Tuesday, February 26, 2013

Basic settings to adjust for high load systems to make sure the server have enough resources to handle big number of network connections.


The main parameter is a maximum number of opened files allowed for the process to keep at the same time. Each network connection uses a file handler therefore if the limit is too low you can quickly run out of handlers and the server can not accept any more connections.

This limit is set on 2 levels - on the kernel level (fs.file-max) and on the system level (nofile).

Another kernel property which can be important in certain configurations (like transports installations or when you use proxy for Bosh connections) is: net.ipv4.ip_local_port_range. This parameter can be set the same way as the fs.file-max property.

fs.file-max = 358920
If you plan to run high load service with big number of server connections then this parameter should be at least as twice big as the number of network connections you expect to support. You can change this setting executing command:

/etc/sysctl.conf

fs.file-max=360000
net.ipv4.ip_local_port_range=1024   65000




Your IP address identifies your machine, and the port identifies a program on your machine.
A connection consists of 5 pieces of info.
    It is determined by protocol (TCP, UDP)
    local IP address and port
    remote IP address and port
for ex. a web server can service many connections on the same port.
Your webserver can even support multiple connections to the same client machine.
Say, you're connecting to google.com from two windows. Your machine will pick an unused port for each connection. So, google's server will have to keep track of (TCP, google.com, 80, yourmachine, someport1) and (TCP, google.com, 80, yourmachine, someport2).
At some point you'd bump into limits, but it's not a hard limit, and is very system dependent.

 each socket is a file descriptor, but not all machines use shorts for the fd table.
cat /proc/sys/fs/file-max gives 1606638

So, there is a limit of 65336, but it has to do with addressing, not number of connections. Number of connections is limited, but more by system config, and how much memory it has.

 64k connections per incoming IP address on a single local port

root@tc01:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
root@tc01:~#

Open files

The current value can be seen using ulimit -a (look for open files).
 set this limit by putting a file into /etc/security/limits.d/ that contains the following two lines:
*    soft    nofile    10000
*    hard    nofile    10000

Ephemeral Ports

increase the number of Ephemeral Ports available to your application.
By default this is all ports from 32768 to 61000.
We change this to all ports from 18000 to 65535.
Ports below 18000 are reserved for current and future use of the application itself.

TIME_WAIT state
TCP connections go through various states during their lifetime.
here’s the handshake that goes through multiple states, then the ESTABLISHED state, and then a whole bunch of states for either end to terminate the connection, and finally a TIME_WAIT state that lasts a really long time
netstat -tnlp

we care about is the TIME_WAIT state, and we care about it mainly because it’s so long.

By default, a connection is supposed to stay in the TIME_WAIT state for twice the msl.(maximum segment lifetime)
Its purpose is to make sure any lost packets that arrive after a connection is closed do not confuse the TCP subsystem

The default msl is 60 seconds, which puts the default TIME_WAIT timeout value at 2 minutes. Which means you’ll run out of available ports if you receive more than about 400 requests a second.

if a webserve apache/ngnix translates to 200 requests per second then it is Not good for scaling.

We fixed this by setting the timeout value to 1 second.

The two TCP tuning parameters were set using sysctl by putting a file into /etc/sysctl.d/ with the following:

net.ipv4.ip_local_port_range = 18000    65535
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 1


Connection Tracking


Note
Both the “hard” and the “soft” ulimit affect MongoDB’s performance. The “hard” ulimit refers to the maximum number of processes that a user can have active at any time. This is the ceiling: no non-root process can increase the “hard” ulimit. In contrast, the “soft” ulimit is the limit that is actually enforced for a session or process, but any process can increase it up to “hard” ulimit maximum.

A low “soft” ulimit can cause can't create new thread, closing connection errors if the number of connections grows too high. For this reason, it is extremely important to set both ulimit values to the recommended values


Saturday, February 23, 2013

NATing

echo 1 > /proc/sys/net/ipv4/ip_forward
Then you'll need to configure iptables to forward the packets from your internal network, on /dev/eth1, to your external network on /dev/eth0. You do this will the following commands:

# /sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
# /sbin/iptables -A FORWARD -i eth0 -o eth1 -m state
   --state RELATED,ESTABLISHED -j ACCEPT
# /sbin/iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT 
 
 

Wednesday, February 20, 2013

Apache Mod_Proxy | Port Redirection by Apache


Redirect port from abc.com:8111 to abc.com

sudo apt-get install libapache2-mod-proxy-html
sudo apt-get install libxml2-dev
sudo a2enmod proxy proxy_html

cd etc/apache2/mods-available

vim proxy.conf




# If you want to use apache2 as a forward proxy, uncomment the
# 'ProxyRequests On' line and the block below.
# WARNING: Be careful to restrict access inside the block.
# Open proxy servers are dangerous both to your network and to the
# Internet at large.
#
# If you only want to use apache2 as a reverse proxy/gateway in
# front of some web application server, you DON'T need
# 'ProxyRequests On'.

ProxyRequests Off

        AddDefaultCharset off
        Order deny,allow
        Deny from all
        Allow from all
ProxyPass / http://abc.com:8111/
ProxyPassReverse / http://abc.com:8111/

# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
ProxyVia Off




/etc/init.d/apache2 restart

IN case you have SSL


/etc/init.d/apache2 restart
sudo a2enmod proxy proxy_html

in : /etc/apache2/sites-enabled
#cat default-ssl

ServerName hostname.domainName.com
        SSLProxyEngine On
        ProxyPass / https://localhost:8181/
        ProxyPassReverse / https://localhost:8181/



Tuesday, February 19, 2013

Multiple Java versions on ubuntu

$ wget http://download.oracle.com/otn-pub/java/jdk/6u34-b04/jdk-6u41-linux-x64.bin 
 $ chmod u+x jdk-6u41-linux-x64.bin 
 $ ./jdk-6u41-linux-x64.bin 
 $ sudo mkdir -p /usr/lib/jvm
 $ sudo cp -a jdk1.6.0_41/ /usr/lib/jvm/
 $ sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.6.0_41/bin/java" 1
 $ sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.6.0_41/bin/javac" 1
 

sudo update-alternatives --config java
There are 3 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      auto mode
  1            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      manual mode
  2            /usr/lib/jvm/jdk1.6.0_41/bin/java                1         manual mode
* 3            /usr/lib/jvm/jdk1.7.0/bin/java                   1         manual mode

Press enter to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/jdk1.6.0_41/bin/java to provide /usr/bin/java (java) in manual mode.

Monday, November 5, 2012

Git + CentOS + Step by Step

Git + CentOS + Step by Step

Install the git

yum install git

Define Global variables
[root@GOLDMASTER ~]# git config --global user.name "myName"
[root@GOLDMASTER ~]# git config --global user.email "myName@myDomain.com"

Make a directory where you want your repository
[root@GOLDMASTER ~]# mkdir /opt/repository
[root@GOLDMASTER ~]# cd /opt/repository/
[root@GOLDMASTER repository]# git config --global color.ui auto

Make your repositry
[root@GOLDMASTER repository]# mkdir project
[root@GOLDMASTER repository]# cd project/

Initialize the repository
[root@GOLDMASTER project]# git init
Initialized empty Git repository in /opt/repository/project/.git/

It will create a .git directory

[root@GOLDMASTER project]# ls -la
total 12
drwxr-xr-x 3 root root 4096 Nov  5 19:47 .
drwxr-xr-x 4 root root 4096 Nov  5 19:47 ..
drwxr-xr-x 7 root root 4096 Nov  5 19:47 .git
[root@GOLDMASTER project]#

Add docs and files in the repository    
[root@GOLDMASTER project]# mkdir docs
[root@GOLDMASTER project]# cd docs/
[root@GOLDMASTER docs]# vim testdoc.txt
[root@GOLDMASTER docs]# cd ..
[root@GOLDMASTER project]# ls
docs
[root@GOLDMASTER project]# git add .
[root@GOLDMASTER project]# git commit -m " my first commit"
[master (root-commit) 600533f]  my first commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 docs/testdoc.txt


Each time you make a change you want to track, you need to commit it.
In Git committing a change and sharing that change are two different processes...which is not a case in CVS.

Clone the Project Repository


[root@GOLDMASTER repository]# ls
project
[root@GOLDMASTER repository]# git clone project/ /var/local/project
Initialized empty Git repository in /var/local/project/.git/
[root@GOLDMASTER repository]#










If you create the file ~/.gitconfig  & put write the below lines then you can create the aliases as shown below.

[user]
  name = YOUR NAME
  email = you@yourDomain.com

[alias]
  st = status
  di = diff
  co = checkout
  ci = commit -am
  br = branch
  brr = branch -r
  bra = branch -a
  sta = stash
  pl = pull
  ps = push
  lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative
  wc = whatchanged --pretty=oneline

[color]
  ui = true
  diff = auto
  status = auto
  branch = auto

[push]
  default = tracking

[color "branch"]
  current = green bold
  local = yellow
  remote = green

[color "diff"]
  meta = cyan bold
  frag = magenta bold
  old = red bold
  new = green bold

[color "status"]
  added = magenta
  changed = green
  untracked = cyan

[core]
  pager = less -FRSX
  whitespace=fix,-indent-with-non-tab,trailing-space,cr-at-eol
  excludesfile = /home/YOU/.gitignore
  editor = vim
  autocrlf = input
  quotepath = false

[merge]
  tool = vimdiff

[apply]
  whitespace = warn

[gc]
  reflogexpire = 300
  reflogexpireunreachable = 90

[format]
  pretty = %C(yellow)%h%Creset %s %C(red)(%cr)%Creset

[github]
  user = YOU
  token = YOURS

# vi:filetype=gitconfig:tabstop=2:expandtab

 

Thursday, November 1, 2012

Memcache + CentOS

Install Memcached from Source
Download the latest source from http://www.memcached.org website.
  • # cd /usr/src
  • # wget http://memcached.googlecode.com/files/memcached-1.4.13.tar.gz
  • # tar zxvf memcached-1.4.13.tar.gz
  • # cd memcached-1.4.13
  • # ./configure –with-libevent
  • # make all && make install
Install the libmemcached
libMemcached is an open source C/C++ client library and tools for the memcached server (http://danga.com/memcached). It has been designed to be light on memory usage, thread safe, and provide full access to server side methods. You can download the latest version from http://libmemcached.org/libMemcached.html
  • # wget https://launchpad.net/libmemcached/1.0/1.0.4/+download/libmemcached-1.0.4.tar.gz
  • # tar zxvf libmemcached-1.0.4.tar.gz
  • # cd libmemcached-1.0.4
  • # ./configure
  • # make && make install
It is advisable to run/start the memcached process as non-root user. We need to create  a daemon starter scripts for memcached.
  • # touch /etc/memcached.conf
in /etc/memcached.conf place the following code.
#Memory size  (adjust this according to your settings)
-m 64
# default port
-p 11211
# user to run daemon nobody/apache/www-data
-u nobody
# only listen locally
-l 127.0.0.1
 
Create the startup files.
Using SSH Terminal, execute the following command
  • # touch /etc/init.d/memcached
  • # chmod +x /etc/init.d/memcached
in /etc/init.d/memcached place the follwing code.
#!/bin/bash
#
# memcached This shell script takes care of starting and stopping
# standalone memcached.
#
# chkconfig: – 80 12
# description: memcached is a high-performance, distributed memory
# object caching system, generic in nature, but
# intended for use in speeding up dynamic web
# applications by alleviating database load.
# processname: memcached
# config: /etc/memcached.conf
# Source function library.
. /etc/rc.d/init.d/functions
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/memcached
DAEMONBOOTSTRAP=/usr/local/bin/start-memcached
DAEMONCONF=/etc/memcached.conf
NAME=memcached
DESC=memcached
PIDFILE=/var/run/$NAME.pid
[ -x $DAEMON ] || exit 0
[ -x $DAEMONBOOTSTRAP ] || exit 0
RETVAL=0
start() {
echo -n $”Starting $DESC: “
daemon $DAEMONBOOTSTRAP $DAEMONCONF
RETVAL=$?
[ $RETVAL -eq 0 ] && touch $PIDFILE
echo
return $RETVAL
}
stop() {
echo -n $”Shutting down $DESC: “
killproc $NAME
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f $PIDFILE
return $RETVAL
}
# See how we were called.
case “$1″ in
start)
start
;;
stop)
stop
;;
restart|reload)
stop
start
RETVAL=$?
;;
status)
status $prog
RETVAL=$?
;;
*)
echo $”Usage: $0 {start|stop|restart|status}”
exit 1
esac
exit $RETVAL
 Using SSH Terminal, execute the following command
  • # touch /usr/local/bin/start-memcached
  • # chmod +x  /usr/local/bin/start-memcached
in /usr/local/bin/start-memcached place the following code.
#!/usr/bin/perl -w
# start-memcached
# 2003/2004 – Jay Bonci
# This script handles the parsing of the /etc/memcached.conf file
# and was originally created for the Debian distribution.
# Anyone may use this little script under the same terms as
# memcached itself.
use strict;
if ($> != 0 and $< != 0) {
print STDERR “Only root wants to run start-memcached.\n”;
exit;
}
my $etcfile = shift || “/etc/memcached.conf”;
my $params = [];
my $etchandle;
# This script assumes that memcached is located at /usr/bin/memcached, and
# that the pidfile is writable at /var/run/memcached.pid
my $memcached = “/usr/local/bin/memcached”;
my $pidfile = “/var/run/memcached.pid”;
# If we don’t get a valid logfile parameter in the /etc/memcached.conf file,
# we’ll just throw away all of our in-daemon output. We need to re-tie it so
# that non-bash shells will not hang on logout. Thanks to Michael Renner for
# the tip
my $fd_reopened = “/dev/null”;
sub handle_logfile {
my ($logfile) = @_;
$fd_reopened = $logfile;
}
sub reopen_logfile {
my ($logfile) = @_;
open *STDERR, “>>$logfile”;
open *STDOUT, “>>$logfile”;
open *STDIN, “>>/dev/null”;
$fd_reopened = $logfile;
}
# This is set up in place here to support other non -[a-z] directives
my $conf_directives = {
“logfile” => \&handle_logfile
};
if (open $etchandle, $etcfile) {
foreach my $line (<$etchandle>) {
$line =~ s/\#.*//go;
$line = join ‘ ‘, split ‘ ‘, $line;
next unless $line;
next if $line =~ /^\-[dh]/o;
if ($line =~ /^[^\-]/o) {
my ($directive, $arg) = $line =~ /^(.*?)\s+(.*)/;
$conf_directives->{$directive}->($arg);
next;
}
push @$params, $line;
}
}
unshift @$params, “-u root” unless (grep $_ eq ‘-u’, @$params);
$params = join ” “, @$params;
if (-e $pidfile) {
open PIDHANDLE, “$pidfile”;
my $localpid = ;
close PIDHANDLE;
chomp $localpid;
if (-d “/proc/$localpid”) {
print STDERR “memcached is already running.\n”;
exit;
} else {
`rm -f $localpid`;
}
}
my $pid = fork();
if ($pid == 0) {
reopen_logfile($fd_reopened);
exec “$memcached $params”;
exit(0);
} elsif (open PIDHANDLE,”>$pidfile”) {
print PIDHANDLE $pid;
close PIDHANDLE;
} else {
print STDERR “Can’t write pidfile to $pidfile.\n”;
}

Test the startup script. This command will start memcached server.
  • # /etc/init.d/memcached restart
Add memcached as autostart daemon. This command will make sure that memcached will start automatically on when your server restart.
  • # /sbin/chkconfig memcached on

Install the Memcache PECL Extension for PHP
Even though memcached is happily running on the server, it’s not accessible from PHP without the PECL extension. Fortunately this is a very easy process, just use the pecl command.
  • # pecl install memcache
Make sure that extension=memcache.so is in your php.ini (it should be added in there automatically if not, add it).
Restart Apache

/tmp noexec

 remount /tmp to make it executable by issuing the following command:
# mount -o remount,exec /tmp
Once you’re done,  set /tmp back to noexec:
# mount -o remount,noexec /tmp

Thursday, October 25, 2012

Few good links




The following are comprehensive configuration examples.

1) You can obtain 3DES key free from:

2) Free Software Download Page:

3) Good link to obtain TFTP, SYSLOG, FTP etc:
http://tftpd32.jounin.net/   (Only tftp server)

4) Complete list for IANA Ports Information:

5) Bandwidth Utilization:

Apache benchmarking tool (ab) + CentOS

yum install apr-util
yum install yum-utils

ab -n 50 -c 10 http://www.myTestSiteAddress.com/
 
It will test 50 requests + maximum of 10 requests running concurrently


The -c parameter specifies the number of connections; the -k stands for HTTP Keep-Alive; and the -t parameter sets the time in seconds for which each connection is alive
 
ab -n 50 -kc 10 -t 60  http://www.myTestSiteAddress.com/




Run this command on the webserver to analyze the traffic as well 
 
 
tcpdump -nn 'tcp[tcpflags] == tcp-rst' and port 80 and src host x.x.x.x 

Tuesday, October 23, 2012

Easy Twiki backup

#!/bin/sh
cd /var/www
/etc/init.d/httpd stop
sleep 5
tar -zcf twiki.tgz twiki
mv twiki.tgz /home/public/sync/
/etc/init.d/httpd start
cd /home/public/sync
ls -la | mail -s "backup $HOSTNAME -'date'" myUser@myDomain.com

Monday, October 22, 2012

Search & Delete the logs + cron

Delete from a path
sudo find /Path-to-directory/ -name *.log  -exec rm -rf -v {} \;

Delete from the current directory
sudo find . -type f -name "*.bak" -exec rm -rf -v {} \;



By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .
>/dev/null 2>&1

To collect the cron execution execution log in a file :
30 18 * * * rm /home/someuser/tmp/* > /home/someuser/cronlogs/clean_tmp_dir.log



Saturday, October 6, 2012

Delete emails from mailbox/es – Exchange 2007



First Grant all the permission to the mailboxes you are to have empty.
Get-Mailbox "target-username" | Add-MailboxPermission -AccessRights FullAccess -User "my-username"


Put the below command in the a file and save it as a .ps1 file - It will delete all the emails from a mailbox catchall without prompting

Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Admin

Get-Mailbox -Identity catchall | Export-Mailbox -DeleteContent  -Confirm:$false

--------------------------------

Put the below command in a file and save it it as a  .bat file and schedule it in windows scheduler
Powershell.exe -command "& {d:\scripts\catchallMailDelete.ps1}"


------------------------------------------


Delete everything: (from all Mailboxes)
Get-Mailbox | Export-Mailbox -DeleteContent


Export everything: (and then delete the mails from the mailbox)
Get-Mailbox | Export-Mailbox -PSTfolderPath "c:\temp" -DeleteContent



Delete Emails with a given sender name:
Get-Mailbox | Export-Mailbox -SenderKeywords abc@mydomain.com -DeleteContent


Friday, October 5, 2012

Deploying with Capistrano

  1. sudo gem install capistrano-ext
  2. cd yours_rails_directory
  3. capify .
  4. mkdir config/deploy
  
Now we will create two deployments staging and production and a new staging environment.

  1. touch config/deploy/staging.rb
  2. touch config/deploy/production.rb
  3. touch config/environments/staging.rb
  1. config/deploy.rb
  2. config/deploy/production.rb
  3. config/deploy/staging.rb
  4. config/environments/staging.rb:  
  5. config/databases.yml
Now run the capistrano commands to set up the deployment. 
  1. cap staging deploy:setup
    (this creates the correct directory tree)
  2. cap staging deploy:check
    (this checks to make sure setup ran)
  3. cap staging deploy:update
    (this updates the server with the app code)
  4. ssh your_user_name@your_deployment_server
    (fill in the correct user and server here)
  5. cd your_deployment_directory/current
    (again fill in the correct deploy to dir)
  6. RAILS_ENV=staging sudo rake gems:install
    (this installs gems that need native compiling)
  7. RAILS_ENV=staging rake db:schema:load
    (loads the database)
  8. RAILS_ENV=staging rake db:seed
    (loads any seed data)
  9. script/console staging
    (load the console for testing)
  10. app.get('/')
    (test to see if this returns ‘200’)
  11. exit
    (exit the console)
  12. exit
    (exit ssh)
  13. cap staging deploy:restart

Saturday, September 29, 2012

SMTP Relay in Exchange 2007 OR Exchange 2010 (for testing )& Restricting Outbound Email



You may want to do some email testing in the staging environment for real customers minus sending them actually.
By Default exchange is not open realy.
If you already have exchange in your environment and it is  used in the production. then you need to create another dummy domain for the testing. e.g stage.local.
You can use a VM for installing both AD & Exchange on a single machine.
Open up the
exchange MMC
                Server Configuration
                                Hub Transport
 create a "new Receive connector"

 

  

 
 


Now Open the powershell
Get-ReceiveConnector "StageTest" | Add-ADPermission -User "NT AUTHORITY\ANONYMOUS LOGON" -ExtendedRights "ms-Exch-SMTP-Accept-Any-Recipient"

The above will allow the relay to the Anonymous group for that connector.

Now as we may not want that email will go outside the machine (to real customers). We need to create a "transport rule".
Create a user mailbox  e.g  "catchall" .  This will be catch all the emails that you will be sending to the real customer.

open exchange MMC
Organization Configuration==>Hub Transport ==> Transport Rules


Click on the new transport rule on the right side>
Give it a name e.g Mail Redirect


 
 

This should do the trick. To be extra sure you can remove the DNS and Gateway entry of the machine. (in that way this machine will  work inside the LAN only)


A Receive connector that is configured to accept messages from all remote IP addresses through SMTP port 25   This connector typically accepts connections from all IP address ranges. The usage type for this connector is Internal. This connector is created automatically during setup. This connector only accepts mail from other Exchange servers that are part of the same Exchange organization. By default, this connector does not accept anonymous submissions.