Tuesday, September 28, 2010

Which process is eating away the memory

which process is using up your memory using ps, awk, sort

The following command lists all the process sorted based on the used memory size.

$ ps aux | awk '{if ($5 != 0 ) print $2,$5,$6,$11}' | sort -k2n

PID VSZ RSS COMMAND

3823 3788 484 /sbin/mingetty

3827 3788 484 /sbin/mingetty

3830 3788 484 /sbin/mingetty

3833 3788 488 /sbin/mingetty

3834 3788 484 /sbin/mingetty

3873 3788 484 /sbin/mingetty

2173 3796 568 /usr/sbin/acpid

1835 3800 428 klogd

1832 5904 596 syslogd

2054 5932 540 /usr/sbin/sdpd

2281 6448 360 gpm

The above command lists the PID, Used virutal memory size, Used resident set-size and process command. The output is sorted on VSZ.

While debugging performance issues, use this command to find out which process is using up the memory.

Find out Top 10 Largest File or Directory Using du, sort and head

du command shows summarized disk usage for each file and directory of a given location (/var/log/*). The output of a sort command is reversely sorted based on the size.

# du -sk /var/log/* | sort -r -n | head -10
1796    /var/log/audit
1200    /var/log/sa
612     /var/log/anaconda.log
512     /var/log/wtmp
456     /var/log/messages.4
92      /var/log/messages.2
76      /var/log/scrollkeeper.log
72      /var/log/secure
56      /var/log/cups
48      /var/log/messages.1

How to Resolve VC Service Problem - VC data base limit 4 GB




Posting this with the permission of my colleague Mike


Problem:
Surface Problems:
No clients can connect to the Virtual Center; error usually states a problem with “remote computer”. The remote computer is the virtual center server. Upon checking the virtual center server, you should notice that the “VMWare virtual center server” service is not running. The Virtual Center service will not start. It repeatedly stops after attempting to start the service manually or after reboot (automatically).

Root Problem:
The Virtual Center Database was at the 4GB limit.






The first error I noticed was the “unexpectedly shutdown error” (first error shown above). After digging through the logs you may start to see other errors indicating a database problem.

Resolution:
Use KB article http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1000125
In short, do the following:
Stop services
Backup database
Run script to trim database size
Shrink database
Restart services

Stop/Start Virtual center services:
Log into the virtual center server, go to services and stop/start the VMWare VirtualCenter Server service

Backup Database for Virtual Center
We use MSSQL Express Database for our VC environment.
Open SQL Server Management Studio application; login in with windows authentication (default settings)

Browse to the database being used by Virtual Center – In our case VIM_VCDB



Right Click the DB, select tasks and backup. A backup window will appear:

Ensure that the appropriate DB is selected and that a FULL backup is selected.

Create a name for the backup and note where the backup will be located once complete. Then press OK.

Your backup will run and the progress bar (bottom left corner) will indicate when it is complete. This can take more than 30 minutes.

Run Script to clean up database:

I have downloaded the script on our virtual center server. It is located on the mtalbot desktop. The script can be downloaded from the KB article posted above.

I highly recommend that you watch the video in the KB article! It is short and easy to understand.

In short you need to open the script; it should open in SQL server management studio. YOU MUST CHANGE THE DATABASE TO THE VIRTUAL CENTER DATABASE VIM_VCDB.

Execute the script to get an idea of how many rows will be deleted.

There are two things that need to be adjusted in order to execute the script and delete rows.

1. The “SET @DELETE_DATA = 0 “

a. If set to 0, then no data will be deleted.

b. If set to 1, then data will delete from the DB

2. The “SET @CUTOFF_DATE = GETUTCDATE – 180”

a. Change the number to the amount of days you would like to delete data

b. A suggestion is to delete minimal days and review the size of your database. If you are comfortable with the size of the database, you do not need to delete more days.



Notes about Database and Script:

*SQL Express has a data limit size of 4GB. If our database is approaching or at 4GB this is when we run into this problem. We need to backup database, trim records from db.

*The SQL script will delete tasks, events, and performance data. Simply comment or delete the applicable section of the script if any of these items are not to be purged.

*You can build a new query to monitor the transaction log progress. This will tell you how the log files and space are being updated during the execution of the original script run.

Again, review the video in the KB article for more information.



Shrink Database:

Open SQL Server Management

Browse to the database being used by Virtual Center – In our case VIM_VCDB



Right Click the DB, select tasks, shrink and database.

A window will pop up to allow you to manage the process, review your settings and click OK. Verify you are working with the correct DB. Also, note the size and free space available.



The progress bar in the bottom left corner will let you know when the process is complete.

Once the shrink is complete, restart the server or service for VMWare Virtual Center Server.

Stop/Start Virtual center services:

Log into the virtual center server, go to services and stop/start the VMWare VirtualCenter Server service