How to Resolve VC Service Problem - VC data base limit 4 GB

Posting this with the permission of my colleague Mike


Problem:
Surface Problems:
No clients can connect to the Virtual Center; error usually states a problem with “remote computer”. The remote computer is the virtual center server. Upon checking the virtual center server, you should notice that the “VMWare virtual center server” service is not running. The Virtual Center service will not start. It repeatedly stops after attempting to start the service manually or after reboot (automatically).

Root Problem:
The Virtual Center Database was at the 4GB limit.






The first error I noticed was the “unexpectedly shutdown error” (first error shown above). After digging through the logs you may start to see other errors indicating a database problem.

Resolution:
Use KB article http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1000125
In short, do the following:
Stop services
Backup database
Run script to trim database size
Shrink database
Restart services

Stop/Start Virtual center services:
Log into the virtual center server, go to services and stop/start the VMWare VirtualCenter Server service

Backup Database for Virtual Center
We use MSSQL Express Database for our VC environment.
Open SQL Server Management Studio application; login in with windows authentication (default settings)

Browse to the database being used by Virtual Center – In our case VIM_VCDB



Right Click the DB, select tasks and backup. A backup window will appear:

Ensure that the appropriate DB is selected and that a FULL backup is selected.

Create a name for the backup and note where the backup will be located once complete. Then press OK.

Your backup will run and the progress bar (bottom left corner) will indicate when it is complete. This can take more than 30 minutes.

Run Script to clean up database:

I have downloaded the script on our virtual center server. It is located on the mtalbot desktop. The script can be downloaded from the KB article posted above.

I highly recommend that you watch the video in the KB article! It is short and easy to understand.

In short you need to open the script; it should open in SQL server management studio. YOU MUST CHANGE THE DATABASE TO THE VIRTUAL CENTER DATABASE VIM_VCDB.

Execute the script to get an idea of how many rows will be deleted.

There are two things that need to be adjusted in order to execute the script and delete rows.

1. The “SET @DELETE_DATA = 0 “

a. If set to 0, then no data will be deleted.

b. If set to 1, then data will delete from the DB

2. The “SET @CUTOFF_DATE = GETUTCDATE – 180”

a. Change the number to the amount of days you would like to delete data

b. A suggestion is to delete minimal days and review the size of your database. If you are comfortable with the size of the database, you do not need to delete more days.

Notes about Database and Script:

*SQL Express has a data limit size of 4GB. If our database is approaching or at 4GB this is when we run into this problem. We need to backup database, trim records from db.

*The SQL script will delete tasks, events, and performance data. Simply comment or delete the applicable section of the script if any of these items are not to be purged.

*You can build a new query to monitor the transaction log progress. This will tell you how the log files and space are being updated during the execution of the original script run.

Again, review the video in the KB article for more information.

Shrink Database:

Open SQL Server Management

Browse to the database being used by Virtual Center – In our case VIM_VCDB

Right Click the DB, select tasks, shrink and database.

A window will pop up to allow you to manage the process, review your settings and click OK. Verify you are working with the correct DB. Also, note the size and free space available.

The progress bar in the bottom left corner will let you know when the process is complete.

Once the shrink is complete, restart the server or service for VMWare Virtual Center Server.

Stop/Start Virtual center services:

Log into the virtual center server, go to services and stop/start the VMWare VirtualCenter Server service

Linux memory check script

Please save this with .sh ext and run it ./memmon.sh



#!/bin/bash

# create a memmon.sh script that tracks the current date, memory usage and running processes
echo '#!/bin/bash' > /root/memmon.sh
echo "
date;
uptime
free -m
vmstat 1 5
ps auxf --width=200
" >> /root/memmon.sh

chmod +x /root/memmon.sh

# create a cronjob that runs every few minutes to log the memory usage
echo '0-59/10 * * * * root /root/memmon.sh >> /root/memmon.txt' > /etc/cron.d/memmon
/etc/init.d/cron* restart

# create a logrotate entry so the log file does not get too large
echo '/root/memmon.txt {}' > /etc/logrotate.d/memmon

EXCHANGE SMTP CONNECTORS

In Exchange Server 2007 there are two receive connectors.
All SMTP mail is routed through a Send connector.

Server to Server SMTP and Client to Server SMTP.

Client---> server SMTP is used for POP3 and IMAP clients to send mail via SMTP.
Client Usage Type will allow the Permission Group of Exchange Users

In Exchange 2007, Microsoft wanted to comply with updated RFC standards and kept Server to Server SMTP communication over port 25 and segregated Client to Server communications over port 587

By default, Client to Server Authentication is encrypted using TLS via this Client Receive Connector

Example:

If you want to allow an anonymous application such as a Web Application to relay off of your Exchange 2007 server, you would do the following:

1. Create a new Receive Connector with the Custom Usage Group
2. For Remote Network Settings, remove 0.0.0.0-255.255.255.255, and then add the IP Address of the remote server that requires relaying permissions
3. Once the new Custom Receive Connector is created, go into the properties of this connector, go to the Permission Groups Tab > Add Anonymous Users


If you look at the Authentication Tab, only Transport Layer Security will be selected. This is called Opportunistic TLS which means that TLS will be accepted and is the preferred method for communication, but TLS will not be required.

To activate Anonymous users to use this connector for relaying, you must issue the following command:
Get-ReceiveConnector “Receive Connector Name” | Add-ADPermission -User “NT AUTHORITY\ANONYMOUS LOGON” -ExtendedRights “Ms-Exch-SMTP-Accept-Any-Recipient”

lsof command - whos doing what

lsof will show list of open files

#lsof
[root@rock ~]# lsof | more
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
init 1 root cwd DIR 8,5 4096 2 /
init 1 root rtd DIR 8,5 4096 2 /
init 1 root txt REG 8,5 38620 6422543 /sbin/init
ncftpput 15940 root 3u IPv4 16636687 TCP rock.transfer.my:47940->10.10.10.115:ftp (ESTABLISHED)

Show all open tcp files
#lsof -i TCP


#lsof -i -P | grep sshd

To see list of open files
#lsof -i -U

"Paradigm shift” - An Old Story

A Group of Scientist placed 5 monkeys in a cage and in the middle, a ladder with bananas on the top.

Every time a monkey went up the ladder, the scientists soaked the rest of the monkeys with the cold water.

After a while, every time a monkey went up a ladder, the others beat up the one on the ladder.

After some time, no monkeys dare to go up the ladder regardless the temptation.

Scientists then decided to substitute one of the monkeys. The 1st thing this monkey did was to go up the ladder.

Immediately the other monkeys beat him up.

After several beatings, the new member learned not to climb up the ladder even though never know why.

The 2nd monkey was substituted and the same occurred. The 1st monkey participate in beating for the second monkey. Same way all the monkeys were replaced.

What was left was a group of 5 monkeys that even though never received a cold water shower, continued to beat up any monkey who attempted to climb up the ladder.

If it was possible to ask the monkeys why would beat up all those who attempted to go up the ladder ....i bet you the answer would be.....

" I do not know - that's how the things are done around here"

Does this sound familiar .....

Law of attraction

I was reading the below blog and it was so true in every way.


http://www.forbes.com/2010/06/28/chinese-billionaire-beijing-leadership-managing-rein.html


What I Learned From A Chinese Billionaire
He taught me the basic principles he relied on to go from grinding poverty to enormous wealth.

n the last decade more new billionaires emerged in China than anywhere else in the world. There are now at least 79 Chinese billionaires. China's rich are fueling the country's 15% annual growth in luxury sales, which have reached $9 billion a year, making the country the world's second largest market for top-end products. You can wait a year for a new Ferrari in China. Porsche launched its Panamera sedan there before it did so in the U.S. Long lines often stretch outside Louis Vuitton and Hermès stores.

Who are the superrich Chinese? What are they like? How did they get so wealthy?
To find answers to those questions I've talked to a half dozen Chinese billionaires. Some of them made their money by taking Internet companies public in the U.S., others by investing in real estate or beverages. Unlike many of today's American billionaires, such as the Rockefellers or the Waltons of Wal-Mart, China's are almost all self-made. Half of the world's 14 self-made billionaire women are Chinese. They all overcame hardship and failure, and they all are highly optimistic about China's future.

One Chinese billionaire, a real estate mogul, met with me more than a dozen times over five years to share his lessons for succeeding in business. Sometimes we met at his palatial home in Beijing, sometimes at my much smaller one in Shanghai. We ate clams the size of footballs on a beach in Australia, and one time we went to McDonald's for french fries. I liked spending time with him not only because I could briefly live the life of a billionaire but also because it became obvious why he was so successful. He actually embodied the lessons he taught me. He didn't just preach them.

There was one condition to our talks and my writing about him, though. He insisted on remaining anonymous. Therefore I will call him Mr. Chen. Like many wealthy Chinese, Mr. Chen prefers to fly under the radar, because he doesn't want to accidentally get on anyone's wrong side. Lest you think his fears are exaggerated, realize that 70 of the people who have appeared on the Forbes China Rich List in the last decade have publicly gotten into trouble. China's affluent now sometimes half-jokingly call the ranking the Death List.

Mr. Chen's first lesson is to believe that anything is possible and the only thing that can stop you is yourself. He never ceased to believe that he would make something of himself. As a poor peasant child with no high government connections, he had the odds stacked again him. He had to drop out of school in his teens because his family had no money.

Yet he believed in himself and refused to give up. With no powerful family behind him, he had to take out loans at five times the normal rates of his well-connected competitors. He had to do smaller projects that no one else wanted and endure derision and slights from others. Slowly, over time, he developed a reputation for doing what he said he would, and his projects got bigger and bigger.

Now he pays for the food, medical costs and living expenses of more than 80 members of his extended family. He donates millions a year to set up schools in rural areas, and he often walks the halls of hospitals with his offspring and pays for the medical costs of uninsured patients. But before he could do any of that he spent decades in sheer perseverance, ignoring all the better-educated, wealthier people who disdained him.

The second lesson Mr. Chen taught me is that to get what you want you have to respect everyone and sometimes eat humble pie. A decade ago, when nearly everyone in China was poor, it took less than $10 million to make the China rich list. Last year you needed $120 million. The creation of wealth has been staggeringly quick. As Mr. Chen says, you could be a waiter today and the head of a food and beverage conglomerate tomorrow--so you'd better respect everyone, or your attitude may come back to haunt you. Practically everyone in China knows someone who a decade ago raised pigs and today drives a Mercedes and buys Tiffany jewelry.

Finally, Mr. Chen believes in sharing wealth. He lets business partners make more than he does, so that the next time they secure a good deal they'll think of working with him first. He sees no point in ever trying to shortchange business partners.

He is right. I have been in meetings with him with different partners with projects all over the country. I asked several of them why they worked with him rather than anyone else, and every one of them said it was because they knew he'd take a smaller piece of the total pie. But so many small pieces add up to a whole lot of pie.

Mr. Chen believes in sharing wealth also in the sense of ensuring good working conditions for rank-and-file employees and letting the most productive ones get rich. He does not believe in letting lieutenants scrape by while a CEO is paid absurd sums. And he believes that companies like Apple and Dell need to improve the conditions at the factories where they source their products.

Mr. Chen built a fortune out of dogged determination and a reputation for being assiduously honest, fair and respectful. He sees no reason why the same shouldn't work for anyone else. As he told me once, "If I can do it with as little as I had growing up, anyone can do it."

Steps towards Green IT

1) What are the Green initiatives that have been taken within your company?

- We shut down the desktop after office hours and during weekends, direct result is 60% reduction in electricity bill.

- Optimizing windows start up time

- We are using thin client technology for desktops machine which are critical, they are more energy efficient and also easy to mange.

- AC shut down after 8 PM and during weekends.

- Use of Virtualization technology, we consolidated the servers, any new application/machine deployment is a contender of virtualization, all the legacy machines/application also replaced with virtualization technology, It direct results in lower power consumption.

- We are using SaaS (Software as a Service) model for payroll and HR dept.

- We have migrated our solution to SaaS (Kind of hosting model).

- Migrated the application from SUN servers to Intel based architecture.

- Energy efficient CFL lighting in the office

- Use of TFT/LCD screen instead of old CRT monitors, TFT/LCD are more energy efficient

2) What are the benefits that have resulted from such green initiatives?

- Cost Saving , better ROI , it is major motivating factor

- Reduced carbon foot print , Greener Planet

- Less Hardware (using virtualization and consolidation of servers )

3) What is the road-map for furthering the green agenda, i.e., what other green initiatives is your company intending to take?

- For new IT equipment we are in process for contract with the Hardware vendor for hardware replacement and refresh after its life cycle(in our case it is Dell)

- Every new application should be deployed using Virtualization technology.

- Relocating our office servers to data center colo. It will save cost for us and bigger data centers can afford Green IT Data center. Many times is not feasible for a smaller company to afford green data center (design, build and its operation) in house.

- Dell servers with Intel high-efficiency power supplies, fans and processors to reduce our total watts per server.

- We partner with a company that refurbishes and resells our old server hardware that we can't reuse internally.

How to extend the virtual disk size in ESX/VM

1. Power off the VM-1
2. Make clone of this VM -1 , call it VM-2 (for backup , in case you mess with VM-1)
3. From Vsphere client , increase the size of the VM-1 , you can also increase with a command line tool. #vmkfstools -X 40G VM-1.vmdk (-X is the switch for the size that you want the disk file to be , not the size you want to extend).
4. Power down VM-2
5. On Hardware tab add a new existing HDD and point it to the original VM-1.vmdk (the disk we resized few minutes back), mount it as second HDD.
6. Boot VM-2
7. From run menu type “diskpart”, this utility resize disk partition in windows 2k3
8. DISKPART> list volume , will show you the list volume , select your volume that you want to extend.
9. DISKPAER> extend , run this command to finally to extend the HDD size.
10. You can cross check the HDD size in HDD management snapin (mycomputer->manage->Disk Management)
11. Shut down VM-2 and remove the disk we added
12. Power on the VM-1
13. Voila you are back to business

another way of doing this

1. Power off VM
2. Add a blank larger target disk
3. Add a temp disk (formatted NTFS) to hold a ghost or acronis image
4. Boot to Ghost or Acronis CD
5. Image source disk to image on your temp disk
6. Restore temp image to your larger disk
7. dismount your source and temp disks and test a boot
8. If everything goes well then you are done, you can delete your temp and source disks.
9. if something bad happen then you can revert back to your original disk.

Catch infected machines with TCPDUMP

tcpdump -l -n arp | egrep 'arp who-has' | head -100 | awk '{ print $NF }' |sort | uniq -c | sort -n

Squid Made Simple

1. Modify Squid.conf
Open [root@localhost root]# vi /etc/squid/squid.conf
Edit the following entries:
http_access deny all to http_access allow all
After run:
[root@localhost root]# squid -z
Start squid service by:
[root@localhost root]# service squid restart

2. Create a file with any name .... for our example say iptables.sh
# List all hosts allowed --- separate them by single space
HOSTS="x.x.x.x y.y.y.y"

for host in $HOSTS
do
iptables -I FORWARD -s $host -j ACCEPT


iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -s $host -j REDIRECT --to-port 3128
iptables -A INPUT -i eth1 -j ACCEPT
iptables -A OUTPUT -o eth1 -j ACCEPT

iptables -A INPUT -i $host -j ACCEPT
iptables -A OUTPUT -o $host -j ACCEPT

done

save this file and execute
chmod +x iptables.sh
[root@localhost root]# ./iptables.sh
[root@localhost root]# iptables-save
[root@localhost root]# iptables-save > /etc/sysconfig/iptables

3.
[root@localhost root]# vi /etc/sysctl.conf
net.ipv4.ip_forward = 1
4.
[root@localhost root]# service iptables restart
Now configure users Browser for internet.

Troubleshooting SMTP Servers

$ telnet localhost 25
ehlo
mail from: a@b.com
rcpt to: me@mydomain.com
250 2.1.5 Ok
data
.
Subject: SMTP testing
Greetings from me
.
quit
221 2.0.0 Bye
Connection closed by foreign host.

$ telnet localhost 25
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
That means the server is not running, which you can confirm with Netstat:
# netstat -pant|grep :25
If it returns nothing, your SMTP server is not running