Hello Everyone,
I'm looking if I can get some advice from your group on how to handle a rather frustrating situation.
In one of my branch offices, I have a rather small VMWare ESXi 4.1 environment running on two IBM HS22 Hosts. The hosts are running on BladeCenter S storage.
In the last few months, I have been getting horrible performance from a majority of the VM's that are running on those hosts and I am out of ideas on how to prove what is causing the issues. I am starting to think that it is storage related, but I am having trouble coming up with an argument which will support my theory.
Here is the info on our system:
- There are 6 shared datastores on each host.
- There are two storage pools in the BladeCenter S.
- One storage pool contains Five 15k 420 GB SAS disk, the other contains Five 7.2k 2 TB SAS disks.
- Both Pools are configured in individual RAID5 arrays
When i have ESXTOP open, I can see that the DAVG/cmd value is often jumping from 68-620. I even saw it jump to 1887 at one point.
Looking at the Performance Tab in the vSphere client for each host, in the last 1 hour, I have a max of 476 ms write and 114 ms read latency on the storage adapters for one host, 74 write/34 read for the other host.
Also looking at the Performance Tab in the vSphere client for each host, in the last 1 hour, I have around a max of 383ms write/406ms read latency on two of the datastores for one host, 266ms write/527ms read latency for the same two datastores on the other host.
I am not seeing this issue on other hosts running on different storage in a different geographic location.
I have attached screenshots of the performance I am seeing.
What are your recommendations on how to proceed with troubleshooting the source of the performance issues on these hosts?
Thanks in advance for your help.
-Sean