Thursday, March 27, 2008

VMware & PowerShell: Creating a PSdrive

I'm not the first to report it, but I wanted to share this here. My first reaction was: 'Cool!', and so is my second reaction... It is related to the fact that VMware released the beta of the PowerShell toolkit for VI.

Imagine you can browse your virtual infrastructure using a command line interface like this:

cd vi:
cd Folder01\DataCenter01\host\Web\LiveHost01


The original article can be found here. Have fun!

Monday, March 10, 2008

VMware VscsiStats: Measuring at the virtual SCSI level

I mentioned earlier that one of the presentations at VMworld Europe 2008 was about measuring at the level of the virtual SCSI adapter of a VM. A wealth of information is available when looking at this kind of information.

A tool is available on ESX 3.5 that creates histograms by default (and complete traces if wanted) is VscsiStats. As an option, one provides the vSCSI handle ID and the VM World ID. In order to get some statistics at all, one first needs to start the monitoring:


./vscsiStats -s

After some time, the relevant statistics can be fetched by issuing a command like:

./vscsiStats -i 8260 -w 1438 -p ioLength

This, for instance, yields an histogram of the size of IO pacakges sent to the virtual SCSI adapter (and thus to the storage array). To finalize the monitoring, it has to be stopped as well:

./vscsiStats -x


The result of the command above, in my test, was a graph like the one below:


This is a VM running Windows 2003. Remember this is a histogram, we put the measures points in 'buckets' according to their size and plot their relative frequency.

Remark that there is some IO with sizes 4095 bytes 8191 bytes. This is a sign that the file systems are not aligned properly. The fact that 4K sizes are the majority is nice, because VMFS is optimized for IO of 4KB.

Thursday, March 06, 2008

VMware: Virtual VCB Server/Proxy for iSCSI storage

(I know, I did not get past day 1 of VMworld Europe 2008, perhaps I will write some more about the other two days later this week...)

Something different now: It is known that VCB for VI 3.5 adds support for iSCSI devices, in itself this is not a big deal, but there is something really 'cool' about this: it means we can use a virtual server to act as a VCB proxy! This is a big step forward in my opinion.

How can this be done?
1. Install a virtual server
2. Connect the virtual network card of the server to the iSCSI network (either giving it the correct VLAN tag, or connecting it to the proper vSwitch).
3. Install a software intiator (e.g., the one you can download from the MS website. This step is pretty much the same as for a physical server.
4. Install & Configure VCB.
5. Configure the SAN to correctly present the LUNs to the backup proxy (based on the iqn).

Now, combine this with the possibility to add an iSCSI or NFS storage appliance to your virtual infrastructure, and you're ready to have a complete virtualized backup solution that is no longer tied to physical hardware.

Tuesday, February 26, 2008

VMworld Europe 200: Day 1

Here are some quick thought, remarks, things I picked up today in a random order. The moment I'm home with a decent internet connection, I might update some of this info.

General things:

  • From this morning's key note: 3% of all energy generated goes into our data centers. This is the reason why green data centers are important.
  • As you can read elsewhere, Novell has acquired PlateSpin. Let's hope prices will drop.
  • Site Recover Manager is nice, but does not offer anything that can not be done by hand or script.

Performance related:

  • File system alignment tends to become more and more important. Two of the speakers today claimed it to be very important for file system performance. Just make sure you use the VI Client to create the file systems, it does the alignment for you.
  • Some Linux kernel use a Khz CPU timer, which causes CPU overhead in the guest. The kernel boot option 'divider' modifies this behavior.

SAN/NAS related:

  • A lot of people were interested in the talk discussing differences between FC SAN, iSCSI (SW & HW) and NFS. NFS, however, was not really covered in this talk. All in all, there were no big surprises here: FC is usually better than all the other alternatives, especially for large block sizes (less SCSI overhead), software iSCSI uses more CPU cycles than hardware iSCSI, etc. A whitepaper has been published with this info, but I don't have the link ready.
  • iSCSI has been optimized for 8K block sizes, as this is block size is encountered a lot. The result is clearly reflected in the stats.
  • An experimental tool is available to analyze guest disk I/O statistics. It basically creates histograms of throughput, latency, average read/write distance, etc. The command line is 'vscsiStats'. I could not yet test it out yet, as I don't have an ESX server in my hotel room. This alone makes it worth being here...
  • In order to troubleshoot SAN performance issues, allocate a small LUN (e.g. 100MB), so that everything can be cached. This way, you avoid effects of physical disks, spindles, etc.

Network related:

  • In order to use the enhanced vmxnet driver in 3.5, you need to first remove the existing vNIC and add a new one. Then you can select the new enhanced interface with support for all the new features.
  • When setting up network failover policies, it is important to take into account the fact that by default the spanning tree protocol takes 30 seconds to open the uplink port on a physical switch. During this time, the virtual switch sees the link (to the physical switch) as up. 30 seconds is twice the default timeout for VMware HA. Rebooting a switch may cause a lot of havoc in this case

As you might notice, I am particularly interested in everything that relates to performance. Furthermore, I have a lot of references to interesting KB articles, but I need to check them out first before posting any info.

Friday, February 22, 2008

VMworld Europe 2008 Timetable - Abstracts

Somebody noticed (in the comments here) that the abstracts for the talks are not in the Excel file I created. I don't have time to add this info now, but I created a quick and dirty html page (with some sed, grep and bash scripting) that can be found here: Session_Abstracts.html

Hope this helps...

Custom Search