VMware Capacity Planner: taking the data offline (Part 1: Introduction)

Lately, I have been involved in some VMware Capacity Planner (VMCP) tracks. VMCP deals with monitoring a set of (physical) servers in order to assess possible virtualization scenarios.

VMCP works in the following way: a monitoring server is set-up with a tool that does (but is not limited to) regular performance measurements. This tools sends the data to a VMware database through the web (over a secure channel). Via a web interface, one can then query the information, get reports, view trends, show graphs, configure and create consolidation scenarios, etc. Usually, we let the system run for around 4 weeks to get a realistic idea of the performance characteristics.

What I like about VMCP is that the data is not located at the customer site, and available at all times (once it has been uploaded). This gives me the opportunity to regularly check on the status of the performance measurements.

The biggest disadvantage of VMCP is that the web interface is not the most flexible and fast interface around. Some things I would like to do are not available (lack of flexibility) but could easily be done in, e.g., Excel and at times when everyone in the world is awake it takes ages to refresh a page and get the result of a query. Moreover, it is not easy to get good-looking information to paste in a document.

When it comes to writing a report, the customer is obviously not only interested in a statement like: you will need 5 ESX server of type X to cope with the load. Therefore, I like to add tables with the most useful metrics (CPU, network, Disk I/O, ...) for later reference. I add this information as an appendix.

This is where I would spend at least half a day exporting CSV files from the VMCP website, loading them in Excel, laying it out as a nice table and paste it in the document. I started thinking about automating some of the steps required, and I covered the most time-consuming already: exporting the information from the website as a CSV file.

In the following part, I'll explain how I started this little adventure...

VDI vs. TS/CTX

Check out this blog article by Brian Madden.

VCB for NFS

Read this post: http://storagefoo.blogspot.com/2007/09/vmware-on-nfs-backup-tricks.html. It discusses a free alternative for VMware Consolidated Backup, but over NFS instead of Fibre Channel (SAN).

The thing we want Softgrid to do (Active Upgrade+)

This entry started as a comment on
http://blogs.technet.com/softgrid/archive/2007/09/25/methods-for-upgrading-or-updating-virtualized-applications.aspx

The thing with Active Upgrade is that (as the article points out), the user gets the update automatically on next launch of the application. This is a major step forward for application updates. However, it does not solve the whole application testing problem: before pushing changes to the users, I need to test the application. In order to test, both version need to coexist but with Softgrid it is not X and Y but X or Y. The fact that a rollback to an earlier version of an active upgrade is not possible adds to the complexity.

What I basically want is:
1) Make a parallel application branch Y (as it is called in the article) from version X
2) Test this branch (without impacting the existing version)
3) Update the parallel application branch if required (using active upgrade)
4) When tests are successful, use Active Upgrade to update the production version X

This scenario is inherently not feasible because in stage 1, a new asset directory and a new GUID is created so that settings are stored in a different location.

The scenario that comes closest to what I want is the following:
1) Make a parallel application branch Y
2) Test this branch
3) When tests are successful, apply exactly the same procedure to the original version X, but this time using active upgrade.

The main issue in this case is the fact that you need to record the exact upgrade scenario in order to apply it again in step 3.

An other workaround exists, though:
1) Make an active upgrade
2) Test this update by manually importing the SFT file on the client (instead of streaming it)
3) When successful, do the update centrally with active upgrade.

I hear you thinking: but we have a Test envionment in order to do application tests. Ok, right, do you have a test copy for every production database to name just one example of why production tests may be required.

File Versioning

File versioning is something I have been interested in since I started out doing programming for my simulations back in university. Later, I was lucky enough to be able to use (La)TeX to type my PhD thesis, which enabled me to use CVS to track changes to my text.

Years later, I'm less lucky in that I do not use LaTeX anymore (neither do I program in C ;-), but I do still want my documents and scripts to be versioned, especially when actively working on them. I used to create copies for major versions of Word or PowerPoint documents. At the end, I deleted most of the intermediate copies because they were no longer relevant.

But now, everything has changed dramatically... due to FileHamster. FileHamster is a versioning tool, that automatically tracks changes to files and directories and allows you to keep those changes, revert to earlier versions, backup and by means of some handy plugins add notes and remarks to versions. And ... it is free !



Behind the scenes, the tool creates a copy of every version you save, so you do use a lot of space for big files, but you cannot blame a tool for the fact that most of the files we work with nowadays are composed of binary data!