Rob’s Blog

Step by step - Host Clustered Virtual Server 2005 R2 Guests

Over the last few weeks at work I’ve been working on host clustered Virtual Server 2005 R2 SP1 guests. Here’s how I go about setting up a new guest - it’s not particularly complicated but it is very long-winded. Caution: this article contains almost exactly the same information as Microsoft’s guide. I just re-wrote it to make it (a) more specific to my environment and (b) more readable for the people I work with.

Well, first off, you need to allocate your storage. The great thing about Microsoft’s Cluster Service is that it supports a wide range of storage options, so you can choose the expensive Fibre Channel, the cheap iSCSI, or shared SCSI or SAS. In my environment we’ve got an IBM SAS array which has a lot of the benefits of FC, whilst keeping the price point relatively low.

So, allocating disk space. Where to start? Well, first of all you need to work out exactly how much space you want. In my environment, I’m starting off with estimated disk size plus double the memory size plus an extra 4GB for anything extra I may need in that guest. This might seem a little on the cautious side but hey, with a couple of terabytes of fast storage to play with, what’s a few gigabytes between friends? Also, remember that if you want to use undo disks you’ll have to increase this even further.

So anyway, in my environment (and anyone not using the IBM DSxxxx series can skip ahead here after configuring their own storage subsystem!) what we need to do is open up IBM’s Sytem Storage Manager. Under the Configure tab, we go to Create Logical Drives and then select the free capacity we want to create this drive from. Stick in the size of the drive you worked out earlier and a descriptive name - I tend to use Servername-Disk. The final step in Storage Manager is then to map that drive to a storage group (I’m assuming you’d be using storage groups as it’s the most sensible thing to do in a cluster).

So, that’s your storage system configured. Next, to Windows - to make sure that all of your server nodes can see the storage and, just as importantly, they’re mapped as the same disk number on each node (else Cluster Server will be unhappy). First off, log on to your first cluster node. Open up Computer Management (Start -> Administrative Tools -> Computer Management) and navigate to the Disk Management node. Take a note of the physical disk number. Now, log on to each of the nodes in the cluster (or connect to them remotely through your Computer Management console - however in Windows Server 2003 SP2 be aware that Windows Firewall blocks remote access to the Disk Management console) and check that your new disk has been given the same physical number in each of the cluster nodes. This is extremely important - if these disk numbers do not match, the cluster will not be able to fail over correctly.

Assuming everything is okay, you’ll need to initialize and then format the disk ready for Windows to use. Do this from your first cluster node, and ensure you initialize the disk but don’t convert it - MSCS does not support dynamic clustered disks. Once you’ve done this, create a new partition to fill the disk and assign it a drive letter (ensuring of course this letter is free on every node of the cluster). Now, on each cluster node, you’ll need to add this drive letter to the partition you’ve just created. I’m not 100% certain you need to do this step - I think MSCS can handle this for you - however with something mission critical it’s always best to be safe rather than sorry. You don’t want to be failing something over to another node at 2am and for it to go horribly, horribly wrong - and yes, I do have personal experience with this…

Next up you’ll need to begin to create your cluster group. Open up Cluster Administrator on any cluster node, or on any workstation that has the Windows Administrative Tools pack installed (Start -> Administrative Tools -> Cluster Administrator). Right-click pretty much anywhere and choose New -> Group. To make things a little more obvious, I usually call this new group the name of the server I’m going to create; however if you’re running more than just Virtual Server in this cluster then you might want to think about this a little bit, perhaps prefixing the group VM or something. In our two-node environment I don’t set any preferred owners; however if you have a larger cluster node you may have specific servers you want your guests to run on.

Now you have your new cluster group, it’s time to add something to it - the shared disk resource. Right-click on your freshly created cluster group and choose New -> Resource. Give it a name and a description and set the type to Physical Disk. Leave the Possible Owners as default (so any node can own the resource). You’ll also want to leave the dependencies blank for two reasons - firstly, there’s nothing else in this group so there’s nothing to add, but more importantly, this disk will always need to be the first thing that’s available in a Virtual Server cluster group. Finally, choose the disk you created earlier as the disk resource you wish to cluster. Have a go with failing this group over to each cluster node - make sure the disk resource comes online on each node.

Okay, so now you’ve prepped your cluster nodes, it’s time to start working with Virtual Server. The first step is to create a new Virtual Server network. Now, before doing this, it’s extremely important that your NICs are absolutely identically configured in each cluster node. For example, I have two NICs in each node. The first NIC, which the OS labels Broadcom #1, is called LAN and the second, Broadcom #2, is called Cluster. If you have multiple devices of the same type in your configuration you must make sure that these devices match across each node. Although MSCS doesn’t care, Virtual Server does - where MSCS just looks at the interface name, Virtual Serve binds itself to the physical NIC. So, if you’re using card 1 for the LAN on node 1, but card 2 for the LAN on node 2, when your Virtual Server fails over network connectivity will fail. Make sure you get it right, otherwise you’ll spend plenty of time either scratching your head, or, if you’re more like me, swearing loudly at your display.

Assuming your cluster is perfectly configured we’ll need to create a new Virtual Network. Log into the Virtual Server Web Management page of the cluster node which currently owns the shared disk resource you created earlier. In the web management console, find the Virtual Networks bit on the left and choose Create. Give it a name (again, I use the name of the server I’m going to create) and bind it to a physical NIC in the server. First of all, from the web console you’ll need to remove the virtual network from the server. Go to Virtual Networks -> Configure -> View All and then click the network you’ve just added and choose Remove. Next up, you will need to log on to this cluster node. Open up Windows Explorer and navigate to C:\Documents and Settings\All Users\Documents\Shared Virtual Networks. Cut (don’t copy!) the file from this location and paste it into the root of your clustered disk resource for this server.

Back in the Virtual Server Web Management page, navigate back to Virtual Networks but this time Add a new Virtual Network - type in the full path of the .vnc file’s location into the Existing configuration (.vnc) file: box. You’ll need to add this virtual network to each cluster node - so once you’ve completed this step initially on the first cluster node, move the cluster group across to the next node and add the Virtual Network.

Right, now you have the Virtual Network available on each node, it’s time to get down to the business of actually creating a new Virtual Server Guest. So, log back on to the Virtual Server Web Management thingy on whichever cluster node currently owns your cluster group. Navigate across to Virtual Machines and choose Create New. Now, for the server name, type in the full path - for example, if the disk you created previously has the drive letter X:, type X:\NewServer.vmc as your server name. This ensures that the Virtual Machine Guest configuration file resides somewhere that all of the cluster nodes can read it from. Next, configure your guest’s memory as you usually would, attach the network adapter to the network you created earlier, and choose to Attach a virtual hard disk later (none) as the hard disk. "Why not create a hard disk now?" I imagine you’re asking. Well, if you create the hard disk here, it’s a dynamically expanding disk. We do this for performance reasons - if you’re adding data to this hard disk it’s only one write operation to modify this disk. If you have a dynamically expanding VHD, the server first has to grow the file before you can write the data to it.

Click on the Create button to create your guest, and navigate back to Virtual Machines - this time choosing to Configure your newly-created guest. Firstly, click on the SCSI Adapters link and choose to Add SCSI adapter. Accept the defaults and click okay. Now we’re ready to create and connect a hard disk for this guest. Click on the Hard Disks link and choose to create a new Fixed Size Hard Disk. Choose the location for this hard disk - the drive letter you’ve saved the .vnc and .vmc files to - and name your hard disk. For example, I tend to use X:\ServerName-C.vhd for the server’s C: drive and so on for any other disks your server needs.

It takes a little while for the fixed size disks to get created, so go and grab yourself a coffee, or perhaps that bottle of tequila if you’re really pleased with yourself for making it this far. After a few minutes (although if you did take the tequila option I’d recommend waiting a few hours or even days before continuing) your VHD will be created and you can then add it to your guest. Navigate back to your guest’s config (Virtual Server -> Configure -> NewServer) and click on the Hard Disks page again. Add the VHD you have just created, ensuring that it is attached to the first available SCSI node (which will almost certainly be SCSI 0 ID 0). In our environment we don’t use Undo Disks, so we don’t enable this, but of course if you do use them, enable it. Obviously. I’m not going to go into why we use SCSI instead of IDE here, but if you’re interested, read this.

Finally, we want to fiddle with the networking configuration a little bit. On your guest’s configuration page, click on the Network Adapters link and change the MAC address assignment from Dynamic to Static. This ensures that the server always get the same MAC address when failing over from node to node. This isn’t necessary, but highly recommended, especially if you have devices on your network with long ARP cache times. It also means that one of your nodes won’t start up another virtual machine with a MAC address that’s already in use somewhere else.

So, that’s most of the hard work done now - but don’t be tempted to turn on your guest just yet! Just a few more steps, then it’s all ready to go. Before you can think about running your new guest, you need to make sure each node knows about it. As Virtual Server isn’t quite as cluster aware as some of Microsoft’s other products (SQL Server especially), we need to add the guest to each node - much in the same way as we did with the virtual network configuration. So, for each node, move the cluster group that contains your new guest to the node, log into the Virtual Server Web Management, navigate to Virtual Machines -> Add and type in the full path to the guest configuration file (in our example from earlier, X:\NewServer.vmc). Repeat this step on each of your cluster nodes. Easy, but time consuming.

Last step, I promise, then you can go and have a nice cold beer (or the rest of that tequila) to congratulate yourself. Now, you may remember when following Microsoft’s guide that you configured a script that lives in MSCS’s folder on each server. This script handles how your server reacts to the cluster group starting, stopping and moving from node to node. You don’t need to know how it works, but you do need to configure it as a resource for your new cluster group. It’s really easy to configure, but you must be logged on to one of the nodes in your cluster to perform this step - you can’t do it from a machine that has the management tools on (okay, this may actually be a lie, but it’s just easier to do it this way for now). Open up Cluster Administrator and right-click on the cluster group for your guest. Choose New -> Resource, give it a name and a description and choose the type Generic Script. Again skip through the possible owners screen, but ensure you add your disk resource as a dependency. This ensures that all the config files etc. are available to Virtual Server before the script attempts to bring it online (or do whatever else it needs to do). When you get to the Parameters page, you’ll need to choose the following as the script path:%windir%\Cluster\Havm.vbs

Now, the final step. Open up a command prompt, and tap in the following:
cluster res "Name you gave the script a minute ago" /priv VirtualMachineName=Name of your guest

And you’re done! Right click on your cluster group and bring it online and watch it start up through Virtual Server Web Management on the active node - assuming this is successful, test failing it over across to all of your nodes. You now have a fully redundant virtual guest!

	Enter your search terms Submit search form
	Google This Site

This entry was posted on Monday, July 9th, 2007 at 1:56 pm and is filed under Storage Area Networking. Find similar posts by selecting any of the following tags: . You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.