Linux Software Raid + VMware

Posted: February 22, 2008 in Linux, VMware

In my previous entry, I wrote about the problem I faced with VMware using SCSI as the virtual disk emulation. After a few experiment, I notice, the problem only exist in my Linux that uses software raid. All of my 3 machines that hosts VMware are using software raid mirror. I tried with one machine without software raid, the default SCSI virtual disk works fine, even in I/O high load.

I had mentioned this in one of my response to the comments I received. So far, I notice, in software raid setup, Linux run best with IDE as the virtual disk emulation. Below are the steps taken by me to convert the SCSI disk to IDE disk. The steps are quite simple, so far no lost data or guest OS failure encountered. However, I am using Slackware, which uses lilo as the OS loader. So, this little guide only applicable to Linux that uses lilo.

  • Step 1: Delete the Virtual SCSI disk and remove SCSI from the system – This may sound like a very drastic measure, but removing SCSI disk does not destroy the virtual disk images. This can be done by doing: Edit Virtual Machine Settings -> Select the SCSI disk -> Press the “Remove” button
  • Step 2: Edit the .vmx file and .vmdk file of the guest OS – For example, in my guest OS, Linux, it will have a Linux.vmx file. Inside this file would contain the following entries:
    scsi0.present = "TRUE"
    scsi0.virtualDev = "lsilogic"

    Set the “TRUE” to “FALSE” and remove the scsi0.virtualDev line. After this, edit the Linux.vmdk file, and change:
    ddb.adapterType = "lsilogic" to ddb.adapterType = "ide"
  • Step 3: Add the modify virtual disk image – Goto Edit virtual machine settings again, this time use Add to add a new disk, and make sure we select Use an existing disk and browse the the modified Linux.vmdk that we had done in step 2.
  • Step 4: Make the guest OS boot from hda instead of sda – First, use the Slackware installer CD or ISO image to boot up. After boot up the system using the installer disk, mount the virtual disk partition to /mnt. From here, edit /etc/lilo.conf and /etc/fstab, make sure we change all the entry of sda to hda. Once this is done, run ‘lilo -r /mnt’. Create the hda* device file in /mnt/dev if required.

Once we reached here, we can remove the CD installer or ISO image from the guest OS and let it start as per normal.

I had converted many guest OS systems using the steps mention, it works quite well and it took not longer than 10 minutes to finish.

However, I am using Slackware, so these steps are typically for Slackware. For other distro, we will need to know how to manually make a partition bootable. This is very important! Else we won’t be able to boot up the disk image we had modified

Lastly, (though it may seems obvious to some) do remember to try these steps with a few test guest OS before actually apply to a production guest OS. We should backup the production guest OS just in case something went wrong.

  1. hsn says:

    Thanks for the useful article. I was trying to run around 15 vms simultaneously for some testing I have to do and I faced the problem you described about the slow vms and SCSI errors. I just wander what editor did you use for the vmdk files as my files are around 4 gb and tend to get corrupted when casual editors such as textpad is used.

  2. lenrek says:

    To be honest, I did not aware there is such issue until I downloaded a demo VM guest from Internet. Now, I am also facing the same problem. It seems, the virtual disk that I had created, were not structure in this way.

    As far as I understand, if the virtual disk has been created using “monolithicSparse” as the createType, then all your vmdk files will be join as one large file.

    For my case, I always create the virtual disk using: createType=”twoGbMaxExtentSparse”. I have no idea how to convert a “monolithicSparse” virtual disk to “twoGbMaxExtentSparse” virtual disk.

  3. Ben D. says:

    @hsn: if you have monolothic disks the disk-descriptor database is contained in the first few bytes of the file. There’s good information about how to edit the disk-descriptor under either Windows or Linux in these threads:

    The thread is a little hard to follow but it should get you pointed in the right direction.

  4. Ben D. says:

    This guide was very helpful but I ran into a problem because I had made snapshots and so I had a chain of three sets of vmdk files:

    Ubuntu.vmdk, Ubuntu-s001.vmdk, Ubuntu-s002.vmdk … Ubuntu-s011.vmdk

    Ubuntu-000001.vmdk, Ubuntu-000001-s001.vmdk … Ubuntu-000001-s011.vmdk (this set is a child of the set above)

    Ubuntu-000002.vmdk, Ubuntu-000002-s001.vmdk … Ubuntu-000002-s011.vmdk (this set is a child of the set above)

    To make the changes you describe I had to edit both Ubuntu.vmdk and Ubuntu-000001.vmdk. Then I made a mistake when I added the new IDE disk: I pointed it at Ubuntu.vmdk — which describes the original set of disk files — rather than Ubuntu-000002.vmdk — which describes the latest child set.

    When I booted the guest after the change I quickly realized that the disk had reverted to the state it was in a few months ago. So I shut it down, removed the disk, and added it again, this time trying to point it at Ubuntu-000002.vmdk, but I couldn’t because the chain had been broken when I booted into the “grandfather” disk.

    Luckily the fix was simple, I just had to edit Ubuntu.vmdk and change the disk’s CID to the CID that its child disk was expecting.

  5. lenrek says:

    Thanks for the links, and, nice to know you had solved your problem.

  6. hsn says:

    Thanks Ben D. In my case I found the most clean solution is to create new virtual machines using ide vmdk’s. Although this took a bit of time to reconfigure the guests like those made using scsi vmdk’s.

  7. biosphere says:

    Eh, yeah. Don’t try this for Windows guests. :p

  8. lenrek says:

    It was never intended for Windoze.

  9. Boris says:

    Thanks, it worked fine for me.

  10. I have been having these kind of problems too. I have some servers using Software Raid. I have Ubuntu Server on them, as host. After I install everything, my Virtual Guests run very slow when there is high i/o on the disks. I haven’t tried using IDE virtual disks, though, I’ll try them and then let you know. Have any of you found the way to improve performance using this setup? If I don’t use the Raid1 on the host…will this improve performance?? I need to try some configuration, but if you did it already, please tell me about your experiences!

  11. lenrek says:

    I suggest you try the latest version of the vmware server. From my experience, it seems to suffer less of such problem.

  12. […] Linux Software Raid + VMware February 200811 comments 3 […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s