Why Hyper-V cannot boot off of SCSI disks (and why you should not care)

Why Hyper-V cannot boot off of SCSI disks (and why you should not care)

  • Comments 20

Under Hyper-V we have two kinds of disk controller that you can add to a virtual machine – IDE disks and SCSI disks.  A rough list of the differences between these controllers is as follows:

IDE:

  • Works on operating systems without integration services installed / available
  • Can be used to boot a virtual machine

SCSI:

  • Supports hot add / remove of virtual hard disks

The type of disk controller that you use in the virtual machine has nothing to do with the type of disk that you are using in your physical computer.  SCSI VHDs can be stored on IDE disks, and IDE VHDs can be stored on SCSI disks.

Why can’t you boot Hyper-V virtual machines off of SCSI disks?

Well, calling our SCSI controller “SCSI” is somewhat of a misnomer.

When we were working on Virtual Server we heard, loud and clear, that servers had SCSI disks – so virtual servers needed to have virtual SCSI disks.  To this end we invested in the emulation of an Adaptec SCSI controller for Virtual Server.  Unfortunately, this SCSI controller is a very advanced piece of hardware – and as a result was actually slower to emulate than the more simplistic IDE controller that we were already emulating.

In the end we had to extend our emulated Adaptec controller and write custom drivers for our supported operating systems in order to get good performance for SCSI in a virtual machine.

With Hyper-V we dropped the emulation of an Adaptec controller altogether.  Instead we implemented our traditional emulated IDE controller and a new completely virtual, VMBUS based, storage controller - with no traces of emulation present.

It is this virtual storage controller that you are adding to a virtual machine when you choose to add a SCSI controller to a virtual machine.

The problem is that the BIOS that is used by our virtual machine has no knowledge of VMBUS and is only able to boot to emulated devices.  This is why you can boot to an IDE controller and a legacy network adapter – but not to a SCSI controller or network adapter.

Why you should not care about not being able to boot off of SCSI disks in Hyper-V.

When talking to users about needing to boot off of SCSI disks in a virtual machine – there were two reasons that came up.

The first one was that SCSI could support larger virtual hard disks than IDE could.  To address this we made the IDE controller in Hyper-V use 48-bit LBA.  This allows you to attach virtual hard disks that are up to 2TB in size to an IDE controller.

The second reason was performance.  But that too is not an issue.

Let me start by grabbing an architectural diagram I have on file (you can tell that this is an old diagram because it uses our codename – Viridian – which means it was originally drawn before we had been given “Hyper-V” as our name).  Now, the parent partition diagram is not 100% correct for Windows Server 2008 R2 – but the child partition diagram is accurate for both Windows Server 2008 and Windows Server 2008 R2, and that is where I want to focus.

storage

The first thing to notice in the child partition side of things is the “Virtual Storage Minport (VSC)”.  The is essentially the driver that gets loaded when you attach a SCSI controller to a virtual machine.  It connects to VMBUS and allows us to perform Disk I/O without any emulation involved in a very high performance manner.

The next thing to notice is the “Fast Path Filter”.  This is a filter driver that gets installed on all disk objects in the virtual machine – whether they are IDE or SCSI.  It allows us to pass directly to the VMBUS based path for everything except low level disk operations (like partitioning a disk).

What this means is that once the integration services are installed the same code path is used for disk I/O whether you use an IDE disk or a SCSI disk.  There are two limitations that remain for IDE disks:

  • Disk commands to IDE disks on the same controller are serialized by the guest operating system (note that you can only have two IDE disks on a single controller)
  • The IDE disk is limited to I/O block sizes of 512kb or less – while the SCSI controller can go up to block sizes of 8mb

However I have yet to see a test where either of these limitations resulted in a noticeable performance difference between IDE and SCSI.

Cheers,
Ben

Leave a Comment
  • Please add 2 and 1 and type the answer here:
  • Post
  • In the standard configuration for Windows 7 the System partition (which the BIOS boots to) is separate from the partition where Windows actually lives.  I believe these can be on separate drives.

    So, can Windows itself be installed on a virtual SCSI disk provided the System partition is on a virtual IDE disk?

  • Ben,

    The issue we have is that the guidance for VS2005 was to put the HDD images on the SCSI bus as this provided the best performance.  We now have an issue with these machines migrating to Hyper-V (around 50-60 of the pesky little buggers).  From what I can determine we need to rebuild these machines to migrate them to Hyper-V - is this the case?

  • Hi Ben,

    My name is Alex Bethlehem CEO of desktopsites. I would very much like to have the opportunity to introduce myself and what we are doing as a software company with Hyper-V integration into our Konect Elite product. If you could please send me your contact info I will be sure to follow up with you to set up a time that work best for you.

    desktopsites: 403-538-0105

    email: alexb@desktopsites.com

    Thanks - look foward to hearing from you.

  • @harry. No you can only see the virtual SCSI disks once the OS has loaded the drivers for them; while the OS is loading you only have emulated IDE.

  • Hi Ben,

    The one other thing which comes in to play is that optical drives won't attach to the new SCSI controller and you can only have two IDE controllers. With a boot disk on one of the IDE locations that leaves room for only three optical drives. I've not seen a requirement yet for more than three, but this is one of the other planning considerations.

    For instance, on my WDS/WAIK VM I have ISOs for Windows 7 and Windows Server 2008 R2 in two optical drives at all times to save remounting them for WAIK. While it isn't necessary, it could be useful to have another optical drive or two for additional ISOs.

    Cheers,

    Tristan

  • Tristan,

    Install MagicISO Virtual CD/DVD software. It works brilliantly!

    http://www.magiciso.com/tutorials/miso-magicdisc-overview.htm

  • Hi BBWI,

    Yep, I use it and agree it's excellent. I suppose there's a difference in that one is installed within the guest VM and the other is a part of the virtual hardware. I was thinking there might be a need for more than three optical drives and sometimes Magic ISO might not be desirable, for instance if the virtual machine is network-isolated and you don't want to increase the VHD size by copying the ISO file locally. I suppose there could be a situation where you want the Hyper-V adminsitrators to be the only people who cab control what disk is inserted as well.

    I was slightly surprised that optical drives couldn't be added to the SCSI controller when I first discovered this. It's not a big deal at all though.

    Cheers,

    Tristam

  • Is a quick moral to the story to boot OS drive from IDE, and use the virtual SCSI controller for data drives?

  • We need to boot from SCSI because the HAL is too far off to move an SCSI NT installation to Hyper-V for recovery purposes.  The server was installed on Scsi and trying to clone it to IDE causes BSOD and may or may not be repairable ??

  • There's a simple reason to maintain the old SCSI emulated adapters Ben, and that's the INCOMPATIBILITY that could appear for not suing the standarization MS introduced with its previous virtualization products. Not to mention about doing P2V, V2V and V2P operations wich can come more buggy and unstable.

    Doing the things using a syntethic driver to appear like an emulated SCSI adapter to the end can produce never seen problems on a real scenario, like the ones you probably'll see soon.

    Virtualization is based on a only principle, EMULATION. Let say a bar exist and in one extreme we have EMULATION, in the other we have SIMULATION. When you move the arrow more to SIMULATION (wich is that you do with Synthetic SCSI drivers, CPU pass through, guest tools, etc.) more unexpected errors can come out, because that new situation doesn't exist in the real scenario, while the problems you could have with all the hardware emulated, are exactly the same that they happened before with that piece of hardware in the real world.

    Imagine that you have the arrow all on the SIMULATION side. Sure you'll need to be a hardware supporter for synthetic VM but not only for MS based os, you'll need to bring that support for all the operating systems that are supported as guests under Hyper-V or whatever the virtualization system MS offer.

    But for a customer this is the less important thing compared to others that are really critical, for example:

    what will happen when you migrate from a piece of software that is sensible to hardware calls or checks that are only available on the emulated hardware? simple, that software won't work. Not at least till you modifiy the synthetic driver to support that kind of requirement wich could be a really a hell in time for solving the problem, ensuring that the change doesn't affect to other hardware checks of other software, etc. and the customer waiting for the resolve of the problem.

    The proyects, intervention planning, and posibily failures are totally unexpected from synthetic drivers.

    I think that you and the rest of the MS virtualization should take in mind this: virtualization means the conversion from physical to virtual of an EXISTING thing.

    That simple statement has some obvius considerations, and the first of it is this: the best virtualization for an existing hardware is that wich comes 100% emulated perfectly.

    Maybe you'll ask why, and the answer is simple, because getting all the hardware 100% emulated, you only have to connect it, and it will make the job as if it were on the physical world.

    All the things that are out of emulation means inserting a layer of posible problems in the virtualization. Maybe you're lucky and you have few (because you make very good synthetic drivers or so) or even none of those problems, but the layer is efectively inserted and exist. But even more, all the things that are out of emulation, means that you need to do the work that that piece of hardware does, and I don't suggest you to reinvent the wheel.

    The only scenario were synthetic drivers and models could have a place is on future proyects where the hardware is totally independant (for example, the cloud), or were a piece of hardware is merely an appliance that virtualice some other process, or part of the system that is used with Windows Server 2008 (because it doesn't use the HAL that all the previous Windows do), etc.

    I don't know if you understood what I wanted to mean with all of this because I know that my english is horrible.

    Cheers,

    Akuma.

  • >>The server was installed on Scsi and trying to clone it to IDE causes BSOD and may or may not be repairable ??

    I converted 6 VMs from VMWare SCSI to Hyper-V IDE via CloneZilla and about half of them went BSOD when accessing the disks... A swift repair install solved the problem.

  • Hi Ben,

    Well I think the number one reason to not like Hyper-V SCSI disks is that you cannot do a system repair using Windows Backup since it cannot see the drives. Perhpas you can release a methodology around this which would include the driver for system repair.

    Now I know it might be better to copy VHD files or od an export, however some people need to use the windows backup features for a more robust real-time backup that simple snapshots or complete VHD exports.

    Thanks,

  • hi,I couldn't extend the disk which is connected with iSCSI in windows 2008 server R2. extend volume option is greyed out. What could be the reason?

    thanks..

  • Harry Johnston -

    You could do this, but there would be no benefit to it.  The performance for SCSI and IDE is the same.

    Rick Pearson -

    These systems should work correctly using IDE on Hyper-V.

    Tristan Watkins -

    Good point!

    NT-Recovery-SVC / Akuma -

    Providing an emulated SCSI controller would do nothing to help with P2V / HAL compatibility.  Every SCSI controller has seperate drivers - so unless we emulated every SCSI controller out there you would still need to handle replacing drivers in systems in order to P2V them.

    Jeff Loucks -

    Windows 7 / Windows Server 2008 R2 should be able to see SCSI disks when doing system repair.

    oz -

    I do not know about iSCSI capabilities in this regard.

    Cheers,

    Ben

  • Sorry Ben (first of all for my horrible english), but in fact it helps a lot, because when you P2V a machine the SCSI destination is another machine with a replaced driver for the SCSI disk driver and cleaned HAL (now I see that the announcement from Microsoft that Windows Vista and later won't have HAL is just false), that means than in our case, using VMM 2008 R2 for example, and converting a machine that comes from vmware, the modification of the original disk is a fact (IDE and SCSI have different structure), so I could never use a cloned image or similar tools taken from SCSI disk to restore a machine in Hyper-V, because a SCSI disk attached directly to an IDE will never work. It needs conversion or structural change.

    But it would be even worse, because converting a machine using traditional tools (those used before VMM, vCenter Converter, Disk2Vhd, etc. appeared) would never work because you can't inject manually a synthetic driver for that SCSI because doesn't exist in the real world. This scenario would be valid for example with not supported OS for conversion (too old or too newer), or in cases where the need to use sector by sector copied disk is a must (forensic scenarios), etc.

    I think that you don't need to emulate every SCSI controller, I think that you need to emulate the simplest or easiest generic SCSI that could be found (for example those legacy from LSI). You could use some HLE at first to get improved and get replacing the code eventually with better emulated system. Or even better, you had emulated a real Adaptec adapter (Virtual Server), you have half of the path walked, you only need to implement it or get better for Hyper-V.

    Come on, I can't believe that ppl who made Virtual Game Station so good (sure that's more complex that emulating an only bus card adapter) or getting MAME to the next level (thx Aaron) can't emulate a legacy (and properly documented) SCSI card to the perfection.

    Cheers,

    Akuma.

Page 1 of 2 (20 items) 12