sanbarrow.com ~ View topic - Boot a complete XP in ram from a WIM (diskless)

*Last edited by joakim on Wed Dec 23, 2009 12:58 pm; edited 9 times in total*

Files; http://www.mediafire.com/file/mcz4mdyy3gz/wimboot_unixp_drivers.rar

The drivers capable of performing rambooting are at least 5; ramdisk.sys, firadisk.sys, winvblock.sys, wdsys.sys, wimfsf.sys. They can be separated into 3 different methods of rambooting.

1. Firadisk, WinVBlock and WDSYS(DisklessAngel) are mostly the same and mappes preformatted disk/partition images to ram using Grub4Dos or memdisk. Actually, the mapping to memory is not what the driver does, but what Grub4Dos does. The disks created in ram are recognized as harddisks after boot.

2. Ramdisk.sys from MS uses preformatted disk/partition images too, but must be loaded to ram with one of the MS loaders; ntldr or setupldr.bin. This is what rambooting traditionally has been about, for the last years.

3. Third method is about wimbooting as is the standard way for WinPE of NT6.x and later.
The methods that map complete disk/partition images to ram work great, but they require a large amount of ram (size of image + ram necessary to run system). The wimboot method require much less ram, because it does not map a complete image to ram, but instead mount the WIM onto a tiny 3 Mb partition image (boot.sdi). The system then exist in a "mirror" on this tiny partition image that is just a mount point for the WIM.

The size is reduced drastically for 2 reasons;

1. much better compression (compared to gzipped images with NTFS compressed FS)
2. writable free space has moved away from preallocated on-disk, to dynamically allocated cache (handled by the File Based Write Filter, FBWF)

Drawback
1. For now it is limited to roughly 172 Mb writable cache (ScratchSpace) on NT5.x (512 Mb on NT6.x and NT7.x).
2. Junction points (hardlinks) are not supported because the WIM technology has put some restrictions on NTFS metadat handling inside mount points.

Big thanks to Nikzzzz, that made the patches to ramdisk.sys and wimfsf.sys (and setupldr.bin). He is the author of wimboot.script that is for PE usage only.

What do we need to boot a complete non-PE NT5.x, XP in this case, with wimbooting?

1. Prepare a clean system as you otherwise would have (uninstall what you don't need).
2. Delete all unnecessary files. You only need to see 3 folders at the root, see screenshot. My base system is reduced only 550 Mb uncompressed. Updates will follow on how to optimize size.
3. Get 5 files off WAIK, http://www.microsoft.com/downloads/details.aspx?familyid=94BB6E34-D890-4932-81A5-5B50C657DE08&displaylang=en and add to your base;

The registry hive patching tool is finished; http://sanbarrow.com/phpBB2/viewtopic.php?p=6768#6768

Nothing fancy, just a lame autoit script (uses reg.exe and regini.exe).

From the readme.txt;

Updated Registry Hive Patcher; http://sanbarrow.com/phpBB2/viewtopic.php?t=1694

It now solves 100% of the C:\ references, even the last REG_MULTI_SZ hurdles. At least on my english xpsp2 source.

Joakim

Tutorial updated with PXE instructions. Video of complete PXE Wimboot; http://www.mediafire.com/file/yznmuyzmzgh/PXE_Wimboot_Universal_XP.zip

Short answer; it works great!

Took only 2 min 50 sec including complete transfer and boot.

Joakim

More interesting news:

Any bootmgr version can wimboot NT5.x (anything from Longhorn - Win7).

The special bootmgr/winload.exe uploaded is actually from Longhorn version 6.0.5259.0. Which also proves that this MS document is incorrect; http://www.microsoft.com/whdc/driver/tips/Debug_Vista.mspx (They should know better).

The version of winload.exe don't have to be from Longhorn either. As a matter in fact, you can just as well use the one from Vista (6.0.6000.16386). Winload from server 2008 will bsod ..4E. Winload from Win7 rc1 will cause a kernel stack fault (hardware reset) in vmware. All tests from wimbooted xpsp2. Setting the "detecthal" value will cause any version to hang forever.

Joakim

Booting directly over WAN (from internet) can be done with gPXE. In this test it was done through the HTTP protocol, with a server on the internet hosting the files on port 80. Both the diskless client and the server where on slow wireless endpoints. The transfer speed averaged at 724 Kb/s, with a full bootime of about 7 minutes (image transfer and boot).

The gPXE version had to be latest development branch. If earlier version is used, then workarounds are needed because gPXE did not handle memdisk correctly until very recently. The diskless client was booted with an iso containing gPXE only, in a VM configured with a bridged wireless network connection.

We need to put bootmgr, boot.sdi, boot.wim and boot.ini/BCD onto a properly created disk image. We will then boot the image with memdisk. Since we map twice to ram, we will actually wasted ram equivalent to the size of the disk image, so keep disk image size not larger than just to fit the 4 required files. Preferrably zip the image afterwards to optimize the size of it. We then put memdisk and the zipped disk image on some HTTP file server somewhere. Booting from memdisk this way don't require special drivers as the system will boot from a second mapped ram location, instead of the memdisk-created (hence wasted ram). But the waste is necessary to achieve this. The reason why I did not put whole xp install on the memdisk directly and boot with winvblock is that compression would not be so good. In fact the image would be 93 Mb's larger, and thus add over 2 minutes more to the boot time!!

At the prompt on the gPXE bootcd type ctrl + b and then;

*Last edited by joakim on Tue Dec 08, 2009 8:22 am; edited 2 times in total*

Instead of "wasting" the ram we can use it by activating winvblock or firadisk. By deleting all the content on the disk image, we can have it as a regular ramdisk or some other temporary container/cache.

You will cause no harm to the system by deleting all the content, because it is just the "kicker" anyway and not used in any way in the actual Windows boot process. In my case this is now a NTFS compressed disk image with 227 Mb of free space (mapped as C:\). Just make a batch that delete the content during boot;

Video of an internet booted XP using HTTP/xPXE/memdisk/wimboot; http://www.mediafire.com/file/ydhzii2xgim/WimBoot_http_wan_gpxe.rar

The part when the IP is displayed is blurred to not reveal too much about the "borrowed" network. But the commands are the same as already described. This particular video was with "wasted" ram.

Joakim

Playing some more with HTTP booting, I just booted Win7Pe. This really cool and thumbs up for the gPXE developers.

A 2,4 Mb video of http-booting the recovery disk of Win7 RC1; http://www.mediafire.com/file/d3vwjdt4bym/gPXE_HTTP_Win7PE.rar

It is still the same memdisk "kicker" trick being used. Winvblock will not work for this, but there is is a fair chance that the DisklessAngel driver will be able to use the "kicker" disk as a secondary ramdisk. The DisklessAngel driver is unrestricted for images up to 640 Mb (equivalent to roughly 1,5 Gb uncompressed wim!!! And a total of 512 Mb + 640 Mb of free space for your diskless station!!!)

Let's hope Sha0 Miller will continue development of his Winvblock, the only free memdisk capable windows driver...(currently only supporting XP).

Btw, since memdisk now supports iso mapping/emulation, it is also possible to boot the iso directly over the Internet. But, you will not be able to "reuse" any kicker as a ramdisk if done this way. Total size should be roughly the same for both methods.

I assume any WinPE could be HTTP booted one way or the other.

Joakim

For those testing gPXE vmware, it is advisable to change network adapter type to e1000 in your vmx. Also change your gPXE boot rom to the e1000 specific (the package with all drivers in it does not perform as good, for some weird reason).

The Intel e1000 will transfer at an average 30% faster than Amd PCNet32 when network booting with gPXE (at least with HTTP).

And that will make a substantial time-difference when transferring large images.

Joakim

Btw, I tried modifying the Option Rom in the bios to include gPXE, but without success (no error messages, just hanging at black screen). Anybody out there with a working gPXE capable vmware (Phoenix) bios? At least boot logos was easy to change...

Clarifying some of the points in the above procedure;

@dara

Basically, we prepare a folder with a complete XP install, lets call the folder wim_xp. My test system is a virtual machine (power off vm and mount the virtual disk offline, then copy all files into the wim_xp folder.

Then we must prepare this offline XP installation before we finally pack it into a wim. Forget for the moment the "Universal" part of this because it is an unfinished/ever-improving procedure with a lot of extra work to even resemble something hardware independant.

Preparation is as follows;
- Inside the folder wim_xp you will need 3 subfolder; WINDOWS, Program Files, Documents and Settings.
Very important update
- Get http://www.mediafire.com/file/mcz4mdyy3gz/wimboot_unixp_drivers.rar and put the 4 drivers into wim_xp\windows\system32\drivers
- Put winload.exe into wim_xp\windows\system32
- Copy the SYSTEM and SOFTWARE registry hive (taken from wim_xp\windows\system32\config) into a folder like C:\temp together with the registry hive patcher tool. Run the tool on both hives. When finished, copy them back into wim_xp\windows\system32\config. Load the SYSTEM hive either with the registry GUI or via "reg load HKLM\sys SYSTEM". Merge wimboot.reg into registry. Unload the hive with the GUI or via "reg unload HKLM\sys".
- For including the WinVBlock driver, see separate note.
- Get gimagex.exe, an easy frontend to the command line tool imagex.exe. Capture wim_xp and save as boot.wim and tick the "boot flag". Set compression to max in one of the drop-down boxes. Then click capture.
- Put boot\boot.sdi, bootmgr, boot.ini and boot.wim into the 238.img (mount descriptor with vdk). When finished copying, unmount the image.
- Finally zip the image (238.img), and it's ready to be served from HTTP.

Get your gPXE boot media from; http://rom-o-matic.net/gpxe/gpxe-git/gpxe.git/contrib/rom-o-matic/
Choose iso or whatever you want and click "Get Image". Boot iso and type commands as already described. Optionally create your boot media with an embedded script so that everything is automated at boot-time.

Joakim

After some investigation on "writecache" on systemdrive, it appears to be a limit for nt5.x on 183.

That means we can use its full potential by setting the WinPECacheThreshold or the ScratchSpace (same thing) to B7.

Merge this into your SYSTEM hive;

Finally figured out how to set systemdrive to C:\.

Some more notes, about the /minint switch and how it works.

Basically, /minint for nt5 means different things for the "Setup Loader" and the "Boot Loader".

- setupldr.bin will start PE and load non-volatile hives as volatile and keep registry changes in memory only.
- ntldr will load non-volatile hives as volatile and keep registry changes in memory only.

For nt6 and nt7 there is just 1 loader, winload.exe, but located in 2 different places. Winpe set on in the BCD will;
- Load your specified winload.exe that in turn load the SYSTEM hive and the other hives (all as volatile) which has the exact same paths for PE and non-PE. PE vs non-PE will then be autodetected (not exactly sure how, but most likely from several places in registry). Therefore readonly registry is also possible for non-PE on nt6 and nt7 as well.

But, winload.exe (only certain versions) can also load nt5. However it can only understand what ntldr can boot (ie setupldr.bin/setupreg.hiv will not be understandable to winload.exe). Therefore wimbooting nt5 results in a confused state for winload.exe, which require us to override the default, X:\, as mentioned earlier.

It appears to be somewhat problematic to set systemdrive to anything other than X: on PE from nt5 and nt6, Windows 7 is different though.

Joakim

research Joined: 09 Feb 2007 Posts: 559

Joakim, I just want to thank you for all the work and knowledge you sharing here.

I'll test this RAM network boot approach one day... but meanwhile, this is very interesting!