Fun with your new Firefox - running on a ramdisk

[DMcCunney]

Warning: serious geekery follows. Feel free to avoid if browser tech isn't your preferred recreation.

I've been playing with Firefox 4.0 Beta12pre (a/k/a Minefield), on a Windows XP Pro SP3 box with 3GB RAM. (It has 4GB, but the BIOS only reports 3 to Windows. The 4th GB is apparently unusable. )

I've got it pretty well where I want it, and at this point, it's replaced FF 3.6.13 as my production browser.

One of the things I've been playing with is something I used to do in the old MS-DOS days. My old PC XT clone had 640K of main memory, and a card with a megabyte of add-on memory that could be utilized through software. So at bootup, I'd load a driver that configured 512KB of that add-on memory as a ramdisk, that looked and acted like a disk drive but was actually in RAM. My boot process would copy frequently used utilities to it, and add the ramdisk as the first entry in the PATH. MS-DOS and programs that created and used temp files were told to use the ramdisk for it. It sped things up a treat.

Ramdisk software exists for Windows, and since I have 3GB of RAM, I can afford to carve off a chunk as a ramdisk and fiddle. So I have a 256MB ramdisk seen by Windows as compressed NTFS volume Z:.

The first thing I tried was keeping Firefox's cache there. That's easy. Go into about:config and create the new string preference browser.cache.disk.parent_directory. Set the value to the location you want the cache to be, which in this case is Z:\\. Poof! FF is using the ramdisk for cache. (On Linux I put cache in POSIX shared memory - /dev/shm - which exists in RAM or swap but not in the HD file system.)

Well, why stop there? Why not have the Firefox profile on the ramdisk? That's fairly easy, too. First, I copied the contents of my standard FF profile to the ramdisk. Next, I needed to get FF to use it. To do that, I created a new profile called "Ramdisk", and used Choose Folder in the Profile Manager to point to the folder on the ramdisk as the one to use. Poof! FF 4.0 was running, using the profile on the ramdisk.

Okay, so far so good. Now, can Firefox itself run from the ramdisk? Yes. I copied the directory in Program Files where Minefield is installed to the ramdisk, then navigated to it in Windows Explorer and executed Firefox.exe. It ran fine, and Sysinternals Filemon confirmed that all FF related disk I/O was to the ramdisk, not the hard drive.

So I created a shortcut on the desktop that runs FF 4.0 from the ramdisk, and tells it to use the ramdisk profile. Thus far, it's working fine, and it's probably about as fast as FF can get on the box.

It's still a work in progress. I'm in the middle of creating WinXP startup process that will create the proper directories and copy the needed files to the ramdisk when Windows is booted. The bigger issue is entries that will copy changed files back to the hard drive on shutdown, so changes don't get lost if I have to reboot. It appears I can use gpedit.msc, the Group Policy Editor, to run a shutdown script. (This is only available in XP Pro. XP Home lacks it.) I will unavoidably lose stuff if I crash and must use the three-fingered salute to restart Windows, but I can live with that.)

Mostly, this is a "because I can" exercise, and not something I recommend to general users, but it's been fun so far and FF run this way is quick.
______
Dennis

[LCF]

Wow! I loved playing with ramdisk, when I had the XP. I'm definitely going to try this with the final FF4. I will have to fight will Win 7 and reading your post I'm a few light years behind your skills, but that's what google is for

If you manage a solution for saving the changes and it is usable for win 7, do please share how it's to be achieved!

[DMcCunney]

Originally Posted by LCF

Wow! I loved playing with ramdisk, when I had the XP. I'm definitely going to try this with the final FF4. I will have to fight will Win 7 and reading your post I'm a few light years behind your skills, but that's what google is for

There is nothing about what I did that is specific to FF 4. The same thing can be done with FF 3, the same way. I just happened to be playing with FF 4 when I got inspired to do it.

If you manage a solution for saving the changes and it is usable for win 7, do please share how it's to be achieved!

Which Win7 version? The trick is something that happens automatically as part of shutdown. Under XP Pro, it appears I can do it with the Group Policy Editor. I don't know if that exists on your version of Win7. (Do Start/Run, and enter gpedit.msc in the Run box to find out.)
______
Dennis

[phenomshel]

Originally Posted by DMcCunney

Warning: serious geekery follows. Feel free to avoid if browser tech isn't your preferred recreation.

I've been playing with Firefox 4.0 Beta12pre (a/k/a Minefield), on a Windows XP Pro SP3 box with 3GB RAM. (It has 4GB, but the BIOS only reports 3 to Windows. The 4th GB is apparently unusable. )

I've got it pretty well where I want it, and at this point, it's replaced FF 3.6.13 as my production browser.

One of the things I've been playing with is something I used to do in the old MS-DOS days. My old PC XT clone had 640K of main memory, and a card with a megabyte of add-on memory that could be utilized through software. So at bootup, I'd load a driver that configured 512KB of that add-on memory as a ramdisk, that looked and acted like a disk drive but was actually in RAM. My boot process would copy frequently used utilities to it, and add the ramdisk as the first entry in the PATH. MS-DOS and programs that created and used temp files were told to use the ramdisk for it. It sped things up a treat.

Ramdisk software exists for Windows, and since I have 3GB of RAM, I can afford to carve off a chunk as a ramdisk and fiddle. So I have a 256MB ramdisk seen by Windows as compressed NTFS volume Z:.

The first thing I tried was keeping Firefox's cache there. That's easy. Go into about:config and create the new string preference browser.cache.disk.parent_directory. Set the value to the location you want the cache to be, which in this case is Z:\\. Poof! FF is using the ramdisk for cache. (On Linux I put cache in POSIX shared memory - /dev/shm - which exists in RAM or swap but not in the HD file system.)

Well, why stop there? Why not have the Firefox profile on the ramdisk? That's fairly easy, too. First, I copied the contents of my standard FF profile to the ramdisk. Next, I needed to get FF to use it. To do that, I created a new profile called "Ramdisk", and used Choose Folder in the Profile Manager to point to the folder on the ramdisk as the one to use. Poof! FF 4.0 was running, using the profile on the ramdisk.

Okay, so far so good. Now, can Firefox itself run from the ramdisk? Yes. I copied the directory in Program Files where Minefield is installed to the ramdisk, then navigated to it in Windows Explorer and executed Firefox.exe. It ran fine, and Sysinternals Filemon confirmed that all FF related disk I/O was to the ramdisk, not the hard drive.

So I created a shortcut on the desktop that runs FF 4.0 from the ramdisk, and tells it to use the ramdisk profile. Thus far, it's working fine, and it's probably about as fast as FF can get on the box.

It's still a work in progress. I'm in the middle of creating WinXP startup process that will create the proper directories and copy the needed files to the ramdisk when Windows is booted. The bigger issue is entries that will copy changed files back to the hard drive on shutdown, so changes don't get lost if I have to reboot. It appears I can use gpedit.msc, the Group Policy Editor, to run a shutdown script. (This is only available in XP Pro. XP Home lacks it.) I will unavoidably lose stuff if I crash and must use the three-fingered salute to restart Windows, but I can live with that.)

Mostly, this is a "because I can" exercise, and not something I recommend to general users, but it's been fun so far and FF run this way is quick.
______
Dennis

\

Dennis, see bolded above... for Windows to report and use more than 3 GB RAM, you have to be running 64 bit. You did already know this and just said it wrong...right?

[DMcCunney]

Originally Posted by phenomshel

\
Dennis, see bolded above... for Windows to report and use more than 3 GB RAM, you have to be running 64 bit. You did already know this and just said it wrong...right?

Wrong. I'm well aware of the 32bit Windows limitation.

The problem here is that my BIOS only sees 3GB of it. (This issue is noted in the BIOS manual, tucked away in fine print.) There is software that could make use of the 4th GB if it knew it existed, but it can't know about it if the BIOS doesn't report its existence.

Win XP SP3 supports PAE mode, and XP here thinks it's in it, but nothing I've looked at can see and use that 4th GB.

I'd love to be wrong about this, but thus far it appears I'm not.
______
Dennis

[phenomshel]

Originally Posted by DMcCunney

Wrong. I'm well aware of the 32bit Windows limitation.

The problem here is that my BIOS only sees 3GB of it. (This issue is noted in the BIOS manual, tucked away in fine print.) There is software that could make use of the 4th GB if it knew it existed, but it can't know about it if the BIOS doesn't report its existence.

Win XP SP3 supports PAE mode, and XP here thinks it's in it, but nothing I've looked at can see and use that 4th GB.

I'd love to be wrong about this, but thus far it appears I'm not.
______
Dennis

Ooooohhhh. Ok, got it. Well, I couldn't figure it out, because I was sure you had to know about the 32 bit Windows limitation.

[DMcCunney]

Originally Posted by phenomshel

Ooooohhhh. Ok, got it. Well, I couldn't figure it out, because I was sure you had to know about the 32 bit Windows limitation.

Well, I don't know everything, but I do know about that.

Unfortunately, the fix would be a new motherboard. The current one is an ASUS circa 2005, and while I generally like ASUS, this one doesn't seem to support PAE. For various reasons that are all variants of "I have other things to do with the money", a new mobo isn't happening any time soon.

Windows sees 3.1+ GB of the available memory, and there's enough I can carve off a 256MB slice to play games like this without missing it, so I do. I could actually do this with a ramdisk about 100MB smaller, based on the space actually used by this effort, and may shrink it for the "production" version once I have things like auto load on boot and save on shutdown working. Meanwhile, I'll settle.
______
Dennis

[AprilHare]

Amiga ram disks were better in my opinion (dynamically-resizing RAM disks were seriously useful), but it sounds cool. It would still use swap space if the application used too much ram though (theoretically you can disable swap but apparently Microsoft don't like this).
Why not ramp up the geekage and run a whole OS in a RAM disk, running Firefox and Calibre?

[LCF]

Originally Posted by DMcCunney

There is nothing about what I did that is specific to FF 4. The same thing can be done with FF 3, the same way. I just happened to be playing with FF 4 when I got inspired to do it.

I also thought so, but until I have the time to play with it, FF4 will probably be out

Which Win7 version? The trick is something that happens automatically as part of shutdown. Under XP Pro, it appears I can do it with the Group Policy Editor. I don't know if that exists on your version of Win7. (Do Start/Run, and enter gpedit.msc in the Run box to find out.)
______
Dennis

Pro, 32 bit

Originally Posted by AprilHare

Amiga ram disks were better in my opinion (dynamically-resizing RAM disks were seriously useful), but it sounds cool. It would still use swap space if the application used too much ram though (theoretically you can disable swap but apparently Microsoft don't like this).
Why not ramp up the geekage and run a whole OS in a RAM disk, running Firefox and Calibre?

That's what I did back then - put my swap on the ramdisk. I'm not really sure it was faster, but it was fun

Which software can recognize the 4th GB? I have Win 7 Pro, 32 bit and PAE should be supported in the architecture of my CPU (Intel Core Duo).

[DMcCunney]

Originally Posted by AprilHare

Amiga ram disks were better in my opinion (dynamically-resizing RAM disks were seriously useful), but it sounds cool.

I had a dynamically resizing ramdisk under Win98. It was not updated for 32 bit NT based Windows, or I'd still be using it. There may actually be a payware dynamic ramdisk that works under XP, but I haven't looked. Available freeware solutions (including a sample program provided by Microsoft) meet my needs.

It would still use swap space if the application used too much ram though (theoretically you can disable swap but apparently Microsoft don't like this).

Yes, you can run without a page file. It's not recommended - Windows gets snarky if you do. You can reduce the page file to a minimal size, but you have to have one.

Other tricks that may be useful include forcing a fixed size to pre-allocate the required space and avoid the overhead involved in growing it when needed, and if you have more than one physical drive, putting the page file on a different drive and IDE channel. (I do that here.)

Why not ramp up the geekage and run a whole OS in a RAM disk, running Firefox and Calibre?

I should only be able to do that.

If my mobo would actually see the 4th GB of RAM installed (2 2GB sticks, but it only sees 3GB of it), I'd put pagefile.sys on the ramdisk.

I wouldn't run Calibre from the ramdisk, however. It would speed it up a treat, but I don't use Calibre enough to justify the effort. I use Calibre for ebook conversions and metadata editing,. I do not use it as a library manager.
______
Dennis

[DMcCunney]

I wouldn't run Calibre from the ramdisk, however. It would speed it up a treat, but I don't use Calibre enough to justify the effort. I use Calibre for ebook conversions and metadata editing,. I do not use it as a library manager.

Incidentally, there's a simple reason for that. I have about 4,000 ebook volumes, but I have at least as many paper books, and whatever book catalog solution I use must handle both. There are an assortment out there that claim to. The leading candidate here at the moment is Readerware.
______
Dennis

[DMcCunney]

Originally Posted by LCF

Pro, 32 bit

See whether you have gpedit.msc.

That's what I did back then - put my swap on the ramdisk. I'm not really sure it was faster, but it was fun

Fun is why you do it. Measuring speedup tends to be a complex issue.

Which software can recognize the 4th GB? I have Win 7 Pro, 32 bit and PAE should be supported in the architecture of my CPU (Intel Core Duo).

If you are running 64bit Win7, pretty much everything should see it.

One ramdisk that is supposed to see and use extra memory under 32bit Windows is Superspeed's: http://www.superspeed.com/desktop/ramdisk.php

In my case, it's a motherboard limitation hamstringing me.
______
Dennis

[DMcCunney]

Still working on the auto-load/auto save to HD on shutdown.

One bit that requires fiddling is FF updates. I'm using FF 4.0 beta12pre. This means I get updates in the form of nightly builds fixing additional issues every few days. FF is configured to look for them and grab them if found. The update works fine on the copy run from the ramdisk, but it updates that copy. It doesn't update the master copy on the HD.

I need to exit the one running from the ramdisk and run the version on the HD against a different profile to make sure it gets the update.

The solution for that will be including the FF program files on the ramdisk as well as the profile in the "save back to HD on shutdown" script.

Meanwhile, it's an "Oops! Forgot about that..." moment, and not a particular problem to work around.
______
Dennis

[DMcCunney]

Updates - the startup and shutdown scripts are mostly done.

As mentioned, I used the Group Policy Editor available in XP Pro (gpedit.msc) to set them up. They get run automatically as part of Windows startup and shutdown process.

The startup script looks like this:

@echo off
:; ldramdsk.bat - Populate ramdisk (Z:) on Windows startup
:; check if ramdisk already populated and skip to end if so
if exist Z:\RD_Made goto End 
z:
f:\console\dos\unzip d:\mozilla\profiles\firefox\4.0\minefield.zip 
f:\console\dos\unzip d:\mozilla\profiles\firefox\4.0\standard.zip 
echo. > z:\RD_Made
:End

and the shutdown script looks like this:

@echo off
:; stramdsk.bat - Save ramdisk contents to HD on shutdown
f:\console\dos\zip -1 -r d:\mozilla\profiles\firefox\4.0\minefield.zip z:\minefield
f:\console\dos\zip -1 -r d:\mozilla\profiles\firefox\4.0\standard.zip z:\standard

F: is my WinXP boot drive. An assortment of command line utilities live in the \console\dos directory, and it's in the system PATH. D: is a data drive holding stuff largely common across OSes. (The desktop multi-boots Win2K Pro, WinXP Pro, and Ubuntu Linux.)

After some experimenting, the simplest and quickest approach proved to be using zip archives. The open source Info-zip versions of zip and unzip are used.

It turned out to be fastest to simply unzip archives of the directories to the ramdisk: details like creation of sub-directories happen automatically. And opening and extracting the zip file means only finding and opening one file. The previous method simply copied the directory contents, and required finding and copying many files.

Likewise for storing the files on shutdown. A recursive zip of the directories on the ramdisk updates the zip files on the HD. Oddly, it was actually a bit faster to use the "fast" compression compression method when creating the zip archives, instead of simply storing the files uncompressed.

There are a few tweaks to be made, like adding error checking and logging - running the scripts the way I do means they are invisible background tasks, so if something fails, I won't see it when it occurs, and will need a log to find out what happened.

But so far, so good.
______
Dennis

[LCF]

looks really promising