Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Mea Culpa (it's corrections time).

Mea Culpa (it's corrections time).

  • Comments 13

One of the rules in Tim Bray’s version of Sun’s blogging policy is “Write What You Know”.

Well, I should have listened to this when I posted my 3rd post, “So why does NT require such a wonking great big paging file on my machine”.  I’ve since been gently corrected by the guys on the memory management team who really DO know how this stuff works, and there are a couple of really important mistakes in my post that need to be corrected.

The biggest one is in the title itself.  NT doesn’t require a wonking great big paging file.  In fact, NT works with a paging file that’s too small, or even without a paging file at all.

But, when you run without a paging file, you run certain risks. 

One of the things that the system keeps track of is the “system wide commit limit”.  It’s composed of the system RAM plus the aggregate size of all the pagefiles on the system.  If there’s no pagefile, then the commit limit is lower.  When you allocate memory that requires commitment (some examples would be if you allocate memory with the MEM_COMMIT flag, or you create shared memory backed by the paging file), the memory is “charged” to the commitment limit.  If there’s not enough room in the commit limit, then the memory allocation will fail.  So your app won’t run until you either free up other memory (by shutting down running applications) or you increase the commit limit by increasing the pagefile space (or adding more RAM).

The other risk of running with too small a paging file occurs when NT attempts to create a dump file in the event of a bluescreen.  If the paging file isn’t big enough to hold physical memory, then the dump file isn’t saved, and an opportunity to improve the system is lost.  You could specify a minidump or a kernel-only dump, but won’t necessarily have all the information needed to debug the problem.

When NT runs setup, it chooses a default paging file size based on 1.5x memory (for certain sizes of RAM, the actual size chosen will be different, but this covers most of the case).  This is a guess, but it’s a pretty good guess, especially since the cost of NOT getting it right is likely a call to PSS.

One of the people commenting on my blog asked: “Is there any reason to set the paging file greater than 4095 (ie. the maximum addressable memory space)?  If so, why.”  The answer to this is actually pretty straightforward: If the memory load on your system is more than 4G+physical RAM, then you will need to have a bigger paging file.  If you’ve got 20 copies of autocad running on your machine, and each of them is sucking down 400M of memory, then you’re going to need 8G of paging file space to hold all those pages.

Btw, one of the tidbits that came out of my discussion with the MM guys was the maximum paging file size on a system:

·         On a regular x86 machine, you can have 16 paging files, each is 4G in size, for a total of 64G.

·         On an x86 running in PAE mode, you can have 16 paging files, but each can be 16tb in size, for a total of 256tb of paging file space.

 

  • This is a good and useful blog, and I think writing about something so intrinsically part of OS as paging is difficult because I am sure it has gone though several generation of changes even though the underlying scheme of how it works has been the same. I had a couple of questions I am hoping someone can shed more light on:

    - How do I set up 16 paging files? I thought each paging file is mapped to a physical disk and hence you would need to partition your drives into 16 disks to get 16 paging files.

    - Isn't the size of paging file suggested by NT/2000/XP also 1.5 times the physical memory because of something to do with hibernation?

    - What is PSS and PAE?


    /C
  • Hmm. First, the easy ones:
    PSS - Product Support Services = Microsoft Product Support
    PAE - Physical Address Extensions = A set of extensions to the x86 processor that allows addressing more than 4G of RAM.

    You may be right /C about hibernation, the MM guys didn't mention that one, so I just don't know :).

    Setting up more than 16 paging files. The easiest way is to have 16 physical disks :)

    Realistically, there's no reason to ever have more than one paging file on a given spindle - you get no benefit and potential difficulty (excessive head motion, inability to defragment the disk, etc).

  • I want 256TB of page file(s)...

    I think I'll ask our SAN guys if I can borrow some space...

    :|

  • How can I create two page files on a single partition? Or do I need 16 partitions to have 16 page files? And is there a way to create a page file on a partition that is mounted to a folder istead of having a drive letter?
  • if ihave 2gb ram but never use morethan 800mb peak, would i suffer any performnce hit setting the pagefile to 2mb or switching it off, then it would never page, which must be faster, right ?
  • Jerry/Greg: I don't know. You'd have to ask someone in the memory management team.
  • Hibernation doesn't use the page file. It uses a separate hidden file called hiberfil.sys, which is the same size as your physical memory.

    When actually hibernating, Windows compresses the contents of memory when writing to disk and decompresses when resuming from hibernation. It doesn't do this to save space, it does it to speed up the disk transfers (less I/O = less time).

    My 512MB Windows XP (desktop) system hibernates in under ten seconds and resumes in under five (P4 2.8GHz, 10k rpm HDD). It's still just about worth doing, since a cold boot takes about 20.
  • >setting the pagefile to 2mb or switching it off, then it would never page, which must be faster, right ?

    Wrong. First of all, this will not eliminate all paging. Binaries (EXEs and DLLs) and memory mapped files that are not backed up by pagefile will still page in and out as necessary.

    So if the system really wants to free some RAM it will always find something to page out. Running without pagefile only restricts what it can choose from.

    Which brings us to the second point. A lot of stuff that is normally paged out to and rarely if ever paged in will now sit in memory all the time, reducing the amount of RAM that's avaliable for other purposes.
  • Thanks Mike and Pavel, I always like learning new stuff.
  • PingBack from http://workfromhomecareer.info/story.php?id=18547

Page 1 of 1 (13 items)