Virtual memory and its impact on games

Ok, time for another educational "little" thread.
I've seen a few people ask about what virtual memory settings are best for games, and similar question. There are also a lot of myths floating around about when/if you need a pagefile and similar, and what happens if you disable it.

So let's see if we can clear up the confusion a bit. As usual, this is going to be a long rambling post. Don't say you weren't warned. Even now, before I've started on the actual post, I can think of at least 3 separate subjects I'll have to cover, and more will probably pop up.

The same disclaimers apply as in this thread
(Although the graphics-related ones are a bit less relevant.)
Instead I suppose we could add these:
I have coded a small OS kernel
No, it did not support virtual memory

What is virtual memory
Lets start with a little detour.
How many processes do you have running on your computer?
How many of them do you think were written explicitly to work together on the same system? (Did the winamp developers consult with the MS Word devs to ensure they didn't create any conflicts, that they didn't try to use the same memory addresses?)
For that matter, do developers have to take into account any possible hardware configuration? (Allocate so much memory on a machine with 1GB RAM, and so much if you have 768mb)

Of course not, so how does it work? How come the 20+ processes you have running don't step on each others toes and conflict every other microsecond?
And how come your games work regardless of whether you have 512, 768, 1024, 2048 or any other amount of memory?

The answer is surprisingly simple. The OS acts basically like a virtual machine.
Its job is to provide the illusion for every application running, that it is alone on the system, that no one will interfere with it, and that it won't accidentally interfere with anything else. Oh, and also that the application has infinite memory at its disposal... Sounds like a tall order? Makes you glad you're not an OS, right?

So, what if, every time an application tried to read/write memory address x, the OS intercepted this, and translated the address to some other address that the OS knows no one else are currently using? That way, the application doesn't have to worry about which bits of memory are in use. And if we're going to translate all memory accesses *anyway*, it should be easy enough to shuffle things around a bit so some of the data might not be in memory at all. We could put it on the harddrive, and just load it into memory when an application tries to access it.

That way, we can pretend to have an infinite amount of memory, and the application developers don't need to worry about how much memory is free, which areas are in use and which aren't, and all that messy stuff.

Of course, there's a downside to it as well. If we need to go and ask the OS every time an application tries to access memory, we'll never get anywhere. We'd easily get a 10-50x slowdown.

Shame, it seemed like a promising idea.

So we do what developers have always done when they come up with a nice idea that'd be ridiculously slow. They say "We need hardware support for this! Build it into the CPU!"

Which is what we've got. The CPU holds a list internally of memory mappings, a big dictionary saying which address should be translated to what. (Of course coupled with some logic to perform the lookup on the fly without wasting any time) Of course, this table doesn't have infinite size, so sometimes, we'll still have to go and ask the OS. But like with the CPU's cache, this can catch all the common cases, practically eliminating the speed penalty.

And because we're changing all memory accesses on the fly anyway, we can create a pagefile to act as extra RAM. When an application tries to access an address that corresponds to data we've put in the pagefile, we can just pause the show for a moment, load the data into RAM, translate the memory access to point to the address we just loaded into, and voila. We can now resume the application, and it won't even be aware that it was paused or that we had to read from the pagefile. Of course, if we do this a lot, the user might notice that the application is running really slow, but that's at least better than crashing because we didn't have enough RAM.

So, with this little trick, and with appropriate support form the CPU, the OS can present the illusion to every running process, that this process is all alone on the computer, and can use the full 4GB of memory (on 32-bit systems), even if the computer only has 256MB of RAM, and even if there are 15 other processes running.

I called this an illusion, and the problem with illusions is that they tend to break down if you touch them. The same applies here. Each running process can pretend that it has 4GB of memory all to itself, as long as it plays nice, and asks the OS for permission before touching new areas of memory. (If it doesn't do that, the OS can't perform the address translations, and then it all breaks down. That's what happens when you get a segmentation fault on *nix systems, or an Access Violation on Windows. A program tried to read/write memory that hadn't been granted to it by the OS. (in DOS, which didn't support virtual memory, the application could just read/write any memory address freely. Of course, if it went above the max amount of physical memory present, it'd crash, so application developers had to keep careful track of how much memory was present, and whether the memory address they tried to access actually existed.

So how does this affect games
On the most basic level, it doesn't. The entire point, after all, is that the OS (with some support from the CPU itself) keeps track of all this behind the scenes, without even telling the running applications that their memory accesses are being redirected, or that the entire process is being paused while we go and read from the pagefile.
The application has no choice, it can't say "Ok, disable memory access here, I want to go directly to this address in physical memory". That address might be in use by another process. So all applications have to go through virtual memory for all memory accesses. And they can't even order the OS to "put this in the pagefile", or "keep this in memory". The OS decides that for itself.

Of course, the fact that all this is going on under the hood can have a noticeable impact on performance. If there's too little RAM on the computer, then the application will be paused very often, waiting for the OS to move data back and forth between RAM and pagefile. But there isn't much to do about it, short of upgrading your computer. If this didn't happen, your game would just crash with an "out of memory" error.

But the key point to take away here is that all this happens whether the application likes it or not, and whether or not you have a pagefile.
You can disable the pagefile if you like, but the OS still has to translate memory addresses to allow multiple processes to run on the computer without conflicting. So you don't really gain any performance. On the other hand, if you disable the pagefile, and then run out of RAM, applications will crash, because they don't have the safety net that a pagefile offers.

So what should I do to maximize performance?
Fundamentally, as said above, there isn't anything you can do. All memory access goes through virtual memory, and there is no way to disable it (short of running MS-DOS). Disabling the pagefile won't change this, resizing the pagefile won't change it.

Of course, you can try to free up as much memory as possible (by shutting down other programs), so that we won't have to push as much data out to the pagefile.
But even this generally won't help much.

When you're running a game, there's this to say about your computer:
It is running your game. It is pumping out frame after frame as if there was no tomorrow. Every spare moment it has (And usually, your computer spends 99.9% of its time in spare moments) is put into the game.

Other applications or processes on the computer won't really get much time to do anything, and because the OS always puts the least used data in the pagefile, guess what ends up there very quickly. That's right, all the background apps.
All the processes that have been loaded, but don't really have anything to do currently, are tossed out to the pagefile as soon as we run out of RAM. The game's data probably won't end up there, because it's being used every 1/60th second.

So, am I saying that there's no point at all in trying to free up memory?
Not quite. Of course the OS can't do a perfect job of this. It can't predict what will be used next (because obviously, whatever is gonna be used next should be in RAM, not in the pagefile), it can only look at what has been used a lot recently. So sometimes, it will push a small bit of game data out to the pagefile. Sometimes it will load a bit of MS Word, MSN, Daemontools or whatever other background apps you have running, into RAM because they just needed to wake up and do stuff for a millisecond.

My point is simply that you won't gain more than a few percent of performance. It might feel good to trim your task manager down by shutting down processes, and it certainly doesn't hurt performance to do so. But it won't gain you a lot either.

Pagefile settings? No big deal. Make sure it's big enough to cover when you run out of RAM. Performance? No difference. If there's enough RAM, the pagefile won't be used. If there isn't enough RAM? Well, then it's use the pagefile or crash. Again, no big loss in having a pagefile then.
Of course, before some smartass tells me "yeah but I have 800GB of RAM, and Windows XP still accesses the harddrive sometimes when the pagefile is enabled", I'd better make a disclaimer here:
Yes, it does. There are two reasons for this. One is that no OS is perfect. They can't look into the future, and they can't tell for sure that they won't suddenly need 800.1GB of RAM. They have to play it careful.
And the second point, since the OS has to assume that it might suddenly need the pagefile, what is the most sensible thing it can do?
Let's say your RAM is filling up, there's maybe 8MB free.
Now your game decides to suddenly request 1.5GB more memory.
If the OS had simply avoided using the pagefile until now, it'd have to freeze everythingwhile it wrote 1.5GB of data to the harddrive, just to make room for this new allocation. How would you like your game to freeze completely for 20 seconds? Because that's what you'd get.
Instead, the OS is a bit more pessimistic. Even when there's plenty of RAM free, it plays it safe, and gradually writes a bit of data that hasn't been used for ages, to the pagefile. That means the occasional harddrive access, but it doesn't take more than a moment (because it writes maybe 4KB each time), and just as importantly, it does this when the harddrive isn't otherwise needed, so your game doesn't have to wait for it to complete, but can just run along at (almost) normal speed. But then, when the game decides that it needs 1.5 GB more data, the OS can handle it fairly quickly, because it's been foresighted enough to put a lot of unused data in the pagefile already. So there might even be 1.5GB of RAM free, and your game can continue without missing a beat.
In other words, even when it seems that the pagefile is accessed where it isn't necessary, it's part of a long term strategy. If the extra memory is never needed, you've wasted the occasional millisecond. Hardly a big deal. But if it is suddenly needed, you've saved maybe 20 seconds.

Additionally, for those who don't want to see the pagefile being used as they have 4GB RAM, have you tried just letting Windows manage the pagefile? I can't tell you exactly how Windows treats the pagefile, but my personal experience is that if you try to specify the pagefile size, Windows will use the pagefile a lot, even if there's plenty of free RAM.
If you let Windows manage the pagefile, it basically won't touch the pagefile unless it's low on RAM. When I tested this, I was surprised by how much better performance you get by letting Windows manage the pagefile.

Disabling virtual memory? Can't be done. It's an essential part of any OS. Simple as that. Same goes if you're the one developing the game. You have to go through virtual memory. You can't choose "Ah, but for this bit of data, I think I'd like to bypass virtual memory, and just access the physical memory directly".

So roughly speaking, give or take a few percent of performance, the conclusion is:
Don't bother trying to squeeze extra performance out of your computer in this way. There isn't really anything you can do.

 

Geez, let me be first to congradulate you on a good thread

confusing, yes, but really detailed and helpful to understand what virtual memory does.

 

Nice one Jalf.

So there's no way to turn off the pagefile completely? That sucks for people who have 4GB of RAM.

 

There is boon to Virtual Memory some forget, while you play a game you are not actively using 'explorer.exe' and a few other core system components. Since these are not inuse they will be swapped to the pagefile and your game can get more effective memory. You can experience this is when you tab out of a game and have your screen redraw very slowly or have applications that are already open not just pop to the front. The reason they take a bit longer is because they were pushed to the pagefile and they are getting swapped back into active memory when you try to access them.

This is probably of little relevance here, but if you have 2 hard disks in your system, place windows on 1 HD, and put your VM on the other, with a smaller amount on the disk with Windows. If you have the 2 HDs raided together ignore this. But windows will see a benefit in the paralell load time of using 2 disks.

Edit:

Forgot to mention for the above poster, the use of Virtual Memory is governed by a chip on your board. The Memory Management Unit handles the translation he describes, and it sits between your CPU and the RAM / Hard Disk, so there is no penalty for checking Virtual Memory, the penalty for VM comes when you actually need to load something from it,

 

You can turn off the pagefile. You can't turn off virtual memory. The point is, the OS will always translate addresses and make each running process believe that it has 4GB all to itself.
All you can change is whether or not the OS is allowed to support this illusion by using the harddrive to provide extra "memory".

Actually it's integrated into the CPU. But yeah, it's a dedicated bit of hardware doing this very efficiently, so there isn't really a performance loss (untill data from the pagefile is needed)

 

Thanks for the lecture!

 

Very nice Jalf!

 

Well done Jalf. I knew all that, but still, good guide for those who don't

 

Jalf, I think I love you.

 

^^lol at that. But i kinda see were pete come from