Another thread about RAM, only this time --> RAM usage under Win 7, pagefile, utilization, etc.

This post is a duplicate of #92 in THIS THREAD, but I decided to move it to a new thread so that more people can join the discussion, which has a proper Title now. Look at the old Thread for preliminaries if you wish.

Ok, I'm back in the game and enjoying the noticeable difference of having 6GB over 4GB RAM, for the second time

However, after some experimenting I still have a big question mark in my head!!

Whenever I copy a large amount of data from/to the HDD, ALL of my available RAM seems to be used for that process, forcing windows to start caching everything in the Page file instead. This makes working on the PC during data transfer of big folders (say over a few GB) very hard. It seems to be sliiiiiightly better than with 4 GB, but still nothing I have experienced with Win XP!!

Why does a file transfer need SO MUCH RAM? If it is kind of a buffer, it is really huge to take up nearly 4GB RAM!! Do you also have this problem?? Try copying say 10 or 50GB folder from one place on the HDD to the other. Or even from external HDD to the internal or the other way around!

Here some pictures:

1st one: During the file transfer (looks like that a few minutes after I start - All of the free memory is used up and becomes "standby")



2nd one: Right after the file transfer ended.



You can see the Hard Faults per sec counter .... everything is very slow just as if I had 2 or 4GB RAM!

Any explanation/suggestion? Is it the same on your PCs? Try it out...

Here a live demonstration VIDEO ( Alternative link) of what I mean! I didn't play around with minimized programs and windows too much not to cover the RAM usage area windows, but you can see the Hard Faults appearing... normally I get them up to 40-50, hence everything is moved back to the HDD and the system is slowed down immensely!

 

show me your disk activity tab(if there is no privacy concerns).

when you said system is slowed down immensely, is it after the copy is done ?

 

No - only DURING the copying process and shortly after that while everything is being read out of the HDD and stored in the RAM (just like if you just started the PC). But the PC is almost unusable during the copying process - I would say due to the constant need of reading everything out of the page file on the HDD instead of the RAM. Firefox windows very often even turn to "Not responding" for a while, etc.

You mean the disk activity tab during copying process or...?

Here another 2 screenshots during the data transfer. Nice Hard Faults per sec despite the 6GB of RAM and the fact that only 2GB are actually in use by applications! Why does a copying process take up such amount resources and slow the computer down to almost unusable!?? As I said - a bit better with 6GB instead of 4GB, but not gone. Many of us might not realize it as you don't really copy such amount of data every day... But it it really frustrating when you do. I was copying 120GB yesterday - didn't have enough patience to do anything else on the PC during that time... almost impossible to browse, everything lagging, everything taking a lot of time...

 

As I mentioned in your other thread; this is a Win7 'feature'.

However, the 'extreme' lagging is not what I experience with 8GB RAM on my systems (I too move 100GB+ data around almost daily).

Are you running a gigabit network with these transfers, or wireless?

If gigabit, do you have jumbo frames (etc.) all optimized?

Also, what is the cpu usage during these copying/lagging scenarios?


Note that if I'm trying to access the Internet from the copying/copied to computer(s), then it will be 'laggy'. However, any internal programs that I care to run are not affected (too much, and certainly not 'laggy') by the background copying of the data.

Hope we can track this down.


Oh! To keep everything in one thread:

Did you disable the 'system managed' pagefiles on each partition?

Good luck.

 

Thanks for joining me again!!
down in the text

 

Okay, you're copying locally (not to a network).

What exactly are your pagefile specs?

Can you post a screen shot of your disk(s) Policies page in the Device Manager?

How many programs (during a copy session) are you running to experience this lag (one, or more)?

I'm the one who mentioned the benefits of multiple pagefiles to you (one on each partition - each one 'system managed') - just curious if you have found other sources on the 'net for such setups.

Finally, what is your HDD again? Is it a 7200 RPM model...

 

down there again

 

I don't think any amount of RAM would help in this case. You are just overloading the HDD which is not good at handling multitasking work pattern

 

How exactly am I overloading it? By maximizing a minimized Firefox window with a web page? Or by maximizing skype from the taskbar? And this is still a 7200rpm HDD.

I tend to blame Windows, as I never had such a problem under XP with any HDD - neither 5400 nor 7200 ...

 

I agree with chimpanzee; you're simply overloading the HDD (which is known to not multi-task too well either).

You may want to check both boxes in the drive Policies window.

Don't disable your pagefile though - instead, enable a 'system managed' one on each partition.

Oh, and I would highly recommend installing the IRST driver 10.1.1008 if you haven't done so already.

See:
http://downloadcenter.intel.com/Det...apid+Storage+Technology+(Intel®+RST)&lang=eng


More RAM would help, but I just don't know if it would be enough to alleviate the symptoms that you're experiencing with your usage pattern though.

Keep us informed.

 

Blame whatever you want. Your disk monitor said that it is working very very hard.

Your firfox request is just like you are going into a bank with a long queue of customer each requires lots of thing to be done(that is your copying task).

Some HDD has a better multitasking policy which would balance this out so this troublesome customer would not hold up the rest, your HDD seems to be doing the opposite.

I did a simple copy test(a 3GB file from one partition to another) on my laptop. Other than the initial time(2-3 seconds or so), it works as if nothing happens during the copying process, I can do the other thing just as well. Yes, a slight delay(may be a second or two) but nothing more than that. Never lose responsiveness. Mine is Scorpio Black, known for its multitasking capability(at the expense of throughput).

Unfortunately, I don't have that many large files to test exactly like your scenario.

BTW, your disk monitor didn't show any page file activity.

 

I had similar results transferring a 15.7GB folder from WD AV 500GB 5400RPM (via eSATA) to WD Black 500GB 7200 RPM internal drive in optical bay SATA caddy. I had Firefox, iTunes, uTorrent and Excel running at the same time. And this is with just 4GB RAM, T7350 CPU. No pagefile because of 60GB Vertex SSD as main drive. Power management set to performance.

 

That's the one I am using. Exact same version.

3GB is nothing. I have no problems with a 3Gb file either - it would all fit into my RAM I'm talking about 10 to 150 GB of data!
I had the exact same problem while using the original Toshiba HDD (5400) that came with the laptop. And I don't really know about multitasking, but Scorpio Black is actually noticeably slower than 7K500... according to all benchmarks and reviews at least...

 

Yeah, 3GB transfers are easy.

Just curious, is this any better with 6GB RAM vs. when you had 4GB's?

If there is no difference at all, then it surely points to the HDD as being the bottleneck for you.

 

that may not be the same situation as hers. you are have 3 seperate disks and my guess is the firefox etc. are all using the SSD so your copying activity would have no impact on those.

She is asking the same 7K500 to do a huge copy(saturated the queue) so any other request would find it hard to squeeze in between.

This actually echo back the why SSD shows huge improvement during boot up. That is a typical scenario of multiple unrelated disk request all at the same time. The only difference between that and hers is that her copy task is HUGE whereas most Windows startup task are short burst.

 

i would try to create a larger file when I went home, what size you think is large enough ?

also, you need to know how to read the number of review and benchmarks. what you read are mostly 'throughput'. That is one of the most deceiving number and least useful for modern day OS or usage.

Why people(well those in the industry anyway) keep on stressing the IOPS when taking about SSD ? That is usually the more important number if you want a smooth ride.

My Intel X25M 80G has a slower sequential write throughput than my old HDD yet everyone at my office who has replaced them(on my recommendation) don't want to go back to the HDD.

 

True, but in each case with those programs, they are not actively and constantly accessing the SSD. Which, in her case, like mine, would be a momentary access which should not be enough to slow down the entire process and consume all that much in resources.

 

Anything above 10-15GB. Doesn't have to be a single file - I copy folders with multiple files.

Open at least 6-7 Firefox windows in advance each with 3-4 or more tabs with different webpages and minimize Skype to tray. Start copying and wait for at least about 20-30 seconds. To be sure - even wait until at least 4-5 GB are already copied! Then start clicking the windows and tabs one by one and assess the latency. Refresh a few tabs and while waiting go to another tab or window. Maximize skype which has been minimized the whole time and scroll up and down. But make sure the page-file is enabled!

 

well in her case, the momentary access has to wait because the 7K500 is busy handling the same drive copy. That is my bank analogy. This is what I experienced, in the bank. If there is a nanny in front of me, she can keep on chatting with the teller and it is the culture in this part of the world(Canada) that I cannot just drop a letter(and nothing else) until she has been served. In some other country(mostly Asian), I can interrupt briefly and drop my letter and go.

 

I don't use firefox but I would try to mimic that in IE and enabling page file is something hard as I always have that disabled.

 

Here is probably the reason for your productivity I will try disabling mine too in a few minutes and report back after trying the scenario

Other then that I found out IE is a bit easier to handle, whereas Opera is the lightest. But still the lag is quite noticeable with IE too

 

LOL. Good analogy.

I'll run the test again on my HDX notebook later - another computer. Though I never experienced these issues as I am constantly moving very large file directories. For sure, it has nothing to do with RAM as on all my rigs, there's only 4GB RAM. Though, I run very small fixed pagefiles (250 or 500MB) or none at all. That might be contributing to her problem along with having a drive that has relatively poor IOPS.

 

Don't know how to benchmark/compare IOPS, but here are the simple typical results of my HDD using HDTune...:

 

You could use IOmeter to bench it: Iometer project

 

Gracy123,

For IOPS numbers go to the Random Access Tab and run the tests there.


The graph you are showing looks normal, btw.

 

Here the results from the 1st run. 2nd run was slightly faster, but pretty much the same:

 

I'm typing this as I am copying my experimental 15GB from one partition to the other with DISABLED pagefile!

HUGE HUGE HUGE DIFFERENCE!!!

There is almost NO delay at all! I say almost, because my fine senses did catch a ms delay here and there while loading a web page or starting a program, but actually if I didn't know I am copying a file - I would probably have never noticed anything different!! Everything is as responsive as it should be or let say the reaction speed is over 95% of what it is when not copying anything, which I consider perfectly normal under such circumstances! Lovely!

I don't really know what is so great about Windows 7 and pagefile managing, but I am definitely keeping it disabled unless I run in some troubles?

The difference is enormous!

 

Unless you hit OOM(then please add more RAM, still have 2G room as far as I can tell) or a brain dead application, disable page file has no harm.

 

Gracy123,

Glad you've seemed to find a simple fix (I can't call mutilating Windows a 'solution' ).

Just for fun, I disabled the pagefile on a fully updated (SP3) WinXP install with nothing else installed and 3GB RAM. The system crawls. Noticeably. With nothing else running.

When trying to use it 'fast' (ie. doing multiple things at once) it visibly stutters.

With the pagefile put on system managed (single partition 80GB HDD), the system feels much like Win7 to me - multi-taskable.

I'll try this in a few minutes on my Win7 based systems - I'm ready to be proved wrong.

 

I have 2 XP VM running (and of course page file disabled) and haven't experienced the 'crawl' and I am running multiple things inside it.

Not trying to prove it as right or wrong, just that I cannot repeat your experience.

and disabling page file is not mutilated. If it is such a critical thing, there is no way Microsoft allows you to disable it. It is an option from day 1 of NT, just that back then most system has fewer RAM and the speed difference between RAM and HDD is narrower than now, and the price difference was much larger than now.

Page file is just slow RAM, nothing magical.

 

This is why I want to test/prove it for myself once more...

Btw, XP in VM is simply running in memory anyway, right?

 

noop. it still needs to access its files from the SSD/HDD.

 

Very excellent. Glad you got that sorted out. I keep a fixed page file of 500MB on my HDX (gaming notebook) only because some games require a pagefile or they won't load. I did test out a transfer. I had to transfer 86.1 GB of media files to a new WD black that's installed on the media bay. One thing I recalled is that I don't cache to disk in Firefox. Only to memory. You might want to set that in about:config.

In any event, there was a very slight degradation in transfer performance when I loaded a website that streams data. That occurred around the 60GB mark of that 86.1GB transfer. CPU util shot up to near 50% at one point. The whole transfer took about 15Mins.

As a note, on my media notebook, no page file. I transfer large media files and folders often. Usually weekly. Sometimes I stream from it. My tabletPC's don't do anything that requires a pagefile. On my other notebook, I have a 250MB pagefile for Adobe Photoshop. Seems to need it. Especially since I'm only running 4GB.

 

Oh, I didn't even know you can disable that in firefox locally!! Is this the right option to disable: browser.cache.disk.enable? Do I need to disable any of the neighboring ones as well?

I also use Photoshop daily - I have ran into troubles with it while I had the pagefile disabled with 4GB ram. Will see how it goes with pagefile off with 6GB now!

 

Gracy123,

Okay, disabled all pagefiles.

Rebooted (although Windows didn't require me to) and proceeded to load programs as I normally would.

In twenty minutes of real world use, I had 4 programs running. Viewing a youtube video resulted in sound stuttering (system otherwise sitting idle - with 4 apps going...), opening up Adobe Bridge resulted in a 'Windows needs to close this program because of a problem'...

Not quite a blue screen of death, but the Win7 equivalent if you ask me (programs just don't close for no reason).


I enjoyed the fast shutdown time - the speedy boot up time (saved a whole 10 seconds!) and the responsiveness of the system when running/using one program at a time.

However, that is where the honeymoon ended with no pagefile in use (for me, anyway).

I had my accounting program running, IE (with 4 tabs - one with the youtube video), LR3 and PS CS5 in addition to attempting to run Adobe Bridge when it crashed.


After enabling the pagefiles on all (4) partitions of my drive, I have been running the system for the last hour with over 40 programs running, with no hiccups (stability-wise).

Keep in mind that I have 8GB RAM.

Also interesting is that LR3 was pegging the CPU for a few minutes (although the system still seemed responsive at the time) with no pagefile enabled.

With pagefiles enabled, 40 programs running, 148 processes and less than 4GB RAM used (Commit (GB) is 4/39), the system is still ready to do real work and idles between 0 and 4% cpu usage.


I hope this has helped you understand why a pagefile is not optional (yet).

 

Hmmm, I will report back once I collect some experience. So far everything is better, no issues. Haven't used the PC actively though. Will report tomorrow.

 

That's the one. Also the SSL one below it. But that should be disabled by default. Make sure the memory cache is enabled. browser.cache.memory.enable, with the capacity (browser.cache.memory.capacity) set to -1 so it can use whatever available memory as needed.

Photoshop benefits nicely with more RAM. Most of Adobe products do for that matter.

 

page file is optional or else Microsoft would not allow you to disable it. No matter how your test is run, this is a simple fact.

 

chimpanzee,

No doubt it is optional in your usage pattern. No doubt it is not in mine.

Microsoft has never recommended disabling the pagefile - it gives controls to give the user the choice to further optimize it for a specific use (like you have in your case with disabling it completely - and in my case; by enabling it on each of my 4 partitions on a single drive - even with 8GB RAM installed).

The primary reason to have the ability to easily disable the pagefile completely is to clear its contents. This has brought back many systems from unexplainable behavior - including imitating HDD problems and/or Memory problems.



Once such example I actually ran into last week:

I updated a Windows XP system to SP3, updated the nVidia drivers to 266.58, updated the IDE driver to IRST 10.1.1008 (along with switching it in the BIOS, of course) and from a rock solid stable system for the past 3 years, it went to consistently crashing in less than 5 minutes of booting up.

At first, I thought the drive was going: win32.sys was giving errors, but after a few reboots it would have have different OxO errors and with a different file each time so I thought the RAM needed to be replaced.

Instead of testing the RAM or HDD (and going with my gut feeling simply because I wanted to get home), I simply disabled pagefile.sys rebooted a couple of times and when the system seemed (slow! - just 1GB RAM in that system) but stable enough, I re-enabled the pagefile (to 'system managed', of course) and it has been stable and faster than ever.

So disabling the pagefile just because it is possible to do so - is just like a Ferrari that can accelerate faster if you put in the right kind of airplane fuel - but it still doesn't mean it's a good idea to do so.

In conclusion, the 'simple fact' is that the control is there for a different reason than you want it to be. No prob! Enjoy your faster, but potentially unstable system for as long as you can.

 

Care to share what games 2.0? After asking many times where it was stated "some programs need the pagefile to run" I never got answered except for one. Titan Quest. I tried the demo (maybe that makes a difference idk) and ran through it twice without a pagefile and no problems.

Not so sure why the PF is debated so much, IMHO some people need it some don't.

How to determine the appropriate page file size for 64-bit versions of Windows

Also you should be aware that paging can still happen with non modifiable data types if just the pagefile is diasabled. Most executables are non modifiable.

 

The most egregious example (I posted this in the SSD thread a while back) is Dawn of War II, which requires a 1.5 GB(!) minimum pagefile to run.

 

Thanks for the info Judicator. I'm interested to try these things. Do you know if the full game is required or will the Demo suffice?

EDIT: Is that only available through STEAM?

I did find the following (just the messenger here)
Steam Users' Forums - View Single Post - Dawn of War 2 installed, doesn't run

So it seems there's a switch to disable the check and from accounts people seem to have successfully run it without a pagefile. It also seems there are people who tried and did not have enough memory so of course in those cases the pagefile is needed.

 

You mean browser.cache.offline.capacity ?

What does this actually change and what would -1 mean?

 

It's a THQ game that uses Windows Live, there's some Steam integration, but you can buy it standalone (at least the first 2 versions, DoW II and DoW II Chaos Rising... the 3rd, Retribution (preorder) is apparently going to be Steam only). I don't know if the demo version has/had the pagefile requirement (I only own the full versions), and the trick to disable the pagefile check is news to me; not that I'm terribly worried about it, I have a nice big platter to go with my SSD that has my pagefile on it. Still, they could have just as easily _not_ put in that switch to disable the check, so...

browser.cache.offline.capacity is letting the browser store cache data so it's available even if you're offline; in other words, it lets you access the last version of a website cached even if you have no internet connection. A -1 would probably mean that it would use as much memory for said cache as it is asked.

 

Aside from two games already mentioned, I also thought Dragon Age Origins required it. But I disabled the pagefile on my gaming rig and it runs just fine. I don't have Dawn of War installed currently so, for now, I'm good to go without the pagefile enabled.

 

No, browser.cache.memory.capacity. But if you don't have it listed, don't worry. It will default to -1 anyway. Sometimes some addons can change or add this line to increase performance.

As mentioned by judicator, -1 simply means no defined or reserved amount. It will dynamically use whatever memory or capacity that is required.

So you're all good then.

 

That is exactly why it is 'optional'.

Noop. disable page file is a very huge operation(it usually requires restart) so it is not designed as a way to clean up and page file don't need clean up.

Pontentially unstable, that is true. But that is simply because there may be OOM situation. The analogy is a bit like should I wear helmet ? If I always appear around construction site area, I think I should. Otherwise, I doubt anyone wear a helmet just because there is a chance that something falls, in day to day life.

I am not against page file(my work machine has it enabled because it use SSD and I have the potential of exceeding the RAM size), just don't like the urban legend that page file is a must no matter what, which is simply not the case. Enable it if you like but all tests I have done show that under a situation when they are not needed, the responsiveness of the system is suffered(not much but noticeable).

 

My personal, not professional opinion based on the experience I have collected so far, is that the pagefile is a wonderful thing whenever the installed RAM is insufficient! It not only grants stability, but also enhances system's speed and reaction latency. No doubt about it!

But unless it leads to programs crashing (which I have experienced with disabled pagefile and 4GB RAM), it seems to speed multitasking A LOT to have it disabled as nothing is being cached to the HDD! In my case, described above, it makes a HUGE difference!

I will report about eventual stability problems, until now - nearly 24h since I have disabled it - no problems at all. The system is faster and more responsive, HDD activity is lower.

 

Exactly, I have been running with no page file for years on multiple systems and not a single issue. Out of memory OOM is relatively easy to avoid. Simply put, don't run so many memory intensive programs at once. When you're not using them, close them down.

In the vein of chimpanze's famous analogies... running so many programs at once is like leaving all the lights in your house on just because you happen to have been in or might return to a room.

 

Well... this is actually contra-productive - Multitasking requires exactly the opposite But time will show - I am using my PC just as before - I have everything I might need soon minimized, ran Photoshop a few times - everything is stable so far.