XP vs Vista vs sp1 vs 2008 vs 32 vs 64 benchmarked

After seeing enough benchmarks out there that have been mis-run or talk without proof, I have finally finished benchmarking several versions of Windows to show the differences.
I conducted each test using an Dell XPS m1330 with the following specs:

200GB 5400RPM Hard Disk in ATA, not AHCI mode
2 Gigs RAM
Core 2 Duo T7500 @ 2.2GHz
NVIDIA GeForce 8400 GS
174.94 GeForce Drivers

The test was conducted by wiping the previous partition, installing the operating system, installing the chipset and all drivers, installing the benchmark, and rebooting prior to testing. This allowed for each operating to start on an equal playing field.
The test itself was conducted running Pasmark's Performance Test as a 32-bit test on 32-bit operating systems and 64-bit on 64-bit operating systems, with the exception of all graphics tests running as 32-bit. This allowed me to test the 64-bit capability of the systems, unlike most tests out there. Tests conducted on PCMark as well as many older 32-bit benchmarks tests the emulating capabilities of the 64-bit systems, rather than the 64-bit system itself. This has led to the misconception of 32-bit systems running faster than 64-bit systems.
3DMark and similar graphics benchmarks including in game benchmarks test 32-bit graphics as well as 32-bit system performance, thus testing the emulation capabilities again. Unfortunately, Passmark only tests 32-bit graphics as well, but this will be applicable for almost every game out on the market. There have been a few games (to include Half-Life 2) and several 3D applications that have been ported to 64-bit to experience the improvements the 64-bit throughput can give; however, this will not be tested today.
The test compared Windows XP, XP 64-bit edition, Vista 32-bit and 64-bit Home Premium, Vista with SP1 upgrades, and Server 2008 Enterprise 32 and 64-bit editions. This test gives the most common operating systems and tackles the 64-bit as well as the latest claims of Windows Server 2008. Vista was set up using full Aero as well as Server 2008 being set up using the Desktop experience and full Aero.

I apologize for the massive test results, but there's a lot of information included:
http://www.imagebam.com/image/9786675364575

Summary of test results:
CPU:
All 64-bit OS dominates with the exception of Prime Numbers and SSE.
Best-Worst: 2008, Vista, XP

2D Graphics:
64-bit performed better. XP tested very well with the exception of 2d shapes.
Best-Worst: XP, 2008, Vista

3D Graphics (no full screen):
32-bit performance better except with 2008.
Best-Worst: XP, Vista, 2008, Vista SP-1

3D Graphics (full screen/complex):
64-bit performance better except Vista SP-1
Best-Worst: Vista, 2008, XP 64, Vista 32, Vista 64, XP 32

Memory:
64-bit performance better
Best-Worst 64: XP, Vista SP1, 2008, Vista
Best-Worst 32: Vista, Vista SP1, 2008, XP

Disk:
32 vs 64-bit had little difference except with Vista's sequential writing. 64-bit performed better there.
XP and Server 2008 both showed even, but noticeable faster results vs Vista.

Overall:
64-bit is improving and shows a better overall average performance. The 3D graphics are dependent upon the drivers released and I have seen consistently improving performance with nVidia's drivers. The 64-bit performance has exceeded in this benchmark which surprised me. The push for Vista has finally exceeded XP's performance, but SP1 has been a step back for gaming in vista. I've also noticed this in 3dMark tests I have run.

Vista has provided more stability with more compatibility per year of existence vs XP. I could never install XP 64 with 100% drivers on any of my computers until last year. I could install Vista 64 years before that. I noticed XP would constantly become corrupted after certain software was installed. Beta software could actually damage the operating system. I have not once run into this for Vista. Furthermore, Vista has been much more tolerance to installing and uninstalling tons of trial software. XP would constantly become slower without an install monitor to protect it. This prompted me to reinstall XP about once ever four to six months. I only reinstalled vista because I accidentally hit the magic "format me" media direct button included with the M1330.

Server is not designed to be a desktop operating system, but it is designed to be very stable. Currently, you can use Vista drivers, but as changes continue, this may not be an option. Server did perform noticeably better than Vista or XP having been in development longer than any other Windows operating system. Server 2008 would probably be the better choice over XP 64 due to XP 64's compatibility issues it still suffers from. The only real downside would be the cost of obtaining a server software.

I expected XP to perform better in graphics but the push has been making some headway with Vista. The only real show of XP's performance is on the disk performance and non of the tests I conducted included a boot timing. PCMark does include a boot speed test, but can be inaccurate due to AHCI drivers and DMA changes during the boot process. The best result you can obtain would be to use a stopwatch and time the boot, but this can even change due to prefetching.

The boot results differs between computers with different performances. My M1330 Boots Vista faster than XP and about the same as Server 2008. My wife's HP dv2000 boots much faster in XP than in Vista. The drives are the same speed, but she has 1GB of RAM and a slower CPU. I don't have an answer to that swing in performance, but my guess is that it has to do with Vista using more RAM than the pagefile.

Another important thing to note is that even though Vista without SP1 performs better in these tests, I would not recommend staying that way due to several bugs such as transfers between drives going 10% of the speed of XP. The same logic is applicable with running games on windows 98 to gain even more performance.

For those who think the plunge into 64-bit will leave you missing drivers and having compatibility issues, don't worry. Most computer vendors do not provide 64-bit drivers or are very slow to update them, but with a little googling you will find around 95% of Vista 64's drivers, if not all. The drivers for XP 64 tend to be more difficult to find. The 64-bit side does not support 16-bit software (it won't function at all) and drivers have to be digitally signed. Be aware that while most software will work, there may be issues with custom software schools use for VPN or similar.

Work pending, I may provide additional benchmarks using Vista Business and Windows SP3 and include 3dMark benchmarks. I could not perform that this time due to the M1330 downclocking due to heat in 3DMark. It takes a bit more time to set a profile to prevent that.

Until next time, have a great day.

Respectfully,
Trebuin

 

Wow Treb.... Thats alot of work! It is nice to see you back again.

With respect to the testing, I didn't find many surprises as I have been an avid supporter of 64 bit. Not being a gamer, I also believed the word of pretty much every gamer out there that the graphics were better in XP. I think you may see some XP gamers running to make sure they get 64bit installed with the outstanding performance of XP64 in graphics.

There is alot of work here; something that should be considered for front page news. After all, I believe its probably the most comprehensive report as such I have seen anywhere.

Again... Kudos and a job well done! Now go and protect our skies big guy.

 

Thats extremely interesting....great job...I mean it...

 

Wow! Thanks for the thorough comparison. Much appreciated.

 

Wow that is pretty darn thorough. You did a lot of work. I am sad that SP1 actually hurt performance. But I guess that can come hand in hand with stability? I doubt I;ll take the 64bit plunge yet. The performance looks good but I just don't know enough about support for all the software I need. And I don't need to buy another OS.

Great job thanks! Repped

 

Thanks for the numbers.

I feel safe using XP32.


However I know that 64bit is the future and hopefully sooner or later all Windows OS will go that way.

I'd like to see how how XP 32 with SP3 compares.

 

Wow good work here

 

That's a very impressive body of work - congratulations! One point I would add, however, is that the short-term operating behaviour of _Vista may not be a good measure of it's long-term behaviour, particularly with respect to I/O and especially disk I/O. The I/O manager in _Vista is an adaptive manager that over time will adjust the way in which it schedules and manages behind-the-scenes tasks like disk I/O in order to optimize the visual appearance of performance - sometimes at the expense of actual through-put performance.

Basically, because of the adaptive feature, the initial performance times you get with _Vista when you first install it will be different from the performance times you eventually get when the I/O manager learns enough about the user's habits to modify the I/O scheduling. There's a whitepaper on this somewheres on the MS website, but I don't have the link (or the title) handy right now.

 

Great job,thx!!!!

 

Errr...what happen to SP3 for XP?

 

Very detailed work...Excellent!

You hit the nail on the head with one of the KEY differences between XP and Vista that I really haven't been able to put into words till just now...

XP is super-fast after a fresh install, and then with all the windows updates, driver installs and program installs, starts to slow down, little-by-little... Thats why you're always having to re-install the whole OS every few months, to get that snappy, clean feel again...

Vista on the other hand, doesn't slow-down with time (in fact it seems to get more responsive with time if anything...).

More reason to stick with Vista SP1 32-bit for the time being...

 

Awesome.

That's a lot of stuff to do.

+rep ;]

 

Very informative read Trebuin, thanks!

 

Going 64bit Vista. There's always VMware for that one of two program that might not work. But so far, of all the software I use, no one has reported compatibility problems and I plan on buying all new hardware so 64 bit drivers won't be an issue.

 

IF someone like you doesnt deserve a rep, then nobody deserves it.

Thanks for the hard work.

 

Excellent work Treb. I hope this forum as well as our specific 64bit dell threads encourage many more users on the net to make the switch to 64-bit. Once accomplished, it really is a great thing.

 

Thanks for the comparisons Trebby! Great work!

 

Nice work!!!! This is very interesting. Thanks.

I agree with you. Vista is slower after a fresh install (and even right after SP1 update) but get faster when you give it some time.
Also you can Tweak Vista to run very fast.

 

That was incredible, it's good to see someone take the time to do something like this. I've been wondering about this for a while.

+rep.

 

Ok, here's the Microsoft whitepaper I referred to earlier that discusses the I/O optimization/prioritization scheme in _Vista, and how it differs from WinXP. The paper is titled " I/O Prioritization in Windows_Vista;" a brief discussion, plus the link to the paper and some related links can be found on this MS webpage.

 

Sweet *hugs own 64-bit Vista with SP1*

 

Damn nice piece of testing.

Thanks.

I have Vista Ultimate 64bit SP1 and found that a fresh clean install of XP 32bit boots up about 10 seconds faster than a fresh clean install of Vista.

Same HDD, same 4 gig's of ram. Vista is a hog but I prefer the "Mac" feel it
has that XP lacks.

Thanks again, it was a very informative report.

 

Great stuff. Outstanding work ...

RR

 

Man this is a hard pill for the XP fanboys to swallow.

So you are telling me I didn't upgrade when I installed XP over Vista that came with my comp?

I don't know I still feel that a fully tuned and tweaked XP is still better than Vista fully tuned, but I obviously don't have data to support this.

Good work though. Now I am less hesitant when claiming that XP is way better. I would love the day when I can switch to Vista without any worry of performance loss or bloating. Yet even microsoft claims they are learning from the mistakes they made in Vista as they are working on windows 7.

I guess I'll still cling to XP a little longer. Lets see what this SP3 will do.

 

The numbers are interesting, but most of the fluff about Vista being "better and more compatible than XP" is anecdotal evidence based on *your* personal experience with *your* computer, and would have been better left out. (The "Vista was stable years before thing is also not true because Vista has not been out for that long)

The data is interesting, but you're a bit too quick to jump to conclusions.

 

The change from Win2K (or worse, WinME) to XP should have been substantially more complicated than the change from XP to _Vista, in large part because Win2K and XP are two qualitatively different OSes, whereas _Vista is essentially XPSP3 with a funky new GUI. There are, to be fair, a number of other differences as well, but that represents more a matter of whether or not certain otherwise XP-compatible modules were upgraded or not.

That being the case, it should not be surprising that most people who went from XP to _Vista did not experience the sort of gut-wrenching issues people faced going from Win2K to XP.

That being said, I have no doubt that the average user will notice little or no difference between _Vista and XP in their ordinary everyday use (other than the aforementioned funky new GUI ). However, in certain areas, XP does, in fact, continue to outperform _Vista - gaming, for example - and thus continues to be a legitimate alternative to _Vista, particularly when one takes into account that (a) _Vista consumes considerably more system resources, and (b) at least with respect to _Vista Home, which is what most individual consumers have, will get two years' less security support than WinXP.

 

Yes, it is sad that gaming and CAD is outperformed in XP. My past tests have shown Vista with higher 3dmark scores and even the higher full screen 3d tests here (3d complex=full screen, others=windowed) on Vista shows how greatly Vista suffers with gaming compared to XP. The CAD work I also do seems to render much faster and smoother in Vista so I know that quality has been lost somewhere. I probably should go back to XP for some of the games I do play (oblivion, Halo 3, Call of Duty 4) but the drop in FPS in XP doesn't seem worth the increase in graphics performance in XP.

 

Thanks a lot for the hard work on this, it is much appreciated. I have a free Vista Ultimate 32/64 bit coming for participating in a MS something-or-other and I plan on going 64 bit shortly after I receive it. This only cements that thought.

 

Have you been running any of your _Vista systems long enough with the same user (or with consistent usage patterns) to be able to detect any noticeable (or, more likely, measurable) effect of the new I/O management algorithm and/or the new superfetch function?

 

I've played around with Longhorn since 2001 and Vista since 2005 when testing it. I've noticed somem effects of superfetching and it has given me some heartache on past benchmarks due to caching.
I've done some readyboost tests but pretty much found that it seems great in theory, but not in reality. Plus, the more readyboost, the more boot thrashing Vista has. I've also tested that on memory slower than my HD so basically you should see a faster start response...but again, that's just theory and I could never provide any bench results. The only thing I could get numbers on was battery life improvement.
The network IO initially (vs XP) sucked. It's made improvements and tweaking has helped it, but it still is behind XP, and both are far behind OSX, but OSX is designed for a specific hardware, unlike Windows. I end up syncing external hard drives every night through Vista, the transfer used to be 300kb/s either direction. With a few patches and SP1, it's improved to 10mb/s one direction (on a gigabit) and 40mb/s the other (on firewire). I'd be much better off with a server OS.

 

Thanks for the info. One of the reasons I asked is that I just came across Mark Russinovich's 3-part article in one of the MS in-house magazines discussing the changes in the _Vista kernel and, at least on paper, they sounded very impressive; however, as you say, theory is one thing, practice another. In particular, I was interested in his discussion of the new algorithm for prioritizing processes, and of being able to super-prioritize certain processes and permit them to reserve bandwidth, in no small part because he referred to registry keys where some of the settings are kept, and which (presumably) could be modified. Have you tried playing around with changing any of the prioritization defaults to see what effect that had on performance?

 

I actually have and it has helped a lot. That algorithm itself is what has slowing things down. The problem is that it keeps trying to update and that slows things down in general. With SP1, I haven't had to do any tuning to get that better performance, but I think it could use just a little more work and we'll probably see that in either SP2 or Windows 7.

 

I believe that you are mistaken.

The transition from 2000 to XP should not have been very gut wrenching since 2000 and XP are based on almost the same code base, have almost the same driver model and if you choose the Classic theme in XP your UI would basically be identical to 2000. It was first with SP2 that XP introduced changes that definitely separated it from 2000.

However, when going from XP/2000 to Vista you have substantial differences: different code bases, different driver models and different UI:experience (even if you choose Classic theme in Vista)

 

hmmm Impressive...

 

So contrary to a different thread on here, installing XP on a M1330 (250GB, 4GB Ram 2.2Ghz) will actually make it slower not faster?

There is a huge thread about how to install XP on an M1330 and I was all geared up to do it, before I found this thread. I'm not a gamer or a techie, I just had heard bad things about Vista and thought it would be more trouble than it was worth e.g. it doesn't work with ZoneAlarm Pro

http://forum.notebookreview.com/showthread.php?t=153941

 

Thanks so much to trebuin for his excellent post last year! I am familiar with the Passmark tests and used their data to select my laptop, but that was real work wiping, installing, and obtaining drivers - way to go!

I purchased a Dell Vostro 1510 laptop with Core2Duo T5870s and 3 GB RAM. The key here is that the processors are 64-bit. This laptop is necessary for my business: MS Office, C programming, and perl/Rexx scripting. For this use the Vostros are hard to beat!

I also got a Vista Business downgrade; XP was preinstalled and all is fine. BUT I noted they sent me a Vista 32-bit version so when the time comes to upgrade to Vista, I will have an OS that, performance-wise, is 1.8% better than XP! Believe it or not, Dell tells me I need to return my Vostro (which I've been configuring for 3 days now) to get 64-bit Vista - it's not a matter of just sending me a new disk!

However, I note that even Vista 64 XP1 is only 15% better than XP, and from the individual summaries, the processor and memory tests are faster; an advantage for XP is its 35% superiority in disk transfers, which is where the most time gets wasted.

Since trebuin was able to readily find Vista 64 drivers, should I just insist that Dell send me the Vista 64-bit CD and I should handle the driver situation, or given the modest performance differences, should I just let it go even though there was a cost for the downgrade? Thanks for reading this and any feedback you have!

 

Nice job buddy!

I was an XP hold out for a long time but since the arrival of Vista sp1 I changed my mind.

I also have Vista 64bit and will never use a 32bit OS again.

 

Good question. I hope this isn't vanilla XP with no SP. That's not very representative.

 

First: Yes, Filecopying sucked before sp1 (and thus, networking).

But: I have 100MB/s read / write on my network all the time. gigabit between vista on a quadcore, and an atom with winhomeserver.

so it may be that your drivers are messed up somehow? or something is NOT gigabit net somewhere.



and thanks for the big test, it's nice to have a reference post now..