3DMark08 'Vantage' Review

:wink: 3DMark08 'Vantage' Review :wink:
from Extremetech.com



We have recently talked about the new 3DMark and its proprietary use of Vista and DX10 cards, but I would like us to step back and take a look at this bleeding edge benchmark that brings the most expensive systems to their knees. And we should see what the future of 3D gaming can be.

Download:
www.futuremark.com
www.majorgeeks.com/download5948.html

Here are some things to look for:

Introduction:

Benchmarking 3D graphics cards and CPUs for gaming is tricky business. Of course testing real games is the preferable method, but that can only tell you how well a certain piece of hardware runs the stuff that's out there today. You want to buy a graphics card that will be good at future games as well, right? Then there's the issue of how you test—pre-recorded timedemos and built-in benchmark modes have the advantage of perfect repeatability and reliability.

Measuring frame rate over time during actual gameplay is a more accurate representation of the performance you can expect while truly playing games, but it carries with it the fallibility of each test not being exactly like the one before—variance in performance may be the result of an imperfect tester, rather than true differences in the hardware you're testing.

3DMark has for years now filled a wholly different role than "real game testing." It's a synthetic benchmark, not a game, but it's made with some very game-like code. The scenes and graphics techniques represent the same kind of stuff games use. What's more, by not being a game, 3DMark can afford to be very future-looking.

The tests and graphics techniques in 3DMark are meant to represent what we'll see in games over the next year or two, not what's on the market right now. In this, it takes its biggest drawback (that it's not a real game and therefore does not represent real game performance) and turns it into its biggest strength (by using forward-looking graphics techniques it can give some insight into how graphics cards may perform in future games).

3DMark06 has gotten a bit long in the tooth, and no longer reflects this forward-looking approach. Even inexpensive graphics cards run it incredibly well, and many graphics techniques in modern games are beyond what are used in the benchmark. And so, after perhaps too long a wait, Futuremark has finally released their follow-up, 3DMark Vantage. Let's take a closer look at the new tests and scoring methods, and find out what the new version of the industry-standard benchmark means for us.


Major Changes

The first thing you'll notice about the new 3DMark is that it's DirectX 10 only, which means it's Windows Vista only. The other system requirements are pretty steep, but then again, this is supposed to be a forward-looking benchmark. The required CPU is a dual-core, equivalent to an Intel Core 2 Duo E6600 or Athlon 64 X2 6000. It will run with any DirectX 10 graphics card with at least 128MB of RAM, and there's no minimum system RAM requirement beyond that for Vista, but in our experience you'll want 2GB of RAM to avoid hard drive activity during the tests.

Futuremark has also changed the pricing structure. Now the free version only allows you a single benchmark run. If you want to run the benchmark more than once, you need to shell out $6.95 for the Basic Edition, which allows you to run a single "performance" preset (more on that later) as much as you want, but requires a network connection in order to view your results. If you want more, you can pay $19.95 for the Advanced Edition, which lets you run all four presets or any custom settings you desire, though you still need a network connection to view your results. For commercial use, there's the $495 Professional Edition that includes technical support, automated command-line scripting for batch testing, and the ability to view your results without going online.

There are other changes, too. Where there used to be a single default 3DMark setting (1280x1024 with no anti-aliasing or anisotropic filtering), there are now four "presets." These represent four different performance levels of graphics cards, and represent changes in resolution, AA/AF settings, texture resolution, shadow quality, shader complexity, and post-processing effects. The four presets are presented in the table below.

	Entry	Performance	High	Extreme
Resolution	1024x768	1280x1024	1680x1050	1920x1200
Multisample count (AA)	1	1	2	4
Texture filtering	Trilinear	Trilinear	Anisotropic	Anisotropic
Max. Anisotropy	N/A	N/A	8	16
Texture quality	Entry	Performance	High	Extreme
Shadow shader quality	Entry	Performance	High	Extreme
Shadow resolution quality	Entry	Performance	High	Extreme
POM shader quality	Entry	Performance	High	Extreme
Volumetric rendering quality	Entry	Performance	High	Extreme
Post-processing scale	1:5	1:2	1:2	1:5
Disabled post-processing effects	Motion Blur, Depth of Field	None	None	None

Test Setup and Procedure:

Running 3DMark Vantage couldn't be easier. You just fire up the program, pick the tests and preset you want to run (or custom settings, if you have the Advanced or Pro Edition), and go. For our first look at the program, we'll run benchmarks in the following machine:

Component	Make/Model
Processor	Intel Core2 Extreme X9650 (3 GHz)
Motherboard and chipset	Intel X38 Motherboard
Memory	2 x 1GB DDR3 1333MHz
Hard drive	Seagate 7200.10 160GB SATA Drive
Optical drive	ATAPI DVD-ROM Drive
Audio	Integrated HD Audio
Operating system	Windows Vista Ultimate with SP1

We're going to test a handful of recent and popular cards from Nvidia and ATI.
* ATI—3870 X2, 3870, 3850
* Nvidia—9800 GTX, 8800 GT, 9600 GT

We wanted to include the newer 9800 GX2 dual GPU card from Nvidia, but weren't able to because of power supply issues. We'll follow up in the future with more on the 9800 GX2.

Naturally, both companies released last-minute drivers to improve 3DMark Vantage performance. Nvidia's Forceware 175.12 is the driver of choice, and ATI released a hotfix with driver revision 8.471. The ATI driver, in particular, came with a few warnings about some potential gotchas—you have to reboot when enabling CrossFire, quad CrossFire X performance isn't what it should be, and so on.

Naturally, the release of a new 3DMark is just the beginning of a driver and product battle between graphics companies that won't be settled anytime soon. In the short term, we can expect each new driver to improve 3DMark performance in some way. In the long term, new products will tout new 3DMark records, and driver releases for the first few months after a new product's release will offer 3DMark gains. That's just the way of it. Fortunately, improving 3DMark performance will often lead to performance gains in other DirectX 10 games.

GPU Test 1 - Jane Nash:

<!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //--> <!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //--> <!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->

Let's take a look at how a set of modern GPUs performs in this test.

<!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->


GPU Test 2 - New Calico

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //--> <!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //--> <!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->

How do our GPUs stack up in this second scene, performance-wise?

<!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->

CPU Test 1 - AI

<!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->

Performance is measured in Operations Per Second. This is equal to the number of 3D paths calculated for the airplanes during the test, divided by the total test time.

<!-- start ziffimage //-->

/images/spacer.gif

<!-- end ziffimage //-->

CPU Test 2 - Physics

<!-- start ziffimage //-->

/images/spacer.gif

click on image for full view (right-click > open new tab/window)

<!-- end ziffimage //-->

We're not sure where this one is going, now that AGEIA has been acquired by Nvidia. Nvidia has announced that some form of PhysX support will be added to their graphics cards. This may give them a distinct advantage in this test.

The result is measured in Operations Per Second, where one physics "step" of one "world" (pair of gates) is considered one operation. Remember that the CPU tests represent only a part of the overall 3DMark score—at most 25% on the Entry setting, and only 5% on the Extreme setting.

<!-- start ziffimage //-->

/images/spacer.gif

<!-- end ziffimage //-->

________________

What It All Means....

Now that we have this mountain of information at our disposal, what does it all mean? Frankly, right now, not as much as you would think. The problem with a new release of 3DMark is that it's a very difficult and forward-looking test application that has a ton of importance placed upon it by the industry. In 4–6 weeks, if you ran all these tests again, you could almost guarantee different performance thanks to inevitable driver optimization.

Still, we can take away some important info. The PhysX cards, when utilized heavily as they are in the second CPU test, can outperform even a high-end CPU, but not as much as you would think. It's not a factor of ten, or even a factor of two. With the limited support and sales of PhysX cards so far, and future CPUs like Nehalem surely offering better performance at these tasks, it's hard to see this benchmark making the case for that hardware. However, the addition of a PhysX card can improve the Entry preset 3DMark score by 5% or so (its influence tapers off on higher settings), so make sure you know if the card is present when comparing numbers. We plan to test without it, unless specifically noted.

It's also very interesting to look at the specific feature tests, where the cloth physics and particle rendering tests are dominated heavily by Nvidia cards. Is this a difference in hardware capabilities, inherent to the GPU architectural designs? Or will we see ATI catch up with driver updates?

3DMark Vantage's GPU tests makes use of a fair amount of GPU-generated data, which is a difficult situation to deal with when developing CrossFire and SLI drivers. Watching 3DMark numbers for multi-GPU scaling may be a good indication of how we can expect many DX10 titles to scale on dual-GPU cards and multi-GPU setups.

Finally, we see that even some pretty dramatic driver optimizations won't allow current hardware to run the new tests at what would be acceptable frame rates in a real game, unless you choose the Entry setting where everything is greatly simplified. This is actually good news—it creates a work load for graphics cards that will truly show what new hardware is capable of, even when many modern games may not. It cements 3DMark Vantage as a very forward-looking test.

It goes without saying that 3DMark Vantage is not a real game, and shouldn't be treated as such. We care primarily about graphics performance in games, not synthetic benchmarks, and 3DMark scores should never be used to replace real game benchmarks. However, the very forward looking nature of the tests, and its propensity for using graphics features that hammer GPUs in ways current games don't, make it a very useful additional tool in evaluating GPU performance. At some point in the future its relevance will be diminished, but now, in the early days when increasing 3DMark performance often means increasing real game performance, it's a good benchmark.

There's always a certain amount of controversy about a new 3DMark, and we don't expect less this time around. Why no DX 10.1 support, for instance? (Futuremark says 10.1 was available too late in the development of 3DMark Vantage to incorporate.) Why support the PhysX cards when so few games do, and don't look likely to in the future? (Frankly, it doesn't make that big a difference.) We expect ATI and Nvidia each to, at some point, call the other out for driver shenanigans when they don't like the other's 3DMark scores. This stuff goes with the territory, and our job is to keep an eye on the situations as they develop and deliver to our readers as much clear, accurate information as possible.


Read more details at the review site:
http://www.extremetech.com/article2/0,1697,2289638,00.asp


________________


Game On People,
-Gophn

P.S. Jane Nash kinda looks hot... dont you think?

 

Very nice review.

 

Great Review Gophn!!!!

 

Nice review. Will be usefull.

 

yeah,great job!

 

great work Gophn! i might pay for it just because of the girl in test 1 ;P

 

haha I was thinking the same thing!

great review btw!

 

Nice review!

 

I doubt itll be as popular as o6 though, just due to the fact that you have to pay even for basic

 

You can run it once for free, anyone use it to benchmark their machine yet, I tried to download it but the site was super slow.

 

there are many mirrors, I recommend:
http://www.majorgeeks.com/download5948.html

 

Cool thanks I'll try it tomorrow!

 

Classy review, nicely done!

 

Um, Gophn didn't write this review, extremetech did! He even linked to it in the very first line!

 

Correction.... 1st and last line.

I just re-did some HTML to bring it into the forums for the lazy folks.... like me

 

Haha I thought you were just saying that extremetech also did a review that was worth taking a look, not that this review is from extremetech.

 

lol, i thought i made it obvious... with two links to the same place.

 

I agree. People who just want to compare their own pc for fun wont buy it...

 

I already know someone that bought the crazy Professional version for $495 !!!

He is freakin crazy, but he is already a crazy early adopter that changes computers every 6-9 months... and he likes to have the latest in hardware and software. He likes to OC, so he does not want a watered-down free or basic edition.

I will probably try to borrow his license when I get around to benching my new systems.

 

tried it, here is my score....
http://service.futuremark.com/home....8D46C761391EC245?resultId=67752&resultType=19

 

Ohh, 1/2 the score 0_o

 

im shocked to see the ATI 3870X2 too be nearly doubling the 9800's FPS

:O:O:O:O:O:O

when did the 3870 get better than the 9800 haha

i know its obviously not, and the reason its running better is something complicated, but still, nvidia will not be happy when the 3870 starts showing higher than their monster card on 3d marks site.

 

I stopped reading at "...Vista only"

 

Interesting that you say that..

I am working on bring it to XP right now.

I will post results when I get it going.

 

hows this score? had to take the q6700 down to 3.9ghz becase my room is boiling hot
http://service.futuremark.com/home....8675378BA750A8B?resultId=140507&resultType=19

 

Anything with such rig is great XD / also look at the fastest system, this is totally crap! CPU is really boosting the score , this is supposed to be related to games right? not to compressing

 

yea the fastest system's cpu demolishes mine, u need nitrogen cooling to hit over 5ghz on the quad

 

yeah i see, beware vantage can get kind of sluggish if you have a physx GPU, you'll get a GREAT leap in score o_0

 

Man Gophn, I KNEW that review looked familiar. You need to slam a huge EXTREME TECH ARTICLE on the top...because I missed it and thought you were several levels + awesome for a few minutes too many.

Freaking rep hoe. (said with all the love in the world).

 

Hmm, it's strage I encounter this problem about 10 seconds into the first CPU test.

Anyone know what it could be? I have Vista SP 1 Home Premium.

 

Maybe update your DirX?

 

Do you have patch installed:
http://www.futuremark.com/download/3dmva_hotfix_101/

 

i agree gophn, she does look hot. Only problem is I can't view the image larger =/

 

You need to connect your laptop to a monitor that can display higher resolutions. The tests cannot finish because your monitor cannot properly display the higher resolution image. Hook up your system to an external monitor capable of at least 1280x1024.

OH, and make sure you are set up with a single monitor displayed at a time. No clone or extended monitor setup.

 

I would have liked to have seen 2x8800GT's compared to the 3870X2. That would have been a more even result. IMO.

Here is a list (from Oc'net) from best to worst.
9800GX2
HD3870X2
8800Ultra
9800GTX
8800GTX
8800GTS 512MB (G92 Revision)
8800GT 512MB
HD3870 512MB

As you can see from it, the HD3870X2 is right below the 9800GX2, and way above a 9800GTX.

 

Ah thanks a lot - I've patched it to 1.01 and now I get that error on start-up. Looks like I won't be able to run this for a while then because my laptop only has an HDMI port and I don't have a converter to VGA (what my desktop monitor is).

Is it not strange that his happens? I thought you could change the resolution under options and make it run in that one.

Thanks fo the help anyway and great review!

 

hehe well done hd3870x2 =) very impressive benchmarks , nice job!