3Dmark 2008 'Vantage' - to add or not to add

3Dmark 2008 'Vantage' - to add or not to add

This discussion is coming for hwbot... the database for benchmark results and a good community that supports enthusiasts.

I am bringing it here since I have been pondering about how NBR should deal with this benchmark.

Should it be added to the new NBR reviews, or should it not be?

Read the following arguments and tell me what you think.
____________________



With futuremark releasing 3DMark Vantage today, we finally have a successor to the popular 3Dmark 2006. It looks incredibly smooth and is sure to tax your quad core, three way SLI system, but it does have its downsides.

The free version only allows one single benchmark run, which comes down to being completely useless for an overclocker. The cheap but not free basic version allows you to run vantage unlimited with the "performance" settings. You'll need the professional version to run vantage on all(*) settings. You'll also need Vista, as this benchmark is DirectX10 only.

System Requirements:

One of the requirements a benchmark has to comply with in order to be available in the hwbot suite, is being free of cost, Futuremark has put us in a awkward position. Do we make an exception to our own rules, or do we stick by them and will 3Dmark Vantage be as unpopular as PCMark Vantage? Should we only add the "performance" version, which is affordable in cost and can be run on any pc but does not make full use of your quad core / sli system, or should we also add the high or extreme ranking?

We believe that adding Vantage Performance + Extreme, but not rewarding any points, would be the best solution for now, as we don't want to force people to buy Vista or a benchmark program.

What is your thought?
Voice your opinion in the hwbot forum poll.

(*) All settings:
* Entry - 128MB DX10 Video Card and a monitor capable of displaying 1280x1024 resolution
* Performance - 256MB DX10 Video Card and a monitor capable of displaying 1280x1024 resolution
* High - 512MB DX10 Video Card and a monitor capable of displaying 1680x1050 resolution
* Extreme - 512MB DX10 Video Card and a monitor capable of displaying 1920x1200 resolution

Screenshots:

_________________

My Thoughts:

I am a bit disappointed at Futuremark because users would have to purchase Vista... which is almost dead with the confirmation of Windows 7... while XP is still going strong and may be extended further than the June 2008 cut off date.

I agree there should be a DX10 supported benchmark, but it should still support DX9.

I am against a benchmark only allowing a single run... making it useless for testing. So would you have to uninstall it and install it clean again each time you want to run it.... RIDICULOUS.

Lastly, if Windows XP using DX10 (using the latest DirectX 10 RC2) can use this benchmark, then I would add it.

What do you think, it is worth making standard for NBR?

Game On People,
-Gophn

 

I would say "no" because the free version only allows one run... it's unfair to impose a paid standard in a forum where people donate their time freely to help the community.

 

If it were free, of course I'd give it whirl on my desktop just for fun. But I'm not spending $7 for a benchmark program that has had a past of questionable favoritism towards one vendor or another. It's one benchmark of many that people should consider. I can find lots of others that are free that will do the same thing. If nothing else, I can run 3DMark06 to tax my system for stability for the next year or two anyhow.

 

Hi.

3DMark06demo RULES

with the demo, no changes can be made, so it`s easier to compare

As long as people say if they are using XP or Vista

Regards

John.

 

No because many people still use XP with DX9, and even if it does work under the DX10 RC2, it should atleast be free to run the default more than once.

 

i think NBR should add it to test powerful/gaming/performance laptops (like Dell 1730,Sager 92..,Sager 5793,AW m15x,...).point. .

 

Right now I have no plans to add this benchmark to my reviews because no one is going to know what the results mean. I still believe 3DMark06 is a decent synthetic benchmark so I will continue to use that.

 

I think that machines being reviewed that are capeable of running it should. I also disagree with the negative thoughts on it being Vista only, its a DX10 benchmark, it should be a given that you would need a DX10 capeable OS and hardware. As for the ticket price, $7 for unlimited use isnt asking a whole lot, it'd be great if it were free, but widely as the program is used, you cant really blame them for wanting to make a little more scratch, though I personally feel they should omit the flagrent advertising if they are charging for it.

Bottom line its a current benchmark for current hardware, 3DM06 is a great DX9 benchmark, and while DX9 is still very relevent right now, it probably wont be that long before its number is up.

 

Poll is biased by the way... my answer is not in the list.

I will use Vista to benchmark since its a DX10 benchmark (support for XP+DX10 if Microsoft ships it would be even better)

but asking $7 for a baseline benchmark is not right so NO overall.

3dmark06 was useful because everyone could d/l and run it and it was the same on every machine.

The price should be for:
-custom controls
-CPU-only (baseline test should NOT run CPU-only tests)
-tweak/perfromance suggestions

All the things that were positive about 3dmark06 are gone in the new version.

 

The baseline test always has a cpu component. Your cpu has to be tested to see how it fares because for higher end desktops the cpu is the bottleneck.

 

by making their defacto standard benchmark no longer free, they basically opened up the door for someone else to swoop in with an excellent free benchmark and steal the limelight. 3dmark 08 won't gain much ground imo.

 

I do not see any reason why not to add it, it just gives you an extra option for a benchmark.. Nobody says you have to use it or it has to be the new standard.

Like adding a new mustang at the used car lot. It may not fit in but its there if somebody wants to pay for it.

I already said in the other thread about this benchmark that there is nothing wrong with 3dmark06 and I do not feel it is outdated.

 

By not making it free, Futuremark Co. are shooting themselves in the foot. Having to pay for the most basic functionallity seems ridiculous to me. A 1 time trial is really not the alternative to a "free" version when compared to 3dmark06 Demo. Therefore, I vote No, its not free and I do not want to use Vista to benchmark

 

I say yes. If a person reviewing has a liscense, why not? It's completly new and makes use of DX10. We all know that the 9800GX2 or the HD 3870 X2 will blow on 06.

PS: It's awesome. SM4.0 HDR is really beautiful. Too bad I couldn't enjor physics because it was a slide show >_>

 

The problem is the CPU-only test... this biases the test when used for comparisons by percentages that are not realistic. The score includes a CPU score which over biases for the test. (note having a better CPU already gives you a better score, the scoring simply weights it more than it should.)

note the 3dmark06 tests are actual game-like scenes which are chosen to be difficult for the hardware of the time. The only actual synthetic component is the CPU-only test and the scoring!

While I agree that for overall testing the CPU-only test is an excellent idea, the free version does not require it.

Quite frankly NO game renders with only the CPU anymore.
truthfully 3dmark06 (free version) would be better for basic system comparison, personal hardware decision-making, and comparison shopping without it. The FULL version shoudl indeed include it for those that want to test everything.

 

Well its the same as 3dmark06. There it would average the cpu tests and gpu tests. Heck I can overclock my e6600 to 3.1 ghz on my c90 and get over 4k in 3dmark06 with a minimal gpu overclock.

I'm kinda confused about what your saying.

 

3dmarks 06 for life

 

Exactly and no xp support (and dx9 support) included is a total deal breaker for me

 

Its not meant to be for DX9. Its supposed to be built form the ground up for the DX10 engine and renders alot of things not possible on the DX9 engine.

 

+1, we have yet to have a game thats built from the ground up as DX10, they start as dx9 and get features tacked on, thats why there is little to no difference visually and in terms of performance.

 

I don't know yet... but...
*sigh
It would take another 3 to 4 years to get that benchmark run decently on our computers. Also i wonder what graphics they use to render those pictures. i mean it is even probably hard for even graphic cards that just came out! Minimal requires x10 cards :/

 

I understand that the development of Vantage essentially forced Futuremark to charge 7 bucks for the 'demo' version, but it is really going to hurt them in the short run with sites like NBR who want to be able to compare scores against a massive database of other systems. I'll still buy the 20 dollar version, but that's because I'm a high roller with a credit card.

Still, in my opinion no new hardware should come with antiquated software. I can see using XP on old and underpowered systems or specific applications that only have XP support, but otherwise it is time to let go. The aforementioned systems should rarely ever be gaming machines in need of gaming benchmarks anyway! I can't change people's initial negative impressions of Vista, but I do believe people need to give it a second chance on current midrange hardware sometime soon lest the train leave without them. It is REALLY time to move on. As hardware becomes faster, and the percentage performance benefit of XP over Vista decreases into decimals of a percent, XP becomes irrelevant, no?

3DMark has always been about pushing hardware toward future software. Future software will run DX10, not DX9. Support for DX9 would've been a terrible idea and a step backwards in technology. We know that our systems can run DX9, that is what 3DMark06 is for testing! There was no reason to remake 06 into Vantage just to please people who refuse to buy Vista.

Microsoft's worst mistake wasn't Vista's performance, it was releasing the OS over a year late. Now users are cemented into XP, and the worst part will be when Windows 7 comes out and ends up being nothing more than Vista SP2, further entrenching XP users in the past.

Yep, it's going to be great when everyone is still using XP in 2020 because Windows 9 runs too slow on their Pentium 4 in DX9 games. It's going to come back to bite those guys, that's all I'm saying. If we're lucky, by that time there will be a viable alternative to DirectX anyway, and gamers wont still be chained to Microsoft anymore!

So yeah, I'd like to see it on benchmarks from here on out, but I expect sites like NBR to forego Vantage until the bigger sites take the plunge and feel out the waters. Here's hoping Anand, Extremetech, Tom's Hardware, and Maximum PC aren't still looking in the rearview.

 

Every time a new 3dmark comes out it usually has stressed even the most powerful gpus to the limit. It won't take 3-4 years. High end desktops already run them fine. I also believe alot of the problems lie within drivers for it and vantage itself perhaps. Considering there are a bunch of glitches and such.

 

Since its not free, and it is only for semi powerfull gpu's, no, do not use it. There needs to be a good opensource benchmark. Somewhere i have a cuda program that i can convert into a benchmark.

 

But then you get a benchmark measuring how fast a GPU can do math, not how fast it can render frames. Creating a benchmark for modern gaming is quite a bit harder than you'd think.

Best way to do it is to play the game and take an FPS measurement.

 

Storm what scores are you getting?

 

Hmm...well I don't think I'll be getting my disk for a few weeks, so I'll download the trial and run it to give you a quick number.

 

Throw Vantage out the window with all the other junk. It's not needed.

 

OH just checking. Didn't hear about 8600m scores or much laptop scores in general. One guy on another forum said like 350 which is weird cause i get about 850ish stock.

 

Its a benchmark, and it designed to scale overtime, upon release its inteded to bring hardware to its knees and show off what can be done with hardware and the platform. As hardware advances scores will go up. As for dx10 being required, it'd be hard to make a DX10 benchmark that didnt run on DX10 hardware....DX10 differes from previous versions as its a whole new ballgame, all the titles we have now that use DX10 only have tidbits tacked on as an afterthought, its not like the previous versions that simply added new features, and required little effort to be compatible with older versions, to fully support all the goodies you can achieve via DX10, the title has to be built from the ground up in DX10, and in doing so, you cant just disable certain bits to make it jive with DX9, you'll have to write a whole new codepath basically from the ground up for that as well, hence we have no true DX10 games yet.

 

It's always good to have more info, but whether or not those reading the review would appreciate it or know what to do with it (other than think of them as e-Wang scores) is questionable.

It's like adding SPECPerfView results, probably not worth the time spent benching it for the usual crowd you get reading the reviews, but for a small minority they'd love the added information. Of course it's more important the subtest information than the final scores which is unfortunately what is mostly published anyways.

Personally I would've prefered a voting option to remove all 3Dmark final scores from NBR reviews, and replace them with something else more descriptive. Most people nowadays consider raw 3Dmark scores as useless.