Is notebookcheck extremely Nvidia biased?

Mobile Grafikkarten - Benchmarkliste - Notebookcheck.com Technik/FAQ

Apparently a GTX 460 (Nr. 20 on the list) is more than 2 times better than a 5850 (Nr. 42 on the list), and somehow the 5870 (non crossfire) which is just an overclocked 5850 is about 30 times higher on that list..

List makes no sense

edit
ok so I just looked at the benchmarks for the 5850 and out of all the laptops tested only one had the GDDR5 version, the rest are exclusively DDR3. So much for fair ranking

 

rank #20 > rank #42 =/= "2 times better". That's like saying a gold medal athlete is twice the athlete of a silver medalist.

And a 5870 is better than a 460, which is better than a 5850.

 

I have a feeling that is the GDDR3 version of the 5850, not the GDDR5 version found in the MSI gx640, which performs quite a bit better because it has a lot more memory bandwidth.

 

I don't really like the notebookcheck ratings... keep in mind that for example a 5850 from Dell will perform differently than a 5850 from clevo... it'S not that easy to just make a ranking and this is definitely not good enough imo.

 

a 5870 is only slightly better than a 5850.. not enough to drop the 5850 to Nr. 42 on the list, their benchmarks must be hilariously inaccurate

 

They're roughly accurate, they just don't draw a distinction between the GDDR3 and the GDDR5 5850, which is why the scores are so much lower. You can argue that as stupid, but their scores are mostly sane from my experience.

 

They've never drawn distinction between memory types, even back with the DDR2/GDDR3 generation cards (Radeon HD 3650, GeForce 9600M GT for example)

 

We want them to differentiate between ddr2, ddr3 and ddr5 lol.

We also want them to fix that quadro fx 3600m BS, its not at all the same as the 9800m Gt, its a 8800m GTS

...and the fx 1600m, how can it be the same as the 1700m? no way. Its the same as the 8700m gt.

 

It's not even close to that -- more like 80% or so. Enough to get a decent impression, but I would definitely look elsewhere for confirmation if you're researching a laptop to buy rather than just discussing things on the internet.

 

Yeah they should be doing that, even if it means having multiple entries of the same GPU in the table.

 

Hahaha, no wait HAHAHAHA. Please be sarcasm.

Listen. All you should use Notebookcheck for, is the basic specs of the GPUs, and maybe there Vantage GPU scores.

There rankings were genuinely good, from around the time of the 7950 GTX -> 8700M GT -> 8800M GTS, but after that they just started throwing GPUs up there with no regard for it being a ranked listing. It's a total joke now.

 

...and even then its still not always great, like I was saying with some of the quadro cards.
TDP should be a basic spec, and yet they are really bad at it... all the quadro cards are listed as about 5-10 more watts then their geforce counterparts which are exactly the same.

I know most people don't care about quadro cards, but these are just examples.

Yea, I know its been established that your statement entirely inaccurate already. but what I want to know is how do you get off saying that as if any single person could possibly know that? seriously, do you have over 100 mobile GPUs that you have compared on the same system?

 

it's not perfect, but it's not nvidia biased.

If I get 20th place in a marathon 10,000 person marathon, and you get 42nd place, am I twice as good a runner as you?

Common sense. You aren't interpreting the ranking system correctly.

 

i talked to one of their reps and he told me that only they test systems on stock, and whatever units they receive, i prefer TH more.

 

notebookcheck is a very credible source. I've owned a few mobile cards and the most recent is the GTX 460m, and again they're right on the money especially with the benchmarks. I get the same framerates in game and same ratings in 3dmark6, they don't make up numbers. I trust them and they're more professional and technical then other sites.

 

Once again, You've owned a few mobile cards have you?
...therefore you're willing to say how credible benchmarks are for over 100?

Nope, I don't think so....

 

Nobody in this thread said notebookcheck make up numbers.

 

I wish they'd fix their listing of GeForce 9600M GT to VP2.

 

I think they make up TDPs and quadro card specs based on what they've heard is the equivalent card and stuff.

I was looking at their chart, what the heck is up with the mr3870?
The 3dmark06 and vantage scores are accurate so how on earth did they decide to put it in 80th place?? No, a 4670 does not beat a 3870, lol....

 

It's good for the cards specs but yea their GPU ranking has severe flaws. For the 5850 there's no way to tell apart the 5850 with GDDR5 and the one with DDR3 therefore it gets a crappy ranking even if the first version is a very good performer.

 

ok I definitely worded that wrong.

I just looked at their 5850 benchmarks and yea, if you click on the framerate for each game you can see which system it belongs to and all the laptops in the test are Acer Aspires with the DDR3 version. Only laptop tested with GDDR5 is the MSI GX640, so the ranking is not even barely fair

ATI Mobility Radeon HD 5850 - Notebookcheck.com Technik/FAQ

 

For all cards where the vendors can select the memory type, notebookcheck handles the situation equitably. Therefore, it is fair, and further, it is not biassed. The end.

However, you have noticed that their measurement system is problematic. Note that the issues with their performance metrics are not related to bias or unfairness, but poor measurement (overly reliant on synthetic benchmarks) and poor data analysis (considering the 5850 a single entry with a wide performance range instead of separating the versions)

 

One could argue that due to the most common version of the 5850 is the DDR3 version, the performance noted is correct.

However that is to clear someone from doing a proper job.

The differences between those models should have been noted and made more clear in the article. Notebookcheck tries to cater a large audience with the promise of a consistent cpu and gpu list. However the lack of differentiation and information on those lists leads to it being a not good reference, due to the laptops that they congregate in those results being the single most important source of information, they dont rely on several respectable sites and they use whatever is available in the web for the purpose of input, one of the things is that it leads to some inexplicable deviations in the results.

Another thing is that we dont know how well do they perform those tests, we lack the metrics:

what are the drivers?
what is the power mode in windows?
was the windows a clean install or a OEM install?

The heavy use of synthetic benches are another problem, due to them not being that reliable, and tend to favor one brand over the other. Aside that the synthetics only serve for one thing:

WOOT I HAS A GOOD COMPUTER! (said the cats at the icanhascheeseburguer)

The proper benchmarks are real use, enconding decoding, and so forth. Incredibly or not I dont use my pc to calculate the Pi, although the idea was very tempting sometimes, I was too afraid for it to end in 42 and thus leading it to be a perfect number and inevitably the meaning of life.

So notebookcheck, biased? Yes.

For which brand? None

For who them? themselves.

 

While the ranking is wrong due to the memory variables, it simply shows the imperfections of their ranking system. Simply separating the DDR3 and DDR5 cards would help.

Both this site and laptopcheck are about finding real differences and by how much and demystifying laptops and their components. The laptopcheck site lists their benchmarks so YOU can be the judge.

OP, If we ranked 100 cards all within 1% of each other, is the card in 42nd place twice as good as the card in 84th place due to its ranking?

No, they are all good (or bad lol) and within 1%, its just one card performed slightly better than the others.

I think the OP should now look at himself and ask why some site's numerical ranking matters THAT MUCH to him.
I think fanboi is the right word.

 

I already admitted to wording that badly, it is not bias. The right thing to do would be to have multiple entries for each card if it is available in different versions.

It matters because a laptop is an investment and the benchmark lists are supposed to help in choosing the right laptop, and something so minor could well lead someone to paying $500 more for falsely believing a certain card is not so good due to notebookchecks indifference to DDR3 and DRR5. No fanboism there.

 

we all agree that notebookcheck's analysis is flawed.

 

I guess I don't like to admit it all the time but the reason I use notebookchecks GPU list is mainly to see what I can market, what is sellable, what could work in some systems with vbios flash.

in light of that I would really love if they showed the actual core the GPU is based off as one of the main things. Its not even one of their specifications, its just in the description (e. g. 9800m GT based off g92 same as 8800m GTX etc)

I've suggested that NBR makes something better, in a thread in site suggestions but it wasn't related.
Does anyone else think it would be a good idea? Should I post a thread on it?

We could do it as a forum really easily of course but I'm talking about making something official and part of the main site.

 

It'd be a nice feature, but not sure whether NBR sees enough laptops to make it work.

It'd be great if one day there was a proper GPU performance chart, and also a list of MXM notebooks, and what GPUs are supported or can be made to work in what chassis, but I've never seen one that shows any strength of completion. NBCs is the closest we've got in terms of a GPU list, and MXM-upgrade probably has/had the best list of chassis and compatibility.
MXM is basically to help ODMs integrate different GPUs, rather to help users upgrade, so I suspect this may be deliberate, to ensure people keep buying!

 

I think I could make that list and do it pretty well for MXM 2.1 systems as long as they aren't too old, and by that I mean pentium dual core or single core.

Yea, MXM upgrade does not include some models that have BIOS whitelisting because technically its not fully completing the standard, even tho they include other models that have similar BIOS issues. I mean, no laptop supports every MXM card of its type.
Specifically they really need to include a lot of HP elitebooks and the HDX dragon, Lenovo 17" and a few ideapads, toshiba, and get a lot more info on Asus.

One of the more complete lists I've seen is specific to Acer but the Acer MXM thread, it really has a lot of good info and is well organized.

But MXM is really a bit off the subject, some cards simply do not exist in MXM even tho they are pretty high performance E. G. 9700m GTS.

 

I dont know much about notebookcheck. I have really heard its name only over here as a site where reviews are done.

Seriously though, if you have been buying PC hardware for a while, the amount of places to get accurate figures off is surprisingly low. One very reliable source is Anandtech. They almost always never let people down and have fair reviews. Another is HardOCP. Not as detailed as Anandtech but still good enough to get some views off. The only other place is user reviews at xtremesystems and notebookreview. And that is IT.

If you are looking at reviews from other sources (eg toms fart ware), you have no one to blame but yourself. I am willing to bet that they are shills of nvidia like countless other sites. Why do you think nVidia chips are so expensive compared to ATI? All that income goes to "advertising" and that stupid logo at the beginning of games. If anyone has any doubts about the price gouging practices of nvidia, dont look any further than the fated 9800 GTX. A card that dropped half in price after few weeks of release before release of the 4XXX ATI series.

 

Coz it is based on a Turion X2 machine.
[Fujitsu-Siemens Amilo Sa 3650 (AMD Turion X2 Ultra ZM-86, 13.3")]
CPU restricts the performance of MR3870.
MR3850 suffers in the same situation.
[MSI Megabook GT735 (AMD Turion X2 Ultra ZM-82, 17.0")]

Accurate while misguiding.

BTW: MR3870CF and MR3850CF get the right place I think.

 

Thats so dumb, they couldn't test one of their laptops with 2 3870 and just turn CF off?

Btw I don't know what you're talking about with that Amilo, no way does it have a 3850/3870

 

Their ratings/rankings are pretty stupid. That's why you always want to dig deeper with benchmarks of actual systems with those cards to give you a better idea.

 

I don't think notebookcheck.net is nVidia biased. It's just that most of the games they test happen to run well on the nVidia cards. Even though their tests suggests that the GTX 460M is only 5% behind a ATi 5870, my tests with both of them share a different story, especially when you crank up the settings in which the 5870M simply blows the GTX 460M away.

 

Yes they do, it is called Ati XGP




They score 6.9 for games and graphic in WEI which is higher than GT335M

 

One could spend all day picking through the inaccuracies of that list of rankings.

 

I kind of get the point of it though, it's flawed, sure, because they are really comparing *notebooks* and not *graphics cards* - but in a way if laptops shipping with the 3870 tend to get a certain type of bench, then posting that bench makes more sense to me than getting a 3870 CF machine and just running one of the cards.

Do you want to present the base case performance scenario, or the more realistic average case? I would think the average case, especially for notebooks. With desktops, the parts are very mix-and-match, so it's important to get an idea of the performance of the part, but with laptops, you are generally getting an entire package.

Just a philosophical difference, you can disagree if you want, but that's likely their thought process.

 

notebookcheck isn't nvidia biased. For quite some time, some ati mobility card in crossfire reigned supreme before the nvidia 280 SLI knocked it down.

My DDR3 9600M GT is borderline class 2 now

Time to upgrade to GTX 460M