?? questions about caches

what exactly is a cache and what is its purpose in the cpu? also what are the advantages of the double sized caches found in the new core2duo chips? thanks for the help.

 

As Derek Zoolander would say: "Cache is really really really, really, ridiculously good looking memory"...in that it's really fast, and stores program instructions and data likely to be accessed again in the near future by the CPU. So the more of this you have, the faster your performance in programs that tend to re-use a particular portion of code (nearly all programs).

 

If you have time, read this http://en.wikipedia.org/wiki/CPU_cache

Otherwise, I suppose it could take a while to explain it.

EDIT: It's kinda like RAM

 

The best answer is probably to start from the other end.

Imagine a computer with no RAM, only a harddrive.
That'd be sloooooow. Looking up data on a harddrive takes ~10 ms. If you had to do that for every instruction, your computer would only have time to execute 100 instructions per second. (Woo, a 100hz CPU)

Enter the RAM. With RAM, it takes maybe 100 nanoseconds to read data. Now you can do roughly 10 million accesses per second, so if every instruction relied on this, you'd have a 10mhz system. Much much better, but still hardly what we need to play Oblivion.

So, what's needed is an even faster place to store data.
That's what CPU caches are for. Accessing data there takes only a couple of CPU clock cycles. (Typically around 15 cycles for level 2 cache, which is what they have ~2-4MB of).
And of course, we can do even better than that. Level 1 cache takes only 2-4 cycles to access... But it's smaller, so you only have 32 or 64KB of it, typically.

The basic idea is that the faster you want your memory, the less space-efficient it is. That's why I have a 300GB harddrive, 1 GB RAM, 512KB L2 cache and 64K L1 cache.

And the reason it's worth it to have these tiny amounts of fast memory is the idea of temporal locality. If we needed a certain chunk of data a moment ago, the odds are *very* good that we're going to need it again soon. So the first time we need that data, we load it from wherever it's located (in the worst case, that's the harddrives pagefile. If you're a bit luckier, it's in RAM already), and into the CPU's cache.

Then later, when we need it again, we don't have to wait for the harddrive or RAM, we can just grab it from the lightning-fast cache. Of course, the L1 cache is tiny, so whenever we load new data, some of the old data has to be pushed out of the cache (it's been longer since we used that, so according to temporal locality, we probably won't be needing it as much as the data we just read).

That's why the L2 cache is handy. When data is pushed out of the L1 cache, we can still fit it in the L2 cache, which isn't quite as fast, but still a hell of a lot faster than RAM. And when that is full? Well, then we have to throw data out all the way to RAM, but you'd be surprised at how rarely that happens.

The level 1 cache typically has a hit rate well above 90%. That is, 90% of the time, the data you need is actually found in the tiny 32 (or 64)KB level 1 cache.

The level 2 cache is huge by comparison, so if you include that, you get a hit rate that's pretty darn close to 100%, which means we hardly ever have to actually read from the comparatively slow RAM.

Some CPU's have a level 3 cache as well (AMD's upcoming quad-cores are going to use one), but the basic idea is still the same. When data is pushed out of the level 2 cache, we can put it in the slower, but bigger level 3 cache, to avoid going all the way out to the even slower RAM.

I'm leaving out a lot of complications here to keep it relatively simple, and the performance difference isn't quite as big as I made it sound, but it's still very noticeable. On some CPU's cache literally makes up half the die area. That's how vital it is to good performance.

 

Wow awesome explanation Jalf...you sound like some professors I've had, except more concise. Rep'd!!!

 

wow. really, really great explanation. thanks a lot. i feel a lot smarter now hah.

 

This looks like a great place to ask this question, I have been hedging getting the system below. My big fear is the cache situation, the cpu on the system below is a Intel® Core™ 2 Duo processor T5500 w/2MB L2 Cache. The thing that is stopping me is the fear I should be building a system with the Intel® Core™ 2 Duo processor T7200 w/4MB L2 Cache instead, but then I would only be able to afford 1gig ram instead of 2. Is this a valid concern, any idea what % performance difference I would notice if doing big Photoshop files. Was hoping Jalf would have the answer...

My simple logic seems to thinks about 30%. (10% for cache, 20% for processer speed (1.66 vs 2.0)), but I could be way off and would love know if I am.

Product Name dv9033cl
US Product Number RG341UA#ABA
Microprocessor 1.66 GHz Intel® Centrino® Duo mobile technology featuring Intel® Core™ 2 Duo processor T5500
Microprocessor Cache 2MB L2 Cache
Memory 2048MB DDR2 System Memory (2 Dimm)
Memory Max 2048MB
Video Graphics NVIDIA GeForce Go 7600
Video Memory 256MB
Hard Drive 200GB 5400RPM Dual Hard Drive (2 X 100GB) (SATA)
Multimedia Drive LightScribe SuperMulti 8X DVD±RW with Double Layer Support
Display 17.0” WXGA+ High-Definition BrightView Widescreen Display (1440 x 900)

 

There is a minimal difference in performance between the 2MB L2 cache Core 2 Duos and the 4MB versions:
http://www.bit-tech.net/hardware/2006/07/14/intel_core_2_duo_processors/5.html
You would be hard-pressed to notice the difference.

Nice explanation Jalf.

 

Thx for the link, looks like the CPU speed will have more effect than the cache..

 

Pretty much what Chaz said.

I'm not exactly an expert on Photoshop and image manipulation stuff, but I'd expect cache to have very little effect there. (You're constantly running through a really big data set which won't fit into cache no matter what. As soon as a pixel has been moved into cache, the CPU is finished with it, and starts loading the next. So the cache will pretty much always contain old, useless data (by the time you get around to that pixel again, it's been pushed out to memory to make room for more data that's no longer being used either)

Just my guess, but it seems to match the test Chaz linked to.

 

So if all the data is in the L1 and L2 cache and never gets sent to the RAM, what is the RAM used for?

 

Well, you have at most 4MB L2 cache on current CPU's. How much space did the last game you played require? 4GB HD space? And all those files are there for a reason. All 4GB are needed at some point. True, the .exe file might be used more often than the file containing the 3d model for some obscure easter egg you're never going to find, but even so, there's a lot of data you need to access from time to time. And only a tiny fraction of the data can be fitted into cache.

The idea in a cache is the same as your desk. If you're studying, you don't want to have to go to the library every time you need to look something up. Instead, you just put the book on your desk, so you can access it quickly when you need to. But that doesn't change the fact that the library has tons and tons of books you might need at some point, and your desk doesn't have room for them all. So sometimes, you have to go to the library. Then you can return one of the less useful books, to clear some space on your desk, and then borrow the book you need now.

But no matter how big your desk is, it's never going to be big enough to hold all the books you're ever going to need.