What is Memory Cache for ?

I noticed that different processors have different Memory Cache storage, and also Hard Drive Cache also. What is the purpose of Memory Cache ?

 

Not exactly recent, but it has the basics of why CPUs have cache: How a CPU Works | Hardware Secrets.

The tl;dr: so that data is immediately available to be processed rather than the CPU having to fetch it from the RAM which at a time scale relative to how fast a CPU can process data is slow, having to get things from the HDD gets to insanely slow levels.

I'm pretty sure the purpose of the cache on a hard drive is similar. Here you go for the HDD: http://www.tomshardware.com/reviews/understanding-hard-drive-performance,1557-5.html, again, old, but the basics remain the same even today.

 

There are different levels of memory heirarchy. These are generally the L1 cache, the L2 cache, the L3 cache, system memory, and finally the hard drive. Each different level down the chain holds much more data, but it is accessed much slower. Every time the CPU needs to do something, it gets the data it needs to work with from the highest level that has the data. The lower it goes, the more time it has to spend clock cycles waiting instead of doing work. Keeping the highest levels populated with relevant data before the CPU needs to call on it is a huge priority.

 

It's essentially data that is temporarily stored transparently, so that future requests for data can be served faster. With CPUs, it's meant to circumvent the access time to a degree between the CPU and main memory.

 

What type of temporary memory ? any example of what would be temporary stored for future requests ?

 

I knew there was a better article on cache somewhere on hardware secrets, but I had trouble finding it yesterday: How The Memory Cache Works | Hardware Secrets. Yes, it is nine pages long, but it should answer most of your questions about how CPU cache works better than I could.

As for what gets stored in the cache, the instructions and data to be processed next (ideally). Cache is essentially extremely low latency RAM that is on the CPU die to make the transfer of data between the execution units on the CPU cores and the cache as near instantaneous as possible. Each level of cache is slower and larger than it's predecessor, the L1 is the fastest and smallest.

Courtesy of Anandtech, CPU die for a quad core ivy bridge, the L2 and L1 caches are in the parts labeled cores and each core has it's own.

 

Since others have already explained what cache is in great detail, I'll just say this:

Modern CPUs have gotten extremely good at figuring out what to keep in the various caches to maximize performance, to the point where differences in cache size between 2 CPUs in the same family/generation only makes a tiny difference in performance at best - even synthetic benchmarks can only measure differences in the low single digit percentages. So you should never let cache size affect your buying decision.

 

On Conroe/Merom, going from 512KB - 1MB - 2MB - 4MB had a huge impact on game performance. With 4MB of L2 cache it would have around an 80% improvement compared to 512KB.

 

Sure, older chips, but if you got a quad core with 6MB L2 cache vs one with 8MB there would be negligible performance differences, except for some extremely specicialized apps that may make best use of it. that's like the 3740QM with 6MB "Smart Cache" vs the 3840QM with 8MB cache will make no difference.

 

Very interesting. I was going to chime in because I was worried as I thought this might be misunderstood with too much benifit given too cache. Consider 512KB to 4GB is 800% increase so each 100% increase in size offers 10% improvement. Others should keep in mind 2GB to 4GB is only 100%. In real world application testing in the past on the C2D 2GB vs 4GB (mobile) I have seen 2% to 3% average difference in several different application types. The most extreme was 10% and on several types 0% difference. And of important note these are L2 cache. The i7 is L3 cache so less significant I would guess. Notice Intel reduced the the L2 to 256KB and added large L3.

To OP:
My simple explanation. Using algorithms (formulas) it is determined what Instruction/data is most likely to be requested and that is left in cache and which level. You want a high likelyhood of a "hit" or else it is just a waste of time. So let's say 70%? So you are most likey to get a hit in L1, that diminishes the significance of L2/L3 but for 30% remianing L2/L3 are back in play. Well exact same principal in play. If we are not likely to get a hit it is just a waste of time so we need a high likelyhood. But for reasons I won't (can't) explain it must be larger or you will lose the ability to maintain close that 70%. With L2 to get an 60% "hit" on the final 30% the L2 is 4 times the size of the L1. We are paying a penality with size to get performance and try and reduce diminishing returns. So we get a "hit" great back to work or we don't. On to L3, I said 70% on L1 leaves 30% for L2 left then 60% of that leaves 12% we are left to hunt. Now see how huge our L3 gets just to get a high "hit" rate. Because if we are unlikely to get a hit we are just wastung time. And while the actual individual time/per/request might be very small multiply by millions (+) and it adds up. To get a 50% "hit" rate on the final 12% we know have say 4GB/6GB/8GB. The 8GB gets 50% of 12% so it leaves 6% left so on to RAM/HDD/SSD. If it was 6GB L3 and all else was equal and it is not because of algoritms. 6GB is 80% of 8GB, 80% of the 50% is 40%. 40% of 12% leaves 7.2% on to RAM/HDD/SSD. 4GB leaves 9%, on to the RAM/HDD/SSD. On the difference between the three which of the 100% is 3% at it's greatest. As said before greater size offers diminishing returns the 50% size increase does not offer an equal return. So with the lack of direct proportionality of size and the effect of algorithms the performance difference and % of "hit" "miss" would be closer.

An application/Program would need to use almost the similar amount of intructions/data as the total amount of the larger cache to truly get a perceivable performance increase. If requests were above both quantities of cache then both would suffer the "miss" penality the larger less.. The greater the the larger L3 misses the closer peformance will equalize. So as said the situation that perfectly uses the extra space above my 4GB and does not exceed 8GB would get the best results. In my made up numbers that is 3%. But because of algorithms it would be less than equal to the size disparity. The 4GB of L3 above my 4GB would be much more likely to be misses than the 4GB I have.

Important to consider: I did simply make up my numbers to express a point. I think they likely are more favorable to % of hits in L3 and as such increases the significance of the increased size. If "hit/miss" ratio in L3 is less hits size matters less.

To sum it up I would pay very little for increased L3 if performance what I thought I was buying. I personally pay way too much for little increases in performance because that is just the way I am.

I 100% agree with you. But just to make clear for other both have "Smart Cache". It is marketing. Funny someone went to college to learn "Dumb Cache" was not good marketing.

PassMark:
i73740, 6GB L3 2.7GHz 8430
i73820 8GB L3 2.7GHz 8783

Same Family/Generation that is your L3 difference.