[?] SSD provisioning

Hello,

I've read for the SSD to do Garbage Collection effectively, it needs at least some 10% free space. To get that free space, do I need to just simply keep 10% of my partition free of files? Or do I need to actually make the partition 10% smaller?


Thank you very much!
-Jeremy

 

Just keep the partition free of files. Alternately you can make the partition smaller than the actual capacity of the drive, that way even if you fill it to capacity you'll still be fine. The actual SSD controller doesn't really care about partitions.

 

thank you

 

No, the partition you're using needs to be 10% (or more) smaller.

Why? Because.

With the full capacity available to the O/S - all the nand can, and will be, 'dirty' at the same time and the drive can be put into a corner (performance-wise) depending on how 'real-time' the GC and/or TRIM is performed (controlled by the firmware).

Intel and SandForce (funnily) are the only two SSD controllers that do this kind of 'real-time' GC and TRIM cleanup and will be the least likely to be put into this 'corner', but, the performance can still plummet even with these SSD's. With other SSD's which rely on idle time GC and TRIM cleanup routines - a smaller than full capacity partition will greatly enhance the ability of the drive to come out of this performance 'corner'.

What happens when we partition the drive to anything smaller than the full capacity - we are actually limiting how many nand chips are allowed to get 'dirty' - a much different scenario than simply using 90% of the capacity as I'm sure you can see now (which allows the O/S access to all the nand chips and gets them All dirty just simply doing things like booting and shutting down Windows and not even using the system for anything else more demanding - like photo/video editing, document creation or copying/moving movies and music back and forth).

I'm not sure where you got the 10% figure from, but in my experience even the Patriot Inferno with way over 28% over-provisioning (100GB advertised, 93GB usable and 128GB nand chips on board) simply bombed with heavy writing tasks.

The reason we want to do GC and TRIM 'effectively' with enough over-provisioning is that it will keep the performance of the drive as high as possible (hopefully, as high as when it was brand new/first installed).

The best way to guarantee that (in my, specific usage scenario) is to use much less than 50% of the available capacity (this is how much over-provisioning is needed for my usage to keep the drive operating at SSD speeds for me). To be more exact: with an Intel 250GB, 510 Series SSD - I'm using a 100GB formatted partition (out of 232GB available capacity).

I'm not saying you need that much, but keep in mind that this is in addition to the 9% spare area (for the 250GB 510 version - the 120GB 510 version has 12.7% spare area by default) that my write heavy workflow requires.

I hope you can now see that using an SSD at 70/80/90% filled is different than using it at 100% filled (or, as close to 100% filled as Windows will let you) but partitioned to the same 70/80/90% of the total capacity.

If you're writing to the drive a lot (a LOT) and/OR you require the performance to stay as high as possible and/OR you also want the SSD to last as long as possible (at least as long as you own that system - 2-3yrs or even more): then I would consider using (much) less than a 60% partition size - even if this means that you would need to get a larger capacity SSD so that you're able to sacrifice some additional capacity for the longetivity and the sustained performance level required.

If the above leaves you with a cross-eyed look just reading it: simply read the second paragraph or simply just the third sentence again.

Hope this helped?

 

Ah, thank you

So I guess that is related to deleted files not really being "deleted," on emptying the recycle bin? I know my Vertex 2 slowed down BIG TIME after a while of useage as my laptop's only drive (m11xr2), but for some reason, my desktops C300 is still seemingly okay... (both 120/128GB), though I have not filled my C300 anywhere near over 50% yet.

 

You're welcome.

Keep in mind though that a desktop has (usually) much more time to idle than a notebook which we turn off as soon as we're finished with it.

The greater than 50% free space is also a factor for the C300's 'feel' too.

 

Hi tiller,

can you give me an advice how can I get 10 % provisioning with my matrix RAID0 setup?
Windows sees both volumes as 2 physical disks, 2 primary partitions.
Should I decrease each partition after install/loading offline with a partitioning tool?

 

Software RAID, correct?

Yeah, since Windows shows each drive separately, you would need to 'shrink' each one 10% to reach your goal.

Pay attention that the formatted capacity remaining is identical for each drive (don't know if this will make a difference here, but with RAID, matching everything, and I mean everything is never a bad idea) I would be verifying this to the nearest MB.

If you have two partitions on each physical drive (and it seems like you do, 64GB and 174GB), then it won't matter if you decrease each partition by 5% of the total capacity of a single drive or if you decrease one partition by 10% of the total capacity of a single drive - the effect with regards to 'over-provisioning' will be identical.

Keep in mind though that the best time to do this is when the drive is brand new and no data is written to it yet - doing it after it has been used for a while might be a case of too little, too late (depending on how hard you push your storage subsystem, how effective it's SSD toolbox is and how aggressive the GC is implemented on your specific firmware).

Hope this helps?

 

again, ssd doesn't see the partition. just make sure you don't fill it >90%, no matter how (having smaller partitions, or just not filling it up that much, or what ever.. it doesn't matter if you have the space at the end of the partition, the start of it, in between, etc.. it can't see it).

i personally have the full space partitioned, and just try to not fill it to the max. if i do, i just make sure i get back to below 90% asap, and then give it time to 'recover'.

same as with hdds, actually. you don't want to fill them up to 100% and expect them to perform well. but, unlike a hdd, an ssd can clean up itself to nice performance quite well once you 'freed it from the burden'. a hdd, typically, has to be defragmented manually afterwards.

 

tiller, thank you for the answer.
I have 2 SSDs in Intel matrix RAID0, configured in 2 volumes(64 GB and 174 GB). This 2 volumes are also my 2 primary partitions(C: and D: ) which are split(striped) on the 2 SSDs.
You're right, shrinking one partition should do the job, as the data is split(striped) on both SSDs. Therefore it's sufficient to shrink only the D: partition.
It seems better to me to create and format the second partition(D: ) with only 90 % of the capacity. This avoids the risky shrinking with a partition manager or within Windows.
Until today Samsung has no SSD toolbox(Magician) that works with a RAID setup. But it's no problem for me to "Secure Erase" the SSDs with Ubuntu if there's a performance problem with the storage subsystem.

According to davepermen there's is no need to this overprovisioning as long as one doesn't fill the partitions to the max(assuming one formatted the partitions with the "quick" option!).
Is this common sense? What is your opinion about it?

 

as said, the ssd has no clue on what you do on it. so if you don't write more than 90%, that's all it sees. doesn't matter if it's because of partitions, or simply you not filling the disk. it doesn't matter what file system you use (osx based, one of the many of linux, one of windows, etc..).

obviously, not doing a quickformat is not a good idea since quite some years anyways. just remember, the ssd does not understand what you do. it just gets "put this there", "remove this there (if it has trim)", and tries to manage itself around it (and it cheats, never physically puts something where you asked it for. it just has a list to remember where it put the stuff really, and uses the rest for provisioning).


to shrink the partition, you don't need no partition manager. just go to the disk manager of windows, rightclick the partition and choose shrink.

 

davepermen,

as I mentioned in my post, I know how to shrink a volume within Windows.
Regarding quick format I'm pretty sure there are people that do a format without "quick" in the cmd.exe window or within diskpart.

 

ah ok, missread it. thought you've planned to use other partition tools.

yeah, there are always stupid people. but trying to format a 3tb disk without doing it quick, just don't... really.. takes about 3hours minimum.

there are some reasons to do a full format, but not for setting up partitions. more for cleaning-like processes.

 

I've already expressed my opinion in post 4.

Yeah, exact opposite of what davepermen states.

But, to quote myself from that post:

and

and

Please see the entire post (4) to read the above quotes in context.


Simply put:

'Hidden/unallocated' nand capacity is used exclusively by the SSD to keep itself healthy and performing at top speed.

'Formatted' nand is used primarily for (user) storage requests - and secondly for GC/TRIM internal cleanup operations.

If you use your systems like I use mine (they are rarely 'idling'), then using the 'Formatted but not filled' method can and will degrade their performance. Even if you are not filling the drive past 50% with 'static/long term' data.

At least that is my experience with SSD's, so far.

To touch on the 'risky' shrinking of partitions within Win7's Disk Management environment: I have never had an issue with it - ever. Much more than I can say about any third party partitioning tools I've tried. Of course, I make sure a current backup is available; but the ability to shrink and extend partitions inside Windows Vista/7 has been one of the great features of upgrading (because of it's reliability) to the newer Windows versions for me and my clients.

Hope this helps.

 

With TRIM, it is ok to partition the full disk so long you know you have some free space per the OS. I left part of mine as unallocated only because I am the kind of person who don't like to free up space until it get all filled up.

 

it's independent of trim unimportant how you leave the space free. it just matters to have space free.

in case of trim (and a good ssd doing it all right), filling up completely, and then freeing up again (as per your behaviour) is no problem at all. it's just slow WHILE FULL, then. true so far for samsung and intel. both get filled randomly to near100% from time to time, and then freed again.

 

Wow, here is davepermen actually admitting that SSD's get slow when full.

Good to see you turning your story around.

How to put this as succinctly as possible:

Ahh!

See:
Write amplification - Wikipedia, the free encyclopedia


While davepermen thinks his usage of SSD's covers the entire world's scenarios - it doesn't.

Users who actually push the storage subsystems in their computers will find a use for partitioning to a smaller capacity not only useful, but necessary for an SSD to function at the level they require.

Quoted from the link provided.

davepermen, could we stick to helping others instead of simply trying to sabotage the help I'm trying to offer? Thanks.

 

Without TRIM, the drive has no concept of free once a NAND is used(well it can deduce but that is hard). That is why OP was the only recommended way before TRIM, to leave some buffer to the SSD so it can juggle around.

 

@tilleroftheearth, does trim work with RAID?

 

OS can, drivers do not support. Therefore, TRIM = no support for RAID. Need awesomesause GC (which I just found out Marvell's recent controllers are BAMF at, even more so than SF2).

 

Nope.

/10

 

So if the array is filled up and then files deleted to make room, how is it communicated to the SSD firmware that those pages are no longer required?

 

This is the job of the hopefully in the SSD firmware implemented Garbage Collection.

 

IRST help seems to say RAID0 supports pass-through of the trim command or am I reading this wrong?



EDIT: idk, perhaps older drivers didn't support trim on RAID0. Maybe someone could check their own IRST help and see if that is so.

@maximinimaus, how does GC know if a specific part of a media page that is mapped to a physical sector is no longer required if it's not possible to communicate that through the OS?

 

What version of the IRST are you running?

With v10.6.2.x.x here is the 'correct' answer:

Bold mine.

 

Following is an excerpt of the Intel document

Intel® Rapid Storage
Technology 10.5 Release
Technical Product Specification (TPS) Differences Document
April 2011
Revision 1.20


page 47

3.6.3.2 RAID Mode

3.6.3.2.1 TRIM on Pass-thru Disks
The driver processes the host ATA Data Set Management (DSM) TRIM command on
any pass-thru disks that support ATA DSM / TRIM in accordance with the ATA-8
specification.

3.6.3.2.2 TRIM on RAID Volumes
The TRIM command is not supported on RAID volumes.

3.6.3.2.1 means one also can attach a SSD per SATA/eSATA to a Intel Raid controller. For this type of attachment, TRIM is enabled/supported.

 

Thanks tiller, thought that might be it. I've been running 11.0x since August.

 

Cool, I've just installed the Alpha version on my U30Jc Intel 320 Series system and am trying it for a few days.

What are you running it on and have you seen any performance improvements with v11?

 

I practically started straight off on 11.0 so couldn't tell you if there are improvements, no problems so far. I suppose if trim in RAID 0 isn't available in earlier versions that must be an improvement. Thought it a bit strange you could send ATA commands to each disk of the array and read SMART etc but not send a trim command.

Originally I was curious to try RAID with 2 cheap disks, (2x 60GB disks cost about $10 more than the 120GB disk where I am) then split them, one for the laptop and one for the desktop but shortly after I bought them I read all the horror stories about SF. I need my laptop to be as reliable as practical even if it is getting old, so decided to leave them both on the desktop with the updates. Been sweet so far, no drop outs or BSOD's.

EDIT: Forgot to mention you might need the 11.0 option ROM FW also for it all to work or maybe the 11.5 option ROM will work as well. You can now use the Anvil SSD Utility Beta9 to check OROM version.

 

TRIM doesn't work through RAID...TRIM DOES work on non-RAIDed drives on a RAID controller though, and this I think is where the confusion is...

 

madmattd,

It seems like the v11 IRST drivers might support TRIM in RAID configs too.

If it's not simply a typo on Intel's documentation team (and missing a 'not' in the quote/screenshot by Dufus).

 

Although it specifically says RAID0 and not RAID it could possibly be a typo as you suggest.

How to check? What would happen if a large file is created and then trim command was sent for those clusters without deleting the file? Is marking the blocks with trim enough to not get back the original data? i.e. for those with trim, if you delete a file does trying a file recovery program/undelete fail to properly recover the file if used straight away?

Still leaves the earlier question "how does GC know if a specific part of a media page that is mapped to a physical sector is no longer required if it's not possible to communicate that through the OS?"

 

Dufus,

We'll see if it is a typo or not when it goes WHQL.

The 'test' you propose is hard to implement because although TRIM is sent immediately to the SSD - the SSD can choose to implement that TRIM command at any time (not necessarily right away or even 'soon' for us).

As to your last question, I think you may have missed it:

See:
http://forum.notebookreview.com/8028652-post23.html


I basically agree with maximinimaus that GC should be taking care of it..

 

No I didn't miss the answer, but while it may be GC responsibility I don't know how GC would know which pages are no longer required. Guess I've got a fair bit of reading to do if I'm going to have a chance of knowing what is going on under the hood.

Thanks to everyone for the answers, appreciated.

 

Dufus, which SSDs do You have in RAID-0?

 

I've done following tests to check if TRIM works for SSDs in RAID0 with IRST 11.0.0.1015.
I filled up my 2 partitions C: and D: to 99 %, then deleted some files on C: to make room for the Anvil benchmark.
I benched after fill up to 99 %, after delete and idle for about 8 hours, after a subsequently Secure Erase.
It seems IRST 11.0.0.1015 doesn't include support for TRIM for SSDs in RAID0.
GC kicks in partially after idling some time, but doesn't restore the full write performance.
A SE is necessary to get back the full write performance.

Screenies from left to right:
normal load of C: and D:, after filled up to 99 %, after delete of the files for filling up and idling for 8 hours, after the SE

 

Great post! Maybe Intel is being optimistic in their help file and hoping to have this added by the time it goes WHQL?

Either way, thank for for this definitive answer.

 

Tomy, 2x OCZ 60GB Agility 3, firmware 2.11 which is what they came with.

maximinimaus, that was awesome. Thanks very much for doing that. I was hoping there would be a simpler way to check without having to do so many writes.

 

As I'm using a RAID0 setup, I was curious if Intel finally delivered a solution.
I think, this was the first time my Sammies were challenged.
It was a great fun to explore the facts.

 

I wonder why it seems none of these SSD manufacturers use something like a super* page. A page that say contains all zero's and isn't stored on the media but either kept in a reserve space of the internal cache or generated by the firmware.

All those media pages full of zero's could instead be freed by mapping to the super page. Want to do a full format, point all the clusters that are zeroed at the super page, no P/E cycles. Need to do a trim, fill the unused portion of the disk with zero's, no media writes needed, just the mapping changed to the super page while at the same time telling the firmware all those previous media pages that were mapped before can be returned to the pool. Or does something like this exist already?

Plus imagine how much bigger peoples e-peen will be with 0-fill benches lol.

*just made up that name, don't know what to call it really.

 

Because writing all zero's is not the same as proper GC/cleaning for each type controller and nand type.

I do like your super* page name though.

 

Interestingly, I was joking with OCZ support staffs back when I had the Vertex 2 that I can make a SSD that is faster than their beloved SF using the above tricks, if ATTO is the only thing we care

Of course, in real life, it is of little use as completely 0s or 1s sector are very rare.

 

I'm not so sure that is that rare to have complete 0's on a sector and wouldn't it be better to have all those sectors that aren't written yet pointing to the super page to avoid media write/read, assigning media to it?

What happens if sectors are read. Say you have a program that reads all sectors of the disk, including those outside the partition, wouldn't it be better for those sectors that haven't been written to to return zero from the super page rather than assigning media?

When a file is deleted its sectors can be pointed back to zero without having to assign flash media on a subsequent read.

From reading so far when reading sectors that haven't been written it seems some firmware may return indeterminate data, determinate data or determinate data using zero's. With the latter case that would seem to suggest some sort of mechanism exists for doing something similar to using a super page.

Since the RAID0 I'm using allows for ATA commands to be passed to an individual disk of the array it should be possible to send TRIM commands and I shall look into this to see if a forced TRIM can work as it doesn't work natively. It looks a little tricky as one needs to know the mapping of the OS side to the LBA's on each disk but first need to find out how to send the correct command through the SCSI device.

If I understand it right it seems the more free media you have the bigger the pool will be for GC to work with. With no TRIM that pool can be reduced substantially and the only way to try and keep it fairly efficient is to provide user provisioning by partitioning to use less of the disk. Of course some may find the reduced efficiency not to be a problem, YMMV.

 

all I can say is, theory is different from actual usage.

 

iirc, the Corsair Performance 3 drives had an extremely overagressive GC that hampered instantanious performance, though without TRIM, mantained performance well after being "dirty."