Very layman's ignorant question about storage

I have a question pertaining to any storage, e.g. hard disk drive, flash drive, or micro SD card.

If I use a, say, 32GB micro SD card to save a short video file (say 3GB), then delete it, and save a new small video, then delete,.... keeping repeating. Does it mean each time the small file is always saved at the same location in the card, and if so does it make that part of the card prone to being worn out or damage?

Would it be better for the longevity of the SD card to save enough files to use up its storage before deleting them? Or it makes no difference?

Same issues for other storage devices...

 

makes no difference. SD cards and SSD's provision themselves normally to handle wear like that if a memory cell goes bad

 

I see. Thanks. So, I was a little bit right, in that the file does go to the same location and that location is prone to "going bad"; it's just they have a way to handle it?

 

To save large files (bigger than ~2MB) even once a day on a quality SD card and erase them and do it again tomorrow is not an issue. This is what the card was designed to do. If you do this a few dozen times a day; no guarantees.

If you were to use the card as an O/S drive, that will kill it very quickly.

The primitive controller on an SD card will try to rotate the cells being used, so no need to fill it up first (that may be worse, even).

One thing I have found though: I always format the card in the device (cameras, in my case) in which I plan to use them. Once in a while, I'll also do a Windows format for good measure (about once every 5th time I've used it). With this method, I haven't had a card wear out on me since (but I'm always buying new/bigger cards and giving the older/smaller ones away).


For HDD - you can't wear out the media.

For a flash drive, same as the SD card advice above - though I have seen a few dead USB 'keys' in my day (not from over use though, afaik).


Hope this helps.

 

Wow, thank you for the informative lesson! :hi2: I record and delete the video file about once a day (sports/cycling video).Sounds like it's ok, but I'll follow your advice to format it from time to time. Highly appreciated! (Always had this question in my mind)

 

The logical sectors seen by the OS are usually remapped by the device when overwritten so writing to the same logical sectors doesn't write to the same memory on the flash storage.

 

It's called wear leveling, and yes it is managed for least amount of wear on the device. On SD cards though, if you write say 16GB on a 32GB card, unlike an SSD that will move data so that it evens out wear to cells as best it can, I think if you kept the original 16GB in place, it would constantly write to the other 16GB over and over again as you used it and deleted files. In that case, on occasion, it is a good idea to copy the files to your hard drive, format the SD card, and copy the files back over if you do frequently writes, but keep a large amount of static files as well.

 

Good point. Or you could buy SD with static wear leveling and forget about it.

Features of High-Reliability SD Memory | SD Memory (Industrial Use) | Recording Media | Panasonic Global

 

Modern USB drives (well, at least the high quality ones) do have wear leveling. For SD however, you need something like Dufus's suggestion.

Edit: did some Googling. Looks like most SD/microSD cards sold those days have wear leveling already.

 

They have wear leveling, but not all have static wear leveling afaik.

 

Thanks to everyone supporting (with more details) what I originally posted for the OP.

I just want to clarify one point:

Having just re-read it; what I actually wanted to convey was:

Always format the card in the actual device (cameras, in my case) in which you plan to use them. Once in a while, also do a Windows (Quick, defaults) format for good measure (about once every 5th time I've used it). With this method, I haven't had a card wear out or corrupt data on me on me since (but I'm always buying new/bigger cards and giving the older/smaller ones away).

As for leaving files on the card; use it as you must. Make sure you have backups though.

And if/when you format it in Windows occasionally (at least every 10th use), copy the 'static' data to your computer.

Format it in Windows (Quick, defaults), then, insert the card in the device it will be used in and format it there again.

Now; copy the 'static' data back to the card. And, we're good to go!


I know that the OP got the answers he needed - just wanted to make sure the info was complete.

 

In that case you might want to add what type of format (low level / quick) with Windows and what to do with files the OP or someone else might want to keep that are on the card.

 

Elaborate please

 

You don't need to, really. There is no logical reason for doing so other than being anal about everything. The only time I ever do is to delete everything on the memory card.

 

By allowing the device to format the card it will use, it also sets up the expected directories (and possibly other low level parameters) for optimum (re: STABLE) performance of the storage device. I have followed this for ~15 years so far (since CF cards) and it has served me well.


Qing, you may not like my being anal - but you may still learn something too.

 

Thanks for the new replies, everyone. The elaborated explanations especially make sense to me. I will follow the advice you give here.

Despite these real issues, I wonder how practical my concern is? i.e. Has anyone actually run into damage that can be attributed to such overwriting "abuse" (maybe it becomes an issue only after huge amount, such as thousands of times overwriting)?

 

Also, I did run in a few instances where a certain device required a certain file format that was not always the default option when formatting from an OS. Very few instances and some rather more obscure lab equipment, but I would rather avoid edge cases like those.

 

The video cam I use always has the directories DCIM, Picture and Video folders. Yesterday, I formatted the micro SD card in the card reader plugged into computer, then inserted it in the cam, there were no more those directories. If I had formatted it directly when the SD card is in the camera (not card reader), does the above you said mean that the formatted SD card should have retained those directories/folders?

 

I think your concern is real. I have seen many photographers who couldn't understand what was killing their cards until I showed them a couple of these tricks. I have also seen cards being worn out when used as a torrenting (slow...) drive - they're just not meant to be used like that (small files, often and continuous).

A couple of the photographers that changed their ways saw their new/working cards last for years afterwards (some of them were using the same card between different (brand) cameras on the same shoot!).

While we're told as consumers that 'it will just work' - that is NOT how the cards and devices were designed. Even (or maybe, especially) the Pro equipment - there was always some special (proprietary) sauce added that to me, made sticking to the method outlined above a necessity to survive all these years without losing my head over lost/corrupted files too (on top of all the other pressures of the shoot).


Glad that the suggestions I've offered are appreciated.

(And tickled that the same text has somehow pi$$ed off Qing Dao).

 

Indeed, there are so many small mysteries when something goes bad without apparent reason for the average user--we almost consider it normal for some memory cards to go bad after a "reasonable time" and decide it's time to buy a new and more upgraded one, esp. technological products get cheaper and more advanced.

So for my camera that records a video daily, I guess a good idea is to keep the recorded videos in the SD card until the card is near full, at which time delete all the files, then continue the same cycle... and periodically format it. Sounds good?

 

In keeping with the 30% OP'ing I advocate on everything else (not that I'm using this for my shoots, for much smaller image files than your avg. video file): fill it to 2/3's full and then copy/save/delete the files and do the format in camera - once a month; format in the computer too.

 

Windows doesn't know it is (SD card for camera) and just formats it. The Camera requires those subfolders before processing so recreates them sometimes with the first snap. Formatting in the device will also set these folders up automatically if they are not present.

Using the Windows Quick format as described would free up the formerly static data area allowing the card to apply wear leveling better.

----

Hey Tiller, glad I found this thread - good info! Always kinda wondered and half did some of those things mentioned - better now to be a little more aware and conscientious.

 

Thanks for the informative discussions and tips (and glad it's worth discussing ).

 

The card itself has no comprehension of a file system, it's only interested in LBA translation. So all quick formatting will do is overwrite the first few logical LBA's and the file allocation tables, only when the old logical LBA's of the previous "static" data are overwritten will they become free to be recycled.

That also makes an interesting question as to how effective copying the old data, formatting and copying it back is. As the data is being written will the controller push the least worn blocks to the front as they become available. IOW a lot of the data could just be written back to the least worn flash just to sit there and not help much at all. There is of course benefit with limited data retention as the cells will be refreshed and perhaps the quick format will help with media fragmentation.


I was hoping tilleroftheearth (man that name is long, do you have a shortened version you like?) was going to address this but maybe not.

We can look at the some LBA's with data on the SD card using a disk editor.

Using the latest W8.1 lets do a quick format, this takes about 3 seconds. If we look at the same LBA we can see the data is still there. I will not bother posting a pic as it is identical to the one above.


Now lets introduce a free formatter from the SD organization called SD Formatter 4.0 downloadable here https://www.sdcard.org/downloads/formatter_4/eula_windows/



With this, if your SD supports it we can use the erase function when formating. Erase takes about 18 seconds on this 16GB microSD.


Finished.


And if we look at the previous LBA's with data.

All gone. Isn't that cool?


Here's an older 4GB SD I've had for some years.

Not supported. However, never reformatted and it's lasted all this time and never had a problem although the plastic casing has somewhat disintegrated.

If tilleroftheearth's camera uses the erase function then what he says makes perfect sense except for the Windows format. Maybe that will come with W9 or W10. Gotta give people some reason to upgrade their Windows and keep MS in Business.

 

I'm not to worried for myself since I only usually take snaps, not videos, and when I download the snaps I have Adobe Bridge delete them from the card; also I rarely fill the card before performing this.
However it would be more of a concern leaving large files (or lots of picture files) there and then adding and deleting more files into the remaining space since it is doubtful there is sufficient space to perform wear leveling and that this wear leveling is probably not as efficient as an SSD.

I'm pretty sure that camera's don't perform any low level formatting: Low level formatting as in zero write the drive (Erase). It may be redundant to do a Quick format from Windows if they are exactly the same thing, though it certainly shouldn't hurt anything. My assumption here is that devices such as Cameras take shortcuts unless the SD card is in a semi-new like state (such as formatted by Windows), at which point whatever extra is needed is performed by the device to make it compatible.

To do a Low level format in W-Vista, all that needs to be done is perform a full format, i.e., sans the QUICK specification. In W7, W8, W8.1 the CLEAN ALL command needs to be invoked first then format (Quick type recommended for NAND). Even faster and less wearing would be to use simply CLEAN omitting ALL (Wipe the partition info), then QUICK format.

 

For me a low level format is writing zero's across the storage device and that can take quite some time to do. The erase function above does not write to the SD storage. It would be similar to a secure-erase on an SSD, much quicker and properly leaves the blocks free.

Everyone has their own preference as to how they do things so hope we all can appreciate that.

 

Absolutely!

Always just trying to figure out exactly how something works, and understand the available built-in utilities.

Zero writing an SD card can take a long time. And if there is nothing confidential there - kind of unnecessary. By that I mean that if the device needed a faster way to clear blocks for new writes it would have to be implemented somewhere, and for the moment they seem to keep up with demand.

 

Dufus, (you can call me 'till' for short...),

Thanks for that post above; you really went above and beyond.

As I mentioned before, I trust (a little) the engineers who designed the CF/SD card spec's - but a little extra help is always appreciated.

I would be surprised if the cameras did a proper erase as you demonstrated. Actually; they don't (I've recovered images for clients after they've formatted the cards), not Nikon, Canon, Olympus or Sony that I know of.

To use your nifty program you dug up from the SD card association would allow cells to be cycled properly - but as you say; it may not matter much if you leave a lot of static data on it anyways.

Thanks again and Cheers!