*OT* TeraByte Me *OT*

Every few years, I make it a habit to see just how much it would cost, using off the shelf components, to build a storage array THAT I KNOW WILL WORK in the next "Quantum Amount" of storage.

Over the years I've done an analysis on GigaByte quantity, then TeraByte quantity... and then watched as technology whittled those costs away to that of a single hard drive. Of course now the next level is PetaByte (For those of us NOT working for the US Government or WalMart), so I got my calculator out and started working it up.

Using the direct connected storage/localized server model, my best numbers were in the $184,000 price range using an HP Blade Center and 8 HP MDS600 70-slot SAS rackmount boxes full of the cheapest 2TB drives I could find. Not really as bad as I figured; these configurations are USUALLY the most expensive, and I DID go name-brand all the way. They will also usually produce the lowest latency connectivity for local users.

My next attempt was much easier to calculate; off-the-shelf generic 12-slot servers and LOTS OF them (46 total); that configuration of course sucks up LOADS of space (AND JUICE), but the benefits are that they can be easily managed over the network, and you can easily break them up into subclusters to better serve user concentration points, and it's not nearly so painful to "keep a few spare" servers ready to deploy at a moment's notice. The weak point is... there are SO MANY possible points of failure. Price point? $245,000. By shopping around a little more, or by building my own servers from scratch using larger-capacity boxes, I could probably shave another 15-25% off that; NOW we're getting into the same price range as the HP Solution. UGGHHH.

I'm NOT impressed; I'm used to being able to undercut the big guys by at least 20% by DIY efforts. In the course of my investigations I came across this article:

Petabytes on a budget: How to build cheap cloud storage | Backblaze Blog

This article is a year old; thanks to cheap & plentiful 2TB drives, their cost per PB should be CONSIDERABLY LESS today.

The single thing that most impresses me is how thorough they are in their outline. They give you a detailed list of EVERYTHING YOU NEED to build your own exactly the same way they do; they even give the CADD file and tell you where to go to get the enclosures made. :GEEK:

Definitely a thought-provoking read. :smile:

mnem
Food for thought. Thought from food.

 

Hey mnem! I saw this on another forum, an individual there was using it as his profile statement and I thought you would appreciate the point....

"Meddle not in the affairs of dragons, for thou art crunchy and good with ketchup!"

 

Hmmm; now where have I read that before???

"Do not meddle in the affairs of dragons, for thou art crunchy and go well with ketchup."

 

I knew I had seen that somewhere previously....

 

LOL... ummm... guys... it used to be my sig before I decided to update it with all my ToughBook stuff.

mnem
"Meddle not in the affairs of dragons, for thou art crunchy and taste good with ketchup!"

 

At my old company Ameagle, we had EMC2 SANS and we had 2 x 42 U racks of them at each datacenter (One rack for drives, and one rack for the server, monitor, KVM's). Eacy one had 25 x 80GB in each row. Each row took up 2U's so you could get 525 x 80GB, 5400RPM crappy Maxtor SAS hard drives per rack which was just about 42TB x 2 datacenters connected by dual/redunant multi-mode fiber (10G) = 84TB. Now the company spent about $250,000 on these per set up about 5 years ago plus the time involved with setting up the LUNS.

Now, 84TB is no where NEAR close to a petabyte...

The dwagon is pretty good at $184,000...

Mnmenth, what are the specs on the drives and sizes, yada yada yada...?

Fun stuff, fun stuff

 

Heh... EMC2 SANs like the ones that took down the whole state of Virginia for a week just now? :wink:

I just priced out the cheapest, low-tech stuff off the shelf from Newegg. Remember, HDD size has grown tremendously over the last 5 years. My numbers are pretty average, possibly on the high side compared to what a provider with access to wholesale pricing could do; what I find really impressive is that the guys as BackBlaze are able to do it for 118,000... and that was a year ago. They can probably get under $100,000 with today's cheaply available 2TB drives.

mnem
Tanked.

 

If their enclosures will support media conversion from SATA to SAS like the MDS600, they COULD get right close to a PB with their existing hardware. I dunno what the comparative latency would be; I was gearing towards long-term storage/archival use.

Newegg has 20-packs of 2TB WD Caviar 3GB/s drives for $2216; that comes up to $58170 to fill 525 slots.

Allowing 15% overhead for RAID, 1050 x .85 = .8925 PB total capacity.

If you can cut that overhead to 12% like the guys at BackBlaze, you get within a few points of a PB: 1050 x .87 = .9135 PB. Not too shabby...

mnem
Archetypal.

 

How much pron have you downloaded?????
CAP

 

How did I know that this was gonna come up?

yeah, cuz 525 x 2000GB = 1050000GB = 1.05PB of RAW, unformatted storage.

 

ALMOST all of it... :wink:

mnem <~~~ HornDwagon~~~<<<

 

My company also had a EMC^2 (had to get the ^ in cuz it's really squared ) SAN failure a week ago and brought down a few major websites along with it!