Would this work? Storage Device, SSD, 8PB for cheap?

[Taverner]

Imagine this

We use commodity HW, and create SAN/NAS device that uses SD Chips

These can be very small, and not need a lot of cooling

Imagine instead of disks, you put in "wafers" that are cards with about 2-3TB of space, and each of these can be added to a storage unit/filesystem that could eventually hold 8PB in under 2U of rack space?

[Taverner]

1. That is a "wafer" is a sheet with several 128GB SD chips on it

Thoughts?

[TheMadMonk]

2. Yes/No. Cost remains a huge factor.

Solid state comes in at roughly 100 times the cost per GB as spinning platters.

It's cheaper to pay for the space, the juice, and the juice necessary to run the AC at 20 bellow arctic.

[Taverner]

3. Exactly, but say you didn't use SSDs but specifically cheap commodity HW?

that is, the SD mini chips that you put in your camera

RAID them all together for fault tolerance - and put them in sheets

[TheMadMonk]

4. Because if you want petabytes of storage, you need...

...access speeds to match. Write times on these devices is often very slow.

It's doable, and being done. but it's not just a matter of grabbing 16000 x 32 GB thumb drive chips and cramming them into a box.

Nor will relying on ordinary RAID protocols, safeguard you against data loss. With SSDs, and a relatively fixed number of write cycles, it's quite probable that the backup device will fail within a very short timespan of the primary.

[Taverner]

5. Ahhh makes sense

Thanks!

[Taverner]

8. What about if you create multiple redunancies, not through RAID

But through something traditionally used for "Big Data" like Hadoop or Gluster?

[Taverner]

10. BTW...why couldn't you use RAID?

Say, RAID 6 and multiple raid groups to make up a volume

[Recursion]

6. Finite writes make this a problem, and you'll need a logical filesystem that will be a PITA

You can only write to an SD a certain number of times before it goes kablooie. Now, you could have LEDs like RAID arrays do to show you which one to replace. But that gets to another point: what kind of redundancy do you want? RAID 6? 1+0? Something else? Write times and seek times are going to be problematic, and since the filesystem metadata and block map will have to be either in memory or on a faster filesystem, that's going to add a ton of complexity. That said, if you build one, I'll write the filesystem for it. That sounds fun.

[Taverner]

9. What about Hadoop for the FS?

You would have to do RAID 6, and not just that but several raid groups to make up one volume/FS

[Recursion]

11. Doesn't Hadoop rely on an underlying block FS?

I haven't looked at that stuff in a while.

[Taverner]

12. I don't know - but you could use Gluster for the FS

or really any good base block FS that has redundancy

[Recursion]

13. How about this: format each SIM card with FAT or something similarly fast and stupid

and then use each individual SIM card as a block in the overall FS. This could be cool.

[Taverner]

14. Hmmm I like that

[Phillip McCleod]

7. you're cool.

and you're all so damn smart! i do love DU