I just bought 40TB of hard drives

[Signa]

Not sure why this needs its own thread. I just wanted to post it so someone can react to it. Also to test out the site while it's experiencing Shrodinger's death.

It was 5x8TB drives, and I'm going to put them in RAID5, so it will only be 32TB usable. Still, if one crashes, I can rebuild the array without any data loss. I still need to build the rest of the PC. Just pricing the parts is making my head spin. I can get a relatively cheap i5 processor, and then I have to choose if I want an ATX or ITX form factor. Heat generation is going to be a big deal for me, because my current 4x2TB server heats my room year round. The thing that's really fucking with me is that 16GB of DDR4 RAM will cost just as much as the processor, and I'm not cool with that.

 

[Terminal Blue]

Why would you ever need that much storage?

Also, this may be the wrong time to point it out, but if you're going to spend that much shouldn't you add an SSD for the operating system and anything you need to load quickly and not slow down in a couple of years?

 

[Avnger]

That sounds... excessive. I'd also second the SSD argument. You'd have been better off grabbing like 10TB (or however much) of solid storage for the same price.

 

[EscapistAccount]

Why would you ever need that much storage?

Also, this may be the wrong time to point it out, but if you're going to spend that much shouldn't you add an SSD for the operating system and anything you need to load quickly and not slow down in a couple of years?

It's perfectly legitimate to need that much storage based on your hobby or business, particularly raw photo storage.

What I would say is OP should consider using something a bit more intelligent than pure RAID 5 with spinny rust, for that money I'd look at smarter tiered storage rather than a conventional RAID array. I also wouldn't do RAID 5, hard drives have a lifespan and when one pops others from that batch don't tend to be far behind. You should also keep some spare because the tendency is to run RAID 5 with no hot spare, pop a disk, go 'oh wow I can't afford to replace that I'll just cope for a few weeks', lose another disk aaaand you're fucked.

 

[Samtemdo8_v1legacy]

I wish I had that amount of Space on my PS3 and especially my PC.

What with new games going into 50+ GB download territory.

 

[KingsGambit_v1legacy]

That's a lot of space, good luck dude, but I have some technical advice and this is very important!
Do not use RAID5. I have been a proponent of RAID5 for many years, but regrettably it is deprecated in the face of increased HDD sizes. RAID5 is *not* feasible for arrays with drives over 3TB, and I'll explain why.

In RAID5, the data and parity is striped across all drives in the array. When one drive fails, the controller can compare the data on the remaining functioning drives to rebuild the missing one. However, with larger hard drives the chances of a read failure somewhere on one of the other drives is staggeringly high and with RAID5, a single read-error anywhere on a compromised array will result in a loss of all data.

With drives over 3TB, an the MTBF on hard drives on average, the chances you'll be able to successfully rebuild a RAID5 array is lower than 20%. Here is an article [https://superuser.com/questions/912673/are-raid-5-systems-suitable-for-larger-disk-sizes#912726] with numbers if you're interested.

I will be happy to explain further, but in short, DO NOT use RAID5, it is not good enough to keep your data safe with 8TB HDDs. You should use RAID6 or RAID10 (though RAID10 (stripe + mirror) uses an even number of disks, so RAID6 is better suited to 5 drives.) In RAID6, you will lose two drives to parity, and as a result have 24TB for storage.

Lastly, I would not use it in a PC. Put that storage in a Synology NAS and if you're making a PC, use an SSD for snappy boot times, loading times and a responsive experience. Happy to advise or discuss further.

 

[Redlin5_v1legacy]

TBH

I see myself needing that level of storage in the next five years. I work primarily with video files and let me tell you, a TB vanishes quickly. Unfortunately I don't have much in the way of funds so I'm in the process of reshuffling everything I've got to maximize the storage I do have for future projects. I've been too slap-dash about it.

 

[Dirty Hipsters]

I feel like that's way too much to have at the start of building a new computer. It'll probably take years to fill up that much space, and HDDs only get cheaper with time. You'd be better off just getting however much space you need right now and expanding your storage as needed over time rather than buying in bulk and installing it all now with the plan of future proofing.

 

[Signa]

Why would you ever need that much storage?

Also, this may be the wrong time to point it out, but if you're going to spend that much shouldn't you add an SSD for the operating system and anything you need to load quickly and not slow down in a couple of years?

Storing videos. Lots and lots of videos.

Don't need a SSD because this is going to be a media server with some game server hosting, such as Terraria or Freelancer. Just partitioning off a section will be enough for the booting of the computer. My current machine runs 24/7, and only needs to reboot every few months. I think I made 10 months straight once before a reboot.

I've been running a RAID5 in a 4x2TB setup for 7 years now, so it's time for a replacement. I've had to rebuild just one drive twice (the replacement died before the other 3 originals died). I also always keep a spare ready, and run a data check periodically to ensure integrity. I didn't buy a spare now, but I can get one next time I see a sale.

I also bought a RAID card, so I expect that's going to help a bit with keeping the UREs from cropping up because of the hardware checking.

SNIP

Thanks for the concern, and the article. Since it is an absurd amount of space (I decided on having 5 drives a year or two ago, before I decided on the actual capacity of each drive) RAID6 is actually an OK idea. I still haven't filled my 6TB, but I've been keeping my videos standard definition instead of 720p-1080p because I didn't want to fill it too fast. With 32TB, that won't be a problem, and I question if 24TB won't be a problem either. I was expecting to end up with 5x4TB or 5x5TB originally, so RAID6 would have dented that a lot more. I may even add an extra drive to make it 6x8TB and at that point RAID6 will be an attractive option. The RAID card I got has two SAS ports that I'm getting those 4 port SATA cables for, so I could get 3 more drives to stack into this baby.

I only have the drives and the RAID card right now, and I plan on building the rest of the PC next month. I just got a new job and I'd rather spread out the costs instead of putting it all into one month's costs.

TBH

I see myself needing that level of storage in the next five years. I work primarily with video files and let me tell you, a TB vanishes quickly.

It's amazing how much an absurd size becomes not absurd. When I built my last server in 2011, a single 2TB was kinda absurd, and getting 4 in a 6TB RAID was pretty extreme. Now I'm getting drives that equal all of them combined, and it's not even the largest drive I could have bought.

 

[KingsGambit_v1legacy]

Thanks for the concern, and the article. Since it is an absurd amount of space (I decided on having 5 drives a year or two ago, before I decided on the actual capacity of each drive) RAID6 is actually an OK idea. I still haven't filled my 6TB, but I've been keeping my videos standard definition instead of 720p-1080p because I didn't want to fill it too fast. With 32TB, that won't be a problem, and I question if 24TB won't be a problem either. I was expecting to end up with 5x4TB or 5x5TB originally, so RAID6 would have dented that a lot more. I may even add an extra drive to make it 6x8TB and at that point RAID6 will be an attractive option. The RAID card I got has two SAS ports that I'm getting those 4 port SATA cables for, so I could get 3 more drives to stack into this baby.

The space isn't an issue, people need what they need, no need to justify that. The point that needs stressing is that RAID5 just isn't good enough for HDDs of that size. The crucial points to note, from that above article:

With RAID 5, when one drive fails, there exists no redundancy whatsoever, which means that any error is non-correctable: any problem reading anything from any of the other drives ...your array breaks down.

And the reason that's an issue, from this article [https://www.zdnet.com/article/why-raid-5-stops-working-in-2009/]:

SATA drives are commonly specified with an unrecoverable read error rate (URE) of 10^14. Which means that once every 100,000,000,000,000 bits, the disk will very politely tell you that, so sorry, but I really, truly can't read that sector back to you.

Essentially, while a RAID5 array is compromised there is no further redundancy and an error anywhere on any drive will prevent a rebuild. Based on the larger sized disks, the chance of an error is crazy high and gets higher as disk size and number of disks increases. In fact, the chance of having an error is much higher than not having an error.

What RAID6 does, by essentially giving you (the equivalent of) another parity disk, it means that there would need to be two read errors on the same exact data for a rebuild to fail and the chances of that are much, much smaller.

 

[Signa]

snip

Do you know anything about the RAID Media Patrol features for error finding and correcting? I heard about UREs with RAID5 years ago, but the article I was linked then seemed far more obsessed with enterprise environments. Maybe I'll buy a 6th disk when I build the rest of the PC next month, and then I'll get the space I was expecting. Money isn't that much of an issue for me at this moment

 

[Chessrook44]

Why would you ever need that much storage?

Some people REALLY love their porn.

Sorry, I had to make the joke. I'm only surprised nobody else made it.

 

[Signa]

Why would you ever need that much storage?

Some people REALLY love their porn.

Sorry, I had to make the joke. I'm only surprised nobody else made it.

I very nearly did.