40 TB HTPC Monster: What components to buy?
#16
(2014-06-02, 02:01)_Andy_ Wrote: If you need data safety i always recommended hardware raid. SAS because of the enterprise usage. The drivers and hardware are much better than for SATA controllers. Because of the mainly usage it's obvious why. I'm working as a system administrator and if you need safety and stable data systems use SAS hardware RAID. The disadvantage is the price. You have to initially pay more money. That's true.

If it's ok to loose data if a hard drive fails you can use SATA controllers. Software RAIDs are mostly working but in my experience i don't trust the systems. My friend lost his complete RAID of his Buffalo NAS. And these NAS systems are always software RAIDs.
After a HD failure he wasn't able to rebuild the RAID. He bought a new HD but it wasn't working.
I understand your point, I was a sys admin and have worked with IBM / HP and Dell systemas for years. Buit I have seen hw RAID5 systems fail also, so hw is not warranty of no failure. In fact, the sw does what the hw would do, just with more effort.

(2014-06-02, 02:01)_Andy_ Wrote: SAS is easy to enhance. A expander card from HP cost about 130$. With the expander you can connect 24 more drives and mix as RAID or NON RAID. Another HP expander and 24 more drives :-) As you like.
My question is, what should I need to buy? I'm sure that it would not only be 130 dollars. I'm sure there is some case to use and probably more. What would it cost for me to implement 10 hard drives space like you mention?

(2014-06-02, 02:01)_Andy_ Wrote: File systems like ZFS are working too but you should have Linux knowledge. In daily usage you can do most things from a GUI but if something went wrong it's sometimes better to use the command line. If the system is fine and working all is good. But if the system fails and something is wrong, i prefer enterprise solutions. I can speak from my 11 years of experience only.
I have experience with Linux and AIX, I'm not an expert but today the internet helps a lot.

(2014-06-02, 02:01)_Andy_ Wrote: A good 650VA PSU should handle fine. With my setup (24 drives but staggered spinup) i use a 750VA PSU. During the first 5 seconds after starting the system, i have a load of 580VA. If all drives are powered up (takes about 2 minutes) the load is 380VA. Your chosen PSU should work. I don't like the power efficiency much but from the power aspect it works.
I think I already mentioned it, but I have 2 x 20TB towers in RAID 0, and I plan to create a 3rd tower with 40 TB that I would backup to the 2 20TB towers. So the redundancy is provided by the backup towers.

That said, I would love to have what you mention, but my budget is 450 dollars, do you think I can make it with that little?

(2014-06-02, 02:09)MrCrispy Wrote: Hi kwanbis, you can read about RAID levels on Wikipedia. FlexRaid isn't RAID0, it implements it's own system for storing recovery information.
I know about RAID levels, thanks but it is not very clear what FlexRaid does.

(2014-06-02, 02:09)MrCrispy Wrote: Andy's suggestions are great for higher budgets, bujt much more suited for the enterprise. You don't need that level for the home. And you want to be able to expand storage needs without having to buy more cards.
Well I sure don't want to loose 40 TB of information, but I have to find a valance between the cost and the security.

If anyone can give me a better option for 450 dollars, 500 tops, I'm open to suggestions!
Reply
#17
(2014-06-02, 03:01)MilhouseVH Wrote: Honestly, nobody recommends hardware RAID controllers anymore and with very good reason... Nowadays the recommendation for RAID is to use a dumb SATA/SAS HBA controller and some form of software RAID on top. With software RAID, in the event of controller failure, any other dumb HBA controller (different make, firmware level, etc.) can be used as a replacement - you can even read your disks in a regular PC motherboard port if you need to... Hardware RAID controllers as a recommended solution died out years ago, and good riddance. Do yourself a favour and stop recommending them.
That is also a good point.

Can you recommend a good 8 ports SATA/SAS controller?
Reply
#18
kwanbis, I will let you read the details here - http://www.flexraid.com/faq/

In summary, FlexRaid implements data parity protection similar to RAID but without the limit on disk sizes, storage protocols etc. Such software approaches are also implemented in systems like unRaid, ZFS, WHS etc, basically RAID is very old technology that doesn't scale well at all. You can read this post about RAID limitations here - http://blogs.technet.com/b/homeserver/ar...ology.aspx, and similar documents on the unRaid site.

e.g. with FlexRaid you can have data in any number of 3TB drives and protect all of them using a single 3TB drive (the parity drive), which will protect against single drive failure. The mode I recommend to people is snapshot Raid, which they call RAID-F. The main advantage is that media storage changes very infrequently, only when you add more media. So as long as you take a snapshot regularly, you don't need to monitor the disks in real time. In a real Raid system, every single drive is running all the time, increasing power draw, chances of failure etc. Not in the other systems I mentioned.

There are many possible builds for your budget, you have to take a look and decide on parts cost where you live, what you can reuse, how powerful you want it to be, does it need to do transcoding etc etc. There is certainly no way you can do h/w Raid, let alone SAS, on that budget.

e.g. you can look at this - http://forum.xbmc.org/showthread.php?tid=185337
Reply
#19
(2014-06-02, 04:10)MrCrispy Wrote: kwanbis, I will let you read the details here - http://www.flexraid.com/faq/

In summary, FlexRaid implements data parity protection similar to RAID but without the limit on disk sizes, storage protocols etc. Such software approaches are also implemented in systems like unRaid, ZFS, WHS etc, basically RAID is very old technology that doesn't scale well at all. You can read this post about RAID limitations here - http://blogs.technet.com/b/homeserver/ar...ology.aspx, and similar documents on the unRaid site.

e.g. with FlexRaid you can have data in any number of 3TB drives and protect all of them using a single 3TB drive (the parity drive), which will protect against single drive failure. The mode I recommend to people is snapshot Raid, which they call RAID-F. The main advantage is that media storage changes very infrequently, only when you add more media. So as long as you take a snapshot regularly, you don't need to monitor the disks in real time. In a real Raid system, every single drive is running all the time, increasing power draw, chances of failure etc. Not in the other systems I mentioned.

There are many possible builds for your budget, you have to take a look and decide on parts cost where you live, what you can reuse, how powerful you want it to be, does it need to do transcoding etc etc. There is certainly no way you can do h/w Raid, let alone SAS, on that budget.

e.g. you can look at this - http://forum.xbmc.org/showthread.php?tid=185337
I understand RAID very well, but it is difficult to understand FlexRaid. But it seems to me that you are saying that it works with a parity drive? So if I have four 3TB hard drives and four 2TB hard drives, how much real space do I have?

Today I have a "manual" RAID 1, which means I have four 3TB hard drives and four 2TB hard drives on one tower and the second configuration on the second tower. So I have 2 "merged" 12TB hard drives and 2 "merged" 8TB hard drives (20TB per tower), and I manually synchronize them weekly.

Now what I want to do is create a 10 4TB tower (40 TB), and use the remaining towers to backup this 40TB.

So it is not clear to me which of the tRAID or RAIDoFS would better suit me. It seems to me RAID over File System is the one, but then it says:

"That’s not to say RAID-F does not have its place. For users that can separate their static/semi-static data from data that changes frequently, a lot of efficiency can be gained from running a Snapshot array. If the bulk of your data is comprise of disc images, movies, music, pictures, zip archives and the likes, backup images, etc., it might make sense to create a Snapshot array that only hosts such data. Then, you could put your dynamic data on a RAID 1 array if it can fit on a single disk or on a tRAID array if it spans more than one disk.

And that confuses me because I think the snapshot option is what I want, but I have a lot of hdds, and it seems to me it does not works?
Reply
#20
(2014-06-02, 03:48)kwanbis Wrote: Can you recommend a good 8 ports SATA/SAS controller?

I have 2 of these in my build and have no complaints
8 port card
Reply
#21
(2014-06-02, 06:03)Harro Wrote: I have 2 of these in my build and have no complaints. 8 port card
Thanks harro, but those are 110 dollars! more than half the mb cost! I was thinking inf instead of spending 160 on the mb and about 30 more on the 2 SATA card, I could get a "normal" mb and add 8 SATA, but it seems it is more expensive.
Reply
#22
So, so far, I would be buying this:
CASE: Rosewill Gaming ATX Full Tower Computer Case THOR V2 Black $110 http://amzn.com/B0058P5S9A
PSU: XFX PRO650W Core Edition 80+ Bronze $84 http://amzn.com/B0045L5LGI

Now, my doubt is the motherboard.

Do I buy the ASRock Fatal1ty Z87 Professional $157 http://www.newegg.com/Product/Product.as...6813157377 which has 10 SATA ports? Or buy any of this ones?

ASRock PRO4-M: http://www.asrock.com/mb/Intel/Z77%20Pro...ifications
ASRock Z77 Extreme4: http://www.asrock.com/mb/Intel/Z77%20Ext...ifications
ASUS P8Z77-V: http://www.asus.com/Motherboards/P8Z77V/
ASUS P8Z77-V PRO: https://www.asus.com/Motherboards/P8Z77V_PRO/

The Fatal1ty looks very nice, BUT, I would have to buy a new i3 also, as it is 1150 and my processor is 1155.
Reply
#23
The way I look at this is you need to first make a long term goal on what you actually are trying to solve. From what I gather on your posts is that there are 4 concerns:
1) having more storage
2) have additional servers to back up your storage.
3) using hardware you already have
4)$$

My thought process works that if I was to build another storage unit , I would think about how much storage I thought I may need for say 5 years. I then would start researching on what hardware would get me there now and at what price. If the price was too much for me now to build, what would be a cheap substitute or band aid until I could get to build my dream storage. For me it was adding external drives and then building the cheapest 6 drive NAS. This held me over until I built my server I have listed in my sig. The server in my sig will handle up to 24 drives even though I am only using 18 right now. And with the bigger and bigger hard drives coming out I do not foresee me needing to build another. Because even if I swapped only 15 drives out for 6TB drives that would net me 90 TB of storage.

Sorry I am not much help for you but thought I would share my thoughts on how I went about the solution of media storage.
Good Luck
Reply
#24
(2014-06-02, 07:02)Harro Wrote: The way I look at this is you need to first make a long term goal on what you actually are trying to solve. From what I gather on your posts is that there are 4 concerns:
1) having more storage
2) have additional servers to back up your storage.
3) using hardware you already have
4)$$
Thanks for your comments. Actually, I know exactly what I want. I want a reliable way of accessing 10 4TB hard drives as a single drive (actually I rather have access to them as 2 "virtual" drives), and spend as less as possible, ideally less then 450.

So far, I think the only option is a HTPC and so I'm asking for recommendations on Case, PSU, and Motherboard. Since I know it is very difficult to find an 11 ports motherboard, I'm also asking for recommendation on expansion SATA cards.

Then, Andy and other suggested other options, and since I'm open to better ideas, I have asked, but as it is now, my solution is the one I already mentioned, a "monster htpc".

By the way, it looks like you went for a similar process! I'm gonna read, because I assume the specs on the first post are not the ones you are mentioning.
Reply
#25
For 350 plus shipping this is all you need and under budget, Smile
14 drive server with all hardware except drives
Reply
#26
(2014-06-02, 07:26)Harro Wrote: For 350 plus shipping this is all you need and under budget, Smile
14 drive server with all hardware except drives
hahah, yeah, but it is already sold or something.

By thw way, what case are you using on your server? I mean, the "Norco SS-500 Hot Swap Rack Moduke" seem to plugged into something, and last, you mean a 550w PSU can handle 18 hard drives without issues?
Reply
#27
The Coolermater 590 which is discontinued, and yes my 550 PSU is doing fine. With Unraid the drives are spun down most of the time, so no heavy draw on PSU. I have upgraded the CPU to an i5 but everything else stayed the same. I may have rebooted the server about 6 times since those pictures were taken, otherwise is running smoothly 24/7. I also still have my 6 drive NAS also but it is not being used at this time, and soon will be selling it.
The Norco's plug into the sas 4 port cards.
Reply
#28
(2014-06-02, 07:43)Harro Wrote: The Coolermater 590 which is discontinued, and yes my 550 PSU is doing fine. I have upgraded the CPU to an i5 but everything else stayed the same.
So going for a 650w PSU is way overkill?

(2014-06-02, 07:43)Harro Wrote: The Norco's plug into the sas 4 port cards.
I don't understand, it seems they are inside a tower case, correct? What is the case?

http://i1110.photobucket.com/albums/h442...eTower.jpg
Reply
#29
In my sig is my build which has pictures of the inside.
Image
Reply
#30
(2014-06-02, 07:53)Harro Wrote: In my sig is my build which has pictures of the inside.
But I mean, it is obvious the "3 x Norco SS-500 Hot Swap Rack Moduke" are inside a case, they are not left lying around, or I'm wrong?
Reply

Logout Mark Read Team Forum Stats Members Help
40 TB HTPC Monster: What components to buy?1