[jsaddiction]

I have been interested in building my first custom home media server for a while now to eliminate some issues I have with my current build.  Mostly my limiting factors have been cost.  Let me first identify my current configuration and the limitations I have unfortunately come to live with.  [font][font]I am not really looking for support for these issues as I have spent the last several years trying to solve them through the company and community without much forward progress. [/font][/font]  Then I would like to spotlight some extra features that the new system should have and hopefully with your input I can nail down some good hardware choices.

My current system

• 8 bay Synology NAS with 32TB of space using WD RED drives which is currently 93% capacity.
  
• I have upgraded this box to its max capacity of 3GB of ram. 
  
• It is connected to an 88U ASUS router through link aggregation. 
  
• I have all the common USENET applications to manage my TV shows, Movies, and Music. 
  
• I manage most everything remotely so reverse proxy and "let's encrypt" is used everywhere. 
  
• I have MariaDB running to centralize my database
  
• Headless Kodi running in a docker to scan new content into the library
  
• Synology apps such as Audio Station, Video Station, Photo Station
  
• 3 Chrome boxes, 1 RPI2 client machines. One chromebox is wired, all others are 5G WIFI
  

Issues with my current build

• Any "Over the internet" uploads are capped at around 350Kb/s.  This speed will not support any video unless its quality is severely degraded.  File downloads from a remote location take forever. 
  
• Secondly DSM goes to great lengths to prevent you from using HTTP -> HTTPS redirects or enabling web sockets.  The work around for this is to SSH into the system and manually configure the nginx server to allow it.  Once a software upgrade is completed on DSM, you have to reconfigure again.  I could probably write a script to circumnavigate this issue but it seems a bit hacky. 
  
• Third, Synology's media software (Audio Station, Video Station, Photo Station) requires an "indexing" service to run on your system.  This service is "triggered" every time a file change is initiated or a command is sent from one of the USENET applications.  It then proceeds to scan your entire library and takes a very long time while pegging your volume utilization to near 100% reducing overall system performance greatly. 
  
• Fourth, while having a 1gb upload and download internet connection NZBget can only download up to 35Gb/s.  This is plenty fast enough but the bottleneck is the cpu's ability to decrypt and the volumes speed. 
  
• Fifth, occasionally and at random, one of my in use chromebox clients will freeze during playback.  This happens when the volume cannot keep up with the demand (reads/writes)
  
• Lastly, mariaDB runs pretty slow for library scans, especially for music.  I am not sure if this is related to cache settings not being large enough but I am sure slow volume speeds and slow CPU don't help.
  

What I would like to achieve with the new build

• Software
  • sickrage, couchpotato/radar, headphones, nzbget,
    
  • plex (to stream content over WAN),
    
  • headless Kodi (VM or docker),
    
  • web server (nginx or similar for web application development),
    
  • VPN server, etc...
    
• Hardware
  • 32TB to start with and expandability to 64TB (10 or more bays)
    
  • At least double the RAM to 6GB (room for more applications and faster mariaDB)
    
  • SSD for operating system and applications
    
  • CPU or CPUs that will handle transcoding on the fly and or during download process (2-3 simultaneous non native PLEX clients requiring transcoding)
    
  • Liquid cooling? maybe not sure.
    
• Functionality
  • Remote management (web interface preferred can't use RDP from work)
    
  • Stream content over the internet (1-3 clients)
    
  • User management to configure both machines and people to have own login credentials (help with troubleshooting)
    
  • Fast RAID (possibly pci hardware controller)
    
  • Rip BDs
    
  • Reverse Proxies
    
  • Saturate the 1gbe WAN connection (WAN should be the bottle neck not hardware)
    
  • Eventual support for 4k videos once native client devices are obtained within the local LAN
    
  • Super Stable.  (Synology slows after about 2 weeks of run time)
    
  • any other functionality that would be nice for this type of server
    

Last note.. I fully understand and don't mind this server dimming the street lights when it is actively transcoding or downloading but I would like minimal power usage when it is idle.  Maybe store the SQL database on the SSD for speed and ability for the RAID to "sleep" until download or stream.  I apologize for the long winded post but hopefully it defines what I need well enough to initiate a focused discussion.
TIA!!!

[t2ffn]

The least detailed answer you'll likely get on here but I have a Qnap TS-453 Pro with 8Gb Ram, and the expansion add on module, for (4+5) 9 drives total. (x6Tb WD Red) in Raid 5 and it covers all of those bases, at least the ones I use or can speak to. It also allows for the ethernet ports to be coupled for higher throughput if needed (?!) but I don't use this and and can stream 4K via normal gigabit Lan to multiple clients just fine so not sure what you'd need to be doing to need this. It has 4 gigabit ports so can be configured with multiple Lan connections, again, never needed it so can't comment if useful.

It's a couple years old so the newer Qnap Pro models probably have it pipped in terms of overall performance but it handles everything I throw at it, same applications, without the issues or weird software hangups you mention from Synology. 

I'm very pleased with it, and as it allows for expansion modules it grew with me as I needed more volume and could spread the cost rather than all up front.

[jsaddiction]

Very interesting that you suggest QNAP.  When I was first getting into the home "server" realm I looked at both Synology and QNAP as viable options.  As of 6-7 years ago I felt that Synology was the better alternative.  I can't remember the specific details for the why but for whatever reason I chose what I did and it worked impressively well.  My media collection grew as did my software requirements and general need for speed.  The more I play around with this array of software I realize that all I really have is a "file server".  It does well at storing and sharing content but fails miserably at transcoding and anything else that requires performance due to the "Atom" processor and general configuration. Low power requirements are awesome, however, my current system seems inefficient and thus the desire to build my own.  I would like to run the OS on an SSD, use an intermediate SSD for post processing downloads (extraction / transcoding) then finally move to RAID for storage.  This should keep the RAID activity as low as possible.  Currently my OS is on the same volume as my media. (scary) Additionally any functionality that requires database reads (opening the movie list in KODI) should be crazy fast.  I would like to speed up library updates but I think my problem currently is the speed (available speed) of the volume for reads. 

While I do appreciate your input I think building my own server will be both cost effective for what I want and have the ability to grow as my software requirements and media collection do the same.  I will however take another look at QNAP just to see what their current offerings are and what price point fits my requirements as a reference.

[wrxtasy]

For those on a budget these look likely to provide great Bang for the Buck

Based on Hardkernel’s own tests, you should be able to max out the Gigabit Ethernet bandwidth while transferring a files over SAMBA in either directions. tkaiser, an active member of Armbian, also got a sample, and reported that heat dissipation worked well, and that overall Hardkernel had a done a very good job.

Hardkernel ODROID-HC1 Home Cloud One Stackable NAS is now available for $49

HArdkernel ODROID-HC2 Linux NAS System for 3.5″ Hard Drives Launched for $54

[Harro]

Very interesting that you suggest QNAP.  When I was first getting into the home "server" realm I looked at both Synology and QNAP as viable options.  As of 6-7 years ago I felt that Synology was the better alternative.  I can't remember the specific details for the why but for whatever reason I chose what I did and it worked impressively well.  My media collection grew as did my software requirements and general need for speed.  The more I play around with this array of software I realize that all I really have is a "file server".  It does well at storing and sharing content but fails miserably at transcoding and anything else that requires performance due to the "Atom" processor and general configuration. Low power requirements are awesome, however, my current system seems inefficient and thus the desire to build my own.  I would like to run the OS on an SSD, use an intermediate SSD for post processing downloads (extraction / transcoding) then finally move to RAID for storage.  This should keep the RAID activity as low as possible.  Currently my OS is on the same volume as my media. (scary) Additionally any functionality that requires database reads (opening the movie list in KODI) should be crazy fast.  I would like to speed up library updates but I think my problem currently is the speed (available speed) of the volume for reads. 

While I do appreciate your input I think building my own server will be both cost effective for what I want and have the ability to grow as my software requirements and media collection do the same.  I will however take another look at QNAP just to see what their current offerings are and what price point fits my requirements as a reference.

I would highly suggest you look at Unraid. https://lime-technology.com/

I started off years ago with a small test server. From there I built the one in my sig, which has grown to 90TB storage and still running strong. Upgraded the CPU, PSU and kept installing larger drives. Now with the release of stable 6.4 with included partner https://letsencrypt.org/ makes it even better.

[jsaddiction]

For those on a budget these look likely to provide great Bang for the Buck 

Based on Hardkernel’s own tests, you should be able to max out the Gigabit Ethernet bandwidth while transferring a files over SAMBA in either directions. tkaiser, an active member of Armbian, also got a sample, and reported that heat dissipation worked well, and that overall Hardkernel had a done a very good job.

Hardkernel ODROID-HC1 Home Cloud One Stackable NAS is now available for $49HArdkernel ODROID-HC2 Linux NAS System for 3.5″ Hard Drives Launched for $54 

 This is a really cool design and plays to my hardware developer side but don't think it would transcode on the fly especially if it were running the common software.  I am going to look at this for other applications though.. maybe a mobile KODI

[jsaddiction]

Very interesting that you suggest QNAP.  When I was first getting into the home "server" realm I looked at both Synology and QNAP as viable options.  As of 6-7 years ago I felt that Synology was the better alternative.  I can't remember the specific details for the why but for whatever reason I chose what I did and it worked impressively well.  My media collection grew as did my software requirements and general need for speed.  The more I play around with this array of software I realize that all I really have is a "file server".  It does well at storing and sharing content but fails miserably at transcoding and anything else that requires performance due to the "Atom" processor and general configuration. Low power requirements are awesome, however, my current system seems inefficient and thus the desire to build my own.  I would like to run the OS on an SSD, use an intermediate SSD for post processing downloads (extraction / transcoding) then finally move to RAID for storage.  This should keep the RAID activity as low as possible.  Currently my OS is on the same volume as my media. (scary) Additionally any functionality that requires database reads (opening the movie list in KODI) should be crazy fast.  I would like to speed up library updates but I think my problem currently is the speed (available speed) of the volume for reads. 

While I do appreciate your input I think building my own server will be both cost effective for what I want and have the ability to grow as my software requirements and media collection do the same.  I will however take another look at QNAP just to see what their current offerings are and what price point fits my requirements as a reference.

I would highly suggest you look at Unraid. https://lime-technology.com/

I started off years ago with a small test server. From there I built the one in my sig, which has grown to 90TB storage and still running strong. Upgraded the CPU, PSU and kept installing larger drives. Now with the release of stable 6.4 with included partner https://letsencrypt.org/ makes it even better. 

 I was actually looking at this design concept.  Very interesting.  Do you see any negative / shortcomings while utilizing that system?  There isn't much on their website (going to dig in the forums) in the way of what capabilities it has.

[noggin]

For those on a budget these look likely to provide great Bang for the Buck 

Based on Hardkernel’s own tests, you should be able to max out the Gigabit Ethernet bandwidth while transferring a files over SAMBA in either directions. tkaiser, an active member of Armbian, also got a sample, and reported that heat dissipation worked well, and that overall Hardkernel had a done a very good job.

Hardkernel ODROID-HC1 Home Cloud One Stackable NAS is now available for $49HArdkernel ODROID-HC2 Linux NAS System for 3.5″ Hard Drives Launched for $54 

 This is a really cool design and plays to my hardware developer side but don't think it would transcode on the fly especially if it were running the common software.  I am going to look at this for other applications though.. maybe a mobile KODI   

No point looking at it for Kodi - it has no video output!

It does make a neat TV Headend server, and OpenMedia Vault runs nicely on it too (I've got an HC1 with OMV + TV Headend running with an HD Homerun dual IP tuner and they work nicely together)

I absolutely support anyone who suggests unRaid. I've been using it for 10 years. I'm on my second hardware build (cheap Core I-based Celeron) iteration of one server (and have a second licence on an HP Proliant Microserver in a secondary location), and have migrated all my data from 1TB drives to 4TB, though am now looking at upgrading them again at some point to increase capacity. unRAID has done me very well.

[Harro]

 I was actually looking at this design concept.  Very interesting.  Do you see any negative / shortcomings while utilizing that system?  There isn't much on their website (going to dig in the forums) in the way of what capabilities it has.

I see no negative or shortcomings on my end. Well maybe the weight of my case with all hard drives in it, but I never move it.
The forum is the best place to look around, ask questions and browse all the dockers. 
Unraid just works and is expandable to your needs.

[tential]

As others have said OP, Unraid is your friend here.
It does everything you asked for.

I only used it for storage, but now that I use it as my actual server, it's amazing.  It has simplified my setup immensely.  I used a Rosewill Chassis for my build with 15 bays.

It's now full, ~90 TB of data, but I still have 5 HDDs in boxes (I thought I broke my server and rage bought 10 8TB HDDs....), so I'll be making my second UNRAID server to handle Steam In Home STreaming (since it takes up my main PC which is unacceptable so off to the server it goes!!!!)

What I'd get
UNRAID OS
Core i3 Processor (Ryzen may be better, but I don't know for sure.  I imagine it would be, but for pure simplicity/compatibility, I'm telling you an i3, but if you want to get the max performance you may want to look into how Ryzen transcodes for Plex as intuitively, I believe it would be better.
HDDs - I buy 8TB enclosures and dismantale them.  Cheapest way to get HDDs
SSDs - Buy on sale.
Ram - 8 GB minimum IMO.  Upgrade to 16GB later if needed.

Really, what you're asking kind of is a hardware/implementation.  The hardware part is pretty straight forward though, looks like you've got a great grasp on that.

Just need to pull it all together, and that's UNRAID OS.  Honestly, I think if you just look into UNRAID OS OP, you'll be fine and probably don't need much guidance from us, but more than willing to give it.  I think once you look at that, you'll find you just need help implementing it, which people are more than happy to do.

Unraid OS is one of the best purchases of software I've made though for sure.

[a5ian300zx]

I was about to suggest qnap to.

I have a qnap 871 with i7 and 16GB ram and can transcode on the fly multiple WAN users using plex server.

It also supports docker Station, virtualisation Station, vpn, encrypted Web browser. You can access virtual desktop via WAN.

Has loads of apps on offices qnap store but also loads more on the unofficial qnap club https://www.qnapclub.eu/en Inc kodi 17.1.

Only downside is hdmi 1.4 port but really like the always on hd Station with YouTube, Web browsers, kodi, etc can. Be used with flic remote.

I have 10gbe installed in mine so I do all my remuxing on pc and transfer across but can do it on the qnap using the ssd cache feature on virtual machine.

You can also consider Supermicro tn4 superserver which I also have but used for esxi only.

Thanks

[nickr]

What about just straight Linux distro like ubuntu server? This will give maximum flexibility, and always access to the latest kodi builds.

[speedwell68]

Why you docker and headless Kodi to scan the library?  That seems like an awful waste of resources to do something that could be done with an addon on your Raspberry Pi.

[atcusb]

more of a lurker than poster, but seeing as unraid is being mentioned thought I'd add my experience to the mix. I went for unraid beta at 6.3 originally for IGD (allow intel graphics to be used in virtualisation without having another GPU) more than anything else, keeping my htpc as a VM, and nas on the same device to cut down on the ever growing bits of hardware appearing in my front room. The NAS bit went perfectly, and the advantages of being able to use any odd disk rather than having to standardise on one size was excellent.

However what really makes unraid for me, is the virtualisation and docker options it gives. Admittedly, this wasn't straight forward, but now I have a dual ethernet intel nic passed through to a pfsense VM (multiple VLANs and load balanced VPN WAN's), unifi controller as docker to control my wireless AP, TVHEADEND running as a docker with a dual DVB tuner , and the icing on the cake, a Krypton ubuntu VM, with Skylake graphics and audio passed through, connected via HDMI to my flatscreen.  Access to my library is lightning fast as it's all running on the back plane of the server rather than over the network.

I've gone from modem, router, switch and 4 devices to just modem, switch and unraid. Only downside is the total outage (and consequently the family nagging) if the server crashes or is rebooted. In a year and a half, I've only really had one major issue when I tried to apply an upgrade to unraid with some incompatible addons installed.

[tential]

more of a lurker than poster, but seeing as unraid is being mentioned thought I'd add my experience to the mix. I went for unraid beta at 6.3 originally for IGD (allow intel graphics to be used in virtualisation without having another GPU) more than anything else, keeping my htpc as a VM, and nas on the same device to cut down on the ever growing bits of hardware appearing in my front room. The NAS bit went perfectly, and the advantages of being able to use any odd disk rather than having to standardise on one size was excellent.

However what really makes unraid for me, is the virtualisation and docker options it gives. Admittedly, this wasn't straight forward, but now I have a dual ethernet intel nic passed through to a pfsense VM (multiple VLANs and load balanced VPN WAN's), unifi controller as docker to control my wireless AP, TVHEADEND running as a docker with a dual DVB tuner , and the icing on the cake, a Krypton ubuntu VM, with Skylake graphics and audio passed through, connected via HDMI to my flatscreen.  Access to my library is lightning fast as it's all running on the back plane of the server rather than over the network.

I've gone from modem, router, switch and 4 devices to just modem, switch and unraid. Only downside is the total outage (and consequently the family nagging) if the server crashes or is rebooted. In a year and a half, I've only really had one major issue when I tried to apply an upgrade to unraid with some incompatible addons installed.

Are you using community applications for installing/running dockers?
https://www.youtube.com/watch?v=su2miwZNuaU

This makes it so easy to manage a server that my simpleton self moved all of my stuff to my server in a couple of hours.

With community applications now, unraid is just so easy to use.  

A QNAP
https://www.amazon.com/QNAP-TVS-871-i7-1...B00S0XRY7Q
Isn't what the OP is looking for as it's only 8 bays.  It's still a nice solution if the OP doesn't plan on building anything himself.  But I think, and this is just judging from the OPs post and me making a guess based on his/her level of expertise, they look like they are planning to build something themselves.

As for cooling OP, I saw that mentioned.  You can use stock intel coolers.... That's what I'm doing.  With my server chassis currently open, I can't hear anything.  I actually am wondering if it's on... ya it is.  Ok. But I'd still replace all your fans with Noctua fans, or Phantek (or whatever your favorite high end cooler brand is).  I just really like my builds to be quiet though.  Liquid cooling is not worth it.

You really won't get the combination of ease of setup/performance/flexibility in any other package than unraid in my opinion.  Yes, you could just run ubuntu server, but I honestly am way too stupid for that.  Unraid + Community Applications really made it stupidly easy for me to utilize unraid to a far better potential.  That youtube channel I posted really has made it stupidly easy for me to use Unraid now.

My next step is a new unraid server for my additional harddrives.  That will also serve as a "gaming server" to allow people to game on my high end system from any client box using the steam in home streaming.  Looks like  you can launch steam from kodi and exit back to kodi pretty easily which I'm excited about.