(2014-01-22, 01:19)jacintech.fire Wrote: [ -> ]... maybe someone will see this thread on the web and that would spur a thought and something will come off it.
That much is already possible. Distributed storage, Ceph is my personal favourite; I have a small 8TB block storage array working across 3 nodes, I then mount those blocks, using RADOS (as others have mentioned) at the OS level. Sure, that doesn't leverage the object component, but ...
All that's not happen right now, is an object aware media player modification (I'm no dev, are they the right words?!). Most don't have a (need for) PB of storage; let alone the media to fill it, so as has been sung many times in this thread, I don't think this will happen soon if only for that reason.
I like the scale of your storage; perhaps not the power bill!
But, if you want to create a project from it, start with OpenStack maybe. I don't know - if you have the vision, run with it!
@
n1md4,
I favour openStack simply because in addition to storage it has both a compute and networking shared services (which will give me the flexibility to continue experimenting).
I can get OpenStack up and running and plug it into samba and export the volumes that way...but what I want is a "Object Store-Aware" player to interact with the Object Store Natively (i. e query the store, pull and manipulate the objects without the need of any mapping mechanism similar to the Ceph metadata server cluster; which provides a service that maps the directories and file names of the file system to objects stored within RADOS clusters. ).
I will probably move my whole storage to openStack or a similar model very soon (There is still more reading and experimenting before I can safely pull the trigger)...
(2014-01-21, 18:23)jacintech.fire Wrote: [ -> ]@TR800
a) 512 1TB: No reason whatsoever. I have always partitioned my drives, figuring that if something happens (loosing the index, for example), it would be restricted to 1TB of data and not the whole drive.
b) Duplicate Titles: Streaming a 10GB movie over wifi (or remotely) is a crap shot. Sometimes it works, most times it does not. A 650-750MB .mp4 file on the other hand, is far more forgiving.
c) RAID: Scaling to a petabyte using RAID is <insert adjetive>!!! Knowing that, I decided to read up on alternatives. Distributed filesystem, object storage and the rest seem to offer promise that RAID does not.
d) How cool would it be to try something new...see where it would take me...
a) You say no reason whatsoever and then give a weak reason. I had 8 x 2TB partitions of movie drives before I moved to unRAID and it was horrible to look after. God knows how it is doing it with 512 of them.
b) That's why you transcode and not have thousands of duplicate files.
c) I get why you don't go for certain types of RAID but not why you don't opt for others like unRAID or something similar.
d) If you're talking about having duplicate files and all that partitioning, not even slightly remotely cool.
(2014-01-22, 08:32)T800 Wrote: [ -> ] (2014-01-21, 18:23)jacintech.fire Wrote: [ -> ]@TR800
a) 512 1TB: No reason whatsoever. I have always partitioned my drives, figuring that if something happens (loosing the index, for example), it would be restricted to 1TB of data and not the whole drive.
b) Duplicate Titles: Streaming a 10GB movie over wifi (or remotely) is a crap shot. Sometimes it works, most times it does not. A 650-750MB .mp4 file on the other hand, is far more forgiving.
c) RAID: Scaling to a petabyte using RAID is <insert adjetive>!!! Knowing that, I decided to read up on alternatives. Distributed filesystem, object storage and the rest seem to offer promise that RAID does not.
d) How cool would it be to try something new...see where it would take me...
a) You say no reason whatsoever and then give a weak reason. I had 8 x 2TB partitions of movie drives before I moved to unRAID and it was horrible to look after. God knows how it is doing it with 512 of them.
b) That's why you transcode and not have thousand of duplicate files.
c) I get why you don't go for certain types of RAID but not why you don't opt for others like unRAID or something similar.
d) If you're talking about having duplicate files and all that partitioning, I don't think it's even slightly remotely cool.
This thread is not about his current setup, it is about the possibility of getting object store support into XBMC.
(2014-01-22, 08:40)nickr Wrote: [ -> ]This thread is not about his current setup, it is about the possibility of getting object store support into XBMC.
The problem is that the current setup undermines any technical credibility the OP brings to the discussion of an object store (and quite possibly, anything to do with IT).
Obviously the two are (or should be) unrelated, but anyone that would build such a horror then try to justify it is going to be on a losing wicket when attempting to "educate" the unwashed masses on alternative storage solutions.
(2014-01-22, 09:09)MilhouseVH Wrote: [ -> ] (2014-01-22, 08:40)nickr Wrote: [ -> ]This thread is not about his current setup, it is about the possibility of getting object store support into XBMC.
The problem is that the current setup undermines any technical credibility the OP brings to the discussion of an object store (and quite possibly, anything to do with IT).
Obviously the two are (or should be) unrelated, but anyone that would build such a horror then try to justify it is going to be on a losing wicket when attempting to "educate" the unwashed masses on alternative storage solutions.
Yeah I see your point.
Frankly I just mhddfs to join my JBOD disks in one pool (well one for movies, one for TV).
(2014-01-22, 09:09)MilhouseVH Wrote: [ -> ] (2014-01-22, 08:40)nickr Wrote: [ -> ]This thread is not about his current setup, it is about the possibility of getting object store support into XBMC.
The problem is that the current setup undermines any technical credibility the OP brings to the discussion of an object store (and quite possibly, anything to do with IT).
Obviously the two are (or should be) unrelated, but anyone that would build such a horror then try to justify it is going to be on a losing wicket when attempting to "educate" the unwashed masses on alternative storage solutions.
MilhouseVH describes it perfectly. I admit, I've probably been too harsh in what I've said in this thread, but when you have someone with such a crazy setup telling everyone else that they're the ones who are limited and not thinking outside the box... it tends to rub some people the wrong way.
@
ned Scott
Dito, + something along the lines of:
My personality is who i am, but my attitude depends on who you are.
@
nickr, @
ned scott, @universal,
At risk of beating a dead horse, please consider this: should I join all the volumes into two pools (one for movie, one for tv) and export said volumes as movies and tv (remember, I have THOUSANDS of titles) where to browse these volumes I would see a single folder with THOUSANDS upon THOUSANDS of entries. Try finding ANYTHING with THAT mess. If however, I label each partition such as "Movie Server #8 - Feature Films Drive #4"? Now, when I need to find a title, I just look up the Movie Information from XBMC (or c22 from movie in MYSQL), look at the PATH variable and viola, I know exactly where it is.
As far as the lack of redundancy, THAT is provided by the original DVDs and BluRay boxes packed ceiling high on a corner...Remember, 3.5 years on, no a single drive lost. At worst, I stand to lose a partition, or an average of 117 titles; which can be rebuild from the secondary copy with handbrake. In the maintime, one single SQL query removes these titles from the library. I remove the damaged drive (or simply don't export that partition) and power the server, everything is back online. On a RAID6 array, what is my average HHD loss rate per annun?
--------------
"...With today’s high capacity disk drives, 4TB today, 6.4TB in 2014, and predicted to be 60TB within the decade , the probability of data loss using RAID6 from a non-recoverable read error during rebuild will soon be a certainty. These large capacity disk drives will proportionally increase RAID rebuild times to multiples of days. According to an Intel study (3), rebuild times between 8.3 hours and 41.5 hours have been measured for 3TB SATA drives, depending on the IO rates being attained from the drives...
...This is the reason all large-scale cloud services today, such as Amazon, Microsoft Azure and others, use either multiple distributed copies (up to 6), erasure coding, or a combination of the two for data protection and durability. Now is the time to abandon obsolete RAID based storage, and eliminate the overhead and vulnerabilities of asynchronous replicated copies...."
(2014-01-22, 14:08)jacintech.fire Wrote: [ -> ]I would see a single folder with THOUSANDS upon THOUSANDS of entries. Try finding ANYTHING with THAT mess. If however, I label each partition such as "Movie Server #8 - Feature Films Drive #4"? Now, when I need to find a title, I just look up the Movie Information from XBMC (or c22 from movie in MYSQL), look at the PATH variable and viola, I know exactly where it is.
Ctrl + f in a file browser, then type in what you want to find. Easier than querying databases, looking at variables or sorting through thousands of titles.
(2014-01-22, 14:22)teeedubb Wrote: [ -> ] (2014-01-22, 14:08)jacintech.fire Wrote: [ -> ]I would see a single folder with THOUSANDS upon THOUSANDS of entries. Try finding ANYTHING with THAT mess. If however, I label each partition such as "Movie Server #8 - Feature Films Drive #4"? Now, when I need to find a title, I just look up the Movie Information from XBMC (or c22 from movie in MYSQL), look at the PATH variable and viola, I know exactly where it is.
Ctrl + f in a file browser, then type in what you want to find. Easier than querying databases, looking at variables or sorting through thousands of titles (what do you think Ctrl + f does?).
You are STILL dealing with THOUSANDS upon THOUDANDS of folders...in addition, each partition contains root level folders a, b, c...z. It's click, click and I am there. Since I am already looking at my library on XBMC, it's just right-click, movie information...and viola...
(2014-01-22, 14:28)jacintech.fire Wrote: [ -> ] (2014-01-22, 14:22)teeedubb Wrote: [ -> ] (2014-01-22, 14:08)jacintech.fire Wrote: [ -> ]I would see a single folder with THOUSANDS upon THOUSANDS of entries. Try finding ANYTHING with THAT mess. If however, I label each partition such as "Movie Server #8 - Feature Films Drive #4"? Now, when I need to find a title, I just look up the Movie Information from XBMC (or c22 from movie in MYSQL), look at the PATH variable and viola, I know exactly where it is.
Ctrl + f in a file browser, then type in what you want to find. Easier than querying databases, looking at variables or sorting through thousands of titles (what do you think Ctrl + f does?).
You are STILL dealing with THOUSANDS upon THOUDANDS of folders...in addition, each partition contains root level folders a, b, c...z. It's click, click and I am there. Since I am already looking at my library on XBMC, it's just right-click, movie information...and viola...
And you're still dealing with thousands of folders, just spread across multiple disks, partitions and root levels. I dunno I'd still rather recover then go through re-encoding media.
@
teeedubb
Ctrl + f stands for control, find. In this case, the local operating system is queying the SMB partition (Exported volume) across the network, it will have to interate through everything until it finds what you are looking for (i. e. spend CPU cycles).
right-click, Movie information on the other hand, queries a MYSQL database (in miliseconds) and returns a result before the window is drawn...
click, open file browser, type movie location takes about a second (Carpal tunnel syndrome, you see)...
You tell me, which is more optimal?...Take your time...
My first post here was....
(2014-01-19, 12:19)uNiversal Wrote: [ -> ]Im not here to join this discussion, Im posting this as a reference for the OP He should read its especially the setup part.
http://www.admin-magazine.com/HPC/Articl...Filesystem
This was my second.
(2014-01-19, 12:33)uNiversal Wrote: [ -> ]Also please stop crossposting http://forum.xbmc.org/showthread.php?tid=183746 Is a exact duplicate of this.
What you want has to be done in Linux and then XBMC will read whatever. XBMC is and app not a OS.
This was your reply to me.
(2014-01-19, 12:35)jacintech.fire Wrote: [ -> ] (2014-01-19, 12:33)uNiversal Wrote: [ -> ]Also please stop crossposting http://forum.xbmc.org/showthread.php?tid=183746 Is a exact duplicate of this.
What you want has to be done in Linux and then XBMC will read whatever. XBMC is and app not a OS.
Dude,
Do you even know what an object store is?...
"...One of the design principles of object storage is to abstract some of the lower layers of storage away from the administrators and applications. Thus, data is exposed and managed as objects instead of files or blocks. Objects contain additional descriptive properties which can be used for better indexing or management..."
This is where you lost any credibility and along with it, my attention and respect.
The use of the word DUDE and the presumptions you make about others is very condescending.
Either implement your discovery into XBMC source code or dont. SIMPLE!
VNC or SSH. Once you've found the location of the files in xbmc what do you do with them?