• 1
  • 12
  • 13
  • 14(current)
  • 15
  • 16
  • 22
Native Object-Based Storage Support for XBMC
(2014-01-22, 15:36)MilhouseVH Wrote:
(2014-01-22, 15:18)jacintech.fire Wrote: ...and enlighten me, how EXACTLY is this crazy? Please list a couple of nightmare scenarios that await me in some distant future...Take your time, I will wait...

You're not managing your resources effectively, and you're adding masses of overhead for no benefit. You're in denial that disks fail, as you are about database corruption. As you've come to realise, a bit late I might add, accessing your media scattered across 1024 independent partitions is a major headache. Your lack of redundancy (and your attitude towards it) is frankly breathtaking (no doubt born out of naivety).

There are a multitude of volume management options available to you that would have meant you didn't need to create this monumental embarrassment, and could have avoided all the problems you have created for yourself. Nobody in their right mind, with any technical competency or common sense, would have created such a system as you now have.


Many of us work in environments with more storage than yours, and with more files than yours, and so can probably speak with some authority when they say your setup is a complete joke (a bad one).

Ok. Give an example in which this setup (as it applies to XBMC) is a nightmare...

(2014-01-22, 15:41)MilhouseVH Wrote:
(2014-01-22, 15:27)jacintech.fire Wrote: Con you please describe with examples as it relates to XBMC, the "hell" or "nightmare" you believe I am experiencing...

It seems that you've created your setup just so you don't have to search for a file. I mean, really? Or is that some sort of backward justification for creating such a horror show?

It's difficult for us to describe what you might consider to be "hell" when you seem to be appalled by the concept of searching for a file.

Again, give ONE example in which this setup, as it applies to XBMC is a nightmare...
Reply
(2014-01-22, 15:42)jacintech.fire Wrote: Ok. Give an example in which this setup (as it applies to XBMC) is a nightmare...

In your own words:
(2014-01-18, 18:45)jacintech.fire Wrote: As you can imagine, it has become increasingly resource intensive to manage such a large setup (considering that each 4TB HDD is further partitions into 4x 1TB partitions; and each partition represent a difference media source, the sources.xml file is rather large).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Could you please explain the merits of an object store compared to say an Unraid server?
Reply
(2014-01-22, 15:27)jacintech.fire Wrote:
(2014-01-22, 15:22)T800 Wrote:
(2014-01-22, 14:44)jacintech.fire Wrote: @teeedubb
Ctrl + f stands for control, find. In this case, the local operating system is queying the SMB partition (Exported volume) across the network, it will have to interate through everything until it finds what you are looking for (i. e. spend CPU cycles).
right-click, Movie information on the other hand, queries a MYSQL database (in miliseconds) and returns a result before the window is drawn...
click, open file browser, type movie location takes about a second (Carpal tunnel syndrome, you see)...
You tell me, which is more optimal?...Take your time...


I have 3000 movies on one pooled volume of 13 disks. If I include all attached data it's about 30000 files.
On a networked computer or a file browser on my phone via SMB I can type a search in and have the result right in front of me within 5 seconds ready to manipulate in anyway I like.

Optimal..I don't know but it works well and don't have any of the nightmare seem to have and XBMC luuurrves it!

(2014-01-22, 15:18)jacintech.fire Wrote: @uNiversal,
"...Benzodiazepines are effective in reducing stress that may provoke hyperventilation syndrome (HVS) and are thought to reset the central nervous system (CNS) response to a variety of “panicogens.” Selective serotonin reuptake inhibitors (SSRIs) have been reported to reduce the frequency and the severity of episodes of hyperventilation..."


What's wrong with a text editor. I once wrote an AES implementation on textpad :-)

...and enlighten me, how EXACTLY is this crazy? Please list a couple of nightmare scenarios that await me in some distant future...Take your time, I will wait...

What we call hell, you seem to call home (not a compliment). Your nightmare is with you now imo.
Con you please describe with examples as it relates to XBMC, the "hell" or "nightmare" you believe I am experiencing...

I'm gonna go with 'sources.xml'
Reply
@TR800,
Hahahahahahahaha!!!
sources.xml? That's ALL you've got?
Hahahahahahahahaha!!!

@Harro,
An Object Store model w/ Aggregated (distributed) storage is infinitely scalable. 10GB, 10TB, 10PB is all the same.

Redundancy and failover is built in.

Application-defined, or User-defined metadata negates the need for a central database

It's just AWESOME not to worry about files, or directories, or volumes...
Reply
Yes can you explain how how you came to this conclusion while experimenting with your server? More precisely all the steps and what happened that day when you had that eureka moment!
Reply
(2014-01-22, 15:44)MilhouseVH Wrote:
(2014-01-22, 15:42)jacintech.fire Wrote: Ok. Give an example in which this setup (as it applies to XBMC) is a nightmare...

In your own words:
(2014-01-18, 18:45)jacintech.fire Wrote: As you can imagine, it has become increasingly resource intensive to manage such a large setup [This is one thought onto itself] as in the number of individual media files. Not a function of storage

(considering that each 4TB HDD is further partitions into 4x 1TB partitions; and each partition represent a difference media source, the sources.xml file is rather large) [this is an aside, ad in it is large because...]

See above...
Reply
Considering XBMC applies a metadata layer over your filesystem, essentially becoming a poor mans object store, from the narrow perspective of XBMC what else is likely to be the problem? Even in your own words the sources.xml problem has been sufficient for you to begin investigating an object store (which you seem to have only just learned about) and hence this entire thread.

From a more general perspective, your system is very poorly designed, resource intensive, inefficient and with no redundancy (which you either don't appreciate, or understand, or you have far too much time on your hands). XBMC doesn't care about any of that, as it does it's best to isolate your from what you have created and it's a miracle it works at all.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Ok I decided to make headway and started coding the object store API support into XBMC vfs
Reply
(2014-01-22, 15:37)uNiversal Wrote: https://jacintech.fire.com

Domain Name: TROLOLOLOLOLOLOLOLOLOLO.COM
Registry Domain ID: 1586850471_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.publicdomainregistry.com
Registrar URL: www.publicdomainregistry.com
Updated Date: 01-Oct-2011
Creation Date: 26-Feb-2010
Registrar Registration Expiration Date: 26-Feb-2020
Registrar: PDR Ltd. d/b/a PublicDomainRegistry.com
Registrar IANA ID: 303
Registrar Abuse Contact Email: 
Registrar Abuse Contact Phone: +1-2013775952
Domain Status: clientTransferProhibited
Registry Registrant ID: DI_15267424
Registrant Name: c/o RespectMyPrivacy, LLC
Registrant Organization: trololololololololololo.com Domain Administrator
Registrant Street: PO BOX 484   
Registrant City: COCOA
Registrant State/Province: FL
Registrant Postal Code: 32923-0484
Registrant Country: US
Registrant Phone: +1.8664570078
Registrant Phone Ext: 
Registrant Fax: +1.8663909061
Registrant Fax Ext: 
Registrant Email: 
Registry Admin ID: DI_15267424
Admin Name: c/o RespectMyPrivacy, LLC
Admin Organization: trololololololololololo.com Domain Administrator
Admin Street: PO BOX 484  
Admin City: COCOA
Admin State/Province: FL
Admin Postal Code: 32923-0484
Admin Country: US
Admin Phone: +1.8664570078
Admin Phone Ext: 
Admin Fax: +1.8663909061
Admin Fax Ext: 
Admin Email: 
Registry Tech ID: DI_15267424
Tech Name: c/o RespectMyPrivacy, LLC
Tech Organization: trololololololololololo.com Domain Administrator
Tech Street: PO BOX 484  
Tech City: COCOA
Tech State/Province: FL
Tech Postal Code: 32923-0484
Tech Country: US
Tech Phone: +1.8664570078
Tech Phone Ext: 
Tech Fax: +1.8663909061
Tech Fax Ext: 
Tech Email: 
Name Server: ns1.dreamhost.com
Name Server: ns2.dreamhost.com
Name Server: ns3.dreamhost.com

DNSSEC:Unsigned

Registration Service Provided By: NEARLYFREESPEECH.NET

URL of the ICANN WHOIS Data Problem Reporting System:http://wdprs.internic.net/

What's your point...?
Reply
(2014-01-22, 15:46)Harro Wrote: Could you please explain the merits of an object store compared to say an Unraid server?

http://www.ddn.com/pdfs/BeginnersGuideTo...epaper.pdf

For the average user, I'm not sure there are noticable differences; but then I'm short-sighted when it comes to being visionary with object storage. I would say, should the feature exist, I'd configure my storage as objects Smile But then, I'm that sorta person.
HTPC RPI3 Kodi 17 (Krypton) v8.0.1 MR
Storage BPI 1x 500GB SSD UPnP server
Display Sony Bravia 32"
Reply
Just for the sake of discussion and I am not familiar with object store or how it is set up, but from this thread have gotten a crude idea of what it may do.

I am not educated in object store as a storage means or system and I think everyone has their own personal way to handle their storage, but if XBMC did add.objs or whatever the object store file system extension is along with SMB,NFS etc.. I could see some interesting possibilities from it. Not saying XBMC has to run an object store just recognize it, with the store running on your personal machine as my unraid does, XBMC would access the store and pull all the info from it. (add source) The store would have all the metadata, locations of files or nodes (whatever your media is), machine specs and other info you have set up on your personal store.

So with that said and my minuscule knowledge of how anything works, someone could set up a cloud based store, which would act as the master server, holding all the data (no physical media just info) from your store. Now I set up my unraid server to be and object store named "Harro", I install XBMC on my laptop and add source "Harro, XBMC scans my object source and adds all my media into XBMC along all the metadata, nothing different at this point than the way XBMC does things now, except I now send the same info I just scanned into XBMC to the object store on the cloud based server as I will call the "Master". Tongue Master is now holding all the locations for my info and what machine I am running XBMC on along with my settings and other items pertaining to how XBMC is set up on my machine. As Master grows from others that have and object store running on their machines, gathering all the same type of info, it will constantly check for new or updated info on all the items in it's store, passing the updated info down to my store "Harro". No need to update ratings, or fanart, actors, etc.. , since my store is cross checking with "Master". As "Master" grows through the input of other personal stores, I could see XBMC becoming an app for all platforms without any user configuration besides the set up for a user name on the "Master" for their own store. So you down load the XBMC app, prompt asks you for a username, then where your media is located, then what skin you would like, with this info inputted, XBMC app then looks to "Master". Master sees that XBMC app has been downloaded to a NUC. Master sees all the NUC configs from it's store, along with the skin selected, along with all the media the new user has. Sends all the info on the input screen back to XBMC app and XBMC app is up and running in a matter of a few minutes. Master now has created a store for said user and will update said store as needed.

As XBMC is now and with you being so far the only person requesting an object store for XBMC, I see it as a null point. But can see how it could work.

I can go on but I may have had too much coffee this morning. Rolleyes
Reply
(2014-01-22, 15:57)MilhouseVH Wrote: Considering XBMC applies a metadata layer over your filesystem, essentially becoming a poor mans object store, from the narrow perspective of XBMC what else is likely to be the problem? Even in your own words the sources.xml problem has been sufficient for you to begin investigating an object store (which you seem to have only just learned about) and hence this entire thread.

From a more general perspective, your system is very poorly designed, resource intensive, inefficient and with no redundancy (which you either don't appreciate, or understand, or you have far too much time on your hands). XBMC doesn't care about any of that, as it does it's best to isolate your from what you have created and it's a miracle it works at all.

Again, give one example which would illustrate the above comment and result in a "nightmare scenario"...

The reason for investigating an Object Store Approachnwas the large numbernof individual media files I need to keep track of (whether RAID or JBOD).
The fact that as it scales up (RAID) becomes unfeasible.
The fact that an Object Store model has built in redundancy, it scales infinitely and provides a mechanism to simplify and unify both content and metadata into a single container..

Is my setup resource intensive? How
Am I not using my available storage its maximum potential?
Are my network resources not used uptimally?
is my setup consuming more CPU Cycles than otherwise?
Am I using more RAM?
Is the lack of RAID6 compromised other, more simple forms of redundancy?
Is 3.5 years with Zero loss statistically inferior to the RAID6 HHD loss average per annun?

@Harro,
FINALY!!!!
Now that that thought and run it to its ultimate conclusion...and wait for your holy $#!7 moment :-)
Reply
Conceptually, would it be possible to create a 'layer' for Ceph Object Gateway Swift API to work with XBMC JSON-RPC API XBMC? I mean, is that what's required?

n.b. I'm not an expert with either Ceph or XBMC! Happy to be educated though.
HTPC RPI3 Kodi 17 (Krypton) v8.0.1 MR
Storage BPI 1x 500GB SSD UPnP server
Display Sony Bravia 32"
Reply
[quote='jacintech.fire' pid='1607427' dateline='1390398577']
@TR800,
Hahahahahahahaha!!!
sources.xml? That's ALL you've got?
Hahahahahahahahaha!!!



Whats with the 'R' you keep adding to my name?

Having had 8 movie sources and then going to one I can tell you with absolute certainty that one is way easier. That plus no good reason for so many partitions makes your life harder than mine is.

I also have some redundancy, you may not see the need for it but you could have it anyway at pretty much zero cost to you.
Reply
  • 1
  • 12
  • 13
  • 14(current)
  • 15
  • 16
  • 22

Logout Mark Read Team Forum Stats Members Help
Native Object-Based Storage Support for XBMC5