Kodi Community Forum

Full Version: Frodo - Shared Library Thumbnail Updates & Mulitple PCs
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3
Hi All,

I have a number of PCs in my house running XBMC 11 and am looking to upgrade to 12 but I am concerned with the new library update changes, the removal of the path substitution and how it will different PCs updating the library and thumbnails.

Currently the PCs are turned off when not in use but they all update the library when XBMC first loads. TV/Movies etc. are on the NAS, as are all the thumbnails (using path substitution on each XBMC instance). This means everything is in sync because of the central database and central thumbnail storage.

The upgrade instructions for Frodo (http://wiki.xbmc.org/index.php?title=HOW...BMC_to_v12) explain how to remove the path substitution and copy everything locally and then it will keep it in sync itself but it doesn't explain how this actually works and I don't see how it can.

What if all the computers aren't turned on or the computer that did the update is then turned off when one of the others is turned on? The library will be up to date but the thumbnails missing I am guessing until that PC is turned back on at the same time as one of the others.

A good example is our home theatre which is used rarely compared to the living room. If I start XBMC and it updates the library and the computer is turned off for a few weeks then none of the other instances will "sync" the thumbnails stored locally on that PC only.

It appears to me that I'd need to somehow run a headless XBMC instance that is updating the library all the time on some type of server to have a centrally fully up to date database and then not do any updates on the actual clients.

Am I misunderstanding something with this new implementation?
Can path substitutions still work in Frodo (exactly the same as Eden) to get around this problem?

Regards,
Andrew
If I'm understanding your question correctly, it sounds like you are using MySQL for the centralized database. In Frodo, the database logs the location of the originating artwork, so if one device updates the library, when you turn on another and go to a view that shows the new media without art, XBMC goes right to the path set in the database and collects it. This is quite different from previous releases where if you updated the library on one machine you had to manually sync the others or select the movie/show and click refresh/update or whatever it was. On a side note, if you change the art on one machine with either online or local art, the new art will be collected from the original source (NAS for you, since that is where you store it) and updated automagically on the other devices.

I'm a low post newbie, so somebody please correct me if I am incorrect or if it isn't working for everybody.
(2013-01-31, 04:24)Nitroburner77 Wrote: [ -> ]If I'm understanding your question correctly, it sounds like you are using MySQL for the centralized database.

Yes I am using mySQL, apologies I should have mentioned this in the initial post.

(2013-01-31, 04:24)Nitroburner77 Wrote: [ -> ]In Frodo, the database logs the location of the originating artwork, so if one device updates the library, when you turn on another and go to a view that shows the new media without art, XBMC goes right to the path set in the database and collects it.

This is my big issue and question. The machine that performed the update has the thumbnail/banner/fanart locally for the newly updated library entries may not be turned on so there is no way any of the other xbmc instances can access the files so they won't be able to copy them locally. This doesn't happen currently because the artwork is stored and accessed on the NAS which is always turned on.

(2013-01-31, 04:24)Nitroburner77 Wrote: [ -> ]This is quite different from previous releases where if you updated the library on one machine you had to manually sync the others or select the movie/show and click refresh/update or whatever it was. On a side note, if you change the art on one machine with either online or local art, the new art will be collected from the original source (NAS for you, since that is where you store it) and updated automagically on the other devices.

No you didn't, there was no manual syncing at all. With path substitution and mySQL every instance looked at the same database and artwork location so it was always up to date. The library update on startup which I run on all the XBMC instances would scan the NAS for new shows or movies and update the library.

I can install XBMC on a new PC (or Mac), update the advancedsettings.xml with the mySQL and path substitutions and it is instantly up to date with the central library and artwork, watched flags etc.

Updating the art (say a new movie cover) works the same, update it on one PC then its updated on the others since they are all looking at the same thing.

I understand that the new system would be faster as everything is access locally (not that I have any issues with the current system) but I can't see how any of my eight XBMC instances would ever have a complete copy of the artwork.







Quote:The machine that performed the update has the thumbnail/banner/fanart locally
.

No, it has a local cache of the image which is not shared at all. This is completely separate from the shared database. The original URL of the image is kept in the shared database. See the art table.

Your other clients have the original URL from the shared database, and create their own local cache of the image automatically.
(2013-01-31, 05:15)jmarshall Wrote: [ -> ]
Quote:The machine that performed the update has the thumbnail/banner/fanart locally
.

No, it has a local cache of the image which is not shared at all. This is completely separate from the shared database. The original URL of the image is kept in the shared database. See the art table.

Your other clients have the original URL from the shared database, and create their own local cache of the image automatically.

Ahh, now that makes perfect sense. I didn't think of looking at the new database table schema. Thanks for the clarification.

Not that it really matters but it is worth pointing out that with it working like this each XBMC client will need to download all the missing artwork each time from the internet (using the URL stored in the database) so if you have 10 clients then you'll be downloading all the artwork 10 times. Am I understanding that correctly? This will also mean that if you add a new instance in say 5 years some of the URLs may no longer work, what happens in this case?

If my understanding is correct an additional feature could be added to have a local central library that all machines use to update their local cache to avoid the internet downloading. The URL stored in the database could be the image location on the NAS.

Appreciate your time.
I thought about this a little more. If you wanted to create a new XBMC instance then you'd just copy the cache off an existing machine.

It's quite possible that the URL stored though may no longer exist if a machine wasn't turned on for a period of time so what would you do in this instance? We have a couple of machines that may not be turned on for 6 months.
You are really overthinking this in my opinion. Artwork URLs should exist for over 6 months, at least from the sources xbmc is scraping from. Ive been using the new cache system almost as soon as it was available in a nightly build. My raspberry pi (secondary xbmc in my house) is constantly being wiped as a try new things and i have yet to see miss any artwork when rebuilding the library.

As a side note. If you export your library, it will also export the artwork. If you ever need to reimport or find specific artwork you can get it locally then. If you are super paranoid just do an export every few months.

The new cache system really works great in my opinion.
Thanks for your thoughts. I am possibly over thinking it, the XBMC team have always had well thought out solutions so I doubt this time is any different.

Having 9 machines to update with a few different operating systems I wanted to understand how the new system works before fully I started the upgrade process.

Thanks to everyone who contributed to this thread.

Thanks XBMC team for a wonderful piece of software.
Excuse me if I jump into this topic, but I am really worried about what I am reading here:

(2013-01-31, 06:20)robweber Wrote: [ -> ]Artwork URLs should exist for over 6 months, at least from the sources xbmc is scraping from.

But what if the artwork was never scraped by XBMC, therefore there are no URLs?
I usually retrieve the artwork by external tools (ember or MediaElch), some of it by scraping the well known medie sites as XBMC would do, but some others by using pictures I downloaded myseld and simply putting
them in the movie directories as poster.jpg or fanart.jpg or whatever.

These files are used by the Transparency-Skin of my XBMC without any problems, there is no way any of this images is connected to a valid URL.
The Thumbnail-Cache is currently synced between 3 different XBMC-Installations by a robocopy script.

So my question: If I add an new fanart.jpg into a movie directory on machine X (just by storing the file in the movie directory, not scraping it from a URL), will it ever be visible on machine Y? As machine Y can not scrape it itself from a URL in the internet, were will it get the image from?
(2013-01-31, 08:54)grumbler Wrote: [ -> ]So my question: If I add an new fanart.jpg into a movie directory on machine X (just by storing the file in the movie directory, not scraping it from a URL), will it ever be visible on machine Y? As machine Y can not scrape it itself from a URL in the internet, were will it get the image from?
If you have the fanart on a movie directory on machine X and machine X is turned off, you should be worrying about where machine Y will get the movie itself from.

(2013-01-31, 08:54)grumbler Wrote: [ -> ]So my question: If I add an new fanart.jpg into a movie directory on machine X (just by storing the file in the movie directory, not scraping it from a URL), will it ever be visible on machine Y? As machine Y can not scrape it itself from a URL in the internet, were will it get the image from?

The image path does not have to be an http URL. As an example, when I decided to move from SQLite to the MySQL database structure I did an export of my library, which exported all artwork into the movie/tv show folders along with NFO files. Once I had MySQL working I did a rescrape of the library to get the contents into MySQL. The URLs to the image paths were now nfs://ip/path/folder/movie/whatever instead of http://website/imagepath. Looking in the artwork table of the database all the image paths just point to local network images. As I added new media some of these paths became web URLs, others local paths as I added custom artwork.

The media location doesn't matter so much as long as XBMC can a) read the path from the DB and b) the path exists. As Kibje mentioned, if machine X is turned off and that is where you movies/artwork are; you'll have quite a few problems.
I am using MySQL with 1 TV Server, 1 Movie Server and 1 Music Server. I also have 4 XBMC Clients. I have all my posters, fanart and thumbnails in the source folders. For anyone using a central database, i believe this is the best way to go about it.

I upgraded to Frodo yesterday but decided NOT to go the extra safe way like explained in the upgrade FAQ. At first, i didn't want to have to copy my 12 gig thumbnail folder to the XBMC clients that have small SSDs and especially not to my Raspberry PI! I figured that over the years, some of those thumbnails were obsolete.

When XBMC started the Video Art Upgrade, i saw this would be a lengthy process so it definitely stuck to my original idea of just letting all the clients generate their own art.

This worked just fine. While the main DB was being upgraded, i fired up the clients and started navigating and the artwork slowly got generated.

For those of you with large libraries like myself (4250+ Movies, 785+ TV Shows, 30000+ MP3s), here is a list of how long different steps of the conversion took (all on the main server) :

Updating the database Textures13 from version 6 to 13 : 1 Minute
Updating the database xbmc_music32 from version 18 to 32 : 45 Minutes
Updating the database xbmc_video75 from version 60 to 75 : 49 Minutes

Updating the Movies & TV Shows Art : 4 Hours 56 Minutes
Updating the Actors Art : 5 Hours 41 Minutes
Updating the Music Art : Started at 10:19am and it is now 11:35am and it is 40% done.

This is on an i5-750 with 8 GB of RAM.

-Pr.
PR i feel your pain, i really dont want to do local images since my XBMC client are using USB drives or small disks. I really wanted to keep all my images on a larger SSD that i have on my machine that holds my DB and does the auto update since its always on. Ive been tinkering with using the path substitution but ran into some issues since the machine in question is running LION.

I need to upgrade to full FRODO this weekend so will do some other testing with path substitution on my nas but directly to another SSD drive to see how that pans out.
Pr. Sinister: No way in hell it should take that long. The textures update for instance is essentially: "drop everything and change a couple tables" so anything more than a few seconds is way too long. Same for the others.

The music scan could well be about right if you have a really, really large collection - essentially it's a complete (local) rescan of your music. I've only got a small collection (some 5k songs) and it takes about 5-10 minutes (over wifi).
From the previous discussion my assumption would be that the URL stored would be the URL of the computer not and internet URL so if X is always on then XBMC on Y could get the file from the original location.

If it's not on then as Kibje says it won't be able to get the fanart.jpg but it also couldn't play the movie.

I still think a local central cache on a NAS (where the DB stores the NAS location rather than the internet location) would be a useful addition that would save downloading the same thing for every XBMC instance. I'd be interested to know if this was considered with the new design.


I think the idea of storing everything locally is a great idea. My systems are not slow with the current Eden implementation but locally stored images/fanart are definitely faster though I do see the problem if the clients have small disks.
Pages: 1 2 3