v19 SQL and scraping
#1
Hopefully I'm in the right section to ask this!

So after I ballsed up my sql database and rolled it back and found it was still messed up I took the opportunity to rebuild from scratch. Not ideal but needed.
  • v19 installed
  • avancedsettings.xml pointed to the right IP address
  • Sources.xml pointed to the SMB drive
  • Every single nfo/tbn removed from the unraid box so not to confuse the scraper
  • Everything has the right naming convention
  • //TV/Show/SeasonX/SXXEXX.*
  • JPGs left in place
  • Most things scanned via TMDB (historically they all went via TVDB due to the age of most things on there)
  • A handful of shows didn't scan properly.
Either:
  1. at all and we're talking very popular shows like GOT and I could see an empty top level folder
  2. some folders just ignored
  3. partially scanned and stopped a few seasons ago randomly mid season
for 2 and 3 I removed everything but the avi/mkv files and they're scanning correctly now but that seems a really weird quirk
What I can't work out is where all the jpgs/nfos etc now live. I would have assumed they'd all be dumped to the unraid server like before but there is nothing there.
I checked the size of the database and it's 85Mb so they're not there either...

So when I go add another box is it going to have to pull every single image down again from the net?
Reply
#2
(2021-03-12, 18:59)Rodimus Wrote: at all and we're talking very popular shows like GOT and I could see an empty top level folder
https://github.com/xbmc/xbmc/issues/15768

(2021-03-12, 18:59)Rodimus Wrote: What I can't work out is where all the jpgs/nfos etc now live
In the Kodi cache.
If you want to download local artwork, (which Kodi has never done without an add-on), then you need to install Artwork Dump... https://forum.kodi.tv/showthread.php?tid=360078
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#3
(2021-03-12, 20:10)Karellen Wrote:
(2021-03-12, 18:59)Rodimus Wrote: at all and we're talking very popular shows like GOT and I could see an empty top level folder
https://github.com/xbmc/xbmc/issues/15768
(2021-03-12, 18:59)Rodimus Wrote: What I can't work out is where all the jpgs/nfos etc now live
In the Kodi cache.
If you want to download local artwork, (which Kodi has never done without an add-on), then you need to install Artwork Dump... https://forum.kodi.tv/showthread.php?tid=360078
Thank you Karellen 

Ah right, time to digest the bug.
Interestingly enough I'm using the new tvdb tv series scraper and it's happening though!
Thank you!

Re: the local storage, even things from a week ago pulled down a jpg when kodi scraped it. I didn't have anything extra installed that I'm aware of.
Base kodi 18.7, aeon nox (I think was the skin) and that was it.
Reply
#4
(2021-03-12, 21:01)Rodimus Wrote: Interestingly enough I'm using the new tvdb tv series scraper and it's happening though!
Yea, I don't think that scraper was ever coded to limit the artwork links. It was created before the problem was discovered, and as that scraper is now unmaintained and soon to be marked broken, it hasn't received any updates for a while.

(2021-03-12, 21:01)Rodimus Wrote: Re: the local storage, even things from a week ago pulled down a jpg when kodi scraped it. I didn't have anything extra installed that I'm aware of.
Install Atwork Dump, and you will have artwork downloaded automatically. It's not in the repo yet, so install via zip method.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#5
(2021-03-12, 21:35)Karellen Wrote:
(2021-03-12, 21:01)Rodimus Wrote: Interestingly enough I'm using the new tvdb tv series scraper and it's happening though!
Yea, I don't think that scraper was ever coded to limit the artwork links. It was created before the problem was discovered, and as that scraper is now unmaintained and soon to be marked broken, it hasn't received any updates for a while.
(2021-03-12, 21:01)Rodimus Wrote: Re: the local storage, even things from a week ago pulled down a jpg when kodi scraped it. I didn't have anything extra installed that I'm aware of.
Install Atwork Dump, and you will have artwork downloaded automatically. It's not in the repo yet, so install via zip method.
Ah i thought it was the right scraper to be using! 
Which one is the right one then? The default TMDB or the one marked new?

I'll get artwork dump going over night! Thanks!
Reply
#6
For now it is the right one to be using, at least if you want data from TVDB.

But soon they are revising their API to a new version, and after that the old API will be phased out (and when it does happen the scraper will stop working).

They are currently working on writing a new scraper for the new API, but it is not complete yet. They won't do anything though until that scraper is available and tested (or at least so they tell us).
|Banned add-ons (wiki)|Forum rules (wiki)|VPN policy (wiki)|First time user (wiki)|FAQs (wiki) Troubleshooting (wiki)|Add-ons (wiki)|Free content (wiki)|Debug Log (wiki)|

Kodi Blog Posts
Reply

Logout Mark Read Team Forum Stats Members Help
SQL and scraping0