Guest - Testers are needed for the reworked CDateTime core component. See... https://forum.kodi.tv/showthread.php?tid=378981 (September 29) x
  • 1
  • 7
  • 8
  • 9(current)
  • 10
  • 11
  • 47
Release TMDb TV Show scraper (Python - Default Matrix Scraper)
@Hebotsuki

Thanks for the feedback and the explanation on your fix. I will keep this in mind as another method to fix that issue of the scanner not seeing certain episodes. I would love to know why this happens, but I don't have the shared library setup and none of the users have really looked at the problem in depth once they got it working again.

If you come across a reason, please let us know.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2020-11-20, 00:05)pkscout Wrote:
(2020-11-16, 21:26)Hebotsuki Wrote:
(2020-11-15, 05:34)pkscout Wrote: Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it.
Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.

Anyway, thanks a lot for the work!
Do you have a couple show names I can use for testing where someone has the roll screenplay or storyboard? I think I see how to use the department rather than the role, but without a couple shows with which to test, I won't know if this works.
Sure. The show Violet Evergarden has writer, storyboard and scenario writer as writing credits. And the show After The Rain has screenplay, storyboard and writer. If you need a show not having wirter, tell me and I tried to check for one.

Thank you very much for taking the trouble to look into it.
Reply
@Hebotsuki @Karlson2k There is a new version of the scraper on Github that has the two things you two had asked about.

1- More writer credits.  The scraper now includes anyone in the Writing department as a writer for a show. Tested with Violet Evergarden.
2- Cache files are now stored in special://temp/scrapers/metadata.tvshows.themoviedb.org.python/cache.

One note on #2. On my development machine (running OSX), there is no indication that Kodi does any kind of regular cleanup of the temp directory (i have some files in there from 2014). So the scraper still manages the cache files (it will delete anything over 3 hours old). If another platform handles cleanup differently (or you are handling cleanup through some other process), it shouldn't matter, as the scraper won't try and delete files that aren't there.

Available at:

Leia: https://github.com/xbmc/metadata.tvshows.../tree/leia
Matrix: https://github.com/xbmc/metadata.tvshows...ree/matrix

I consider these both non-critical fixes, so they will be included in the next release of the scraper, but I'm not going to push an update specifically for them right now.
Reply
(2020-11-20, 04:49)pkscout Wrote: @Hebotsuki @Karlson2k There is a new version of the scraper on Github that has the two things you two had asked about.

1- More writer credits.  The scraper now includes anyone in the Writing department as a writer for a show. Tested with Violet Evergarden.
2- Cache files are now stored in special://temp/scrapers/metadata.tvshows.themoviedb.org.python/cache.

One note on #2. On my development machine (running OSX), there is no indication that Kodi does any kind of regular cleanup of the temp directory (i have some files in there from 2014). So the scraper still manages the cache files (it will delete anything over 3 hours old). If another platform handles cleanup differently (or you are handling cleanup through some other process), it shouldn't matter, as the scraper won't try and delete files that aren't there.

Available at:

Leia: https://github.com/xbmc/metadata.tvshows.../tree/leia
Matrix: https://github.com/xbmc/metadata.tvshows...ree/matrix

I consider these both non-critical fixes, so they will be included in the next release of the scraper, but I'm not going to push an update specifically for them right now.

Thanks for the new version.
Kodi doesn't perform clean-up inside special://temp/, but Kodi cleans old files inside special://temp/scrapers/metadata.tvshows.themoviedb.org.python/ according to the scraper setting in addon.xml. I suggest to drop "/cache" from the end of the path as second level directories are not checked.
Reply
(2020-11-20, 19:20)Karlson2k Wrote:
(2020-11-20, 04:49)pkscout Wrote: @Hebotsuki @Karlson2k There is a new version of the scraper on Github that has the two things you two had asked about.

1- More writer credits.  The scraper now includes anyone in the Writing department as a writer for a show. Tested with Violet Evergarden.
2- Cache files are now stored in special://temp/scrapers/metadata.tvshows.themoviedb.org.python/cache.

One note on #2. On my development machine (running OSX), there is no indication that Kodi does any kind of regular cleanup of the temp directory (i have some files in there from 2014). So the scraper still manages the cache files (it will delete anything over 3 hours old). If another platform handles cleanup differently (or you are handling cleanup through some other process), it shouldn't matter, as the scraper won't try and delete files that aren't there.

Available at:

Leia: https://github.com/xbmc/metadata.tvshows.../tree/leia
Matrix: https://github.com/xbmc/metadata.tvshows...ree/matrix

I consider these both non-critical fixes, so they will be included in the next release of the scraper, but I'm not going to push an update specifically for them right now.

Thanks for the new version.
Kodi doesn't perform clean-up inside special://temp/, but Kodi cleans old files inside special://temp/scrapers/metadata.tvshows.themoviedb.org.python/ according to the scraper setting in addon.xml. I suggest to drop "/cache" from the end of the path as second level directories are not checked.
Well, I learn something new every day. I'd never seen that option in an addon.xml before.  And it took me awhile to realize that Kodi doesn't automatically clean that directory, it only does so when the scraper is called (and the file has expired). So I did a bunch of waiting for 15 minutes and then wondering why the files hadn't disappeared. Anyway, there are updated versions for Leia and Matrix at the same place as before that deal with the cache location and expiry properly now. Thanks for pointing these options out.
Reply
@pkscout

Thank you very much for the new version.

I just downloaded the new version from Github and tried scraping the show Ao Haru Ride, but it throws an exception and cannot scrape it. In the debug log I found this error: Error Contents: 'module' object has no attribute 'translatePath'.

I also tested other shows and didn't work either.

By the way, looking into the log I see there are references to the Kodi profile as it was located in AppData of the Windows user profile, but I have installed the Windows Store app. I don't know if this is related to the issue.

Sorry for causing you troubles. I know this update it's not important, so I would understand it's not something you are going to prioritise.
Reply
(2020-11-21, 01:51)Hebotsuki Wrote: looking into the log I see there are references to the Kodi profile as it was located in AppData of the Windows user profile, but I have installed the Windows Store app.
It is a bug... https://github.com/xbmc/xbmc/issues/18260
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2020-11-21, 01:51)Hebotsuki Wrote: @pkscout

Thank you very much for the new version.

I just downloaded the new version from Github and tried scraping the show Ao Haru Ride, but it throws an exception and cannot scrape it. In the debug log I found this error: Error Contents: 'module' object has no attribute 'translatePath'.

I also tested other shows and didn't work either.

By the way, looking into the log I see there are references to the Kodi profile as it was located in AppData of the Windows user profile, but I have installed the Windows Store app. I don't know if this is related to the issue.

Sorry for causing you troubles. I know this update it's not important, so I would understand it's not something you are going to prioritise.
That one is my bad. I merged some code from the Matrix branch and forgot that the translatepath is in a different module for Leia (and didn't test the Leia branch, which was not great of me either).  I will always make time to fix stupid mistakes I make.  I've fixed that, tested in Leia, and pushed an update to the code repo at the same links as before. Thank you for letting me know about the error.
Reply
@Karellen , thank you for letting me know it's a known bug. I was thinking I might have a bad setting.

@pkscout , thank you very much!, it's working now. I did a couple of test and it's getting the writing credits department, including storyboard and screenplay. You're a genius!!!
Reply
Hi.  Is the release group functionality implemented as of yet? 

Just switched over to the new scraper and the intended order for Firefly is not working.  It does work with the old scraper.

Kodi 18.7 on a pi4 with 1.1.26 of the TMDb TV Shows plugin.

If you need logs let me know.

Thanks!
Reply
(2021-01-05, 18:19)fourtysixand2 Wrote: Just switched over to the new scraper and the intended order for Firefly is not working.  It does work with the old scraper.
Yes it works. Explain how you are trying to scrape the alternate order and I will let you know where you are going wrong.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2021-01-05, 18:51)Karellen Wrote:
(2021-01-05, 18:19)fourtysixand2 Wrote: Just switched over to the new scraper and the intended order for Firefly is not working.  It does work with the old scraper.
Yes it works. Explain how you are trying to scrape the alternate order and I will let you know where you are going wrong.
Like I said, the old plugin works.  Everything gets copied using a script to standardize the naming. 

Episodes named as per intended order.

TV\Firefly\Season01\Firefly.S01E01.Serenity.mkv 
TV\Firefly\Season01\Firefly.S01E02.The Train Job.mkv
                             \<etc>
TV\Firefly\tvshow.nfo

tvshow.nfo contains just the link
https://www.themoviedb.org/tv/1437-firef...738a004068
Reply
(2021-01-05, 19:45)fourtysixand2 Wrote: tvshow.nfo contains just the link
https://www.themoviedb.org/tv/1437-firef...738a004068
Just tested it and it works fine here.

Provide a Debug Log which captures you scraping the show.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2021-01-05, 20:16)Karellen Wrote:
(2021-01-05, 19:45)fourtysixand2 Wrote: tvshow.nfo contains just the link
https://www.themoviedb.org/tv/1437-firef...738a004068
Just tested it and it works fine here.

Provide a Debug Log which captures you scraping the show.
Debugging started at 08:24 in the log.

http://paste.ubuntu.com/p/R6WZGQxgyK/
Reply
(2021-01-05, 20:55)fourtysixand2 Wrote:
(2021-01-05, 20:16)Karellen Wrote:
(2021-01-05, 19:45)fourtysixand2 Wrote: tvshow.nfo contains just the link
https://www.themoviedb.org/tv/1437-firef...738a004068
Just tested it and it works fine here.

Provide a Debug Log which captures you scraping the show.
Debugging started at 08:24 in the log.

http://paste.ubuntu.com/p/R6WZGQxgyK/
I think the problem might be that you did a scrape of Firefly with the new scraper before fixing the nfo file.  I see from the log that there is a cached copy of the show being used. You can either wait 3 hours for the cache to expire or manually delete the cache files. The cache file location varies depending on platform, so, honestly, it might be easier to wait.
Reply
  • 1
  • 7
  • 8
  • 9(current)
  • 10
  • 11
  • 47

Logout Mark Read Team Forum Stats Members Help
TMDb TV Show scraper (Python - Default Matrix Scraper)0