[pkscout]

Just as an aside, it might also be safer to sort the episodes list by the 'order' key before sequentially numbering them. While I haven't seen any cases where the API returns episodes in a different order (and unlike with the season groups, you can't explicitly set or edit the order #) I'm not sure it's safe to assume the API will actually return the episode list in a particular order.

Good point. I hadn't really noticed that order field in the API return until you pointed it out. I've made that update, and it'll be included in the next release.

[Karlson2k]

@pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"?
I have ".kodi/temp" mounted on RAM disk to avoid SSD trashing and speedup data processing and now it's required to use some tricks to move this scraper temp files to RAM drive.

I'm not familiar with the temp location.  I use a python call to get the addon_data location and then store the addon's data there. If there is some other Python call that will return the path to that temp directory on every platform, I'd be happy to change it. I'd rather not start doing platform specific code though.  As an aside, I'm using the same location the TV Maze scraper uses to store it's cache files.

I'm not an addon API expert, but by quick checking of kodi source code, it seems that simplest way to use temp path is something like os.path.join('special://temp/scrapers', ADDON.getAddonInfo('id')). This path is automatically created before run of scraper and cleared from outdated items (no need to check for old files in your code). Expire time is defined in addon.xml in <addon> <extension cachepersistence="00:15"/></addon>.

[pkscout]

@pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"?
I have ".kodi/temp" mounted on RAM disk to avoid SSD trashing and speedup data processing and now it's required to use some tricks to move this scraper temp files to RAM drive.

I'm not familiar with the temp location.  I use a python call to get the addon_data location and then store the addon's data there. If there is some other Python call that will return the path to that temp directory on every platform, I'd be happy to change it. I'd rather not start doing platform specific code though.  As an aside, I'm using the same location the TV Maze scraper uses to store it's cache files.

I'm not an addon API expert, but by quick checking of kodi source code, it seems that simplest way to use temp patch is something like os.path.join('special://temp/scrapers', ADDON.getAddonInfo('id')). This path is automatically created before run of scraper and cleared from outdated items (no need to check for old files in your code). Expire time is defined in addon.xml in <addon> <extension cachepersistence="00:15"/></addon>.

Thanks. I'll definitely take a look at that.

[Hebotsuki]

Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it.

Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.

Anyway, thanks a lot for the work!

Writer is the only type of crew that is added in Kodi. That's to keep it consistent with the movie scraper. I can look at adding the other ones, but it's a bit of a whack-a-mole problem. Anytime another kind of writer is added, I'd have to update the scraper.  I'll take another look though. There might be a way to do it using "department" instead of roles.

Much appreciated. If there were a way to get a "department" writing credits would be great. I don't know how scrapers work, but the XML version of the scraper gets these credits I mentioned. I'm interested in this because many anime shows use this kind of credits and and even many have no writer role.

[montezuma]

Sorry.
The problem is, that no episode from this TV Show could be scraped now, although it worked ~2 weeks ago.
In the logs I can see an empty result. Is it moviedb issue? Or maybe the query is wrong?

Ok, and yes I can confirm this problem also. I suspect the issue is due to none of the episodes having an overview.

With this scraper we are still trying to catchup to the changes that have occurred over at TheMovieDB, and while for the majority of users it works, it is these edge cases which involve different languages that are still causing problems.

Please change to the new Python scraper, as all the issues you reported do not exist and it works correctly... https://kodi.wiki/view/Add-on:TMDb_TV_Shows

Thanks. I have installed the Python scraper, but it also fails with:
https://www.themoviedb.org/tv/47877-5-dn...ia-emeryta

Here are the logs:
https://paste.kodi.tv/igijuqovup.kodi

0-11-18 21:32:43.369 T:1806672768   DEBUG: [metadata.tvshows.themoviedb.org.python (1.1.18)]: Calling URL "https://api.themoviedb.org/3/search/tv?query=s01e01+Smierc+pana+Kaliny&api_key=af3a53eb387d57fc935e9128468b1899&language=pl-PL"

2020-11-18 21:32:43.584 T:1806672768   DEBUG: [metadata.tvshows.themoviedb.org.python (1.1.18)]: the api response:
                                            {u'page': 1, u'results': [], u'total_pages': 0, u'total_results': 0}

[Karellen]

@montezuma
I have moved you to the correct thread.

Thanks. I have installed the Python scraper, but it also fails with:
https://www.themoviedb.org/tv/47877-5-dn...ia-emeryta

It works fine here for me.
What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source...

/media/HDD6/Seriale/5 dni z zycia emeryta [1985] (PL)/s01e01 Smierc pana Kaliny.mp4

xml:
: Searching for TV show s01e01 Smierc pana Kaliny (None)

[montezuma]

@montezuma
I have moved you to the correct thread.

Thanks. I have installed the Python scraper, but it also fails with:
https://www.themoviedb.org/tv/47877-5-dn...ia-emeryta

It works fine here for me.
What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source...

/media/HDD6/Seriale/5 dni z zycia emeryta [1985] (PL)/s01e01 Smierc pana Kaliny.mp4

xml:
: Searching for TV show s01e01 Smierc pana Kaliny (None)

> What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source...

"5 dni z zycia emeryta [1985] (PL)"

What is actually wrong with the folder name (as I said it worked ~2 weeks ago) ?

https://kodi.wiki/view/Naming_video_files/TV_shows

Naming TV Shows follows the same guidelines as for Movies.
Name the TV Show folder the same way as shown on the TV Show page at the scraper site.
It is always best practice to include the year along with the title. This makes scraping more accurate and differentiates between original and remake versions.
This guideline also applies to Country of Origin TV Shows. For example Hells Kitchen (UK), Hells Kitchen (US), Hells Kitchen (AU), Hells Kitchen (IT) etc.
The title and (year) displayed at the scraper site is how you should name your show.

Anyway I use the "mixed" approach and provide a URL for every TV show in the tvshow.nfo file (to avoid disambiguity):

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<tvshow>
    <title>5 dni z życia emeryta (PL)</title>
    <tag>PL</tag>
</tvshow>
https://www.themoviedb.org/tv/47877-5-dn...ia-emeryta

And btw. I could scrap today the TV show with the XML scraper 3.5.8

[montezuma]

And btw. I could scrap today the TV show with the XML scraper 3.5.8

I also tried the Python scraper and it works, too (previously  I selected the folder with the TV show for scraping (my mistake) and not a folder containing the folder with the TV show).

However from 32 TV shows I have on the disk, I could only scrape 22 with the Python scraper, and the XML scraper 3.5.8 managed all 32.

[Karellen]

> What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source...

"5 dni z zycia emeryta [1985] (PL)"

What is actually wrong with the folder name (as I said it worked ~2 weeks ago) ?

There is nothing wrong with the folder name, but you have set the source on the wrong location. The Source should be set on /Seriale/. But it sounds like you figured that out.

However from 32 TV shows I have on the disk, I could only scrape 22 with the Python scraper, and the XML scraper 3.5.8 managed all 32.

Without any details, there is nothing I can do about that.

[Hebotsuki]

@Hebotsuki

Can you please switch to the new Python scraper and see if that solves your issue... https://kodi.wiki/view/Add-on:TMDb_TV_Shows

@Karellen 

I tried the Python scraper as asked:

• Ao Haru Ride: It works - The guest actors show up in each episode view.
  
• Violet Evergarden: It doesn't work - Same result than with the XML scraper. It only gets the special episode, but without title. Debug log in case it's useful.
  

[Karellen]

I tried the Python scraper as asked:

• Ao Haru Ride: It works - The guest actors show up in each episode view.
  
  
• Violet Evergarden: It doesn't work - Same result than with the XML scraper. It only gets the special episode, but without title. Debug log in case it's useful.
  

That is not a scraper issue. That is MySQL being difficult again.

I have seen this problem a few time already, but never been able to figure out the cause. The Kodi/MySQL combination has a little glitch and the episode files are not seen or scanned, but if you were to switch over to local SQL, it would work perfectly.

Your show is here... smb://192.168.1.100/Animoteca/TV/Violet Evergarden (2018)/

You just need to change the path so Kodi reads it as a new tv show and starts the scan fresh. Maybe smb://192.168.1.100/Animoteca/TV/Violet Evergarden Special (2018)/ which is a valid alternate title at TMDB.

Let us know if that works.

[Hebotsuki]

I tried the Python scraper as asked:

• Ao Haru Ride: It works - The guest actors show up in each episode view.
  
  
• Violet Evergarden: It doesn't work - Same result than with the XML scraper. It only gets the special episode, but without title. Debug log in case it's useful.
  

That is not a scraper issue. That is MySQL being difficult again.

I have seen this problem a few time already, but never been able to figure out the cause. The Kodi/MySQL combination has a little glitch and the episode files are not seen or scanned, but if you were to switch over to local SQL, it would work perfectly.

Your show is here... smb://192.168.1.100/Animoteca/TV/Violet Evergarden (2018)/

You just need to change the path so Kodi reads it as a new tv show and starts the scan fresh. Maybe smb://192.168.1.100/Animoteca/TV/Violet Evergarden Special (2018)/ which is a valid alternate title at TMDB.

Let us know if that works.

I tried renaming the folder as suggested, but, unfortunately, it didn't work. Before scraping, I cleaned the libray, just in case, but it only gets the special episode. Nor the Python scraper worked.

I had scraped the show before without any issue, but I changed the files -same name but different extension-. Could it be related?

[Karellen]

I tried renaming the folder as suggested, but, unfortunately, it didn't work.

Do you have the Debug Log? Can you also post a screenshot of the Season 1 folder so I can clearly see every file in there.

I had scraped the show before without any issue, but I changed the files -same name but different extension-. Could it be related?

No, not now that you changed the name of the tv show folder.

The episodes are not being seen by the scanner, so the scraper is not called.

Can you scan it with the default SQL setup?

[pkscout]

It's not working for me either. According to Travis from TheMoviedb, some new fileds have been added to the credits in order to a future support for translated person names. That's probably what broke the scraper. I guess the scraper developpers are already aware of the problem.

Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it.

Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.

Anyway, thanks a lot for the work!

Do you have a couple show names I can use for testing where someone has the roll screenplay or storyboard? I think I see how to use the department rather than the role, but without a couple shows with which to test, I won't know if this works.

[Hebotsuki]

I tried renaming the folder as suggested, but, unfortunately, it didn't work.

Do you have the Debug Log? Can you also post a screenshot of the Season 1 folder so I can clearly see every file in there.

I had scraped the show before without any issue, but I changed the files -same name but different extension-. Could it be related?

No, not now that you changed the name of the tv show folder.

The episodes are not being seen by the scanner, so the scraper is not called.

Can you scan it with the default SQL setup?

Here is the debug log. And this is a screenshot of season 1 folder:


By the way, when I navigate through the Video>Archives to the show folder, there aren't files showing up, so what you say about Kodi not seeing the files is true. These are screenshots of the video archives:


So, having in mind what you said about Kodi not seeing the files, I decided to create a fake video with the name "Violet Evergarden - S01E01 - This is a fake video.mp4" and this fake file did show up in the Video archives:


If I scrape again the show with all the episodes I have plus this fake video, then it scrapes the fake episode!!!


So it's not an issue with the scraper nor with the database, but with my files. For whatever reason, Kodi doesn't recognise my files. Considering this, I tried moving the files to another HD and the moving them again to the show folder and now Kodi are seeing them!!! That's quite strange, it seems an issue with the file system or something.

Anyway, after doing that, I can now scrape the show even keeping the show name "Violet Evergarden".


@Karellen, @olympia, much appreciated your help and time to fix the problem!!! And sorry for the long post!!