+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+---- Forum: TinyMediaManager (https://forum.kodi.tv/forumdisplay.php?fid=204)
+---- Thread: More flexible scraper settings (/showthread.php?tid=347093)
More flexible scraper settings - Zerginator - 2019-09-08
TMM really is a near perfect program for managing media,
I only have some issues / wishes for the scraper, as settings are not very flexible here.
I mainly have issues with auto renaming files containing the media languages.
It would be perfect to be able to use the ${audioLanguageList} variable, but as I use *.m4v Files mainly that have an AAC fallback audio line,
this results in duplication [deu, deu, eng, eng], as there are a AAC and AC3 stream for every language.
Perfect solution would be to allow for detection of AAC fallback streams.
Using the Language field filled in manually is also trouble, as this field is updated and overwritten with garbage data every time scraping is done (it takes some random language data of the IMDB,...) entry.
And there is no way to deselect scraping that field in the settings.
Another issue is the rating field. I would love to use the rating frmo IMDB, but use the other fields frmo the TheMovieDB scraper.
Only solution is to scrape twice, and do a second run with IMDB rating to be scraped only.
It would be so great to do scraping in one run with different fields being filled from different scrapers.
The rating field is always completely overwritten, even though it allows more than one entry.
So if you manually add ratings and do scraping, everything is gone, as the scraper does not append, but overwrite information in there.
RE: More flexible scraper settings - myron - 2019-09-09
Well, we only utilize MediaInfo for that.
When we have 4 embedded audio streams, we simply list em - there is no comparison or whatsoever done.
Language should be also read by MediaInfo from stream, so you might wanna post a XML output from that file....
(never seen/heard of fallback stream?!)
Please have a look at the UniversalScraper, which should be able to mix different Metadata based on your preferences....
Need to check the overwriting, though...
RE: More flexible scraper settings - mlaggner - 2019-09-17
Overwriting data from a scraper if the essence of tmm! Whenenver you do a scape you expect tmm to use the data from the metadata provider to fill into tmm (and thus overwriting the old one).
I see no problem here! if you have custom content, you shall not pull online data into that fields!
RE: More flexible scraper settings - vibe666 - 2019-09-21
(2019-09-17, 09:02)mlaggner Wrote: Overwriting data from a scraper if the essence of tmm! Whenenver you do a scape you expect tmm to use the data from the metadata provider to fill into tmm (and thus overwriting the old one).
I see no problem here! if you have custom content, you shall not pull online data into that fields!
will there be an option for using a different subtitle scraper(s) at some stage? My ISP has a lot of sites that offer downloading of movie/tv related content (toerrent sites etc.) blocked and unfortunately, opensubtitles.org is on that list so it doesn't work for downloading subs in TMM.
RE: More flexible scraper settings - mlaggner - 2019-09-21
there are not much sites offering a _robust_ API for downloading subtitles.
we already tried thesubdb but for some reason (I cannot remember exactly) we did not continue with that
RE: More flexible scraper settings - Zerginator - 2019-10-13
(2019-09-17, 09:02)mlaggner Wrote: Overwriting data from a scraper if the essence of tmm! Whenenver you do a scape you expect tmm to use the data from the metadata provider to fill into tmm (and thus overwriting the old one).
I see no problem here! if you have custom content, you shall not pull online data into that fields!
I would absolutely agree in general with you, the issue is, the language field contains just garbage data. Does not matter if IMDB oder TMDB is used there are Languages in there, e.g. sometime "English", sometime "English, Japanse" for Film is have in English/German. I have no idea what data is stored there in the scraped database but it seems be just random. So this field makes sense, if it would contain the original language of the film, or the language of the audio streams, as this is at least for me an essential information for file renaming.
But even if I use the Universal Scraper and set Language to "-" the field is just overwritten with an empty value.
As stated above I would love to use the available audiostreams for renaming purposes, but at least for MPEG4 files there is a duplicate AAC fallback Stream for every AC3 stream, and also director's comments are added as a duplicate language.
So I currently use the Language field for storing the languages available in the video file, but is overwritten every single time I do any scraping, and it is not possible to unselect overwriting this field, as there is no checkbox available for it!!
Quote:I see no problem here! if you have custom content, you shall not pull online data into that fields!This is exactly what I would therefore be able to do. It is possible for most data field, but not for Language somehow.
Also it would be really nice to set more than one provider for Rating in the Universal Scraper, I usually add ratings from imdb, tmbd and if available from rottentomatoes, but that means the times scraping currently.
RE: More flexible scraper settings - mlaggner - 2019-10-14
Imho that is no garbage - it is rather not what you expect
The spoken languages field is filled with the different languages which are spoken in the original version (non translated version from the movie creators).You expect the languages which are spoken in the version you have on your hdd (aka the audio streams)..
I see that we need to improve the scraper settings for the data field (which get "overwritten" at each scrape - here we combine some fields in one setting..).
btw: in the universal scraper we take the rating from the scraper you have chosen as preference, but if any other scraper has been set (which should be always _true_ in the universal scraper otherwise you wouldn't use it
) we take the other ratings "for free". try to take omdbapi for the scraper fields which holds more different ratings
RE: More flexible scraper settings - Zerginator - 2019-10-14
(2019-10-14, 07:32)mlaggner Wrote: Imho that is no garbage - it is rather not what you expectOh, that makes sense. I never thought of that field meaning the spoken languages in the US version.
You expect the languages which are spoken in the version you have on your hdd (aka the audio streams)..
I see that we need to improve the scraper settings for the data field (which get "overwritten" at each scrape - here we combine some fields in one setting..).
btw: in the universal scraper we take the rating from the scraper you have chosen as preference, but if any other scraper has been set (which should be always _true_ in the universal scraper otherwise you wouldn't use it
Thanks for clarification! I must I admit I kind of misuse this field then. But to be honest, the audio langagues available in my videofile is the information that really matters
If there is any possiblility to avoid overwriting of this field, as possible with most others, that would be great.
And thanks for the hint with omdbapi, I will try that.