• 1
  • 138
  • 139
  • 140(current)
  • 141
  • 142
  • 167
Release Universal Movie Scraper
(2021-06-17, 10:11)olympia Wrote: Right, thanks for testing!
I just pushed UMS version v5.5.1 please see if it is any better.

Thanks for your time. I'm still occassionaly getting the Japanese original title. Looking at the changes that you've made in the commit on github, I can see you've added a forward slash before the pipe character but there is still a forward slash at the end of accept-language=en-us on 4 lines - I think the imdb web server is seeing this as "en-us/" instead of "en-us".
Reply
(2021-06-17, 14:05)Zippy79 Wrote:
(2021-06-17, 10:11)olympia Wrote: Right, thanks for testing!
I just pushed UMS version v5.5.1 please see if it is any better.

Thanks for your time. I'm still occassionaly getting the Japanese original title. Looking at the changes that you've made in the commit on github, I can see you've added a forward slash before the pipe character but there is still a forward slash at the end of accept-language=en-us on 4 lines - I think the imdb web server is seeing this as "en-us/" instead of "en-us".

Yay, see v5.5.2
Reply
Thanks for the updates. I am seeing the film plot and the film score correctly now, but I still see film duration as "0".

metadata.universal version 5.5.2.
metadata.common.imdb.com version 3.2.2.

Film I have seen this issue: https://www.imdb.com/title/tt0106226/

Thanks for your effort.
Reply
(2021-06-19, 02:10)jcea Wrote: Thanks for the updates. I am seeing the film plot and the film score correctly now, but I still see film duration as "0".

Duration is not a scraped field; Kodi core is extracting it from the file itself, so the issue will be somewhere else.
Reply
(2021-06-19, 08:27)olympia Wrote:
(2021-06-19, 02:10)jcea Wrote: Thanks for the updates. I am seeing the film plot and the film score correctly now, but I still see film duration as "0".

Duration is not a scraped field; Kodi core is extracting it from the file itself, so the issue will be somewhere else.

I humbly disagree, @olympia. I was referring to "runtime" (sorry for the wrong vocabulary) and "runtime" is a scraped value according to (outdated) https://kodi.wiki/view/HOW-TO:Write_media_scrapers

I currently see "runtime" as zero in newly scraped films.

Thanks a lot for your time and effort, @olympia.
Reply
Oh, if you know it better based on an outdated wiki page, then probably you can fix this yourself.
...or else, if you can't fix it, then please don't try to know it better.

Duration/ runtime - or whatever else we call it - doesn't suppose to be a scraped value (UMS didn't scrape runtime from imdb for ages by now) as Kodi should handle this itself.
As I said, you should look elsewhere to get this issue resolved; Maybe you have extract video information from files disabled or something...
Reply
(2021-06-20, 11:14)olympia Wrote: Oh, if you know it better based on an outdated wiki page, then probably you can fix this yourself.
...or else, if you can't fix it, then please don't try to know it better.

Duration/ runtime - or whatever else we call it - doesn't suppose to be a scraped value (UMS didn't scrape runtime from imdb for ages by now) as Kodi should handle this itself.
As I said, you should look elsewhere to get this issue resolved; Maybe you have extract video information from files disabled or something...
Unfortunately I lack the scraping expertise you have @olympia. I am pretty used to regular expresions, but I find the kodi scraping machinery bizarre and difficult to understand. The ony available information is documented as "outdated" and that doesn't help.

Checking the sourcecode of "Universal Movie Scraper" version 5.5.2, I see something apparently related in file "universal.xml", line 193:

xml:

                        <RegExp input="$$1" output="&lt;runtime&gt;\1&lt;/runtime&gt;" dest="5+">
                                <expression noclean="1">&quot;runtime&quot;Sad[0-9]+)</expression>
                        </RegExp>

I don't know why you are enclosing "runtime" with "&quot;", for instance, of if "runtime" in the regexp would match "Runtime" (capitalized) present in the IMDB webpage.

The outdated wiki page is not very clear about the meaning of "noclean" parameter but I have been unable to find any other relevant and up to date documentation in Kodi. Could you possibly point me to a Kodi up to date documentation page explaining how scraping works?. I don't want to learn from aged and incorrect documentation.

That source code seems to suggest that "runtime" scraping is something that should work. If it is stale code from ages ago, I would humbly suggest to delete that dead code to avoid confusion and unnecessary arguments
Reply
I just compared "universal.xml" between versions of Universal Movie Scraper. This block of XML was deleted between 5.4.6, published in late december, and 5.5.0, published June 15th.

Those dates are consistent with my experience of losing "runtime" scraping a few days ago. Until june, IMDB changes and last week UMS updates, those details were scraped just fine.

xml:

<RegExp input="$$1" output="&lt;runtime&gt;\1&lt;/runtime&gt;" dest="5+">
    <expression trim="1">&lt;h4[^&gt;]*?&gt;Runtime:&lt;/h4&gt;[^0-9]*([0-9]+)</expression>
</RegExp>

I just added that XML fragment to UMS 5.5.2, in the right place (after current "runtime" regex", as in UMS 5.4.6). After scanning a handful of movies, "runtime" information seems to be scraped correctly again. Good!!

Please, consider releasing UMS 5.5.3 with that fix, or an improved version.

Thanks for your time and effort, @olympia .

PS: I think the the fragment "[^&gt;]*?" doesn't do what you think, it will not eat everything until the next ">" in the HTML. That is my understanding, but I could be wrong. EDIT: My fault, you are right, the XML machinery will change that "[^&gt;]*?" to the real regex executed: "[^>]*?". That is what I would expect. I beg your pardon. That code is fine. Sorry.
Reply
@jcea

Thanks for taking the time to find the difference. Smile

Do you have this setting disabled... https://kodi.wiki/view/Settings/Media/Vi...from_files

Are you using iso's, bluray/dvd folders, single video files?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2021-06-21, 07:47)Karellen Wrote: @jcea

Thanks for taking the time to find the difference. Smile

Do you have this setting disabled... https://kodi.wiki/view/Settings/Media/Vi...from_files

Are you using iso's, bluray/dvd folders, single video files?

Yes, I do have "Extract video information from files" disabled. My media content in accessed remotely via WebDAV and that option was enormously costly and slow. Until UMS 5.5.0, published a week ago, "runtime" was scraped just fine, so I consider it a regression. Are we hitting a Hyrum's Law situation? https://www.hyrumslaw.com/

My film content is single files, with an occasional NFO companion when a film is not correctly identified.
Reply
I think your situation is extremely rare, but a valid use case, so I added back scraping of runtime at UMS v5.5.3.

(Would have been good to have clarity on this from the beginning, so we could've been more efficient debating this.)
Reply
Hi.
I'm French, so sorry... I don't necessarily have understandable English.

I use Universal Movie Scraper on SPMC, on an Android TV 5.0 box (and not Kodi), with the maximum version offered 2.9.13
I've been using this version for a long time, without worries.

But now I realize that for each new movie I add, some info is missing (year, imdb rating, ...)
I updated manually with the zip from version 5.5.3. Nothing changes...

Why ?
(except I don't understand the difference between Gotham, Krypton and Leia. I took 5.5.3, it installed without problem)
Reply
If UMS v5.5.3 was compatible with older Kodi versions, then we would be make it available to install on those versions.
In other words, we make it available as from Leila on purpose. Installing manually a version which doesn't support the older version is not too wise.
Reply
(2021-06-25, 20:24)olympia Wrote: If UMS v5.5.3 was compatible with older Kodi versions, then we would be make it available to install on those versions.
In other words, we make it available as from Leila on purpose. Installing manually a version which doesn't support the older version is not too wise.

I have a very odd problem. I've recently upgraded to 18.8 (18.8.0). I go to the Video - Files then a hard drive then a subdirectory (ACTION), I select Change Content and select Universal Movie Scrapper 5.5.3 and for Content Scanning Options I've selected Movies are in separate folders that match the movie title and Scan recursively and click OK. Back at the ACTION subdirectory, I right-click and select Scan for New Content. When look in the subdirectory none of the movies' Information has the Year. I've created a Playlist for the same subdirectory - (Path starts with x:\000\ACTION) as the only rule. After it returns a list, I place it in Order by Year and none have the Year. In the Playlist list, I right-click and select Information (no year), BUT, if I select Refresh, the Movie Information message box asks Locally stored information found. Ignore and refresh from Internet. If I click Yes, it presents a list of targets - if I select the movie, it loads the movie details and updates the Year. The contents of the folder look like this "Jack.Reacher.2012.m720p.BluRay.x264-BiRD" I'm assuming at some point I exported to each of the folders. What should I do to get everything in order so should I lose my library, my files and scrappers are tuned to load everything back. Thanks, John
Reply
After recent update of UMS none of the movies are getting scaped, tried both on Android Kodi Leia and CoreELEC 9.2.7 installations. When I changed the scraper to TMDB it is working fine.. Might be some bug there. Please check.
Reply
  • 1
  • 138
  • 139
  • 140(current)
  • 141
  • 142
  • 167

Logout Mark Read Team Forum Stats Members Help
Universal Movie Scraper9