Why no Film-Noir genre on TMDB? Why no titles on UMS?
#1
Hey everyone, new user here.

Question about scrapers:  Universal Movie Scraper is not returning a sortable title and The Movie DB Scraper is not returning "Film-Noir" as a genre.  Is this a settings issue on my end?  Is there a solution?

Another question about scrapers:  I am managing a large (40K) collection and when I redo scraping it gets hung up a lot and is very slow.  Any tips?

thanks in advance
Reply
#2
i would suggest using linux would be faster for you but something tells me you wont...
Reply
#3
(2023-02-07, 23:38)LinuxHater Wrote: Universal Movie Scraper is not returning a sortable title
Read the UMS thread for the solution. Rather, switch to The Movie Database Python scraper, which is the one you changed away from.

(2023-02-07, 23:38)LinuxHater Wrote: The Movie DB Scraper is not returning "Film-Noir" as a genre
TheMovieDb does not use Film-Noir... Click here for full genre list

(2023-02-07, 23:38)LinuxHater Wrote: I am managing a large (40K) collection and when I redo scraping it gets hung up a lot and is very slow.  Any tips?
Yes, scrape your very large collection once. Also download the artwork locally using Artwork Dump.
Export your library to nfo files (don't export artwork) and use the nfo files from then on to rebuild your library when necessary.
Also split your movies into multiple sources and scrape a source at a time. I hope you don't have 40,000 movies in one source.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#4
No, it would take actual months.  All you linux guys should block me, I don't want linux help nor to associate with linux people.  They are worse than Apple People.
Reply
#5
Hey thanks for the reply.  

So I should turn off any downloading of images?  (I will investigate Artwork Dump, TFTT)

"Don't export artwork" seems like good advice.  I will export/import and see how it goes, probably using a much smaller test library.  This may save me vs. rescraping all my files.

When you say, "Also split your movies into multiple sources" what do you mean?  Do you mean scrape some with a different scraper?

thanks again and enjoy the sunshine,
John
Reply
#6
(2023-02-08, 01:54)LinuxHater Wrote: So I should turn off any downloading of images
No. Leave artwork to download. All I am saying is don't export artwork, when you do run the export function of Kodi if you have local artwork downloaded by Artwork Dump (or any other method)

(2023-02-08, 01:54)LinuxHater Wrote: When you say, "Also split your movies into multiple sources" what do you mean?  Do you mean scrape some with a different scraper?
No, you can use the same scraper for all your sources.
All I am saying is don't place 40,000 movies into a single source. That is a huge amount.
When Kodi scans your sources, it inspects each and every folder and file. It'll probably take days to go through 40,000 movies worth of folders and files. And the TMDB's Cloudflare protection will most likely block that many continuous requests coming from your IP address.
So you need to separate your content into more sources and slowly build your library source by source.
The trick here is that once you have built your library, you don't want Kodi scanning all your sources each time you run an Update Library.
So you keep a single source as active, where you add new content. All other sources are locked off from scanning (doesn't affect the library listings)
If keeping a single active Source does not suit the way you separate your content, then the other method is you don't use Update Library at all once your library is set up. Any new movie you add, find it it Videos>Files>Source then press i on your keyboard which will run an individual scrape, not the whole library scan.

If you are using Windows, you can install a Portable version and use it for testing... https://kodi.wiki/view/Portable_mode
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#7
(2023-02-08, 02:59)Karellen Wrote:
(2023-02-08, 01:54)LinuxHater Wrote: So I should turn off any downloading of images
No. Leave artwork to download. All I am saying is don't export artwork, when you do run the export function of Kodi if you have local artwork downloaded by Artwork Dump (or any other method)When you say, "Also split your movies into multiple sources" what do you mean?  Do you mean scrape some with a different scraper?
Quick aside.  Just curious, why do you prefer not to export artwork?  I know I got bit early on when using Kodi and it exported some very low-res versions of downloaded art, but after changing advancedsettings to avoid that, it seems to export fine and avoids having to re-download art if I have to re-scan from scratch on a new install.  Do you avoid it just for the sake of storage space over time or some other reason?  You know this stuff way better than me and figured it was worth asking.
Reply
#8
(2023-02-08, 03:22)NeoCortex Wrote: Just curious, why do you prefer not to export artwork? 
I think having local artwork is important for a number of reasons.
My comment, though, is aimed at exported artwork from Kodi.
Artwork exported from Kodi comes from the Kodi cache which is a modified version of the original artwork.
If users don't have the correct advancedsettings.xml entries (and most don't, judging by the logs I see on the forum) the 3840x2160 version of artwork scraped from the scraper site will be exported as 1280x720 IIRC.
I always advise the use of Artwork Dump to download the original artwork directly from the provider. Artwork Dump can download the artwork during the scraping process.
I am not against having local artwork, rather I urge it, but I think most users will expect the original version of the artwork, not a modified version.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#9
(2023-02-08, 03:54)Karellen Wrote:
(2023-02-08, 03:22)NeoCortex Wrote: Just curious, why do you prefer not to export artwork? 
I think having local artwork is important for a number of reasons.
My comment, though, is aimed at exported artwork from Kodi.
Artwork exported from Kodi comes from the Kodi cache which is a modified version of the original artwork.
If users don't have the correct advancedsettings.xml entries (and most don't, judging by the logs I see on the forum) the 3840x2160 version of artwork scraped from the scraper site will be exported as 1280x720 IIRC.
I always advise the use of Artwork Dump to download the original artwork directly from the provider. Artwork Dump can download the artwork during the scraping process.
I am not against having local artwork, rather I urge it, but I think most users will expect the original version of the artwork, not a modified version.
Gotcha.  That's what I figured.  Yeah, I learned the hard way myself ages ago that by default, export is pretty bad.
Reply
#10
(2023-02-07, 23:49)Karellen Wrote:
Quote:The Movie DB Scraper is not returning "Film-Noir" as a genre
TheMovieDb does not use Film-Noir... Click here for full genre list

Do you know of a link for imdb so we can compare?  I think imdb offers Sci-Fi and Science Fiction and tmdb only offers Science Fiction.
HOW TO - Kodi 2D - 3D - UHD (4k) HDR Guide Internal & External Players iso menus
DIY HOME THEATER WIND EFFECT

W11 Pro 24H2 MPC-BE\HC madVR KODI 22 GTX960-4GB/RGB 4:4:4/Desktop 60Hz 8bit Video Matched Refresh rates 23,24,50,60Hz 8/10/12bit/Samsung 82" Q90R Denon S720W
Reply
#11
(2023-02-08, 18:03)brazen1 Wrote: Do you know of a link for imdb so we can compare? 
https://help.imdb.com/article/contributi...6R742JRGAG#
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply

Logout Mark Read Team Forum Stats Members Help
Why no Film-Noir genre on TMDB? Why no titles on UMS?0