v21 Strange Behaviour Universal Movie Scraper on MariaDB10
#1
Smile 
Hi all, following a brand new & clean install of Kodi 21, I have noticed an unusual issue where I am unable to scrape only a handful of movies when using the Universal Movie Scraper. This issue only started since the use of advancedsettings.xml to successfully share my library with my other devices.

I confirm that I have read **** READ FIRST- Library and Scraping Problems ****

My preferred scraper is Universal Movie Scraper as it provides a lot more information i.e Rotten Tomatoes.

Very sorry for such a long post - but I wanted to capture as much as possible. Any help will be greatly appreciated Blush

My Configuration:
  • The following steps have been carried out on a Windows 10 device, but the issue is exactly the same if carried out on my other 2 Android devices (Nvidia Shield Pro)
  • Video tags are disabled
  • MariaDB10
  • Kodi 21.0.0
  • Skin AEONNOX Silvo V10.0.0
  • Universal Movie Scraper v 5.5.14
  • The Movie Database Python v3.0.1
  • No local .NFO file
  • My naming convention for all of my folders / files is exactly as for the one I have used in this example. Each movie has its own folder i.e Folder name “The Lord of the Rings The Fellowship of the Ring 2001” containing the file “The Lord of the Rings The Fellowship of the Ring.mkv”
  • Coincidentally all 3 of the LOTR trilogy movies have this same issue as described below !

These are the steps Running Kodi Locally without advancedsettings.xml: Kodi Log - Running Kodi Locally No advancedsettings.xml
  1. Temporarily removed advancedsettings.xml from the Database folder
  2. Started Kodi 21 and removed LOTR from library
  3. Cleaned the Library
  4. Open the folder where LOTR resides and scraped it using Universal Movie Scraper and am provided a short list of possible matches

    Image

  5.  I select “The Lord of the Rings: The Fellowship of the Ring (2001)” and it is scraped in successfully

    Image


Steps with advancedsettings.xml : Kodi Log Using advancedsettings.xml
  1. I add back the advancedsettings.xml into the Database folder and to ensure I was not using my local databases by renaming MyVideos131.db to MyVideos131.db.bak and MyMusic83.db to MyMusic83.db.bak
  2. Start Kodi and remove LOTR from library (as it had already been previously successfully scraped into my MAraiDB10 database)
  3. Clean the Library
  4. Open the folder where LOTR resides and scrape it using Universal Movie Scraper. I am provided with a long list of possible matches and I select “The Lord of the Rings: The Fellowship of the Ring (2001)”

    Image

  5. After a few moments attempting to find the title, I receive an error “No Information Found!” and I click OK

    Image

  6. I receive anther error message “Make sure the selected folder is writeable and that the new folder name is valid” and I click OK

    Image

  7. I look at my playlist (small icon view Sorted in Ascending Title view), and LOTR is the very first icon on there. I look at its information and it has no title and "No Information available"

    Image

  8. I remove it from library, but it is still there, but this time without a thumbnail
  9. I remove it again from library, and this time it is removed
  10. Clean the Library
  11. Open the folder where LOTR resides and this time I scrape it with The Movie Database Python. I am now provided with a much shorter list of possible matches. I select “The Lord of the Rings: The Fellowship of the Ring (2001)”

    Image

  12. This time it is scraped in successfully

    Image

Reply
#2
(2024-05-03, 12:38)ARAM10 Wrote: Steps with advancedsettings.xml : Kodi Log Using advancedsettings.xml
xml:
<network>
<cachemembuffersize>314572800</cachemembuffersize>
<readbufferfactor>4</readbufferfactor>
</network>
No longer valid settings. Moved to here... https://kodi.wiki/view/Settings/Services/Caching

(2024-05-03, 12:38)ARAM10 Wrote: My preferred scraper is Universal Movie Scraper as it provides a lot more information i.e Rotten Tomatoes.
Unfortunately UMS is unmaintained, has been for about a year and it is known to have issues.
The differences you are seeing with and without the advancedsettings.xml file is due to MySQL. It uses a slightly different process after scraping to add info to the database.
The problem you are experiencing is item 1i here... https://forum.kodi.tv/showthread.php?tid=336969
The error can be seen on line 2257 of the log with MySQL

The easiest fix is to use TheMovieDatabase Python scraper, otherwise follow the fix in the linked github issue report.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#3
Hi Karellen,

Many thanks for your quick reply Blush

I have commented out the network section of my advancedsettings.xml.

Having read Your Post - feedback from the Mariadb team about type TEXT for the column c06 in the table tvshow.

Would it make any difference if i change the relevant field for the movies to Longtext? And if yes, what do I need to change please?

Thanks
Aram
Reply
#4
(2024-05-03, 14:20)ARAM10 Wrote: Would it make any difference if i change the relevant field for the movies to Longtext? And if yes, what do I need to change please?
Yep, that should fix the problem according to the comments in the Issue report. I think Medium text will also.
As how to do that, sorry I cannot help you there. I don't use a MySQL/MariaDB setup, so have no experience with it. It should be explained in the report.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#5
Finally SUCCESS  Big Grin

Having spent the last few hours individually changing the field types in my MariaDB10 MyVideos131 database from text to longtext, and then removing my LOTR, cleaning the Library & scraping with the Universal Movie Scraper - I had no joy !

I decided to look at the error message once again in my log file. I actually copied the lines with the errors to a Word document and immediately realised it was attempting to scrape a HUGE amount of artwork.

That directing me to the Artwork section in my scraper where I realised that I had Enabled Everything. So after making the changes as below:

Image

Image

Image

I was finally able to successfully scape all of the movies with Universal Movie Scraper that i was previously having an issue with Blush

Image
Reply
#6
Great!
Hopefully the artwork types you disabled are saved as local artwork on your system?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#7
Yes they are
Reply

Logout Mark Read Team Forum Stats Members Help
Strange Behaviour Universal Movie Scraper on MariaDB100