Scrapers for movie / tv data have bugs that appear to corrupt database
#1
I have encountered some bugs that appear to span TMDb TV / Universal Scraper / TMDb Movie scrapers. This seems to point to a bug or bugs that is systemic and not just related to specific scrapers. I will attempt to document them to assist in debugging for the developers. Note that this is the Windows version of Matrix. Most current stable release. 19.3

These bugs occur when the type is set to TV Shows and/or to Movies and regardless of the scraper. 

1. Not all items/shows allow the information to be updated. Most items will display an Information menu choice if you right click on the item. You are then given the option to update the information. Some items do not display the Information menu choice. You have no way to view or update the information in the database. You can also not simply clear the item's information from the database. This results in 2 scenarios. If the scraper has entered data in the database and the data is for the wrong show, there is no way to correct it or clear it since it is impossible to access any menu that will allow the data to be re-scraped or manually updated. If the item has no data in the database, there is no way to refresh / obtain data. There is no option to manually provide data to the database via form. No way to clear incorrect data. 

1B. There is a related problem to the one above. Sometimes if you REFRESH the data on the item, it finds the data via the scraper then updates the data. BUT the information is no longer available. If you right click and select SCAN to DATABASE the information will appear but will be gone the next time you attempt to view the information. 

2. Some items give the Information option or have a REFRESH button in the Information screen BUT if you attempt to update the information it fails without error message to the operating system. (crash) Select the item and REFRESH. When asked if you want to ignore the data in the database and refresh from the Internet answer YES. The process will respond with No Information Found. Now repeat the operation following the same steps on the same item. The system will now find information and attempt to update the database. It will then crash to the Windows 10 desktop without further ado. 

3. There is no way to clear a record from the database when the scraper has populated it with the wrong show or movie and it is unable to find the correct show or movie. It just keeps the old and incorrect data. 

4. Changing the scraper from TMDb to "local information only" results in a variety of unexpected behaviors. Some of the items simply vanish from the database. They do not reappear when the database is updated. They are still on the server and the path for that media type. On the next database update, the database (set to local information only now) will still update from the TMDb database scraper even though it is set to "local information only". The system at the end of this process may or may not crash to the Windows 10 OS desktop. Nothing seems to make the now missing items reappear. Searching for the title returns No items found. 

5. I do not see any way to drop the database and rebuild it. I fear that mine is now corrupted and there appear to be no tools to repair the database or to purge and rebuild it. The first time I built the database using scraper data, it too 3.5 days so I'm not looking forward to that process but at this point missing items that can't be recovered, items with wrong data that can't be updated or fixed, and crashing seems to point to the need to fix or replace the database. Any feedback on this is gratefully welcomed and appreciated. 

This is an attempt to help developers fix this software and should not be viewed as a screed, whine or complaining session. I know that software of this type is complex. I want only to provide as much helpful information as I can to make Kodi a solid product. If any developer wants additional information from me to help please let me know and I will do what I can to provide the information to you. I can now replicate most or all of these errors on demand. 

Installation information:
Windows 10 with KODI Matrix 19.3
Libraries are all shares on a Windows Network residing on a large volume on a server local to the network. (about 50 TB of files. Some are DVD folder images. Some are MP4 video files. They are segregated by type (DVD image / MP4 / TV / Movie) and each type given its own MEDIA entry in Kodi. (DVD TV, DVD Movie, MP4 Movie, MP4 TV, etc.)
The KODI database resides on the local media PC on a local Hard disk internal to the computer
All MEDIA repositories are associated with their correct type and scraper (TV - TMDb TV database / Movies - Universal scraper and IMDB, etc. )

Please reach out if additional information is desired or required. If you want a file or log, please remind me where to find it so that there is no delay in providing it to you. Thanks for any feedback or help. 

David
Reply
#2
@dwreid55

Hmmm, how long have you been using Kodi?
Your post makes me think you are a beginner, probably using kodi for a few days? I say this due to your use of the statement "There is no way to...", when in fact everything you say that cannot be done, can be done.

But the crashing indicates a problem. Provide a log that captures the crashing and we might be able to see what is causing it... Debug Log

I'll move you to the correct forum.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply

Logout Mark Read Team Forum Stats Members Help
Scrapers for movie / tv data have bugs that appear to corrupt database0