Too many fanart images for scrapers: Data too long for MySQL tables (error 1406)
#4
(2019-02-24, 12:23)lcg_ Wrote:
(2019-02-16, 21:02)schapplm Wrote: I stumbled over a potential bug in the The Movie Database Scraper.
If I add a directory with tv shows to the library, only some of them are scraped.
Shows with many fanart images on tvdb cause the scraper to try to insert all these images into a field of the MyVideos database, which is refused by the mySQL server.

I use Kodi 18.0 with addons up to date: E.g. The Movie Database version 3.1.0. The database is not shared with other clients.

[...]
In the log I get
Quote:19:03:19.587 T:140703594206144   ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
Query:  Query: UPDATE tvshow SET c00='The Walking Dead',c01= .... c06='<thumb aspect=\"poster\" preview ... \" preview=\"w780/ywHdL2ip9OaClGmHJ7mKb2CPoft.jpg\">original/ywHdL2ip9OaClGmHJ7mKb2CPoft.jpg</thumb></fanart>',c12='21682',c13='18',c14='AMC',c15='',c16='', userrating = NULL, duration = NULL WHERE idShow=142
19:05:29.582 T:140703594206144   ERROR: ExecuteQuery - failed to execute query
19:05:29.593 T:140703594206144   DEBUG: Mysql rollback transaction

I shortened the line, but is has over 100000 (one hundred thousand!) characters, since there are numerous preview images listed up. Error 1406 is "Data too long for column".   
I recently upgraded my Kodi installation to a centralized file storage and database (Kodi 18.1 and MariaDB in my case). Since Kodi switched to TMDB for TV series, I started to re-scrape my TV series and ran into the same problem with shows like Doctor Who. The show remained empty and I had to dig through the debug logs to find out what was going wrong.

I have found that I can get around the problem by increasing the size of the 'c06' column in the 'tvshow' table above its 'text' type 65535 character limit:
sql:
ALTER TABLE MyVideos116.tvshow MODIFY COLUMN c06 mediumtext DEFAULT NULL NULL;

I do not know if this might have some side effects in the future when the database is migrated to a newer schema. This currently makes me really hesitant to go through with all the scraping if I may have to do it again at some point.

Cheers,
L. 
I think I'm having this issue.  I'm using Library Integration Tool to add movies from an addon to the Kodi library.  LIT creates a folder and matching strm file in the kodi library, but when I update the library only some of the movies are added even though there's a folder and strm file for the missing movies.  I've come to figure out that it has to do with the size of the strm files.  If they're over 4KB in size, they are not read correctly by Kodi and therefore not imported.  If I edit the strm file and remove some actor data to bring it below 4KB, then it imports just fine.  How do use that ALTER TABLE command to fix it?
Reply


Messages In This Thread
RE: Too many fanart images for scrapers: Data too long for MySQL tables (error 1406) - by eric859 - 2019-05-24, 01:46
Logout Mark Read Team Forum Stats Members Help
Too many fanart images for scrapers: Data too long for MySQL tables (error 1406)0