Too many fanart images for scrapers: Data too long for MySQL tables (error 1406)
#2
(2019-02-16, 21:02)schapplm Wrote: I stumbled over a potential bug in the The Movie Database Scraper.
If I add a directory with tv shows to the library, only some of them are scraped.
Shows with many fanart images on tvdb cause the scraper to try to insert all these images into a field of the MyVideos database, which is refused by the mySQL server.

I use Kodi 18.0 with addons up to date: E.g. The Movie Database version 3.1.0. The database is not shared with other clients.

[...]
In the log I get
Quote:19:03:19.587 T:140703594206144   ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
Query:  Query: UPDATE tvshow SET c00='The Walking Dead',c01= .... c06='<thumb aspect=\"poster\" preview ... \" preview=\"w780/ywHdL2ip9OaClGmHJ7mKb2CPoft.jpg\">original/ywHdL2ip9OaClGmHJ7mKb2CPoft.jpg</thumb></fanart>',c12='21682',c13='18',c14='AMC',c15='',c16='', userrating = NULL, duration = NULL WHERE idShow=142
19:05:29.582 T:140703594206144   ERROR: ExecuteQuery - failed to execute query
19:05:29.593 T:140703594206144   DEBUG: Mysql rollback transaction

I shortened the line, but is has over 100000 (one hundred thousand!) characters, since there are numerous preview images listed up. Error 1406 is "Data too long for column".  
I recently upgraded my Kodi installation to a centralized file storage and database (Kodi 18.1 and MariaDB in my case). Since Kodi switched to TMDB for TV series, I started to re-scrape my TV series and ran into the same problem with shows like Doctor Who. The show remained empty and I had to dig through the debug logs to find out what was going wrong.

I have found that I can get around the problem by increasing the size of the 'c06' column in the 'tvshow' table above its 'text' type 65535 character limit:
sql:
ALTER TABLE MyVideos116.tvshow MODIFY COLUMN c06 mediumtext DEFAULT NULL NULL;

I do not know if this might have some side effects in the future when the database is migrated to a newer schema. This currently makes me really hesitant to go through with all the scraping if I may have to do it again at some point.

Cheers,
L.
Reply


Messages In This Thread
RE: Too many fanart images for scrapers: Data too long for MySQL tables (error 1406) - by lcg_ - 2019-02-24, 12:23
Logout Mark Read Team Forum Stats Members Help
Too many fanart images for scrapers: Data too long for MySQL tables (error 1406)0