[Klojum]

The Python scraper itself in v19 is slower AFAICT (perhaps things can be optimized in the future) and perhaps v19's overall handling is a little slower as well, but when having the exported nfo files already present, things do go a lot quicker because metadata doesn't need to be retrieved from the internet.

[zebraitis]

The Python scraper itself in v19 is slower AFAICT (perhaps things can be optimized in the future) and perhaps v19's overall handling is a little slower as well, but when having the exported nfo files already present, things do go a lot quicker because metadata doesn't need to be retrieved from the internet.

"... a little slower..."

In the newly loaded 18.9, it took about an hour to complete 450+ TV series.

In 19.1, it took 12+ hours, and failed consistently.

With the same files, the same NFO's.   As I watch the dawn break on another day of troubleshooting, I think that I can clearly communicate that I believe that there may be  some serious DB issues in 19.1

Sidebar:  The problem that I have now going back to 18.9 is that the TV Show Scrapers that are available likely have been modified to work with 19.1, so I am not getting my most recent TV shows to scrape (posted log at TVMaze scraper).  But save to the DB the files did, and I can exit and restart Kodi 18.9 without any MySQL lockup issue.


OPINION:

Having a DB that has been around for more than a few versions of KODI, upgrades of the DB schema driven by Kodi upgrades allowed it to pass from 18.9 to 19.x with little challenges.

HOWEVER... since I had that nasty self-inflicted Windows 11 issue, AND as a result having to start over from scratch, THEN needing to create an all new DB has exposed a problem with Kodi writing to a MySQL DB.

SUGGESTION:

While those of us that use MySQL and profiles are a relatively small group, it is likely a group that has had years working with Kodi and providing input.   It would be appreciated if someone on the KODI team could do a fresh install of Kodi 19.1 on Windows, leveraging a MySQL DB against a large source of content and see if they can replicate this.

Because I am telling you through this thread, with 19.1 and a LOT of different versions of MySQL and many rescans, it is not working at all as designed.


---------------------


MY NEXT STEPS:

I have no choice but to try to rebuild my DB's and profiles in 18.9.  and then to learn how to export the DB's in MySQL, before trying to upgrade to 19.1x  (albeit slowly).

[zebraitis]

OK...  to close this as a troubleshooting issue, I want to state the following:

* On the same Windows10 computer
* with the same shared folders
* with the same 250GB SSD and 4 10TB drives
* running the same version of MySQL

...  I was able to load a local copy of Kodi 18.9 and rescan all my media without any challenge or drama.  The rescan (once media was copied from backup) went very quickly compared to a 19.1 scan.  Of course time (days) were lost in clearing and restoring my media.

However, this means that I can now access this MySQL shared DB on 4 remote instances of Kodi.   As I said, this closes my troubleshooting.

---------------------------------

What it DOES mean is that there appears to be a problem with Kodi 19.1x and a shared DB using MySQL on Win10.

Does the Kodi team still support this configuration?

[zebraitis]

What it DOES mean is that there appears to be a problem with Kodi 19.1x and a shared DB using MySQL on Win10.

Does the Kodi team still support this configuration?

OK...  so since this has gone unanswered, I will start a different thread.

Likely because this was seen as a troubleshooting thread, the question of if expected features are no longer being supported may be overlooked.

In that new thread, I will discuss my config and ask if a 19.1x migration is possible.

[Klojum]

OK...  so since this has gone unanswered, I will start a different thread.

Creating multiple threads on this forum on the same subject, simply because you didn't get a fast enough answer, will not make this a more transparent problem nor will it be solved faster.

Not all of us are database experts / Not all of us have 9-5 time for Kodi / Not every problem is as easy to solve. If this all was a generic Windows/MySQL/MariaDB problem, there would be many more complaints on the subject.

So for some reason your Windows PC setup is now allergic to MySQL. I'm sure you'll understand that for us it's hard and perhaps impossible to reproduce your database experiences. Do you maybe still have a Windows 7 setup available to test with?

[zebraitis]

OK...  so since this has gone unanswered, I will start a different thread.

Creating multiple threads on this forum on the same subject, simply because you didn't get a fast enough answer, will not make this a more transparent problem nor will it be solved faster.

Not all of us are database experts / Not all of us have 9-5 time for Kodi / Not every problem is as easy to solve. If this all was a generic Windows/MySQL/MariaDB problem, there would be many more complaints on the subject.

So for some reason your Windows PC setup is now allergic to MySQL. I'm sure you'll understand that for us it's hard and perhaps impossible to reproduce your database experiences. Do you maybe still have a Windows 7 setup available to test with?

I think what you are saying is fair, so I will stay in this thread.  

However, as Windows 7 was end of support on Jan 14, 2020, respectfully, I cannot in good faith troubleshoot on a platform that is not receiving security updates. 

Additionally, I do disagree with the allergy metaphor (and you will see why, below), but let's get to the good stuff: 



Let's discuss my implementation today that is running Kodi 18.9 w/MySQL successfully:

As I have completely rebuilt both the server layer and the presentation PC layer, I can honestly say that this is a current fully functional Win10 platform.

My home sever is a home-built box with Gigabyte GA-Z68XP-UD3P motherboard.  Not a big fan of Gigabyte, but I did need some SATA 3 ports and it had 'em, and it was a cheap used upgrade choice on craigslist. 

Running on that motherboard is an Intel 8-core i7-3770 CPU @ 3.40GHz.  The box also has 16GB RAM, and a gig ethernet port  There are no additional boards-in-slots.  The boot drive is a 240GB SSD, and I have 4 @ 10TB drives. 

I now use Macrium Reflect Free and another 240 GB drive as my boot drive backup via imaging.  To backup my 40TB drives, I have a Buffalo Terrastation TS5800D NAS connected via ethernet cable.

The software running on my home "server" is Win10 Pro 64 bit ver 21H1 and MySQL 5.7.33.  The MySQL implementation followed the standard instructions for MySQL implementation from the Kodi Wiki.  The only deviation is that I chose was to allow for more resources to be accessible to MySQL by selecting "server machine" rather than "development machine" for config type

I have loaded a local copy of Kodi 18.9 so that I can update locally on that machine to the local MySQL DB.  This is new, as I used to do that on the living room TV, but it finally dawned on me that the WAF would be improved if I did this on the home server either at my desk or via remote desktop.

My Kodi implementation is multi-room and also is implemented with 8 profiles.  All profiles appear on all remote machines and connect to the MySQL DBs.  All remotely connected Kodi running machines are windows based, same Kodi version, and all ethernet connections are hard-wired except for one tablet.  Nearly all run the confluence skin. In practice, no more that 3 of these remote machines would connect to the shared MySQL DBs at any one time.

So that you can get a view of what I have just said, here is a debug-enabled log of the Kodi 18.9 that is running on my home server:
https://paste.kodi.tv/hucapagafe.kodi 


SUMMARY:  As I said, this implementation works FLAWLESSLY for Kodi 18.9  and, this has been working successfully in many other previous major release Kodi versions.  (And when I say "previous versions", I'm taking MySQL all the way back to XBOX w/ XBMC.)

WHAT CHANGED: 
1) When I went to Kodi 19.0, I ran into the Kodi Crash-on-launch issue that was resolved by KodiSetup-20210525-fd5acfb5-Matrix-x64
2) When I went to 19.x, Kodi automatically updated my DB's, and that seemed to work, but updating seemed slower.
3)  Tried reloading a remote machines to see if a new fresh version of windows & Kodi would make scanning for new media faster, but did not seen improvement.
4) With the Win11 rollout,  I gave that a shot on my home server, was unimpressed, and all-too-late found out that the only way that I could undo that "development" implementation was a complete wipe of the hard drive.   And with that wipe, and reload from scratch in Win10 running Kodi 19.x environment, that is when I found that I COULD NOT get through a Kodi scan of media without crashing the MySQL application.  Of course, that led to this thread and why I listed it:  [BROKEN].  I too have a "regular job" but I spent well over a week and many sleepless nights in this thread trying to get 19.x to work with MySQL.

And that is why I wiped everything and fell back to 18.9, and a successful stable system.

So, while there may be a handful of folks regularly using MySQL (compared to standard Kodi deployments) that have migrated from 18.9 to 19.x with a DB upgrade occurring automatically, there may be very few that have actually tried to load this new in 19.1  And that may be why I am the only one that is saying that something is wrong.  This, however, is just a theory and even with my experience I would say it's a possibility, but I cannot be sure.


WHAT I WOULD LIKE TO KNOW:  Is this MySQL implementation still a supported configuration in 19.1x ?  Does the KODI development team feel that they can stand behind Kodi with MySQL with confidence?

I am more than willing to provide more info, additional logs or test various options in an attempt to better the product.  We would be starting from a configuration that works successfully, so any glitches that we found may be easily traced.

Let me know the next steps that you all recommend.

[black_eagle]

Just my two cents here, but what might help is a log from the MySQL server when it goes AWOL from Kodi.  Your kodi log(s) showed that happening but it might be helpful to see the MySQL log as well because it may contain some pertinent info. FWIW, I replaced MySQL with Mariadb because it is faster - might only be my experience of course but might be worth a try.

[zebraitis]

Just my two cents here, but what might help is a log from the MySQL server when it goes AWOL from Kodi.  Your kodi log(s) showed that happening but it might be helpful to see the MySQL log as well because it may contain some pertinent info. FWIW, I replaced MySQL with Mariadb because it is faster - might only be my experience of course but might be worth a try.

I appreciate your response.

I would only see AWOL behavior that once I did the 19.1 upgrade or reinstall.    I would be happy to do that, if the team recommends that I upgrade and tell me the path that they want me to take to complete it.  I volunteer to be a test crash dummy here, but I would like to ensure that I do that with purpose.

I would have to be told what logs to provide then... as I am not a DB specialist either.

As to MariaDB, in attempting to resolve this earlier I did see that many have made a migration away from MySQL to MariaDB, but without links and instructions on the wiki, I was unsure where to begin.

[Klojum]

I would have to be told what logs to provide then... as I am not a DB specialist either.

There is this tool on the internet called 'Google', and you can ask it anything. Such as mysql 5.7 log file location windows

This is what you get in return:
The error log is located in the data directory specified in your my. ini file. The default data directory location is C:\Program Files\MySQL\MySQL Server 5.7\data

[black_eagle]

As to MariaDB, in attempting to resolve this earlier I did see that many have made a migration away from MySQL to MariaDB, but without links and instructions on the wiki, I was unsure where to begin.

MariaDB is a direct drop-in replacement for MySQL.  You simply remove MySQL and install MariaDB.  Everything else stays the same and anything that would have previously connected to MySQL will now connect to MariaDB instead.  Your original databases will still be available but via Maria not mysql.

[zebraitis]

@Klojum   Hmmm... the google...  yeah, doubt if that will ever catch on when there are awesome card catalogs in large libraries.  Dewey Decimal fo-eva  


Still, checking the the existing functional Kodi 18.9 implementation, on the box where MySQL is installed, I did find the info located in C:\ProgramData\MySQL\MySQL Server 5.7

MySQL my.ini - https://paste.kodi.tv/uwacefohap

MySQL Log file - https://paste.kodi.tv/ecuvojufek

MySQL error file fresh after "net stop / net start MySQL", stating KODI and scanning for new content in primary profile - https://paste.kodi.tv/anoyimuyig

I restarted MySQL to get a new error log as the one that was there had like 27,000+ lines, and pastebin puked on taking that in.  I saved a copy if that longer file is preferred.


Those files could set a baseline if you would like my to try to upgrade to 19.1x

But a recommendation to upgrade to 19.x w/MySQL would be based on your / Kodi Team's strong recommendation that MySQL would be the right thing for moving forward in 19.x 

Because the reality is that troubleshooting takes time, and so does your attention and review.  I respect you and the folks that do this and I really don't want to waste time going down a path that wastes everyone's time.



@black_eagle  - So I am getting a pretty strong clue that you prefer MariaDB.  I think your passion is winning me over to make that move, especially since you have said that the performance is better in 19.x

Honestly, I really don't wish to fix MySQL's challenges with Kodi if MariaDB is just flat-out better.  But understand my hesitancy as MySQL has been working for me & Kodi for many years...  and suddenly it is a dog in 19.x  - So, from this side of the keyboard, one has to ask:  is it really a MySQL issue or a Kodi DB issue.   I don't know, and I don't know enough to speak with any confidence on that.

As to MariaDB ... I feel ready to do this in the Kodi 18.9 implementation, but I have to ask for clarification.  You say:  "You simply remove MySQL and install MariaDB.  Everything else stays the same and anything that would have previously connected to MySQL will now connect to MariaDB instead.  Your original databases will still be available but via Maria not mysql."

My DB's look to be in C:\ProgramData\MySQL\MySQL Server 5.7\Data  

If I install MariaDB, I am venturing a guess that MariaDB would not use MySQL's same directory structure, would it?

So:  Should I do a Net Stop MySQL, move those subdirectories that exist under Data somewhere for safe keeping, uninstall MySQL and once MariaDB is installed then move those subdirectories/DB's back into MariaDB's data subdirectory?

OR: Export & overwrite my library to disk, wipe MySQL completely,  install MariaDB and just rescan everything fresh?



My Ultimate Question right now:    I look forward to the team's input on if I should go forward with MariaDB or MySQL with Win10 on 18.9 before the 19.x upgrade.   Thanks!

[black_eagle]

My Ultimate Question right now:    I look forward to the team's input on if I should go forward with MariaDB or MySQL with Win10 on 18.9 before the 19.x upgrade.   Thanks!

Ah, awkward questions!!

I really can't comment on what to do with regard to Win10 as there is not a chance of me ever installing it on any of my Kodi production kit.  However, I think @HeresJohnny uses Windows and also knows a fair bit about the configuration of MariaDB so hopefully he can shed some light on those Win10 questions.

Let's see what comes from that before deciding on a path to take.

[HeresJohnny]

Everything was working great on my home server (win 10 64 box) that was running MySQL for a shared DB.   Then I had this idea that I would try try the new Windows 11.  Long story short, decided to fall back to 10,  Which I could do...

All other problems aside, how did you get back to Win10? If you did something other than recreate it exactly from a backup image you took before you might be in for a a world of problems in the future - not only with Kodi. If you didn't go back in a way that eliminated the whole experiment you might probably be better off by re-installing everything from scratch. I doubt that switching from MySQL 5.x to MariaDB 10.x would be a solution in your case. Normally both should work reasonably well with only some differences in speed due to different versions although I have only first-hand experience with Kodi 18+ and MariaDB.

As much as MariaDB is called a drop-in replacement for MySQL that's not exactly true. Both products have seen some development and have unique features that are not 100% compatible in their latest branches. AFAIK MariaDB will import your current database via a nice assisstant and convert it to its own structure. You'll have to be careful to keep the database name to "MySQL". Once converted to the latest 10.x format you'll not be able to go back directly to MySQL (only via dumping SQL statements and re-importing them) .

That said, the latest MariaDB 10.6 branch is very fast without having to tinker with the .ini file. Take a snapshot of your current Windows configuration with system protection or even better make a full backup so you can go back if things are not to your liking.

[zebraitis]

I appreciate the thoughtful response.

All other problems aside, how did you get back to Win10? If you did something other than recreate it exactly from a backup image you took before you might be in for a a world of problems in the future - not only with Kodi. If you didn't go back in a way that eliminated the whole experiment you might probably be better off by re-installing everything from scratch.

As I had been lulled into a false sense of security with the very smooth Win10 patches and upgrades of late, I had thought that Win11 would be a breeze.  As a result I had not made a backup of any type (that included NO backup of the MySQL DB data).

When I realized that I could not just undo, and that I had no choice but to make a quick copy then (to save some key personal data), format the C Drive and do a complete reload of everything (Win10, MySQL, Office, etc.) from scratch.  So my current install is completely fresh since that experience.  Loaded from original software, not from a backup or clone.  
 

I doubt that switching from MySQL 5.x to MariaDB 10.x would be a solution in your case. Normally both should work reasonably well with only some differences in speed due to different versions although I have only first-hand experience with Kodi 18+ and MariaDB.

Kodi 18.9 is working right now smoothly with MySQL 5.7 

But before this all happened...  my experience was that Kodi 19.x scans were substantially slower following the Kodi 18.9 to 19.x db upgrade.  And while it was a drag, I was living with it.

But then I ended up doing the Win11 upgrade.  And following my Win11 debacle, once I did that complete reload back to Win10 I just could not get Kodi 19 to do a complete fresh scan of data without the MySQL service crashing in Win10, requiring a reboot to get MySQL to just function as a DB. 

I feel that was the key difference:  The Kodi managed MySQL DB upgrade (18.9 -> 19.x) allowed the DB version to move up.  and whatever was causing that scan problem did not show up that way...  But in redoing every thing from scratch and trying to recan, it just would not complete it in 19.

So that is why I fell back to 18.9 on al the devices that were attached.  So that I could get that scan to complete.  It would not (could not) in 19.  No matter of how many times I tried.

That is the core of my hesitancy in moving to Kodi 19.1x today.  I am not sure if the incompatibility is anchored with a problem in KODI 19.x or if it is MySQL.  My gut feeling is that it is Kodi 19 and the new DB structure. 

I guess that I would like to have the team say:  Your best path forward would be to do _________________________  (fill in the blank)

And then if it works, great, and if not then we can have a baseline before and data after and that can add value to troubleshooting the possible underlying issue.

As much as MariaDB is called a drop-in replacement for MySQL that's not exactly true. Both products have seen some development and have unique features that are not 100% compatible in their latest branches. AFAIK MariaDB will import your current database via a nice assisstant and convert it to its own structure. You'll have to be careful to keep the database name to "MySQL". Once converted to the latest 10.x format you'll not be able to go back directly to MySQL (only via dumping SQL statements and re-importing them) .

That said, the latest MariaDB 10.6 branch is very fast without having to tinker with the .ini file. 

Thanks for that info.  That makes me feel a bit more confident if recommended to go to MariaDB.

I have never messed with the MySQL ini file, as the wiki says nothing on that topic.

And I understand that in branches of products there will always be differences. 

But if there is not a key difference / reason that would benefit a Kodi implementation, I would like to stay with the DB that Kodi would like to recommend for this type of distributed implementation.

And that goes back to my previous comment:  I would like to have the team say:  Your best path forward would be to do _________________________  (fill in the blank)

Choice is great, but I want to make sure that I go down a path that Kodi is going as well.
 

Take a snapshot of your current Windows configuration with system protection or even better make a full backup so you can go back if things are not to your liking.

This is something I've learned through all this.  I had previously connected two drives in a mirror, but while that would be good if I had a hardware error and lost a drive, it did not address having a backup if I did something, well...  just flat-out stupid.

I have broke that mirror and now am using Macrium Reflect Free and clone the C drive entirely.  My drive C is copied to drive I.  and Drive I is the same size SSD as drive C.  So, I think that that part is covered.  

But if I do the upgrade to 19.1x to see what happens, that means that I will have to upgrade Kodi on several other machines and the time involved in doing that and then falling back if the whole thing goes south.  Which plays into the whole hesitancy thing again.

So:  willing to give it a try, and willing to share results with the team.  Just let me know which way to go.

[HeresJohnny]

Just as an afterthought: Kodi N runs fine here with MariaDB 10.6.x. I cannot say if MySQL would present any problems with Kodi N since I have not tried that specific combo. I also doubt that you will receive a thumbs up or recommendation for any central database solution from Team Kodi as that is and problably will forever be deemed experimental and thus is, unsupported. All info provided in the Wiki and the forums comes from lone but bold adventurers and must be taken as is, I'm afraid....