Solved Issue with MySQL & large music libraries resulting in duplicate artists
#9
@DaveBlake - I captured kodi logs and JSON output showing the issue, first with 17.6 and then with the latest Leia 18.0 nightly

Unfortunately, Leia still shows the exact same problem as 17.6.

Both 17.6 and 18.0 are sending to MariaDB the exact same SQL query.

The resulting files are here:   https://filebin.net/9qd3lxnzwmiqaorg

There are two folders, one with 17.6 results and the other one with 18.0. Each test set has the JSON output (3 files, one per JSON call / table chunk, and the kodi log corresonding to the last two calls (since the duplicates always happen in the last two chunks, this is consistent)


I do the JSON queries like so:
Quote:/usr/bin/curl --user kodi:kodi --header 'Content-Type: application/json' --data-binary '{"id":609,"jsonrpc":"2.0","method":"AudioLibrary.GetArtists","params":{"albumartistsonly":false,"properties":["style","description","born","died","thumbnail","instrument","genre","fanart","songgenres","isalbumartist"],"limits":{"end":750,"start":0}}}' http://192.168.122.119:8080/jsonrpc  > kodi-17.6-artistsPart1.json

Part1: {"end":750,"start":0}
Part2: {"end":1500,"start":750}
Part3: {"end":2250,"start":1500}

Where Part1/2/3 are the 3 chunks

Sample repeated artists always show in the 2nd and 3rd chunks, there are about 30 of them, examples: "The Velvet Underground", "Santullo", "Rosario": they can all be found in both chunks (#2, #3)



17.6, for Part2, JSON request and corresponding SQL query
Quote:00:05:42.942 T:139754102224640   DEBUG: JSONRPC: Incoming request: {"id":609,"jsonrpc":"2.0","method":"AudioLibrary.GetArtists","params":{"albumartistsonly":false,"properties":["style","description","born","died","thumbnail","instrument","genre","fanart","songgenres","isalbumartist"],"limits":{"end":1500,"start":750}}}
00:05:42.982 T:139754102224640   DEBUG: GetArtistsByWhere query: SELECT artistview.* FROM artistview  WHERE ((EXISTS (SELECT 1 FROM song_artist WHERE song_artist.idArtist = artistview.idArtist AND song_artist.idRole = 1) OR EXISTS (SELECT 1 FROM album_artist WHERE album_artist.idArtist = artistview.idArtist)) AND (artistview.strArtist != '')) AND (artistview.strArtist <> 'Various artists') LIMIT 750,750


18.0 Leia, for Part2, JSON request and corresponding SQL query
 
Quote:00:25:16.716 T:140208452392704   DEBUG: JSONRPC: Incoming request: {"id":609,"jsonrpc":"2.0","method":"AudioLibrary.GetArtists","params":{"albumartistsonly":false,"properties":["style","description","born","died","thumbnail","instrument","genre","fanart","songgenres","isalbumartist"],"limits":{"end":1500,"start":750}}}
00:25:16.759 T:140208452392704   DEBUG: GetArtistsByWhere query: SELECT artistview.* FROM artistview  WHERE ((EXISTS (SELECT 1 FROM song_artist WHERE song_artist.idArtist = artistview.idArtist AND song_artist.idRole = 1) OR EXISTS (SELECT 1 FROM album_artist WHERE album_artist.idArtist = artistview.idArtist)) AND (artistview.strArtist != '')) AND (artistview.strArtist <> 'Various artists') LIMIT 750,750


Could this help as a fix:  when a table segment is requested (as opposed to the entire table), would it be possible for Kodi to do some kind of SORT of the table before splitting it into chunks. That would guarantee that there will never be repetitions with the same artist showing in different chunks, I think.
Reply


Messages In This Thread
RE: Issue with MySQL & large music libraries resulting in duplicate artists - by htpcero - 2018-06-22, 06:48
Logout Mark Read Team Forum Stats Members Help
Issue with MySQL & large music libraries resulting in duplicate artists0