2016-12-07, 20:44
It seems to be that the way Kodi currently does sorting by multiple fields is fundamentally flawed. A couple of users reported some odd inconsistencies in sorted results, and looking into it I can understand why Kodi is unable to produce consistent sorting. That is very unexpected from such a mature component of Kodi.
For those that don't know it sorting is not done in the databases using ORDER BY statements in SQL, but as explained by Montelesse in the big PR that implmeneted it back in 2012 (PR1047), so that smart playlists and default nodes can use the same sorting it is done in memory on the linked list of file items that was populated from the results dataset.
That is all well and good, but item order in the list is determined by combining the various fields of the sort into a single string value and comparing that alphanumerically. That results in inconsistent ordering.
The issue is best shown with an example from the music library, although the same sorting is also used by video. Say you have 3 compliation albums by "Various Artists" called "Back To The Old Skool", "Back To The Old Skool 2", "Back To The Old Skool Drum N Bass", and you view them from the albums node.
Sorting albums "by artist" is a sort by Artist and Album title, it uses a string <artist name> <album title>.
In our example that gives sorted strings
"Various Artists Back To The Old Skool"
"Various Artists Back To The Old Skool 2"
"Various Artists Back To The Old Skool Drum N Bass"
- the albums appear in the expected order.
Sorting albums "by album" is a sort by Album title and Artist, it uses a string <album title> <artist name>.
In our example that gives sorted strings
"Back To The Old Skool 2 Various Artists"
"Back To The Old Skool Drum N Bass Various Artists"
"Back To The Old Skool Various Artists"
- the resulting album order is unexpected.
How the albums get suffled with depend on the artist name, and how similar the titles. It is right to sort by title and artist, you could be looking at a list of albums and some have the same title but different artists e.g. "Greatest Hits". But the secondary sort order should not casue the albums to appear in the wrong order.
Multiple field sorting need the fields to considered separately, accurate item comparison should not be attempted by concatening these to a single string.
It should be possible to correct this by reworking SortUtils.cpp, it has the separate field values for the items it sorts, but just doesn't use them when comparing items. But since it is such a fundamental part of how the libraries work, I didn't want to rework anything without some input from others.
Perhaps there was a good reason for trying to sort things like this? Maybe @Montelesse could comment, although I know a lot happens in 4 years, and he is busy with other things. I guess I am looking at least for a sanity check, as I'm so surprized that sorting has been so flawed for so long unnoticed.
For those that don't know it sorting is not done in the databases using ORDER BY statements in SQL, but as explained by Montelesse in the big PR that implmeneted it back in 2012 (PR1047), so that smart playlists and default nodes can use the same sorting it is done in memory on the linked list of file items that was populated from the results dataset.
That is all well and good, but item order in the list is determined by combining the various fields of the sort into a single string value and comparing that alphanumerically. That results in inconsistent ordering.
The issue is best shown with an example from the music library, although the same sorting is also used by video. Say you have 3 compliation albums by "Various Artists" called "Back To The Old Skool", "Back To The Old Skool 2", "Back To The Old Skool Drum N Bass", and you view them from the albums node.
Sorting albums "by artist" is a sort by Artist and Album title, it uses a string <artist name> <album title>.
In our example that gives sorted strings
"Various Artists Back To The Old Skool"
"Various Artists Back To The Old Skool 2"
"Various Artists Back To The Old Skool Drum N Bass"
- the albums appear in the expected order.
Sorting albums "by album" is a sort by Album title and Artist, it uses a string <album title> <artist name>.
In our example that gives sorted strings
"Back To The Old Skool 2 Various Artists"
"Back To The Old Skool Drum N Bass Various Artists"
"Back To The Old Skool Various Artists"
- the resulting album order is unexpected.
How the albums get suffled with depend on the artist name, and how similar the titles. It is right to sort by title and artist, you could be looking at a list of albums and some have the same title but different artists e.g. "Greatest Hits". But the secondary sort order should not casue the albums to appear in the wrong order.
Multiple field sorting need the fields to considered separately, accurate item comparison should not be attempted by concatening these to a single string.
It should be possible to correct this by reworking SortUtils.cpp, it has the separate field values for the items it sorts, but just doesn't use them when comparing items. But since it is such a fundamental part of how the libraries work, I didn't want to rework anything without some input from others.
Perhaps there was a good reason for trying to sort things like this? Maybe @Montelesse could comment, although I know a lot happens in 4 years, and he is busy with other things. I guess I am looking at least for a sanity check, as I'm so surprized that sorting has been so flawed for so long unnoticed.
