Improving Recently Added Albums
#1
[Split from https://forum.kodi.tv/showthread.php?tid=336750 as a topic I would like to discuss for improvements in v19. This is where your words went @HomerJau  Smile  - DaveBlake]

One of the things that drives me nuts when I do a new install is losing the Recently Added history (timestamp). I think I’ll look at solving that too (if it hasn’t yet been done?). I was thinking of updating the Kodi music db with the filesystem’s DateCreated which is probably what I want. Otherwise the initial source scan loads it’s timestamp based on the order it scans the enitire source which is alphabetic by artist and album folders names (my most recently added albums after a new install are always from ZZ Top Smile)
Reply
#2
Yes, recently added does have issues and things that could be improved.

They start because Kodi uses file modified timestamp for dateadded. This can be switched by an entry in advancedsettings.xml with  0 =  currrent time (when scanned) , 1(default) = modified, 2 = newer of creation and modification. Unfortunately only using <dateadded>0</dateadded> tells Kodi about when they were added to the library, the default use of timestamp does not. In an attempt to show the user the most recently added albums (not just their newest music files) the Recently Added Albums node does not use dateadded, it uses album ID instead. The most recently added items will have the newest ID, so even adding old files to the library they will show as recent.

Of course this implementation of the Recently Added Albums node means that when you completely rescan all your music files the last scanned (which is recursive through folders alphabetically) is shown as the most recent. Hence ZZ Top.  Sorting the albums node by dateadded will give you them in music file date order (newest file of all those in album).
 
Quote:I was thinking of updating the Kodi music db with the filesystem’s DateCreated which is probably what I want.
No that isn't going to help you. Kodi will keep giving the most recent thing it actually added no matter what date is stored.

In the case of a fresh install all items have been added to the library at (approx) the same time, what do you want recently added to show? I suspect in that situation what you want is file date order not recently added, but other occassions you may want recently added even if the files themselves were old. Or is it just me that adds old things to the library?

I would like to actually store the date music files are first added to the library as dateadded. This means a change to the current delete/insert approach to updates, which is significant work, but that could bring other benefits too. Then store file timestamp (created or modified?) too and offer sorting and filtering by both values.

Otherwise perhaps we rename "recently added" and dateadded as "newest" and "filedate", and use timestamp for everything?

I am interested to hear user views.

Meanwhile workarounds....
Because I hate a small tagging change (usually editing genre) placing the album at the top of dateadded order, I use a batch update tool (BulkFileChanger)  that resets file modified date to that of created. The option to have Kodi use creation date (as file date) would be useful
Reply
#3
Thanks very much for this post Dave. Lots of info I was unaware of.

I’ll digest over the weekend as pretty busy at the moment.
Reply
#4
Storing the date of the file stamp (last modified) or something like that would be very welcome but I'm not sure how this would work on all platforms?
Reply
#5
Hi Dave, thanks again for your detailed post.

The issue with losing correct recently added sort only occurs when I do a new install. I tend do do this on new Kodi versions and during testing too. I have a NUC with two SSDs, one used for test builds and one for current build, just swap them arround.

Based on your response I’ve created a workaround by building a SQL script that recreates the album table based on each album’s song table max dataadded value so the idAlbum primary key is in ‘correct’ order. The script then updates the idAlbum foreign keys in various tables (also removes old indexes then adds the indexes again at end).

This seems to do the trick.

I’ll post the script here once I’ve fully tested but it is only suitable for current Leia RC1 database. You may consider using this script in Kodi or a modifed version as part of the music dB upgrade process?

Is there something it can test for to ensure it won’t run on any other version. If not I can add it to from Music Media Helper and hardcode the dB name (MyMusic72.db) before it can run.

Index Problem:
One interesting thing is one of the existing indexes is a unique constraint on musiscbrainzid. I think this may cause issues for some users at some point. I could have a single disc with multiple same album for example: One a stereo mix version, or quad version of a BDA disc and one a 5.1 mix released on one disc. I think this needs to be changed to a non unique index.

THX
Garry
Reply
#6
I’ve tested the script and it looks like it’s working as expected.

However, after I did another scan there were additional updates (not sure why but I think the latest Leia scanning seems to find additional info). Of course, due to the delete/insert method used these existing albums now have new idAlbum keys so they now appear in recently added.

Without asking for this to be changed to an Update method I think I’m going to move the script into an app to update the dateAdded values in the song table to DateCreated (file time stamp)

Run Scan Library in Kodi
Run the Update dateAdded code
Run the Script above (recreate the album table)

This would solve my issue as I just want to see my Recently added albums and their song creation dates are always the most recent I’ve purchased.

Not sure if you have any further thoughts/comments Dave?
Reply
#7
I’ve changed the <dateAdded> setting to 2 as mentioned and rescanning the original dB. Then I’ll run the idAbum key update script. I think that will solve the issue I mentioned in my previous post.

I’ll post again once the rescan is finished and I run that script.
Reply
#8
(2018-11-11, 21:54)HomerJau Wrote: Index Problem:
One interesting thing is one of the existing indexes is a unique constraint on musiscbrainzid. I think this may cause issues for some users at some point. I could have a single disc with multiple same album for example: One a stereo mix version, or quad version of a BDA disc and one a 5.1 mix released on one disc. I think this needs to be changed to a non unique index.
Not sure what you are seeing as a unique MBID contstraint? I am aware that Musicbrainz Track ID may not be unique on an album, recordings are sometimes repeated e.g. "[silence]" or on a disc set, although I think the examples you give would have different Musicbrainz ID. Of course the same recording could also appear on different releases e.g. on a orginal albuim and on a Greatest Hits. Hence the unique index on the song table is idAlbum, iTrack, strMusicBrainzTrackID combined.

But thank you for thinking about it Smile

As for a script to recreate the album table messing with ID etc. ....
Well I would much rather get you putting your efforts into contributing to v19 than apps that meddle with such DB fundamentals. Do whatever you like with your own data of course, but please don't offer other users enough rope to hang themselves!
HomerJau Wrote: I just want to see my Recently added albums and their song creation dates are always the most recent I’ve purchased.
Right, you want a Recent Album (by file timestamp) node. Can almost create this using a smart playlist (or custom node), just need to add "dateadded" as a possible smart playlist sort order. No idea why it was left out before. That is a really easy change, like to try it?

Perhaps that is what most users want? Am I the only guy that adds old files to my library but wants to see them as recently added?
Reply
#9
(2018-11-12, 11:58)DaveBlake Wrote: Perhaps that is what most users want? Am I the only guy that adds old files to my library but wants to see them as recently added? 
 Nope not just you Wink

I always thought both methods should be available, File Timestamp AND Recently Added. They are 2 different things.
Reply
#10
(2018-11-12, 11:58)DaveBlake Wrote: Perhaps that is what most users want? Am I the only guy that adds old files to my library but wants to see them as recently added? 
 No you're not the only one, thats what I want as well. I want to see when items are added to the library not the underlying timestamp, although at times I want to see the timestamp as well but for that I just use windows explorer
Reply
#11
Dave. Thanks for your post.

Here is the Unique Contraint om MusicBrainzID:

CREATE UNIQUE INDEX idxAlbum_2 ON album(strMusicBrainzAlbumID)
Reply
#12
(2018-11-12, 22:01)HomerJau Wrote: Here is the Unique Contraint om MusicBrainzID:
CREATE UNIQUE INDEX idxAlbum_2 ON album(strMusicBrainzAlbumID)
Ah, album having unique MBID.... strMusicBrainzAlbumID is Musicbrainz Release ID which will be different for each stereo mix, quad, BDA and 5.1 mix releases (whereas album title, artist(s) and year will be same). Admittedly users probably misuse Musicbrainz tagging, picking any release that has the right number of tracks rather than ensure they have the right mix or add a Musicbrainz entry if it doesn't exist yet. But if you are a person that has that kind of multiple mix collection then you are also behoven to tag that accurately too.

It is because of using Musicbrainz Release ID that Kodi music lib can contain multiple mix versions of an album. Make sense to you now Gary?

EDIT: Kodi also stores Musicbrainz Release Group ID which is non-unique but used to look up art and common album info
Reply
#13
Thanks again Dave. Yes, I'm one of those that uses the album release MusicBrainzID incorrectly. I'll need to be careful now I know it needs to be unique!

I almost always use a 3rd Party iOS app to play my music, occasionally the official Kodi iOS remote. I would imagine their interfaces use an API/JSON call to display Recently Added. So I assume a new Recently Added menu node wouldn't help with those apps. Does this sound right?


BTW: The script to reset the AlbumIDs is working. I've just reset mine to give me the Recently Added view I want (based on Created/Modified date). This 'repaired' the Ids changed with the updated albums (the database DELETE/INSERT method). Basically I've done a new Kodi install (new drive) and I now have the Recently Added order I had on my old install.

If you want I'll PM the script to you to have a look at. I feel the method I've used to build the IDs based on create/modified date gives a 'better' result than building a new database alphabetically, but it needs the album's songs loaded first and rebuilding the music db would add a bit more time to the initial scan so I'm not expecting you will do anything with it.
Reply
#14
Yeap Gary, add those different mixes of an album to Musicbrainz as more releases Smile
 
(2018-11-13, 06:10)HomerJau Wrote: I almost always use a 3rd Party iOS app to play my music, occasionally the official Kodi iOS remote. I would imagine their interfaces use an API/JSON call to display Recently Added. So I assume a new Recently Added menu node wouldn't help with those apps. Does this sound right?
I always play music with TV off too. Actually some of the apps can access smart playlists, for example Yatse does, but no not custom nodes. But apps take various approaches: some make a JSON call for every display and hence would use GetRecentlyAddedAlbums JSON call to get exactly what the Kodi GUI default node gets. Others - including Yatse and Kore - make their own copy of the library so can show whatever they choose as the recently added albums (and probably use datadded). I'm not familiar with iOS apps but would suspect the same range of behaviour as the Android apps.
 
(2018-11-13, 06:10)HomerJau Wrote: If you want I'll PM the script to you to have a look at. I feel the method I've used to build the IDs based on create/modified date gives a 'better' result than building a new database alphabetically, but it needs the album's songs loaded first and rebuilding the music db would add a bit more time to the initial scan so I'm not expecting you will do anything with it.
I understand your enthusiasm to use the skills you have to get things looking as you want the quickest way you can, but again would encourage you to apply yourself to Kodi itself. The id an item has really should not matter, unfortunately it is currently having an undesireable impact in some situations for some users. Best to rework those situations and support user requirements than try to define an ID order and set it in post processing. I worry for your db that you have updated all the album ID in link tables, but you are a big boy so it is up to you.
Reply
#15
Thanks again Dave.

I hear what you are saying on a long term solution vs my hack. I’d like to take some time to help but still too busy here at the moment.

I’ll have a think about this while I’m painting tomorrow Smile
Reply

Logout Mark Read Team Forum Stats Members Help
Improving Recently Added Albums0