Kodi Community Forum
Solved Song sorting by dates (TDOR, TDRC, TDRL) not just YEAR - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Music Support (https://forum.kodi.tv/forumdisplay.php?fid=263)
+--- Thread: Solved Song sorting by dates (TDOR, TDRC, TDRL) not just YEAR (/showthread.php?tid=313952)

Pages: 1 2 3 4


Song sorting by dates (TDOR, TDRC, TDRL) not just YEAR - AnalogKid - 2016-08-23

[EDIT: Split to sperarate thread so I can find discussion - DB]
On the issue of Sorting Order...

There's been a long standing problem that I've never managed to crack...

If an artist releases two albums in the same year, there's no easy way to chronologically show one album before the other.


RE: Song sorting order - jjd-uk - 2016-08-23

The id3 standard has a date "TDAT" tag which should be used in the form "ddmm" so in theory we could use that in conjunction with year "TYER", looks like Picard can write these from Release Date in Musicbrainz info https://picard.musicbrainz.org/docs/mappings/


RE: Song sorting order - jjd-uk - 2016-08-23

Would also be nice if we supported the sort tags:

Album sort "TSOA"
Title sort "TSOT"
Artist sort "TSOP"
Album Artist sort "TSOP2"


RE: Song sorting order - AnalogKid - 2016-08-23

(2016-08-23, 18:32)jjd-uk Wrote: The id3 standard has a date "TDAT" tag which should be used in the form "ddmm" so in theory we could use that in conjunction with year "TYER", looks like Picard can write these from Release Date in Musicbrainz info https://picard.musicbrainz.org/docs/mappings/

It's deprecated, in favour of TDRC as of v2.4.
Sadly most taggers just seem to support Year but I'm not sure if they all use TYER or if some use TORY. Still, it would be nice to have a solution for greater granularity that just year. It's small thing (to me) in the grand scheme, but I'm surprised more people have never mentioned it
!


RE: Song sorting order - scott967 - 2016-08-23

I use MediaMonkey as my tagger/manager. It uses the TDRC in ID3v2 yyyy-mm-dd and also TYER.

scott s.
.


RE: Song sorting order - AnalogKid - 2016-08-24

Does / can Kodi use TDRC at all?

It should probably use TDOR or TDRL though. I think TDRC is the time when the user recorded the track rather than when it was actually released (or at least that was the intention!).


RE: Song sorting order - jjd-uk - 2016-08-24

Not sure how reliable this is http://kodi.wiki/view/Music_tagging but it seems like we can read TDRL, don't know if we use it anywhere though.


RE: Song sorting order - scott967 - 2016-08-24

(2016-08-24, 13:43)jjd-uk Wrote: Not sure how reliable this is http://kodi.wiki/view/Music_tagging but it seems like we can read TDRL, don't know if we use it anywhere though.

Code:
else if (it->first == "TYER") tag.SetYear(strtol(it->second.front()->toString().toCString(true), nullptr, 10));
else if (it->first == "TDRC")   tag.SetYear(strtol(it->second.front()->toString().toCString(true), nullptr, 10));
else if (it->first == "TDRL")   tag.SetYear(strtol(it->second.front()->toString().toCString(true), nullptr, 10));

scott s.
.


RE: Song sorting order - DaveBlake - 2016-08-25

(2016-08-23, 18:36)jjd-uk Wrote: Would also be nice if we supported the sort tags:

Album sort "TSOA"
Title sort "TSOT"
Artist sort "TSOP"
Album Artist sort "TSOP2"

I have a branch that does just that Smile
But needs a little more work (and review) before it can merge into master. Sad
I had hoped to get it into v17, but since we are in beta it looks like it will be v18 now. To be widely useful it needs to be propagated via JSON API, and that is a slow and painful process in my experience.

As for date - yes Kodi reads the various tags, but currently only stores the year.

Biggest complaint about year is that so much music gets tagged with the relase year rather than the original recording year, and of course you may want both?


RE: Song sorting order - jjd-uk - 2016-08-25

Well TDRC is supposed to the recording date and TDRL the release date, so separate values can be stored, not sure on how widespread usage is on tagging software though.


RE: Song sorting order - AnalogKid - 2016-08-25

As a first implementation, the UI could stay as 'Sort on Date' whilst internally the code had some priority for which date to use? possibly configurable through advancedsettings.xml?

Off the top of my head I am imagining something like this:

<datedetection>
<tagpriority>
TDRC,TDRL,TYER
</tagpriority>
</datedetection>

This would effectively be instructing Kodi to use TDRC if available, else TDRL, else TYER.

I think it would be too confusion for most users to choose 'sort by year' 'sort by release date' 'sort by recording date' as there are possibly 4 or 5 date tags and their precise usage can be confusing.
There is probably a case for being able to define the priority under music settings UI in the future though, but the XML is probably good enough for the moment.

Also, for an Album Date, either this has to be detected when all album track dates match, or if it's supplied in the Album NFO (but again, most taggers / NFO creators only supply a year). What is Kodi going to do if there are differences in the dates for Album tracks? what should it assume the date for the Album is?

I would suggest something like this:

1) Read each track date tag (according to the priority order set in the XML) - thus you now have one 'date' for the track
2) When all dates have been read for all tracks, set about determining the album date:
2a) Set the album date to the latest date found on any of the tracks (latest date makes more sense than earliest date, because it's illogical to have an album date earlier than any child / track).


RE: Song sorting order - jjd-uk - 2016-08-25

Since it appears Dave will be introducing the sort tags then it makes sense to me to have the sort criteria configurable in the settings, so you could have:

Configure "Sort by Date" to use: Release date / Recording date

Release date uses TDRL and falls back to TYER if not present
Recording date uses TDRC and falls back to TYER if not present


RE: Song sorting order - DaveBlake - 2016-08-25

Like all your thinking AnalogKid Smile

To be clear while tagging offers a number of dates, Kodi currently only stores year (of the last date tag it encounters), and sorts by that. It seems to date from id3v1.1 that only has a year tag. We do need to think about other tag formats than just ID3. At the moment the date related tags Kodi processes and stores as year are

TDRL or TDRC (id3v2.4) - extracting year.
TYER (id3v2.3)
YEAR (APE or Vorbis or id3v1.1)
DATE (Vorbis)
WM/Year (ASF)
"\251day" (iTunes MP4)


In the ID3v2.4 standard it is clear what the dates are

Code:
TDOR
   The 'Original release time' frame contains a timestamp describing when the original recording of the audio was released.

  TDRC
   The 'Recording time' frame contains a timestamp describing when the audio was recorded.

  TDRL
   The 'Release time' frame contains a timestamp describing when the  audio was first released.

but I am not so sure about the other file/tag formats.

Musicbrainz Picard uses
release date: TDRC (id3v24), TYER+ TDAT (id3v23), DATE (Vorbis), Year (APEv2), ©day (MP4), WM/Year (ASF)

Original Release Date: TDOR (id3v24), TORY (id3v23), ORIGINALDATE (Vorbis), WM/OriginalReleaseTime (ASF)

Original Release Year: ORIGINALYEAR (Vorbis and APEv2), WM/OriginalReleaseYear (ASF)

Notice it does not use TDRL nor does it have every kind of date in every file format. The date section summary here http://wiki.hydrogenaud.io/index.php?title=Tag_Mapping is interesting too. What that shows is that we can expect to get a year but more than that isn't always available.

The years node depends on albums/songs table having a year field, so I propose we keep that but make it clear how that field gets populated when songs are tagged with several dates. I also suspect that sorting by an integer value like year is quicker than sorting by a full date. Sorting by a field requires every record to have a value for that field, so what to do if all we have is a year? Take it as 1st Jan I guess?

What I'm not sure about, given much older music will just have a year tag, is do we add 3 date fields (like the ID3v2.4 spec) or 2 (as Picard supports).


RE: Song sorting order - AnalogKid - 2016-08-25

I think the ideal option is to have it in the Music Settings UI. The XML option was suggested as a potential interim method if the UI option was difficult.

When it comes to the UI option though, I think it's more flexible to have a priority order rather than a specific option - that way, there's a graceful fallback However, I do understand this might be trickier to implement and represent in the UI (it would probably need a list with 'move up / move down' ability for the user to set the order).

One of the reasons / issues for having the priority order than just a simpler Release Date vs Recording Date is that the support for these tags in tagging software varies a lot. TYER is ubiquitous, but the others aren't. If the user can specify the specific tags, then it's more flexible.

Perhaps a compromise would be to allow a simple choice of TDRL, TDRC, TDEN, TDOR etc - followed by the fallback of TYER ?

Failing that, I agree with jjd-uk, if you have to keep it super simple, then just offer two options for date:

Option 1: Release Date, else Year
Option 2: Recording Date, else Year

Then the user has to find a tagger that supports those specific fields / frames.

Still need to work out how an album date is supposed to be deduced!


RE: Song sorting order - AnalogKid - 2016-08-25

(2016-08-25, 15:51)DaveBlake Wrote:
(2016-08-25, 15:05)jjd-uk Wrote: Since it appears Dave will be introducing the sort tags then it makes sense to me to have the sort criteria configurable in the settings, so you could have:

Configure "Sort by Date" to use: Release date / Recording date

Release date uses TDRL and falls back to TYER if not present
Recording date uses TDRC and falls back to TYER if not present

Happy with making this configurable, that is also how my use of artist sort name etc. implementation works. However it does get harder when it comes to passing this out to the JSON API. As I understand the principles from Montellese the API needs to be independant of the effects of user system settings that it can not change. That is it will need to be able to set whether it is using release date or recording date.

Do we follow Picard and have (release) date (TDRC/DATE) and original release date (TDOR/ORIGINALDATE), or do we also have TDRL and if so what Vorbis tag is equivalent?

The way sorthing works it needs a populated field of data, and is not able to fallback to another value in the middle of sorting, so we are talking about adding 2 (or 3) fields to the data stored and deciding how they are populated when the file tags are scanned. This also means a database conversion from Year values to full date values when upgrading.... Just thinking aloud really.

If there's no fancy SQL query, then yes, I guess you'd have to create a faux / deduced field (something like 'fauxDate'). I believe you'd only need to use one field for this though.
When a file is scanned, and all tags read, at that point, you're able to 'deduce' the date (based on your priority algorithm).
This raises a new issue - if the user changed the preference for which date fields to use, you'd have to go through the entire DB and refresh the 'fauxDate'.

If you allow the user to choose which field they want, then you don't have to worry about what Picard or any other tagger uses, although as you highlight, Vorbis comments aren't the same is ID3, and appear to be much more limited with only 'Date' as some sort of standard.

For the JSON API, can you not have a field which is 'Sorting Date' (i.e. your 'fauxDate') ?



p.s. If you're able to use a database query, you SHOULD be able to create a query that CAN prioritise the fields - but it might be a medium complexity query. For performance reasons, the 'fauxDate' might be easier. A few database purists might object to it though.
Using some medium complexity query should be able to generate the 'fauxDate' on the fly without having to actually store it in the database.