[fiveisalive]

Just chiming in from a user perspective, would these changes allow albums with multiple artists to be grouped under each individual artist? (e.g. this is how MusicBrainz's NGS works). The way I see it, using MBIDs for the artist has three primary uses from a UI/artist browsing point of view.

The first is to have a have a single listing for a primary artist (in the MusicBrainz sense), this gets around the minor variations problem, e.g. "Eno", "Brian Eno" would be listed under a single "Brian Eno", as it is on the MB site:

http://musicbrainz.org/artist/ff95eb47-4...c30f02e872

The second would to allow cross-listing of albums under multiple artists, e.g. the album "Spinner" by Brian Eno & Jah Wobble:

http://musicbrainz.org/release-group/3e5...792874ef79

is displayed under both Brian Eno and Jah Wobble on MB. It would be nice if this was mirrored in the artist hierarchy in XBMC. Of course, there should probably be an option that could switch off the artist MBID mapping, so that the user could default back to the textual representation and ignore the MBIDs for artist groupings if they wanted to opt-out.

The third is to disambiguate identically named artists, perhaps the scraper could pull the "disambiguation" field and optionally display that, e.g. MBID has 5 artists called "Lino":

http://musicbrainz.org/search?query=lino...od=indexed

Of course, in the actual album view (and lists), it would show the actual artist name as it appeared in that release. The Artist MBID would only be used for the grouping purposes (and the standardized name for the Artist MBID in the top-level list of Artists).

[jmarshall]

XBMC already supports listing an album under more than one artist, it's just that currently it's done using the strings in the (album)artist tags. So yes, using the musicbrainz ids allows better disambiguation.

[Martijn]

XBMC already supports listing an album under more than one artist, it's just that currently it's done using the strings in the (album)artist tags. So yes, using the musicbrainz ids allows better disambiguation.

Could we do the same for record label perhaps? I know it's not that important however there are quite a lot record labels that have the exact same name and are totally different from each other (different coutry and music genre)

[night199uk]

Yes, I want to get record label in too... :-)

A quick update, a lot of work on this.
I've added proper Compilation flag support and I'm just finishing that off, I've converted the exalbumartist and exsongartist tables and updated all of the major dependencies.

This should help me to clean up some of the music database stuff and get acquainted with the codebase..

I much better idea of how I need to add the MusicBrainz IDs now.

[night199uk]

Hey.. I think I got the first set of music database patches to the point where I'm happy with them...

These are just foundation stuff at the moment.. Would appreciate your feedback on Pull Request #97.

Basically it:
- Deprecates / Removes the ex* tables, and creates link tables for Song->Artist, Album->Artist, Song->Genre, Album->Genre
- Removes idArtist, strExtraArtist creates strArtists (you'll see all of your artists not just the first)
- Removes idGenre, strExtraGenre creates strGenres (ditto)
- Fully support the Compilation flag and uses the Compilation flag instead of detecting Various Artists releases

The commit history isn't as ordered or structured as I'd like as I was feeling my way through and I made a few mistakes in the way I ordered commits - adding the Compilation flag halfway through, and MBIDs to the tables which made it very difficult to reorder some commits. If I had to do it again it'd be more structured, but I think the end result is good. My artists table looks a lot cleaner with the Compilation flag support instead of the way Various Artists worked before.

I'm going to be running this version on my media centre with 50,000 odd music files to test it further. Need to look at some of the queries with the new table structure - GetArtistsNav() query currently takes around 2.7secs for me to run even with the indexes in place, but I think this query can be optimised to a SELECT /JOIN rather than SELECT .. WHERE idAlbum IN ... I also wonder if some of the GSoC work going on will make this less of a concern?

Let me know your thoughts. If this is good the second part will be to start fixing up some of the GUI / FileItem elements to try and ensure we always query by ID and make use of some of the new queries, rather than query by strings.

You will need to rescan your music to take advantage of the Compilations flag...

---

Pull Request 1040, even :-)

[Martijn]

Will definitely give this a try

[night199uk]

Great :-)

Two things I noticed while doing this:
1) The AllMusic scraper is broken (ergh)
2) The Universal Scraper can query MusicBrainz too fast... When I scan my library MBs web service starts rejecting my queries as they seem to have a rate-limit at 22queries / second and reject anything above that. I think we might need to look into implementing a rate-limit per-domain on queries...? Is there already one?

[night199uk]

I've been looking at the GUI side of this for a few days trying to understand the best direction to go with this and I had a few questions..
Wanted to ask a few questions:

The musicdb:/3/1 style URI schema.. is that coming or going? I notice music FileItems use their real path but then later we convert that to a musicdb://3/1 style URI just to pass through the ID to the schema and get the scraper for the path. It seems like in the Video DB, either the files have the videodb:// style schema in the first place or we use a DB call to get it from the file path?

The VideoDB GUI/database seems to work almost entirely from FileItems, rather than the music code which is a mix of paths, strings, Album and Artist type structs and FileItems. E.g. for GUIWindowMusicBase we take an Album or Artist from the DB and convert it to a FileItem for listing... In GUIWindowMusicBase we then convert it back to an Album or Artist, then just extract the strings from it for FindAlbumInfo and find the album and then pass an Album to MusicInfoDialog.... I think it would make sense to try and tidy some of this up for consistency and use one method throughout.. That could be FileItem, or I could propagate Albums/Artists through or MusicInfoTag... Then there is also MusicAlbumInfo which is propagated to the GUI code in some places I think unnecessarily.. I feel this just needs to live in DownloadAlbumInfo for showing the Selection dialog.

Wanted to get a few ideas around this... it won't change the functionality as I think I can make this work with any of those schemas, it's just to understand which way people envision this working.

[night199uk]

Also what do the SetPropertiesForFileItem, SetPropertiesFromArtist etc calls do? There seem to be text based properties on FileItems, but I can't find where these are actually read or used. Can you point me to a reference please?

[jmarshall]

They're read directly by the skin - the former in lists, the latter most likely in the artist info dialog.

[fiveisalive]

OK, so I patched my system, and removed my library to test. I rescanned a single directory with this release:

http://musicbrainz.org/release/c7c5e6b3-...92595c8c19

It's a release with 2 artists: "Technical Itch" and "Kemal", I expected that now with using the MBIDs that I would see two separate artists, with the same release listed underneath them, but I still see one single artist. I guess that part isn't implemented yet. (This is the killer feature for me, as well as disambiguating identically named artists).

I should mention that I've already tagged the files with picard and they already contained MBIDs etc. Here's the contents of the db:

Code:

$ sqlite3 MyMusic34.db
sqlite> select * from artist;
1|Technical Itch & Kemal|
sqlite> select * from artist;
1|Technical Itch & Kemal|
sqlite> select * from album;
1|The Calling / Signal Trace||Technical Itch & Kemal|Drum & Bass|2003|0|1
sqlite> select * from song;
1|1|2|Technical Itch & Kemal|Drum & Bass|The Calling|65537|470|2003|3244260757l|01 The Calling.mp3|6dfce1c7-7a6b-48da-bc40-a9c6ffe170bf|79f48fec-2ef2-431a-ba4d-9a5ea06bb655|c7c5e6b3-9feb-4e2e-8231-9e92595c8c19|79f48fec-2ef2-431a-ba4d-9a5ea06bb655||0|0|0|0|1||0|Media Jukebox: Tool Name
2|1|2|Technical Itch & Kemal|Drum & Bass|Signal Trace|65538|486|2003|2798081215l|02 Signal Trace.mp3|9b23e165-5aad-48c2-9a4b-057d325f4d4b|79f48fec-2ef2-431a-ba4d-9a5ea06bb655|c7c5e6b3-9feb-4e2e-8231-9e92595c8c19|79f48fec-2ef2-431a-ba4d-9a5ea06bb655||0|0|0|0|1||0|Media Jukebox: Tool Name

[night199uk]

Yes, the MusicBrainz IDs/disambiguation are not done yet. I need to clean up the GUI code as there are a bunch of areas we rely on artist/album name that will break if we add duplicate artists right now. But I wanted to get some direction from the XBMC team (See 3 posts up), and also I'm semi waiting for the first round of patches to go in first and get the feedback from that cycle so I don't repeat any mistakes and to manage the workload/support concerns. This PR contains all of the required database changes though so the support concerns for the next set of patches should be significantly smaller.

Given the feedback so far I suspect there are maybe 3-4 more PRs to get this done:
1) GUI code cleanup
2) Any other disambigutation/uses of GetArtistByName etc.
3) MusicBrainz into the DB
4) Scraper code

If I don't get any direct feedback in the next day or so I'll just go ahead and put a PR with it done my own (using CAlbum/CArtist).

As a side note while I'm writing, in the latest upstream/master the Universal Music Scraper is broken for some of my artists (stack overflow due to recursive calls in Run/RunNoThrow in ParseMBDiscography). This happens with a fresh music DB on a clean upstream/master (I had to go check as I figured it was my code). I need to go file a report on the Universal Music Scraper threads but the bug looks to be in core.

[Martijn]

About the stack overflow (i think):
http://forum.xbmc.org/showthread.php?tid...pid1129456

Core code isn't my expertise but sure hope some one will look at this.

[night199uk]

Thanks Martijn, thats the one I saw. It seems he's fixed the scraper, but I think it really tickled a bug in core. I spent 2-3 hours with this up in the debugger last night but I'm not aware enough of the deep internal scraper code yet to find a fix myself. And I hate XML parsing code :-)

[Martijn]

Did you also link the scrapers to the database or is that something that needs some more work?