[fantasticn]

Hi guys,

after updating to Gotham I had to take a closer look into into musicbrainz, because (unfortunately) XBMC seems to scan for all Musicbrain-data stored in the MP3-tags and makes use of it - no matter what other information might be included in other tag fields. It also seems that XBMC is scraping and using those Musicbrainz information no matter what kind of scraper you use (even if you select the scraper "local information only").

At first I tried to comply with this new feature and benefit from it. So started tagging my (already tagged) MP3-Files with the Musicbrainz-Infos using Picard. But I came across the following issues:

1. Some of my albums include songs that are not officially included in the retail version, like "Bonus-Tracks" or tracks that I just added myself because they somehow fit into the album. E.g. I have the 12-inch version of Depeche Mode's Strangelove and I also have some remixes like blind-mix and pain-mix that I added to that album. My tags do show "Depeche Mode" as Album Artist and "Strangelove (12-inch)" as album for all included songs. But Picard does not seem to recognize it that way and adds different IDs for those songs. Therefore the songs are finally mixed up in my XBMC-library.

2. Musicbrainz sometimes has different IDs for the same album or even same artists, causing files to be scattered throught my library. E.g. I Anna Vissi suddenly was listed four times as a different album artists, each carrying different albums. Some albums just showed part of the songs that were "physically" included while the rest was listed under the same album name but within another "Anna Vissi"-column.

3. Musicbrainz - of course - cannot identify compilations (= albums with various songs of different artists) that are selfmade. But like a lot of people I creat my own compilations for certain subjects, like "UK Top 100 of 2012" or "Various Hits of the 80s". Musicbrainz tries to identify the included songs individually and assigns each of them to its separate "album" that Musicbrainz deems to be its correct origine. Some of the songs will not be identified at all and therefore do not get any kind of assigment.

After a lot of trying I decided that I will not tag my songs with Musicbrainz-IDs because - besides the amount of work to do it - I cannot assure any file getting a reliable ID and therefore I also cannot avoid messing up my whole database. Unfortetely Picard did not only add the Musicbrainz IDs but also replaced all other tags of the files. Therefore I had to restore my music-folder's backup and start over again. I then used mp3tag to scan all tracks and made sure to delete all Musicbrainz-ids that were possibly included in some of my files.

Afterwards I had XBMC scraping the library again by using Universal Artist Scraper and Universal Music Scraper. Basicly everything went well and the database was built up the way I wanted it, based on the information provided in the respective tags for album artist and album. But unfortunately a lot of artists and albums were not supplied with any kind of biography or information by the scrapers this way. E.g. ABBA, Adele or ABC - all famous groups or artists - were shown with empty biographies and their famous albums were shown with empty album information (though their names were spelled correctly).

After some research in the internet and especially in this forum I found out that you should be able to include additional information in files called "artist.xml" (stored in the artist-folder) and "album.xml" (stored in the respective album folder). But I was not able to find any kind of tool automatically pulling thos infos from the internet and creating the XML-files based on the names of the artist- and album-folders. So this was no way to proceed,

Next I found out that you should be able to just provide the respective Musicbrainz-links within those XML-files and the Universal Artist Scraper and the Universal Album Scraper would download those information from Musicbrainz themselves. Again I was look for a tool to do the jov, but could not finde one. So I decided to do it manually. I created an "artist.xml"-file for every artist containg just the http-link to the artist-page at musicbrainz-org and stored it in the artist folder. I also created an "album.xml" for every album containg just the link http-link to the album-page at musicbrainz.org and stored it in the album-folder. Then I had XBMC scrape music-files again and came across the following issues:


1. The artist.xml seems to be ignored by Album Artist Scraper. Having just this xml-file in the artist-folder (and having no album-xml-file in the subfolders) did not get me any kind of biography or artist-infos.

2. Adding the album-xml-file to the album-folders would result in not only getting album-infos, but also the artist infos. Therefore artist.xml-file seems to be ignored totally.

3. The scraped information will not only be used for the requested purpose (providing biographies and infos about the artists and albums), but also for naming and sorting of the artist and the album within the database again. Therefore I ended up having about the same problems again that I had when I used to tag the MP3-files with Picard (especially issues no. 2 and 3 as outlined above).

So do you have any suggestions as to solving my Problem? I would like to have my database built up on the album and artist infos I have included in the respective tag fields. I would like to get additional artist and album information pulled from the internet by the scrapers based on the names included in the tags or - alternatively - the musibrainz-iDs that I might include in an artist.xml and album.xml-file, but without using them for sorting and naming.

At least it would be fine if Universal Artist Scraper would only use the artist.xml file and I thereby could get biographies and artist-infos without having to also include an album.xml-file in the subfolder (because the Musicbrainz-Album-IDs are the major reason for the creation of the database-chaos).

Brgds,

FantasticN

[jmarshall]

If common artists are not being scraped, then that's a bug.

Supply clear details on how to reproduce, ideally with a sample mp3 file that shows this behaviour.

[fantasticn]

Hi jmarshall,

thanks for your reply. Of course I cannot supply you with all the MP3-files (that's too much data and would not be allowed from a legal perspective). But it should be easy to simulate:

Step 1:
Copy two albums of any artist to some folder. Name the folder "ABBA". Name the albums "Gold - Greatest Hits" and "More Gold - More Abba Hits". Now Tag the MP3-files in the albums with Artist "ABBA" and Album Artist "ABBA" and the respective names of the albums. I think you do not actually need to adjust the titles and rename the files. The Scraper will probably just look for the tagfields "album", "album artist" and maybe "artist". You can also add covers and fanarts for the artist and the albums (but this should not affect the results in any way either). The only thing you have to make sure is that you erase ALL MUSICBRAINZ Tag in the MP3-Tag-fields(!). Otherwise the Scraper would make use of them and totally ignore the tags like album and album artist.

Now start XBMC, select "Universal Artist Scraper" and "Universal Album Scraper" with standard settings (I also have it set to "de" for the preferred language setting, but it's falling back to english anyway when no biography is found in German language). Now scrape the folder. You will surely get the Artist "ABBA" and the Album "Gold - Greatest Hits" and "More Gold - More ABBA Hits" in your database. But they should be shown without any infos/biography.

You can also try it with Adele (Albums "19" and "21") or ABC (Album "How to be a Zillionaire"). I could continue with a lot of famous artists and albums. In my database only about 50% of the artists and their albums get recognized by their names.

Step 2:
Now you can try to edit a new file called "artist.xml", include just the http-address of the musicbrainz-webside for ABBA and put it into the artist folder. Then scrape again.
Result: Nothing. You are not getting any infos as to ABBA either. Universal Artist Scraper seems to ignore the xml-file.
You can also try it with the other artists I suggested.

Step 3:
Now do also create "album.xml"-files for the albums and have each including the link to the musicbrainz-webside for the respective album. Then put them into the album folders and have them scraped again.
Result: Suddenly you get infos for the artist ABBA and the two albums.

Step 4:
Now change the artist.xml-file and have it including a musicbrainz-link to a totally different artist (let's say Depeche Mode). Also edit the album.xml-files and have them include musicbrainz-links to totally different albums (let's say "Violator" and "Song of Faith and Devotion"). But do not change the tags of the included MP3-files. Now scrape the folders again.
Result: You get a listing for Depeche Mode and the albums Violator and Song of Faith and Devotion with the songs from ABBA included. You will not get a listing for ABBA and their two albums, though the tags in the MP3-files still have their names included. The musibrainz-links in the album.xml-files obviously get priority.

Conclusion
Even famous artist and albums are not always recognized by their names included in the respective tag-fields of the MP3-files. If there are problems in identifiying you can include the musicbrainz-links into artist.xml and album.xml files. But there are two problems with this methode:

1. Artist.xml seems to be ignored totally by Universal Artist Scraper whereas album.xml will identify the album AND the Artist

2. Identification of the Artist and the Album via album.xml will unfortunately not only lead to grabbing the respective infos (like biography and/or album description), but also replace the infos taken from the tags "Album Artist" and "Album" from the respective MP3-files. In certain cases this will mess up the database as described in my former post. I would really appreciate if links to musicbrainz.org included in the album.xml and artist.xml would only be used for dragging the respective infos but not also influence the sorting.

Solution:
- Step 2 (as shown above) should result in already getting infos for the artist "ABBA", but not for the albums.
- Step 4 (as shown above) should result in still having the artist listed as "ABBA" and the Albums as "Gold - Greatest Hits" and "More Gold - More ABBA Hits" (because that is what the tag-fields say), and only showing the infos for Depeche Mode and their respective albums (because those were linkes in the album.xml and artist.xml files).

Otherwise you will never be able to "force" a certain set of artist and artist-info, or album and album-info.

Brgds,

FantasticN

[zag]

Nice feedback thanks, I'll do some testing now on the local artist XML files.

Moving to MusicBrainz support is a big change and is bound to have some side effects but it will make scanning much more accurate in the future.

[zag]

I can confirm that the artist.xml containing an artist MusicBrainz URL does not seem to be overriding the online lookups.

[night199uk]

@jmarshall: you added an option for this, right? ('Allow online data to override music tags' or something like that?)
Can you turn off that option and test again?

Essentially, the DESIRED behaviour is:
1) If you don't have MusicBrainz tags and you have the option turned off, you should see the exact same as you saw in Frodo
2) If you don't have MusicBrainz tags and you have the option turned on, you should see the exact same as you saw in Frodo
3) If you have MusicBrainz tags and you have the option turned off, you should see the exact same as you saw in Frodo
4) If you have MusicBrainz tags and you have the option turned on, you should get dynamic metadata from MusicBrainz. i.e. once you've MusicBrainz tagged your files once, your library and metadata is built dynamically from up-to-date MusicBrainz data, and later KEPT up-to-date and refreshed automatically so you always have good tags.

Essentially the option is pretty redundant but it should allow us to bring the feature in more smoothly. Or that's the idea.

Can you let me know if this option is on or off for you, and what behaviours you see in the above cases?

[olympia]

By the way artist.nfo and album.nfo... NOT .xml
...and did you enable download additional information in music settings? Without that xbmc is only scanning in the tags into the library.

[fantasticn]

@olympia
You are absolutely right. Sorry, it was just a typo in my post. Of course I meant and always used "artist.nfo" and "album.nfo".

@night199uk
Your referring to the option in Settings / Music / Database. I have it turned to "off" but its taking Musicbrainz.IDs anyways if they are included in the MP3-tags or in the album.nfo

I would say the best behaviour would be to ALWAYS use the infos supplied in the tags "Album Artist" and "Album" for display and sorting in the database as well as the tags "Artist" and "Name" for the songs. The Musicbrainz-ID - if any included in the tags or supplied in a separate file (album.nfo, artist.nfo) - should only be used for dragging the additional information.

I think there are three possibile scenarios:

1. There are no Musicbrainz-IDs included in the files or in separate artist.nfo/album.nfo (this might currently be the most likely case, because the avarage Joe might not even have heard about that service or how to make use of it yet). Then the Scraper needs to make use of the regular tag-files (album artist, album, artist and title) for naming and sorting in the database as well as dragging additional infos from the web.

2. There are Musicbrainz-IDs included in the files or in separate artist.nfo/album.nfo and they comply with the infos provided in the regular tag fields (this is the ideal case). Then using MusicbrainzIDs or regular tag fields for Scraping would lead to the same result. So why use the Musicbrainz-IDs then for naming and sorting? Just remain with the regular tag fields and use the Musicbrainz-IDs only for grabbing the additional ARtist- and album-infos from the web.

3. There are Musicbrainz-IDs included in the files or in separate artist.nfo/album.nfo, but they do not comply with the infos provided in the regular tag fields. Since the tagger that applied Musicbrainz-IDs would usually have "corrected" the regular-tag-fields the only reason for that is that the owner of the files as set those tags manually to a different value. So he might have a good reason for that, e.g. the infos supplied by Musicbrainz were not accurate, or the owner is adding some songs to a pre-existing album that are usually not included in that album, or the owner is even creating a self-made compilation of songs (that's what a lot of people do, like "UK Top 100 Songs of 2013" , "Best Hits of the 80s" etc.). If you then use the Musicbrainz-IDs supplied in the files for sorting and naming, it will create kind of a mess in the database. THerefore it's almost essential to make use of the regular tag-fields, not of the IDs. You can maybe use the Musicbrainz-IDs for additional infos (e.g. artist-biography or album-info) if they really are the same for all files.

So the main advantage of the Musicbrainz-service is not for scraping the music files, but for tagging them. Because if somebody uses a tagger for including the Musicbrainz-ID into his MP3-files, the tagger will most probably use the IDs to also include the matching tags for album artist, album, artist and title at the same time. And if he does replace them afterwards with other infos, he might do it for a good reason (e.g. selfmade compilation, adding non-official titles to an official album, wrong infos supplied by musicbrainz etc.etc.etc.) so i think it's no good idea to force an override of these tags when scraping them into the database.

Brgds,

FantasticN

[night199uk]

Actually no. I disagree with this assessment of MusicBrainz tags - it's not how they were designed to be used.

MusicBrainz tags were designed to allow you to keep up-to-date metadata in your files, i.e. tags. Most MB users such as myself WANT the dynamic metadata -

MB tags are NOT a replacement for .nfos. They're for dynamic tags, not resolving metadata. The fact they help with the latter case is just a handy byproduct that we can use.

If someone added e.g. A featuring artist to a track online, I want my metadata kept up to date. I don't want to have to go retag for that. The idea behind MusicBrainz is that you tag once, and then use the DB for tags - giving you the power of 3000 editors to keep your tags clean.

Jmarshall was trying to add a halfway house to the PR like you suggested so we can use MB tags just for art/metadata but I don't really support that use case as it is fraught with problems - been there and seen that in earlier revisions - and I think that's what you're seeing now. I will take a look if I get chance.

If you don't want the dynamic tag feature you can turn it off or remove MB tags from your files (if someone is so keen on using their own tags surely they'd tag their own way anyway). If turning it off is not working, then there is a bug - that needs to be fixed.

[fantasticn]

Hi night199uk,

thanks for your statement on this which I really respect. However, I do not see the point in "dynamic" metadata. I actually cannot see what kind of "dynamic" there is supposed to be in those infos like artist, title, album and album artist or even date and genre of the respective song. Those infos are static at the day the song or the respective album is published. I also do not want it to be "adjusted" to something else afterwards.THat could probably cause a lot of confusion when I start my XBMC-PC, open my music libry and suddenly some artist or albums have been replaced. The only thing that might actually be dynamic is what you call the "metadata" (biographies of the artist, opinions and infos on the album etc.). That is why I suggested using the Musicbrainz-IDs for those.

However, including an option to ignore the IDs for sorting and naming in the database would be great. But you called it just a way of "bringing the feature smoothly". So I am afraid that this is just a temporary option and you will finally go for only using the Musicbrainz-ID anyway. That would be the day I would have to turn off the database and only use file-mode, because there is no way in having all my music only consisting of "official albums" that are totally in line with the data supplied to Musicbrainz, while this data itself is only built on user uploads and therefore does include a lot of mistakes as well. Like the tags of my movies and tv-shows I want to be in control of my database. If I name the Movie "Batman 3 - The Dark Knight rises" instead of only "The Dark Knight rises" I have a good reason for that and want it listed that way in my database. And if I create a music-album called "Mixed Songs of the 90s" and put in a bunch of songs and also tag them with this album name and the album artist "various artists" I am doing this for a good reason and want this album and its songs to be listed exactly that way.

Brgds,

FantasticN

[jmarshall]

Actually, it's quite simple. If you use musicbrainz data then you have 2 choices. 1. Use it for everything. 2. Use it help disambiguate artists and albums that are similarly named. This is what the setting "don't touch my damned tags" does.

It can't and won't work, however, if your musicbrainz tags do not agree with your normal tags. For example, if you use the wrong musicbrainz ID for a particular artist, then that artist will be named how it is in your tags, but the metadata fetched for that artist (stuff that isn't in your mp3 tags) will be from whatever artist your musicbrainz ID is pointing to - as that is the more accurate information. Another way they can be out of sync is if you have multiple musicbrainz artist id's in a song, but only a single normal artist name tag - obviously the second artist exists, but we have no idea what the name is (in this case you'll get duplicate artists with the same name - they differ as they have different MB IDs).

As long as your tags are in sync, all will be good - the naming you choose will be what is used, and the MBIDs (if they exist) will be used to disambiguate when fetching additional metadata (scraping).

[fantasticn]

Hi jmarshall,

absolutely. I totally agree with you as to this point. But it's a fact that there are several reasons why you sometimes actually want(!) the MB-IDs (for the fetching of metadata) and the tags (for the naming and sorting) to vary. This is especially the case when you add songs to a certain album that are usually not included in this album or if - as most people do ever since the tape recorder was invented - create your own "compilition" of songs. You want them to be listed with a certain artist and/or album, and you do not want them to be fragmented all over the database.

From a logical perspective there actually is no benefit in using the Musicbrainz-ID for scraping, but only for tagging. Because to have the Musicbrainz-ID included you always need to tag the file first. And when you already tag the files by using a tagger-program you will be able to also include the respective tag-fields automatically based on the same Musicbrainz-ID. So when you scrape the files afterwards you should normally get exactly the same naming and sorting by using the tags as by using the Musicbrainz-IDs. But the benefit of the tags is, that you have the option to change them if you like and thereby create a different naming and sorting in your database if you like (e.g, in the cases I just mentioned). So just using the Muscbrainz-ID will not grant you any quicker or easier process of integration. But you will bei bound to the fixed "package" of artists, albums and songs that you cannot modify anymore and that only follows the "original album organization" as listed at Musicbrainz, while by using the tags for scraping you will still have the freedome of combining artists, albums and songs as you like.

However, I do not want to insist on that matter too much. Of course it is up to you how to proceede. So I only hope you understand my point and may consider to at least preserve the option to ignore the Musicbrainz-IDs for sorting and naming (permanently, not only temporarily).

Btw: Since the option had not been working for me yet, has it been fixed now? Does the artist.nfo work again as well? (I actually do not want to sound "pushy" though I cannot avoid these question to look like that).

One again thanky you very much for your support and sorry for not totally agreeing with your opinion on the advantage of using Musicbrainz-IDs. Anyways XBMC is and will always be the best mediacenter software available!

Brgds,

FantasticN

[jmarshall]

Again, varying is OK as long as it means the same thing!

e.g. you might want "Rolling Stones, The" with the MBID for The Rolling Stones. That's perfectly acceptable, and is exactly what the option mentioned is for. It's not going to go away.

What won't work well is if you name the artists "Bob Saget" when the MBID is pointing to Bob Dylan. Bob Saget will still be listed in your library, but the metadata for Bob Saget will be that for Bob Dylan (just named Bob Saget).

If something isn't working, then I need sample files and detailed steps to reproduce. Usually a single mp3 file is enough. 9 times out of 10 I end up looking in the tags and spotting the error in the tags. Every now and then it's a real bug, and I fix it.

I will not address artist.nfo issues until we're sure the tag reading stuff is doing what it should. They're two completely separate things.

Jonathan

[fantasticn]

Yep, you are right. Providing a "Bob Saget"-Song with the ID of an "Bob Dylan"-Song would defenitly not make sense. But as long as I COULD do it if I'd like to and the song would then be listed under "Bob Saget" because of its tags (and not be listed under "Bob Dylan" because of ist ID) everything is fine. Because this means that I stay in control of my songs and my database and I can use this control to organize my albums the way I like them to be, which is necessary in certain cases (meaning all albums that do not content an "offcial" song set and that I have built-up myself, or all albums that somehow are not recognized correctly via Musicbrainz).

As to the "bug": I had my songs tagged with Musicbrainz-ID and then amended the tag-fileds (album artist and album) in certain cases where the MB-IDs did not "fit" or lead to chaos. I deleted the music-database-file and scraped the songs again with Universal Artist Scraper and Uinversal Album Scraper (the option "allow online data to override music tags" was always switched to "off"). But again the system was using the MB-IDs for soring and naming of artists and albums instead of the respective tags. When I now heared that the option was supposed to deactivate the usage of the MB-ID for sorting and naming, which it did not do for me, I thought we discovered a bug. The phenomenon should be reproducable with almost every song. Just tag it with an MB-ID and then tag the fields "Album Artist" and "Album" with different values. It shoudl take die MB-IDs for sorting and naming anyways. This does not seem to be depending on my specific MP3-files.

However, I will try it myself one more time and come back with the results.

Brgds,

FantasticN

[jjd-uk]

If you want the music scraping to make decisions based only on your tags, why do you have Musicbrainz ID's? I've not had chance to do a proper test yet but if you want to return to the behaviour before Gotham then simply delete any Musicbrainz ID's, at least that's my understanding on what I've been told.