Efficient music artist scraping
#46
(2016-03-31, 15:22)black_eagle Wrote: I don't think that (certainly initially) it would be for all artists, just non-album ones artists for whom there isn't a defined path. The current artist.nfo works pretty well IMHO but only allows for adding non-albumartists provided the user creates a directory structure for them. The problem (in my head anyway), is that this makes a mess of my neatly organised library as it becomes full of artists who I have no albums for. One directory containing all of them would (again in my view) be infinitely more desirable. It also allows those users without an artist/album structure to be able to make better use of the music library.

+1 Wink
Reply
#47
(2016-03-31, 15:22)black_eagle Wrote: The current artist.nfo works pretty well IMHO but only allows for adding non-albumartists provided the user creates a directory structure for them.
Huh
No it really doesn't. GetArtistPath does a query on the album_artist table

Code:
SELECT strPath FROM album_artist
JOIN song ON album_artist.idAlbum = song.idAlbum
JOIN path ON song.idPath = path.idPath
WHERE album_artist.idArtist = 516
GROUP BY song.idPath

And then takes the common path of all the ones it finds. You can create directories for non-album artist if you like, but Kodi will never find them for nfo because this query will return no records. For non-album artists GetArtistPath returns blank string, which could be taken as root and all kinds of oddities follow (remember Adam Ant Black_eagle). Even for album artists with a directory of thier own the attempt to locate this directory path from album and song data can go wrong.

I am guessing that the cdART addon does create extra directories for art work for artists that don't have one in your music file structure, and that is what has lead to this false memory.

Quote:The problem (in my head anyway), is that this makes a mess of my neatly organised library as it becomes full of artists who I have no albums for. One directory containing all of them would (again in my view) be infinitely more desirable. It also allows those users without an artist/album structure to be able to make better use of the music library.

Yes, I agree. It is the inconsistency of some artists having metadata being in artist.nfo in a folder above the albums (assuming it is found correctly), with others being in artistname.nfo in a nominated folder that worries me.
Reply
#48
(2016-03-31, 19:25)DaveBlake Wrote: No it really doesn't. GetArtistPath does a query on the album_artist table

Code:
SELECT strPath FROM album_artist
JOIN song ON album_artist.idAlbum = song.idAlbum
JOIN path ON song.idPath = path.idPath
WHERE album_artist.idArtist = 516
GROUP BY song.idPath


Yes, I agree. It is the inconsistency of some artists having metadata being in artist.nfo in a folder above the albums (assuming it is found correctly), with others being in artistname.nfo in a nominated folder that worries me.

Presumably then that this is the reason some users do things like this

(2014-01-29, 14:27)whitebelly Wrote: Hah, it is a bit messy of a process, but I don't change any files in XBMC.

A very simplified version, for a particular artist, is to create a text file called musicdb.xml with the following included (or you can put more that one artist in if you like)

<musicdb>
<artist>
<name>Leon Redbone</name>
</artist>
</musicdb>

Replace special characters (&) and (') with &amp; and &apos ----> ("Little Charlie & The Nightcats" becomes "Little Charlie &amp; The Nightcats")

Then import this file from "import music library" in settings. Once you do this, you can navigate to that artist name, and when you select Artist Information you will get to the GUI that allows you to manually add art rather than get stuck in the keyboard GUI.

Also, if you wanted to add other artist info, such as BIO information, you could add that tag in the text file as well and write whatever text you wanted.

This is neither user-friendly nor particularly intuitive and your average user more than likely won't bother.

Regarding inconsistency.

I don't see it as being inconsistent. It's an either/or situation. Either the artist is an album artist, in which case the current artist.nfo applies in the artist directory, OR it is an artist for which there is no clearly defined path in which case artistname.nfo applies in the user defined central directory. I would argue that it is consistent because only one of these scenario's can apply. Either we have a album artist or we don't. If we do, look in the artist directory. If not, look in the central directory.
Learning Linux the hard way !!
Reply
#49
Got these interesting ideas on another thread
(2016-03-31, 21:12)scott967 Wrote: 1. Export should allow to export artists, albums, or both.

2. It would be nice if export would allow for artist-name.nfo and album-name.nfo, though I don't know how to handle illegal file name (maybe standard "makelegalffilename" function) . But the database should add based on nfo file contents, not file name.

3. Single file export for artists will include <art></art> element containing currently assigned thumb and fanart (at least for local image files) but is ignored on import.

In light of the fact that music uses file meta-tags as main data source, I would consider getting rid of nfo file reading for music. Instead just use the music library import function. User can turn off online scraping during update.

scott s.

Moving away from Kodi maybe finding an nfo file as part of scraping, if user has it in the path Kodi wants, if we can come up with default locations for all artists, is attractive. What would be lost is the flexibility to skip scraping online for just a few items because an nfo exists.

I think 2) is key. It is the xml tags that should determine what artist or album the info applies to not the filename, nor the path.
Reply
#50
Managed to cross post this morning Black_eagle

Whitebelly is describing manually using importing, which unfortunately doesn't cover artwork. Yes it is the only current way to load artist info for those artists for whom Kodi can not locate a unique folder. Scott is suggesting we make this more mainstream.

(2016-04-01, 09:24)black_eagle Wrote: This is neither user-friendly nor particularly intuitive and your average user more than likely won't bother.
So it would seem you would not be kean? It is helpful to me to talk this through.

I would guess the average user doen't want to use nfo at all. They want it all automated, info and images just appear by magic and look lovely. Remind me again why you have nfo files? Is the import approach really naff, or could it be make approachable?

Quote:Regarding inconsistency.

I don't see it as being inconsistent. It's an either/or situation. Either the artist is an album artist, in which case the current artist.nfo applies in the artist directory, OR it is an artist for which there is no clearly defined path in which case artistname.nfo applies in the user defined central directory. I would argue that it is consistent because only one of these scenario's can apply. Either we have a album artist or we don't. If we do, look in the artist directory. If not, look in the central directory.
The more I look at it the less either /or it seems.Sad

Kodi can have difficulty finding that clearly defined path. Say I have 10 albums by artist1, 3 albums by artist2, the songs all in obvious music/artist-name/album-name folders. But I also have 1 album by artist1 & artist2, a collaboration with both artists on the title, and I either put the songs for that album under artist1 or artist2 or in something different e.g. music/MixedArtist/album14. From song path, that is what data we have, how would you find the correct paths for artist1 and artist2? The current algoirthm fails, the songs for album14 making music/ the common folder for both artist1 and artist2, info for one overwrites info for the other in artist.nfo. I guess checking for artist name in the folder name with some kind of fuzzy logic could help, but common path can be confused.

Also what happens when the user adds to their music library over time. So initially I have no albums by artist2 just them on a few songs, hence no appropriate artist folder, but I view and scrape song artists so artist2.nfo is created in our non_album artist folder. Later, having liked that track so much, I get albums by artist2, create a folder etc. I have an nfo but in the wrong place. Do I check and move it?

While it is fair so say that many album artists will have a matching folder not all will e.g. classical music albums will likely be in a composer folder, so no folders for other album artists orchestra or conductor. Also do we force users to physically organise thier music by album artist?

Album nfo in the album folder (assuming you have one) makes sense (allowing for CD multi disc collections and folders, something currently broken), but artists simply do not have a guarenteed, identifiable, unique physical location. Seems silly to go though hoops trying to find it and use it sometimes.

Verbalising that helped Smile
Reply
#51
Hi,

Sorry I am a little late in the discussion, I missed the beginning, but I also consider music metadata scrapping as a weak point in Kodi at the moment.

For reference, I submitted a pull request a while ago to address some of the issues:
https://github.com/xbmc/xbmc/pull/8876

It didn't get through, but I believe the concerned issues should be solved (however I am ready to accept that my proposed solution is not the right one).

Beyond scrapping methods, I think the most confusing thing about music library is the user interface. The results of music library scan are different when the scrapping is launched from the library view or from the dialog AlbumInfo which doesn't make sense. As a user, I would expect a single action to scrap my entire library (Album and Artists), with eventual helper dialogs when some information is missing.

A second thing I would expect is a simple way to fix metadata manually form the UI, and store the manually input info locally. It may seem stupid, but this is a big frustration for me.

A third thing I don't understand is why we don't use the file tags more? I personally have a collection of music files that I tagged manually over the years, these are displayed fine in other players, but for some reasons I don't understand, they are not properly used in Kodi.

To conclude, if there is some coding to be done on this topic, you can count me in :-)
Reply
#52
@DaveBlake, please if possible verify this bug Kodi Krypton, when try create new music source..

http://forum.kodi.tv/showthread.php?tid=268098
MediaBrazil forum Website - Youtube Channel
MQ9-1.6.0.29 - 09.15.2023 - Aeon MQ Skin Team
MarcosQui Website Donate and support us.
Reply

Logout Mark Read Team Forum Stats Members Help
Efficient music artist scraping0