Kodi Community Forum

Full Version: Music scraping help
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Hello,

I am having some weird issues with music scraping. First, a lot of titles in my library are small local bands without any info on the web. I recently discovered library view and want to view by artist because I think it's a great way to view my collection. So for these artists who aren't on the web I've been adding some kind of image to go with them. I do this by hitting context menu, then artist info, then get thumb, browsing to local file, etc. This has worked fine on many but not on all. Some artists I go to artist info and it prompts me to input the name again, which I do, then it goes into an infinite loop prompting for the name until I X out of the dialog, then it says "download artist info failed".

Ok so I assume downloading is the problem so I tried making an artist nfo and putting it in that artist directory, then changing artist scraper to "local NFO only", then clicking artist info again. No good, same exact error.

I am not sure how much I need to put into the artist NFO. I have tried just

<artist>
<name>artist name</name>
</artist>

then I have also tried

<artist>
<name>artist name</name>
<genre clear=true>Rock</genre>
<genre></genre>
<genre></genre>
<style clear=true></style>
<style></style>
<mood clear=true></mood>
<mood></mood>
<yearsactive clear=true>19??'s to 20??'s</yearsactive>
<born>Birth date of artist</born>
<formed>Date band formed</formed>
<instruments></instruments>
<biography></biography>
<died></died>
<disbanded><disbanded>
<thumb></thumb>
<album>
<title></title>
<year></year>
</album>
</artist>

Neither make any difference, it acts as though it's still trying to download from the web and ignoring the NFO file even if I switch artist scraper to local NFO.

I created the file using Notepad and saved it as artist.nfo in the artist directory, which I believe is correct.

Is it sufficient that I am clicking on "artist info" to try to rescrape? Or would I need to rescrape the entire library from local info to make this work? (That would be terrible)

Basically all I want is to be able to add an artist image to every entry otherwise what is the point of having it on some but not others. Is there any other way to do this?

Been at this for a few hours now and I can find some similar issues on the web but not exactly.

I really do not care if the biography is accurate I just want the ability to add thumb and fanart to all of the artist entries.

edit:
Solved my own issue, all I had to do was put a cover.jpg file into the artist directory and this ended up being used as the artist thumb. I haven't been able to figure out the same for fan art yet but thumb is sufficient.

edit2:
This does not always work. Sometimes it's necessary to remove (move) the entire directory out of the music folder. Then clean the music library. Then put the folder back in and do an update. However, even this does not always work 100% of the time. And, if you decide you want a different image later, you can't simply change the jpgs in the artist directory and clean and update, it will pull the first image it scraped from the HD from wherever Kodi keeps these files. I have tried the removal, clean, put back, update process and it sitll pulls the first image scraped. So be careful.

The process only works at all if your directory structure is set up as /Artist/Album and only if your Artist directory name is exactly the same as it appears in Kodi.

(It's taking me 5-15 minutes per artist to get a thumbnail to show up because of the various voodoo procedures I described above. The inconsistency is frustrating.)

edit3:
I finally got NFO scraping to work which solved all of my problems I think.

edit4:
NFO scraping works in some cases but not in others. Seems there is no end to this.
First step is make sure your music is tagged or you use cue sheets that are consistent with entries for "artist", "album artist", and "album". Adding an embedded album cover art also is good.

Second, use a "folder.jpg" art in each folder/subfolder. This can be artist pix or album cover for example.

Third, it is difficult to know if an artist did or didn't scrape, or if local art was or wasn't found. Typically the scanner will determine the album artist and album title from the tags. For the artist, it will look in the parent folder to the one containing tagged tracks for specific files "folder.jpg" and "fanart.jpg" to set as the local thumb/fanart for the album artist.

For artists and albums that don't scrape, what I found works best is to use an xml file (same format as those nfo files, but you can have as many artists and albums in a single file as needed).

A trick I use is to use an SQL tool to force the Kodi music database to indicate every artist and album is scraped, then do a single file music library export. The last added artists and albums are always at the bottom so it isn't too hard to find the new ones that didn't scrape. Then just fill in the missing data with whatever you have locally available. I truncate the export file so it only contains the new artists/albums (that way import is much faster) and then import the edited file back into the music library.

An alternative if you use the "universal" scrapers. These use the "musicbrainz" site as a canonical data source. Ideally you could update musicbrainz, but in my experience that is hard because to protect its quality MB has tight editing controls. And sometimes you might want to use tags in a non-standard way to get around limitations, for example artist aliases or classical music.

There is work being done on a third party tool (I think it is called "symphony") that would simplify this process, but work on that seems to go very slowly. Alternatively there are various tagging programs out there that allow for creating reports, and potentially you could tailor a report to generate the needed xml for kodi.

Whatever you do, please export a single file from the library and keep that as a backup.

scott s.
.
My toolkit for getting unknown bands to get recognised involves MP3TAG, folder.jpg and SQLiteBrowser ( https://github.com/sqlitebrowser/sqlitebrowser/releases )

Also some idea of what the application is attempting, so reading the wiki helps http://kodi.wiki/view/Music_library

Like you I also get confused by KODI's own internal "Information" tool when it gets stuck in that odd loop. Which is why I tend to do all my work outside of the running KODI. I see you've now found the folder.jpg trick. There is a wiki page that takes the guesswork out of getting those names right http://kodi.wiki/view/Adding_music_to_th...thumbnails

Clean up all the tags on those music files with MP3TAG. Especially worth understanding what happens with ARTIST, ALBUM ARTIST and those fields. Especially when there are multiple artists listed for once track. http://kodi.wiki/view/Adding_music_to_th..._File_tags

Be aware that any MUSICBRAINZ tags in the tracks will over-ride other tags and be used to set the artist choices.

Changing artwork can be a pain. Mainly because KODI has been written to speed-up rescans. This doesn't just make changing artwork tricky, but sometimes correctly MP3 TAGs is not always picked up. One way to absolutely force everything to be rechecked is to go back to where you setup the file scraper and make a change there. Briefly set the scraper to NONE. Save that setting. Then go back and set the scraper to MUSIC again. This will kick off a deeper rescan of everything.

I'm comfortable with hacking the SQL Database directly. I shutdown KODI and then use SQLiteBrowser to open up the Music database and directly edit it. This allows me to do bulk cleaning up of data which isn't always picked up with a file scan. Or OCD levels of correcting a band name or changing a path.

When artwork is being manipulated it is worth remembering that KODI caches it locally, so you may want a play with this (http://forum.kodi.tv/showthread.php?tid=158373). Some people can do wonders with it, but personally I get lost in a 100+ page thread. One day I'll have time to read it!! Big Grin
Thanks guys!

Just to update since I have figured out two things since my last post (and in case this might help anyone in the future)

1. Kodi likes a single Artist\ directory followed by Album\ directories. In general, with this setup, local thumbs (folder.jpg) and fanart (fanart.jpg) are either scraped or available to select as "local thumb" or "local fanart" from artist info. Also, artist.nfo is recognized correctly in this scenario.

1a. All of the situations where I was running into issues didn't follow #1.

2. My directory structure is not exactly as Kodi likes it because I have two main directories \mp3\ and \FLAC\, and then I have \Artist\Album etc. So, there can be two directories for an Artist, one in MP3 one in FLAC. Don't do this if you can help it because local thumbs, artist nfo, etc will not be recognized even if you put them in both directories. At some point maybe I will change everything to how Kodi likes it but I have already put too much work in right now to do that.

3. One other instance where artist.nfo wasn't working was when I had a directory with \Artist\Album\Album (where the nested album was a different rip). This made it not work. Once I moved both \Album\ to the \Artist\ directory, artist.nfo worked fine. So Kodi is very picky about the directory structure.
Yes, directory structure dependance has its undesirable consequences. I'd love to see a move away from it, but for the most part I can live with workarounds.

One approach is to add art to theaudiodb, I've done a few artists like this with success. It can be a pain to get art to fit the specs.
You really should read the Wiki as it will take some of the guesswork out of things. Yes, the \Artist\Album\ structure is very important to the scraper.

I also have folders like you - different parents for MP3 or FLAC or based on the source of that part of my collection. Each time I had a different parent like that I just added it as a new source. Partially because I know that I'm replacing old rips with cleaner EAC FLAC rips.

For duplicates, my main solution was to get rid of the lower quality MP3s in most cases. OR rename the MP3 version of the album using the MP3 TAG so it appears separately in KODI's library.

I also had to get aggressive on all my two disk albums. I make use of the disk number so they now all correctly show 1/2 or 2/2 etc. with the album name set the same for both disks.


The biggest gotchas I get with the directory layout is when I have a boxset or two CDs making up an album. When there is a \Artist\Album\CD1\ and \Artist\Album\CD2\ then I find that due to a bad assumption within the Artist artwork code the folder.jpg at that Album level can often get swapped in as the artist's image in error.

Maybe the MUSIC side of KODI needs a concept of a "Collection" like on the videos side. This would then allow better handling of box sets. And the more common CD1\CD2 setups.
I got all my songs, 33k, to scan in, but they're all in one big directory and pre-tagged using mp3tag. Seems to me going the extra step of having images for every song is not worth the trouble.

What's the value?
(2015-07-04, 17:30)music_lover Wrote: [ -> ]I got all my songs, 33k, to scan in, but they're all in one big directory and pre-tagged using mp3tag. Seems to me going the extra step of having images for every song is not worth the trouble.

What's the value?

I started embedding album art in every song and moving away from just having folder.jpg. Its nice because it stays with the file no matter what you do with it (copy to MP3 players, phones, etc). SInce you can do an entire album at once, there really is little extra effort required. MP3Tag does it quite well.
@music_lover: if you just have one huge flat heap of music then you'll want to just rely on the artwork coming in from the web. The conversation here is more about unusual bands that the scraper cannot recognise. If you have a flat heap like yours there is no simple way of adding in the missing artwork for unrecognised artists.

Or maybe you can have two sets of folders. One is your original Flat Heap. Then alongside you start making a cleaner \Artist\Album\Track\ layout for those few albums that don't get identified correctly. Then slowly grow new music into the new organised structure migrating away from the original "heap"
I have a lot of alternative music from the 70-90's, etc. Freedb has a lot, but not all. My question is, what value is having the artwork when listening to the tunes?
(2015-07-04, 18:25)music_lover Wrote: [ -> ]I have a lot of alternative music from the 70-90's, etc. Freedb has a lot, but not all. My question is, what value is having the artwork when listening to the tunes?
Personal choice. Which is the beauty of KODI. We can all have our collections in slightly different ways. Personally I like to flick through a list of artists with images, and then flick through a list of albums. But then that is because I still pick up and play my Vinyl and CDs too.

Other people grew up with MP3 playlists and never see the music. Personally, listening to MP3s make my skin crawl as I can hear the flattened deadened sound of the music when played through decent speakers.

We all have different needs. Wouldn't life be boring if we all did things in the same way.Cool
(2015-07-04, 18:45)BatterPudding Wrote: [ -> ]
(2015-07-04, 18:25)music_lover Wrote: [ -> ]I have a lot of alternative music from the 70-90's, etc. Freedb has a lot, but not all. My question is, what value is having the artwork when listening to the tunes?
Personal choice. Which is the beauty of KODI. We can all have our collections in slightly different ways. Personally I like to flick through a list of artists with images, and then flick through a list of albums. But then that is because I still pick up and play my Vinyl and CDs too.

Other people grew up with MP3 playlists and never see the music. Personally, listening to MP3s make my skin crawl as I can hear the flattened deadened sound of the music when played through decent speakers.

We all have different needs. Wouldn't life be boring if we all did things in the same way.Cool
I hear ya, but with 33k songs, it's hard to flac copies for all.
So I'm having a new issue now, some of my artist thumbnails are reverting to the album cover for Modest Mouse's "This Is a Long Drive for Someone With Nothing to Think About", an album I don't even have except for one song, and that song has no thumbnail saved in the tags according to mp3tag. It probably is a JPG file sitting on my HD, but I have no idea how I am supposed to figure out where it's located. I only have two source directories, and neither have any JPG in the root folder.

What happens is I'll change an artist thumbnail, and it's fine, then later I'm doing a "clean" to get rid of a deleted album entry that won't disappear for some reason, and when I check the artist I set before it has reverted back to this album cover again. It's pretty frustrating.
@Litmusdragon: turn on hidden files viewing in those folders. I bet you have a JPG in there from Windows Media Player. Annoyingly Microsoft adds their thumbnails as hidden system files. This then has the side effect that you are describing.
Thanks Batter, that was it!
Pages: 1 2