Scraping music data
#46
(2016-04-29, 13:13)Mike_Doc Wrote: I always have library problems be it scanning or whatever, take for instance anything by Satoshi Tomiie all tagged via MBP with artist as Satoshi Tommie but in Kodi the artist it null which I believe is because Madnox cannot handle Chinese / Japanese chars which I believe the songs have been loaded with. But that is going off track slightly.

I guess I'll have to bite the bullet and start planning to update Madnox to Krypton compiliant, scares me the amount of changes to deal with, and then hopefully I can take adavantage of your improvements in Krypton.

And thank you for continuing to improve Music in Kodi.


Just checked again after seeing one of zag's posts. I do have ARTISTS but it is defaulting to the artist of the track so I would guess that is down to how the album is in MBP, not what kodi is doing.


As far as non-Latin-1 scripts, I have no problems with Kodi handling this. I have seen problems with tagger software where ID3 v2.3 uses UTF-16 encoding and the software was inserting the BOM incorrectly. This can create "duplicate" artists in the music db (I mainly have this problem with multiple artist credits) (when you look at the two entries with a hex viewer, one is correct and one has the BOM as first "character"). Of course, you need a skin font with unicode coverage of the needed codepoints.

scott s.
.
Reply
#47
(2016-04-29, 13:12)zag Wrote: I spoke with the developer "zas" from metabrainz, he pointed me to this ticket and PR

http://tickets.musicbrainz.org/browse/PICARD-700

and

https://github.com/metabrainz/picard/pull/268

He said make a comment there about the default tagging, maybe it will be accepted but apparently its not supported enough yet.

At the moment, its only possible via the above script.

Thanks for that Zag, and Black_eagle for the Picard script - I have been testing so many kinds of tagging I lost sight of what and how for as moment.

I have posted on Picard ticket system and see if that moves forwards. Otherwise we need to mention that script setting in the Picard guide/tagging tips Zag if you can. ID3 v2.3 and its lack of multiple value tag frames may still present difficulties in Jarvis, but I will see if Krypton can be more versatile in parsing artist names when we know from mbids how many we are trying to identify.
Reply
#48
Hmm i'm hesitant to add a script to the guide for now as its a little advanced for the new user to setup.

I've added my support to the musicbrainz ticket. Lets hope someone picks it up!
Reply
#49
To answer the original question:

I doubt I have a standard setup, but it makes sense for me, and it's still a work in progress.

I have 3 primary folders for music:
1) "Complete" - These are full albums painstakingly OCD tagged with Picard.
2) "Singles + Various" - these are artists who I only have a few songs from, as well as any various artists albums. These are kept separate from "Complete" for Headphones purposes (I don't want new releases from these artists)
3) "_downloaded" - this folder is new downloads that I have not tagged yet (mostly from Headphones)

Each of these has a folder structure of: \Artist\Album\01 filename.mp3
folder.jpg is added for every single album folder. When I was tagging everything with picard initially, I made sure that every album had artwork. I ran scripts to find missing folder.jpg's, and created scripts to scrape the info, and did a bunch of manual searches as well.

for 1 and 2, I am in the works of moving everything to local nfo. This is because - I guess much of my media is not particularly popular, so there were a lot of artists with missing fanart and bios. In order to get a baseline of data, I used CDArt Manager, and I exported the database file from Kodi. Next, I needed to fill in any gaps, which I did with scripts that scraped bing for large artist images and logos, which I sifted through and confirmed. The fanarts were resized to the correct proportions using birme. Since I did this in bulk, they are probably not perfectly cropped, but better than nothing. Then I used some more scripting to get all artist tags from Last.fm to use as artist genres (which I tidied up) and I also scraped any bios that were missing from the database. Then I used this info along with the exported database info to create individual nfo files following this - the only tags I skipped were <thumb> and <album>

Why in the world did I do all this work? Primarily for the following 4 reasons:

1) I wanted multiple genres for each artist (main reason why I'm using local artist nfo files)
2) I wanted to have artwork for every album (main reason why I'm using local album data)
3) I wanted bios for each artist
4) I wanted fanart for each artist

I just stumbled upon this thread, because I'm still in the process of doing this. I just placed all the nfo files into the artist folders and will be running a scan to see if everything is working correctly. I did not create album nfo files, because I believe all of that is included within my tags.

Mainly, it was because I was realizing that set content and scan isn't available for music. Since I keep new music in a separate folder, I would like to be able to scan that path using scrapers, and the other 2 directories to only use local data. I do the exact same thing with movies and tv, so when I just discovered I can't do that with music, I was a bit disappointed.
Reply
#50
(2016-06-01, 15:39)nuentes Wrote: Mainly, it was because I was realizing that set content and scan isn't available for music. Since I keep new music in a separate folder, I would like to be able to scan that path using scrapers, and the other 2 directories to only use local data. I do the exact same thing with movies and tv, so when I just discovered I can't do that with music, I was a bit disappointed.

I think you can achieve this by using two (or more) separate music sources. Source 1: Your two manually tagged folders, source 2: The other folder.

Scan folder 1 in first and after that scan folder 2 in.

Also I believe local nfos will always override any online information.
Reply
#51
(2016-06-01, 15:54)DarkHelmet Wrote:
(2016-06-01, 15:39)nuentes Wrote: Mainly, it was because I was realizing that set content and scan isn't available for music. Since I keep new music in a separate folder, I would like to be able to scan that path using scrapers, and the other 2 directories to only use local data. I do the exact same thing with movies and tv, so when I just discovered I can't do that with music, I was a bit disappointed.

I think you can achieve this by using two (or more) separate music sources. Source 1: Your two manually tagged folders, source 2: The other folder.

Scan folder 1 in first and after that scan folder 2 in.

Also I believe local nfos will always override any online information.

I meant that's why I stumbled upon this thread, not why I'm moving to nfos Smile

I do have the paths set as separate music sources - I'll be scanning the sources in this afternoon, so I'll be sure to do that in the correct order.

Local nfos may always override online info, but local fanarts and logos apparently do not (I'm unsure about folder.jpg's)
Reply
#52
(2016-06-01, 16:09)nuentes Wrote:
(2016-06-01, 15:54)DarkHelmet Wrote:
(2016-06-01, 15:39)nuentes Wrote: Mainly, it was because I was realizing that set content and scan isn't available for music. Since I keep new music in a separate folder, I would like to be able to scan that path using scrapers, and the other 2 directories to only use local data. I do the exact same thing with movies and tv, so when I just discovered I can't do that with music, I was a bit disappointed.

I think you can achieve this by using two (or more) separate music sources. Source 1: Your two manually tagged folders, source 2: The other folder.

Scan folder 1 in first and after that scan folder 2 in.

Also I believe local nfos will always override any online information.

I meant that's why I stumbled upon this thread, not why I'm moving to nfos Smile

I do have the paths set as separate music sources - I'll be scanning the sources in this afternoon, so I'll be sure to do that in the correct order.

Local nfos may always override online info, but local fanarts and logos apparently do not (I'm unsure about folder.jpg's)

I had the same phenomenon with missing local fanart, folders etc. Maybe try this advice

http://forum.kodi.tv/showthread.php?tid=...pid2341616

Basically make sure that the separate artists folders get scanned in before any compilations folder. It solved the issue for me.
Reply
#53
(2016-06-01, 17:04)DarkHelmet Wrote:
(2016-06-01, 16:09)nuentes Wrote:
(2016-06-01, 15:54)DarkHelmet Wrote: I think you can achieve this by using two (or more) separate music sources. Source 1: Your two manually tagged folders, source 2: The other folder.

Scan folder 1 in first and after that scan folder 2 in.

Also I believe local nfos will always override any online information.

I meant that's why I stumbled upon this thread, not why I'm moving to nfos Smile

I do have the paths set as separate music sources - I'll be scanning the sources in this afternoon, so I'll be sure to do that in the correct order.

Local nfos may always override online info, but local fanarts and logos apparently do not (I'm unsure about folder.jpg's)

I had the same phenomenon with missing local fanart, folders etc. Maybe try this advice

http://forum.kodi.tv/showthread.php?tid=...pid2341616

Basically make sure that the separate artists folders get scanned in before any compilations folder. It solved the issue for me.

Thanks, but it's kind of a moot point, since I've already decided to proceed with only local info. It's how my movies and tv are archived, so it made sense for my music collection to follow suit.
Reply

Logout Mark Read Team Forum Stats Members Help
Scraping music data0