• 1
  • 4
  • 5
  • 6(current)
  • 7
  • 8
  • 9
Export/Import and local artist art improvements
#76
I gave the test build a quick go:

I imported a clean music library but got a new pop up asking where I save the artist-info. I'm not sure why any normal user needs to see this. Surely it should be saved in a default location in userdata for consistency?
I ended up accidentally cancelling the setting and it imported anyway. Now where is my artist-info saved? Rather confusing.
I tried setting the location in the settings and re-scan, but no change as far as I can see.

NOTE: also got the "do a full scan even though tags are unchanged?" dialogue, cool feature but is that part of this PR? Or did I just miss this getting merged Smile Great if so!

EDIT: tested export but the location is blank (what did the popup from earlier do? seems confusing to me).

- Export to single file works as expected (no artwork present)
- Export to separate files works as expect (but I think artwork should be enabled by default, do we really need such granular control over this? When I click export I just want it "to work")
- Export to library folders (sounds dangerous, not sure I want to test that...)

Image

EDIT2: I see that album covers are named "folder.jpg", since there is no album sub folder, what is the logic for multiple albums? Oh and where is the artist thumb? Maybe these got confused? Smile

Nice work though, a good improvement I think!
Reply
#77
Zag thanks for feedback. I am trying to move to having an Artist Info Folder (and solve all the funnies that go with trying to mix it in with the music folder layout) in a backwards compatible way. That may need some fine tuning.

Quote:Surely it should be saved in a default location in userdata for consistency?
Well no, userdata is not the best location for nfo files and art work, for example Kodi and hence userdata could be installed on a small SD card in a RPi . The artist nfo and artwork wants to be on some kind of mass storage device, similar to the music files themselves, but not mixed in with the music. Until the user adds some music we have no way to know where that will be.

Quote:I ended up accidentally cancelling the setting and it imported anyway. Now where is my artist-info saved? Rather confusing.
I tried setting the location in the settings and re-scan, but no change as far as I can see.
The Artist Info Folder is a where the user can put artist art and nfo files for Kodi to use. It is not mandatory, and for backwards compatibility Kodi will still find artist art nfo located above the music files, but with the old limitations (i.e. only for album artists that have all the music by that artist and only that artist under one folder, and all music by an artist in same music source, and no collaborations).

If the user does not give a location on first adding music source, like you didn't, then it is not a problem. But the prompt appears then to allow those that do have additional artist stuff (including art for song artists, and those on separate sources, or collaborators etc. ) to enter it.

Artist nfo and art is only saved to the Artist Info Folder when you export, or if an addon saves it there (that isn't implemented yet, but will be as soon as Marcel and I can work together on it).

I hope that things can be made clearer, changing messages perhaps, certianly writing guides and wiki. Smile

Quote:NOTE: also got the "do a full scan even though tags are unchanged?" dialogue, cool feature but is that part of this PR? Or did I just miss this getting merged.
Feature to forced full rescan has been in nightlies since start of year available via as.xml setting, but took this chance to improve the UI for it.

Exporting:
Quote:Export to separate files works as expect (but I think artwork should be enabled by default, do we really need such granular control over this? When I click export I just want it "to work")

The export of images has always been optional, I would not want to start forcing it, would you? Maybe a user wants to capture the artist info (nfo stuff), but feel the images are crap so doesn't want to save them. Also the images saved in this way are at cached resolution, possibled compressed from original, so overwrite needs to be optional. Could easily want to overwrite nfo but not art etc.  Mostly I see art being added to the Artist Info Folder by addons or the user manually rather than exported.

This work adds 3 new options: what items exported, export of unscraped items, and (when to separate files) to do art and not NFO. These are because export now has the facility to output a lot more artists, hence they give the user the option to retain the old functionality. Not everyone will want to create a folder for every artist in their library everytime they export.

But the key thing to consider about this is why are you clicking "export"? What you do want to just work?
Export is an advanced feature, not something a beginner would need to use. Unlike video export it is not about making a backup. Importing previously exported files to an empty music library does nothing. The purpose of the music export facility is to help (more advanced) users to manipulate/control the local additional data that Kodi can handle for artists and albums. It is support for local scraping, and really useful if your music collection is not mainstream.

In time I also hope it will be used to reduce the need to refetch the same data from the online sources, when say transfering to a new device or doing a clean install. But those activities are also advanced.


Quote:Export to library folders (sounds dangerous, not sure I want to test that...)
Not dangerous, it is mostly what old export to separate files did but using the Artist Info Folder for artists stuff. Export to separate files creates folders and files for both artists and albums under the destination folder you give, export to library folders puts the album stuff with the music files, and the artist stuff in folders/files under the Artist Info Folder.

Zag I am interested in what happened with your DJKhaled example. What albums do you have for that artist, and can I see the album.nfo file please. An album folder should have been created under the artist one.
Reply
#78
(2017-10-11, 16:17)DaveBlake Wrote:
Quote:The artist nfo and artwork wants to be on some kind of mass storage device, similar to the music files themselves, but not mixed in with the music. Until the user adds some music we have no way to know where that will be.
Is there a logistic reason why it can't be in with the music?  My music is in a folder called "music".  It makes sense to me to shove the "artist info" folder in that folder too, but I can put it wherever it has to go.


Quote:The Artist Info Folder is a where the user can put artist art and nfo files for Kodi to use.


Quote:The purpose of the music export facility is to help (more advanced) users to manipulate/control the local additional data that Kodi can handle for artists and albums. It is support for local scraping, and really useful if your music collection is not mainstream.

In time I also hope it will be used to reduce the need to refetch the same data from the online sources, when say transfering to a new device or doing a clean install. But those activities are also advanced.

Is Kodi making use of this info in your build, or is that still being worked on?.  The export seemed to work, it took all the "mishmash" artists out of the kodi library and put them into the "artist info" folder (which I'd actually created in the "music" folder, I hadn't reread this thread properly before I started mucking about.)  Is it possible to scan that extra info into Kodi, or get it to show up automagically, or do we just need to browse to it to "suck it into" Kodi?

Thanks again for all your work.
Reply
#79
My test data is just 38 standard popular albums, tagged but with no MBID tags.

On export, all that is created is the Artist folder with 4 files.

Code:
Album.nfo
Artist.nfo
folder.jpg
fanart.jpg

It looks to me like something went wrong with the album sub folder creation.

The folder.jpg file appears to be the album artwork.

https://pastebin.com/396HNzkt

EDIT: ticking the Output only artwork, not nfo files and overwrite files makes the artist folder.jpg work.
Reply
#80
(2017-10-12, 06:27)bilgepump Wrote:
(2017-10-11, 16:17)DaveBlake Wrote:
Quote:The artist nfo and artwork wants to be on some kind of mass storage device, similar to the music files themselves, but not mixed in with the music. Until the user adds some music we have no way to know where that will be.
Is there a logistic reason why it can't be in with the music?  My music is in a folder called "music".  It makes sense to me to shove the "artist info" folder in that folder too, but I can put it wherever it has to go.


Quote:The Artist Info Folder is a where the user can put artist art and nfo files for Kodi to use.


Quote:The purpose of the music export facility is to help (more advanced) users to manipulate/control the local additional data that Kodi can handle for artists and albums. It is support for local scraping, and really useful if your music collection is not mainstream.

In time I also hope it will be used to reduce the need to refetch the same data from the online sources, when say transfering to a new device or doing a clean install. But those activities are also advanced.

Is Kodi making use of this info in your build, or is that still being worked on?.  The export seemed to work, it took all the "mishmash" artists out of the kodi library and put them into the "artist info" folder (which I'd actually created in the "music" folder, I hadn't reread this thread properly before I started mucking about.)  Is it possible to scan that extra info into Kodi, or get it to show up automagically, or do we just need to browse to it to "suck it into" Kodi?

Thanks again for all your work.

Think perhaps you've misunderstood, what Dave means by not having the artist info with your music, is actually directly alongside the music files, so what you have done is perfectly fine and is similar to what I've done for testing.

My structure is:

Music
  • Albums
    --> Artist folders
    ---->Artist album folders
  • Singles
    --> Decade folders
    ----> Year folders
  • Artist Info

Where Artist Info is the folder I've set for the metadata export/import.

What Dave meant (I think) was to not set Albums in my case to be the artist information folder as the metadata folders on export become jumbled in with the folders containing the music files.
Reply
#81
(2017-10-12, 06:27)bilgepump Wrote: Is there a logistic reason why it can't be in with the music?  My music is in a folder called "music".  It makes sense to me to shove the "artist info" folder in that folder too, but I can put it wherever it has to go.
The Artist Info Folder can be wherever you like, also for backwards compatibility it is not mandatory, but it could be useful to have one especially if you want to provide artist and nfo for song artists, or those on collabloration albums etc.  But if the Artist Info Folder is an existing folder with other things already in it then it could cause confusion.

Within the folder you give as Artist Info Folder Kodi will look for a folders that match artist names (or name + partial mbid if not unique within library), and within those for the artist.nfo and artwork. Structure like this
PHP Code:
My_Artist_Info (Artist Info Folder)
---
ABBA (folder)
----------
artist.nfo
----------fanart.jpg
----------folder.jpg (artist thumb)
---
John Williams_53b1(folder)
----------
artist.nfo
----------fanart.jpg
----------folder.jpg (artist thumb)
---
John Williams_8b8a (folder)
----------
artist.nfo
----------fanart.jpg
----------folder.jpg (artist thumb)
---
U2 (folder)
----------
artist.nfo
----------fanart.jpg
----------folder.jpg (artist thumb)
... 

If you already have a <my music>/<artist name>/<album name> structure, you might even want to set  <my music> as the Artist Info Folder. If you have ever used cdArt addon, or skins that use Skin helper Service you will understand this has consequences. Your music collection will gain folders for other artists that just contain art not music. And if your artist name is not an exact match for what Kodi is using for that artist it will have a second folder with art.

Some people are happy with that mix, others are not. So my recommendation (just to avoid confusion) is to have an Artist Info Folder different from your music folder. It can be a folder under the music folder, as @jjd_uk described, but best not the one,  if you already have that structure, already containing artist folders with music them unless you clearly understand what is happening.

Quote:
Quote:The purpose of the music export facility is to help (more advanced) users to manipulate/control the local additional data that Kodi can handle for artists and albums. It is support for local scraping, and really useful if your music collection is not mainstream.

In time I also hope it will be used to reduce the need to refetch the same data from the online sources, when say transfering to a new device or doing a clean install. But those activities are also advanced.
Is Kodi making use of this info in your build, or is that still being worked on?  The export seemed to work, it took all the "mishmash" artists out of the kodi library and put them into the "artist info" folder (which I'd actually created in the "music" folder, I hadn't reread this thread properly before I started mucking about.)  Is it possible to scan that extra info into Kodi, or get it to show up automagically, or do we just need to browse to it to "suck it into" Kodi?
This build makes it possible to load and save local art and nfo for all artists (something previously only possible for album artists within limits). This means that experienced users could use it to minimise repeat online scraping when reinstalling Kodi, but that is not something I am expecting beginners to do.

If you are a beginner with nothing but music files, then nothing here changes anything for you. You scan the tags, and scrape artist and album art and info from online sources (either as part of scanning because scan + scrape that works together OK in v18, or as a second action).

But if you have local artist art you want used, then you can put it all in the artist info folder and Kodi will find it as part of scraping (automagically as you say). If you have album artist stuff already saved with you music files Kodi will continue tofind it, but now you will be able to provide local art for composers and conductors, or song artists etc. too.

Or say you have fetched everything online, and pick images you like etc., and want to keep thay after a clean reinstallation or move to new device. Well export that art and nfo. The tell the new Kodi installation where the artist info folder is, and Kodi will pick up the local art without having to scrape online.

Although it seems that this change is about exporting, actually is is accompanied by some nice improvemements to local art and nfo handling too.
Reply
#82
(2017-10-12, 12:30)docwra Wrote: My test data is just 38 standard popular albums, tagged but with no MBID tags.
...
It looks to me like something went wrong with the album sub folder creation.
Yep, sounds like in the rebasing and edits I picked up a glitch. Thanks for the log, I will look into it.
Reply
#83
(2017-10-12, 15:06)DaveBlake Wrote: Or say you have fetched everything online, and pick images you like etc., and want to keep thay after a clean reinstallation or move to new device. Well export that art and nfo. The tell the new Kodi installation where the artist info folder is, and Kodi will pick up the local art without having to scrape online.

Unfortunately, although all the exported artist info is there in the "artist info" folder, Kodi doesn't pick any of the pictures up for the "compilation" artists.  It only has what it used to have - the album artists' pictures.  This is a "new" installation in that it's a portable mode installation of your build.  All the "extra" artists from compilations have an icon (bloke in a suit and tie wearing headphones) with their name underneath, but no picture of them. 

The "artist information folder" is identified under Media/Music/library/Artist information folder.  Kodi doesn't seem to have "sucked up" the pictures, although they are there.
Reply
#84
(2017-10-12, 15:51)bilgepump Wrote: Unfortunately, although all the exported artist info is there in the "artist info" folder, Kodi doesn't pick any of the pictures up for the "compilation" artists.  It only has what it used to have - the album artists' pictures.  This is a "new" installation in that it's a portable mode installation of your build.  All the "extra" artists from compilations have an icon (bloke in a suit and tie wearing headphones) with their name underneath, but no picture of them. 

The "artist information folder" is identified under Media/Music/library/Artist information folder.  Kodi doesn't seem to have "sucked up" the pictures, although they are there.
I'm not quite sure what you are saying. Is it that you have albums with songs by "artistX", but no albums by that artist. And you have an image for artistX, in a file called folder.jpg located in a folder called "artistX" in your Artist Info Folder.

If you then do ""Query info for all" from the context menu within the artists node (where you see the bloke in a suit and tie wearing headphones) that wil pick up the image. But I may have misunderstood.

I think you were expecting scanning to pickup local art, and it didn't. Now that may not be an unreasonable expectation, but it isn't what I have implemented. Scraping picks up nfo and art. But I will give it some thought.
Reply
#85
DaveBlake Wrote:
(2017-10-12, 15:51)bilgepump Wrote: Unfortunately, although all the exported artist info is there in the "artist info" folder, Kodi doesn't pick any of the pictures up for the "compilation" artists.  It only has what it used to have - the album artists' pictures.  This is a "new" installation in that it's a portable mode installation of your build.  All the "extra" artists from compilations have an icon (bloke in a suit and tie wearing headphones) with their name underneath, but no picture of them. 

The "artist information folder" is identified under Media/Music/library/Artist information folder.  Kodi doesn't seem to have "sucked up" the pictures, although they are there.
I'm not quite sure what you are saying. Is it that you have albums with songs by "artistX", but no albums by that artist. And you have an image for artistX, in a file called folder.jpg located in a folder called "artistX" in your Artist Info Folder.
I have a LOT of "artistX" entries, because I have a lot of albums which are compilations/movie soundtracks/album artist+guest artist and that type of thing where different people are performing the songs, not just one person/band. 
Since installing your build and exporting, all of those "non-own-album" artists have a folder with an artist .nfo and many also have their photo as folder.jpg in their artist folder (I also messed with 'create an .nfo anyway' or whatever the selection is, so they all have an .nfo, although some of the more obscure don't have a photo.  Anyway, there are many known artists who DO have a photo in their artist folder).  Set media source, add to library.  All the albums appear, all the album artists appear with photos, but the "artistX" artists appear without photos.  

Quote:If you then do ""Query info for all" from the context menu within the artists node (where you see the bloke in a suit and tie wearing headphones) that wil pick up the image. But I may have misunderstood.
If I right-click on an artist I know to have a photo in their artist folder and select "query info for all artists", it will show at the top-right that it is going through everyone, but it does not pick up any of the photos that are there or the information in the artist's .nfo (which is also there, I've opened it and looked).

If I select "information" and "refresh" it seems to (try to) look online, not just in the local information (which is the setting for default scraper).  I select "information" then "refresh" and I get "enter the artist name".  If I hit "ok" (which it is, because it matches the artist folder name exactly) it just brings that up over and over.  If I then "cancel" out of that it says "Searching artist: Downloading artist information failed" (as if it tried to look online?).  If I choose "change information provider" and "set for this artist" or "set for all artists shown" it says "this directory contains artists" and "local information only" so it is set properly to only look locally.
Quote:I think you were expecting scanning to pickup local art, and it didn't. Now that may not be an unreasonable expectation, but it isn't what I have implemented. Scraping picks up nfo and art. But I will give it some thought.
I have no expectations but am just "playing" with it to see what it does and giving any feedback about what it is doing.  I don't have any coding experience so I don't understand when "you guys" talk about that, but I'm quite familiar with Kodi from a user's perspective.  I think I've done what's required to have it scrape the .nfo and picture files.  Kodi must be using the "artist info folder" to create all the artist heads, it just isn't scraping the artist .nfo and picture.
Reply
#86
So if I understand correctly you've done the export and the metadata info for all artists is there in the artist info folder. So then you've removed the library sources to start over again from scratch by adding the source back, however the metadata for non-album artists that is contained in the artist info folder is not added back.
Reply
#87
(2017-10-13, 04:14)jjd-uk Wrote: So if I understand correctly you've done the export and the metadata info for all artists is there in the artist info folder. So then you've removed the library sources to start over again from scratch by adding the source back, however the metadata for non-album artists that is contained in the artist info folder is not added back.
More or less, but I didn't remove any sources, it's a whole new portable "install" on a different computer with sources added to that new "install", but with the artist info folder copied from the other computer.  Everything seems to be there, and set correctly, but, as you say, the metadata for non-album artists does not get "picked up" by Kodi.
Reply
#88
All good feedback @bilgepump, thanks for playing with it, all very helpful. It seems I have introduced a glitch with the C++ style edits and rebase I did. So a bit of tweeking for me to do. But that is what testing and feedback is all about.
Reply
#89
(2017-10-13, 05:24)bilgepump Wrote: More or less, but I didn't remove any sources, it's a whole new portable "install" on a different computer with sources added to that new "install", but with the artist info folder copied from the other computer.  Everything seems to be there, and set correctly, but, as you say, the metadata for non-album artists does not get "picked up" by Kodi.
@bilgepump could you go to settings?media>music and check that "Artist Info Folder" is set to the folder that you generated using export and copied over from the other computer. It will be a folder containing sub-folders with artist names etc. Then go to the artists node and repeat "Query Info for all"

That is assuming to still have the test portable install in the same state. I can't repeat the issue, so the most likely problem is that this was not set. I take on board that this use is not intuitive enough yet, and needs documenting clearly at least, but I do want to check it works (or find why it doesn't).

Thanks, the testing and feedback is appreciated.

However the glitch Zag spotted is still present (added by some code tidy up) . It means that album art overwrites artist art when both are exported. So @bilgepump you might have album images rather than artist ones!
Reply
#90
(2017-10-13, 19:38)DaveBlake Wrote:
(2017-10-13, 05:24)bilgepump Wrote: More or less, but I didn't remove any sources, it's a whole new portable "install" on a different computer with sources added to that new "install", but with the artist info folder copied from the other computer.  Everything seems to be there, and set correctly, but, as you say, the metadata for non-album artists does not get "picked up" by Kodi.
@bilgepump could you go to settings?media>music and check that "Artist Info Folder" is set to the folder that you generated using export and copied over from the other computer.
Post #83 - yes, it is. 
I originally pointed it to the folder that I'd created in with the music files (before I re-read the thread where it said to put it somewhere else), but when that didn't work, I copied the artist info folder to the usb where the portable install is.  Neither worked.  I then nuked the whole thing, started again, still no joy.  I'll nuke it all again and start from scratch to see whether it's something I've done/not done.
As I mentioned, Kodi seems to be "seeing' that artist info folder, because when it scrapes (locally) it scrolls through all those obscure names that are there.  They aren't anywhere else but in that folder are they?

EDIT: For those folders I've looked at to test getting them to show up, it's definitely an artist image in there and the .nfo is the artist info. God knows what's in all of the folder, but at least some of them seem to be how they are intended to be.
Reply
  • 1
  • 4
  • 5
  • 6(current)
  • 7
  • 8
  • 9

Logout Mark Read Team Forum Stats Members Help
Export/Import and local artist art improvements0