Kodi Community Forum

Hi. Thanks for this scraper and all the devs do to create and maintain Kodi.

I'm writing because I just cannot scrape Album Ratings no matter what I do, and I would value some help please. The log is here.

Context

• I've (it would seem unwisely) done a full database and Kodi refresh after the recent Kodi upgrade.
  
• Album ratings (scraped with Kodi from online sources) were within my Kodi DB up until I refreshed, and I like them to sort and find content in a large database.
  
• Kodi is on a 2017 Nvidia Shield, and the Shield itself is connected via SMB to a Mac which actually has the data.
  
• I use native paths and EmbyNext gen to connect with my Mac, but Emby doesn't sync ratings across, so I'm attempting to scrape the albums on my shield by loading them into Kodi DB, and then running the scrapers after as recommended.
  

Tests today (after a full import of my full library failed):
I set up a test directory with a few albums from a well known Australian band, and have previously (2 or so years ago) run those albums through MB Picard to put the MB tags on them.

The preferred album rating scraping source I'm trying to use with the Generic Scraper is Discogs, as it has may more users putting in ratings than other sites. I have confirmed that Discogs does indeed have ratings for these albums (hundreds - example here), AllMusic also has many but fewer (dozens), and the other sources one or none.

I have the "fetch additional information during updates" setting set to on, and I have followed the instructions on the Kodi Scraping Music page, tried changing the rating provider in the Generic scraper, and tried the Universal Scraper. I've also tried just loading the files into Kodi on the shield without loading Emby in case there is some clash, but still to no avail.

In android on the shield Kodi is set to be able to access local files all the time.

Nothing I've done has yielded album ratings, which I determined using smart playlists for both "ratings" and "my ratings". Can anyone please tell me how I can scrape in album ratings?

I'm grateful for the free and excellent software and community involved in it, so I wish to be clear I'm happy, but I'd value some assistance please as I'm going around in circles. I'd be delighted if someone told me there's some dumb mistake I'm making.

Cheers and thanks, Peter.

@ptvminix A quick glance at your log file suggests that you have local nfo files alongside your albums, and that Kodi is loading its info from them.   You should answer YES to the dialog box that says "local info found, ignore and refresh from the internet" else Kodi won't contact discogs or any other site and will just use the information in the nfo files.

If that doesn't include ratings then ...... you won't get any.

(2023-02-03, 12:27)black_eagle Wrote: [ -> ]You should answer YES to the dialog box that says "local info found, ignore and refresh from the internet"

Hey thanks @black_eagle for the quick help.

I see what you are saying and it makes perfect sense. So I just erased Kodi again and then loaded in my test directory, and also rescanned via "query info for all artists", but didn't see a "local info found" dialogue during either stage. I did however find and apply the Kodi DB setting "prefer online information", and set that, along with "fetch additional information during updates" before doing the scan and rescan.

Still no ratings I'm afraid. Here's the latest log. I'd value any further perspectives from yourself or others. With thanks.

Well, your second log is not a debug log (wiki) so reveals nothing. 
From your first log however

Quote:Found matching nfo file: /storage/MAC-STUDIO/Secondary/Audio/Test/Divinyls/Temperamental/album.nfo

(2023-02-03, 13:41)ptvminix Wrote: [ -> ]I did however find and apply the Kodi DB setting "prefer online information", and set that, along with "fetch additional information during updates" before doing the scan and rescan.

Indeed. However Kodi will always prefer local nfo files over remote info, even with those settings enabled.

You could try doing the following, and see if you can fetch ratings this way.

1) Navigate to the list of albums, highlight any album and get up the context menu
2) Choose "Change information provider"
3) Set "Choose how to apply settings" to Set default information provider
4) Album information provider -> Generic Album Scraper (should be that already)
5) Select "settings" at the bottom and ensure ratings is set to discogs
6) Select OK

Kodi should now re-scrape from the internet and fetch ratings. Or at least, I just tried it and it worked for me and all my albums have nfo files.

(2023-02-03, 14:29)black_eagle Wrote: [ -> ]Well, your second log is not a debug log (wiki) so reveals nothing. 

Sorry for my mistake with the second log, and thank you for the further reply and for testing it out yourself. I really appreciate it. It's late here and I need to work tomorrow so I'll have to try it out tomorrow or Sunday.

I'll let you know how I go, and again, thanks @black_eagle

(2023-02-03, 14:29)black_eagle Wrote: [ -> ]Kodi should now re-scrape from the internet and fetch ratings.

I couldn't sleep, so I got back to Kodi for a moment. Unfortunately that didn't work (debug log here)

I've tinkered with Kodi too much today (and yesterday), so it's entirely likely I'm still doing something silly, but I'm confused, as I once successfully scraped ratings in using this same file structure in the past (with nfo files in each album's folder). Hopefully it's something simple.

With thanks

Hmmm, nfo files are still getting in the way!

So, navigate to an album and get up the album info screen - on a keyboard this is the <i> key but no idea how to do it on your shield so you want the window shown here.



Click the <refresh> button (highlighted in the pic above)

You will get the following dialog appear



Answer YES here, otherwise Kodi will just load your nfo file for whichever album it is. Album should be re-scraped and rating should appear in the right pane (if using Estuary as your skin).

This is a laborious way of doing it though as you have to do each album individually, which is why I was hoping that my other answer would work for you.   Alternatively, having ensured that the ratings are indeed getting scraped by this method, you could nuke all your current nfo files and re-scrape all your albums fresh by following the first method.

(2023-02-03, 18:11)black_eagle Wrote: [ -> ]This is a laborious way of doing it though as you have to do each album individually, which is why I was hoping that my other answer would work for you.   Alternatively, having ensured that the ratings are indeed getting scraped by this method, you could nuke all your current nfo files and re-scrape all your albums fresh by following the first method.

G'day @black_eagle . Thanks for staying with me on this. I really appreciate your assistance.

Laborious method (immediately above)

• I'm happy to say that your most recent ("laborious") method worked, and for one album at a time as you stated.
  
• I overlooked running debug logs when I ran it, but it worked so I imagine they're not needed.
  
• Anyway, I have hundreds or thousands of albums and so I just can't use the laborious method, but I realise you understood that. 
  
• It would be great if Emby server gathered and loaded into/over the NFOs an album rating, but it appears not to.
  

Retrying plan A...
I did however restart the debug log before reattempting your suggestion in post 17 (I did NO nuking beforehand). No luck sadly, that log is here if useful. 

Is it me or bugs? (likely me I realise)

• I'm trying to understand why this is happening, as I was able to scrape ratings before on the same Kodi (different Mac) even though I had local nfos, though I did have trouble. While corresponding with Luke from Emby at the time (2 or so years ago) I stumbled upon a possible solution (setting "fetch additional information during updates"), but I recall you've told me above that couldn't have actually been the solution, so I don't exactly know why I got successful ratings scans in the past.
  
• Regardless, I'd like to offer my help in any changes to Kodi that make it more reliably overwrite local nfos as I presume is intended, though I realise you have limited time and it appears to be just me having the issue so all good if you can't.
  
• If there can't be a fix/update, then ok, I'll nuke nfos...
  

Nuking NFOs
To do so I guess I'd need to alter my Emby server settings to NOT store nfos within the music directories. Is that your understand of how best I "nuke"? I assume that's easy enough, yet hope it doesn't require a rescan. I'll rescan if I have to, or change servers, or whatever. It is what it is, but I notice that the Emby devs strongly recommend keeping the nfos with the music.

I believe the above is a long form version of saying, is this just me?

I'm really happy to be corrected on anything above, and once again, thank you.

Cheers, Peter.

Correction: I meant "It would be great if Kodi loaded into/over the NFOs an album rating, but it appears not to."

OK so I've had a  look at the code and it appears that the single album override where you answer yes to ignoring local info is the only way to override a local nfo.

At this point I should probably say that although I don't use Emby, I do agree with their position regarding local nfo files.  It's much easier and quicker to rebuild your library using them and it avoids hitting up the internet sites that provide this info for free.  This is why Kodi reads them in preference to anything else.

Still, in some cases, it might well be preferential to scrape some info, whilst preserving the rest.  Ratings being a case in point I guess.

In view of that, there is a mac version of Kodi here that can ignore nfo files if told to via an advancedsettings.xml entry.
EG

xml:
<advancedsettings>

        <musiclibrary>

                <ignorenfos>true</ignorenfos>

       </musiclibrary>

</advancedsettings>

This will make Kodi ignore any nfo files and re-scrape from the internet.  Scraped info is merged with the db so this should just allow you to scrape ratings but if I were you, I would make a copy of the db before attempting this.  If everything works out well, Kodi can export album and artist info to a directory of your choice (and re-import it form there), leaving your original Emby generated nfo files intact (I think, I can't test this sadly).  You can then re-build your Kodi music library using that exported info.

Note for anyone else reading this, there are also Win64, IOS-ARM64 and TVOS builds available on the mirrors should anyone want to test this further.

Note2 - This is based on v21 Omega and will therefore upgrade your video db to 121.  There is no music db upgrade and so that should stay at 82.  Once the scraped info has been added to the db, it should be possible to run v19 and see the scraped info in that version (as the db versions for music have not changed).  Videodb 121 can either be left or deleted if not required.

Hey thanks @black_eagle for looking at the code and for your suggestion to update the metadata by putting Kodi on my Mac (which I use as a server for anyone else just tuning in). 

I'll have a go on a small sample of my music later today and report back. I'll definitely back up as you suggest.

Just checking my understanding – when you say "You can then re-build your Kodi music library using that exported info" – I'm interpreting that as me going to my Nvidia shield at this point, and pointing Kodi on the shield to what the new Kodi version on my Mac has collated (albums/directory of choice metadata). I'm pretty sure that's right, and don't imagine much harm if I'm wrong, so I'll proceed that way if I don't hear back before I test this out.

I'll let you know if the nfo files stay intact since you can't test that yourself.

If the test works, I may use the approach for my whole library, but then updating the library every time I buy new music would be a three-step process (Emby and Kodi loads on Mac, then loading into Kodi on shield), so I'm hoping Emby or Plex might still be of service here so I can have the DB info in one place and done at one time. In particular:

• I've been assuming Emby (still) does not scrape album ratings, nor serve them to Kodi via Emby for Kodi NextGen. That was the case when this database was built a few years ago. I'll read up on the latest capabilities or ask the Emby devs.   
  
• I'll also look into if Plex scrapes and serves ratings. If it does I might just move from Emby to Plex as a server. From what I have gleaned today on quick web searches the free Plex version will work for that.
  
• If none of the above apply, I'll do the full DB using Kodi on my mac if the above test works. I'll just do the test for now.
  

I'll let you know how I go, thanks again.

@ptvminix I've split the discussion out into its own thread as there isn't actually an issue with the generic album scraper.

I don't really "get" why people use Emby and/or Plex with Kodi.  It seems to me that it just adds an extra level of complexity that I don't need. 

(2023-02-05, 04:39)ptvminix Wrote: [ -> ]Emby and Kodi loads on Mac, then loading into Kodi on shield

Err, why??  Kodi is capable of sharing its databases between devices so the db's on the shield should (or could be) automatically created and track whatever happens on your Mac.

(2023-02-05, 04:39)ptvminix Wrote: [ -> ]'m hoping Emby or Plex might still be of service here so I can have the DB info in one place and done at one time

Again, why do you think you need Emby or Plex for this ?  One instance of Kodi setup with shared databases can update those db's and every other Kodi client connecting to them gets the same info.

This is how the stuff works in my house.  One instance of Kodi (on the most powerful machine) does all the scanning, scraping and updating of the db's.  These are then shared to 3 bedrooms, the living room and the kitchen.  Playing something on one device automatically updates watched status / play count etc on every device.  FWIW, live TV and PVR recordings are also shared in a similar way.  Every Kodi client can play live TV and / or the recordings.  They can initiate a recording and it will be available to all clients.

I'm not knocking Plex or Emby here (particularly as I have never used either), I'm just genuinely curious as to what one or the other of them is giving you that Kodi can't do or can't do as well ??

Anyway, to answer your questions.

Kodi can export to a directory of choice.  If you export Albums and Album Artists to this directory, you can check that the output is indeed correct without affecting the original nfo files.  So, if they now include ratings and all the other stuff is still correct, then we can assume everything is working correctly.

You will then need to export again (you can delete the original exported stuff as Kodi won't use it) but this time to library folders.  This will write the album.nfo file in the correct directory for each album, overwriting what was there previously (the emby ones).  You will need to nominate an Artist Information Folder first though.  See this for info on the AIF, and this for more details on exporting nfo files.

Thanks very much @black_eagle for the link to the Kodi version. I haven't attempted to use it yet, as I have some matters to attend to for a week or two and believe I would benefit from a break from this, but I am grateful and I will report back how I go. I'm hearing you that this is not an issue with the scraper, it's to do with local nfo files blocking the scraping.
 

(2023-02-05, 09:07)black_eagle Wrote: [ -> ]I don't really "get" why people use Emby and/or Plex with Kodi. 

I just started with Plex some time ago and got used to and liked how I could edit/add/manage metadata for obscure things that will never have online tags, such as recordings of a friend's radio show. I then discovered XBMC thereafter and kept using plex as the server, and moved to Emby/Kodi thereafter.

I'm open to giving up on Emby as a server if Kodi enables me the environment to manage metadata in the way I want, I just never considered or explored that possibility, and I accept I should have before hitting you up for help. Again, I'm most grateful for your assistance. I'll let you know how I go when I can get back to this. Thank you.

"...that can ignore nfo files if told to via an advancedsettings.xml entry."

@black_eagle I have now tested the version of Kodi you kindly provided above, thanks again.

I'm pleased to report I had fantastic results in some testing I did last night on a small (backed-up) subset of my library. More specifically:

• I was able to scrape community ratings with Kodi on my Mac, and confirm the ratings were in the new (Mac) Kodi generated NFOs saved to my user-defined folder, and I could also see these ratings on Kodi on my Shield using the (easy) UPNP settings in both versions. Great
  
• Even better, I was then able to reimport these new ratings from the user defined folder into Kodi (on the Mac) and overwrite the old Emby NFO's without the ratings with the new Kodi ones. I also confirmed that for my bespoke/rare/manually made Emby NFOs, Kodi didn't overwrite them if it couldn't scrape any data for them, which of course is logical and expected but nice to see.
  
• So I believe the above will meet my needs, and I'm most grateful. Thank you.
  

... (final point TBC, just realised something....)

...(I was having trouble today with Kodi recognising the advancedsettings.xml, but then I realised Mac TextEdit makes RTF by default, so I went back to vi editor in the terminal and I got going again).

Anyway, I think my testing is complete, and I'm thrilled with the results, thanks so much.

I'll do the full library when I have more time late next week and will report back. Brilliant stuff, thanks!