Kodi Community Forum

OK I tried all that and it didn't work.

I unchecked the "use xbmc scraper" and re-scraped all and it worked this time.

(2022-06-04, 00:50)artistwantab Wrote: [ -> ]OK I tried all that and it didn't work.

I unchecked the "use xbmc scraper" and re-scraped all and it worked this time.

Yep, the older XBMC/Kodi Scraper probably will be stopped as Kodi has gone onto Python for their addon, so won't be compatible in MC very soon.

Got it.

I will leave it unchecked then from now on.

A great program....

About the only thing I am hopeful for the future is an option to re-scrape fanart even if fanart exists.  

Thanks...

I've only just stumbled upon MC while in search of a solution to the Music Video scraping shortcomings of Kodi.  I was very excited to see Music Video support touted as a major new feature but I can't seem to get it to work very well and hoping someone can help me out.

When I try to scrape my Music Videos I get very little success using any scraper.  With the wiki scraper, my log output looks like this:

php:
05/06/2022 17:45:57



******************************************************************************

* One Or more errors were detected during scraping. See below for details.   *

* To disable seeing this, turn off General Preferences -'Show log on error'. *

******************************************************************************



257 new movies found in [\\192.168.0.3\Music Videos]





A total of 257 new movies found -> Starting Main Scraper Process...





Scraping Title: 2 Unlimited - Here I Go 

Unable to scrape body with refs :



Scraping Title: 2 Unlimited - Tribal Dance

Unable to scrape body with refs :



Scraping Title: 4 Non Blondes - What's Up

Unable to scrape body with refs :



Scraping Title: Aaliyah feat  Timbaland - Try Again

Unable to scrape body with refs :



Scraping Title: Adam F - Circles cam

Unable to scrape body with refs :



Scraping Title: Adam F - Music In My Mind 

Unable to scrape body with refs :



Scraping Title: Adele - Cold Shoulder 

Unable to scrape body with refs :



Scraping Title: Adele - Hello

Unable to scrape body with refs :



Operation cancelled by user

Once cancelled, if I try manually pointing MC at the correct Wiki entry, I see this:



Which kind of looks like all the work done cleaning the filename is ignored when try to manually match.  Could the cleaning process also be getting skipped when scraping?  The log posted above seems to show filename cleaning is working but this manual Wiki matching makes me wonder. Here are a few filename examples from my collection:

2 Unlimited - Here I Go.DVD.FLAC2.0.x264.mkv
2 Unlimited - Tribal Dance.ProRes.FLAC2.0.x264.mkv
4 Non Blondes - What's Up.1993.DVD.FLAC2.0.MPEG2.mkv
Aaliyah - Back & Forth.DVD.PCM2.0.MPEG2.vob

I added ProRes, FLAC, MPEG2 etc to the "Video Source" and "Clean Filename" sections already.

The final issue I'm having is that I don't mind manually Wiki matching every video however once I've corrected the internal Google search, opened the correct Wiki page, verified artwork, release year etc are present and pressed [Go], only the title and artist actually get scraped.  The year and artwork which were present on the Wiki page are left and I have to manually fill them in.

Music video section still needs a lot of work, and without a good dedicated site to get the info from.

As for the naming, If I recall correctly, the format should be only <artist> - <title>.<extn>
Any other info will confuse MC. Again, this section needs a lot more work, so cleaning the filename is one I need to sort for the next release.

As for pulling the info from Wiki, MC is extracting from the HTML page, and as Wiki make changes to their layout, we miss some data sometimes.

Can you give some examples so I can do more testing?

First off I must thank you for Media Companion. It's by far my favourite and most used movie/TV scraper. Scraping music videos is a pain no matter what scraper is used so I can't criticize that part but I do have one request that I believe would be an improvement.

I was quite surprized to find that IMDB actually has a lot of music videos listed. Only problem is I have to scrape them as movies then do some editing of the nfo to convert it from movie type to music video format. Usually it's just a matter of adding the <album> tag and changing the first <actor> tag to <artist> and optionally the top and bottom tags from <movie> to <musicvideo>. I also manually add the Wiki info and use it fill in the missing fields like <studio> and <album> that IMDB doesn't normally have. That's all well and good since it beats creating a nfo from scratch by hand. It speeds things up to use an editor that can do batch file editing.

So, what I'm asking is if IMDB could be added as scraping source option in music video section?

Another trailer question....

Is it possible to store all trailers in one location?   I only ask because sometimes I am not sure what I want to watch.   So I simply search "trailers" via explorer and then grab all and move onto my movie player.   Its primitive but works until I see what I want to watch.  Would be nice if everything was in one place to begin with.

(2022-06-07, 03:52)artistwantab Wrote: [ -> ]Another trailer question....

Is it possible to store all trailers in one location?   I only ask because sometimes I am not sure what I want to watch.   So I simply search "trailers" via explorer and then grab all and move onto my movie player.   Its primitive but works until I see what I want to watch.  Would be nice if everything was in one place to begin with.

@artistwantab 

I would think so. as long as the full path to the trailer is added.

(2022-06-07, 03:52)artistwantab Wrote: [ -> ]Another trailer question....

Is it possible to store all trailers in one location?   I only ask because sometimes I am not sure what I want to watch.   So I simply search "trailers" via explorer and then grab all and move onto my movie player.   Its primitive but works until I see what I want to watch.  Would be nice if everything was in one place to begin with.

Not for Media Companion, and as far as I know Kodi only finds the trailer if it is with the movie.

(2022-06-06, 09:55)Juan Mortyme Wrote: [ -> ]First off I must thank you for Media Companion. It's by far my favourite and most used movie/TV scraper. Scraping music videos is a pain no matter what scraper is used so I can't criticize that part but I do have one request that I believe would be an improvement.

I was quite surprized to find that IMDB actually has a lot of music videos listed. Only problem is I have to scrape them as movies then do some editing of the nfo to convert it from movie type to music video format. Usually it's just a matter of adding the <album> tag and changing the first <actor> tag to <artist> and optionally the top and bottom tags from <movie> to <musicvideo>. I also manually add the Wiki info and use it fill in the missing fields like <studio> and <album> that IMDB doesn't normally have. That's all well and good since it beats creating a nfo from scratch by hand. It speeds things up to use an editor that can do batch file editing.

So, what I'm asking is if IMDB could be added as scraping source option in music video section?

Interesting, I did not know IMDb had music video's listed.

Can you give some search examples and I'll have a look see.

(2022-06-04, 01:28)artistwantab Wrote: [ -> ]Got it.

I will leave it unchecked then from now on.

A great program....

About the only thing I am hopeful for the future is an option to re-scrape fanart even if fanart exists.  

Thanks...

As in overwrite artwork? There is an option in Common preferences that is selected as default Don't overwrite existing artwork, but this will allow overwriting any and all artwork so disable and try with Batch rescrape Wizard but you have been warned.

@vbat99 

Sure, here are a few examples along with respective IMBD ID and link:
Run DMC and Aerosmith: Walk This Way - tt6722960
Aerosmith: Janie's Got a Gun - tt6663460
Alice Cooper: I'm Eighteen - tt9893934
Devo: (I Can't Get No) Satisfaction - tt12063820

If you look at the Aerosmith page for example you'll see 41 listed under "Actor" and a couple more listed under "Self".

The one advantage to scraping music videos as movies is the actors are scraped and added. This is handy because as of Kodi 19 the actors in music videos are shown (the same as movies) on the Video Info screen if the skin supports it.

(2022-06-07, 14:27)Juan Mortyme Wrote: [ -> ]@vbat99 

Sure, here are a few examples along with respective IMBD ID and link:
Run DMC and Aerosmith: Walk This Way - tt6722960
Aerosmith: Janie's Got a Gun - tt6663460
Alice Cooper: I'm Eighteen - tt9893934
Devo: (I Can't Get No) Satisfaction - tt12063820

If you look at the Aerosmith page for example you'll see 41 listed under "Actor" and a couple more listed under "Self".

The one advantage to scraping music videos as movies is the actors are scraped and added. This is handy because as of Kodi 19 the actors in music videos are shown (the same as movies) on the Video Info screen if the skin supports it.

I had a quick look and scraped Aerosmith Janie's got a gun from IMDb, and surprisingly it filled a lot of info.

I'll have a look to see whats involved in adding IMDb to the music scraper.

BUT, just to note. IMDb does have a limit to how many hits it takes before it returns false or no info. This seems to be a web-bot attack defence that no one can get around.
I also wonder what info OMDBapi might have as well.

@vbat99 

Ya, I don't even remember how I stumbled onto it. I know it was purely by accident while looking for something unrelated. But a good find none the less.

Would using a personal API number help defeat the IMDB limit? Or is it IP related? Or something else?

Thanks for looking into it. Scaping music vids has long been a sore spot for me. I have a few thousand of them and in all these years I've only managed to get a few hundred add to my library. But that's slowly changing. Now, using a couple workarounds it's a bit easier.

(2022-06-06, 00:37)vbat99 Wrote: [ -> ]Music video section still needs a lot of work, and without a good dedicated site to get the info from.

As for the naming, If I recall correctly, the format should be only <artist> - <title>.<extn>
Any other info will confuse MC. Again, this section needs a lot more work, so cleaning the filename is one I need to sort for the next release.

As for pulling the info from Wiki, MC is extracting from the HTML page, and as Wiki make changes to their layout, we miss some data sometimes.

Can you give some examples so I can do more testing?

Well the filename cleaning is working nicely, the entries in the log file I posted show it has done a very good job of cleaning the filenames down to just <artist> - <trackname> but it seems that the Wiki search function completely ignores all that excellent work and just pulls the filename, so hopefully this is an easy fix.

I've scanned a few more videos in to reproduce the Wiki scraping problem.  First one had the issue:

Alex Party - Don't Give Me Your Life

Released, Recorded attributes both present on the Wiki page, as well as artwork but only the Artist, Track and Plot actually get populated after the scrape.  Next video pretty similar:

Alice Deejay - Better Off Alone

Only artist, track and album pulled in this case, plot field gets "This Music Video could not be identified." inserted.

After doing another ten or so it seems in all cases, the Year is never successfully scraped, even though "Recorded" and "Released" dates are present on all Wiki pages.  Artwork occasionally gets pulled but only 1 or 2 times out of 10.  Album name is populated most of the time but sometimes missed.