Intelligent movie scraper
#1
Hello,

I have many movies which filename does contain more than only the name of the movie. Thus the movie scraper does not find them and they are not added to my Kodi library. I just entered a filename into the search mask of IMDB and I get correct results there.

So I suggest to make the movie scraper a bit more intelligent to also add movies which does not fit to the style that is suggested.
Reply
#2
I suggest you stfu and rename your files to fit with the standard like you should and like everyone else does. You sound like a prima donna

Too harsh? Wink
Reply
#3
I would do that and also have set up a script to create symlinks which fit the standard, but they do not work correctly via NFS, where the original file name is used again.

Renaming the original files is something I would rather not like to do as I store additional information in them. This is why I came up with this suggestion and I cannot see why a more intelligent scraper should be wrong. The question is, whether it is realizable.
Reply
#4
where should we take that intelligence from?
AppleTV4/iPhone/iPod/iPad: HowTo find debug logs and everything else which the devs like so much: click here
HowTo setup NFS for Kodi: NFS (wiki)
HowTo configure avahi (zeroconf): Avahi_Zeroconf (wiki)
READ THE IOS FAQ!: iOS FAQ (wiki)
Reply
#5
I think our video library scanning needs to become modern and support internal metadata. Then file names can be whatever the heck people want. There's even been work started on this.

Before anyone says "but no one stores metadata in their video files", that's a chicken and the egg problem. You do one and the other will follow.
Reply
#6
Metadata is probably the safest way and it is also used by the audio scraper. The problem is currently the one you mentioned plus the metadata seems to be complicated to edit, but I hope that both things will change in the future.

But regarding the filenames and where to take the intelligence from: Go to the IMDB main page and type the following into the search field: "Avatar xyz zxy" the top entry in the suggestions is still the movie Avatar. (Unfortunately this does not apply to the results). So this would be some kind of intelligent suggestion which is not that strict due to filename conventions.
Reply
#7
we are not stupid. of course it does fuzzy matches to the extent possible. but it will always be at the mercy of the search engine.
Reply
#8
If everything possible has already been done than I am ok with that and wait for the meta data functionality.

Another option would be to tell the scraper how the files are named using wildcards. In my names I have additional information at the end written in brackets. But this might be to complicated/demand is too low.
Reply
#9
have fun
http://kodi.wiki/view/Advancedsettings.xml#cleanstrings
Read/follow the forum rules.
For troubleshooting and bug reporting, read this first
Interested in seeing some YouTube videos about Kodi? Go here and subscribe
Reply
#10
Thanks, will try this. Looks like it should solve my problem.
Reply
#11
I understand the movie (show) folder structure, have the same, although the problem is, that the show name contains additional information (see list below).
Is there a way to strip such information from just the folder-name (and ev. just for shows)?

It would be ideal of scraper could remove such also from show folder names, but i think the same would apply to music album and video folder names...

EXTRAS IN FOLDER NAME:
[Complete, The_Complete, TheComplete, The.Complete] S1, S01, Season 1, Season1, Season.1
S1-S10, S01-S10 (special case)
Video formats
Release (team) details
Dubbing details

TV SHOWS:
20:05:26 T:4808 WARNING: No information found for item 'C:\Shows\RU\Masha and the Bear (2009-2014) (S01E01-45) (gixerk9)\Masha and the Bear S01E01.mp4', it won't be added to the library.
20:01:46 T:4808 WARNING: No information found for item 'C:\Shows\EN-GB\Secretmillionairesclub WWW S01\ep26.flv', it won't be added to the library.
20:05:26 T:4808 WARNING: No information found for item 'C:\Shows\Banshee.S01.HUN.BDRip.x264-HSF\', it won't be added to the library.
20:05:26 T:4808 WARNING: No information found for item 'C:\Shows\Better.Off.Ted.S01-S02.MiXED.XviD.HUN-DART\', it won't be added to the library.
20:05:27 T:4808 WARNING: No information found for item 'C:\Shows\Breaking.Bad.S01.MiXED.XviD.Hungarian-TvTiME\', it won't be added to the library.
20:05:29 T:4808 WARNING: No information found for item 'C:\Shows\Burn.Notice.S01.Web-DL.x264.Hun.Eng-MaMMuT\', it won't be added to the library.
20:05:31 T:4808 WARNING: No information found for item 'C:\Shows\Californication.S01.720p.HDTV.Dual.x264-ZiP\', it won't be added to the library.
20:05:40 T:4808 WARNING: No information found for item 'C:\Shows\Dexter.S06.BDRiP.x264.HuN.EnG-HDTV\', it won't be added to the library.
20:05:41 T:4808 WARNING: No information found for item 'C:\Shows\Dexter.The.Complete.S03.HDTV.XviD.Hungarian-TvTiME\', it won't be added to the library.
20:05:42 T:4808 WARNING: No information found for item 'C:\Shows\Dexter_Season_2_DVDRip_Dual_XviD-CLT\', it won't be added to the library.
20:05:51 T:4808 WARNING: No information found for item 'C:\Shows\Extras.S01.RETAiL.HUN.DVDRip.XviD-DWP\', it won't be added to the library.
20:06:00 T:4808 WARNING: No information found for item 'C:\Shows\Game.of.Thrones.S01.720p.HDTV.x264.HunEng-AXIOME\', it won't be added to the library.
20:06:04 T:4808 WARNING: No information found for item 'C:\Shows\Hallo.Hallo.S01-S09.The.Complete.Series.Hun-NoGrp\', it won't be added to the library.
20:06:05 T:4808 WARNING: No information found for item 'C:\Shows\Heroes S01\', it won't be added to the library.
20:06:09 T:4808 WARNING: No information found for item 'C:\Shows\How.I.Met.Your.Mother.COMPLETE.S04.HUN.DVDRip.XviD-MuTTLeY\', it won't be added to the library.
20:06:16 T:4808 WARNING: No information found for item 'C:\Shows\Its.Always.Sunny.in.Philadelphia.S01.HUN.DVDRip.XviD-SRT\', it won't be added to the library.
20:06:18 T:4808 WARNING: No information found for item 'C:\Shows\Kockafejek.S01.IT.Crowd.Complete.S01.DVDRip.Dual.XviD-SzMoRyBoY\', it won't be added to the library.
20:06:28 T:4808 WARNING: No information found for item 'C:\Shows\Monk S7\', it won't be added to the library.
20:06:33 T:4808 WARNING: No information found for item 'C:\Shows\Outer limits S1 - Vegtelen hatarok E1\', it won't be added to the library.
20:09:05 T:4808 WARNING: No information found for item 'C:\Shows\Vikings.S01.COMPLETE.HUN.WEB-DL.x264-R4Z3R\', it won't be added to the library.
20:09:05 T:4808 WARNING: No information found for item 'C:\Shows\Vikings.S01.COMPLETE.HUN.WEB-DL.x264-R4Z3R\Vikings.S01E01.HUN.WEB-DL.x264-R4Z3R\', it won't be added to the library.

MUSIC VIDEOS:
20:03:29 T:6620 WARNING: No information found for item 'C:\Music VIDEO\HD\David Bowie - Best Of Bowie [x264-ac3 2.0]\David Bowie - Best Of Bowie - 1.27 - Dancing in the Street.mkv', it won't be added to the library.
20:03:29 T:6620 WARNING: No information found for item 'C:\Music VIDEO\HD\David Bowie - Best Of Bowie [x264-ac3 2.0]\David Bowie - Best Of Bowie - 2.00 - DVD Menu.mkv', it won't be added to the library.
Reply
#12
or install couch potato and use the renamer - it'll clear up the mess
Reply
#13
@hidegh

The problem in your case you don't actually have show folders for most of those. You might be able to get it to work using clean strings, so long as they are after the title, as Kodi can combine multiple folders into a single show.
Reply
#14
(2015-09-13, 02:16)Ned Scott Wrote: I think our video library scanning needs to become modern and support internal metadata. Then file names can be whatever the heck people want. There's even been work started on this.

Before anyone says "but no one stores metadata in their video files", that's a chicken and the egg problem. You do one and the other will follow.

I've been adding cover art into my mp4s since I found out you could store that info, and just recently (last 2 days) I decided to go through and tag all of my metadata....so +1 for me! Big Grin
Reply
#15
I'm really surprised at the lack of internal metadata support, not just in Kodi but in most of the major media players out there (both open and closed source). It would make things so much easier if that would catch on. Especially for the online scraper websites, as it would greatly reduce traffic. People would only need to scan just once when they were ripping/preparing their video files. All future scans would be local, without messy NFO files and images spread across hard drives, etc. No worrying about file names or messing with regex. No need to organize files in a special way. There's already some great open source tools that will add data to MP4's and MKV's that hook into The Movie DB and TVDB.

IIRC, notspiff/ironic_monkey did some proof of concept work on this a while back. There is some hope :)
Reply

Logout Mark Read Team Forum Stats Members Help
Intelligent movie scraper0