Kodi Community Forum
Release JAV Movie Scraper - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release JAV Movie Scraper (/showthread.php?tid=200788)

Pages: 1 2 3 4 5 6 7 8 9

RE: JAV Movie Scraper - macminiGIANT - 2015-01-06

Can this be run on OSX? I know there's a built in jar launcher; but i get an error message kicked back to me.

Update: Ah, nvm. Realized I didn't even have Java installed. It works now with a simple double-click.Smile

RE: JAV Movie Scraper - macminiGIANT - 2015-01-07

Follow up:

I am loving this scraper! I was originally planning on manually entering nfo data and editing fanart/poster art for part of my collection over the course of the next week or two. Not any more...all done!

There were a few hiccups and extra time spent trying to figure out what naming schemes fit with which scraping sites...but I think i reliably got 95% of my backlog scraped with the other 5% requiring some manual editing.

Your recent addition of the R18 scrape is a much welcome addition, too. Thanks Doctor D.

RE: JAV Movie Scraper - limkimkeong - 2015-02-03

Dear DoctorD,

First of all, thank you for the JAV Movie Scraper, it really make the task for organize the file time saver.

Being use it for almost a month. Most of the folder with the JAV ID can be detect properly.

However, have still left a portion with

10musume-070413_01 should use what to Scrape? should wait for create new release support for that?

Pacopacomama 093014_256 should use what to Scrape? should wait for create new release support for that?

XXX-AV-21844 should use what to Scrape? should wait for create new release support for that?

Or someone have the experience to solve the detection that, look forward for your kind advice.

Thank you in advance.


RE: JAV Movie Scraper - DoctorD - 2015-02-07

If you don't see the name of the site under specific scrape and it isn't a JAV dvd release, then I haven't made a scraper for it yet. You can request a scraper under the github issues page and maybe either myself or someone else will eventually write a scraper for that site.

RE: JAV Movie Scraper - Arc69 - 2015-04-02

Thanks for the program however I have some sugestions/problems:

Please add title IDs to the front of the title, as mentioned by someone earlier.

Seems to be very buggy scrapping several files at once, I think this might have to do the excessive filename length and illegal characters the the name (If I scape manually file by file I have to often shorten/change the title).
Would it be possible to set a maximum title size and automatically remove illegal characters?

I already have a large collection of movies that I store as ID-title, with covers and a copy of the cover jpeg as a TBN so it shows up in XBMC, hence I don't need or want to rename this (it took a lot of effort to manually find and modify these!).
However when scrapping your program it seems to search for the whole filename not just the product ID (which dosent work), would be possible for your program to have a setting where it would just search the first few characters of the file eg until the first space?

eg file RCT-705 titlename would just search RCT-705 and ignore the title after the space.

Would it be possible to add an option to remove certain words from a title eg rape, incest?

Would it be possible to remove selected sites from the jav scrapper eg I noticed R18 and Javzoo can take a very long time or hang the program.

Any chance of adding DMM as a separate scrapper option?

Sorry for the long post, your program has a lot of potential, the idea of having a JAV database here I can scan my collection for certain studios and actress searchs is very appealing


RE: JAV Movie Scraper - DoctorD - 2015-04-22

Hey Arc69,

I've released 0.1.25-alpha which has support for an option to grab the ID from the first word of the file instead of the end, an option to prefix the ID to the title field, as well as an option to only use DMM as a specific scraper.

For the problems with excessive file names, are you also using the file renamer function at the same time? Can you PM me the full paths of the 3 or 4 files you are scraping that won't work?

The remove certain words thing has been requested by another user or two, so I've added it to my issues page on github. It isn't my highest priority and I have a pretty big backlog, so I probably won't get to it right away. That said, if a contributor wants to write this and submit a patch, I'd be welcome to review the pull request. Smile


RE: JAV Movie Scraper - lesarros - 2015-05-01

DoctorD,you are my hero!Thanks a lot for the scraper.

RE: JAV Movie Scraper - MrTimscampi - 2015-05-04

Hey Pr.Sinister, thanks for making me find one of my old scripts Big Grin Didn't even know people were using it.
I've found the original script and put it up on my GitHub for archiving (Always cool -and embarrassing- to be able to read my old code).

Also, pretty cool to learn about DoctorD's scrapper (Always wanted to make one myself but have always been too busy/lazy to do so). I'll see what it's capable of Smile

RE: JAV Movie Scraper - Pr.Sinister - 2015-05-09

Hehehe... Pretty cool! I love PythonCoverCrop... I am busy with losta projects right now so i haven't had time to try and look at modifying your script so that it generates a poster.jpg AND back.jpg

If you every get the time, it would be appreciated Wink

RE: JAV Movie Scraper - trinity2111 - 2015-05-25

Hi Dr D,

Thanks for this scraper it truly is amazing. I do have a question which must have an easy fix but don't know how to do it.

Once scraping the file from Data18, how do I get the program to NOT rename the title of the movie itself. I would like the title of the movie to remain the same as the file name. E.g. I've already renamed a file before scraping to my liking, but once it's been scraped, in the title section it changes from the file name to the scraped movie name.

For example, I rename a file.mp4 to "Brazzers - Mia Malkova - bla bla bla.mp4"

Once the file gets scraped, I would like this name to stay in tact for the title too not just the file name.

Hope that makes sense, please get back to me when possible, I love the scraper, this is the only issue I have with it.


RE: JAV Movie Scraper - DoctorD - 2015-05-27

Hey trinity2111,

There's no current way to do what you're wanting to do automatically in the scraper program yet. I've added it as a requested enhancement on the issues page here:


When I have some time, I'll add that feature in, or if anybody else wants to code it they can also submit the patch on github as a pull request and I will incorporate their patch into the main build.


RE: JAV Movie Scraper - Enlightened - 2015-07-01

Love the program. This and mediaelch are the only things I've found to scrap porn. But this is way better with it' scene scrapping. Two things that bug me. One is the naming and moving of files.
As of now when I hit write file to data it will rename the file fine. But if I'm processing a folder it will only rename the folder and not touch the files inside. I like the way mediaelch will process a movie folder especially with split scene. I will rename the files inside and add -CD1 _CD2 ect. to the end. So for now any movie mediaelch can't process I use jav movie scraper with data18. Then I'll do the renames in Mediaelch. I would like to just cut mediaelch out all together. The second thing is how long the file names are. I have mine set to <SET> - <TITLE> [<ID>] (<YEAR>). The title is the one that gets really long. Is there a way to limit it?

RE: JAV Movie Scraper - DoctorD - 2015-07-02


For the renaming feature request, I have an open issue about it on github. When I have time, I'm sure it's something I'll try to add. I need to be a little more careful with it, which is why I haven't done it yet, because I don't want to rename the wrong files.

For the issue you mention about the length of the filename, can you be a little more specific? Is what is happening that the filename is so long that the program is truncating the filename and the ID and year get cut out? Maybe what I can do is order the fields by length of characters and prioritize the shortest ones as being the in the filename first before truncating starts to happen on the longest fields. It's typically the <ACTOR> or <TITLE> field that cause super long file names, so another lazier option I could maybe do is just prioritize those two last.

RE: JAV Movie Scraper - Enlightened - 2015-07-02

Its not cutting anything off of the file name which is making it to long for windows. It is the <TITLE> field that is really long. So I thinking you could add a limit to <TITLE> as the other fields are fine. also what is the exact file naming data18 is looking for? <SET> - <TITLE> [<ID>] (<YEAR>) I think ID might need to be moved to the end? Does the scrapper work with just scenes from movies? Because I can find the exact scene and info. Then when adding the link it doesn't scrap anything.

RE: JAV Movie Scraper - DoctorD - 2015-07-02


You may be using the data18 movie instead of the data18 WebContent scraper. Data18 movie is for full length DVD/bluray releases. Data18 WebContent is for split scenes and content that is download only from subscription sites.

It's somewhat lenient on what you have as your name for data18 content since it does a google search if it can't find it the first time. The first pass it tries to do just uses the site's own internal search which is pretty terrible. That usually won't find a match, but if you name your file exactly the same as the the <TITLE> field the internal search may work. But that's pretty inconvenient so you can probably just rely on the google search method. When in doubt, feed in your filename, minus the name of the extension (.wmv, mp4, etc) plus "site:data18.com/content" without the quotes into google and you can see what google would return for your filename.

It should be cutting a little bit off the file name now (though not from the title field, specifically), if you're using the latest development build. It's linked on the main github page under step 2, which has a link to a mediafire download. It should show v0.1.36-alpha in the Help -> About dialog.

This forum uses Lukasz Tkacz MyBB addons.