[AssFaceBook]

Would it not be better to scrape these as TV Shows? Surely there are perverts enough to make some banners etc, for the most popular sites at least. Guess it cant be so hard to batch edit the movie nfo files and make them the episode infos.. actually havent even tried the scraper yet but maybe I'll give the good cause a night and try it out. I'll post some results if I do. Cheers

[AiWaBR]

Would it not be better to scrape these as TV Shows? Surely there are perverts enough to make some banners etc, for the most popular sites at least. Guess it cant be so hard to batch edit the movie nfo files and make them the episode infos.. actually havent even tried the scraper yet but maybe I'll give the good cause a night and try it out. I'll post some results if I do. Cheers

Also wanted so, I think would be more organized and easier to access.

It would be nice if they developed a similar site thetvdb.com adult sites, would help for sure.

---

I've been doing some scrapping for college rules and for some of them rather then putting
<set>College Rules</set>

<set>You guys are awesome, we have been receiving so many submissions from all you crazy college kids, but this one here takes the cake! ... </set>

(repeating the same description as in the plot)

eg http://www.data18.com/content/update_62580.html

Also data18 search seems to be beyond broke for searching for scenes, ie searching for "Scavenger Hunt" brings up the above as a possible link and yet putting "Fucken Scavenger Hunt" brings up nothing at all.

Must admit for all there faults it does make me appreciate thetvdb and themoviedb so much more.

P.S. Why is there not something like theadultdb

Yeah theadultdb would be awesome lol

+1

+1

[AssFaceBook]

Still haven't tried this one out. Tested the scraper and it worked nicely tho. I think I'll make some time this weekend for the Movie->TVShow test..
Made a small porn-test-playlist on AeonNox:


edit: did a quick test and it seems to work fairly easy and good!


All i did here was to export the .nfo files (settings-video-export library). Didnt export any images.
Then replaced (notepad++ have a nice 'replace all in all opened documents' option here):
movie> -> episodedetails>
<epbookmark>0.000000</epbookmark> -> <showtitle>X-Art</showtitle>
<top250>0</top250> -> <episode>0</episode>
year> -> season>

the small problem here is that one then have to rename the vid & nfo file with the year and ep..
2013x00 filename.avi & 2013x00 filename.avi

[DoctorD]

The problem with scraping them as TV shows is exactly what you've found: you got to rename your files and then also everything is episode zero since there's no clear episode number one can use.

On the plus side, tv shows support the date (not just year) the episode aired, which is something movies don't have, which is pretty nice actually since you can then sort by that. It's also nice to be able to group things into one folder in tv shows, as you've done, but you can actually accomplish that with movie sets too (which this scraper does). Just make sure you enabled the option in XBMC to group items into sets when browsing. Then if you want to seperate DVD movies out from websites in your "Movies" category, you can create a smart playlist based on the folder location of your scraped website scenes (just keep them in a seperate root directory than DVD movies). Most of the popular skins then allow you to add the smart playlist as a main menu item.

By scraping everything as a movie, you also get the advantage of not separating your Movie (from dvds, blurays, etc) actor list from your tv actor list from websites, which is nice when trying to search for all work on a specific actor.

I agree with people in this thread that it would be nice to have a better site than data18 to get this data from. At the moment though, I haven't found one. If anyone sets one up or knows of a better site, let me know and I'll make a scraper for it!

[AssFaceBook]

Yeah I know I could save them as movies almost as nicely but i prefer the banners & since I dont use sets for my normal movies either this suits me better. On the Data18 site there is a full date when 'aired' but the scraper just saves the year. I now skipped the 2013x01... and just rename all to 0x0.. then replaced the year tag with <aired>****-1-1</aired> so its possible to sort them by year but not by exact date. Personally I can live without having my porn sorted by exact release date I'm sure there is some easy batch filename renamer to manage big collections as well

I changed over a hundred of these in a matter of minutes with a notepad++ macro so its really an easy task to do. I have lots of other custom 'TV Shows' with all sorts of sports and such so its kind of familiar..

[AssFaceBook]

Is there anyway to make the scraper save the nfo's when it scrapes? I'll try to make a easy step-by-step for the movies to TVShows conversion this weekend..

[Solo0815]

How can I get "Sex-Art" working with this scraper? Does anyone has a tip for me? Thx

[ksdmg]

At first, i wanna say thank you! Works pretty great for me, apart from the fact, that i dont have any fanarts. The covers, models etc. are loaded, but no fanarts show up apart from twistys scenes. So all my brazzers scenes have no fanarts. Any ideas?

[DoctorD]

The fanarts not showing up is because of a bug/missing feature in XBMC dealing with no URL spoofing attribute allowed for fanarts.

See this bug I filed for more info:

http://trac.xbmc.org/ticket/14722

It's rather unfortunate, but there's not really much I can do until the XBMC devs add this feature.

[cjd2004]

The HotMovies Description is not loading (You will need to scrape the 'descfullcontent' Javascript variable).

[Pr.Sinister]

Hi DoctorD,

Somewhere you say that your dmm.co.jp scraper works better in Ember... How did you create it for Ember?

Can your data18.com scraper be integrated in Ember 1.4?

Also, if you are making something for Ember, would you be able to make an Ember scraper for http://javlibrary.com/en/ ? It's so much better than DMM because it's in English and has the basic info needed...

I have a bunch of batch files and csv files i use to create my NFO's for JAV, XXX Movies and XXX Websites but being able to create the nfo from scraped data witha GUI tool would make my life so much easier!

Thanks in advance!

P.S. I have long wanted to create a JAVdb website with an API for easy scraping but i don't know how to setup the API. Setting up the site and the db should be easy enough...

[DoctorD]

Hi All,

Since there was some mention of wanting to scrape Japanese Adult Videos (JAV) within this thread, I wanted to post a link to the new alpha version of the program I wrote to do this. You can find it here:

https://github.com/DoctorD1501/JAVMovieScraper

I wrote this program as a standalone program instead of a XBMC XML scraper so that it can amalgamate data from a variety of different sources, clean up data, crop the poster element, and do machine translations. Please try it out and let me know about any issues or requests you have with it on github (not on here - this is the wrong thread for that!).

Thanks and happy scraping!

[Pr.Sinister]

I have started a new thread to discuss the promising new JAV Movie Scraper by DoctorD.

Here is the new thread : http://forum.xbmc.org/showthread.php?tid=200788

[jiggsaw]

Is there anyway you can make a data18 standalone version as well?

[DoctorD]

I may be able to adapt some code to get that working with the standalone program. It's actually not a bad idea, come to think of it.

I've never been super happy with the experience of the XBMC scraper due to that fanart bug mentioned previously in the thread and the fact that it's cumbersome to correct entries within XBMC if it didn't find a match.

It'll take me a while though to get this working though and I have some features I want to get working with the JAV stuff first before I write this.