[RELEASE] Data18.com Web Content Scraper - Adult Movie Web Downloads

  Thread Rating:
  • 2 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
signumda Offline
Junior Member
Posts: 16
Joined: Jun 2010
Reputation: 0
Post: #106
Hi, tke1384 same problem i get posters and thums but no fanarts.
find quote
tke1384 Offline
Junior Member
Posts: 7
Joined: Feb 2015
Reputation: 0
Post: #107
(2015-07-27 18:01)signumda Wrote:  Hi, tke1384 same problem i get posters and thums but no fanarts.

Do me a favour and send me the link to the movie or better yet a few movies you're not getting fan art for and I'll look into it.
find quote
DoctorD Offline
Member
Posts: 91
Joined: Apr 2013
Reputation: 4
Post: #108
Hi All,

Sorry I've been pretty lax on maintaining this scraper. I've been mostly working on my new adult movie scraping project which works much better and gets the fanart every time:

It's called JAVMovieScraper and can be found at https://github.com/DoctorD1501/JAVMovieScraper

If anyone else is interested in helping to maintain this Kodi XML scraper, please send me a private message as I would love to have a more active maintainer than myself.

TKE1384,

Is your version based on str1567's version? Should I try to merge your changes into the main branch?
(This post was last modified: 2015-07-29 05:59 by DoctorD.)
find quote
tke1384 Offline
Junior Member
Posts: 7
Joined: Feb 2015
Reputation: 0
Post: #109
(2015-07-27 18:30)tke1384 Wrote:  
(2015-07-27 18:01)signumda Wrote:  Hi, tke1384 same problem i get posters and thums but no fanarts.

Do me a favour and send me the link to the movie or better yet a few movies you're not getting fan art for and I'll look into it.

So it looks as though this may be a bug in Kodi itself. I found an old bug report speaking to just this. Some movies will have fan art pulled without issue if the page doesn't need to be spoofed but the pages that do require spoofing will fail for fanart only. If you copy the same XML into say posters for example the pages are spoofed and images are downloaded. I would leave the code as is if I were you and if this bug is fixed in the future you should already have the correct metadata in your database. Here's the bug report page if you're interested in reading it yourself.

http://trac.kodi.tv/ticket/14722
find quote
tke1384 Offline
Junior Member
Posts: 7
Joined: Feb 2015
Reputation: 0
Post: #110
(2015-07-29 05:57)DoctorD Wrote:  Hi All,

Sorry I've been pretty lax on maintaining this scraper. I've been mostly working on my new adult movie scraping project which works much better and gets the fanart every time:

It's called JAVMovieScraper and can be found at https://github.com/DoctorD1501/JAVMovieScraper

If anyone else is interested in helping to maintain this Kodi XML scraper, please send me a private message as I would love to have a more active maintainer than myself.

TKE1384,

Is your version based on str1567's version? Should I try to merge your changes into the main branch?

What's up Doc? I'd love to have you take a look at what I've done and give me your thoughts. I don't want to take away from your hard work but I'd love to help if I can however I can. Let me know if you think my updates are worth while and how I can help maintain going forward.
find quote
signumda Offline
Junior Member
Posts: 16
Joined: Jun 2010
Reputation: 0
Post: #111
(2015-07-29 05:57)DoctorD Wrote:  I've been mostly working on my new adult movie scraping project which works much better and gets the fanart every time:

Sorry, but I trey your Jav but the data18 scraper returned also no fanart the tmdb scraper returned some fanarts for "blockbuster" like pirates II.
find quote
DoctorD Offline
Member
Posts: 91
Joined: Apr 2013
Reputation: 4
Post: #112
Hey tke1384,

Sorry for the late response on this. If you could submit bug fixes as a pull request on github, that would be fantastic!

Here's the link to the github page:

https://github.com/xbmc-adult/xbmc-adult
find quote
StoningtonQB Offline
Junior Member
Posts: 25
Joined: Jun 2015
Reputation: 0
Post: #113
I think this has stopped working. I have tried using both Google and Data18. Tried renaming files to scenes that are definitely in the library to no avail. Suggestions?
find quote
Mordred Offline
Junior Member
Posts: 36
Joined: May 2011
Reputation: 0
Post: #114
(2016-02-16 04:46)StoningtonQB Wrote:  I think this has stopped working. I have tried using both Google and Data18. Tried renaming files to scenes that are definitely in the library to no avail. Suggestions?
It had stopped working (no idea how long ago, so I took it upon myself to fix it). I've submitted a pull-request to the xbmc-adult repo, so hopefully it'll get pushed out soon. You can also manually download the zip here.

Here's the changelist from Git:
  • Fixed Google, Bing, and data18 search engines.
  • Fixed a name collision which was always causing Google search to occur even if Bing or data18 was set.
  • Set Bing as default search engine because Google will temporarily return 503's after a few requests, and data18 requires exact title matches
  • Fanart retrieval was not working properly, and was only getting the single (low quality) video preview image, no matter how many higher quality sample images exist.
  • Poster uses first sample image, and fanart will use 3rd (if available) so that you don't see the same image for both.
  • Fixed local nfo retrieval

Verbose details:
I tried it on about 50 web videos I had. Google returned the most accurate results, but required some massaging of the names. Bing picked the right video about 80% of the time, but was able to return results on some poorly named files that Google wanted manually corrected. Google returns 503s and you get the nice "do you want to disable this plugin" message if you do more than a couple searches within minutes of each other. You can use Google, but I wouldn't recommend auto-updating your library in that case. I changed the default to Bing because of that.

I discovered that at least on my system $INFO[search_engine] caused a name conflict with another plugin that apparently ALSO had search_engine set to Google. No matter what I set search_engine to, it always searched Google. I renamed the setting to data18_search_engine and instantly I could search using Bing and data18 (although both of those were broken). They've obviously been fixed.

One of my 50 videos came up with results that were completely wrong. The title was just a single, easily searchable word, and my manual (oustide Kodi) google, bing, and data18 webbrowser searches using the same search strings the scraper uses all returned the correct video for (the word only appears in one video on data18). Google and Bing always returned some variation on the same 8-11 videos from the website. Adding the performers names didn't change the results. Adding the year didn't help. Adding the studio didn't help. Trying only the performers names didn't fix it. Finally after an hour of beating my head against the wall I gave up. If you have some videos that just won't return the right data for whatever reason, use a local nfo file like this and it'll work fine (cause I fixed that too):

Code:
<details>
    <url>http://www.data18.com/content/1159234</url>
</details>
(This post was last modified: 2016-02-19 09:16 by Mordred.)
find quote
Post Reply