Kodi Community Forum
AdultScraper for Ember released - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+---- Forum: Ember Media Manager (https://forum.kodi.tv/forumdisplay.php?fid=195)
+---- Thread: AdultScraper for Ember released (/showthread.php?tid=360299)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30


RE: AdultScraper for Ember released - VonKrolock - 2023-10-04

(2023-10-03, 19:46)adultmm Wrote:
(2023-10-02, 20:13)VonKrolock Wrote: @adultmm Thanks for getting back to me. Here it is https://paste.kodi.tv/ahoqabigej.kodi (I hope I did this right).

Thanks,

It seems to a normal timeout, however this works fine on my side, the page is really slow but works. The default timeout is 100sec, it should be enough. I'm thinking about how could I reproduce the error on my side...

I've been trying other videos and it seems to be video-dependent.
Any idea if there could be something on a video that could trigger the time-out behaviour? Like something in the file naming scheme, encoding type? Could that affect it?


RE: AdultScraper for Ember released - Juan Mortyme - 2023-10-04

@adultmm

I just scraped some scenes from Fake Taxi and the scene info was scraped but the actor thumb from https://www.iafd.com/graphics/headshots/arishlamborghini_f_arish.jpg was not. I tested the url in a browser and it worked fine. If I go to the scene page on IAFD and copy the actor thumb link  https://www.iafd.com/person.rme/perfid=arishlamborghini/gender=f/arish-lamborghini.htm and paste it in I still get no results. I tried the imgbox link from your post and it worked fine.


RE: AdultScraper for Ember released - adultmm - 2023-10-05

(2023-10-04, 12:20)Juan Mortyme Wrote: @adultmm

I just scraped some scenes from Fake Taxi and the scene info was scraped but the actor thumb from https://www.iafd.com/graphics/headshots/arishlamborghini_f_arish.jpg was not. I tested the url in a browser and it worked fine. If I go to the scene page on IAFD and copy the actor thumb link  https://www.iafd.com/person.rme/perfid=arishlamborghini/gender=f/arish-lamborghini.htm and paste it in I still get no results. I tried the imgbox link from your post and it worked fine.
I've just scraped this scene, and the thumbnail image was uploaded to imgbox: https://images2.imgbox.com/bb/15/3Vdhr2bm_o.jpg
Please send me the log file, maybe it could help to solve your issue. Thanks


RE: AdultScraper for Ember released - Trillian_M1 - 2023-10-06

Hi @adultmm, great work. I was just wondering could you add gaydvdempire.com as well? Would be a great addition.

Thanks


RE: AdultScraper for Ember released - Juan Mortyme - 2023-10-07

@adultmm

Here is a link to a zip file containing the results of some test scrapes using fakehub, theporndb and data18 including the log file for each. In order to keep the size down I did not include the media files but I can supply them if needed.


RE: AdultScraper for Ember released - adultmm - 2023-10-07

(2023-10-07, 05:58)Juan Mortyme Wrote: @adultmm

Here is a link to a zip file containing the results of some test scrapes using fakehub, theporndb and data18 including the log file for each. In order to keep the size down I did not include the media files but I can supply them if needed.

Thanks, 

Great pack. We'll fix every error. Smile

Let's see one of the videos: FakeTaxi - Sexy Mature MILF In Lingerie (2016)
1) Metadata downloaded from: https://www.fakehub.com/scene/4072181/-
2) Only one actor (John Petty) was found.
3) Searching image for John Petty, it was found on IAFD, and download it from: https://www.iafd.com/person.rme/perfid=iantate/gender=m/ian-tate.htm, size 170x200

I think this's the expected behavior. What do you want to happen? 

Thanks


RE: AdultScraper for Ember released - Juan Mortyme - 2023-10-07

(2023-10-07, 16:07)adultmm Wrote:
(2023-10-07, 05:58)Juan Mortyme Wrote: @adultmm

Here is a link to a zip file containing the results of some test scrapes using fakehub, theporndb and data18 including the log file for each. In order to keep the size down I did not include the media files but I can supply them if needed.

Thanks, 

Great pack. We'll fix every error. Smile

Let's see one of the videos: FakeTaxi - Sexy Mature MILF In Lingerie (2016)
1) Metadata downloaded from: https://www.fakehub.com/scene/4072181/-
2) Only one actor (John Petty) was found.
3) Searching image for John Petty, it was found on IAFD, and download it from: https://www.iafd.com/person.rme/perfid=iantate/gender=m/ian-tate.htm, size 170x200

I think this's the expected behavior. What do you want to happen? 

Thanks
@adultmm

That's correct that is the size of the image on IAFD and that would be what I would expect to be downloaded. However you can see in my test folders that no actor thumbs are actually being downloaded from IAFD. That is the main issue.

The rest is basically just my rambling and griping about the quality of thumbs obtained from other sources which understandably you have no control over.

I included ThePornDB and DATA18 results just to highlight the vast discrepancy in image quality and availability to illustrate the importance of why I feel having TMDB (which seems to be working fine) as the first choice and IAFD as the second choice with the studio site as a fallback for actor thumbs. Other sources, including many of the studios' own websites have actor thumbs that are not really suitable as they are full body shots or screenshots from scenes. IMHO actor thumb should be portrait headshots only as on TMDB and IAFD.

In short all I would like is a functioning workaround for IAFD so it works the way it did before IAFD changed. I'm not a coder so I don't have any knowledge of the complexities involved but would it be possible to trick IAFD into thinking that's just a web browser sending a "say as" request?

Thanks for taking the time to address this issue.


RE: AdultScraper for Ember released - brigante - 2023-10-08

(2023-09-30, 19:49)adultmm Wrote:
(2023-09-09, 00:38)brigante Wrote: Image

Hi @brigante,

I hope I fixed it in v0.12.3. Please try it.

Hello and thanks for responding. Between JAV movies I get this error and if I don't accept it, it doesn't advance to the next movie. It's not that it doesn't download the metadata, it's that it stops in the process. To this we add that right now in jav movies they don't seem to be adding actresses. I don't know if this may have to do with my search settings in Ember Media...I only select "JAV DATABASE"

ImageImage


RE: AdultScraper for Ember released - adultmm - 2023-10-08

(2023-10-08, 01:43)brigante Wrote:
(2023-09-30, 19:49)adultmm Wrote:
(2023-09-09, 00:38)brigante Wrote: Image

Hi @brigante,

I hope I fixed it in v0.12.3. Please try it.

Hello and thanks for responding. Between JAV movies I get this error and if I don't accept it, it doesn't advance to the next movie. It's not that it doesn't download the metadata, it's that it stops in the process. To this we add that right now in jav movies they don't seem to be adding actresses. I don't know if this may have to do with my search settings in Ember Media...I only select "JAV DATABASE"

ImageImage

Hello, please give me the movie url or at least the name to reproduce the error.


RE: AdultScraper for Ember released - adultmm - 2023-10-08

(2023-10-06, 17:38)Trillian_M1 Wrote: Hi @adultmm, great work. I was just wondering could you add gaydvdempire.com as well? Would be a great addition.

Thanks

Hi @Trillian_M1,

It's not a big deal, so it'll be available in the next release.


RE: AdultScraper for Ember released - adultmm - 2023-10-08

(2023-10-04, 03:18)VonKrolock Wrote:
(2023-10-03, 19:46)adultmm Wrote:
(2023-10-02, 20:13)VonKrolock Wrote: @adultmm Thanks for getting back to me. Here it is https://paste.kodi.tv/ahoqabigej.kodi (I hope I did this right).

Thanks,

It seems to a normal timeout, however this works fine on my side, the page is really slow but works. The default timeout is 100sec, it should be enough. I'm thinking about how could I reproduce the error on my side...

I've been trying other videos and it seems to be video-dependent.
Any idea if there could be something on a video that could trigger the time-out behaviour? Like something in the file naming scheme, encoding type? Could that affect it?

I really have no idea, but in the next version, I'll implement retrying 3 times on timeout. I hope it'll solve your problem.


RE: AdultScraper for Ember released - adultmm - 2023-10-17

(2023-10-08, 02:15)adultmm Wrote:
(2023-10-06, 17:38)Trillian_M1 Wrote: Hi @adultmm, great work. I was just wondering could you add gaydvdempire.com as well? Would be a great addition.

Thanks

Hi @Trillian_M1,

It's not a big deal, so it'll be available in the next release.

Hi @Trillian_M1
Implemented in v0.12.4. Enjoy!


RE: AdultScraper for Ember released - adultmm - 2023-10-17

(2023-09-30, 19:59)adultmm Wrote:
(2023-09-11, 14:07)Juan Mortyme Wrote:
Quote:What do you mean by "scrape scenes from IAFD"? Could you give me some examples?
@adultmm

IFAD contains scene info as well as movies. In many cases the info on IAFD is more comprehensive than that on the "Official" websites, some of which don't even list the release dates. Finding scenes is bit of a hit and miss proposition as their listings are far from all inclusive but what they do have often times is more thorough, including release date, studio, distributor, description, a more complete performer list including aliases a performer may have used in credits for that scene, and occasionally even directorial credit.

Examples I quickly found:
Official webpage vs IAMD listing for the same scene or Official webpage vs IAMD listing.

I was thinking that it might be handy to be able add IAFD as a scaping option.

Thanks.
Hi @juan Mortyme,

I've fixed IAFD scraper in v0.12.3.

Thanks for the IAFD scene idea. The info that is a web scene or a movie isn't visible on the search result page, but I'll try to implement it.


Hi Juan,
IAFD scene scraper has been implemented in v0.12.4.
You have to enable it before using it. If you find any problem, please report it. Thanks


RE: AdultScraper for Ember released - adultmm - 2023-10-17

(2023-10-07, 23:30)Juan Mortyme Wrote:
(2023-10-07, 16:07)adultmm Wrote:
(2023-10-07, 05:58)Juan Mortyme Wrote: @adultmm

Here is a link to a zip file containing the results of some test scrapes using fakehub, theporndb and data18 including the log file for each. In order to keep the size down I did not include the media files but I can supply them if needed.

Thanks, 

Great pack. We'll fix every error. Smile

Let's see one of the videos: FakeTaxi - Sexy Mature MILF In Lingerie (2016)
1) Metadata downloaded from: https://www.fakehub.com/scene/4072181/-
2) Only one actor (John Petty) was found.
3) Searching image for John Petty, it was found on IAFD, and download it from: https://www.iafd.com/person.rme/perfid=iantate/gender=m/ian-tate.htm, size 170x200

I think this's the expected behavior. What do you want to happen? 

Thanks
@adultmm

That's correct that is the size of the image on IAFD and that would be what I would expect to be downloaded. However you can see in my test folders that no actor thumbs are actually being downloaded from IAFD. That is the main issue.

Hi @Juan Mortyme,

As you can see, the image has been downloaded in the "Results scraped from ThePornDB.zip" and not found in the "Results scraped from FakeHunb.zip".

The problem is visible in the log file:
2023-10-06 04:39:12.3308,EmberAPI.HTTP,EmberAPI.HTTP.DownloadImage,19,ERROR,DownloadImage <https://www.iafd.com/graphics/headshots/iantate_m_iantate_vivthomas.jpg>,"EXCEPTION OCCURRED:System.Net.WebException: The remote server returned an error: (403) Forbidden.

It means that IAFD noticed that we are not a real browser and disabled to download the image. In v0.12.4, I use ImgBox proxy, which makes the scraping slower but solves this problem.
 
(2023-10-07, 23:30)Juan Mortyme Wrote: The rest is basically just my rambling and griping about the quality of thumbs obtained from other sources which understandably you have no control over.

I included ThePornDB and DATA18 results just to highlight the vast discrepancy in image quality and availability to illustrate the importance of why I feel having TMDB (which seems to be working fine) as the first choice and IAFD as the second choice with the studio site as a fallback for actor thumbs. Other sources, including many of the studios' own websites have actor thumbs that are not really suitable as they are full body shots or screenshots from scenes. IMHO actor thumb should be portrait headshots only as on TMDB and IAFD.

Okay, in a future version, I will implement a way to modify the order or method for selecting actor images. The corresponding issue is here.
 
(2023-10-07, 23:30)Juan Mortyme Wrote: In short all I would like is a functioning workaround for IAFD so it works the way it did before IAFD changed. I'm not a coder so I don't have any knowledge of the complexities involved but would it be possible to trick IAFD into thinking that's just a web browser sending a "say as" request?

We could do it, but Kodi or other media managers won't, so they would also get an error. So we do the trick, and upload the image to the ImgBox, as I has written earlier.
(2023-10-07, 23:30)Juan Mortyme Wrote: Thanks for taking the time to address this issue.

You welcome.


RE: AdultScraper for Ember released - brigante - 2023-10-18

(2023-10-08, 02:13)adultmm Wrote:
(2023-10-08, 01:43)brigante Wrote:
(2023-09-30, 19:49)adultmm Wrote: Hi @brigante,

I hope I fixed it in v0.12.3. Please try it.

Hello and thanks for responding. Between JAV movies I get this error and if I don't accept it, it doesn't advance to the next movie. It's not that it doesn't download the metadata, it's that it stops in the process. To this we add that right now in jav movies they don't seem to be adding actresses. I don't know if this may have to do with my search settings in Ember Media...I only select "JAV DATABASE"

ImageImage

Hello, please give me the movie url or at least the name to reproduce the error.

Hello. Download the latest version and adding these two entries downloaded the metadata smoothly and without any hang-ups. EBWH-023 and STARS-923... When I added two films by Saki Okuda, however, I couldn't find any metadata and an error occurred. The films were ADN-146 and DV-1305.