Kodi Community Forum
AdultScraper for Ember released - Printable Version

Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+---- Forum: Ember Media Manager (https://forum.kodi.tv/forumdisplay.php?fid=195)
---- Thread: AdultScraper for Ember released (/showthread.php?tid=360299)

Pages: 1 2 3 4 5 6 7 8 9 10


AdultScraper for Ember released - adultmm - 2021-02-03

Hi Guys, 

I've started to develop a new scraper for Adult Movies and Videos. It's only in beta version, so any remarks or suggestions are very welcome. 

Implemented sources: AdultDVDEmpire, TMDB, Brazzers.com, BangBros.com, DogfartNetwork.com, Kink.com, NaughtyAmerica.com, RealityKings.com. I'm working on implementing more, and you also can ask for your favorite.

I hope, you'll find it useful.

IMPORTANT: When you update for a new version DO NOT override the Profile folder!


UPDATE 2021.09.20:
A new BETA version is available! - v0.8.0-BETA.2

Guys, it was a big challenge, but (I hope,) I finally found a way to use search engines through Tor network to avoid too many request errors. I tested it a lot, but I'm not sure it's really stable in every country in every situation. So I released it in a BETA version. Any error report is very welcome. Thanks

You can download it from Github: https://github.com/adultmm/AdultMediaManager/releases/tag/v0.8.0-BETA.2

v0.8.0-BETA.2
    ADD: AnalVids.com now supports studio information
    ADD: WebClient tries to terminate or reuse existing Tor process after unhandled software exit
    FIX: Brazzers.com - the case when search engines don't see the valid title of the video
    FIX: Brazzers.com search "length cannot be less than zero" error

...and code cleaning, unit tests, and a lot of changes under the hood.

v0.8.0-BETA.1
  • ADD: Possibility to use Tor network for searching to avoid too many request errors.
  • ADD: OCR-support to read information from images.
  • ADD: AnalVids.com (earlier LegalPorno.com) Scraper
  • FIX: 429 or 418 errors occurred when too many requests were sent to search engines
  • FIX: German users can't access some Google results because of German youth protection laws
  • FIX: YahooSearcher - result page has changed
  • FIX: NubileFilmsNetwork sites don't find some videos
  • FIX: Project1Sercvice based Scrapers (Mofos, Babes, PropertySex, Twistys, and a lot more) don't find new videos
  • FIX: MaxHardcore scraper timezone handling



UPDATE 2021.09.02:

A new version is available! - v0.7.8
  • ADD: MaxHardcore scraper
  • ADD: Nubiles.net support
  • MOD: NubilesFilmsNetwork scraper now handles the case when the search result contains the actress' name
  • FIX: NaughtAmerica search sometimes didn't find the video
  • FIX: NubileFilms Network search sometimes didn't find the video
  • FIX: DDG Searcher sometimes didn't find results when the URL contained - (minus) sign
  • FIX: Wankz video url changed
  • FIX: RoccoSiffredi.com video url changed
  • FIX: Brazzers Scraper errors retrieving specific video details (https://github.com/adultmm/AdultMediaManager/issues/13)

You can download it from Github: https://github.com/adultmm/AdultMediaManager/releases 
Enjoy!


UPDATE 2021.07.19:

A new version is available! - v0.7.7
  • FIX: GoogleSearcher 409 error when you used it too much. Now we use multiple search engines for Scrapers: Brazzers, FuckYouCashSites, NaughtyAmerica, NewSensations, NubileFilmsNetwork, RealityKings, TeamSkeetNetworkNoSearch, XEmpire.
  • FIX: video scoring problem with requested year
  • FIX: runtime scraping of Movies (it was buggy)
  • FIX: LetsDoeIt scraper search failed
  • FIX: FuckYouCashSites supports posters on pages without player
  • FIX: now GammaEntertainment Scraper supports videos with empty title (eg: https://www.evilangel.com/en/video/-/122027 )
  • MOD: NubileFilmsNetwork - cut the title after episode info
  • ADD: RateLimiter improvements
  • ADD: UnderTheBed.com subsite to PureTaboo.com
  • ADD: GiveMeTeens.com subsite to Devil's Film scraper
You can download it from Github: https://github.com/adultmm/AdultMediaManager/releases 
Enjoy!


UPDATE 2021.06.28:

A new version is available! - v0.7.6 You can download it from Github: https://github.com/adultmm/AdultMediaManager/releases 
Enjoy! Smile


UPDATE 2021.06.06:

A new version is available! - v0.7.5
  • FIX: VixenGroup has changed the URL of video pages (Blacked, Tushy, Vixen, Deeper, BlackedRaw, TushyRaw)
You can download it from Github: https://github.com/adultmm/AdultMediaManager/releases 
Enjoy! Smile


UPDATE 2021.06.04:

A new version is available! - v0.7.4
  • FIX: VixenGroup search had been changed (Blacked, Tushy, Vixen, Deeper, BlackedRaw, TushyRaw)
  • FIX: XEmpire scraper releasedate format (HardX, AllBlackX, EroticaX, DarkX, LesbianX)
  • FIX: NewSensations scraper releasedate format
  • FIX: DDFNetwork scraper releasedate format

UPDATE 2021.05.31:

A new version is available! - v0.7.3
  • ADD: XEmpire Network's sites (HardX, AllBlackX, EroticaX, DarkX, LesbianX)
  • MOD: use Google as the first search engine for search on Brazzers (so we can support year for results)
  • FIX: Brazzers.com changed the DOM, so we couldn't find tags
  • FIX: NewSensations Scraper updated to the new site structure (Title and Plot)

UPDATE 2021.05.23:

A new version is available! - v0.7.2
  • FIX: Brazzers didn't find some videos, so we now use multiple search engines
  • MOD: NewSensations now uses multiple search engines
  • MOD: TeamSkeetNetwork now uses multiple search engines


UPDATE 2021.05.20:

A new version is available! - v0.7.1
  • FIX: DorcelVision scraper language selector
  • FIX: NewSensations scraper didn't find some video
  • FIX: LetsDoeIt changed the structure of the site
  • FIX: HotBabesPlus.com has become part of VIPSexVault.com
  • MOD: BackroomCastingCouch scraper now supports runtime

UPDATE 2021.05.16:

A new version is available! - v0.7
I've done a huge refactoring on Scrapers to handle some special cases on different sites, improved error handling, made testing easier, and also possible parallel processing in further situations. I also implemented a new ScraperSelector dialog to make it easier to select the appropriate scrapers from the huge list. And the most requested DDF Network Scraper is now available (with some others). The whole change list can be found here: https://github.com/adultmm/AdultMediaManager/releases 



UPDATE 2021.04.15:

A new version is available!
The new v0.6 version supports scraping Fanarts, multiple Posters, and Screenshots from sites. You can control this behavior under Images / AdultScraper settings. I made some processing parallel, so scraping can be faster, improved auto-matching, and much more. Also added support for more than 130 sites/subsites.

From this version I distribute the application on GitHub: https://github.com/adultmm/AdultMediaManager/releases 



v0.5.12
  • MOD: GammaEntertainment scrapers now report Sci-Fi and Award-Winning categories as genres
  • FIX: workaround for EmberMM loses new Studio, Genre, Country for the first time. http://embermediamanager.thebuggenie.com/embermediamanager/issues/EMM-202
  • FIX: app crashes when scrape found unnamed category (eg. PureTaboo)
  • FIX: duplicated search results in AdultDVDEmpire scraper
  • FIX: the actor name was the title of the movie in some cases in AdultDVDEmpire scraper
Bundled version v0.5.12 x64  - Just download, extract and use. 
EMM Plugin version v0.5.12 x64 - Extract it under EmberMediaManager/Module folder, then enable AdultScraper under Edit/Settings/Movies. (Compatible with x64 versions only.)


v0.5.11
  • ADD: Title matching now handles roman numbers (eg. Any Movie VI = Any Movie 6)
  • ADD: TeenMegaWorld.com Network support (36 sites: 18FirstSex.com, TrickyMasseur.com, AnalBeauty.com, and more)
  • ADD: Dorcel Vision Movie scraper (multilanguage)
  • ADD: new Devil's Film site: OutOfTheFamily.com
  • ADD: NewSensations.com Video (scenes) scraper
  • MOD: speed up AdultDVDEmpire scraper a bit
  • FIX: scraping error in some cases when no Actors found
  • FIX: AdultScraperSettings / MovieFieldScrapers design fix
  • FIX: scraping by TMDB ID didn't work in some cases

v0.5.10
  • ADD: settings to add the source (site) to the Title of the video
  • ADD: support Girlsway.com and subsites (16 sites: GirlsTryAnal.com, LezCuties.com, WhereTheBoysArent.com, and more)
  • ADD: support EvilAngel.com and subsites (21 sites: Buttman.com, AnalAcrobats.com, MilkEnema.com, and more)
  • ADD: PureTaboo.com site support
  • ADD: Wicked.com site support
  • ADD: GirlfriendsFilms.com site support
  • ADD: BurningAngel.com site support
  • ADD: 21Naturals.com site, and 2 subsites support (21FootArt.com, 21EroticAnal.com)
  • ADD: GenderX.com site support
  • ADD: DevilsFilm.com site support
  • ADD: RoccoSiffredi.com site support
  • ADD: 21Sextreme.com site and 6 subsites support (GrandpasFuckTeens.com, DominatedGirls.com, TeachMeFisting.com, and more)
  • MOD: VideoScraper now removes invalid (eg. empty) genres
  • MOD: skip media info tag option now removes 'HD Porn' as well
  • FIX: replace <br> in the plot to new line
  • FIX: remove whitespace from both ends of the plot

v0.5.9
  • ADD: Vixen Group scraper supports two more sites: TushyRaw, BlackedRaw
  • ADD: BackroomCastingCouch.com support
  • ADD: Vixen Group scraper now supports Genres
  • ADD 21Sextury.com sites (22 sites: AnalTeenAngels.com, LezCuties.com, AssHoleFever.com, DPFanatics.com, and more)
  • FIX: "Object reference not set to an instance of an object". error, when the Video has no genre and the "Convert genre case" is not "No change".
  • FIX: Brazzers rating cannot be read (DOM changed)

v0.5.8
  • ADD: CzechAV Network sites - (CzechAmateurs.com, CzechStreets.com, CzechMegaSwingers.com and more, 34 sites)
  • ADD: Vixen Media sites: Blacked, Deeper, Tushy, Vixen

v0.5.7
  • FIX: "empty scraper list" error (reported by Strubdog - thanks)
  • FIX: Brazzers.com video URL matcher changed
  • FIX: RealityKing scraper didn't find the video in some cases

v0.5.6
  • fix AdultDVDEmpire search scoring
  • fixed RealityKings changed video URL format



RE: AdultScraper for Ember released - strubdog - 2021-02-18

2 Questions:

1. Do you plan on implementing this as a scraper in Kodi?
2. How do you scrape by specific supported sites. When I run the scraper, it only gives me the options of "The MoviesDB" and "AdultDVDEmpire". How do I use the other supported sites?

Thanks


RE: AdultScraper for Ember released - adultmm - 2021-02-18

(2021-02-18, 01:16)strubdog Wrote: 2 Questions:

1. Do you plan on implementing this as a scraper in Kodi?
2. How do you scrape by specific supported sites. When I run the scraper, it only gives me the options of "The MoviesDB" and "AdultDVDEmpire". How do I use the other supported sites?

Thanks

Hi Strubdog,

1, EmberMediaManager can save movie data into XML files automatically and Kodi (and a lot of other streaming or home theater software) easily reads this information. Sometimes scrapers cannot read the appropriate information so you have to edit it by hand, and it's much easier in Ember.

2, AdultScraper tries to detect the file is a movie or a video from supported sites. If it's failed, you can switch between the modes on the bottom of the Search Results window using the "Switch to Video" or "Switch to Movie" button

I hope this will help you, but if you have more questions, please feel free to ask.


RE: AdultScraper for Ember released - strubdog - 2021-02-18

(2021-02-18, 21:01)adultmm Wrote:
(2021-02-18, 01:16)strubdog Wrote: 2 Questions:

1. Do you plan on implementing this as a scraper in Kodi?
2. How do you scrape by specific supported sites. When I run the scraper, it only gives me the options of "The MoviesDB" and "AdultDVDEmpire". How do I use the other supported sites?

Thanks

Hi Strubdog,

1, EmberMediaManager can save movie data into XML files automatically and Kodi (and a lot of other streaming or home theater software) easily reads this information. Sometimes scrapers cannot read the appropriate information so you have to edit it by hand, and it's much easier in Ember.

2, AdultScraper tries to detect the file is a movie or a video from supported sites. If it's failed, you can switch between the modes on the bottom of the Search Results window using the "Switch to Video" or "Switch to Movie" button

I hope this will help you, but if you have more questions, please feel free to ask.
When I do "switch to video". I get an error message saying "empty scraper list" and then it quits. 

Thanks


RE: AdultScraper for Ember released - adultmm - 2021-02-19

(2021-02-18, 22:13)strubdog Wrote:
(2021-02-18, 21:01)adultmm Wrote:
(2021-02-18, 01:16)strubdog Wrote: 2 Questions:

1. Do you plan on implementing this as a scraper in Kodi?
2. How do you scrape by specific supported sites. When I run the scraper, it only gives me the options of "The MoviesDB" and "AdultDVDEmpire". How do I use the other supported sites?

Thanks

Hi Strubdog,

1, EmberMediaManager can save movie data into XML files automatically and Kodi (and a lot of other streaming or home theater software) easily reads this information. Sometimes scrapers cannot read the appropriate information so you have to edit it by hand, and it's much easier in Ember.

2, AdultScraper tries to detect the file is a movie or a video from supported sites. If it's failed, you can switch between the modes on the bottom of the Search Results window using the "Switch to Video" or "Switch to Movie" button

I hope this will help you, but if you have more questions, please feel free to ask.
When I do "switch to video". I get an error message saying "empty scraper list" and then it quits. 

Thanks

Thank you for this error report, I'll try to reproduce and fix it.


RE: AdultScraper for Ember released - adultmm - 2021-02-19

(2021-02-19, 02:17)adultmm Wrote:
(2021-02-18, 22:13)strubdog Wrote:
(2021-02-18, 21:01)adultmm Wrote: Hi Strubdog,

1, EmberMediaManager can save movie data into XML files automatically and Kodi (and a lot of other streaming or home theater software) easily reads this information. Sometimes scrapers cannot read the appropriate information so you have to edit it by hand, and it's much easier in Ember.

2, AdultScraper tries to detect the file is a movie or a video from supported sites. If it's failed, you can switch between the modes on the bottom of the Search Results window using the "Switch to Video" or "Switch to Movie" button

I hope this will help you, but if you have more questions, please feel free to ask.
When I do "switch to video". I get an error message saying "empty scraper list" and then it quits. 

Thanks

Thank you for this error report, I'll try to reproduce and fix it.

@strubdog  I hope I've fixed the error. Please download the new version, and try again. (Do not overwrite your profile folder!) Let me know if you have any other problems. Thanks


RE: AdultScraper for Ember released - strubdog - 2021-02-20

Do you have the link for the updated program?


RE: AdultScraper for Ember released - strubdog - 2021-02-20

Link appears to be dead to download new version.


RE: AdultScraper for Ember released - adultmm - 2021-02-20

(2021-02-20, 00:44)strubdog Wrote: Link appears to be dead to download new version.

Sorry for the inconvenience, I've uploaded it again.


RE: AdultScraper for Ember released - strubdog - 2021-02-20

(2021-02-20, 16:55)adultmm Wrote:
(2021-02-20, 00:44)strubdog Wrote: Link appears to be dead to download new version.

Sorry for the inconvenience, I've downloaded it again.
Excellent, working without any issues.

Should I just use this thread if I run into any issues or I notice you are missing any particular sites?

Or do you have seperate github page for this program?

Thanks


RE: AdultScraper for Ember released - strubdog - 2021-02-20

Not sure if this possible, are you able to add ManyVids as a scraper source.

Also, please add the sites for the Vixen Media Group (Blacked, Tushy, Vixen)

Thanks


RE: AdultScraper for Ember released - adultmm - 2021-02-20

(2021-02-20, 19:05)strubdog Wrote: Not sure if this possible, are you able to add ManyVids as a scraper source.

Also, please add the sites for the Vixen Media Group (Blacked, Tushy, Vixen)

Thanks

I'm glad it works for you too. 
This thread is perfect to report errors or ask for new scraper sources. 

Blacked.com is also requested by others, so it's on my plan. I'll check the other sites you mentioned.


RE: AdultScraper for Ember released - strubdog - 2021-02-21

One additional suggestion, create a document or section that explains the best folder structure and naming convention so that you can use semi-automatically without re-scraping for an individual video.

Thanks


RE: AdultScraper for Ember released - adultmm - 2021-02-21

(2021-02-20, 20:19)adultmm Wrote:
(2021-02-20, 19:05)strubdog Wrote: Not sure if this possible, are you able to add ManyVids as a scraper source.

Also, please add the sites for the Vixen Media Group (Blacked, Tushy, Vixen)

Thanks

I'm glad it works for you too. 
This thread is perfect to report errors or ask for new scraper sources. 

Blacked.com is also requested by others, so it's on my plan. I'll check the other sites you mentioned.
@strubdog I've added Vixen Group sites. I haven't had too much time to test, so please, tell me if you found any errors.


RE: AdultScraper for Ember released - strubdog - 2021-02-21

One thing I am noticing. The program is having a hard time picking up genre/tag/categories from the sites. I have seen this in particular with Brazzers.

Thanks


This forum uses Lukasz Tkacz MyBB addons.