So I saw some methods about how to scrape "infinite" scrolling websites, but they don't apply to the site I'm trying to scrape.
Maybe someone with more experience with scrappers can take a look and help me.
PS: It's an adult site.
json:
https://www.myfreecams.com/#Homepage
I'm not sure on how to scrape a website like that in "pages"
Once again thanks for any help.
Self-quote from another topic:
(2018-11-05, 14:37)Roman_V_M Wrote: [ -> ]with modern JavaScript-based websites the situation gets more complicated. You need to use either a headless browser engine, which is not a trivial task for a Kodi addon, or to reverse-engineer the front-end JavaScript application to find REST endpoints it gets its data from and use those endpoints in your addon to fetch data.
(2018-11-15, 22:40)Roman_V_M Wrote: [ -> ]Self-quote from another topic:
(2018-11-05, 14:37)Roman_V_M Wrote: [ -> ]with modern JavaScript-based websites the situation gets more complicated. You need to use either a headless browser engine, which is not a trivial task for a Kodi addon, or to reverse-engineer the front-end JavaScript application to find REST endpoints it gets its data from and use those endpoints in your addon to fetch data.
Thanks, will take a look into things like that I see what I can find about it
