2021-08-10, 17:14
Hi Guys,
Sorry for the late answer, but I was on my holiday.
As I can see, DDG searcher sometimes detects too many requests despite the rate limiter. My next idea is to use Tor proxy so we can switch to a new IP address in these situations. It's not too easy to implement, I have to integrate an external software, but I'm on it.
My plans for the next period:
- stabilize the search capability
- fix some smaller bugs in the existing scrapers
- one of my Users sent me money for a special Scraper (thanks for it!), so I'll implement it as we've discussed (it'll be available for everybody)
- finish some semi-finished scrapers
- check the possibility to integrate https://metadataapi.net
- check the work needed to upgrade EMM 1.11.x
Keep in touch here or on GitHub.
Sorry for the late answer, but I was on my holiday.
As I can see, DDG searcher sometimes detects too many requests despite the rate limiter. My next idea is to use Tor proxy so we can switch to a new IP address in these situations. It's not too easy to implement, I have to integrate an external software, but I'm on it.
My plans for the next period:
- stabilize the search capability
- fix some smaller bugs in the existing scrapers
- one of my Users sent me money for a special Scraper (thanks for it!), so I'll implement it as we've discussed (it'll be available for everybody)
- finish some semi-finished scrapers
- check the possibility to integrate https://metadataapi.net
- check the work needed to upgrade EMM 1.11.x
Keep in touch here or on GitHub.
