Device: Windows, Android
Kodi version: 21.0.0
Requests version: 2.31.0
I use requests to get web pages and attempt to bypass Cloudflare verification by setting headers. The code is as follows:
Quote:requests.get("https://ddys.pro",headers={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","Cookie":"X_CACHE_KEY=acb097992b92ab24fc7ddf604c46f77e; cf_clearance=6A2OzzXdNvxxqZd1M2b3w05xGv.62jIg6AfmIX8ew1Q-1713403239-1.0.1.1-AkhJ3yB_wNx2Sh.RRkV3irxjjrdXMmoRbC4LXXF6UrTwZBSpzckxif1i57jun7.xErZ_0waGK5JgcrtgRM1gGw; __cf_bm=tiemtpKjomxGMQfjCZf6T4sdqMvFMabBI5pl1pAnbGU-1713403238-1.0.1.1-eotvanZiYm7QrOY5kkCN162mVwFHi5jMbBGJ6NdBUMihqT8.e1Da1bF6UCo4edHO7Wpo44lGlHzqVl69yUBiDw"})
The same code, when run in Python in PowerShell, returns a status code of 200. However, in the Kodi, the status code always remains 403.
Could someone please help me? Thank you
Hopefully someone will, but you have to give them time. Wait a day or two before bumping the thread.
I'm not seeing obviously wrong with that one line of code. It would be helpful to see the code in context. Can you post a link to the full file?
I use web-pdb for debugging. After the program pauses, I paste and run the line of code mentioned above in the web-pdb console. (The same code runs fine in PowerShell)
This is my actual code. I pass cookies and a user-agent through HTTP server. At that time, requests.get returned a 403 error.
code:
# proxies is set to {} here.
def remotecookies_method(url, proxies: dict, headers: dict):
datas = get_remote_cookies(url, headers)
remote_cookies = datas['Cookie']
remote_useragent = datas['User-Agent']
if 'Cookie' in headers:
remote_cookies = headers['Cookie'].rstrip(' ').rstrip(';') + ';' + remote_cookies
headers['Cookie'] = remote_cookies
headers['User-Agent'] = remote_useragent
res = requests.get(url = url, headers = headers, proxies = proxies)
if not res.ok:
raise ConnectionError
dialog.close()
return res.text
waiting = True
json_data = None
url_ = None
useragent = None
class Server(HTTPServer):
def handle_timeout(self) -> None:
global waiting
waiting = False
return super().handle_timeout()
class ServerHanlder(BaseHTTPRequestHandler):
def _set_response(self):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
def do_GET(self):
self._set_response()
self.wfile.write(json.dumps({"User-Agent": useragent, "url": url_}).encode('utf-8'))
def do_POST(self):
global waiting, json_data
content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length)
json_data = json.loads(post_data.decode('utf-8'))
self._set_response()
self.wfile.write(b"Success")
waiting = False
def run(server_class = Server, handler_class = ServerHanlder, port = 1234):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
httpd.timeout = 300
while (waiting):
httpd.handle_request()
httpd.server_close()
def get_remote_cookies(url, headers):
global url_, useragent
url_split = urlsplit(url)
url_ = url_split.scheme + '://' + url_split.netloc
if "User-Agent" in headers:
useragent = headers["User-Agent"]
else:
useragent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0"
run()
return json_data
Where are you calling remotecookies_method? Specifically, how are the headers being passed in? Kodi has a default user agent I think it uses, so it's possible you are passing that user agent in with the header, and in get_remote_cookies you only set the header for the user agent if one doesn't already exist. It's the only thing I can think of off hand. You could probably stick something in right before the requests call to print the header you have to the Kodi log so you can see it. That might shed some light on things.
I seriously doubt that you can bypass CloudFlare by just manipulating headers. CloudFlare verifies user agent signature by many factors including the ability to run JavaScript. If you opened this URL in PowerShell using built-in commandlets they most probably use Edge browser engine under the hood that has a valid browser signature.
Currently, I have successfully bypassed Cloudflare (status code 200) in Kodi version 20.5 on Android, but versions 20.5 and 21 on Windows were unsuccessful. And I don't know why it's also unsuccessful now in PowerShell. This might not be a Kodi issue.
Thank you all for your help