2016-06-20, 12:14
with https://github.com/xbmc/xbmc/pull/9330 the usual way of setting http headers was removed and this will break all addons that rely on http streaming with header checks (referer=, cookies=, etc).
(2016-09-01, 18:25)ironic_monkey Wrote: Nope. Some devs think it is progress to remove things peeps need, damn the consequences. I will try to find some time to fix this but i have little time these days
(2016-09-01, 18:35)ironic_monkey Wrote: also it is worth noticing that this does not affect vanilla http(s) which is unchanged, it is only for protocols that runs on top like hls and mmsh.
- - [01/Sep/2016:21:21:46 +0300] "HEAD *****.mp4 HTTP/1.1" 200 - "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "HEAD *****.mp4 HTTP/1.1" 200 - "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp4 HTTP/1.1" 206 497689 "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp44 HTTP/1.1" 206 65536 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp4 HTTP/1.1" 206 497689 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
m_userAgent = g_sysinfo.GetUserAgent();
(2016-09-01, 21:01)FatalityX Wrote: So glad somebody noticed these changes that started with Krypton. I don't think my actual problem is happening exactly because of them however it is related.User-agent passes on just fine, as is the Referrer and Cookie header. All others fail. Just append "|http-headers="....." to the url. It's all the other headers that no longer work.
I'm using an addon that parses a website which provides streaming services for a bunch of different TV channels. One of its biggest advantages for me is that has a few days' worth of EPG archive which means I can watch, let's say, yesterday's news broadcast the following day. It's pretty neat and is free of charge. Videos on the site are encoded in x264 and streaming is done via HTTP and HLS. The only check they perform is for User Agent in the request headers. Even for the actual .ts chunks from m3u8.
Up until Krypton everything worked fine with user agent being appended to the URL in python script. Simply putting something in line of "|User-Agent=Mozilla/5.0..." made it work just fine. However v17 had some changes brought in (all alphas and now beta) that completely broke the functionality of it all and as a result it's no longer working.
I can reproduce this easily with a simple change in the addon so that it request files from my own webserver. See the headers:
Code:- - [01/Sep/2016:21:21:46 +0300] "HEAD *****.mp4 HTTP/1.1" 200 - "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "HEAD *****.mp4 HTTP/1.1" 200 - "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp4 HTTP/1.1" 206 497689 "-" "\"TEST\""
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp44 HTTP/1.1" 206 65536 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
- - [01/Sep/2016:21:21:46 +0300] "GET *****.mp4 HTTP/1.1" 206 497689 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
It's obvious User-Agent headers are no longer being passed to actually play the video. Only to request first 64KB to get its mime type etc.
What's even more frustrating is that there is no way to override default User-Agent even with advancedsettings.xml. It's hardcoded in xbmc/settings/AdvancedSettings.cpp:
Code:m_userAgent = g_sysinfo.GetUserAgent();
Any suggestions without recompiling my own version (I would need both for Windows and RaspberryPi) or using a proxy with header rewriting?
(2016-09-01, 21:53)Basje Wrote: User-agent passes on just fine, as is the Referrer and Cookie header. All others fail. Just append "|http-headers="....." to the url. It's all the other headers that no longer work.
uri = 'http://my.own.server.tld/path/filename.mp4|user-agent=TEST'
- - [02/Sep/2016:06:11:14 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "GET ****.mp4 HTTP/1.1" 206 65536 "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 65536 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
(2016-09-02, 05:26)FatalityX Wrote:Indeed, something is very inconsistent here and is really causing issues with a lot of add-ons.(2016-09-01, 21:53)Basje Wrote: User-agent passes on just fine, as is the Referrer and Cookie header. All others fail. Just append "|http-headers="....." to the url. It's all the other headers that no longer work.
I see. However I'm doing just that, well the addon does via appending "user-agent" to the URL just like in the old versions.
PHP Code:uri = 'http://my.own.server.tld/path/filename.mp4|user-agent=TEST'
But... Even direct request for a simple mp4 file over HTTP isn't rewritten with a new header. Consider these log outputs from a webserver with an older Kodi:
Code:- - [02/Sep/2016:06:11:14 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "GET ****.mp4 HTTP/1.1" 206 65536 "-" "TEST"
- - [02/Sep/2016:06:11:14 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "TEST"
and Krypton:
Code:- - [02/Sep/2016:06:16:48 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "HEAD ****.mp4 HTTP/1.1" 200 - "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "TEST"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 65536 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
- - [02/Sep/2016:06:16:48 +0300] "GET ****.mp4 HTTP/1.1" 206 497689 "-" "Kodi/17.0-BETA2 (Windows NT 6.1; WOW64) App_Bitness/32 Version/17.0-BETA2-Git:20160831-f39d849"
Krypton does one extra request it seems (4 vs 5), I can live with that. However, user-agent is back to original one in the headers when it starts actually playing the video.
Strange, isn't it?
(2016-09-05, 11:43)ironic_monkey Wrote: it's not ffmpeg that does the GET, that is CURL (it's pre-parsed to choose the correct stream according to configured bandwidth, as there is no adaptive streams yet). then the show is passed on to ffmpeg and it will not set those headers on the contents of the m3u8 (i.e. the actual streams).