[kowach]

Hi,

I'm owner of tvprofil.net/xmltv. You can find there epg for 40 channels for free download.
I have noticed lot of unnecessary request from Kodis all around the world in my server logs. Problem is that those request are growing in numbers and they will eventually choke down my server.

Examples:

Code:

x.x.x.x - - - [17/May/2015:09:11:06 +0200] "HEAD /xmltv HTTP/1.1" 301 0 "-" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:07 +0200] "HEAD /xmltv/ HTTP/1.1" 200 0 "http://tvprofil.net/xmltv" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:07 +0200] "GET /xmltv HTTP/1.1" 301 184 "-" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:09 +0200] "GET /xmltv/ HTTP/1.1" 200 117836 "http://tvprofil.net/xmltv" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:09 +0200] "HEAD /xmltv HTTP/1.1" 301 0 "-" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:09 +0200] "HEAD /xmltv/ HTTP/1.1" 200 0 "http://tvprofil.net/xmltv" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:09 +0200] "GET /xmltv HTTP/1.1" 301 184 "-" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"
x.x.x.x - - - [17/May/2015:09:11:11 +0200] "GET /xmltv/ HTTP/1.1" 200 117836 "http://tvprofil.net/xmltv" "XBMC/13.2 Git:20140817-0f3db05 (Linux; Android; 3.0.50 armv7l; http://xbmc.org)" "-"

around 50 request which repeats in intervals... all night around 3000 log lines.



another case (which lands to homepage?!):

Code:

y.y.y.y - - - [17/May/2015:09:08:02 +0200] "HEAD / HTTP/1.1" 200 0 "-" "XBMC/13.2 Git:20140901-867305b (Linux; XBian 1.0 (knockout); 3.15.8+ armv6l; http://xbmc.org)" "-"
y.y.y.y - - - [17/May/2015:09:08:02 +0200] "GET / HTTP/1.1" 200 110105 "-" "XBMC/13.2 Git:20140901-867305b (Linux; XBian 1.0 (knockout); 3.15.8+ armv6l; http://xbmc.org)" "-"
y.y.y.y - - - [17/May/2015:09:08:02 +0200] "HEAD / HTTP/1.1" 200 0 "-" "XBMC/13.2 Git:20140901-867305b (Linux; XBian 1.0 (knockout); 3.15.8+ armv6l; http://xbmc.org)" "-"
y.y.y.y - - - [17/May/2015:09:08:03 +0200] "GET / HTTP/1.1" 200 110012 "-" "XBMC/13.2 Git:20140901-867305b (Linux; XBian 1.0 (knockout); 3.15.8+ armv6l; http://xbmc.org)" "-"

around 50 request which repeats in intervals... all night around 3000 log lines.

list goes on...

Code:

z.z.z.z - - - [10/May/2015:09:41:50 +0200] "HEAD /xmltv/data/channel-list.tvprofil.net.xml HTTP/1.1" 200 0 "-" "Kodi/14.2 (X11; Linux x86_64) Ubuntu/15.04 App_Bitness/64 Version/14.2-Git:7cc53a9" "-"
z.z.z.z - - - [10/May/2015:09:41:50 +0200] "GET /xmltv/data/channel-list.tvprofil.net.xml HTTP/1.1" 206 6977 "-" "Kodi/14.2 (X11; Linux x86_64) Ubuntu/15.04 App_Bitness/64 Version/14.2-Git:7cc53a9" "-"
z.z.z.z - - - [10/May/2015:09:41:50 +0200] "HEAD /xmltv/data/channel-list.tvprofil.net.xml HTTP/1.1" 200 0 "-" "Kodi/14.2 (X11; Linux x86_64) Ubuntu/15.04 App_Bitness/64 Version/14.2-Git:7cc53a9" "-"

around 100 same lines...

Code:

q.q.q.q - - - [10/May/2015:11:53:36 +0200] "HEAD /epg/ HTTP/1.1" 200 0 "-" "Kodi/14.1 (Linux; Android 4.2.2; AFTB Build/JDQ39) Android/4.2.2 Sys_CPU/armv7l App_Bitness/32 Version/14.1-Git:2015-01-30-38e4046-dirty" "-"
q.q.q.q - - - [10/May/2015:11:53:36 +0200] "GET /epg/ HTTP/1.1" 200 19382 "-" "Kodi/14.1 (Linux; Android 4.2.2; AFTB Build/JDQ39) Android/4.2.2 Sys_CPU/armv7l App_Bitness/32 Version/14.1-Git:2015-01-30-38e4046-dirty" "-"
q.q.q.q - - - [10/May/2015:11:53:36 +0200] "HEAD /epg/ HTTP/1.1" 200 0 "-" "Kodi/14.1 (Linux; Android 4.2.2; AFTB Build/JDQ39) Android/4.2.2 Sys_CPU/armv7l App_Bitness/32 Version/14.1-Git:2015-01-30-38e4046-dirty" "-"
q.q.q.q - - - [10/May/2015:11:53:36 +0200] "GET /epg/ HTTP/1.1" 200 18982 "-" "Kodi/14.1 (Linux; Android 4.2.2; AFTB Build/JDQ39) Android/4.2.2 Sys_CPU/armv7l App_Bitness/32 Version/14.1-Git:2015-01-30-38e4046-dirty" "-"

Can you help me identify the problem? I think this is some missconfiguration in Kodi (plugin).

I would gladly make instructions on xmltv page or event support some epg plugin just to do it right.

[wsnipex]

you are not the first epg site owner that has this issue. In the past, it boiled down to preconfigured "fully loaded" boxes with tons of addons that are banned here. Its usually a wrongly configured iptvsimple addon. We have taken steps to reduce this on the new versions of the addon itself, but if users/box vendors configure it wrong, there's not so much we can do.

[kowach]

Thx. I have found simple way to detect it.
User agent: Kodi or xbmc
Request method: HEAD

Action: log it.
Fail2ban: block it on firewall for 24h

I don't see other way? I try to put some dummy XMLTV data, but id didn't worked.

[wsnipex]

as long as its only a HEAD request, it shouldn't really be a big problem. Downloading the complete file all the time would be bad.

[kowach]

that is DDOS.

daily average 36.000 Kodi/XBMC requests (18.000 HEAD request, 9.500 redirects, 8.000 pointless GET request and 500 real request to XMLTV documents).

that is 98,61% wrong request from Kodi/XBMC

still it's nothing to total server traffic :-)

[wsnipex]

heh, our mirror redirector has ~500-800 requests/sec

anyway, its your server, so just block them.

[kowach]

I have analyzed the source of the problem and made a suggestion. I hope developer will be happy to assist
https://github.com/kodi-pvr/pvr.iptvsimple/issues/17

[fritsch]

Thanks much for helping to fix our stuff!