Repository Hammering
#1
I recently discovered that there are a number of IP addresses that are hammering my repository. They keep downloading the same updates over and over again and download the addons.xml file far to often. It actually turned out that some IP addresses download an update (10MB) more than 200x on a single day. That is 2 GB of bandwidth and is really causing me to wonder what is wrong.

Now, most (95%) of the IP's actually download them once or twice (if they have multiple XBMC's). So I am wondering what could lead to this behaviour. I double checked, but I can't really related it to a specific XBMC or OS version.
Reply
#2
did you "whois" those IPs? Maybe its one of those meta repositories...
Reply
#3
(2013-09-01, 15:24)wsnipex Wrote: did you "whois" those IPs? Maybe its one of those meta repositories...

Yes, they all seem to be normal ISP clients. I blocked them for now and they will get an 403 error.
Reply
#4
Hi, I would like to continue this topic. Last tuesday I updated my repository from version 1.0.2 -> 1.0.3 -> 1.0.4 (the 1.0.4 was due to a missing icon) and I updated my addon from 3.3.7 -> 3.4.0.

What I see today is (these are only todays counts):
10000 downloads of the 1.0.4 repository
900 downloads of the old 1.0.2 repository by only 8 clients
700 downloads of the old 3.3.7 add-on by the same 8 clients (that is about 7 GB of data)
In total those 8 clients are responsible for a total of 2100 downloads in 24 hours.

I did not check this yesterday, but I guess it will be the same. So I thinks something is not OK at all for those 8 IP addresses. I would really like to know what is wrong with these addresses, but how can this be figured out?
Reply
#5
The only thing I can think of is they have some sort of corrupt add-on database. Thus, the update time of the repository in the database is never updated (can't write to the database) so it continually hits.

I'm not sure what could be done about that to be honest - the database open's fine, but writes to it are probably broken.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#6
So can I advise those people to delete their addons db?

Besides that: would it not be possible to detect this problem when XBMC cannot write to a DB? In case of the addon db it could just be re-created?
Reply
#7
If you know who they are, then sure, see if that's the issue.

I'm not sure we could detect this reliably. When an error occurs, it's not always clear whether that's a coding error on our part or some sort of corruption of the database (which may also be some coding error - I have never managed to achieve this so I don't know!). Worst case all we could do is flag the database as having something wrong with it.

The first step is determining where it's failing - there should be some errors thrown and thus errors in the log. If we can confirm that the database is corrupt then we're in a better position to be able to fix it. Ideally we'd have an example of such a corrupt database to play with.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#8
I would like to re-address this issue, as even with recent versions of Kodi the problem persists.

I have users downloading add-on updates over 500x per day. With a 0.5 MB per download, that makes 250 MB per user per day!

So this is just my simple add-on with 50.000 active users. I would assume that if this is the case on your repository servers you would be wasting gigabytes of bandwidth reach day.

Created some analysis tools (SQL stuff) for my apache logs to analyse the millions of rows of requests. Would you guys be willing to have me have a look at your logs (or some of them) to see how often this occurs for you?

Let me know.
Reply

Logout Mark Read Team Forum Stats Members Help
Repository Hammering0