USA clients sent to EU mirrors?
#1
I'm wondering why USA traffic is sent to EU mirrors? 

I know there was drama in the past about the plugins but the leia exe is being redirected overseas? 

Image
Reply
#2
That can happen when the locations near you have limited bandwidth compared to the monster mirrors in the EU.
I tried with your IP and it gives OS6 mirror about 6% of the time which is right considering the amount of bandwidth each has available.

We could use a few really high-speed mirrors in the US !
Hobbies: playing Torn-City - Massively multiplayer online text based RPG
Reply
#3
(2019-08-19, 00:41)Kib Wrote: That can happen when the locations near you have limited bandwidth compared to the monster mirrors in the EU.
I tried with your IP and it gives OS6 mirror about 6% of the time which is right considering the amount of bandwidth each has available.

We could use a few really high-speed mirrors in the US !
http://mirrors.kodi.tv/releases/windows/...mirrorlist 

there's plenty of 10gig plus servers in the US. The weights are set up funky. it's silly to go halfway around the world for a download when there's 10 other North American servers. 

Oregon State University 50+gbit 
OS6  is a 1 gigabit server that receives very little traffic from kodi.
Reply
#4
The weights are set up based on the bandwidth they have available for us.

It is possible something is set up wrong, but please do not make up data. You state OSU has 50+ gbit. Where do you get this info ?
Hobbies: playing Torn-City - Massively multiplayer online text based RPG
Reply
#5
Uni's
Princeton: 10 Gbit
OSU: 1 Gbit
UMD: 1 Gbit

Companies
Limestone: 1 Gbit
Syringa: 1 Gbit
Xmission: 1 Gbit
Leaseweb (Dallas) : 10 Gbit
Leaseweb (San Fransisco): 10 Gbit

^^ That is pretty much all we have dedicated to the US where most of our users come from

The three 10 Gbit lines are not plenty.
Hobbies: playing Torn-City - Massively multiplayer online text based RPG
Reply
#6
(2019-08-19, 17:46)Kib Wrote: Uni's
Princeton: 10 Gbit
OSU: 1 Gbit
UMD: 1 Gbit

Companies
Limestone: 1 Gbit
Syringa: 1 Gbit
Xmission: 1 Gbit
Leaseweb (Dallas) : 10 Gbit
Leaseweb (San Fransisco): 10 Gbit

^^ That is pretty much all we have dedicated to the US where most of our users come from

The three 10 Gbit lines are not plenty.


I think the USA mirrors are just under utilized.  I manage os6.org and the only time I see spikes in traffic is when other mirrors are removed due to not having the latest files when mirrorbits checks. When mirrorbrain was
 still being used I used to see about 20-30MB of sustained traffic ( ~2,000 concurrent clients ) now i see less than 5MB ( no more than 100 concurrent clients ).  See the below screenshots from mirror.os6.org nginx traffic and network traffic.

I realize other mirrors might not want the full amount of USA Kodi traffic hitting their mirrors, but feel free to send 800mbit of traffic to os6. The server as a whole is underutilized.  

http://mirrors.kodi.tv/releases/windows/...irrorstats           Most of the USA mirrors are always on the bottom of the mirrorstats list.


http://osuosl.org/services/hosting/details/

FTP Mirroring
We have a cluster of three servers behind the http://ftp.osuosl.org name with a total bandwidth capacity of more than 50 gigabits per second. These servers are hosted geographically across the United States. Instead of pushing files and releases out from your own server, let us take care of the dirty work for you. We currently host approximately 100 projects on our mirroring servers using around 11TB of disk space. We do our best to host as many projects as we can, however space is limited.




Image

Image
Reply
#7
(2019-08-19, 17:34)Kib Wrote: The weights are set up based on the bandwidth they have available for us.

It is possible something is set up wrong, but please do not make up data. You state OSU has 50+ gbit. Where do you get this info ?

Not sure what data I'm making up? os6.org is the busiest server in USA per mirrorbits and i don't even see 100 mbit from Kodi yet we somehow we don't have enough USA mirrors? so far today for the busiest USA server, mirrorbits has clocked a total of 191GB downloaded in 20 hours. 

Per the ubuntu mirrors, someone has to sign up and populate the data. https://launchpad.net/ubuntu/+archivemirrors

OSU Open Source Lab not Oregon State University   https://ftp.osuosl.org/pub/xbmc/   50+ GBbit according to website or 2gbit according to ubuntu mirrors
Oregon State University has their own mirror at  http://mirror.uoregon.edu but its not mirroring Kodi and its 1gbit
Princeton University   20 Gbps
Syringa Networks       10 Gbps 
Limestone Networks   2 Gbps
Looks like leaseweb has dropped kodi from all its mirrors.
Reply
#8
You are taking all these speeds from their sites, not the speeds they asked us to use. OSUOSL is 1 Gbit for us (not 50). 
I was taking my data from the mirrorbits config but I did ineed not look at the activity of servers. You are right about Leaseweb and there are therefore even less active servers in the US.

There are not many high bandwidth servers in the US compared to the enormous amount of high bandwidth servers in Europe.
EUROPE: We have a combined 158 Gbit of mirror bandwidth. There are 31 active servers - 2 servers that have 20 Gbit for us, 6 servers that have 10 Gbit for us, and the rest is 5, 4, 3, 2 or 1 Gbit. 
NORTH AMERICA: We have a combined 10.1 Gbit of mirror bandwidth. There are 9 active servers in North America. One is set to max on 2 Gbit (yours), 7 are on 1 Gbit, and one is 100 Mbit.

Mirrorbits makes a weighted list of all the mirrors which have the requested file and are allowing the connection, using the weights of all mirrors. It than randomizes base on that weight.
We set the weights based on the available bandwidth of the mirror.

I'm not sure how we can change this without breaking evrrything since we have our bandwidth mostly in the EU, and our users mostly in the US.
On top of that not all US mirrors are synchronising against all the addons I believe, so they even have less files available.

If I set up restrictions based on continent, your server is immediately flooded and people in US can not download kodi anymore. Meanwhile the Euro servers are underutilized.
If I pretend you have more bandwidth, a spike will mean disconnected users and people can not download anymore.

If you have a brilliant idea on how to improve the weighting system of mirrorbits I am all ears, but for now I think it is just a consequence of the difference in users and servers between the two continents.
Hobbies: playing Torn-City - Massively multiplayer online text based RPG
Reply
#9
(2019-08-21, 22:09)Kib Wrote: You are taking all these speeds from their sites, not the speeds they asked us to use. OSUOSL is 1 Gbit for us (not 50). 
I was taking my data from the mirrorbits config but I did ineed not look at the activity of servers. You are right about Leaseweb and there are therefore even less active servers in the US.

There are not many high bandwidth servers in the US compared to the enormous amount of high bandwidth servers in Europe.
EUROPE: We have a combined 158 Gbit of mirror bandwidth. There are 31 active servers - 2 servers that have 20 Gbit for us, 6 servers that have 10 Gbit for us, and the rest is 5, 4, 3, 2 or 1 Gbit. 
NORTH AMERICA: We have a combined 10.1 Gbit of mirror bandwidth. There are 9 active servers in North America. One is set to max on 2 Gbit (yours), 7 are on 1 Gbit, and one is 100 Mbit.

Mirrorbits makes a weighted list of all the mirrors which have the requested file and are allowing the connection, using the weights of all mirrors. It than randomizes base on that weight.
We set the weights based on the available bandwidth of the mirror.

I'm not sure how we can change this without breaking evrrything since we have our bandwidth mostly in the EU, and our users mostly in the US.
On top of that not all US mirrors are synchronising against all the addons I believe, so they even have less files available.

If I set up restrictions based on continent, your server is immediately flooded and people in US can not download kodi anymore. Meanwhile the Euro servers are underutilized.
If I pretend you have more bandwidth, a spike will mean disconnected users and people can not download anymore.

If you have a brilliant idea on how to improve the weighting system of mirrorbits I am all ears, but for now I think it is just a consequence of the difference in users and servers between the two continents.

If you don't mind I can reach out to other mirrors and try to build a bigger US mirror pool for Kodi.  


Looking at vlc's mirrorbits stats it seems they favor geoIP. I haven't setup Mirrorbits in a while so i can't recommended a setting but the dev that created it seems to be active. There's a setting that might need adjusting but the config doesn't explain the setting. 

https://get.videolan.org/vlc/3.0.8/win32...mirrorlist
https://get.videolan.org/vlc/3.0.8/win32...irrorstats

## Adjust the weight/range of the geographic distribution
# WeightDistributionRange: 1.5
Reply
#10
I can take a look at changing the relative weight of the GeoIP nearness to the bandwidth score and I'll reach out to etix as well and ask him if he has any recommendations. 

Seeing the results I get with your IP at VLC's mirrorstats it looks like his setup only selects 3 mirrors out of the pool they have and they are all US based.
Perhaps they set them up to be continent only or he has made another adjustment.
I will need to figure out what happens when a files is not available in your continent and mirrors are set to continent only though.

Any help in finding more US mirrors would be highly appreciated !
Hobbies: playing Torn-City - Massively multiplayer online text based RPG
Reply
#11
(2019-08-22, 08:48)Kib Wrote: I can take a look at changing the relative weight of the GeoIP nearness to the bandwidth score and I'll reach out to etix as well and ask him if he has any recommendations. 

Seeing the results I get with your IP at VLC's mirrorstats it looks like his setup only selects 3 mirrors out of the pool they have and they are all US based.
Perhaps they set them up to be continent only or he has made another adjustment.
I will need to figure out what happens when a files is not available in your continent and mirrors are set to continent only though.

Any help in finding more US mirrors would be highly appreciated !


Mirrors (wiki) . Are these instructions current?
Reply
 
Thread Rating:
  • 0 Vote(s) - 0 Average



Logout Mark Read Team Forum Stats Members Help
USA clients sent to EU mirrors?00