Kodi Community Forum

Full Version: [Issue] - HTTP(s)/FTP Multi-Threaded Streaming/Download
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
I've Been a follower of XBMC community from Dharma Nice features have been added and numerous bugs have been fixed.
Now What I'm about to ask is what i've researched for more than 2 years, but still i've never really seen a suggestion for a feature like it. (am not really a tech savvy)
I only got to understand the reason behind slow streams in XBMC after numerous research on windows,*nix and other popular PC/Mobile OSes,


What i discovered from my own research, is considering the kinda country am from. many people haven't really had problems with this because either they have large data bandwidth speeds or their ISP's are not been tricky with TCP, Unlike my country where we are limited to 25Mbps internet at-most and *maybe higher.

HTTP Stream/Download from browser (FireFox) = 10-15Mbps
HTTP Stream/Download from XBMC = 3 - 4Mbps*

FTP Stream/Download from browser (FireFox) = 12-18Mbps* (This was done on my personal FTP Box)
FTP Stream/Download from XBMC = 5-8Mbps (From same FTP Box)

I only started to get a good glimpse when i started to understand how Downloaders Like Internet Download Manage [IDM]-(Windows) Axel Download (Linux)
They Use File Chunks by creating different threads of one file that way i could download a file of 100MB in my browser in like 5Mins while In XBMC It's a slow process of buffering,stuttering.

As far as i can tell it's not caching it's just the way XBMC handles Streams using just one connection but that it very bad for us in remote location with just 15-20Mbps connections and not been able to fully utilize it because of a certain feature in XBMC or the way our ISP's try to shape our TCP streams which is bad making XBMC to come down to an annoying 5Mbps = 600 - 500KB/S

But there is a Module for XBMC, Currently in development (Axel Proxy Downloader) has the ability to do all these and more inside XBMC, but still in developmental face, (That why it's prefixed "Work In Progress")
I did a test run although not fully stable yet but it's features are amazing.

After series of search i found a Dev (Eldorado) Was making exactly something like those browser to beat this drawback so i had to organize Devs to help fasten the process it's been almost a year since he started it but it has been a slow process i've sent a an issue on GitHub requesting for other Devs to join him.
he appreciates this well and would love to have finished it but it's been a busy year for him so supporters are welcome

Project Page = Script.module.axeldownloader For XBMC

It would been great if this could become a core part of XBMC itself,

But for now it's best a module as bugs could be fixed from open-source enthusiasts and those with the right skill-set and willing.

Thanks, TerrorKeed Big Grin Wink


Anyone willing to help will be appreciated
It Seems No-one is interested in this.. i would have asked for the thread to be deleted but i choose not to because it will become very useful-very soon
I think this sounds really cool, I just don't have much to add to the discussion since I'm not a programmer.

I imagine it would be really neat to have people be "video mirrors" in the same way that we have software mirrors, and being able to connect to all of them would mean that individual mirrors wouldn't need to necessarily be fast.
I clearly don't understand, but I'm wondering how this would really work if you were streaming a video, given that it only makes sense to pull chunks in parallel if you can cache them locally and re-assemble them in the right order as and when needed... there's no point in having chunks 1, 3 and 5 if you're watching it in realtime and need chunk 2, for example.

Or is this really a downloading issue, like the way a torrent allocates space for the entire file and then simply fills it up as and when it can in whatever order it gets the data?
(2014-02-13, 17:33)Prof Yaffle Wrote: [ -> ]I clearly don't understand, but I'm wondering how this would really work if you were streaming a video, given that it only makes sense to pull chunks in parallel if you can cache them locally and re-assemble them in the right order as and when needed... there's no point in having chunks 1, 3 and 5 if you're watching it in realtime and need chunk 2, for example.

Or is this really a downloading issue, like the way a torrent allocates space for the entire file and then simply fills it up as and when it can in whatever order it gets the data?

Exactly It's a "Download and Feed" Process so it also does us the favor of feeding the file to XBMC On-the-fly.

I don't really have much to say afterall it's open-source you can go ahead and give it a try Big Grin
It seems no one is knowledgable or aware of what am saying... or problly has.never had this issue
Or perhaps doesn't care too much for downloads within XBMC? If people are using it for linear streaming - from a NAS or for iPlayer/Netflix/similar streams - then perhaps there isn't the clamour that you'd expect.
(2014-02-15, 13:22)Prof Yaffle Wrote: [ -> ]Or perhaps doesn't care too much for downloads within XBMC? If people are using it for linear streaming - from a NAS or for iPlayer/Netflix/similar streams - then perhaps there isn't the clamour that you'd expect.

Well this would still be needed as TCP is "tailor-ed" and also The above mentioned TV/Movies networks also use "The Internet" so that's where this plugin comes in... It's more like a booster/accelerator Watching a video from netflix as an instance from browser i.e FireFox (1Hour+) would require like say 3MBPS but because you viewing it via XBMC it's different you are limited to just 1MBPS(now that's bad).

but with axel there is a 500% boost and it gives you the edge over xbmc restrictions and even more Smile
Also if i get what you saying correctly about "download" is a NO NO, why?


reason : becasue axel proxy accelerator only acts as a man in the middle(MITM, lol not the hack term thou.)

1. it tricks xbmc into thinking it's donwloading the file while it's actually feeding you the *Livestream
2. it beats xbmc's current 1-way manner of handling Internet.
3. it makes it possible to download/stream a file in the normal time and not the other way round.

that way you would watch long videos with no buffers from start to finish.(Oh yes even during peak-times)

*** I do understand that the following are factors. Location/Latency, Network Type, and Server Speed. Also at time R/W issues/


But i also stand to say Axel Proxy is a fact!. common you should try this out afterall the project is hosted and everyone is free
to contribute


Stars :
By live stream i meant streaming the file directly and not downloading. (but in some sense streaming is actually downloading too Big Grin )

had nothing to do with LIVETV or Sort of but it also works!
This might strike an eye opener!
Some Useful insights/Stuff Axel is and will be made of(as time permits)
http://en.m.wikipedia.org/wiki/Download_manager
http://en.m.wikipedia.org/wiki/Download_acceleration
http://en.m.wikipedia.org/wiki/Segmented_file_transfer
http://en.m.wikipedia.org/wiki/Compariso...d_managers


Read. Read. Read and Read some more Wink
script.module.axel.downloader is a multi-platform/OS download proxy for XBMC written i various languages like C. C++, C#, Python etc XBMC Plugins/Scripts are pure written in python so is the axel proxy. It uses multipart download to increase speeds, creating multiple connections of different file segments to speed up downloads. script.module.axel.downloader works by splitting files into smaller pieces/chunks and automatically seeking faster mirrors. It also can get a file simultaneously from several sites. and bolsters file reception to the client. (XBMC in this case)

Planned features:
+ability to integrate this into existing plugins/scripts for network downloads
+in each add-on there is a settings to choose how many mirrors of the file you want at a time(User-defined) depends on your processors and RAM.
+Also can increase the way XBMC retrieves pages for single-site addons like(Aljazeera news, RT news, and even for inside XBMC itself like addons download boosting. (Skins,Programs,Scripts)

The general idea is like it's P2P counterpart bittorrent in MagnetBoosting. think of axel as HTTP/FTP "MagnetBooster" Big Grin

Jokingly and for real it can do things you only see in your dreams movies or imagine Wink
Thanks for posting this to help drum up support

At some point I was planning to post a thread in the python development sub forum to get help, I need to wrap my head around the code again to remember where the issues were so I could post up proper questions to ask

But if anyone wishes to help out, check out the code on my git - https://github.com/Eldorados/script.modu...downloader

It's purpose is basically to allow addons to stream in multi-thread rather than the single thread as is currently done

The downloading in multi-thread is working, and you can also download & watch.. but there are some drawbacks to it

Work needs to be done on the cache & stream part, my logic on downloading, assembling and feeding back to XBMC the chunks it's asking for isn't quite right..
(2014-02-21, 18:06)Eldorado Wrote: [ -> ]Thanks for posting this to help drum up support

At some point I was planning to post a thread in the python development sub forum to get help, I need to wrap my head around the code again to remember where the issues were so I could post up proper questions to ask

But if anyone wishes to help out, check out the code on my git - https://github.com/Eldorados/script.modu...downloader

It's purpose is basically to allow addons to stream in multi-thread rather than the single thread as is currently done

The downloading in multi-thread is working, and you can also download & watch.. but there are some drawbacks to it

Work needs to be done on the cache & stream part, my logic on downloading, assembling and feeding back to XBMC the chunks it's asking for isn't quite right..
Yo! no problem bro.
would love to see this

soo many plugin could benefit from this

whats the progress? is there any working demo ?
(2014-05-04, 22:11)nero12 Wrote: [ -> ]would love to see this

soo many plugin could benefit from this

whats the progress? is there any working demo ?

Yeah bro it has gotten so good from the last time I posted here... I couldn't update you guys overe am so sorry about that.


There are already plugins using this.. if you want to test Google "SendToAxel" and install.

remember its still beta.
Pages: 1 2