Scraping a subscribed website for video links
#1
Is it possible for me to be able to code an add-on for Kodi (presumably in Python?) that would obtain all of the possible videos or video links from www.horseandcountry.tv (catch up and on demand) using my personal log-in details and list these for me to watch.

I have a fair bit of experience coding (mostly Java, and a little Python), but have never written a Kodi add-on or done anything that scrapes video links from websites etc. I'm a first year computer science student so have a bit of understanding but not much experience!

First off, is what I am looking to do realistic and possible? and secondly, if so, would someone be able to give a very brief overview of how I would go about it and the necessary principles involved?
Reply
#2
first, learn the basic of creating addons.
start with the simple addon in the HOW TO

I installed kodi on windows and added a simple addon.
find where the addon is installed.
modify that simple addon, usually it is contained in default.py

once you've got that simple addon working.
play around some more with listing directory items.

try connecting to websites using the requests library or urllib.

check for basic html knowledge

learn how includes work and include commonfunctions.py to get familiar with parseDOM for scraping websites
alternatively, you parse websites using beautifulsoup4 (parseDOM is easier)

i'm only familiar with scraping for images. i assume that scraping for video is similar but with an added challenge:
websites do not want you scraping their videos and will pull out all the tricks to prevent that.

learn how the browser sends your credentials to horse and country. and emulate it with python.

I use youtube_dl addon to get a playable video from a link.
I think there is an addon called urlresolver that gets videos that are stored in online 'lockers' but i'm not familiar with that.

this is as far as i can go.
Reply
#3
(2016-12-15, 08:40)gedisony Wrote: first, learn the basic of creating addons.
start with the simple addon in the HOW TO

I installed kodi on windows and added a simple addon.
find where the addon is installed.
modify that simple addon, usually it is contained in default.py

once you've got that simple addon working.
play around some more with listing directory items.

try connecting to websites using the requests library or urllib.

check for basic html knowledge

learn how includes work and include commonfunctions.py to get familiar with parseDOM for scraping websites
alternatively, you parse websites using beautifulsoup4 (parseDOM is easier)

i'm only familiar with scraping for images. i assume that scraping for video is similar but with an added challenge:
websites do not want you scraping their videos and will pull out all the tricks to prevent that.

learn how the browser sends your credentials to horse and country. and emulate it with python.

I use youtube_dl addon to get a playable video from a link.
I think there is an addon called urlresolver that gets videos that are stored in online 'lockers' but i'm not familiar with that.

this is as far as i can go.
 that is very good post, i have question, if i want to scrap the links and then run urlresolver in parellel in android which library is best for it ? right now my code scraps video links in sequential manner and then urlresolver once i click on directory ? i want all this in background in parellel processing to speed up the things ? any tip ?
Reply

Logout Mark Read Team Forum Stats Members Help
Scraping a subscribed website for video links0