[WIP] XBMC Flicks (Netflix Addon) Development Thread - Developers Only!
#1
Lightbulb 
I tried the option of opening the netflix via the external player, there's just too much on the screen to navigate with a controller (i use the xbox 360 controller as my xbmc controller, the keyboard is tucked away and only used when i need to do something else on the machine). So I started digging abit into how to interact with the netflix api.

So i've come up with a way to display the instant queue, and open the movie via the external app (IE), but there's gotta be some cool scripting options that i can use to make it seem more like something that's meant for xbmc.

I have the basic netflix api parts working, i can retrieve the instant queue and provide the url that can be used to play the movie. I can write out, in any format, the movie url that will play the movie, the title, the poster URL (Or just write the image to a local file moviename.tbn)

The POC for that is just a simple windows form that has a picture box that onclick opens the url in the browser.

This is done using the c# oauth base class and the wrapnetflix .net code

From the user perspective in XBMC, i'd like to have it display the poster and when clicked it would play the movie

This could be done with the following
A MOVIE_NAME.htm file is created for each item in instant queue by the .net app
(this is done by a .net app that runs in the background, updating on a user specified inteval)
It could also just have a REFRESH_QUEUE.htm, with a listener in the .net app that would tell it to recreate those items.

source added to video sources where those .htm files are located
on disc it would be
c:\xbmc\netflix - this is the source we add to the video section of xbmc
in there there is a file, Refresh_Queue.htm
in there is also a folder, Netflix_Instant_Queue
in the folder (Netflix_Instant_Queue) are the Movie_Name.htm files that the .net app creates, that link to movie on netflix
when one of those Movie_Name.htm are clicked on, the external player opens ie in full screen and plays the movie.

Thoughts?? Ideas?

Right now i'm just opening up my windows form in full screen and can click on the image using my remote to play the movie, but when it's done i have to close the browser window, then exit the windows form. To interact with the queue, it must stay in my windows app, there's a bunch of oauth stuff and hashes that must be done for each item in order to add/delete/modify/list the queue (dev key, user key, secrets, and a hash in sha1 of the requested resource). Also, I have to auth the user before I make a bunch of requests so that I dont hit the limits per day, 4 per second (app limit) 5,000 per day (that are not authenticated, this would be app level), 5,000 per day authenticated (i.e. that counter is at the user level, this is where those requests need to be, otherwise the daily limit for the app would be hit after only a few users)

The Movie_Name.htm is just a redirect to the real URL of the movie in the browser.
Reply
#2
What do you need to access the netflix queue? Could you do all that part of it in XBMC itself (i.e. from python)?
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#3
jmarshall Wrote:What do you need to access the netflix queue? Could you do all that part of it in XBMC itself (i.e. from python)?

oauth - i'm sure there's a python script for it out there somewhere already
netflix api calls - there is a python example out there for it

pass app key and secret to netflix, user enter's login info and approves it (add's device to approved list), this returns the users key and secret, those are stored for additional calls and won't change as long as the user doesn't remove it from the approved list of devices in netflix

to get the queue, we pass the app key, app secret, user key, user secret, and a hash of the requested resource uri that combines the keys & secrets with the uri data into a 20 char ascii string that is the oauth_secret, we also send a unique id per request (noonce).. sending all that junk to the netflix api via http post makes it an authorized connection (all the keys and the oauth secret make up the required token) and will return XML that contains the user's instant queue.

To play the movie, we only need the app's key and the id of the movie
to change it in the queue (add/delete/remove) we need the token info (as described above)
searching adding to instant queue if available, are all just calls to different parts of the netflix api

I'll post up some details (that includes the library i used for oauth which is not that complicated), the netflix api wrapper i used isn't needed as all the calls can be re-written, it's a nice reference to see what the calls are doing.
Reply
#4
Right, so it sounds like all that side of it could be done by a python plugin in XBMC.

On playback, the python plugin would then evoke an external player to through up the browser and tear it down afterward.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#5
jmarshall Wrote:Right, so it sounds like all that side of it could be done by a python plugin in XBMC.

On playback, the python plugin would then evoke an external player to through up the browser and tear it down afterward.
Correct, it can all be done in python and that would be the best user experience with it by far.

Now that i've got it working in c# code, i think i understand it enough to use python, but haven't done anything with it before.

I'll see about some pseudo code, with the c# version of that code, that goes through the process of pulling the instant queue. I'm skipping the initial user part as there's a nice example in netflix's api on that one, i'll get that link tonight as well.
Reply
#6
Update: I found a c# to python converter and popped in the oauth.cs (from oauth's base class (google code) http://oauth.googlecode.com/svn/code/csh...uthBase.cs) and then cleaned up the comments.

I'll see about converting over the calls and other items in the next few days. It should be fairly easy to convert over now that I found a tool to do most of the conversion from code i understand to py

The post below contains the converted oauth.py class that will allow the tokens to be created and other common functions needed with the api calls (Update: it's currently a mixed bag of calls that I'll need to re-write)
Reply
#7
Code:
from System import *
from System.Security.Cryptography import *
from System.Collections.Generic import *
from System.Text import *
from System.Web import *

class OAuthBase(object):
    def __init__(self):
        # Provides a predefined set of algorithms that are supported officially by the protocol
        # Provides an internal structure to sort the query parameter
        # Comparer class used to perform the sorting of the query parameters
        self._OAuthVersion = "1.0"
        self._OAuthParameterPrefix = "oauth_"
        # List of know and used oauth parameters' names
        self._OAuthConsumerKeyKey = "oauth_consumer_key"
        self._OAuthCallbackKey = "oauth_callback"
        self._OAuthVersionKey = "oauth_version"
        self._OAuthSignatureMethodKey = "oauth_signature_method"
        self._OAuthSignatureKey = "oauth_signature"
        self._OAuthTimestampKey = "oauth_timestamp"
        self._OAuthNonceKey = "oauth_nonce"
        self._OAuthTokenKey = "oauth_token"
        self._OAuthTokenSecretKey = "oauth_token_secret"
        self._HMACSHA1SignatureType = "HMAC-SHA1"
        self._PlainTextSignatureType = "PLAINTEXT"
        self._RSASHA1SignatureType = "RSA-SHA1"
        self._random = Random()
        self._unreservedChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.~"

    class SignatureTypes(object):
        def __init__(self):

    class QueryParameter(object):
        def __init__(self, name, value):
            self._name = None
            self._value = None
            self._name = name
            self._value = value

        def get_Name(self):
            return name

        Name = property(fget=get_Name)

        def get_Value(self):
            return value

        Value = property(fget=get_Value)

    class QueryParameterComparer(IComparer):
        def Compare(self, x, y):
            if x.Name == y.Name:
                return str.Compare(x.Value, y.Value)
            else:
                return str.Compare(x.Name, y.Name)

    def ComputeHash(self, hashAlgorithm, data):
        # Helper function to compute a hash value
        # hashAlgorithm: The hashing algoirhtm used. If that algorithm needs some initialization, like HMAC and its derivatives, they should be initialized prior to passing it to this function
        # The data to hash
        # a Base64 string of the hash value
        if hashAlgorithm == None:
            raise ArgumentNullException("hashAlgorithm")
        if str.IsNullOrEmpty(data):
            raise ArgumentNullException("data")
        dataBuffer = System.Text.Encoding.ASCII.GetBytes(data)
        hashBytes = hashAlgorithm.ComputeHash(dataBuffer)
        return Convert.ToBase64String(hashBytes)

    def GetQueryParameters(self, parameters):
        # Internal function to cut out all non oauth query string parameters (all parameters not begining with "oauth_")
        # parameters: The query string part of the Url
        # returns: A list of QueryParameter each containing the parameter name and value
        if parameters.StartsWith("?"):
            parameters = parameters.Remove(0, 1)
        result = List[QueryParameter]()
        if not str.IsNullOrEmpty(parameters):
            p = parameters.Split('&')
            enumerator = p.GetEnumerator()
            while enumerator.MoveNext():
                s = enumerator.Current
                if not str.IsNullOrEmpty(s) and not s.StartsWith(OAuthParameterPrefix):
                    if s.IndexOf('=') > -1:
                        temp = s.Split('=')
                        result.Add(QueryParameter(temp[0], temp[1]))
                    else:
                        result.Add(QueryParameter(s, str.Empty))
        return result

    def UrlEncode(self, value):
        # This is a different Url Encode implementation since the default .NET one outputs the percent encoding in lower case.
        # While this is not a problem with the percent encoding spec, it is used in upper case throughout OAuth
        # value: The value to Url encode
        # returns: a Url encoded string
        result = StringBuilder()
        enumerator = value.GetEnumerator()
        while enumerator.MoveNext():
            symbol = enumerator.Current
            if unreservedChars.IndexOf(symbol) != -1:
                result.Append(symbol)
            else:
                result.Append('%' + String.Format("{0:X2}", symbol))
        return result.ToString()

    def NormalizeRequestParameters(self, parameters):
        # Normalizes the request parameters according to the spec
        # parameters: The list of parameters already sorted
        # returns: a string representing the normalized parameters
        sb = StringBuilder()
        p = None
        i = 0
        while i < parameters.Count:
            p = parameters[i]
            sb.AppendFormat("{0}={1}", p.Name, p.Value)
            if i < parameters.Count - 1:
                sb.Append("&")
            i += 1
        return sb.ToString()

    def GenerateSignatureBase(self, url, consumerKey, token, tokenSecret, httpMethod, timeStamp, nonce, signatureType, normalizedUrl, normalizedRequestParameters):
        # Generate the signature base that is used to produce the signature
        # url: The full url that needs to be signed including its non OAuth url parameters
        # consumerKey: The consumer key        
        # token: The token, if available. If not available pass null or an empty string
        # tokenSecret: The token secret, if available. If not available pass null or an empty string
        # httpMethod: The http method used. Must be a valid HTTP method verb (POST,GET,PUT, etc)
        # signatureType: The signature type.
        # returns: The signature base
        if token == None:
            token = str.Empty
        if tokenSecret == None:
            tokenSecret = str.Empty
        if str.IsNullOrEmpty(consumerKey):
            raise ArgumentNullException("consumerKey")
        if str.IsNullOrEmpty(httpMethod):
            raise ArgumentNullException("httpMethod")
        if str.IsNullOrEmpty(signatureType):
            raise ArgumentNullException("signatureType")
        normalizedUrl = None
        normalizedRequestParameters = None
        parameters = self.GetQueryParameters(url.Query)
        parameters.Add(QueryParameter(OAuthVersionKey, OAuthVersion))
        parameters.Add(QueryParameter(OAuthNonceKey, nonce))
        parameters.Add(QueryParameter(OAuthTimestampKey, timeStamp))
        parameters.Add(QueryParameter(OAuthSignatureMethodKey, signatureType))
        parameters.Add(QueryParameter(OAuthConsumerKeyKey, consumerKey))
        if not str.IsNullOrEmpty(token):
            parameters.Add(QueryParameter(OAuthTokenKey, token))
        parameters.Sort(QueryParameterComparer())
        normalizedUrl = str.Format("{0}://{1}", url.Scheme, url.Host)
        if not ((url.Scheme == "http" and url.Port == 80) or (url.Scheme == "https" and url.Port == 443)):
            normalizedUrl += ":" + url.Port
        normalizedUrl += url.AbsolutePath
        normalizedRequestParameters = self.NormalizeRequestParameters(parameters)
        signatureBase = StringBuilder()
        signatureBase.AppendFormat("{0}&", httpMethod.ToUpper())
        signatureBase.AppendFormat("{0}&", self.UrlEncode(normalizedUrl))
        signatureBase.AppendFormat("{0}", self.UrlEncode(normalizedRequestParameters))
        return signatureBase.ToString()

    def GenerateSignatureUsingHash(self, signatureBase, hash):
        # Generate the signature value based on the given signature base and hash algorithm
        # signatureBase: The signature based as produced by the GenerateSignatureBase method or by any other means
        # hash: The hash algorithm used to perform the hashing. If the hashing algorithm requires initialization or a key it should be set prior to calling this method
        # returns: A base64 string of the hash value
        return self.ComputeHash(hash, signatureBase)

    def GenerateSignature(self, url, consumerKey, consumerSecret, token, tokenSecret, httpMethod, timeStamp, nonce, normalizedUrl, normalizedRequestParameters):
        # Generates a signature using the HMAC-SHA1 algorithm
        # url: The full url that needs to be signed including its non OAuth url parameters
        # consumerKey: The consumer key
        # consumerSecret: The consumer seceret
        # token: The token, if available. If not available pass null or an empty string
        # tokenSecret: The token secret, if available. If not available pass null or an empty string
        # httpMethod: The http method used. Must be a valid HTTP method verb (POST,GET,PUT, etc)
        # returns: A base64 string of the hash value
        return self.GenerateSignature(url, consumerKey, consumerSecret, token, tokenSecret, httpMethod, timeStamp, nonce, SignatureTypes.HMACSHA1, , )

    def GenerateSignature(self, url, consumerKey, consumerSecret, token, tokenSecret, httpMethod, timeStamp, nonce, signatureType, normalizedUrl, normalizedRequestParameters):

        # Generates a signature using the specified signatureType
        # url: The full url that needs to be signed including its non OAuth url parameters
        # consumerKey: The consumer key
        # consumerSecret">The consumer seceret
        # token: The token, if available. If not available pass null or an empty string
        # tokenSecret: The token secret, if available. If not available pass null or an empty string
        # httpMethod: The http method used. Must be a valid HTTP method verb (POST,GET,PUT, etc)
        # signatureType: The type of signature to use
        # returns: A base64 string of the hash value</returns>
        normalizedUrl = None
        normalizedRequestParameters = None
        if signatureType == SignatureTypes.PLAINTEXT:
            return HttpUtility.UrlEncode(str.Format("{0}&{1}", consumerSecret, tokenSecret))
        elif signatureType == SignatureTypes.HMACSHA1:
            signatureBase = self.GenerateSignatureBase(url, consumerKey, token, tokenSecret, httpMethod, timeStamp, nonce, HMACSHA1SignatureType, , )
            hmacsha1 = HMACSHA1()
            hmacsha1.Key = Encoding.ASCII.GetBytes(str.Format("{0}&{1}", self.UrlEncode(consumerSecret), "" if str.IsNullOrEmpty(tokenSecret) else self.UrlEncode(tokenSecret)))
            return self.GenerateSignatureUsingHash(signatureBase, hmacsha1)
        elif signatureType == SignatureTypes.RSASHA1:
            raise NotImplementedException()
        else:
            raise ArgumentException("Unknown signature type", "signatureType")

    def GenerateTimeStamp(self):
        # Generate the timestamp for the signature        
        # returns: string based off int64 of total seconds since jan 1, 1970
        # Default implementation of UNIX time of the current UTC time
        ts = DateTime.UtcNow - DateTime(1970, 1, 1, 0, 0, 0, 0)
        return Convert.ToInt64(ts.TotalSeconds).ToString()

    def GenerateNonce(self):
        # Generate a nonce
        # returns: randon number
        # Just a simple implementation of a random number between 123400 and 9999999
        return random.Next(123400, 9999999).ToString()
Reply
#8
You need the following items to make the request
string ConsumerKey = "gnexy7jajjtmspegrux7c3dj";
//the key is xbmcflix key for Netflix API, it is the actual key for the app i've titled xbmcflix (Is that name ok to use?)

string ConsumerSecret = THIS_IS_A_SECRET;
//the secret is a secret and needs to be hashed and not displayed in text form, this is provided by netflix for the developer

string UserId = THIS_IS_RETRIEVED_FROM_NETFLIX;
string AccessToken = THIS_IS_RETRIEVED_FROM_NETFLIX;
string AccessTokenSecret = THIS_IS_RETRIEVED_FROM_NETFLIX;

The UserId, AccessToken, and AccessTokenSecret parts are not yet converted to python.

Here is the instant queue in python, returned in XML format, some parts of this might still need translated from c# to py

Code:
def getInstantQueueXML(self):
        curlink = "http://api.netflix.com/users/" + UserId + "/queues/instant"
        oab = OAuth.OAuthBase()
        nonce = oab.GenerateNonce()
        timeStamp = oab.GenerateTimeStamp()
        url = curlink
        curUri = Uri(url)
        sig = oab.GenerateSignature(curUri, ConsumerKey, ConsumerSecret, AccessToken, AccessTokenSecret, "GET", timeStamp, nonce, , )
        sig = oab.UrlEncode(sig)
        httpString = normalizedUrl + "?" + normalizedParameters + "&oauth_signature=" + sig
        
        result = ""
        result = self.getHttpResponse(httpString)
        //parseing of resulting xml would take place next


    def getHttpResponse(self, url):
        # NOTE: must import urllib
        f = urllib.urlopen(url)
        # Read from the object, storing the page's contents in 's'.
        s = f.read()
        f.close()
        return s


Playing the movie
//the videoid can be grabbed from the id in the xml for the item
example:
- <queue_item>
<id>http://api.netflix.com/users/%USERID%/queues/instant/available/1/70135754</id>
70135754 is the VideoID

To play the movie, the url is
string playmovieUrl = "http://www.netflix.com/WiPlayerCommunityAPI?lnkctr=apiwn&nbb=y&devKey=" + ConsumerKey +"&movieid=" + curInstantQueueItem.VideoID;
Reply
#9
https://github.com/simplegeo/python-oauth2

Using this as a starting point for oauth would probably be better than trying to port ironPython code, as it tends to rely heavily on the .net framework.
Reply
#10
galvanash Wrote:https://github.com/simplegeo/python-oauth2

Using this as a starting point for oauth would probably be better than trying to port ironPython code, as it tends to rely heavily on the .net framework.
thanks for the link

after looking at how that one is done, and actually looking up some of the py calls, i noticed the oauth code i posted is a mix of stuff.. i'm going to rewrite it as I need to lookup how the calls are done in py anyway to understand how to get it into xbmc
Reply
#11
or instead of re-inventing the wheel, there's already a netflix api in py done
http://code.google.com/p/pyflix/source/browse/

ok, after figuring out how that works, i can pull the instant queue data with the python netflix api (linked above), basically i just modified the netflixapi from that one to add a function that just grabs the instant queue (based off the one that pulls the rental history)

there's a few requirements, simplejson, and oauth, both of which where already done and required no changes (also GPL'd so we are good there too)

The script has to be run 3 times, and modified with the user information and dev information as it's being processed. So that will need some tweaking there.
first run will create the link out to the netflix page to authorize the app
second run will get the user id, user token and user token secret
third time is the charm as we have all the required keys, now it pulls the instant queue

So assuming we are already past the user id/token parts, we now have a string of all the values of the instant queue

Here's where i'll need some assistance, getting that into the xbmc ui
anyone got a simple Hello World on that or willing to help out?

Edit: found the xbmc hello world type example Big Grin i'll test things out tonight, if it works, i'll post it
Reply
#12
Does oauth contain binary bits? If not, great - we can push that into the repo as a separate script lib.

If it does then things are a bit trickier as we need to build it (or grab it) and distribute with XBMC itself (until binary addons come along)

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#13
jmarshall Wrote:Does oauth contain binary bits? If not, great - we can push that into the repo as a separate script lib.

If it does then things are a bit trickier as we need to build it (or grab it) and distribute with XBMC itself (until binary addons come along)

Cheers,
Jonathan

It's actually the MIT license, not GPL, don't think that'll be a problem
No binary as far as i can tell

source: https://github.com/simplegeo/python-oaut..._init__.py
Reply
#14
That's fine then - we can pull it in as a lib and all addons will benefit Smile

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#15
Ok, i'm getting closer but am missing something with actually displaying the links in the UI

Update: i've got it to load the script, but it keeps loading it over and over again so there's still something missing.

here's the pastebin of xbmc's debug log
http://pastebin.com/8TNC94BT

Update #2: the loop error only occurs in dharma rc1 and in rc2.. however it doesn't happen with an older build (28078) .. very odd, but hey I have the basic info in there now

That means there is 2 things I still need to determine
1) if that loop error occurs with RC2 (i'll test that next)--Update: issue occurs in RC2 as well
2) how to tell it to play the video url link via the external player (iexplorer) .. guessing that needs configured in playercorefactory.xml and then there must be a way to tell it to use that player to play the links

If anyone has an idea for #2, that's a key one for sure
Reply

Logout Mark Read Team Forum Stats Members Help
[WIP] XBMC Flicks (Netflix Addon) Development Thread - Developers Only!1