[TobiasTheCommie]

Ok I have spent some time with this today and wow it's super easy to use - very slick. I am pulling all sorts of things very easily....

I have nothing to compare to really, but this certainly works easily and fast.

Oh good good. Even if you can't compare it is still good to know that you don't think it is a pain to work with.

Though we still have time to fix it if that were the case.

[_Pierre_]

The included fetchPage can currently not be used to post.

We are currently converting the Vimeo plugin to use these dependencies as well. And we are updating the dependencies as we go along.

I "hope" to have login procedure in Vimeo go through the fetchPage, but that is currently not the case. And that would require post ability.

Cool looking forward to that

It's a joy to use, using (getParameters) and (getUserInput) in SoundCloud plugin

[TobiasTheCommie]

If you don't mind getting from trunk, i could probably hook up POST pretty quick.

We already have it supported in our fetchPage in the YouTube plugin(It uses it's own function for various reasons, same name though. Just for the added confusion it brings.)

The function definitely needs more features, but if you want to convert from urllib to fetchPage, feel free to post a code snippet with what you want done in urlilb2, and i'll see if i can integrate it into fetchpage in a non stupid way.

That is basically the only requirement i have for functions in fetchPage. They must not require a stupid implementation to work, or cause problems/complexities there are no use for. (Hence why the YouTube plugin uses its own function, that version is stupid and complex).

[_Pierre_]

If you don't mind getting from trunk, i could probably hook up POST pretty quick.

We already have it supported in our fetchPage in the YouTube plugin(It uses it's own function for various reasons, same name though. Just for the added confusion it brings.)

The function definitely needs more features, but if you want to convert from urllib to fetchPage, feel free to post a code snippet with what you want done in urlilb2, and i'll see if i can integrate it into fetchpage in a non stupid way.

That is basically the only requirement i have for functions in fetchPage. They must not require a stupid implementation to work, or cause problems/complexities there are no use for. (Hence why the YouTube plugin uses its own function, that version is stupid and complex).

I use the mechanize lib now not urllib2

Basically I post the first of 3 forms on the page with filled in user and password and get back the querystring from the resultpage.

br = mechanize.Browser()
r = br.open(url)
br.select_form(nr=1)

# User credentials
br.form['username'] = self.username
br.form['password'] = self.password

# Login
br.submit()

result = br.geturl()

[TobiasTheCommie]

Ah yes, i've seen that. Never used it though.

ETA:

Had to do this for Vimeo anyways. So i've added Refering, Cookie and POST support in the latest trunk.

The Vimeo Plugin can perform a full login(And emulate the user agreeing to the XBMC application access) through the functions in version 0.9.1

[_Pierre_]

Ah yes, i've seen that. Never used it though.

ETA:

Had to do this for Vimeo anyways. So i've added Refering, Cookie and POST support in the latest trunk.

The Vimeo Plugin can perform a full login(And emulate the user agreeing to the XBMC application access) through the functions in version 0.9.1

Will check it out when the update comes to xbmc, I'm lazy

[TobiasTheCommie]

Not only a lazy, but also a wise choice.

[_Pierre_]

Bye the way the wiki page is wrong for fetchPage
It should be "link" instead of "url"
_fetchPage({"link": "http://www.google.be"})

And now I'll continue being lazy

[TobiasTheCommie]

Hah, sorry bout that.

Updated the wiki.

[bossanova808]

ah awesome as I have been struggling for some reason with a urllib POST. Much nicer to do it all with your stuff I reckon.

To get from trunk do I have to pull it from GIT? Do you have your own repo for that or is just in trunk xbmc? Sorry, still a bti new to git.

My code is:

Code:

text = '3051'
        #need to submit the postcode to the weatherzone search
        searchURL = 'http://weatherzone.com.au/search/'
        user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
        headers = { 'User-Agent' : user_agent }        
        #need to fetch a page with POST variables
        values = {'q' : text, 't' : '3', 'action' : '/search/'}
        data = urllib.urlencode(values)
        print 'Encoded:', data
        req = urllib2.Request(searchURL, data, headers)
        response = urllib2.urlopen(req)
        resultPage = str(response.read())
        print response.geturl()
        skimmed = common.parseDOM(resultPage, "ul", attrs = { "class": "typ2" })
        print '*********************', skimmed

...it's just some hacky test code but it's not working grrr...all it does is get the /search/ page without any results as if it is not doing a POST.

[TobiasTheCommie]

We have it in a mercurial repo here http://hg.tobiasussing.dk/hgweb.cgi/commonxbmc/

But i honestly wouldn't suggest fooling around with unstable code while at the same time trying to solve a problem in your own code.

Seems like a bad idea to me.

[bossanova808]

Fair enough, I am sure that is good advice but I have spent several hours on the above, which should work according to everything I have read & understand - but it doesn't.

I have checked a manual POST using http live headers and it appears to be sending the same data, but it simply won't get me any results. So either they are somehow using something other than user agent to detect it's a programmatic submission, or I am missing an obvious error.
Grrrr.

[TobiasTheCommie]

Well, you are adding the headers differently from what i do.

Besides for that it looks fine.

[bossanova808]

Apparently I had to set the Host in the headers...now it works.

Again, though - I am loving this parseDOM thing - really, it's super easy.

[newatv2user]

I'm loving this too. Now can you add more html codes to replaceHtmlCodes so I can be more lazy? Thanks.