Help creating "Next Page" function.

  Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
synnc Offline
Junior Member
Posts: 5
Joined: Apr 2017
Reputation: 0
Post: #1
Hello guys,

I need implement a "Next Page" function.
The website im scrapping has in some categories multiple pages and only has 24 items per page, so i get the url of the next page.
But now since i have the next page url, how i call the same function to deal with the same data but only on a different page. (Maybe to make my function recursive?)
And how i call the function on the "Next Page" Directory?

Or whats the better aproach to implement a next page functionality, on a multipage website?

Quick brief, lets say:
Im on this page "www.lol.com/1" and has 24 videos, and the link to the next page where there has more 24, and this goes on about 20 pages.
I grab the link "www.lol.com/2" on the first page, but how i call the same function to deal with scrapping of the page 2?
And how to implement it on "Next page" directory created by xbmcplugin.addDirectoryItem?

Here's my code at the moment:
Code:
def INSIDEcategorie(url):
    addon_handle = int(sys.argv[1])
    r = requests.get(url)
    match = re.compile('<div class="movie-img img-box pseudo-link" data-link="(.+?)">.+?<img src="(.+?)" alt="(.+?)" />',re.DOTALL).findall(r.content)
    next_page = re.compile('<span class="pnext"><a href="(.+?)"><span class="fa fa-angle-double-right"></span></a></span>',re.DOTALL).findall(r.content)
    for url, image, name  in match:
        addDir3(name, url, 3, image, '', '')    
    for next2 in next_page:
        if next2:
            li = xbmcgui.ListItem('Next Page', iconImage='')
            xbmcplugin.addDirectoryItem(handle=addon_handle, url=str(next_page), listitem=li, isFolder=True)
            xbmcplugin.endOfDirectory(addon_handle)

Thanks for the help.
(This post was last modified: 2017-04-21 15:34 by synnc.)
find quote
Skipmode A1 Offline
Skilled Python Coder
Posts: 277
Joined: Mar 2013
Reputation: 24
Post: #2
here's a working example: https://github.com/skipmodea1/plugin.vid...gs_list.py

start with http://www.lol.com/1 and calculate the next one (http://www.lol.com/2)
find quote
synnc Offline
Junior Member
Posts: 5
Joined: Apr 2017
Reputation: 0
Post: #3
(2017-04-21 18:57)Skipmode A1 Wrote:  here's a working example: https://github.com/skipmodea1/plugin.vid...gs_list.py

start with http://www.lol.com/1 and calculate the next one (http://www.lol.com/2)

I dont need to calculate the next one, i can scrap the link on "www.lol.com/1", my problem is how to call the same function to scrap the new page.
Thanks for the example.
(This post was last modified: 2017-04-21 19:23 by synnc.)
find quote
synnc Offline
Junior Member
Posts: 5
Joined: Apr 2017
Reputation: 0
Post: #4
UPDATE: I've managed to make it work.
Thanks.
find quote