Kodi Community Forum

Full Version: TV Show Next Aired (Script) Addon, now w/TheTVDB data
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
(2020-01-26, 20:48)jinux Wrote: [ -> ]It does not have to do anything with this?
Does it ?

yes,
https://forum.kodi.tv/showthread.php?pid...pid2791034

and then,
https://forum.kodi.tv/showthread.php?tid...pid2791366


side note, my country.db from 2018 still working fine. if you use the 'reset' button in the settings it deletes this file. so you'd need to manually re-create it..  i havent looked at why it cant create it (this script was back when tvrage was still a thing.. so it tells ya how old it is)
looking through the code,

few things to address. the included dateutil is from 2014 so not really sure its even doing much these days. could remove and just use included "script.module.dateutil" instead.
now the pulling down of tv data part still works fine (even as http and no www, but adding those future proofs some things)

the first roadblock,
script.tv.show.next.aired: ### ERROR retreiving updates from thetvdb.com: File is not a zip file

is when it does the main update it tries to hit an invalid file: http://thetvdb.com/api/xxx/updates/updates_day.zip
we can correct this by doing: https://www.thetvdb.com/api/xxx/updates/updates_day.xml
modify the relevant function to use that url and just parse that data instead of expecting a zip to extract then parse.

script.tv.show.next.aired\resources\lib\thetvdbapi.py
function would become:

    def _get_update_info(self, period, filter_func = None, callback = None):
        url = "%s/updates/updates_%s.xml" % (self.base_xml_url, period)
        return self._get_xml_data(url, filter_func, None, callback)

that stops the script from erroring out.. now it actually pulls down the latest data on demand.

the country.db never generates because country_lookup is busted. but meh, trying to have the script augment timezones with data from tvdb is a crapshoot since you dont really know if the data is in what timezone (just user input) and you have to make assumptions that x station is x timezone first.. which worked somewhat okay back in the day of aired tv.. but with streaming it falls flat. but anyways this 'fixes' it enough to make the script work.. but really this script needs a whole re-write (especially for the impending python3 requirement)
(2020-01-30, 08:11)thezoggy Wrote: [ -> ]looking through the code,

few things to address. the included dateutil is from 2014 so not really sure its even doing much these days. could remove and just use included "script.module.dateutil" instead.
now the pulling down of tv data part still works fine (even as http and no www, but adding those future proofs some things)

the first roadblock,
script.tv.show.next.aired: ### ERROR retreiving updates from thetvdb.com: File is not a zip file

is when it does the main update it tries to hit an invalid file: http://thetvdb.com/api/xxx/updates/updates_day.zip
we can correct this by doing: https://www.thetvdb.com/api/xxx/updates/updates_day.xml
modify the relevant function to use that url and just parse that data instead of expecting a zip to extract then parse.

script.tv.show.next.aired\resources\lib\thetvdbapi.py
function would become:

    def _get_update_info(self, period, filter_func = None, callback = None):
        url = "%s/updates/updates_%s.xml" % (self.base_xml_url, period)
        return self._get_xml_data(url, filter_func, None, callback)

that stops the script from erroring out.. now it actually pulls down the latest data on demand.

the country.db never generates because country_lookup is busted. but meh, trying to have the script augment timezones with data from tvdb is a crapshoot since you dont really know if the data is in what timezone (just user input) and you have to make assumptions that x station is x timezone first.. which worked somewhat okay back in the day of aired tv.. but with streaming it falls flat. but anyways this 'fixes' it enough to make the script work.. but really this script needs a whole re-write (especially for the impending python3 requirement)
played with it a little tonight, got it to make a new country.db
so back in the day you could get the list of 'networks' by just going to a tv show and scraping it.
there still is no api command to get a list of networks and now with csrf tokens.. its a bit of a pain.
you have to go to login page, get cspf token + login, then go to edit a tv show and then it can scrape.

I hacked in some code to the country_lookup to do this with requests + bs4 (with my credentials) and it worked.
now the file it generates is pretty crude and really just defaults everything to USA..  so unless the tv network has (##) in the name it doesnt really yeild too much.
now, could just expose the username+pw in the settings and then people could enter their creds and it would work.. or one could just omit the timezone logic and run with a static country.db (or none)..
Nice to see someone taking an interest in fixing it - all I can do is handle the skin side of things.
ok had to go bust out wayback machine to see why country db wasnt 1:1 from before..

back then every station had (country) which is what was parsed then used later on..

<option value="ABC">ABC (Australia)</option>
<option value="ABC">ABC (Japan)</option>
<option value="ABC">ABC (Philippines)</option>
<option value="ABC">ABC (United States)</option>
<option value="ABC Family">ABC Family (United States)</option>
...
<option value="CPAC">CPAC (Canada)</option>
<option value="Crackle">Crackle (United States)</option>

now,

<option value="6">ABC (AU)</option>
<option value="7">ABC (JA)</option>
<option value="8">ABC (PH)</option>
<option value="5">ABC (US)</option>
<option value="9">ABC Family</option>
...
<option value="74">CPAC</option>
<option value="695">Crackle</option>

so the dilemma is on converting the 2 letter country code to the appropriate country which then can be mapped to the tz... ok so lets use pycountry for that...

Then I ran into "ABC (JA)"
https://www.thetvdb.com/networks/abc-ja

JA is not the 2 letter official code for Japan..  JP is. boo.

would be nice to just be able to get this data via api or even on one page.. it being paginated means it would not be fun to scrape:
https://www.thetvdb.com/networks/

ok so making an override mapping to fix stuff.. its looking whats going to be best is i just get it working and make a pretty and nice country.db and then just would include it vs trying to have anyone actually have to do this on their side
Thanks for the update.
Stupid question... Wouldn't is be possible to set your Country as a setting, and just go from there?
I gave up on trying to scrape the network names like the old code did (it resulted in so much bad data, and was quite hackish solution - having to login to edit a show to get the list that way)
Instead I just made it scrape the public tvdb network pages (all 28 pages and get the 1,364 networks).
This gets the network and the country that tvdb say it should be (which is much better than assuming anything from the network name), then clean up some of those countries so they align with the zone map.
This works much better than before and can eliminate the whole override/fix situation that was in the old code.

Now to test to make sure it is doing what it should and then to strip out the old remove/update logic as the countrydb would just be included with the addon (the code to generate the .db would be there if anyone else wanted to manually do it however).
Now this does take away the user ability to 'fix' it locally since it wouldn't be in their dir but instead the 'fix' would have to make it upstream (which imho is better so everyone benefits).
Also doing this way, it eliminates all those wasted hits to geonames/tvdb to get data that really does not change very often.
The data that does change is the actual timezone mapping..  because dateutil was included with this lib (from 2014) it has some pretty old data. Switching the addon to use "script.module.dateutil" dependency means that will be much newer and actually updated.

Will report back later
(2020-02-01, 23:13)thezoggy Wrote: [ -> ]I gave up on trying to scrape the network names like the old code did (it resulted in so much bad data, and was quite hackish solution - having to login to edit a show to get the list that way)
Instead I just made it scrape the public tvdb network pages (all 28 pages and get the 1,364 networks).
This gets the network and the country that tvdb say it should be (which is much better than assuming anything from the network name), then clean up some of those countries so they align with the zone map.
This works much better than before and can eliminate the whole override/fix situation that was in the old code.

Now to test to make sure it is doing what it should and then to strip out the old remove/update logic as the countrydb would just be included with the addon (the code to generate the .db would be there if anyone else wanted to manually do it however).
Now this does take away the user ability to 'fix' it locally since it wouldn't be in their dir but instead the 'fix' would have to make it upstream (which imho is better so everyone benefits).
Also doing this way, it eliminates all those wasted hits to geonames/tvdb to get data that really does not change very often.
The data that does change is the actual timezone mapping..  because dateutil was included with this lib (from 2014) it has some pretty old data. Switching the addon to use "script.module.dateutil" dependency means that will be much newer and actually updated.

Will report back later
Thanks for the update
Hi everyone.

if I lock the country.db file and perform a full rest Next Aired will  not delete it Smile
it works on my Mac not sure if it will work for other OS
So, just looking for some clarity. This script is not working anymore and nobody is picking it up right? Is there any alternative out there?
(2020-02-04, 04:15)jdig4240 Wrote: [ -> ]So, just looking for some clarity. This script is not working anymore and nobody is picking it up right? Is there any alternative out there?

script works fine (there was fixes that were needed for leia when that came out to be compatible)
just the country.db generation (so it offsets the data from tvdb to your timezone) was not. most of us had an old country.db that just kept working (unless you used the 'reset' button which deleted it).

I've updated the script and tested, but I keep finding other things to fix :x 
For example the fanarttv downloader wasnt working due to invalid apikey so I updated that to a newer one and exposed the setting so users could put in their personal api (update it without a new code release).
Current changelog:
Quote:-6.0.17 (2020-02-02)
- Updated settings to better defaults (banner/today style w/inc yesterday/less aggressive on bg scan).
- Switched to script.module.dateutil so tz data is updated.
- Redid country_lookup logic to pull info from tvdb network pages + updated zone map.
- Removed defunct tvrage
- Removed defunct artwork.downloader integration (Users should switch to artwork.beef)
- Updated fanarttv api key and add setting for use of personal api key

-6.0.16 (2018-11-13)
- Updated script for Leia: "StringCompare" -> "String.IsEqual", "!IsEmpty" -> "!String.IsEmpty"
- Added bg container for top metadata when fanart is used to maintain readability.
(2020-02-04, 08:09)thezoggy Wrote: [ -> ]I've updated the script and tested, but I keep finding other things to fix :x 

Great work thezoggy! It's there somewhere we can get the new code yet, or are you just making changes to your personal copy while testing it out?

I made the changes you posted in #708, to my installed copy, which seem to fix the problem with shows disappearing and not updating unless you force an update.

My temporary fix for the annoying "can't update country.db" error message that pops up all the time and makes you click OK, was too just comment out the lines that makes the error message pop up! Wink

Not ideal, but with these two changes the add-on appears to work adequately well for now.

Thanks for all the work, and hope this add-on gets the proper update it deserves.
Correction: I meant the changes you made in post #707 (although they are quoted in 708).
(2020-02-04, 08:09)thezoggy Wrote: [ -> ]
(2020-02-04, 04:15)jdig4240 Wrote: [ -> ]So, just looking for some clarity. This script is not working anymore and nobody is picking it up right? Is there any alternative out there?

script works fine (there was fixes that were needed for leia when that came out to be compatible)
just the country.db generation (so it offsets the data from tvdb to your timezone) was not. most of us had an old country.db that just kept working (unless you used the 'reset' button which deleted it).

I've updated the script and tested, but I keep finding other things to fix :x 
For example the fanarttv downloader wasnt working due to invalid apikey so I updated that to a newer one and exposed the setting so users could put in their personal api (update it without a new code release).
Current changelog:
Quote:-6.0.17 (2020-02-02)
- Updated settings to better defaults (banner/today style w/inc yesterday/less aggressive on bg scan).
- Switched to script.module.dateutil so tz data is updated.
- Redid country_lookup logic to pull info from tvdb network pages + updated zone map.
- Removed defunct tvrage
- Removed defunct artwork.downloader integration (Users should switch to artwork.beef)
- Updated fanarttv api key and add setting for use of personal api key

-6.0.16 (2018-11-13)
- Updated script for Leia: "StringCompare" -> "String.IsEqual", "!IsEmpty" -> "!String.IsEmpty"
- Added bg container for top metadata when fanart is used to maintain readability.
Awesome, so where can i get your fixed version?