• 1
  • 27
  • 28
  • 29(current)
  • 30
  • 31
  • 48
Zap2xml for ATSC in OpenELEC
OK - So with the changes coming in Kodi 18 and the issues with different OSs - I think I'm going to go back to the beginning and write this as a true kodi addon. It was thrown together from a simple python command line program that was actually derived from a perl version. Since kodi and tvheadend were and afterthought and I started this when I knew absolutely nothing about python (and I'm still technically a novice!) I think it wil lbe better to rewrite the whole thing (or big parts of it).

Hopefully that can bring a unfied working addon for all platforms. Since I have a two week trip coming up - I can hopefully get a lot done on the plane rides. :-)
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
That will be awesome, thanks.
Reply
One more thing... @stephr1 - I did test the 0.7.4-test version that I posted and it worked on my end - so with whatever chenages you need for the ubuntu setup it should pull down the xtra details info for you.

If you have a working version right now for basic data - you can manually edit the zap2xml.py file and make changes to lines 341-351 making them match lines 341-354 on the 0.7.4-test version.

It adds a try: around the xtra details downloads and hopefully moves on if it fails.
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
I'm thinking with some of the things that have gone on recently with the system, something related to the grabber might be "off", rather than the grabber itself. If I get some time in the next couple of days, I'll "flatten" the grabber (uninstall/delete any extraneous files) and then reinstall, mod the paths and settings as needed, and try again.

I'll let you know the results.

As usual, thx and cheers....

(2017-09-09, 16:55)edit4ever Wrote: One more thing... @stephr1 - I did test the 0.7.4-test version that I posted and it worked on my end - so with whatever chenages you need for the ubuntu setup it should pull down the xtra details info for you.

If you have a working version right now for basic data - you can manually edit the zap2xml.py file and make changes to lines 341-351 making them match lines 341-354 on the 0.7.4-test version.

It adds a try: around the xtra details downloads and hopefully moves on if it fails.
Reply
UPDATE (9/15/17):

Update happened as normal last night/this morning with this msg near the end of the log file (in .kodi/addons/.....):

2017-09-15 03:31:12,978 Form:
<zbSearchForm GET http://tvschedule.zap2it.com/tvlistings/ZCSearch.do application/x-www-form-urlencoded
<TextControl(searchTerm=)>
<HiddenControl(searchType=simple) (readonly)>
<HiddenControl(aid=tvschedule) (readonly)>
<SubmitControl(=Search) (readonly)>>:Function: loginZAP :Line: 570
2017-09-15 03:31:12,982 Form:
<zbSearchFormAdv GET http://tvschedule.zap2it.com/tvlistings/ZCSearch.do application/x-www-form-urlencoded
<TextControl(searchTerm=)>
<SelectControl(searchField=[*name, episodeName, episodeName, description, crew])>
<SelectControl(searchGenre=[*, movie, sports, children, special, news])>
<CheckboxControl(searchHD=[true])>
<HiddenControl(searchType=advanced) (readonly)>
<HiddenControl(aid=tvschedule) (readonly)>
<SubmitControl(=Search) (readonly)>>:Function: loginZAP :Line: 570
2017-09-15 03:31:12,985 Form:
<zcLoginForm POST http://tvschedule.zap2it.com/tvlistings/...o?category= application/x-www-form-urlencoded
<TextControl(username=)>
<PasswordControl(password=)>
<SubmitControl(<None>=Login) (readonly)>
<IgnoreControl(loginReset=<None>)>
<HiddenControl(zc-login-forwardURL=) (readonly)>>:Function: loginZAP :Line: 570
2017-09-15 03:31:14,765 Matched .*Logout of your.*:Function: loginZAP :Line: 591

...and this msg in .kodi/userdata/add_ons/.... at the end of the log file:

http://tvschedule.zap2it.com/tvlistings/...tvschedule

http://tvschedule.zap2it.com/tvlistings/...tvschedule

Matched .*Logout of your.*:Function: loginZAP :Line: 591
Getting: http://tvschedule.zap2it.com/tvlistings/...tvschedule
[55/56] Parsing: /home/osmc/.kodi/addons/script.module.zap2xml/cache/1506625200000.html.gz
Getting: http://tvschedule.zap2it.com/tvlistings/...tvschedule
[56/56] Parsing: /home/osmc/.kodi/addons/script.module.zap2xml/cache/1506646800000.html.gz
Downloaded 896852 bytes in 2 http requests.

Writing XML file: /home/osmc/.xmltv/tvguide.xml

Completed in 158s (Parse: 79s) 78 stations, 10729 programs, 36152 scheduled.

So, looks like success. I want to watch it for another few days and see how it works (or doesn't work). Looks like (for me, anyway) having xtra details enabled is causing problems. I'm wondering if something else (I have a Dish DVR that does nightly updates) is trying to use my local net around the same time and that's causing problems for the RPI 3/OSMC system. It shouldn't, but variables do and constants aren't Sad To confirm or deny this I'll check out "call home" times for both units and make sure they access at different times at night (This wasn't a problem until recently, tho Sad

Peace out and cheers....


ORIGINAL posting:

Not sure what's going on. I completely uninstalled the 0.7.4-test. Deleted the 2 script.module.zap2zml folders (left the tv_grab_zap2xml file since that did not seem to be an issue) and did a reinstall of V0.7.4-test. I have had mixed results. I was able to do a couple of manual updates (restart tvheadend) over the past few days with no errors. However, the normal 3:30 AM updates are not happening and are producing errors. And a recent manual update failed.

In the .kodi/addons... folder I'm still getting the "HTTP Error 500: Internal Server Error" msgs. In the log file under .kodi/userdate.... I'm getting the following msgs (Can't tell if it's directly related to the Error 500 msg. Based on the 3 retries, I expect it is.):

Getting: http://tvschedule.zap2it.com/tvlistings/...3657560349
:Function: getURL :Line: 225
error<class 'mechanize._response.httperror_seek_wrapper'>
Getting: http://tvschedule.zap2it.com/tvlistings/...3657560349
:Function: getURL :Line: 225
error<class 'mechanize._response.httperror_seek_wrapper'>
Getting: http://tvschedule.zap2it.com/tvlistings/...3657560349
:Function: getURL :Line: 225
error<class 'mechanize._response.httperror_seek_wrapper'>
Failed to download within 3 retries.

this was done with xtra details enabled. So, I've disabled those again and will try both a manual run as well as see what happens overnight.

I'll post back results tomorrow.

Thanks and cheers....

(2017-09-10, 17:58)stephr1 Wrote: I'm thinking with some of the things that have gone on recently with the system, something related to the grabber might be "off", rather than the grabber itself. If I get some time in the next couple of days, I'll "flatten" the grabber (uninstall/delete any extraneous files) and then reinstall, mod the paths and settings as needed, and try again.

I'll let you know the results.

As usual, thx and cheers....

(2017-09-09, 16:55)edit4ever Wrote: One more thing... @stephr1 - I did test the 0.7.4-test version that I posted and it worked on my end - so with whatever chenages you need for the ubuntu setup it should pull down the xtra details info for you.

If you have a working version right now for basic data - you can manually edit the zap2xml.py file and make changes to lines 341-351 making them match lines 341-354 on the 0.7.4-test version.

It adds a try: around the xtra details downloads and hopefully moves on if it fails.
Reply
I finally am back home and get back to testing and reworking this issue.

@stephr1 - since I can not recreate the issue on my setup (my guide data loads correctly, including xtra details)...I would like to test if I can recreate using your lineup. Can you let me know what zipcode and lineup you are using that has the error? I will test and see if the same thing happens on my end if I use your lineup.

Thanks!
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
I'm at 94086.

I'm having mixed results. Since my last posting, in order I had 1 successful updt, 2 unsuccessful, followed by unsuccessful last night.

I'm kinda wondering back to the failed updt when this all began. I have no idea to be sure, but I'm still sensing some kind of network conflict or other related issue. I've come close a couple of times to flattening the system and doing a new build (after salvaging the movies and tv shows already on the system). Not quite yet (cuz I'm optimistic Wink, but......

Thanks for your continued persistence on this. Maybe we'll get it figured out.

(2017-09-21, 01:00)edit4ever Wrote: I finally am back home and get back to testing and reworking this issue.

@stephr1 - since I can not recreate the issue on my setup (my guide data loads correctly, including xtra details)...I would like to test if I can recreate using your lineup. Can you let me know what zipcode and lineup you are using that has the error? I will test and see if the same thing happens on my end if I use your lineup.

Thanks!
Reply
94086 and an OTA (antenna) lineup?
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
I just ran a full day download with xtra details for all channels in the 94086 OTA lineup. Built the xmltv.xml file just fine.

I'm guessing the error is caused when details data is missing. I'll let this continue for a few days to see if the error shows up. Please let me know if you see an issue on your end.

Thanks!
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
Sorry. Yes, OTA.

Mine updated last night, as well. However, I have extra details disabled.

My plan was to send you the associated code (EP0XXX...XXX) for the data (or lack there of) that shows up when errors are flagged in the log file(s). I'll have to wait until I get an error (or see previous posting(s)). Maybe I should re-enable xtra details to speed up the process. I'll check the results tomorrow morning and decide at that point.

What's not clear (and maybe could be more of a concern) is the error may be a result of the possible problem I've mentioned previously about my system. I guess eventually one of us will be able to determine the source of this.

Thanks....cheers...

(2017-09-21, 02:42)edit4ever Wrote: I just ran a full day download with xtra details for all channels in the 94086 OTA lineup. Built the xmltv.xml file just fine.

I'm guessing the error is caused when details data is missing. I'll let this continue for a few days to see if the error shows up. Please let me know if you see an issue on your end.

Thanks!
Reply
@edit4ever

have been somewhat following your progress with stephr1.
Question, is now the time to upgrade to latest version?
I am still using 0.7.1 no extra details. It is working well as is.
other than turning on extra details is not providing any extra details.

Also I have some time on and off for the next week if you need help trouble shooting.
Reply
UPDATE (9/24/17):

Updates 2 nights in a row. I was going to ask whether you knew if using a VPN (OpenVPN) might be a problem but last night the VPN was enabled. The night before (also successful update) the VPN was disabled. I suppose, tho, that the VPN connection itself could be suspect. I'll have to experiment with VPN enabled/disabled with xtra details enabled and see what happens.

Orignial Posting:

I expect OK. Realized if you're trying to simulate my configuration, the best thing would be for you to have the channels I've enabled. I included channel nos. only but if you need the channel call letters, I can provide those as well. As I put together this list, I started wondering if a few of the subchannels might be at the source of the problem (like unsupported or invalid program info). Here they are:

1.1
2.1, 2.3, 2.4
4.1, 4.2, 4.3
5.1, 5.2
7.1, 7.2, 7.3
8.1, 8.2
9.1, 9.2, 9.3
10.1, 10.2, 10.3
11.1, 11.2
13.1, 13.2
14.2, 14.3, 14.4
20.1, 20.2, 20.3, 20.4
22.1, 22.2, 22.3
25.1, 25.2, 25.3, 25.4, 25.5
26.1, 26.2, 26.3, 26.4, 26.5, 26.6
32.2, 32.4, 32.5, 32.6
36.1, 36.2, 36.3, 36.4
38.2, 38.3, 38.4, 38.5, 38.6
44.1
54.1, 54.2, 54.3, 54.4, 54.5
58.1
60.1, 60.2, 60.3
65.1, 65.2, 65.3, 65.4, 65.5, 65.6
67.1, 67.2, 67.3

Cheers....

(2017-09-23, 03:03)stephr1 Wrote: Sorry. Yes, OTA.

Mine updated last night, as well. However, I have extra details disabled.

My plan was to send you the associated code (EP0XXX...XXX) for the data (or lack there of) that shows up when errors are flagged in the log file(s). I'll have to wait until I get an error (or see previous posting(s)). Maybe I should re-enable xtra details to speed up the process. I'll check the results tomorrow morning and decide at that point.

What's not clear (and maybe could be more of a concern) is the error may be a result of the possible problem I've mentioned previously about my system. I guess eventually one of us will be able to determine the source of this.

Thanks....cheers...

(2017-09-21, 02:42)edit4ever Wrote: I just ran a full day download with xtra details for all channels in the 94086 OTA lineup. Built the xmltv.xml file just fine.

I'm guessing the error is caused when details data is missing. I'll let this continue for a few days to see if the error shows up. Please let me know if you see an issue on your end.

Thanks!
Reply
OK - once you get an error again we can look at the episode number and match it to a channel. Then I can run some tests on a fix. AUntil then - it's a bit hard to work on. :-)
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
in .kodi/addons/script.module.zap2xml/zap2xml.log I see this error

2017-09-25 20:24:30,497 Didn't Match .*Logout of your Screener account.* Sleep 1 sec.:Function: loginZAP :Line: 591
2017-09-25 20:24:31,500 Failed to login within ,3, retries.
:Function: loginZAP :Line: 594
2017-09-25 20:24:31,500 None

I logged into zap2it site via browser. That works fine.
I restarted librelec. Same error.

I am running libreelec 8.1.2 (I was on 8.0.2 when error started, I updated it since then).
Shall i try your, script.module.zap2xml-0.7.4-test.zip? (If yes how? i.e. use "install from zip file"?
Thank you and thank you for such an awesome addon.
Reply
Yes you can try the 0.7.4-test.zip version. Just download and install from zip file in the addons section and it will update your version.

Then please let me know if it solved the login issue or not. There seems to be some changes happening at the zap2it website that might be impacting the grabber login.

Thanks!
Current Kodi addon projects: zap2epg, sd4tvh and tvh2kodi (tvh config from inside Kodi)
Testing ATSC single and dual tuners on RPi3 and the occasional s905 box
If you like my work you can buy me a coffee!
Reply
  • 1
  • 27
  • 28
  • 29(current)
  • 30
  • 31
  • 48

Logout Mark Read Team Forum Stats Members Help
Zap2xml for ATSC in OpenELEC4