Beta - WebGrab+Plus Configurator

  Thread Rating:
  • 3 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
donbrew Offline
Senior Member
Posts: 260
Joined: Oct 2013
Reputation: 4
Post: #46
:duh: Slaps forehead. I thought I had done that.
find quote
MikeKL Offline
Fan
Posts: 423
Joined: Aug 2014
Reputation: 47
Post: #47
(2016-08-09 21:55)primaeval Wrote:  0.0.6
- hide channels in Channels (hidden channels are not output to config file)
- Countries sort order fix

Anything else for the TODO list?
Tested 0.0.6 and confirm that above changes working well Cool

Using "select" on any Channel we have added to our Channels list, changes colour of Channel Name from Yellow to White and/or White to Yellow
(Yellow indicates will BE included, White indicates Channel will NOT BE included in config file, when Write Config File option used)
Countries sort order (retained when written to Config) works.

Very happy with the options to organise channels and then use move option for finer position in channel list tweaking CoolCool
When Write Config File option is used all expected modifications are correctly written. CoolCoolCool

Noted that with Move Channels..that possible sometimes move is not completely respected, not sure but may have something to do with distance in list a channel is being moved, need to move twice to be correctly added before expected channel (If this makes any sense?)

Delete, Perhaps move to lower/bottom of channel option list?
Add to favourites, What is the idea for this option, which Add a channel from Guide to user favourites?

@primaeval Thank you, its impressive how quickly you incoperate user prposals into your beta addons Big GrinCoolBig Grin

Now will move onto integration testing of the configurator with LibreELEC webgrabplus addon

@DarrenHill, thanks for heads up on settings to use, Not tested yet and intrigued for example how many days will be captured as setting is not currently in Configurators created WebGrab++.config.xml and awiouy has currently choosen to automatically run epg extract routine every six hours from point of an Rpi boot-up, in addition to providing webgrabplus.start that we can run at any time via ssh and now with primeaval configurator Cool

rPi 2, SanDisk Ultra 8Gb SD & Milhouse LE TestBuild (Kodi 18) -> Onkyo AV -> Philips 42" LCD TV
(This post was last modified: 2016-08-10 11:17 by MikeKL.)
find quote
primaeval Offline
Posting Freak
Posts: 1,733
Joined: Apr 2014
Reputation: 182
Post: #48
(2016-08-10 11:16)MikeKL Wrote:  
(2016-08-09 21:55)primaeval Wrote:  0.0.6
- hide channels in Channels (hidden channels are not output to config file)
- Countries sort order fix

Anything else for the TODO list?
Tested 0.0.6 and confirm that above changes working well Cool

Using "select" on any Channel we have added to our Channels list, changes colour of Channel Name from Yellow to White and/or White to Yellow
(Yellow indicates will BE included, White indicates Channel will NOT BE included in config file, when Write Config File option used)
Countries sort order (retained when written to Config) works.

Very happy with the options to organise channels and then use move option for finer position in channel list tweaking CoolCool
When Write Config File option is used all expected modifications are correctly written. CoolCoolCool

Noted that with Move Channels..that possible sometimes move is not completely respected, not sure but may have something to do with distance in list a channel is being moved, need to move twice to be correctly added before expected channel (If this makes any sense?)

Delete, Perhaps move to lower/bottom of channel option list?
Add to favourites, What is the idea for this option, which Add a channel from Guide to user favourites?

@primaeval Thank you, its impressive how quickly you incoperate user prposals into your beta addons Big GrinCoolBig Grin

Now will move onto integration testing of the configurator with LibreELEC webgrabplus addon

@DarrenHill, thanks for heads up on settings to use, Not tested yet and intrigued for example how many days will be captured as setting is not currently in Configurators created WebGrab++.config.xml and awiouy has currently choosen to automatically run epg extract routine every six hours from point of an Rpi boot-up, in addition to providing webgrabplus.start that we can run at any time via ssh and now with primeaval configurator Cool

0.0.7
- fixed sort countries again Wink
- timespan Setting called Days
- backup WebGrab++.config.xml to WebGrab++.config.xml.last on Write Config

The Move might be a bug or it might be the Channel list not being updated immediately.
Some devices like Android try to be frugal with gui updates.

Delete has to be delete.
You can click the Channels to hide them from the config file.

Add to Favourites is not mine.
It is just a default menu item from Kodi.
I doubt if it is useful here.

I think an option for a longer update period in LibreELEC would be good.
I don't want the providers' servers to be abused and shut us down.

Timespan is now included.
Do you need any of the other webgrab config settings to be included?

You're lucky I am not working at the moment.
When I get a job this development will probably grind to a halt. Wink
find quote
DarrenHill Offline
Team-Kodi Minion
Posts: 5,758
Joined: Oct 2014
Reputation: 132
Location: UK, except when I'm not.
Post: #49
If it's helpful, I'll review and compare my two xml files (one done manually from WebGrab+ and one done via the configurator) tonight and see what is in the former and not the latter to see what other settings may be useful to include.

| Banned add-ons (wiki) | Forum rules (wiki) | First time user (wiki) | FAQs (wiki) | Troubleshooting (wiki) | Add-ons (wiki) | Free content (wiki) | Debug Log (wiki) | Free Movies/TV Shows
find quote
MikeKL Offline
Fan
Posts: 423
Joined: Aug 2014
Reputation: 47
Post: #50
Reviewed 0.0.7 configurator WebGrab++.config.xml with the LibreELEC addon version of WebGrab++.config.xml and previous expierence when using PC and webgrabplus to build guide.xml before passing to Pi over network, Following are the setting fields that I currently believe should always be included.

Already fully covered by WebGrab+Plus Configurator @v0.0.7 Cool

<timespan>
<filename>
<channel>

To possible include in future update of WebGrab+Plus Configurator. Blush

<mode> "m" measure "d" for debug?
<proxy> "Automatic" Auto proxy?
<user-agent> Typical User agent string sent to tvguide website. As some sites may still require it?
<logging> "on" Ensure WebGrab++.log.txt is always created?
<retry> "4" Number of times engine will retry to capture a site page?
<update> To change from standard "I" Incremental to "f" Force to enable a full update of an existing EPG regardless of current/previous state.

Not sure about need for all/any of addtions above to be typically user editable within configurator as webgrabplus may modify options and/or add new additional settings in future that we may need to be able to cover?

Perhaps and advanced settings editable text file, to which lines can be manually edited/added/deleted by user utilising kodi configurator, which are then always included in prepared WebGrab++.config.xml

Hope thoughts/ideas are useful

rPi 2, SanDisk Ultra 8Gb SD & Milhouse LE TestBuild (Kodi 18) -> Onkyo AV -> Philips 42" LCD TV
find quote
MikeKL Offline
Fan
Posts: 423
Joined: Aug 2014
Reputation: 47
Post: #51
(2016-08-10 12:07)primaeval Wrote:  Delete has to be delete.
You can click the Channels to hide them from the config file.
Fully understood, was only meaning to re-review order of the options at some point Delete is now currently top of the list (I asked if you could move it into) and I find myself typically tabbing past it to get to more used options? BlushBlush

rPi 2, SanDisk Ultra 8Gb SD & Milhouse LE TestBuild (Kodi 18) -> Onkyo AV -> Philips 42" LCD TV
(This post was last modified: 2016-08-10 16:33 by MikeKL.)
find quote
primaeval Offline
Posting Freak
Posts: 1,733
Joined: Apr 2014
Reputation: 182
Post: #52
(2016-08-10 16:33)MikeKL Wrote:  
(2016-08-10 12:07)primaeval Wrote:  Delete has to be delete.
You can click the Channels to hide them from the config file.
Fully understood, was only meaning to re-review order of the options at some point Delete is now currently top of the list (I asked if you could move it into) and I find myself typically tabbing past it to get to more used options? BlushBlush

Ok. I didn't get that. Could you give me your preferred order.

Also, coding Settings for the Webgrab Config File aren't too hard to do.
Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off
find quote
MikeKL Offline
Fan
Posts: 423
Joined: Aug 2014
Reputation: 47
Post: #53
(2016-08-10 16:43)primaeval Wrote:  Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off
OK, a first attempt

Quote:timespan default=1 type=number
proxy default=Automatic type=free alphanumeric text?
user-agent default=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; yie9) type=free alphanumeric text?
mode default=m values=m|d|n|v|w (measure|debug|nomark|verify|wget)
logging default=on values=on|off
retry default=4 type=number
update default= values=i|l|s|f|f (incremental|Light|smart|full|force)

Note: Number of these options can also contain attributes, although I have not personally needed to use additional attribute information.
Detailed orginating Reference info @ http://webgrabplus.com/documentation/con...bconfigxml

rPi 2, SanDisk Ultra 8Gb SD & Milhouse LE TestBuild (Kodi 18) -> Onkyo AV -> Philips 42" LCD TV
find quote
primaeval Offline
Posting Freak
Posts: 1,733
Joined: Apr 2014
Reputation: 182
Post: #54
(2016-08-10 17:31)MikeKL Wrote:  
(2016-08-10 16:43)primaeval Wrote:  Could you give me a list of the Settings with the optional values for them and the ones that should always be there.
eg something like
Settings
* timespan default=1 type=number
* mode default=m values=m|d (measure|debug)
* logging default=on values=on|off
OK, a first attempt

Quote:timespan default=1 type=number
proxy default=Automatic type=free alphanumeric text?
user-agent default=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; yie9) type=free alphanumeric text?
mode default=m values=m|d|n|v|w (measure|debug|nomark|verify|wget)
logging default=on values=on|off
retry default=4 type=number
update default= values=i|l|s|f|f (incremental|Light|smart|full|force)

Note: Number of these options can also contain attributes, although I have not personally needed to use additional attribute information.
Detailed orginating Reference info @ http://webgrabplus.com/documentation/con...bconfigxml

Great stuff.

Do you think any of these should be hidden and just set to a default value in the code?

This is the list of context menu items in Channels:

'Delete Channel',
'Move Channel',
'Rename Channel',
'Rename xmltv id'
'Sort Channels',

What order do you want them in?
find quote
MikeKL Offline
Fan
Posts: 423
Joined: Aug 2014
Reputation: 47
Post: #55
(2016-08-10 18:00)primaeval Wrote:  Great stuff.

Do you think any of these should be hidden and just set to a default value in the code?

This is the list of context menu items in Channels:

'Delete Channel',
'Move Channel',
'Rename Channel',
'Rename xmltv id'
'Sort Channels',

What order do you want them in?
Context menu for channels, for me...

'Sort Channels', (To organise a "best fit" intial sort list from selected channels)
'Move Channel', (To move channels that are not quite where you want into final list sequence)
'Rename Channel', (To modify channel name when useful i.e. to clarify a HD channel)
'Rename xmltv id' (Not used very often)
'Delete Channel', (Used infrequently to remove un-required channels from list rather than hide)

Regarding default advanced setting values, dont believe many users will have need to touch any of them and could be fully hidden, but believe we need clean way to keep them within a general WebGrab++.config.xml

Personally I am happy to use ssh and an editor to make any modifications if guaranteed not to be touched when config file is written by the configurator add-on Wink but thought it an idea to make webgrabplus settings more accesible to all kodi users?

rPi 2, SanDisk Ultra 8Gb SD & Milhouse LE TestBuild (Kodi 18) -> Onkyo AV -> Philips 42" LCD TV
(This post was last modified: 2016-08-10 18:49 by MikeKL.)
find quote
gaskell1066 Offline
Junior Member
Posts: 25
Joined: Aug 2012
Reputation: 1
Post: #56
having followed all the intructions when i select run webgrab plus its crashing with the following error

WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/18 -- Jan van Straaten
-----------------------------------------------------------
Job started at 10/08/2016 17:59:01


Error while reading -- WebGrab++.config.xml -- check existance and/or format
Exception details: System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
at System.Data.DataSet.ReadXml(String fileName)
at WebGrab.Config..ctor(String Path, String NameVersionAuthor, DateTime JobStart)

[Image: untitled.png]

I am doing something wrong, can anyone help, everything seems to be correct and the channels are all listed in the config.xml correctly but as soon as i run webgrab i get the error
(This post was last modified: 2016-08-10 19:10 by gaskell1066.)
find quote
primaeval Offline
Posting Freak
Posts: 1,733
Joined: Apr 2014
Reputation: 182
Post: #57
(2016-08-10 19:08)gaskell1066 Wrote:  having followed all the intructions when i select run webgrab plus its crashing with the following error

WebGrab+Plus/w MDB & REX Postprocess -- version 1.1.1/18 -- Jan van Straaten
-----------------------------------------------------------
Job started at 10/08/2016 17:59:01


Error while reading -- WebGrab++.config.xml -- check existance and/or format
Exception details: System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
at System.Data.DataSet.ReadXml(String fileName)
at WebGrab.Config..ctor(String Path, String NameVersionAuthor, DateTime JobStart)

[Image: untitled.png]

I am doing something wrong, can anyone help, everything seems to be correct and the channels are all listed in the config.xml correctly but as soon as i run webgrab i get the error

Make sure you have applied the webgrab Upgrade from:
http://webgrabplus.com/download

Mine says V1.57 at the top of the dos window.
Yours says version 1.1.1/18.

If not please attach the webgrab output log from:
\userdata\addon_data\script.webgrab\webgrab\config\WebGrab++.log.txt
find quote
gaskell1066 Offline
Junior Member
Posts: 25
Joined: Aug 2012
Reputation: 1
Post: #58
thank you so much for your prompt reply, upgraded to v1.57 and now works perfectly :-)
find quote
donbrew Offline
Senior Member
Posts: 260
Joined: Oct 2013
Reputation: 4
Post: #59
Wouldn't 1 days worth of data every 18 hours do the job?
I have my PC set to 4 days every 3 days at 2 A.M.
find quote
primaeval Offline
Posting Freak
Posts: 1,733
Joined: Apr 2014
Reputation: 182
Post: #60
(2016-08-10 22:43)donbrew Wrote:  Wouldn't 1 days worth of data every 18 hours do the job?
I have my PC set to 4 days every 3 days at 2 A.M.

It depends when you watch tv.
The default setting of 1 day is only the current day not the next 24 hours.
So if you are watching close to midnight your program list won't see much ahead.
(This post was last modified: 2016-08-10 23:27 by primaeval.)
find quote
Post Reply