[olympia]

Running Kodi 16.1, and there doesn't seem to be an update for that.  Is support being dropped?

Preflighting the request to "please update to later or latest Kodi" I'll say that this is not an easy undertaking, requiring replacing the media boxes in use.

I'm also posting over at thetvdb.com to ask why their legacy support for their older API changed arbitrarily today, but we'll see how far that gets.

Previously, when thetvdb API issues began cropping up a couple of months ago, a patch was released for Kodi thetvdb scraper for Kodi 16.1.  Please could we have that again?

Did thetvdb api really change to using JSON instead of XML?  That's all I've been able to sniff out at this point.

Please stop spreading this ignorant rumor about JSON vs XML. 
Kodi scraper addons are scraping JSON APIs, then transforming the collected results into an XML which is then gets parsed by Kodi core.
The API being JSON or XML doesn't matter at all, nevertheless there are no modern APIs returning response in XML any more.
This is the same with tvdb. Kodi's tvdb scraper is scraping from tvdb via its JSON API for ages. The changes which brake the scraper is not JSON vs XML related!
It is just that the layout of tvdb's JSON response changes and unfortunately quite frequently these days.

[btasker]

Running Kodi 16.1, and there doesn't seem to be an update for that.  Is support being dropped?

Preflighting the request to "please update to later or latest Kodi" I'll say that this is not an easy undertaking, requiring replacing the media boxes in use.

I'm also posting over at thetvdb.com to ask why their legacy support for their older API changed arbitrarily today, but we'll see how far that gets.

Previously, when thetvdb API issues began cropping up a couple of months ago, a patch was released for Kodi thetvdb scraper for Kodi 16.1.  Please could we have that again?

Did thetvdb api really change to using JSON instead of XML?  That's all I've been able to sniff out at this point.

Please stop spreading this ignorant rumor about JSON vs XML. 
Kodi scraper addons are scraping JSON APIs, then transforming the collected results into an XML which is then gets parsed by Kodi core.
The API being JSON or XML doesn't matter at all, nevertheless there are no modern APIs returning response in XML any more.
This is the same with tvdb. Kodi's tvdb scraper is scraping from tvdb via its JSON API for ages. The changes which brake the scraper is not JSON vs XML related!
It is just that the layout of tvdb's JSON response changes and unfortunately quite frequently these days. 

Thanks for the update, seems to be working now.

> Please stop spreading this ignorant rumor about JSON vs XML. 

Sorry but given the API response is JSON and the debug log explicitly complains about XML you can see how I'd come to that conclusion.

If people arriving at that conclusion is an issue for you, then perhaps improving the logging is in order - particularly given that Kodi's default logging level is particularly unhelpful for this issue - "Unable to parse website" really doesn't tell you much.

From a quick look at the pull and the addon, it looks like the JSON is being parsed with regex rather than a... well... JSON parser. This seems to be why changes in layout breaks things, if the JSON were being deserialised this wouldn't be an issue, unless I've missed it it's not like they've changed any attribute names, just changed the order in which they appear.

All this is from a quick look, so perhaps I'm missing something, but it seems quite unfair to be pointing the finger at TheTVDB when the issue appears to be how you're handling the responses

[olympia]

I don't agree with your first comment about logging - the ones who can help and fix know how to read the logs. Logs doesn't necessarily need to be user (even if advanced user like you) interpret-able under and for all circumstances... 
...and if you are don't know/ unsure on how to read, then you shouldn't go far based on assumptions, definitely shouldn't come to a conclusion and ultimately is better to not post all over (i.e. immediately pull the trigger with a guess on tvdb forums.)

Your second comment on how Kodi parses JSON is valid from an angle. 
...and Kodi will address this from the next version by introducing python scrapers and subsequently proper JSON parser.
However, the regexp based XML scraping engine proved it's power over the past decade and that facility made/ make it possible to develop and maintain scrapers by any non-coder, advanced user. 
This era will change and time will tell...

[btasker]

I don't agree with your first comment about logging - the ones who can help and fix know how to read the logs. Logs doesn't necessarily need to be user (even if advanced user like you) interpret-able under and for all circumstances... 
...and if you are don't know/ unsure on how to read, then you shouldn't go far based on assumptions, definitely shouldn't come to a conclusion and ultimately is better to not post all over (i.e. immediately pull the trigger with a guess on tvdb forums.)

I quite explicitly said the log suggested it was trying to parse XML as JSON.

On the TVDB forums I asked if anything had changed. I didn't mention anything about XML vs JSON (though I did link back to my post here rather than having to type bits of it out).

I agree that logs don't need to be user interpretable, but what they shouldn't be is misleading. Where they are, you have to accept that wrong conclusions will sometimes be drawn by someone looking to see if they can find a fix - we all start by being unfamiliar with the software we're tinkering with after all.

Your second comment on how Kodi parses JSON is valid from an angle. 
...and Kodi will address this from the next version by introducing python scrapers and subsequently proper JSON parser.
However, the regexp based XML scraping engine proved it's power over the past decade and that facility made/ make it possible to develop and maintain scrapers by any non-coder, advanced user. 
This era will change and time will tell...

Being picky, given that it's regex based, the scraper was never RFC compliant - JSON objects are unordered. That the next version is going to address this though, is awesome news

Again, thanks for the fix.

[olympia]

....and the log was correct.
As I explained in one of my previous post, the scraper addon parses the site, then generates an XML from the collected data what XML then gets parsed by Kodi core. So for Kodi core it is an XML just as how the log suggested.
While I agree that if you only pick out the word "XML" from the context of the log, then it's misleading, hence I said you need to know what you are reading before making any conclusions. 

You are right in saying we all start being unfamiliar, but in case you are interested, lets ask it here, discuss it here, at least let's wait 24 hours if someone reacts or maybe even fix the issue.
Actually you did mentioned that Kodi expects an XML response on tvdb API, but let's get over that.

For the rest, thank you for the pitch in your 3. post in the forum suggesting that my 10 years work on non RFC compliant scrapers is a piece of shit.
Maybe it's time to stop fixing things what I really don't like or use (e.g. tvdb in overall).

[btasker]

....and the log was correct.
As I explained in one of my previous post, the scraper addon parses the site, then generates an XML from the collected data what XML then gets parsed by Kodi core. So for Kodi core it is an XML just as how the log suggested.
While I agree that if you only pick out the word "XML" from the context of the log, then it's misleading, hence I said you need to know what you are reading before making any conclusions. 

You are right in saying we all start being unfamiliar, but in case you are interested, lets ask it here, discuss it here, at least let's wait 24 hours if someone reacts or maybe even fix the issue.
Actually you did mentioned that Kodi expects an XML response on tvdb API, but let's get over that.

For the rest, thank you for the pitch in your 3. post in the forum suggesting that my 10 years work on non RFC compliant scrapers is a piece of shit.
Maybe it's time to stop fixing things what I really don't like or use (e.g. tvdb in overall).

I think we've got off on the wrong foot here.

FAOD I'm not for a second saying that your work is a piece of shit. There's a hell of a lot of non-RFC compliant stuff out there that isn't a piece of shit too. As you yourself have said, it's stood the test of being in use for a decade. I get why you might have chosen to go the regex route in the first place - it gives you a great deal of flexibility in adding new things, at the cost of sometimes being fragile - but as I say, it's a little unfair to be pointing fingers at the TVDB's api when they've not changed anything that should matter to a RFC compliant client.

I'm also not underestimating the demand on your time or energy that Kodi as a whole makes.

As far as Kodi goes, I'm a user and I'm doing what users do - having a grumble. It was only really your tone that prompted me to look closer and reply earlier, otherwise I'd pretty much just have posted to confirm the fix was working for me.

Not that it really matters, but I'm not sure the age of my account on the forums is particularly relevant either - it tells you nothing about how long I've been using Kodi, what my background is etc.

[flhthemi]

@olympia Thank you for maintaining this scraper. TVDB IMO is by far the best site for TV info Others just pale in comparison. So thanks and keep up the good work!!

Not butt kissing just giving an atta boy to one who deserves it.

[Parker Evans]

Many thanks - just got the update for Jarvis this morning, and it cleared up a couple of issues here.  That was quick!  Tip o' the hat to the volunteer developers.

[olympia]

Many thanks - just got the update for Jarvis this morning, and it cleared up a couple of issues here.  That was quick!  Tip o' the hat to the volunteer developers.

That's @Karellen 's merit - he created the PR for pre-Krypton versions.

[Parker Evans]

On this topic - I would switch to the MovieDB's TV scraper, but it doesn't function properly on Jarvis for me.  It generates nonsensical entries for ST:Enterprise, for example, linking the first two seasons to a show unrelated to Star Trek in any fashion, and leaving the last two seasons as just files.

Again, many, many thanks to @olympia for the update to the TVDB scraper.  I've some experience in page parsing, and I know it's quite a pain when layout and tags change.

[Parker Evans]

On this topic - I would switch to the MovieDB's TV scraper, but it doesn't function properly on Jarvis for me.  It generates nonsensical entries for ST:Enterprise, for example, linking the first two seasons to a show unrelated to Star Trek in any fashion, and leaving the last two seasons as just files.

Again, many, many thanks to @olympia for the update to the TVDB scraper.  I've some experience in page parsing, and I know it's quite a pain when layout and tags change.

Woops.  Thanks @Karellen

[mmhere123456]

Please stop spreading this ignorant rumor about JSON vs XML. 

Did not know I was being ignorant.  ;-)

All I know is thetvdb API borked their stuff again and a stable Kodi install started to have problems due to thetvdb's negligence in maintaining the APIs they claim they continue to support.

I am very very happy to have this released;  it works for me again:

For various versions of Kodi, these URLs have metadata.tvdb.com-2.1.4.zip --

v13:  http://mirrors.kodi.tv/addons/gotham/metadata.tvdb.com/
v14:  http://mirrors.kodi.tv/addons/helix/metadata.tvdb.com/
v15:  http://mirrors.kodi.tv/addons/krypton/me....tvdb.com/
v16:  http://mirrors.kodi.tv/addons/jarvis/metadata.tvdb.com/

Thanks!

[Karellen]

For various versions of Kodi, these URLs have metadata.tvdb.com-2.1.4.zip --

Yes, that is correct. Gotham v13 - Jarvis v16 use the same scraper. Then Krypton v17 - Matrix v19 use the same, but updated, scraper.

This is caused by new features being added to Kodi that the scrapers are updated to accommodate.

[mmhere123456]

In particular, metadata.tvdb.com-2.1.4.zip allowed me to add two new shows that were previously not in my Kodi libary, and which yesterday (and the day before) refused to add due to thetvdb API shenanigans. Now that I have installed (from zip) that version of the scraper, new shows add once again; new episodes of shows added (previously or just now) scrape as well.

Thanks VERY VERY much for releasing the one-liner pattern change in the scraper logic that facilitates this.

It is MUCH appreciated.

[mmhere123456]

Your comment on how Kodi parses JSON is valid from an angle. 
...and Kodi will address this from the next version by introducing python scrapers and subsequently proper JSON parser.

That is good news indeed that a JSON parser will be used. As a SW dev myself, this sounds less fragile in implementation if perhaps requiring a bit more of a "coder person" to maintain it. That said, maintaining complex regexps like the ones proved to work here for a decade are ... um ... fragile. This is especially so when the upstream data provider who claims to be remaining backwards compatible with their output production is in fact NOT doing so.

When the devs get to it, please LOUDLY highlight the release that first uses JSON parse instead of regexps. This will be a motivation for at least one user (me!) to upgrade my Kodi boxen.