Importing TV Shows named by the date they aired, scan & scrape by air-date ("bydate")
#16
http://thetvdb.com/forums/viewtopic.php?f=8&t=36
Quote:The new API is finished. With this api you pass in a seriesid and an airdate and it will return the Episode information if the episode exists.

The full API doc is now up on the wiki and can be found here: http://www.thetvdb.com/wiki/index.php/AP...eByAirDate

Please post any problems or questions to this thread.

Thanks


Now all that is left is implement this in XBMC's TheTVDB scaper Wink
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Reply
#17
Woooot! Man if I could program I'd be bashing away at it right now.
Reply
#18
Gamester17 Wrote:http://thetvdb.com/forums/viewtopic.php?f=8&t=36


Now all that is left is implement this in XBMC's TheTVDB scaper Wink

SWEET! I wish I could help. I tried to make sense of the source code, but could not find the scraper code or make sense of it. Probably will try again tomorrow although someone else who has experience could do it a lot faster. Blush
Reply
#19
I'dd like to look into this, I just have to make sure i understand the scrapers right. This is the first time i'm trying to patch something into XBMC so I don't really know the source that well. Anyone care to elaborate a bit on how the various classes that use the scrapers work together? Maybe if you got an uml class diagram it would be excellent!

I'm actually a bit confused whether it's even necessary to alter the code, would it be posible to just create a new scraper in xml?
Reply
#20
ssboisen Wrote:I'm actually a bit confused whether it's even necessary to alter the code, would it be posible to just create a new scraper in xml?

It seems like that would be the case, or at least modifying the existing one. I found this yesterday (http://wiki.xbmc.org/index.php?title=Scraper.xml) and have been studying it. Maybe it is also what you are looking for to help explain how it all works?
Reply
#21
It has been stated before by a dev that scraping by anything but season/ep will require a code change in XBMC.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Reply
#22
sho Wrote:It has been stated before by a dev that scraping by anything but season/ep will require a code change in XBMC.

I am wondering if this is because the code needs to know that if it doesn't receive two matches from the regex (for season and episode), it could be receiving one match that will be the episode date and to use that with the thetvdb scraper.

Still looking through code to get a better understanding. Sucks coming from a perl/php only back round.
Reply
#23
looking forward to your findings on this. Thanks!
Reply
#24
I noticed that a feature request ticket has been opened.

This actually effects a lot of shows:
Daily Show
Colbert Report
CBS Evening News
The Soup
The Howard Stern Show
Top Gear
Letterman
Leno
Conan

Hopefully it can updated soon. Much love.
Reply
#25
I have the same problem with videos from Microsoft Tech Days, Sun events, dimecast webisodes, conferencies, etc... All these videos are not movies, not TV shows but act like a TV show. it should be interesting to think about a new video content like "Others" that should use a generic 'genre', 'title', ... template.
Reply
#26
I spent a few hours looking at the code and what would be involved in supporting scrapers that scan for episode dates. Instead of searching for season and episode, I would have a few expressions that would parse for the year, month and date. It would then be a question of changing the code to compare the parsed date with the "first aired date" from the data pulled from tvdb or whatever the source is.

The question I have is to how I should do the compare without making things ugly. The episodes are stored in IMDB_EPISODELIST which is a map based on season and episode. I would either have to change this structure around to support the date-based match or iterate the whole list. I'm normally more of a C developer so I'm not sure if there's simply a modification I could make to this class to support both lookups. Any feedback from the devs would be appreciated.

The other potential issue is with regexp conflicts between date and episode formatted lists. For example the date-formatted shows I have already match with the current expressions with the date looking like episode and season values. The expressions would have to be ordered carefully, or perhaps some kind of explicit option on which one to use for which directory would be best.

-Chris
Reply
#27
Has anyone looked into this anymore at all? The script tlunde linked to on the feature request ticket looks like it could be quite helpful to someone who know how to code.
Reply
#28
the script covers the trivial api usage part. csbook pinpointed what needs to be changed.
Reply
#29
ahhh, I see. Okay. I'll just sit back and wait patiently.
Reply
#30
csbook Wrote:I spent a few hours looking at the code and what would be involved in supporting scrapers that scan for episode dates. Instead of searching for season and episode, I would have a few expressions that would parse for the year, month and date. It would then be a question of changing the code to compare the parsed date with the "first aired date" from the data pulled from tvdb or whatever the source is.

The question I have is to how I should do the compare without making things ugly. The episodes are stored in IMDB_EPISODELIST which is a map based on season and episode. I would either have to change this structure around to support the date-based match or iterate the whole list. I'm normally more of a C developer so I'm not sure if there's simply a modification I could make to this class to support both lookups. Any feedback from the devs would be appreciated.

The other potential issue is with regexp conflicts between date and episode formatted lists. For example the date-formatted shows I have already match with the current expressions with the date looking like episode and season values. The expressions would have to be ordered carefully, or perhaps some kind of explicit option on which one to use for which directory would be best.

-Chris

Suggestion for a bit of a HACK to see if scraping based on air date will work without needing to massage the XBMC backend to get a spike working. The reason I'm suggesting this is because I can easily make some tweaks to CMythFile to support these changes.

If I understand correctly, the episode list is keyed on season and episode number. Both of these are likely integersHuh If so, would the following remotely work. Set season = 0 and episode = epoch date, e.g. ms since 1-1-1970 (which would be set from the original air date for the recording, which would have been from the original EPG). It's a piece of cake for me to set the season and episode number within the code - no parsing magic needed.

I think the scraper would then need to check the episode number. If it's bigger than say 100 for Season 0, it knows it's a date not an "actual" episode number. Season 0 can be used for "special" shows, so that can't be used exclusively as the check. Using the epoch time would allow sorting based on episode number to still work correctly.

I'm sure this would mangle the front end UI though. Perhaps I'll have to wait longer for a way less hacky solution... That and the EPG here is New Zealand is scant and doesn't even have the original air date.
Reply

Logout Mark Read Team Forum Stats Members Help
Importing TV Shows named by the date they aired, scan & scrape by air-date ("bydate")1