How do you make a site scrapeable?
#15
asphinx Wrote:Checked the tvdb framework, after my initial reaction of "what the hell" had settled, I started snooping around. But unfortunately as of now, I am nowhere near grasping the structural layout and inner workings that is thetvdb 0.3.

But I have been working on a prototype that might just do what's necessary anyway. I do however have a question (as I know next to nothing about scraping xml)

If say, an xml data container, contains the following php

Code:
<?php echo date (”Y-m-j h:i:s A (T)”, getlastmod()); ?>

does the scraper only read the data (as is) or is it possible to get the actual result of the echo instead, in this case the last modified date?

0.3 is REALLY old. Sorry about that. We're in the middle of completely overhauling our database and moving it to Postgres (most likely) instead of MySQL, which will make most of the SQL exist only in stored procedures on the database. It should really simplify things.

You missed the point that spiff was making. That PHP code you see will get turned into a date on the server and returned. Client side users and applications won't see any of the PHP code itself.

I'd be happy to give you a bit of guidance for your project... just send me a PM. The main thing is that you'll really need a grasp of PHP and SQL (mysql or Postgres). Also keep in mind that once XBMC and Media Portal hit your site, you're looking at a TON of bandwidth and CPU usage so you'll really need a dedicated host when that time comes (instead of a shared host, which is what many website hosting companies provide).
Reply


Messages In This Thread
[No subject] - by spiff - 2009-01-12, 13:46
[No subject] - by asphinx - 2009-01-12, 13:57
[No subject] - by spiff - 2009-01-12, 14:01
[No subject] - by asphinx - 2009-01-12, 14:22
[No subject] - by spiff - 2009-01-12, 14:23
[No subject] - by asphinx - 2009-01-12, 14:30
[No subject] - by spiff - 2009-01-12, 14:32
[No subject] - by asphinx - 2009-01-12, 14:42
For you information... - by Gamester17 - 2009-01-12, 18:46
[No subject] - by asphinx - 2009-01-12, 20:49
[No subject] - by asphinx - 2009-01-13, 11:40
[No subject] - by spiff - 2009-01-13, 12:21
[No subject] - by asphinx - 2009-01-13, 13:03
[No subject] - by tvdb-scott - 2009-02-15, 09:23
[No subject] - by asphinx - 2009-02-18, 15:21
Logout Mark Read Team Forum Stats Members Help
How do you make a site scrapeable?0