2014-01-17, 21:53
Next to the problem described here, I also run into a different problem when scraping my large list of movies (2500 movies). Whenever I do a full scrape with TMDB (I need that one because it has german descriptions, plots and outlines), it is not only extremely slow but also stops working after several entries. There is no error message or anything, it simply stops working. When I try to restart scraping (even with a single entry) it simply doesnt work, I don't get any content.
First of all I thought TMDB has a limit in its API so people can only scrape a certain amount of movies each hour. Though, when I close Ember, restart it and restart scraping, it works again. Now, the whole problem is, that only about 10-30 movies get scraped, this would make me restart Ember about 125 times until I am finished and that sounds like a big pain -_-
I am using BETA Ember 1.4 - 20140110 btw, I also uploaded the log file to my DropBox if this helps somehow. I really hope a solution can be found because I am very frustrated at the moment.
Link: https://dl.dropboxusercontent.com/u/5462...-01-17.csv
I also noticed that if I do not close Ember, the CSV gets larger and larger and larger... something strange is going on...
Thanks for yout work and help,
best regards
Torm
First of all I thought TMDB has a limit in its API so people can only scrape a certain amount of movies each hour. Though, when I close Ember, restart it and restart scraping, it works again. Now, the whole problem is, that only about 10-30 movies get scraped, this would make me restart Ember about 125 times until I am finished and that sounds like a big pain -_-
I am using BETA Ember 1.4 - 20140110 btw, I also uploaded the log file to my DropBox if this helps somehow. I really hope a solution can be found because I am very frustrated at the moment.
Link: https://dl.dropboxusercontent.com/u/5462...-01-17.csv
I also noticed that if I do not close Ember, the CSV gets larger and larger and larger... something strange is going on...
Thanks for yout work and help,
best regards
Torm