[Batch]

I figured out a solution for IceFilms to have their movies in the database without hammering their servers. I did some script modifications and keep the link to the page containing the links to the different mirrors instead of the mirrors themselves.

(I also did a minor hack to be able to use XBMC.MyLibrary with IceFIlms but I hope you don't mind since I'm avoiding to spam HTTP requests...)

Grown Ups (2010).strm:

Code:

plugin://plugin.video.icefilms/?url=http%3A%2F%2Fwww.icefilms.info%2Fip.php%3Fv%3D109700%26&mode=400&name=Grown+Ups+%282010%29

When selecting a movie the script scrapes the movies page at IceFilms and let me choose what quality and mirror I want to use.

I tracked all traffic with Wireshark to be sure I didn't accidently hammer IceFilms server. When I filled my library (11639 movies) the script only did about one page request every 5 minutes from IceFilms.info since it only needed to load the pages listing the movies in alphabetic order.

Everything works quite good but I got a few problems:
1) I would like to skip the phase where old movies are deleted. Because the library is as big as it is I'm only scraping all movies once. The scrapes after that first scrape only scrapes the movies found on the "recent movies" page. I fixed this by setting the HoursThreshold really high but it would be better if I just could turn it off.
2) Problems seem to occur when the database gets to big.

Code:

07/30/2011 09:15:44 em INFO    ContentScan      Querying JSON-RPC interface for all videos in library.
07/30/2011 09:15:53 em WARNING ContentScan      The response from XBMC is not a valid JSON string:
                                                    {
                                                       "id" : "1",
                                                       "jsonrpc" : "2.0",
                                                       "result" : {
                                                          "end" : 11724,
                                                          "movies" : [
                                                             {
                                                                "fanart" : "special://masterprofile/Thumbnails/Video/Fanart/db7441f0.tbn",
                                                                "file" : "C:\\dropbox\\Movies\\500 Days of Summer (2009) HD.strm",
                                                                "label" : "(500) Days of Summer",
                                                                "movieid" : 1,
                                                                "thumbnail" : "special://masterprofile/Thumbnails/Video/d/db7441f0.tbn"
                                                             },
                                                             {
                                                                "fanart" : "special://masterprofile/Thumbnails/Video/Fanart/fa92ccc9.tbn",
                                                                "file" : "C:\\dropbox\\Movies\\District 9 (2009) HD.strm",
                                                                "label" : "District 9",
                                                                "movieid" : 2,
                                                                "thumbnail" : "special://masterprofile/Thumbnails/Video/f/fa92ccc9.tbn"
                                                             },

                                                              ... (more movies goes here)

                                                             {
                                                                "fanart" : "special://masterprofile/Thumbnails/Video/Fanart/3f84684a.tbn",
                                                                "file" : "C:\\dropbox\\Movies\\The Morgue (2008).mpg",
                                                                "label" : "The Morgue",
                                                                "movieid" : 7115
                                                             },
                                                             {
                                                                "fan
07/30/2011 09:16:07 em WARNING ContentScan      JSON Returned was not valid (attempt 1 of 3), will try again...
07/30/2011 09:16:09 em WARNING ContentScan      The response from XBMC is not a valid JSON string:
                                                    
07/30/2011 09:16:09 em WARNING ContentScan      JSON Returned was not valid (attempt 2 of 3), will try again...
07/30/2011 09:16:29 em INFO    ContentScan      JSON was successfully retrieved after attempt #3
07/30/2011 09:16:29 em INFO    ContentScan      No TV Shows found in XBMC's library for this series: {"id":"1","result":{"total":0,"start":0,"end":0},"jsonrpc":"2.0"}
07/30/2011 09:16:29 em WARNING ContentScan      No TV Shows/Series/Episodes were found in XBMC's library...
07/30/2011 09:16:29 em INFO    ContentScan      No music videos were found in XBMC's library.
07/30/2011 09:16:29 em INFO    ContentScan      Found 11724 unique files in video library (11724 movies, 0 episodes, 0 music videos)
07/30/2011 09:16:29 em NOTICE  ContentScan      11724 videos found in library. (Previous count was 11724, a change of 0)
07/30/2011 09:16:29 em NOTICE  ContentScan      Since no more videos were added in the last 2.0 minutes, continuing on...