moviebackdrops.com
#1
I have no problems hosting this site for the benefit of the XBMC community (as well as hosting and working on XBMC BlackMarket {blackmarket.ictcsc.net})

Both of these sites, including all of my private sites for work, etc have been suspended until +0100 GMT due to the fact that some users decided they would rather do a complete site rip and not just scrap for what it needed. I'm looking into ways to prevent this from happening again (as well as tracking down the users that did it this time).

If you use any of the sites that I host to help out the community, please do me a favor and don't abuse the site. The problems aren't because of bandwidth issues (I have plenty of that to spare) but it's because of high CPU load on a Server Cluster based enviroment. I wont take action against anyone yet, but after the site comes back online I may start to block IP addresses that abuse the servers.

Sorry for the inconvience.
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#2
Quick update since it wont let me edit.

My hosting company allows for a maximum of 400,000 hits a day and somehow I managed 432,178 with about 4 hours left of the day.

The part that I'm not fond of, is the ratio of page hits against unique visitors. There's a lot of people doing full and/or multiple site rips which is uncalled for. If you have a valid reason to request the entire set of images hosted on moviebackdrops.com you can contact me and I will have a compressed copy available to you usually in less then 24 hours.
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#3
Note that part of it is due to us not currently handling the thumb mechanism, so each time someone does a "Get Fanart" request, it downloads all the full images for that particular movie. A user would have to have a lot of time on their hands to use this to the point of abuse. We will, ofcourse, fix this as soon as we are able to - there's a trac ticket posted about it already.

Still, that is a manual action, so probably wouldn't happen all that often. The normal scanning process just grabs one.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#4
At least that explains some of the stats I was seeing, but there's multiple IP's that obviously went to the backdrop folder directly and used software to do a site rip Sad

I talked to my hosting company about if the domain could be moved to a dedicated server or somewhere where the CPU load wouldn't be as big a deal, but sadly they don't offer that ability with their cluster style setup. Sad

--------------------------------------------------------------
Future reference to all XBMC/MediaPortal/etc users that visit these forums and use moviebackdrops.com blackmarket.ictcsc.net or any other site I host if you notice that the page states "Account Suspended" it's due to the CPU usage and will be back online at +0100 GMT the following morning (unless this happens on a regular basis in which they get "really" mad at me.)

Thanks for your patience and understanding.
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#5
Smile 
Site is back up Smile
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#6
I feel your pain!! I run movie-xml.com and have the same problems even when I provide a perfectly good XML API to use Wink

Let me know if you find any good ways to prevent the mass scraping of the site.

It seems Mediaportal - XBMC and Meedio all have great plugins using our sites now.
Reply
#7
Sorry to hear some people can't properly appreciate great things they get for free.
I hope you got it covered for now Smile Thanks for your efforts Nod
Reply
#8
note that we have a diff for using the thumbs, it just havent been polished and applied just yet (to remedy our part of the blame)
Reply
#9
Quote:If you have a valid reason to request the entire set of images hosted on moviebackdrops.com you can contact me and I will have a compressed copy available to you usually in less then 24 hours.

Packs are good for setting backdrops up for skins like Aeon Focus and Mediastream.

Aeon puts packs up on radidshare (or some similar site) to make it easier for users to just drop in a folder without browsing a site picking out images they want (which is where I got mine).

This could be a reason to provide a rar of all the images on your site. Especially if it is dynamically created once a week or something Smile

Having said that, ripping your entire site is for sure a dick thing to do, and I'm not trying to defend them, I'm presenting a reason for providing packs.
Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
Reply
#10
Thanks Spiff, let me know when that gets done Smile

I talked with John over at MediaPortal about his plug-in and we found a way to have his cut the hit count in half so that helps as well.

One thing I wanted to point out for the users reading this, is that the problem isn't being caused ONLY by moviebackdrops.com, granted a lot of the hits are coming from this site. But the server hosting this site is also hosting XBMC BlackMarket as well as a few other personal and company sites I have. Sadly the hosting company I use can't get me detailed logs since it's on a cluster based system, but the logs I can manage to view show a majority of this is the new site (mbd).

If the suspensions continue, I'm going to have to temporarily disable MBD until I can come up with a solution to cut the CPU usage down since it's starting to hinder my other sites and customers.

As always, with this suspension, it's temporary and the sites should be back up around 9pm EST (+0100 GMT) until it gets suspended again anyway. Rolleyes
--------------------------------------------
MovieBackDrops - http://moviebackdrops.com
XBMC BlackMarket - http://blackmarket.ictcsc.net
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#11
How do you suggest corrections to the site?
This also a problem at moviebackdrops
http://postur.itn.is/cgi-bin/mmstdol.cgi...fforums%2f
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Reply
#12
@sho
Not sure the point behind that post unless I'm just in more need of sleep then I though. I'm the one that hosts moviebackdrops and that's the site I was talking about.

As far as the link, I've been talking a little with Travis (from Meligrove) about merging the sites (as Scott suggested). I've already created a full rar of all 1446 movies that MBD has backdrops for so he can start importing them to his site.

Moviebackdrops.com just got my account suspended again about 20 minutes ago, so I'm probably going to alter that site (which will break the scrapers sadly) so that when my account comes back up, the site will redirect to Meligrove.
HTPC 1: 64-bit Quad Core 3.4GHz Athlon X4 / 16GB DDR3 / (8) 2TB / 42" Vizio / Windows 8.1 (x64) / IIS 8.5
HTPC 2: 64-bit Quad Core 2.0GHz / 8GB DDR3 / 1TB / Windows 8.1 (x64) / IIS 8.5
My Personal Blog - NewsDemon | Text size doesn't seem to be working anymore for signatures?
Reply
#13
Point is there is no obvious way to make suggestions on the site.
But if you are merging, maybe there is no need.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Reply
#14
I threw up some info about the API for any developer interested in making some updates to their app.

http://www.meligrove.com/api_doc
Reply
#15
travis' site will be taking over everything, so there's no need to invest more time in the site I designed. I just need to get him the whole mess of images. Smile
Reply

Logout Mark Read Team Forum Stats Members Help
moviebackdrops.com0