xbmc.org website recovery
#1
Is there any chance of recovering last month's messages through an index such as google cache?

I know it's a stretch, but maybe? the forum was being indexed by google several times a day, and everything was text based. . .
Reply
#2
For those that are looking to recover data in the lost threads, most of it should still be cached on google/bing/yahoo if you act fast. There's even a few tools out there that are made for recovering data via this method (see warrick). Is anyone from the xbmc team already working this angle? Perhaps people could just highlight specific threads they are after and we can work together to get them from cache before they disappear. Just trying to be helpful....
Reply
#3
I've posted and subscribed to quite a few threads over the last month that i'd like back.
Is team-XBMC looking at this, or just letting the data disappear?

These are just a few to start out with. I've got a whole list of now-broken links in my browser history.
http://forum.xbmc.org/showthread.php?p=526213
http://forum.xbmc.org/showthread.php?p=522197
http://forum.xbmc.org/showthread.php?p=526213

I'm guessing you could probably find the ID# where it cut off and recover anything greater than that from the cache?
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#4
You can't just cram text into the db. There may be extreme ways of salvaging posts from google cache, but nothing I'm willing to do. Sorry guys, 2 sleepless nights is enough for me.

The info will be back in no time anyway...

TheUni
Reply
#5
It's not that easy as just scraping off googles cache.
First you should make a script that does just that, but then there are still a lot of problems.

You can't just put the data in the database right away. If you post a reply or thread, a lot of things change (post count, cache, ...) and to maintain integrity of the system, you need to use the API.
However, for example posts posted by users that are now also gone (users registererd within the last month) will give problems, because guests are not allowed to post. And there might be a lot of these kind of problems.

Edit: theuni was faster
Reply
#6
I would say not much can realistically be done now about lost data, but I am willing to help if xbmc want to set up offsite database replication. It is a good way of not only handling backups (backing up from slave database rather than primary, and having no "delay" whilst non innodb tables are locked, and also having a copy which is always at most a few seconds/minutes behind the main database.
Reply
#7
I just think everyone should be grateful we didn't go with option 1 which was start the forum from scratch and decided to go with option 2 (use the backup before the site move)
Reply
#8
theuni Wrote:...Sorry guys, 2 sleepless nights is enough for me.

The info will be back in no time anyway...
Thank you for your time and help! I was just curious if XMBC had any plans. Like you said, it is not a clean and easy thing to do.

Jezz_X Wrote:I just think everyone should be grateful we didn't go with option 1 which was start the forum from scratch and decided to go with option 2 (use the backup before the site move)
Very grateful! Starting from scratch would have lost us a lot of knowledge, information and time.
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
#9
I appreciate sleepless nights as well (I've been there). Also, i wasn't advocating reimporting into the database, rather, an html capture and attachment to a new thread.
Reply
#10
For the few odd posts (such as guides which has post 1 updated) then google cache will probably help track down the post contents, such as this linux mavemkv one:

Luckily, you can use teh cache: keywork in a google search to bring back the cached page only:

cache:http://forum.xbmc.org/showthread.php?tid=67420

At least this allows some threads to get the first post back on track - but not for wholesale db restoration...
Reply
#11
I don't think that is that useful. If you find some important post of you is missing, you can grab the cache info and repost.
That will do and will not put into trouble the administrators.
Reply
#12
Hi,
I administrate a similarly sized forum (less visitors but over half a million posts) and I run full backup every day and keep a few days worth of binary log (hourly rotated to avoid corruption). Very simple to setup and very effective, the daily snapshot takes a couple of minutes and nobody notices anything (the forum continues to work as is without any issues, the database guarantees data integrity for the exact moment where the dump has started)
With these backups and almost a week's worth of binary logs I can rollback to any point in time in the past week and can easily recover after a major crash.
Additionally - daily dumps are compressed and copied to an off site storage server so even if my server burns and turns to dust, in my worst case scenario I have lost a day's worth of data (I don't replicate the binary logs).

If you need help setting up a similar mechanism - just drop me a note.
Reply
#13
gibsonlp Wrote:Hi,
I administrate a similarly sized forum (less visitors but over half a million posts) and I run full backup every day and keep a few days worth of binary log (hourly rotated to avoid corruption). Very simple to setup and very effective, the daily snapshot takes a couple of minutes and nobody notices anything (the forum continues to work as is without any issues, the database guarantees data integrity for the exact moment where the dump has started)
With these backups and almost a week's worth of binary logs I can rollback to any point in time in the past week and can easily recover after a major crash.
Additionally - daily dumps are compressed and copied to an off site storage server so even if my server burns and turns to dust, in my worst case scenario I have lost a day's worth of data (I don't replicate the binary logs).

If you need help setting up a similar mechanism - just drop me a note.

Great Idea !
I assumed xbmc.org did a backup a few times per day, at least !
It's hard to imagine how much data was lost in 1 month.
ASrock Ion 330HT - XBMCFreak 10.1 Lucid LiveCD. Everything works but System sounds over HDMI.
Reply
#14
I would be very interested in how to backup mysql regularly. I have read about scripting mysqldump but not really implemented it outside a test site.

I administer a popular forum and backup once every 2 weeks manually at the moment using phpmyadmin.

I've also seen phpmybackup, but not used it myself yet.
Reply
#15
Now I understand why i could not log in... and why needed to register again!
Sad
Reply

Logout Mark Read Team Forum Stats Members Help
xbmc.org website recovery0