Maintain separate files backup continuously
#1
Lightbulb 
Hello

I have a system with a central NAS where all movies etc. are mantained and where several instances of XBMC (3 XBMC-Linux, 1 XBMC-Windows 7 and a couple old xbox) go to get its media. I maintain also a MySQL database where the databases for all but the Xbox are stored, and there are several databases because each instance of XBMC uses several profiles (shared between machines, like "children movies&series", "movies&series in spanish", "subtitled movies&series in original languaje", "documentaries, recipes, etc"). One and only one instance (the windows 7 one) is used to import new data to the database manually several times a week selecting each one of the profiles. All the non-xbox xbmc with all profiles share a "thumbnails" directory in the server.

It mostly works and it is proof of how far has gone and what a high level of sophistication xbmc has reached.

The problem is, sometimes the database gets corrupted somehow, I do not really know why, and some entries get duplicated (or triplicated or more), or I need to move files around or a few other problems that forces to re-create part or everything in the database for a profile. It is not such a big deal because I maintain nfo, thumb and fanart files in place along the media files themselves (created manually, or with Media Companion or with XBMC "export library to separate files" function)

The problem is, the information that is in the database but not on the nfo files is lost, for example, date of incorporation to the database (that allows that useful feature, "recent movies", "recent episodes"), watched status, actual codec and resolution...

It would be great to keep that information up to date, that way almost any problem that I have now could be solved with maybe moving directories, library clean and library update, without affecting anything else.

I suppose that, if I have this problem, devs which are constantly tinkering with the database code would find it now and then too.

The thing would work as, if an option is set, when importing media a nfo/tbn/fanart will be created if it doesn't already exist (as it is created with "export to separate files") in place, and among other things it will add the date of scraping also (so if it is imported later, it will not appear in "recent movies"). Then, every time the files is used and some info updated in the XBMC database, it will go to the nfo file also, and so if the database needs to be recreated, no info will be lost. if a disk becomes full and files are moved to another location, a database cleaning and a database update later, everything would be in perfect order without interfering with the users whatsoever.

Maybe it is too much work or it is too complicated, but if some dev find the idea interesting and could implement it, I think it would be great for many people.
Reply
#2
1) Setup a cron to export info
2) Setup a cron to backup the .db files.
3) Use MySQL and dump it daily.
Code:
GRANT ALL PRIVILEGES ON `xbmc_%`.* TO 'xbmc'@'%';
IF you have a mysql problem, find one of the 4 dozen threads already open.
Reply
#3
darkscout Wrote:1) Setup a cron to export info
2) Setup a cron to backup the .db files.
3) Use MySQL and dump it daily.

1) How? it must start the computer, start XBMC, and in XBMC do the export... and it must do it overwriting the nfo files (thumbs or fanart is not a problem usually), and then turn computer back off. And if somebody is using xbmc, it should not disrupt the experience while doing everything. And doing it manually is a real PIA since the database is huge (several thousand episodes and movies) and it takes a long time. Frankly, I do not think that is a solution

2) I use mysql, so no .db files

3) The problem is, it is the database what gets corrupted (yesterday happened again, I can't find why it happens or the reason it affects each time only a few TV shows, and the entire show gets scanned again and duplicated in the database. Maybe it is a network glitch or something). So if I do it by a cron job I may export the corrupted database. The nfo files themselves seem to be much more robust and never get corrupted.

When you export the database to nfo files, if a nfo file already exists, is it overwritten? that could be a solution to the moving files around problem... Exporting the whole database is a long process, but I do not move lots of files around too often, maybe twice a year or so, so it is indeed a good solution.

But the date of original incorporation to the database will be lost too...

I find the "recent episodes" and "recent movies" very useful, a pity it is not used more, like having the possibility of sorting everywhere by the date of addition, and also keeping that info when importing/exporting nfo files...
Reply
#4
1) So... You want XBMC to add a backup function that doesn't work when xbmc is off, but still maintains backups.

And that's what 'cron' is, it's not manual.

3) So it maintains backups of the DB,but you don't want to rely on the database itself because that could get corrupted, but the backups that XBMC is supposed to maintain isn't going to also get corrupted, right? Because that makes perfect sense.

If you want to backup mysql, put it in cron, time stamp it, restore from last good.
Code:
/usr/bin/mysqldump --add-drop-table --user=xbmc --pass=xbmc --databases xbmc_video xbmc_music | /usr/bin/7za a -si $(date +%d%h%y-%H:%M).sql.7za

Each backup is around 800k on my system.
Code:
GRANT ALL PRIVILEGES ON `xbmc_%`.* TO 'xbmc'@'%';
IF you have a mysql problem, find one of the 4 dozen threads already open.
Reply

Logout Mark Read Team Forum Stats Members Help
Maintain separate files backup continuously0