[Release] Backup (formally XBMC Backup) - Printable Version
+- Kodi Community Forum (http://forum.kodi.tv)
+-- Forum: Support (/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (/forumdisplay.php?fid=151)
+---- Thread: [Release] Backup (formally XBMC Backup) (/showthread.php?tid=129499)
[Release] Backup (formally XBMC Backup) - robweber - 2012-04-20 21:21
For a lot of different reasons (testing, new xbmc system, computer crashes) it is a good idea to backup your XBMC data. I've spent enough time copying my userdata folder around that I decided that being able to just click a button and send it on it's way was worth the time coding an addon for it.
The Backup Addon
Please note that with Kodi 14 (Helix) this addon is now called "Backup" in the addon repository.
This is a real quick way to export your XBMC data to another folder or network share for backup purposes. Scheduling, compressing,and having a max number of archived backups are also configurable options. This allows you to use the addon as a completely hands off backup solution. The types of files you can backup from XBMC include:
Since this addon is using the xbmcvfs python module to actually do the file copy you should be able to backup to any location read/writeable by XBMC. You can also write directly to a Dropbox target without needing the Dropbox client on your HTPC. Right now you can download the addon from the main XBMC Add-on repo or the github page for this project. Please post any problems and suggestions here.
In XBMC: Settings->Add-ons->Get Add-ons->Programs
During the restore process there are a few checks and post-run procedures to know about.
The first is a version check. If you are restoring to a different version of XBMC than the one used to create the backup archive you'll get a warning. In most cases it is OK to proceed, just know that some specific items like addons and database files may not work correctly.
The next check is for an advancedsettings.xml file. If you've created this file and it exists in your restore archive you'll be asked to reboot XBMC. This is so that the file can be loaded and used for any special settings, mainly path substitutions, you may have had that would affect the rest of the restore. XBMC Backup will prompt you to continue the restore process when you reboot the program.
The last bit of post-processing is done after all the backup files have been restored. If you have restored your configuration files the addon will attempt to restore any system specific settings that it can from the guisettings.xml file. This is done by comparing the restored file with settings via the JSONPRC Settings.SetSettingValue method. Only system specific settings can be restored so you will get any custom views or skin specific settings back. See the FAQ for how to restore these.
I can't see any restore points when choosing "Restore", what is the problem?
If you've created restore points with an older version of the addon (pre 0.3.6) you may see this issue. New versions of the addon look for a file called xbmcbackup.val to validate that a folder is a valid restore archive. Your older restore folders may not have this file. All you need to do is create a blank text file and rename it to xbmcbackup.val. Then put this file inside the archive directory. Your restore points should show up after selecting "Restore" in the addon again.
Several settings aren't being restored, this includes views, weather, etc. How do I get these back?
GUISETTINGS.xml is a configuration file used heavily by XBMC for remembering GUI specific settings. Due to the fact that XBMC reads this file on startup, and writes from memory to this file on shutdown; it is not possible to restore this file while XBMC is running. You must manually move this file from your backup archives if you wish to restore it. User SouthMark has posted the following steps for restoring in the OpenELEC system where this is more difficult:
1. Run the restore of your backup
2. SSH using putty to the IP Address of your media centre username: root Password openelec
3. Type "systemctl stop xbmc.service" - Your media center machine should now go blank
5. Connect to your machine using WinSCP and copy the guisettings.xml file to the userdata folder (this is the guisettings.xml file from your backup), alternatively you can copy this file directly to an SMB share and use putty to move it to the right spot.
6. go back to your putty window and type "systemctl start xbmc.service"
Why is the Addon prompting me to restart XBMC to continue?
If you have an advancedsettings file in your restore folder the addon will ask you if you want to restore this file and restart xbmc to continue. This is because the advancedsettings file may contain path substitution information that you want to be loaded when doing the rest of your restore. By restoring this file and restarting xbmc it will be loaded and the rest of your files will go where they are supposed to. If you know your file does not contain any path substitutions you can select "no" and continue as normal.
I've re-installed a newer version of XBMC from scratch and tried to restore my data from an older version. Why isn't it working?
It is working, just not how you are expecting. When re-installing xbmc it is usually easier to install the version you had and do an in-place upgrade by installing the newer version of the top. XBMC will take care of upgrading your content, addons, and databases itself after running it for the first time. By doing a backup of your current, wiping your data, and installing a newer version you're not allowing these processes to take place. What ends up happening is that your old databases and other files get restored, xbmc just doesn't care to use them anymore. In the case of addons this could really mess things up by restoring now non-working addons from an older version of xbmc.
If you want to get your databases back after doing this there is something you can do. XBMC will perform a check on startup for the most current database version. If one is not found it will look for an older one and upgrade it. So you can do your restore process, then quit xbmc. Find the database files under userdata/Databases and delete the ones with the highest DB number. Then start xbmc again. It should take your old database and upgrade it to the most current version - creating a new database file in the process. Please note this only works for the SQLite database.
Compressing my backups isn't working, why?
The most common reason for this is going to be drive space. Non-compressed backups write files directly from your local folders to the remote directory. When compressing the archive the files are first staged locally and then only the compressed folder copied over the remote directory. This means for both backup and restore operations you need to have enough space on your local drive for creation/extraction of the compressed archive. Depending on the folders you are selecting - especially for custom directories - this could be a lot of extra drive space, or very little.
RE: [Release] XBMC Backup - avp387 - 2012-04-22 15:27
Good effort. This is definitely needed. At this point though, on a linux system I find the following method to work well for me:
It is automatic and came in handy for me at least once when I had to restore all my xbmc settings.
RE: [Release] XBMC Backup - robweber - 2012-04-22 16:55
Thanks for the link. As with a lot of things there are numerous ways to get the job done. I guess with this addon I'm aiming at cross-platform, or people on "out of the box" systems like XBMCbuntu that may not want to muck around with the command line. As far as automation goes, I don't think that is something I'll put specifically into this addon, but you could schedule it with something like Cron XBMC (again, keep it all within the media center). Once the restore mode is finished I think this will really bring things full circle. I think it would be great to install XBMC, download this one addon, then pull a restore to get back everything I had previously.
RE: [Release] XBMC Backup - paddycarey - 2012-04-24 15:10
you can walk non-local paths (or local ones for that matter) using the json api instead of os.walk. It's pretty nice and has a few added benefits like being able to walk inside compressed directories (the compressed backup file for instance). I use it in my mmabrowser plugin to walk a defined share, whether local or remote like so:
Might help with some of the caveats above.
RE: [Release] XBMC Backup - robweber - 2012-04-24 15:52
That is awesome, thanks for the code. I had seen the Files.GetDirectory method in the JSON API but never thought of using it that way. My solution for the time being was to write a restore.txt file during the backup that included all files and directories, and then read that file back in later for the restore. I didn't really like this as a permanent solution though, I like your cross platform way of walking through the directory tree much better.
RE: [Release] XBMC Backup - dallasnights - 2012-04-24 18:42
Great work I am using it on ATV2 to my media share on a Windows 7 pc
Would love to see if you can add only backup updates since last backup
RE: [Release] XBMC Backup - Bobby Blixberg - 2012-04-24 21:21
RE: [Release] XBMC Backup - paddycarey - 2012-04-26 03:36
I took a quick stab at writing something to solve the walk problem and i came up with this https://github.com/paddycarey/script.module.devtoolkit/blob/master/lib/DevToolkit/vfs.py
it's still rough round the edges but i'll fix it up as best i can
if you just drop it in your addon for now (until i get the module shipshape), do a:
and you should now have all the functionality of xbmcvfs plus a walk and listdir method which should work on local or remote shares
RE: [Release] XBMC Backup - robweber - 2012-04-26 04:00
(2012-04-26 03:36)paddycarey Wrote: I took a quick stab at writing something to solve the walk problem and i came up with this https://github.com/paddycarey/script.module.devtoolkit/blob/master/lib/DevToolkit/vfs.py
That is really great - thank you for taking this on! Am I understanding the comparepathlists() function correctly? The way it looks to me is that by giving it two paths, say C:\xbmc\userdata\ and nfs:\\192.168.1.102\foo\xbmc\userdata it will return the common directory of "xbmc\userdata". Is that right?
I was working on the code snippet you gave me but this is much better than the one or two functions I would have done. I like the idea of making it a comprehensive module for extending xbmcvfs. I should get around to integrating this with the backup addon within a day or two. I'm watching your repo so as you improve it I can keep up.
RE: [Release] XBMC Backup - paddycarey - 2012-04-26 04:08
comparepathlists isn't quite finished yet, it's intended to be a function to use to compare the contents of two directories. so one list of files that exist in both lists, one list of files that exists only in the first list but not the second list and vice versa.
The results could then be used to perform a sort of one way sync, copying any new files, skipping any common files and removing any deleted files.
Not sure of any way to get a timestamp from a remote file though so you can't be sure which file is newer.
RE: [Release] XBMC Backup - robweber - 2012-04-28 00:33
(2012-04-26 04:08)paddycarey Wrote: comparepathlists isn't quite finished yet, it's intended to be a function to use to compare the contents of two directories. so one list of files that exist in both lists, one list of files that exists only in the first list but not the second list and vice versa.
I think you're right, I saw in another thread once that someone was using the JSON-RPC functions for file operations and they were having the same issue. Might be something to add to XBMC's JSON functionality so it can be exposed for stuff like this.
RE: [Release] XBMC Backup - e2zippo - 2012-05-03 11:19
This looks great, thanks a lot!
RE: [Release] XBMC Backup - klunde - 2012-05-03 12:07
Just downloaded and testet it - it didn't work
RE: [Release] XBMC Backup - robweber - 2012-05-03 21:42
From the error it looks like there was a character in one of you filenames that couldn't be encoded to ASCII (for writing the copy list). I'm curious as to what the filename was but it doesn't look like we're going to get that info from this log file. If you could add the following line after line 69 in the default.py file it should print out the filename to the log right before it calls the function that seems to cause the error. Make sure it is on the same indent line as the next line and it should run. Please post your results back here.
RE: [Release] XBMC Backup - Bond13579 - 2012-05-05 21:00
I'll post the logs with that line of code since I'm having the same issue. Looks like it doesn't like the dash, '-'.
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Republic.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Rhino.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Roadrunner.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Roc-A-Fella.png
14:58:42 T:3808 ERROR: Error Type: <type 'exceptions.UnicodeEncodeError'>
14:58:42 T:3808 ERROR: Error Contents: 'ascii' codec can't encode character u'\u201d' in position 58: ordinal not in range(128)
14:58:42 T:3808 ERROR: Traceback (most recent call last):
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 189, in <module>
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 120, in run
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 133, in syncFiles
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 26, in createFileList
self.walkTree(self.walk_path + "addons/")
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 65, in walkTree
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 68, in addFile
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 58: ordinal not in range(128)