[robweber]

For a lot of different reasons (testing, new system, computer crashes) it is a good idea to backup your Kodi data. I've spent enough time copying my userdata folder around that I decided that being able to just click a button and send it on it's way was worth the time coding an addon for it.

The Backup Addon

This is a real quick way to export your Kodi data to another folder or network share for backup purposes. Scheduling, compressing, and having a max number of archived backups are also configurable options. This allows you to use the addon as a completely hands off backup solution. The types of files you can backup from Kodi include:
 

• User Addons - Addon's you've downloaded
  
• Addon Data - Settings associated with your addons
  
• Database - The SQLite Databases
  
• Playlists - Your playlist folder
  
• Thumbnails/Fanart - downloaded images
  
• Config Files - sources.xml, guisettings.xml, advancedsettings.xml, etc
  
• Other non-Kodi directories on your local system
  

Since this addon is using the xbmcvfs python module to actually do the file copy you should be able to backup to any location read/writeable on your system. You can also write directly to a Dropbox or Google Drive target without needing the client on your HTPC. Right now you can download the addon from the main Add-on repo or the github page for this project. Pretty much everything you need can be found in the Wiki listed below. Please note that the version in the Kodi repo for Leia is very outdated due to the Python 3 migration. Please use the links below for a more current Python 2 version. The Python 3 version is live in the Matrix repo already so download that from the Addon Browser. 

Matrix (Python 3) Version - https://github.com/robweber/xbmcbackup
Krypton/Leia Version (Python 2) - https://github.com/robweber/xbmcbackup/tree/krypton
Legacy Version (current in Kodi repo) -  https://github.com/robweber/xbmcbackup/r...rvis-1.1.3
Code: https://github.com/robweber/xbmcbackup/
Wiki: https://github.com/robweber/xbmcbackup/wiki 
FAQ: https://github.com/robweber/xbmcbackup/wiki/FAQ

[avp387]

Good effort. This is definitely needed. At this point though, on a linux system I find the following method to work well for me:

http://www.htpcbeginner.com/automatic-ba...ntu-linux/

It is automatic and came in handy for me at least once when I had to restore all my xbmc settings.

[robweber]

Thanks for the link. As with a lot of things there are numerous ways to get the job done. I guess with this addon I'm aiming at cross-platform, or people on "out of the box" systems like XBMCbuntu that may not want to muck around with the command line. As far as automation goes, I don't think that is something I'll put specifically into this addon, but you could schedule it with something like Cron XBMC (again, keep it all within the media center). Once the restore mode is finished I think this will really bring things full circle. I think it would be great to install XBMC, download this one addon, then pull a restore to get back everything I had previously.

[paddycarey]

you can walk non-local paths (or local ones for that matter) using the json api instead of os.walk. It's pretty nice and has a few added benefits like being able to walk inside compressed directories (the compressed backup file for instance). I use it in my mmabrowser plugin to walk a defined share, whether local or remote like so:

Code:

def getDirList(path):
    dirList = []
    currentLevelDirList = [path]
    while True:
        prevLevelDirList = []
        if len(currentLevelDirList) > 0:
            for dirName in currentLevelDirList:
                prevLevelDirList.append(dirName)
                dirList.append(dirName)
            currentLevelDirList = []
        else:
            break
        for dirName in prevLevelDirList:
            log('Checking for directories in: %s' % dirName)
            json_response = xbmc.executeJSONRPC('{ "jsonrpc" : "2.0" , "method" : "Files.GetDirectory" , "params" : { "directory" : "%s" , "sort" : { "method" : "file" } } , "id" : 1 }' % dirName.encode('utf-8').replace('\\', '\\\\'))
            jsonobject = simplejson.loads(json_response)
            if jsonobject['result']['files']:
                for item in jsonobject['result']['files']:
                    if item['filetype'] == 'directory':
                        currentLevelDirList.append(item['file'])
    return dirList

def getFileList(path):
    fileList = []
    dirList = getDirList(path)
    for dirName in dirList:
        log('Checking for files in: %s' % dirName)
        json_response = xbmc.executeJSONRPC('{ "jsonrpc" : "2.0" , "method" : "Files.GetDirectory" , "params" : { "directory" : "%s" , "sort" : { "method" : "file" } , "media" : "video" } , "id" : 1 }' % dirName.encode('utf-8').replace('\\', '\\\\'))
        jsonobject = simplejson.loads(json_response)
        if jsonobject['result']['files']:
            for item in jsonobject['result']['files']:
                if item['filetype'] == 'file':
                    fileList.append(item['file'])
                    log('Found video: %s' % item['file'])
    return fileList

Might help with some of the caveats above.

Paddy

[robweber]

That is awesome, thanks for the code. I had seen the Files.GetDirectory method in the JSON API but never thought of using it that way. My solution for the time being was to write a restore.txt file during the backup that included all files and directories, and then read that file back in later for the restore. I didn't really like this as a permanent solution though, I like your cross platform way of walking through the directory tree much better.

[dallasnights]

Great work I am using it on ATV2 to my media share on a Windows 7 pc
Would love to see if you can add only backup updates since last backup

Thanks

[Bobby Blixberg]

Great job.

[paddycarey]

I took a quick stab at writing something to solve the walk problem and i came up with this https://github.com/paddycarey/script.mod...kit/vfs.py

it's still rough round the edges but i'll fix it up as best i can

if you just drop it in your addon for now (until i get the module shipshape), do a:

Code:

import vfs as xbmcvfs

and you should now have all the functionality of xbmcvfs plus a walk and listdir method which should work on local or remote shares

[robweber]

I took a quick stab at writing something to solve the walk problem and i came up with this https://github.com/paddycarey/script.mod...kit/vfs.py

it's still rough round the edges but i'll fix it up as best i can

if you just drop it in your addon for now (until i get the module shipshape), do a:

Code:

import vfs as xbmcvfs

and you should now have all the functionality of xbmcvfs plus a walk and listdir method which should work on local or remote shares

That is really great - thank you for taking this on! Am I understanding the comparepathlists() function correctly? The way it looks to me is that by giving it two paths, say C:\xbmc\userdata\ and nfs:\\192.168.1.102\foo\xbmc\userdata it will return the common directory of "xbmc\userdata". Is that right?

I was working on the code snippet you gave me but this is much better than the one or two functions I would have done. I like the idea of making it a comprehensive module for extending xbmcvfs. I should get around to integrating this with the backup addon within a day or two. I'm watching your repo so as you improve it I can keep up.

[paddycarey]

comparepathlists isn't quite finished yet, it's intended to be a function to use to compare the contents of two directories. so one list of files that exist in both lists, one list of files that exists only in the first list but not the second list and vice versa.

The results could then be used to perform a sort of one way sync, copying any new files, skipping any common files and removing any deleted files.

Not sure of any way to get a timestamp from a remote file though so you can't be sure which file is newer.

[robweber]

comparepathlists isn't quite finished yet, it's intended to be a function to use to compare the contents of two directories. so one list of files that exist in both lists, one list of files that exists only in the first list but not the second list and vice versa.

The results could then be used to perform a sort of one way sync, copying any new files, skipping any common files and removing any deleted files.

Not sure of any way to get a timestamp from a remote file though so you can't be sure which file is newer.

I think you're right, I saw in another thread once that someone was using the JSON-RPC functions for file operations and they were having the same issue. Might be something to add to XBMC's JSON functionality so it can be exposed for stuff like this.

[e2zippo]

This looks great, thanks a lot!

[klunde]

Just downloaded and testet it - it didn't work

Code:

12:02:56 T:10624  NOTICE: -->Python Interpreter Initialized<--
12:02:56 T:10624  NOTICE: script.xbmcbackup: Starting
12:02:56 T:10624  NOTICE: script.xbmcbackup: Local Dir: C:\Users\moluo\AppData\Roaming\XBMC\
12:02:56 T:10624  NOTICE: script.xbmcbackup: Remote Dir: C:\Data\Backup\xbmc_backup/
12:02:56 T:10624  NOTICE: script.xbmcbackup: Remote Path exists - may have old files in it!
12:02:56 T:10624   ERROR: Error Type: <type 'exceptions.UnicodeEncodeError'>
12:02:56 T:10624   ERROR: Error Contents: 'ascii' codec can't encode character u'\xc2' in position 59: ordinal not in range(128)
12:02:56 T:10624   ERROR: Traceback (most recent call last):
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 188, in <module>
                                                backup.run()
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 119, in run
                                                self.syncFiles()
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 132, in syncFiles
                                                self.fileManager.createFileList(self.Addon)
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 26, in createFileList
                                                self.walkTree(self.walk_path + "addons/")
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 65, in walkTree
                                                self.addFile(filePath)
                                              File "C:\Users\moluo\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 70, in addFile
                                                self.fHandle.write(str(filename) + "\n")
                                            UnicodeEncodeError: 'ascii' codec can't encode character u'\xc2' in position 59: ordinal not in range(128)

[robweber]

From the error it looks like there was a character in one of you filenames that couldn't be encoded to ASCII (for writing the copy list). I'm curious as to what the filename was but it doesn't look like we're going to get that info from this log file. If you could add the following line after line 69 in the default.py file it should print out the filename to the log right before it calls the function that seems to cause the error. Make sure it is on the same indent line as the next line and it should run. Please post your results back here.

Code:

xbmc.log(str(filename))

[Bond13579]

I'll post the logs with that line of code since I'm having the same issue. Looks like it doesn't like the dash, '-'.


14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Republic.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Rhino.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Roadrunner.png
14:58:42 T:3808 NOTICE: addons\skin.aeonmq.3\media\flags\music\recordlabel\Roc-A-Fella.png
14:58:42 T:3808 ERROR: Error Type: <type 'exceptions.UnicodeEncodeError'>
14:58:42 T:3808 ERROR: Error Contents: 'ascii' codec can't encode character u'\u201d' in position 58: ordinal not in range(128)
14:58:42 T:3808 ERROR: Traceback (most recent call last):
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 189, in <module>
backup.run()
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 120, in run
self.syncFiles()
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 133, in syncFiles
self.fileManager.createFileList(self.Addon)
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 26, in createFileList
self.walkTree(self.walk_path + "addons/")
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 65, in walkTree
self.addFile(filePath)
File "C:\Users\Parnell\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 68, in addFile
xbmc.log(str(filename))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 58: ordinal not in range(128)