• 1
  • 2
  • 3(current)
  • 4
  • 5
  • 131
[Release] Backup (formerly XBMC Backup)
#31
Love the concept.. I'm always backing up, yet never current.. weeks old, this would seem to take the drudgery outo of this task... but alas as others have noted.

here's a snip of the critical area... the pastebin was a bit too big to post.

11:26:53 T:3600 INFO: -->Python script returned the following error<--
Error Type: <type 'exceptions.UnicodeEncodeError'>
11:26:53 T:3600 ERROR: Error Contents: 'ascii' codec can't encode character u'\xf7' in position 50: ordinal not in range(128)
11:26:53 T:3600 ERROR: Traceback (most recent call last):
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 188, in <module>
11:26:53 T:3600 ERROR: backup.run()
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 119, in run
self.syncFiles()
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 132, in syncFiles
self.fileManager.createFileList(self.Addon)
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 26, in createFileList
self.walkTree(self.walk_path + "addons/")
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 65, in walkTree
self.addFile(filePath)
File "C:\Users\PatK\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 70, in addFile
self.fHandle.write(str(filename) + "\n")
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf7' in position 50: ordinal not in range(128)
11:26:53 T:3600 INFO: -->End of Python script error report<--

I look forward with baited breath fr an update Smile

Reply
#32
love idea - got mq3 and script error.
Reply
#33
quick update, i've looked over a lot of stuff tonight and think i have a solution. each of the errors i'm seeing are a result of ascii encoding errors when writing the "restore.txt" file that is used to write the files between destinations. I've removed this entirely and now pass the filenames around internally via a filename list instead. I'm going to test over the next day using different combinations (including MQ) but shoulld release to the repo soon.

Everyone please hang tight, hopefully this will fix things up.
Reply
#34
any updates?

Matt
Main - LG - OLED65CX6LA 65" Smart 4K Ultra HD HDR OLED TV - Yamaha RXV383 - N2plus running CoreELEC Nexus - Harmony Elite remote
BedRoom - Panasonic DX-700 UHD TV - N2 running CoreELEC Nexus - Harmony Elite remote
Server - debian based 10TB Media Server 
Reply
#35
I pulled the latest git today and backup is running with MQ3 skin enabled, so far so good.
But then i looked at what exactly is being backuped and i still miss a few things form userdata folder.

Folders not backuped are:
- keymaps
- peripheral_data (not that important i guess)

EDIT:
I checked code and noticed that those 2 folders aren't defined so maybe that's why it's not backuped.

In OpenELEC there is also a config folder outside userdata folder , is there a way i can add this?

I also noticed it's doing a full update every time, maybe doing incremental updates is better?
Reply
#36
(2012-05-23, 10:37)schumi2004 Wrote: I pulled the latest git today and backup is running with MQ3 skin enabled, so far so good.
But then i looked at what exactly is being backuped and i still miss a few things form userdata folder.

Folders not backuped are:
- keymaps
- peripheral_data (not that important i guess)

EDIT:
I checked code and noticed that those 2 folders aren't defined so maybe that's why it's not backuped.

In OpenELEC there is also a config folder outside userdata folder , is there a way i can add this?

I also noticed it's doing a full update every time, maybe doing incremental updates is better?

Glad that it is working. I should probably add the keymaps folder at the least as some people may be making custom keymap entries there. As far as OpenELEC support I could add an extra toggle for OpenELEC specific files in the settings. Are there any other folders under the user profile that OpenELEC creates that may be important?

As far as the incremental updates - I totally agree that would be the best. The caveat is that there is no reliable way to get the timestamps for files from remote systems. The XBMC JSON RPC does not return them, and the built in python modules only do local files/folders very well. If I was to restrict the addon to something like only SMB shares I may be able to find a smb python library that would get the timestamps, but I'd like to use the built in xbmc protocols where possible and the functionality for that just isn't there yet. As such it can still take quite a while if you have multiple skins or lots of thumbnails/fanart to copy everything over.

I'm holding off submitting this to the addons repo just yet, I have found an issue with the way the JSON Files.GetDirectory command returns filenames for zip files that garbles them. On github I have added a catch to filter these files out, but am waiting back from some info on if this can be fixed before submitting.

Reply
#37
I'll check OpenELEC for you.

If incremental is hard to accomplish maybe add time stamps to the backup folders?
Just a thought Wink
Reply
#38
(2012-05-23, 22:59)schumi2004 Wrote: I'll check OpenELEC for you.

If incremental is hard to accomplish maybe add time stamps to the backup folders?
Just a thought Wink

Thanks, I'll add the OpenELEC stuff before submitting to the repo, shouldn't be too hard. I got an answer on the zip file paths so that is sorted out.

What do you mean by add time stamps to the backup folders? I'd be happy to consider something but the different ways I've looked at it all require that I can gather some type of timestamp information from the backup location once the files have been sent, which I can't do. Once the files are in the backup local all I can get from them are name and full file path.

The one thing I can think of that might work would be to create a list of files and timestamps that is created locally when the backup job runs, and then saved to the addon_data folder (as a txt or xml file). Upon the next run the timestamps in the file could be used to compare the local file against what it was the last time the backup ran. The restore side would just restore everything, as I think it should. Is that more like what you were thinking? It would require the creation of a map data structure to compare against (which would eat up some processing time) but the trade off would be not transferring large files that may not have changed, which would be worth the extra overhead I think.
Reply
#39
I tried latest git on my OpenELEC system and it fails with the following error.


23:49:20 T:140627724441344 NOTICE: Thread Background Loader start, auto delete: false
23:49:20 T:140627724441344 NOTICE: Previous line repeats 1 times.
23:49:20 T:140627724441344 WARNING: CreateLoader - Unsupported protocol(addons) in addons://more/executable.tbn
23:49:25 T:140627724441344 NOTICE: Thread XBPyThread start, auto delete: false
23:49:25 T:140627724441344 NOTICE: -->Python Interpreter Initialized<--
23:49:25 T:140627724441344 ERROR: Error Type: <type 'exceptions.IndentationError'>
23:49:25 T:140627724441344 ERROR: Error Contents: ('unindent does not match any outer indentation level', ('/storage/.xbmc/addons/script.xbmcbackup/default.py', 36, 64, ' self.walkTree(self.walk_path + "userdata/Database")\n'))
23:49:25 T:140627724441344 ERROR: IndentationError: ('unindent does not match any outer indentation level', ('/storage/.xbmc/addons/script.xbmcbackup/default.py', 36, 64, ' self.walkTree(self.walk_path + "userdata/Database")\n'))

Don't understand what is going wrong.
Going to check it out.

EDIT:
My bad, mixed up spaces and tabs.

EDIT2:
root / # find / -name autostart.sh
/storage/.config/autostart.sh
root / # find / -name Lircmap.xml
/storage/.xbmc/userdata/Lircmap.xml
/storage/backup/backup/xbmc_backup/userdata/Lircmap.xml
/usr/share/xbmc/system/Lircmap.xml
root / #

This is the location of OE specific files.
/storage/.config
*.xml and *.sh

For some reason i keep getting errors, will check out tomorrow.


(2012-05-23, 23:10)robweber Wrote:
(2012-05-23, 22:59)schumi2004 Wrote: I'll check OpenELEC for you.

If incremental is hard to accomplish maybe add time stamps to the backup folders?
Just a thought Wink

Thanks, I'll add the OpenELEC stuff before submitting to the repo, shouldn't be too hard. I got an answer on the zip file paths so that is sorted out.

What do you mean by add time stamps to the backup folders? I'd be happy to consider something but the different ways I've looked at it all require that I can gather some type of timestamp information from the backup location once the files have been sent, which I can't do. Once the files are in the backup local all I can get from them are name and full file path.

The one thing I can think of that might work would be to create a list of files and timestamps that is created locally when the backup job runs, and then saved to the addon_data folder (as a txt or xml file). Upon the next run the timestamps in the file could be used to compare the local file against what it was the last time the backup ran. The restore side would just restore everything, as I think it should. Is that more like what you were thinking? It would require the creation of a map data structure to compare against (which would eat up some processing time) but the trade off would be not transferring large files that may not have changed, which would be worth the extra overhead I think.
And what if you checked file-size and date before overwriting? No need to keep track of files in a separate file.
Again just a thought.
Reply
#40
Ver .8 auto-update report some mixed results. Default settings script dies...(same kind of errors reported previously) exclude user addons and add-on data, and the script seems to work but with some extras... which don't seem to be part of the back-up, but did appear. http://pastebin.com/f9qc4YYA keep in mind these are folders... not files and within the folders are empty... with the exception of the 'userdata' folder which contains... the usual xml files and folders, the thumbnails are populated, and the database, all the rest are empty.. playlists included.

In my case, I'd just prefer a global back-up of the roaming directory, (at least for now) without the selective functions.

Eden 11, T! win7/64
Reply
#41
not working looking for files to backup at about 10 min in it causes XBMC to crash and returns to apple screen
Is there a FIX please advise

Thanks

Here is the log
11:36:19 T:165150720 NOTICE: script.xbmcbackup: Starting
11:36:19 T:165150720 NOTICE: script.xbmcbackup: Local Dir: /var/mobile/Library/Preferences/XBMC/
11:36:19 T:165150720 NOTICE: script.xbmcbackup: Remote Dir: smb://MEDIACENTER-HP/Users/Media Center/Documents/xbmc/
11:36:19 T:165150720 NOTICE: script.xbmcbackup: Remote Path exists - may have old files in it!
11:36:46 T:149860352 NOTICE: weather.wunderground - satellite image downloaded
11:36:47 T:149860352 NOTICE: weather.wunderground - available locations: 1
11:36:47 T:149860352 NOTICE: weather.wunderground - finished
11:37:53 T:110497792 NOTICE: Samba is idle. Closing the remaining connections
11:46:57 T:165150720 NOTICE: userdata/addon_data/
11:46:57 T:165150720 NOTICE: userdata/Database/
11:46:57 T:165150720 NOTICE: userdata/keymaps/
11:46:57 T:165150720 NOTICE: userdata/peripheral_data/
11:46:57 T:165150720 NOTICE: userdata/playlists/
11:46:57 T:165150720 NOTICE: userdata/plugin_data/
11:46:57 T:165150720 NOTICE: userdata/Thumbnails/
11:46:57 T:165150720 NOTICE: userdata/advancedsettings.xml
11:46:57 T:165150720 NOTICE: userdata/favourites.xml
11:46:57 T:165150720 NOTICE: userdata/guisettings.xml
11:46:57 T:165150720 NOTICE: userdata/LCD.xml
11:46:57 T:165150720 NOTICE: userdata/passwords.xml
11:46:57 T:165150720 NOTICE: userdata/profiles.xml
11:46:57 T:165150720 NOTICE: userdata/RssFeeds.xml
11:46:57 T:165150720 NOTICE: userdata/sources.xml
11:46:57 T:165150720 NOTICE: userdata/upnpserver.xml
Reply
#42
hi mate

dont know what youve done, but just updated via git hub ands it working perfect.

Thanks

Matt
Main - LG - OLED65CX6LA 65" Smart 4K Ultra HD HDR OLED TV - Yamaha RXV383 - N2plus running CoreELEC Nexus - Harmony Elite remote
BedRoom - Panasonic DX-700 UHD TV - N2 running CoreELEC Nexus - Harmony Elite remote
Server - debian based 10TB Media Server 
Reply
#43
yay its totally working now Smile
Reply
#44
is there a way to automate this?

ie, if i leave my computer on on a sunday night at 2am it will back it up to my server? say weekly?

Matt
Main - LG - OLED65CX6LA 65" Smart 4K Ultra HD HDR OLED TV - Yamaha RXV383 - N2plus running CoreELEC Nexus - Harmony Elite remote
BedRoom - Panasonic DX-700 UHD TV - N2 running CoreELEC Nexus - Harmony Elite remote
Server - debian based 10TB Media Server 
Reply
#45
Thanks for the feedback everyone, nice to know the most current version (0.0.8) is working better for most people. Anyone still having problems please make sure you are using the latest version (check XBMC repo if not set to auto update). Also, it may be worth deleting your old backup directory and starting over to avoid any issues.

@PatK - the debug log you posted is not anything this addon has produced. Not sure if you meant to post this. XBMC logs where there is actually some kind of python error are most useful, I don't see any errors here.

@dallasnights - I don't have an appltTV so I can't say for certain that this addon works on it, although it uses default XBMC libraries so there is no reason it shouldn't (anyone else have any comments here?). The section of the log you posted does not have any errors in it, it looks like it built the file list just fine, could there be a network connectivity problem writing to your backup directory? Without a log that actually includes an error of some type I don't really know what to advise.

@matt77303 - If you want to automate stuff check out this addon: https://github.com/robweber/cronxbmc As you'll see in the disclaimer it is not 100% finished, but it has instructions on how to edit the cron.xml needed to schedule XBMC scripts to run as needed.
Reply
  • 1
  • 2
  • 3(current)
  • 4
  • 5
  • 131

Logout Mark Read Team Forum Stats Members Help
[Release] Backup (formerly XBMC Backup)10