Kodi Community Forum
[Release] Backup (formerly XBMC Backup) - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=151)
+---- Thread: [Release] Backup (formerly XBMC Backup) (/showthread.php?tid=129499)



RE: [Release] XBMC Backup - kromix - 2013-02-09

Awesome Script!


RE: [Release] XBMC Backup - drexel75 - 2013-02-13

(2012-05-14, 23:36)robweber Wrote:
(2012-05-14, 01:18)lifenoobie Wrote: @robweber: I have the same problem as e2zippo. Was having problems with sound after adding addons. Luckily I backed up xbmc prior to the additions. Although I was getting the script error message described above. After fruitless attempts, I uninstalled xbmc, and reinstalled. I added "xbmc backup" and I'm not able to restore my previous settings. I'm getting the same script error. I tried debugging, but I don't actually see where the "xbmc.log" is stored. Below is the log file. Any ideas what I'm doing wrong? Thanks for the help!

20:24:29 T:5068 NOTICE: script.xbmcbackup: Starting
20:24:29 T:5068 NOTICE: script.xbmcbackup: Local Dir: C:\Users\media center\AppData\Roaming\XBMC\
20:24:29 T:5068 NOTICE: script.xbmcbackup: Remote Dir: smb://FREENAS/media_drive/downloads/xbmc_backup/xbmc_backup/
20:24:29 T:5068 INFO: Loading skin file: DialogProgress.xml
20:24:29 T:5068 DEBUG: DialogProgress::StartModal called
20:24:29 T:5068 DEBUG: ------ Window Init (DialogProgress.xml) ------
20:24:29 T:5068 DEBUG: JSONRPC: Incoming request: { "jsonrpc" : "2.0" , "method" : "Files.GetDirectory" , "params" : { "directory" : "smb://FREENAS/media_drive/downloads/xbmc_backup/xbmc_backup/addons/" , "sort" : { "method" : "file" } } , "id" : 1 }
20:24:29 T:5068 DEBUG: JSONRPC: Calling files.getdirectory
20:24:29 T:5068 DEBUG: CUtil::GetMatchingSource: no matching source found for [smb://FREENAS/media_drive/downloads/xbmc_backup/xbmc_backup/addons/]
20:24:29 T:5068 DEBUG: Previous line repeats 4 times.
20:24:29 T:5068 ERROR: XFILE::CDirectory::GetDirectory - Error getting smb://FREENAS/media_drive/downloads/xbmc_backup/xbmc_backup/addons/
20:24:29 T:5068 INFO: -->Python script returned the following error<--
20:24:29 T:5068 ERROR: Error Type: <type 'exceptions.KeyError'>
20:24:29 T:5068 ERROR: Error Contents: 'result'
20:24:29 T:5068 ERROR: Traceback (most recent call last):
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 188, in <module>
backup.run()
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 122, in run
self.restoreFiles()
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 140, in restoreFiles
self.fileManager.createFileList(self.Addon)
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 26, in createFileList
self.walkTree(self.walk_path + "addons/")
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\default.py", line 57, in walkTree
for (path, dirs, files) in vfs.walk(directory):
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\resources\lib\vfs.py", line 50, in walk
for x in listdir(current_path, extra_metadata=True):
File "C:\Users\media center\AppData\Roaming\XBMC\addons\script.xbmcbackup\resources\lib\vfs.py", line 100, in listdir
if jsonobject['result']['files']:
KeyError: 'result'
20:24:29 T:5068 INFO: -->End of Python script error report<--
20:24:29 T:5068 INFO: Python script stopped

Image

If you're doing a restore, my guess would be there is an error in your remote path settings. According to the log your path is this - smb://FREENAS/media_drive/downloads/xbmc_backup/xbmc_backup/.

My guess is that originally you specified this: smb://FREENAS/media_drive/downloads/ and then specified that the xbmc_backup folder name be given to the name of the folder you want created for backups (these are two different settings). When you re-installed xbmc backup you then specified the entire folder - including the xbmc_backup sub folder - and expected it to find your backup data. Since another subfolder called xbmc_backup does not exist within this directory you get a "file not found" error.

Try modifying your settings to read smb://FREENAS/media_drive/downloads/ as the remote directory and xbmc_backup as the folder name to see if you get any better results.


Dude the part where you explained the path settings just helped me out tremendously. i was a folder down lol


RE: [Release] XBMC Backup - u2pop - 2013-02-15

i get error trying to backup on dropbox some fix?


RE: [Release] XBMC Backup - rojazx - 2013-02-15

OPENELEC is giving me trouble with the plugin when enabling scheduler.

A backup is made every time after bootup regardless of the scheduling settings. Any ideas of a fix?


Re: RE: [Release] XBMC Backup - robweber - 2013-02-16

(2013-02-15, 09:11)u2pop Wrote: i get error trying to backup on dropbox some fix?

What version of the addon are you using? Also, a debug log of the error would be cool.

(2013-02-15, 22:03)rojazx Wrote: OPENELEC is giving me trouble with the plugin when enabling scheduler.

A backup is made every time after bootup regardless of the scheduling settings. Any ideas of a fix?

Please answer the same questions as above. Debug logs are the key!


RE: [Release] XBMC Backup - rojazx - 2013-02-16

Solved it.

The problem was that for some reason the raspberry was having trouble connecting to the internet. Since the Rasp does not have a RTClock when xbmc starts it has a date of 1970 and the backup script was scheduling for the next month. After it connected to the internet and gets real date from ntpdate the backup detects the change and starts the backup because that date has passed.

It was a problem with the network settings. It was taking a while for OPENELEC to decide to connect using wired connection. I had previously configured it to use wireless so when i switched back and settings pointed to wireless it gets tripped up for a while.


RE: [Release] XBMC Backup - u2pop - 2013-02-16

[quote u2pop wrote:
]i get error trying to backup on dropbox some fix?[/quote]


Quote:robweber wrote:
What version of the addon are you using? Also, a debug log of the error would be cool.



i am usng the 0.3.5 the one i get on your github and here is the debug log thank you.

http://xbmclogs.com/show.php?id=35852


RE: [Release] XBMC Backup - robweber - 2013-02-16

(2013-02-16, 03:15)u2pop Wrote: i am usng the 0.3.5 the one i get on your github and here is the debug log thank you.

http://xbmclogs.com/show.php?id=35852

Assuming you pulled from the master branch that version of the addon should work. I know there are some issues with the testing branch at the moment.

Did you follow the dropbox instructions? It is very important that the Dropbox authorize URL connect your account with xbmc before any data can be sent. The debug log you sent didn't have any info in it. It just had part of the startup sequence for xbmc in it. It didn't even load all the addons from what I cuold tell. The xbmclogs addon loaded and that was the end of the log file.


RE: [Release] XBMC Backup - u2pop - 2013-02-18

(2013-02-16, 17:04)robweber Wrote:
(2013-02-16, 03:15)u2pop Wrote: i am usng the 0.3.5 the one i get on your github and here is the debug log thank you.

http://xbmclogs.com/show.php?id=35852

Assuming you pulled from the master branch that version of the addon should work. I know there are some issues with the testing branch at the moment.

Did you follow the dropbox instructions? It is very important that the Dropbox authorize URL connect your account with xbmc before any data can be sent. The debug log you sent didn't have any info in it. It just had part of the startup sequence for xbmc in it. It didn't even load all the addons from what I cuold tell. The xbmclogs addon loaded and that was the end of the log file.


I create the dropbox api but i dont know if they authorize the URL because when i creat it they give me that info..


Code:
App name    XBMC-u2pop
App status     Development (Apply for production status)
App key    xz88ec74c35i1fv
App secret    4rem9wdtihhlvd9
Access type    Full Dropbox



RE: [Release] XBMC Backup - robweber - 2013-02-18

With what you've posted it looks like you've done the first part of the Dropbox requirement. After that you need to do the following:

Code:
Once you have your app key and secret add them to the settings (in the addon). XBMC Backup now needs to have permission to access your Dropbox account. When you see the prompt regarding the Dropbox URL Authorization DO NOT click OK. Check your XBMC log file for a line from "script.xbmcbackup" containing the authorization URL. Cut/paste this into a browser and click Allow. Once this is done you can click "OK" in XBMC and proceed as normal. XBMC Backup will cache the authorization code so you only have to do this once, or if you revoke the Dropbox permissions.

To summarize - enter the app key and secret into the addon settings and then try to run XBMC Backup. Once you click 'Backup' you should get prompted to check your log file for the dropbox authorize url in your xbmc log. It is a cumbersome process but the best that can be done with how Dropbox's API works.


RE: [Release] XBMC Backup - karel1 - 2013-02-24

Rob, thanks for this add-on. I experienced a serious problem though. Maybe because I did not understand the options?
Your program made a backup and then started to delete all my media. When I noticed about 100GB was already gone.

I used version 0.3.3. with XBMC 12 frodo on OpenElec 2.99.3 on a Raspberry Pi (who says we live in a complex world !?).

I made a subdir on my media drive for the backup and set 'backups to keep' to '02'. Maybe that was wrong?
Backup ran and later after restarting the Pi I saw that a subdir with todays date as name was made containing backupped data.

Although I am not a programmer maybe your program does delete directories and contents that it did not create and 'owned'?

I am not angry or mad or anything, just wanted you to know about this 'user-experience'.


RE: [Release] XBMC Backup - robweber - 2013-02-25

(2013-02-24, 22:39)karel1 Wrote: Rob, thanks for this add-on. I experienced a serious problem though. Maybe because I did not understand the options?
Your program made a backup and then started to delete all my media. When I noticed about 100GB was already gone.

I used version 0.3.3. with XBMC 12 frodo on OpenElec 2.99.3 on a Raspberry Pi (who says we live in a complex world !?).

I made a subdir on my media drive for the backup and set 'backups to keep' to '02'. Maybe that was wrong?
Backup ran and later after restarting the Pi I saw that a subdir with todays date as name was made containing backupped data.

Although I am not a programmer maybe your program does delete directories and contents that it did not create and 'owned'?

I am not angry or mad or anything, just wanted you to know about this 'user-experience'.

I've seen this issue (someone posted on github a while back) when trying to put backups in the same directory as your media files. If you've created a separate folder, and set the location correctly, there shouldn't be any reason your other files are getting deleted. The logic for the backup rotation goes something like this:
  • list all folders in the backup directory, sort by name
  • delete all folders until the number we want are all thats left

So yes, it is totally deleting anything else in those directories. It is very important that your paths be set correctly when using this feature. Should the backup rotation be checking for some sort of validation that it created those files - yea probably. I'll be honest, my time to tackle these issues has been pretty scarce lately. I hope to be back in development mode within the next week or two.

I'd be curious to see a log file for this since you say it is writing to a specific backups only directory. It should not be going up any higher than the "root" level you set as the backups folder.


RE: [Release] XBMC Backup - karel1 - 2013-02-25

Well, I was trying to write to a specific backups only directory. I made a mistake there, because the directory created by the add-on (with as name the date it ran) was a subdir of the root directory, and the directory I created for the backup was empty. So it did not go up any higher, but because of my mistake it did not start low enough.
So, yes, that was a mistake on my part. But, users are not always as smart as you expect them to be. My point is that you as the designer and programmer now is informed of the possible consequences. So please change the program so that it only is allowed to delete directories and contents that were created by it. In that case it would never have deleted my media.

Thanks for your quick reply. Case closed for me.


RE: [Release] XBMC Backup - iccaros - 2013-02-26

(2013-02-25, 20:41)karel1 Wrote: Well, I was trying to write to a specific backups only directory. I made a mistake there, because the directory created by the add-on (with as name the date it ran) was a subdir of the root directory, and the directory I created for the backup was empty. So it did not go up any higher, but because of my mistake it did not start low enough.
So, yes, that was a mistake on my part. But, users are not always as smart as you expect them to be. My point is that you as the designer and programmer now is informed of the possible consequences. So please change the program so that it only is allowed to delete directories and contents that were created by it. In that case it would never have deleted my media.

Thanks for your quick reply. Case closed for me.

I second this, I lost 2 tb of movies, not warning, this program should never ever delete data, it is not an expected product of a Backup program, and if it does it should only delete what it creates only and should ask first... ask first.. just to repeat... this gives the user the ability to quit.... you should always assume the user will do bad things, not to pick but this is programing 101 of quality control and computer security.

Looking through your logic for log rotation I see the fault, you count folders, a bad ideal as you have no way of knowing what the folder names are or what is in them, there is no warning to the user. The correct logic would have error checking, where you create folders with a specific name like XBMC backup and create subfolders with a date under that. This way you can search for folders with a given name and date older than what ever is needed and remove those only and pop up a warning showing what is being deleted, Let them unchecked if they do not want to see the warning. This way it is simple to find the backup as I have not found one yet, and two it would not accidentally delete anyones data. My data loss is costing over $100 as that is what the forensics software cost to restore all the movie files, + my time of renaming each raw file, and I charge $150 to manage software teams.

So in short, the logic of counting directories is flawed, it allows data to be deleted that should not be.
The options are not explain well what they do or do not work, like making new folders under Frodo on Mac OSX.
No one would expect a backup program to delete their data, unexpected outcome means people will place this anywhere, in fact the instructions say anywhere you can write.. it does not have a warning or disclaimer, so it would not be expected to test in a safe place, or to pay attention to your settings. as the worst expected outcome is it deletes your last backup of settings.

I was going to create a folder but the tool was grayed out so I picked the root of my media drive, one ironically had its backup mirror die and I have not gotten around to replace. Will now..


Hope this can help make a better project.


RE: [Release] XBMC Backup - robweber - 2013-02-26

(2013-02-26, 06:21)iccaros Wrote: I second this, I lost 2 tb of movies, not warning, this program should never ever delete data, it is not an expected product of a Backup program, and if it does it should only delete what it creates only and should ask first... ask first.. just to repeat... this gives the user the ability to quit.... you should always assume the user will do bad things, not to pick but this is programing 101 of quality control and computer security.

Looking through your logic for log rotation I see the fault, you count folders, a bad ideal as you have no way of knowing what the folder names are or what is in them, there is no warning to the user. The correct logic would have error checking, where you create folders with a specific name like XBMC backup and create subfolders with a date under that. This way you can search for folders with a given name and date older than what ever is needed and remove those only and pop up a warning showing what is being deleted, Let them unchecked if they do not want to see the warning. This way it is simple to find the backup as I have not found one yet, and two it would not accidentally delete anyones data. My data loss is costing over $100 as that is what the forensics software cost to restore all the movie files, + my time of renaming each raw file, and I charge $150 to manage software teams.

So in short, the logic of counting directories is flawed, it allows data to be deleted that should not be.
The options are not explain well what they do or do not work, like making new folders under Frodo on Mac OSX.
No one would expect a backup program to delete their data, unexpected outcome means people will place this anywhere, in fact the instructions say anywhere you can write.. it does not have a warning or disclaimer, so it would not be expected to test in a safe place, or to pay attention to your settings. as the worst expected outcome is it deletes your last backup of settings.

I was going to create a folder but the tool was grayed out so I picked the root of my media drive, one ironically had its backup mirror die and I have not gotten around to replace. Will now..


Hope this can help make a better project.

Setting the location of your backups to be in a separate directory from all the other media on your storage device is "backups 101". So I guess in a way you are correct in thinking it was my mistake for not assuming that users would do something like mix their backup data with all their other stuff. While I can sympathize with anyone's loss of data I didn't hold a gun to your head forcing you to start using the backup rotation feature. If you notice the default is "0", otherwise known as "don't delete anything". Asking the user is not a valid fix in this case as the idea is that this process should be automated and not require input from the user. You enabling the backup rotation was the confirmation that this action is OK. Also, as an FYI - all folders being deleted are presented on screen via the progress bar interface. You also shouldn't have to go looking for backup files as the restore feature will pull these in and list them for you.

Don't take my tone as unwillingness to change anything. I just think it's a little rude to list dollar amounts (including what you think you time is worth) when reviewing a free piece of software that someone has created in their free time and donated to the community in an effort to contribute. Since you've read the code I'll take this opportunity to quote a phrase often used on this forum - patches are always welcome......

You do have several good points, such as creating a separate backup directory as part of the program, that could be potential fixes for this issue. I'll take these into consideration when I look into the issue. The tricky part is going to be doing something in a backwards compatible way so all the backups people have made up until now are not rendered useless when attempting to restore. As I've stated my ability to do anything about this is not due to lack of understanding it's seriousness - just lack of time to do anything about it. Within the next few weeks I hope to carve out some time for addon development again and attend to these items.