2011-02-24, 18:25
Introducing merge.sh version 0.1 [Linux only]
What this does:
This detects when you add any hard drive or other mass storage, or open a network share, and if it finds a directory named Music, Pictures or Videos, it overlays it onto the corresponding folder in your home directory.
It can do this for any number of drives at once. If you add, say, three drives and they all have a Videos folder, then ALL of the files in the Videos folders from ALL three drives will show up, at once, in the Videos folder in your Home folder.
If you then write to the Videos folder on the home folder, it will actually write the file to the drive with the most free space (or at least, the one that had the most free space the last time you added or removed a drive/share).
If you remove a drive or share, only the files on that drive/share are no longer there. All of the files from other drives/shares still show up. Removing a drive/share that you're not reading from or writing to should have no effect on the other drives/shares.
If you have sub-folders, such as TV Shows or Movies under Videos, these work too. If they exist on two or more drives/shares, the all of the files from both drives/shares show up under the one folder.
This is a simple script that gets a couple of programs to do all of the heavy lifting. It's not so clever by itself, and there's lots of room for improvement.
You'll need inotifywait (inotifytools package in all major distributions) and unionfs-fuse (package in all major distributions). For non-Linux, it might work if it's Linux-like enough (OSX, Solaris, BSD). Windows is right out, even with cygwin. You'll also need a fairly up-to-date box, but nothing special; I tested this mostly on Ubuntu 10.04.
Q Does this work for NTFS drives?
A This works for any filesystem that the OS can mount. It operates on the mount points and not the filesystems themselves, so if you can see the drive/share, it will work. So yes, if you have ntfs-3g or whatever.
Q How do I add another drive/share?
A Easy! Mount the drive/share (just by plugging it in, or navigating to the share, like with any other drive/share). If it has a Music, Pictures, or Videos folder, they will be merged. If not, you can just create those folders, safely remove the drive/share, and re-attach/re-navigate to them; now that they have those folders, they'll work.
Q I've lost all of my Music/Pictures/Videos that were in my home folder before the merge!
A They're not gone; they just don't show up when other folders are mounted up over the top of them. Remove all other drivers/shares and they will be visible again.
Really, this is intended to have a boot drive that you don't use except for the OS, and all of your storage on portable hard drives or network shares.
Q It didn't add my folder!
A The folder must be in the root of the drive/share; the script won't go hunting for it. But you could always mount the lower folder manually into /media/, and add it that way.
Q How can I run this every time I start up?
A In Debian/Ubuntu, go to the System menu, select Preferences, Startup Applications. Anywhere else, I'm not too sure, but it should be easy enough. I'll improve this answer as I test on more platforms.
Q I unmounted a drive or share, and it caused problems!
A This can happen, this is a pretty simple script.
Q What do I do if I want to make sure that a file I create/copy/move goes to the drive/share I want?
A There are a few ways:
Q It says it can't write, but I have lots of space!
A When you add or remove a drive/share, it remounts all of the available folders, and by default, any writes will go to whichever one had the most free space at this point. There are two exceptions, though:
Q CD-ROM/DVD-ROM, etc. drives don't get added.
A That is specially disabled at this time. It's not impossible, but for the problem I was personally trying to solve, it's not useful because you can't write to them, and they're slow, and hard drive storage is cheap.
Q I deleted/renamed a file, and it's still there!
A It probably existed on more than one drive/share. Delete it again; once it's gone from all drives/shares, it's gone altogether.
Q I created/copied a new version of a file, but I keep seeing the old one.
A Again, it probably exists on more than one drive/share. Delete/rename the old one, and then the new one should show up.
Q How can I manually unmount the merged folders?
A fusermount -u -z ~/Music ; fusermount -u -z ~/Pictures ; fusermount -u -z ~/Videos
Q It doesn't work.
A Could be for a lot of reasons, including:
Q What does this have to do with XBMC?
A Strictly speaking, nothing. I wrote it for a machine that was going to be used as a HTPC running XBMC, and am talking about it here because I thought others might find it useful too. But in principle, there's no reason it wouldn't work for any media center, or in fact any purpose at all.
Q How can I contact you?
A PMs through the XBMC forum, or posts in this thread. I'm not ready to put my name to it yet, so there will be no other contact details until I do.
POSSIBLE FUTURE ENHANCEMENTS
Download the script: [url="http://rapidshare.com/files/449649498/merge-1.0.zip]merge-1.0[/url]
What this does:
This detects when you add any hard drive or other mass storage, or open a network share, and if it finds a directory named Music, Pictures or Videos, it overlays it onto the corresponding folder in your home directory.
It can do this for any number of drives at once. If you add, say, three drives and they all have a Videos folder, then ALL of the files in the Videos folders from ALL three drives will show up, at once, in the Videos folder in your Home folder.
If you then write to the Videos folder on the home folder, it will actually write the file to the drive with the most free space (or at least, the one that had the most free space the last time you added or removed a drive/share).
If you remove a drive or share, only the files on that drive/share are no longer there. All of the files from other drives/shares still show up. Removing a drive/share that you're not reading from or writing to should have no effect on the other drives/shares.
If you have sub-folders, such as TV Shows or Movies under Videos, these work too. If they exist on two or more drives/shares, the all of the files from both drives/shares show up under the one folder.
This is a simple script that gets a couple of programs to do all of the heavy lifting. It's not so clever by itself, and there's lots of room for improvement.
You'll need inotifywait (inotifytools package in all major distributions) and unionfs-fuse (package in all major distributions). For non-Linux, it might work if it's Linux-like enough (OSX, Solaris, BSD). Windows is right out, even with cygwin. You'll also need a fairly up-to-date box, but nothing special; I tested this mostly on Ubuntu 10.04.
Q Does this work for NTFS drives?
A This works for any filesystem that the OS can mount. It operates on the mount points and not the filesystems themselves, so if you can see the drive/share, it will work. So yes, if you have ntfs-3g or whatever.
Q How do I add another drive/share?
A Easy! Mount the drive/share (just by plugging it in, or navigating to the share, like with any other drive/share). If it has a Music, Pictures, or Videos folder, they will be merged. If not, you can just create those folders, safely remove the drive/share, and re-attach/re-navigate to them; now that they have those folders, they'll work.
Q I've lost all of my Music/Pictures/Videos that were in my home folder before the merge!
A They're not gone; they just don't show up when other folders are mounted up over the top of them. Remove all other drivers/shares and they will be visible again.
Really, this is intended to have a boot drive that you don't use except for the OS, and all of your storage on portable hard drives or network shares.
Q It didn't add my folder!
A The folder must be in the root of the drive/share; the script won't go hunting for it. But you could always mount the lower folder manually into /media/, and add it that way.
Q How can I run this every time I start up?
A In Debian/Ubuntu, go to the System menu, select Preferences, Startup Applications. Anywhere else, I'm not too sure, but it should be easy enough. I'll improve this answer as I test on more platforms.
Q I unmounted a drive or share, and it caused problems!
A This can happen, this is a pretty simple script.
- It's best to cease all drive activity before you try to remove a drive or share. But (especially for reads/playback), it should actually keep working, as long as the data wasn't on the drive/share in question.
- If you remove a drive or share that you're currently reading from or writing to (because a file that you've opened happens to physically be stored there), then obviously that will cause problems.
- If you are writing multiple files to another drive, the re-configuration when you remove a drive might cause them to be written to the home folder, which of course is invisible once you run the script. You'll have to end the script, possibly remove the shares manually, and then copy those files to another drive/share/folder.
Q What do I do if I want to make sure that a file I create/copy/move goes to the drive/share I want?
A There are a few ways:
- Obviously, before you begin, you could remove all other drives/shares, so that there is only one.
- Alternately, you could just open the drive itself, rather than the Home folder. It's still mounted and can be used normally.
Q It says it can't write, but I have lots of space!
A When you add or remove a drive/share, it remounts all of the available folders, and by default, any writes will go to whichever one had the most free space at this point. There are two exceptions, though:
- If a sub-folder exists on only one drive, writes to that sub-folder will go only to that drive. For example, if you had Drive A which had Videos/TV Shows/, and Drive B which had Videos/Movies/, then even though both folders would show up in ~/Videos, writing to ~/Videos/TV Shows/ would always go to Drive A, and writing to ~/Videos/Movies/ would always go to Drive B. The simple solution is to go to the drive which does not have whichever folder, and create it there.
- At the moment, network shares mounted through the shell show up as having zero bytes free, so you can never write to them unless all drives/shares have zero bytes (and by coincidence they overlay earliest). This could perhaps be fixed; I'm working on it. The only workaround is to remove all other drives, or possibly to mount the share into /media/ manually using cifs.
Q CD-ROM/DVD-ROM, etc. drives don't get added.
A That is specially disabled at this time. It's not impossible, but for the problem I was personally trying to solve, it's not useful because you can't write to them, and they're slow, and hard drive storage is cheap.
Q I deleted/renamed a file, and it's still there!
A It probably existed on more than one drive/share. Delete it again; once it's gone from all drives/shares, it's gone altogether.
Q I created/copied a new version of a file, but I keep seeing the old one.
A Again, it probably exists on more than one drive/share. Delete/rename the old one, and then the new one should show up.
Q How can I manually unmount the merged folders?
A fusermount -u -z ~/Music ; fusermount -u -z ~/Pictures ; fusermount -u -z ~/Videos
Q It doesn't work.
A Could be for a lot of reasons, including:
- You have xbmc standalone, without the main OS shell. I haven't even thought about trying to make this work, and it's not likely to.
- You're running a user account that is not a member of the fuse group.
- All of your drives and shares are completely full, and you're trying to write more to them. Free some space (or add another drive)!
Q What does this have to do with XBMC?
A Strictly speaking, nothing. I wrote it for a machine that was going to be used as a HTPC running XBMC, and am talking about it here because I thought others might find it useful too. But in principle, there's no reason it wouldn't work for any media center, or in fact any purpose at all.
Q How can I contact you?
A PMs through the XBMC forum, or posts in this thread. I'm not ready to put my name to it yet, so there will be no other contact details until I do.
POSSIBLE FUTURE ENHANCEMENTS
- Use xdg-user-dirs so that it uses the local name for the folder depending upon the language (for example, for German instead of Music, Pictures, Videos you would have Musik, Bilder, Filme).
- Arguments to close or open shares explictly (rather than waiting for a device event), or end the script.
- Let users specify what folders they want this behaviour for.
- Let users specify what folders to mount, and where.
- Let users specify a different method of determining the order of folders (so that reads/writes happen for the media of their choice).
- etc.
Download the script: [url="http://rapidshare.com/files/449649498/merge-1.0.zip]merge-1.0[/url]