+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Discussions (https://forum.kodi.tv/forumdisplay.php?fid=222)
+--- Forum: Feature Requests (https://forum.kodi.tv/forumdisplay.php?fid=9)
+--- Thread: Source 'abstraction layer' (/showthread.php?tid=81419)
Pages:
1
2
Source 'abstraction layer' - AnalogKid - 2010-09-18
It would be rather nice (and sensible) if the 'source' of media truly was an abstraction layer that XBMC adhered to.
Consider this:
My Movie Source (Logical) --> C:\My Movies (Physical)
When ALL data is scanned and saved to database, it should only ever refer to the LOGICAL source. This way, if people need to move their physical data to another drive or server, the logical source can just be remapped, without ruining the database or requiring a rescan.
The way it works at the moment, the logical source serves very little purpose, since the physical paths are littered all over the database :-(
Not sure if all that makes sense, but it does to me!
- AaronD - 2010-09-18
Amen!
Just bought a new 1TB drive today, and putting off moving content around because I know I will lose the "watched status" as I delete it from the library and add it from the new location!
- theuni - 2010-09-18
I'm beginning to agree with this as well. The best solution I've heard so far is to keep a hash of the media itself, rather than its path. This would be quite handy I think, but might be an ugly transition.
Edit: On second thought, doing a checksum on tons of remote files would probably hurt lots.
TheUni
- AnalogKid - 2010-09-18
AaronD Wrote:Amen!
Just bought a new 1TB drive today, and putting off moving content around because I know I will lose the "watched status" as I delete it from the library and add it from the new location!
I actually already implement my own abstraction layer outside of XBMC, but it would be nice for it to be built in.
The external solutions is this:
C:\MyVirtualMediaFolders\
C:\MyVirtualMediaFolders\Music <--- Symbolic link to any drive I like
C:\MyVirtualMediaFolders\Movies <--- Symbolic link to any drive I like
C:\MyVirtualMediaFolders\TV <--- Symbolic link to any drive I like
Within XBMC I map my sources to the above folders, this way I can move my media to anywhere I please, and XBMC still works without a rescan etc.
You need Vista, 7, Ubuntu or OSX for this... XP doesn't fully support symlinks.
- AaronD - 2010-09-18
How do you do a symlink on Windows 7? Can you share a symlink folder just like a physical folder, so I can do this on my backend server just the once instead of on each client?
- AnalogKid - 2010-09-18
theuni Wrote:I'm beginning to agree with this as well. The best solution I've heard so far is to keep a hash of the media itself, rather than its path. This would be quite handy I think, but might be an ugly transition.
Edit: On second thought, doing a checksum on tons of remote files would probably hurt lots.
TheUni
My current SymLink solution works perfectly, so I'm sure this could work well for users, although it would require a rebuild of the current database (OR it would need a small up to convert absolute paths to virtual sources, that runs once on update)...
So basically, XBMC would no longer ever treat DB paths as physical, just as virtual, and 'replace' them with physical 'on the fly'.
So something in the database like this:
'C:\Music\ACDC\Back In Black\01 Back In Black.mp3'
would be changed to
'Music 01\ACDC\Back In Black\01 Back In Black.mp3
XBMC would parse the first folder (Music 01) which would be the name of the 'source' and map it to 'C:\Music'.
There are smarter ways to do it, but in theory, this would be fairly simple. My worry is that there could be a lot of code in XBMC what directly accesses the database and gets path information rather than through one API which could do the transposition.
When I first got into XBMC years ago, this is how I thought the Sources actually worked... it seemed logical. But I soon discovered that 'sources' didn't do much abstraction at all!
- AnalogKid - 2010-09-18
AaronD Wrote:How do you do a symlink on Windows 7? Can you share a symlink folder just like a physical folder, so I can do this on my backend server just the once instead of on each client?
You need to create a the symlinks on each client...
Let me show you the batchfile I run prior to XBMC... don't worry too much, there's more stuff in there than you need.
I run XBMC in portablemode, but it really doesn't matter...
The important commands are the mklink ones.
e.g. mklink /D "X:\virtual" "C:\Physical" creates a symbolic folder called 'X:\Virtual' and it MAPS to 'C:\Physical'
I can change where it maps to ANY time I like.... but you must ensure your XBMC source is pointing to X:\Virtual. And that's all you have to do.
Code:
@Echo off
Rem ****** Map X: to this folder *********
subst X: /D
subst X: "%~dp0."
Rem ****** Create Virtual Media Folders ********
rd /S /Q ""x:\Virtual Media Folders"
mkdir "x:\Virtual Media Folders"
mklink /D "x:\Virtual Media Folders\Backdrops 01" "G:\Media Folders\Backdrops 01"
mklink /D "x:\Virtual Media Folders\Movies 01" "G:\Movie Pipeline\Stage 4 - Perfect Movies 01"
mklink /D "x:\Virtual Media Folders\Music 01" "D:\Data\Music"
mklink /D "x:\Virtual Media Folders\Music Videos 01" "D:\Data\Videos\Music Videos"
mklink /D "x:\Virtual Media Folders\TV Series 01" "F:\TV Series Pipeline\Stage 4 - Perfect TV Series 01"
Rem ****** Auto Update XBMC *******
"x:\XBMC Updater\XBMCUpdate.exe" /Update
Rem ****** Launch XBMC ********
start /B x:\XBMC\XBMC.exe -p- jmarshall - 2010-09-19
The easier thing is to losely connect the media info from the file full stop. "Abstracting" the sources just takes care of the case where you move ALL your media.
If you move just some of your media or rearrange things then it's still not picked up. Thus, hashing is the way to go here. A fast hash of the file can easily detect for file moves without too much overhead.
Cheers,
Jonathan
- AaronD - 2010-09-20
I couldn't agree more, but it is a good temporary solution. Is a "loosely connected" file on the cards?
- jmarshall - 2010-09-20
Yes it is on the cards.
The problem with using something like this as a temporary solution is you first have to get rid of the thumbnail caching using path of media.
- BlueCop - 2010-09-20
theuni Wrote:I'm beginning to agree with this as well. The best solution I've heard so far is to keep a hash of the media itself, rather than its path. This would be quite handy I think, but might be an ugly transition.
Edit: On second thought, doing a checksum on tons of remote files would probably hurt lots.
TheUni
One could use other data to calculate the unique hash. like filesize + filename. you don't have to hash the file data itself.
- jmarshall - 2010-09-20
I think hashing a portion of the file data (plus use file size) is probably the most reliable method - that way you pick up renames.
- hagayg - 2010-09-20
theuni Wrote:I'm beginning to agree with this as well. The best solution I've heard so far is to keep a hash of the media itself, rather than its path. This would be quite handy I think, but might be an ugly transition.
Edit: On second thought, doing a checksum on tons of remote files would probably hurt lots.
TheUni
you dont really need to checksum the files content , You can do combination of filename + extension + size (+ maybe date) it should create a unique enough key...
- theuni - 2010-09-20
hagayg Wrote:you dont really need to checksum the files content , You can do combination of filename + extension + size (+ maybe date) it should create a unique enough key...
As Jonathan said, using metadata like the filename or mtime defeats the purpose completely. The goal here would be to allow seamless moves/renames.
TheUni
- technojunkie - 2010-09-20
I realize this discussion has centered around abstracting file storage/relocation, but is any thought being given to abstracting other protocols for the purposes of the library? For example using uPnP or DAAP for music?
I'm not very familiar with UPnP, but under DAAP I'm pretty sure that no information about the media is being passed, simply lists of locations/URIs to the various files. It seems that it wouldn't be all that inappropriate to generate a hash of artist/album/song to attach that file to a local DB entry.
(in the case of UPnP, like I said I'm not at all familiar with it, but if it does provide metadata, it would be nice if we could optionally override that with the xbmc library.)