Linux Using pathsubstitution to migrate from SMB to NFS causes duplicate video entries
#1
So I have XBMCbuntu 11.0 running on my HTPC ("htpc"), with my library residing in a MySQL database on separate Xubuntu box ("shodan"), and my videos and thumbnails residing on a separate Windows box ("jarvis").

I had my HTPC configured so that it was using SMB sources for videos. For example:
Code:
sources.xml excerpt:

        <source>
            <name>Movies</name>
            <path pathversion="1">smb://jarvis/Shared/Video/Movies/</path>
        </source>

...and I was also storing thumbnails on the SMB share via path substitution:
Code:
advancedsettings.xml excerpt:

    <pathsubstitution>
      <substitute>
        <from>special://masterprofile/Thumbnails/</from>
        <to>smb://jarvis/Shared/Video/Thumbnails/</to>
      </substitute>
    </pathsubstitution>

However, I set up an NFS server on the Windows box and wanted to try having the HTPC use that instead of SMB (NFS over UDP is supposed to work better with powerline ethernet). Therefore, I set up the HTPC to mount the NFS shares via /etc/fstab (and verified that it worked), then I set up a path substitution rule:
Code:
advancedsettings.xml excerpt:

  <pathsubstitution>
    <substitute>
      <from>special://masterprofile/Thumbnails/</from>
      <to>/mnt/jarvis/Shared/Video/Thumbnails/</to>
    </substitute>
    <substitute>
      <from>smb://jarvis/Shared/</from>
      <to>/mnt/jarvis/Shared/</to>
    </substitute>
  </pathsubstitution>

After restarting XBMC, it is now happily adding duplicate entries to my library. Doh!

Both the old and new entries play fine. Anyone have any idea what's going on here?

What is the best strategy for getting things set up so that multiple XBMC installations can access the same videos via different protocols and/or mount paths, while still sharing the same database?
Reply
#2
Is it truly ADDing duplicates? On a brand new item added, post migration, are their two? Or do you just have duplicates to preexisting entries?
Just cause you added a alternative path reference to the same actual storage it sounds like both technically are still accessible?
So if you scan the video sources into the library it will find the new ones, xbmc puts the paths in the dbase too, so if you clean them and it can still find them they won't get removed.
Search first, provide details and keep forums clean. Mark things solved, to close them out and acknowledge helpful volunteers who share. If I have helped, click the plus button.
Reply
#3
Yes, it was showing two entries for each video. Both of them would play. XBMC wasn't giving me the option of seeing an information screen for any of the duplicates, but looking in the MySQL database showed that there were entries under both the old SMB path and separate entries under the new mount path.

After restoring my old advancedsettings.xml and rebooting (at which point the videos still worked because the mount paths were valid, so cleaning didn't work), I was able to delete the offending entries from the database to restore it to the state prior to my experiment-gone-awry. In retrospect, the best thing to have done to revert the problem would have been to unmount the NFS shares and then do a library clean.

Does this mean it's not possible to use path substitution in conjunction with a shared database? It seems that XBMC is not smart enough to keep itself from adding duplicate entries for videos using substitute paths.

The only other thing I can think of is that maybe my substitution rule was too high-level, and maybe it would work if I made a rule that corresponded to each of my sources in sources.xml?


Edit: What I'm really getting at is that it's frustrating that it doesn't seem possible to have a shared database with different clients accessing the actual video files via multiple different networking protocols.
Reply
#4
Path subs alone cannot create duplicates, as it only takes a source and points it to a different location. There would still only be one source.

You only get duplicates if you have more than one source that is pointing to the same files. In other words, don't use pathsubs -and- add the new path as a source.
Reply
#5
I've learned that lesson that hard way in the past, but unfortunately I don't see how that's the case here:
  • I've got only one XBMC client.
  • All videos in the database are stored using smb:// paths.
  • The client is using all smb:// sources (per the example above), and
  • I did not touch the mediasources.xml nor sources.xml files at all.

Here are the actual userdata XML files on the client:
advancedsettings.xml: http://pastebin.com/fT2wmvVy (note that the x at the end of the paste is not in the file, and that I currently have the the pathsubstitution rules for the OS mount paths to the NFS shares commented out)
mediasources.xml: http://pastebin.com/J20rf1rP (I'm not even sure what this is for, versus sources.xml)
sources.xml: http://pastebin.com/57WnYPdh

Edit: It seems that it would be a logical progression for XBMC to store (1) source numbers or aliases along with (2) relative paths in the database for each file, instead of storing absolute paths (which in some cases bind to a specific networking protocol). Then, each client would be free to define different paths to each source number or alias, and all clients would know which files are being referenced via different means.
Reply
#6
I am having a similar problem.
I copied all my TV files from
/media/SAMSUNG/TV/ to
/media/Expansion/Seagate/TV/
I added
<pathsubstitution>
<substitute>
<from>/media/SAMSUNG/Public/TV/</from>
<to>/media/Expansion/Seagate/TV/</to> <!-- Note the difference between the usage of forward and backslashes -->
</substitute>
</pathsubstitution>
to advancedsettings.xml
I hoped to do this to not upset the existing library setup, or have to rescan it all. (hence resetting all the watched statuses etc)
I haven't added the new HDD to the sources and it isn't listed in the sources.xml
But everytime sickbeard adds a new episode and initiates a scan, the library readds every episode of that program, hence doubling up all the episodes (and screwing up my recently added episodes)
Any suggestions on what I should look for? And how I can fix it?
Reply
#7
Yep, same here. A substitution seems to Effectively create an additional media source, hence a library update Proceeds to add everything under that substitution As if it were a new folder altogether. Damn shame.
Reply
#8
Are you guys using MySQL database server libraries, or standard ones? I'm going to submit a ticket.
Reply
#9
Posted a suggestion here: http://forum.xbmc.org/showthread.php?tid=140596
Opened a bug report ticket here: http://trac.xbmc.org/ticket/13488
Reply
#10
My bug report was shot down Sad
Reply

Logout Mark Read Team Forum Stats Members Help
Using pathsubstitution to migrate from SMB to NFS causes duplicate video entries0