[bossanova808]

It's a big show but I'll do my best. Do printfs go to the log or do I have to start via console.

Wish me luck, disappearing down the rabbit hole....

[Memphiz]

1. oncrpc isn't needed anymore in 1.6
2. only the libnfs target has to be built in the project
3. the branch for our 1.6.3 is https://github.com/Memphiz/libnfs/commits/xbmcgotham
4. No clue what or how to printout in libnfs
5. Instead add CLog::Log(LOGDEBUG, "blah blah blah") in NFSFile.cpp/NFSDirectory.cpp (method stat). Printout any error codes the libnfs functions we call from there show including the path which was stat'ed.

Something like CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: stat returned %d for path %s", err, url.get().c_str()); (not sure about the url ... it might be a path string - then path.c_str() - what ever is given in the parameter to the stat call ...

[bossanova808]

Ok so grrr not seeing stat issues....lots of:

Code:

21:44:58 T:8000   DEBUG: NFS: Refreshing context for 192.168.1.51/MovieLibrary02, old: 56859, new: 56919
21:44:58 T:8000   DEBUG: NFS: Using cached context.
21:44:58 T:8000   DEBUG: NFSDEBUGPRINTOUT: stat returned 0 for path //Source Code (2011)/Source Code (2011) - bluray.mkv
21:44:58 T:8000   DEBUG: NFS: Refreshing context for 192.168.1.51/MovieLibrary08, old: 56891, new: 56951
21:44:58 T:8000   DEBUG: NFS: Using cached context.
21:44:58 T:8000   DEBUG: NFSDEBUGPRINTOUT: stat returned 0 for path //Soylent Green (1973)/Soylent Green.mkv
21:44:58 T:8000   DEBUG: NFS: Refreshing context for 192.168.1.51/MovieLibrary02, old: 56919, new: 56974
21:44:58 T:8000   DEBUG: NFS: Using cached context.

Then boom:

Code:

21:45:30 T:8000   DEBUG: Mysql execute: DELETE FROM files WHERE idFile IN (572,575,595,609,613,619,622,625,631,632,634,636,637,638,639,640,641,642,649,652,653,654,655,656,657,658,668,670,671,674,675,680,683,687,690,692,695,703,8775,705,706,707,714,719,721,723,727,728,729,735,736,743,745,746,750,752,756,772,773,774,777,778,781,782,783,796,802,803,809,811,813,820,822,824,828,834,838,840,849,850,858,8742,571,573,576,577,578,579,580,581,590,592,598,599,600,601,604,629,630,648,650,659,660,661,669,672,673,676,677,678,679,689,691,700,708,715,716,717,718,720,724,725,726,731,739,748,754,755,757,761,763,769,775,780,785,791...

(using latest libnfs with ?master xbmc. Didn't actually check out gotham but I think it's irrelevant here).

I'm not see any stats not returning 0.

Damn.

---

The fact that mysql thing is long (it's super long, goes on for page after page) - kinda implies an issue with each path, not with the root hare, doesn't it?

[Memphiz]

try the same in:

CNFSDirectory::Exists (NFSDirectory.cpp)

CNfsConnection:tat (NFSFile.cpp)

[bossanova808]

Yep already done that. 0s everywhere it seems.

@jmarshall - can you give me a basic run down on how the library cleaning works, in terms of sequence (nad how it builds that monster sql query to dump anything) - and where to look in xbmc code for it??

---

...because clearly something fails to get it to come to the conclusion to remove the paths (I get the 'what do you want to do with missing path xxxx' error on screen).

[patseguin]

@patseguin try what I posted it worked for me as I was n the same boat. Oddly you need to give everyone read access on the folder via smb share. As soon as I did that my windows PC's started to work. You still need nfs permissions for xbmc but also smb access.

I am using a Synology NAS with DSM 5.0 but it appears admin and guest users have read/write access for both NFS and SMB.

[jmarshall]

The CleanDatabase routine simply does:

CFile::Exists(path, false)

If that returns false it's a candidate for removal. We bunch them together and then test the parent paths via CDirectory::Exists(parent, false) and prompt if not found and not already prompted for. The false's just mean we don't use the directory cache.

I'd log in CNFSFile::Stat() and CNFSFile::Exists(). The former you want to log out the value of ret immediately after the nfs_stat() call, before it does anything else, as well as also prompting if it can't grab the connection in Stat() (where it returns -1).

Cheers,
Jonathan

[gibxxi]

@patseguin try what I posted it worked for me as I was n the same boat. Oddly you need to give everyone read access on the folder via smb share. As soon as I did that my windows PC's started to work. You still need nfs permissions for xbmc but also smb access.

I am using a Synology NAS with DSM 5.0 but it appears admin and guest users have read/write access for both NFS and SMB.

I don't know if it's the same idea on Synology devices, as I don't own one, but on the Netgear ReadyNAS series, NFS permissions are declared by IP address as a pose to user/password based access of SMB.

So for me I use the 192.168.0.0/24 designation which allows any client on the local network (192.168.0.1 -> 192.168.0.254) to have read/write access to shares exposed via NFS.

In fact there's 3 distinct fields on the ReadyNAS: Read-only hosts, Write-enabled hosts and Root privilege-enabled hosts for which individual IP's or ranges can be input. You may want to double check your NFS sharing permissions to see which hosts are allowed write/root access. AFAIK NFS doesn't support a username/password approach at all, unlike SMB.

You may also want to check the number of "NFS Threads" being used, especially if you have multiple clients accessing the NAS via NFS. I'm running with 4 threads on mine, but I can see a possible need to up that to 6 (2x threads per client machine). Too few threads will result in read/write actions potentially failing, too many will overwork your NAS so caution must be observed increasing this value.

[Memphiz]

AFAIK NFS doesn't support a username/password approach at all, unlike SMB.

While completly offtopic here imo i just want to tell you that NFS auth is not only based on ip. The IPs/subnets you are putting in /etc/exports only tell the server which clients are allowed to connect. Connecting is the first step but not the final one. After connecting the real read/write permissions are determined based on the uid (user id) of the connecting client. This userid is either remapped to an uid on the nfs server or the same uid has to exist on the server. Based on that uid and the normal linux/unix file system permissions nfs determines if a user can read or write or not even see anything.

So yes - no username/password, but no - its not only the granted ip range.

/offtopic.

[bossanova808]

@Memphiz @jmarshall

I am getting somewhere finally I think (time is the big issue for me, sorry!). CNFSFile::Stat can't seem to get the connection and is returning a bunch of -1s

Code:

bool CNFSFile::Exists(const CURL& url)
{
    CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Exists for url");
    return Stat(url,NULL) == 0;
}

int CNFSFile::Stat(struct __stat64* buffer)
{
  CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Stat for buffer");
  return Stat(m_url,buffer);
}


int CNFSFile::Stat(const CURL& url, struct __stat64* buffer)
{
  int ret = 0;
  CSingleLock lock(gNfsConnection);
  CStdString filename = "";
  
  if(!gNfsConnection.Connect(url,filename))
       CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Stat returning -1 as !gNFSConnection for url %s filename %s",url,filename);
    return -1;

gives:

Code:

21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat returning -1 as !gNFSConnection for url ?Z/
filename (null)
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat returning -1 as !gNFSConnection for url ?Z/
filename (null)
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat returning -1 as !gNFSConnection for url ?Z/
filename (null)
21:46:20 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url

I want to log the url but I am not sure how to get the string from

Code:

const CURL& url

these don't work:

url.get().c_str())

url.c_str()

---

There's also all of these:

Code:

21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists gNfsConnection.Connect failed for url ?Z/
path (null)
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists for path nfs://192.168.1.51/TVLibrary03/Band Of Brothers/
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists gNfsConnection.Connect failed for url ?Z/
path (null)
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists for path nfs://192.168.1.51/TVLibrary03/Ashes To Ashes/
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists gNfsConnection.Connect failed for url ?Z/
path (null)
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists for path nfs://192.168.1.51/TVLibrary04/Hung/
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists gNfsConnection.Connect failed for url ?Z/
path (null)
21:47:00 T:9764   DEBUG: NFSDEBUGPRINTOUT: CNFSDirectory::Exists for path nfs://192.168.1.51/TVLibrary04/How I Met Your Mother/

I am hoping that helps? What should I do from here

(also, should I move this to your gitbuh libnfs @Memphiz - an issue = github is a bit nicer for this sort of thing....)

[Memphiz]

Tbh i don't See that your logging does anything at all? You just log the calls to those functions but not the result of their call? Also be carefull when adding printouts After single line "if" conditions - your change stat Results in the fact that -1 is returned always! You need to add a Block after the if ( "{" log Return -1 "}" )

Don't Take it to my github ... Its Fine here...

Also url.Get().c_str() it is ...

[bossanova808]

Yeah I'm an idiot.

too long in python where indentation matters lately...

[bossanova808]

...gimme 10

[Memphiz]

Also your printouts don't match the code or i am to stupid to get it to match ...

[bossanova808]

@Memphiz Ok bit longer than 10 but...

Code:

bool CNFSFile::Exists(const CURL& url)
{
  CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Exists for url %s", url.GetFileName().c_str());
  return Stat(url,NULL) == 0;
}

int CNFSFile::Stat(struct __stat64* buffer)
{
  CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Stat for buffer");
  return Stat(m_url,buffer);
}


int CNFSFile::Stat(const CURL& url, struct __stat64* buffer)
{
  int ret = 0;
  CSingleLock lock(gNfsConnection);
  CStdString filename = "";
  
  if(!gNfsConnection.Connect(url,filename)){
    CLog::Log(LOGDEBUG, "NFSDEBUGPRINTOUT: CNFSFile::Stat about to return -1 as !gNfsConnection");
    return -1;
  }

...I think this time my braces are ok right?

gives

Code:

00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url TVLibrary03/Buffy the Vampire Slayer/Season 04/Buffy the Vampire Slayer - S04E19 - New Moon Rising.mkv
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat about to return -1 as !gNfsConnection
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url TVLibrary03/Buffy the Vampire Slayer/Season 04/Buffy the Vampire Slayer - S04E20 - The Yoko Factor (1).mkv
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat about to return -1 as !gNfsConnection
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url TVLibrary03/Buffy the Vampire Slayer/Season 04/Buffy the Vampire Slayer - S04E21 - Primeval (2).mkv
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat about to return -1 as !gNfsConnection
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url TVLibrary03/Buffy the Vampire Slayer/Season 04/Buffy the Vampire Slayer - S04E22 - Restless.mkv
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Stat about to return -1 as !gNfsConnection
00:45:14 T:2264   DEBUG: NFSDEBUGPRINTOUT: CNFSFile::Exists for url TVLibrary03/Buffy the Vampire Slayer/Season 05/Buffy the Vampire Slayer - S05E01 - Buffy vs Dracula.mkv

Those files definitely exist and Frodo 12.2 on the same machine is fine.

That seems to be to returning false for things that exist because !gNfsConnection.Connect(url,filename)

I am ridiculously tired though, so may have f*cked up again...

Got to get some sleep now, work is killing me, but I get an endless list of the above then the massive sql query and bye bye library.

---

(yeah I am compiling on one and running on my laptop and I got mixed up with generations but the above it all 1:1)