v17 build can't find libssh symbol sftp_tell64; kodi refs _both_ libssh & libssh2?
PGNd
Junior Member
Posts: 43
#1
2016-01-02, 02:20
Building latest kodi-head,
Code:
    git log | head
        commit 4bac6f8c6a9fc41ec4b20dd5e0a3f9b64d3ad2dd
        Merge: c852b64 78cfab5
        Author: xxx <xxx>
        Date:   Fri Jan 1 22:33:54 2016 +0100

my build's failing at
Code:
    ...
    checking for DRM... yes
    checking for EGL... yes
    checking for sftp_tell64 in -lssh... no
    configure: error: == Could not find libssh. ==

checking config.log, the libssh check is
Code:
    ...
    configure:28411: checking for sftp_tell64 in -lssh
    configure:28436: /usr/bin/gcc-5 -o conftest -O2 -g -D_DEBUG -Wall ...
    /usr/lib64/libssh.so: undefined reference to `CRYPTO_set_id_callback'
    /usr/lib64/libssh.so: undefined reference to `DSA_generate_parameters'
    collect2: error: ld returned 1 exit status
    configure:28436: $? = 1
    configure: failed program was:
    | /* confdefs.h */
    | #define PACKAGE_NAME "kodi"
    | #define PACKAGE_TARNAME "kodi"
    | #define PACKAGE_VERSION "16.9.701"
    | #define PACKAGE_STRING "kodi 16.9.701"
    ...
    | #define HAVE_X11 1
    | /* end confdefs.h.  */
    |
    | /* Override any GCC internal prototype to avoid an error.
    |    Use char because int might match the return type of a GCC
    |    builtin and then its argument prototype would still apply.  */
    | #ifdef __cplusplus
    | extern "C"
    | #endif
    | char sftp_tell64 ();
    | int
    | main ()
    | {
    | return sftp_tell64 ();
    |   ;
    |   return 0;
    | }
    configure:28445: result: no
    configure:28455: error: == Could not find libssh. ==
    ...

Starting to poke around, my distro's apparently stripped symbols
Code:
    nm /usr/lib64/libssh* | grep sftp_tell64
        nm: /usr/lib64/libssh2.so: no symbols
        nm: /usr/lib64/libssh2.so.1: no symbols
        nm: /usr/lib64/libssh2.so.1.0.1: no symbols
        nm: /usr/lib64/libssh.so: no symbols
        nm: /usr/lib64/libssh.so.4: no symbols
        nm: /usr/lib64/libssh.so.4.4.0: no symbols
        nm: /usr/lib64/libssh_threads.so: no symbols
        nm: /usr/lib64/libssh_threads.so.4: no symbols
        nm: /usr/lib64/libssh_threads.so.4.4.0: no symbols

Digging in the kodi sources,
Code:
    find . | grep libssh
        ./tools/depends/target/libssh2
        ./tools/depends/target/libssh2/Makefile
        ./tools/depends/target/libssh2/libdl.patch
        ./tools/depends/target/libssh
        ./tools/depends/target/libssh/ntohl.patch
        ./tools/depends/target/libssh/darwin.patch
        ./tools/depends/target/libssh/removelegacy.patch
        ./tools/depends/target/libssh/fix-gcc-5-compile.patch
        ./tools/depends/target/libssh/android.patch
        ./tools/depends/target/libssh/Makefile
        ./tools/depends/target/libssh/md5.patch

I see references to BOTH libssh & libssh2.

atm, I libssh won't build locally (https://www.libssh.org/archive/libssh/20...00000.html), but libssh2 does, with libssh2-specific sftp_tell64 symbols,
Code:
    nm /usr/local/lib64/libssh2*so* | grep sftp_tell64
        000000000001767a T libssh2_sftp_tell64
        000000000001767a T libssh2_sftp_tell64
        000000000001767a T libssh2_sftp_tell64

Applying this patch to source
Code:
    --- configure.ac.ORIG    2016-01-01 12:09:23.777073577 -0800
    +++ configure.ac    2016-01-01 15:44:15.647714859 -0800
    @@ -1405,12 +1405,12 @@
       USE_TOUCH_SKIN=0
     fi
    
    -# libssh
    +# libssh2
     if test "x$use_ssh" = "xno"; then
       AC_MSG_NOTICE($ssh_disabled)
       use_libssh="no"
     else
    -  AC_CHECK_LIB([ssh], [sftp_tell64],, AC_MSG_ERROR($ssh_not_found))
    +  AC_CHECK_LIB([ssh2], [libssh2_sftp_tell64],, AC_MSG_ERROR($ssh_not_found))
       AC_DEFINE([HAVE_LIBSSH], [1], [Whether to use libSSH library.])
     fi

allows configure to get past the ssh symbol error
Code:
    ...
    checking for libssh2_sftp_tell64 in -lssh2... yes
    ...

but I'm still getting _other_ configure errors that're preventing me from completing make, so can't tell whether kodi builds with this libssh2 in place


Q: Why does kodi src refer to both libssh & libssh2? And is the sftp_tell64 what's wanted/needed?
Find
Reply
PGNd
Junior Member
Posts: 43
#2
2016-01-02, 08:41
Kodi source is apparently using deprecated Openssl crypto thread functions which, when building against new/external openssl libs, cause make fails

e.g., later in the build
Code:
        cat    ./xbmc/utils/CryptThreading.cpp
            ...
    70        CryptThreadingInitializer::CryptThreadingInitializer()
            {
              bool attemptedToSetSSLMTHook = false;
            #ifdef HAVE_OPENSSL
              // set up OpenSSL
              numlocks = CRYPTO_num_locks();
    !!!          CRYPTO_set_id_callback(thread_id);
              CRYPTO_set_locking_callback(lock_callback);
              attemptedToSetSSLMTHook = true;
            #else
              numlocks = 1;
            #endif
            ...

note, this is the same problematic function reference as above in/for libssh
Code:
/usr/lib64/libssh.so: undefined reference to `CRYPTO_set_id_callback'

here's the deprecation notice
Code:
    https://www.openssl.org/docs/manmaster/crypto/threads.html
        CRYPTO_THREADID and associated functions were introduced in OpenSSL 1.0.0 to replace (actually, deprecate) the previous CRYPTO_set_id_callback(), CRYPTO_get_id_callback(), and CRYPTO_thread_id() functions which assumed thread IDs to always be represented by 'unsigned long'.
Find
Reply
PGNd
Junior Member
Posts: 43
#3
2016-01-02, 16:46
Reading further at the openssl deprecation notice link,

...
If the application does not register such a callback using CRYPTO_THREADID_set_callback(), then a default implementation is used
...

so simply disable it?

edit +76 ./xbmc/utils/CryptThreading.cpp
...
- CRYPTO_set_id_callback(thread_id);
+ //CRYPTO_set_id_callback(thread_id);
...

with that 'naive' edit, the config+make gets past previous error, but I hit lots of 'sftp'-related error next.

at this point, doubt that it's UN related Undecided

...
make[1]: Leaving directory '/usr/local/src/xbmc/lib/libdvd'
CPP xbmc/CompileInfo.o
AR xbmc/xbmc.a
LD kodi.bin
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::CreateFileHande(std:Confusedtring const&)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:119: undefined reference to `sftp_open'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:122: undefined reference to `sftp_file_set_blocking'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::CloseFileHandle(sftp_file_struct*)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:137: undefined reference to `sftp_close'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::Stat(char const*, stat64*)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:262: undefined reference to `sftp_stat'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:276: undefined reference to `sftp_attributes_free'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::Seek(sftp_file_struct*, unsigned long)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:296: undefined reference to `sftp_seek64'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::Read(sftp_file_struct*, void*, unsigned long)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:303: undefined reference to `sftp_read'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::GetPosition(sftp_file_struct*)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:310: undefined reference to `sftp_tell64'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::VerifyKnownHost(ssh_session_struct*)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:320: undefined reference to `ssh_is_server_known'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:334: undefined reference to `ssh_write_knownhost'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:342: undefined reference to `ssh_get_error'
xbmc/filesystem/filesystem.a(SFTPFile.o): In function `CSFTPSession::Connect(std:Confusedtring const&, unsigned int, std:Confusedtring const&, std:Confusedtring const&)':
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:356: undefined reference to `ssh_new'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:364: undefined reference to `ssh_options_set'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:370: undefined reference to `ssh_options_set'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:376: undefined reference to `ssh_options_set'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:382: undefined reference to `ssh_options_set'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:383: undefined reference to `ssh_options_set'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:412: undefined reference to `ssh_connect'
/usr/local/src/xbmc/xbmc/filesystem/SFTPFile.cpp:426: undefined reference to `ssh_userauth_none'
...
Find
Reply
PGNd
Junior Member
Posts: 43
#4
2016-01-02, 17:00
Those'll need the 'new' libssh2_sftp* symbol references.

bit more than I can bite off atm. trying "--disable-ssh" config option for yucks ...
Find
Reply
wsnipex
Team-Kodi
Posts: 7,650
#5
2016-01-03, 16:46
nm: /usr/lib64/libssh.so.4.4.0: no symbols

this is really strange. Please provide
ls -l /usr/lib64/libssh.so*
readelf -a /usr/lib64/libssh.so.4.4.0 | grep sftp_tell
Team XBMC stable PPA -- Team XBMC unstable PPA -- Team XBMC nightly PPA
DEBUG log
Find
Reply
PGNd
Junior Member
Posts: 43
#6
2016-01-03, 19:39
Code:
ls -l /usr/lib64/libssh.so*
    lrwxrwxrwx 1 root root   11 Nov 27 17:19 /usr/lib64/libssh.so -> libssh.so.4*
    lrwxrwxrwx 1 root root   15 Nov 27 17:11 /usr/lib64/libssh.so.4 -> libssh.so.4.4.0*
    -rwxr-xr-x 1 root root 457K Oct  7 03:17 /usr/lib64/libssh.so.4.4.0*
readelf -a /usr/lib64/libssh.so.4.4.0 | grep sftp_tell
   327: 0000000000037950     5 FUNC    GLOBAL DEFAULT   12 sftp_tell64
   432: 0000000000037940     5 FUNC    GLOBAL DEFAULT   12 sftp_tell

and just for ref,
Code:
ls -l /usr/lib64/libssh2.so*
    lrwxrwxrwx 1 root root   16 Jan  1 12:27 /usr/lib64/libssh2.so -> libssh2.so.1.0.1*
    lrwxrwxrwx 1 root root   16 Nov 27 16:21 /usr/lib64/libssh2.so.1 -> libssh2.so.1.0.1*
    -rwxr-xr-x 1 root root 168K Sep 30 03:35 /usr/lib64/libssh2.so.1.0.1*
readelf -a /usr/lib64/libssh2.so.1.0.1 | grep sftp_tell
   166: 0000000000016580    19 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell
   208: 00000000000165a0    19 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell64

ls -l /usr/local/lib64/libssh*.so*
    lrwxrwxrwx 1 root root   16 Jan  1 15:57 /usr/local/lib64/libssh2.so -> libssh2.so.1.0.1*
    lrwxrwxrwx 1 root root   16 Jan  1 15:57 /usr/local/lib64/libssh2.so.1 -> libssh2.so.1.0.1*
    -rwxr-xr-x 1 root root 201K Jan  1 15:57 /usr/local/lib64/libssh2.so.1.0.1*
readelf -a /usr/local/lib64/libssh2.so.1.0.1 | grep sftp_tell
   177: 00000000000175ba    16 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell
   219: 00000000000175ca    16 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell64
   366: 00000000000175ca    16 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell64
   533: 00000000000175ba    16 FUNC    GLOBAL DEFAULT   12 libssh2_sftp_tell

Fwiw, with my jumble of tweaks -- incl the latest --disable-ssh -- I can successfully build both the master & Jarvis branches. The app's up & running ...
Find
Reply
wsnipex
Team-Kodi
Posts: 7,650
#7
2016-01-03, 20:28
getting rid of libssh2 should allow you to build kodi with ssh support enabled. libssh2 is not supported and its probably found first.
Team XBMC stable PPA -- Team XBMC unstable PPA -- Team XBMC nightly PPA
DEBUG log
Find
Reply
PGNd
Junior Member
Posts: 43
#8
2016-01-03, 21:11
libssh2 can't be removed ... it's used elsewhere on the system.

If libssh2 is "not supported", why's it in kodi at all, and listed in DEPENDS= ?
Code:
git log | head
    commit d0f5e1ef86448f980cc977bb648d65da3ab5f9be
    Author: txtranslation <[email protected]>
    Date:   Sun Jan 3 17:52:13 2016 +0000

        [lang] updated language files from Transifex for Skin Confluence

    commit 34438112884cb424158834825bfd2d881b1a3f15
    Author: txtranslation <[email protected]>
    Date:   Sun Jan 3 17:52:13 2016 +0000

find . | grep libssh2
    ./tools/depends/target/libssh2
    ./tools/depends/target/libssh2/Makefile
    ./tools/depends/target/libssh2/libdl.patch

grep -rlin libssh2 .
    ./tools/depends/target/curl/Makefile
    ./tools/depends/target/libssh2/Makefile
    ./tools/depends/target/config-binaddons.site.in
    ./tools/depends/target/config.site.in
    ./tools/depends/target/Makefile
    ./.gitignore

cat ./tools/depends/target/Makefile
    ...
    DEPENDS = \
        pcre expat gettext sqlite3 libgpg-error \
        libgcrypt bzip2 liblzo2 libzip freetype2 fontconfig \
>>>        openssl gmp nettle gnutls libssh2 curl \
    ...

Even if libssh2 is present, kodi's checks should check for the correct symbols/packages, instead of depending on removal of packages from the system, no?
Find
Reply
wsnipex
Team-Kodi
Posts: 7,650
#9
2016-01-04, 10:52
the configure check is from a time when libssh didn't ship a package-config file, so there was no other way to check for libssh and distinguish between libssh2.
But this has since changed, so I'll update our checks.
Team XBMC stable PPA -- Team XBMC unstable PPA -- Team XBMC nightly PPA
DEBUG log
Find
Reply
PGNd
Junior Member
Posts: 43
#10
2016-01-04, 16:05
(2016-01-04, 10:52)wsnipex Wrote: so I'll update our checks.

That should take care of the libssh issue. Will that fix be in the Jarvis branch, or just in master?

And the deprecated Openssl crypto thread function usage? Looks unrelated to the libssh prob, but still needs an update fix.
Find
Reply
PGNd
Junior Member
Posts: 43
#11
2016-01-23, 22:48
> getting rid of libssh2 should allow you to build kodi with ssh support enabled. > libssh2 is not supported and its probably found first.

Again, if libssh2 is "not supported", why's it in kodi at all, and listed in DEPENDS= ?

> the configure check is from a time when libssh didn't ship a package-config
> file, so there was no other way to check for libssh and distinguish
> between libssh2. But this has since changed, so I'll update our checks.

Building latest Jarvis source,
Code:
    ./configure --enable-ssh

still fails
Code:
    ...
    checking for sftp_tell64 in -lssh... no
    configure: error: == Could not find libssh. ==

Have those checks been updated, but don't work? Or not updated?

> And the deprecated Openssl crypto thread function usage?

latest Jarvis sources still reference the deprecated functions
Code:
cat ./xbmc/utils/CryptThreading.cpp
    ...
    CryptThreadingInitializer::CryptThreadingInitializer()
    {
      bool attemptedToSetSSLMTHook = false;
    #ifdef HAVE_OPENSSL
      // set up OpenSSL
      numlocks = CRYPTO_num_locks();
      CRYPTO_set_id_callback(thread_id);
      CRYPTO_set_locking_callback(lock_callback);
      attemptedToSetSSLMTHook = true;
    #else
      numlocks = 1;
    #endif
    ...

Any intention of fixing these before the release?
Find
Reply
wsnipex
Team-Kodi
Posts: 7,650
#12
2016-01-24, 10:47
the configure checks were fixed in master. And no, this won't make it into jarvis. The openssl callbacks haven't been looked at yet.
Team XBMC stable PPA -- Team XBMC unstable PPA -- Team XBMC nightly PPA
DEBUG log
Find
Reply
Paxxi
Team-Kodi Member
Posts: 252
#13
2016-01-24, 12:36
This tripped me up as well when looking at depends for Windows. Kodi itself uses libs she and libcurl has a dependency on libssh2(same developer), afaik we don't build curl with ssh support so it's possible it's not needed at all. Wsnipex probably knows these parts of the code best but with the amount of libraries we require it's easy that some cruft gets left behind so be patient.
Find
Reply
wsnipex
Team-Kodi
Posts: 7,650
#14
2016-01-24, 14:10
libssh2 is not used, the curl dependency on it is bogus.
Apparently it has been like this ever since unified deps was introduced...
Team XBMC stable PPA -- Team XBMC unstable PPA -- Team XBMC nightly PPA
DEBUG log
Find
Reply

Home
Search
Help
Logout Mark Read Team Forum Stats Members Help
build can't find libssh symbol sftp_tell64; kodi refs _both_ libssh & libssh2?0