[bitstreamout]

On my Tumbleweed I'm using Matrix with less addons as well as with

                <memorysize>1703936</memorysize>
                <buffermode>1</buffermode>
                <readfactor>4</readfactor>

here I see a really stable physical memory consumption below 3% of RAM ... but on the NAS with more used(!) addons even if with less memorysize (7*128*1024 .. yep 7 is a prime) the physical used RAM is fluctuating between 6% and more than 30% which is 600Mb and this clearly triggers swapping and hence stutter in the audio stream and sometime (even with reduced swappiness in the new created cgroup memory:kodi) kodi interrupts the stream or is oomed by the kernel.

And I'm not the only one which sees this behaviour with Matrix

https://discourse.coreelec.org/t/matrix-...sage/16080
https://github.com/xbmc/xbmc/issues/19332
https://www.kodinerds.net/index.php/Thre...e-TV-Model

[bitstreamout]

Beside the trouble with Matrix have I've reworked some addons I had used for Leia for my Denon AVR as the old upstream had never react on the issue I had open for the support of Matrix, see github.com:bitstreamout/script.kodi.denon.control and github.com:bitstreamout/denonremote

[bitstreamout]

NB:

(gdb) bt
#0  __GI___mmap64 (addr=addr@entry=0x0, len=len@entry=163536896, prot=prot@entry=3, flags=flags@entry=34, fd=fd@entry=-1, offset=offset@entry=0) at ../sysdeps/unix/sysv/linux/mmap64.c:51
#1  0x00007f216e51542f in sysmalloc (nb=nb@entry=163534000, av=av@entry=0x7f20c0000020) at malloc.c:2329
#2  0x00007f216e516180 in _int_malloc (av=av@entry=0x7f20c0000020, bytes=bytes@entry=163533984) at malloc.c:4135
#3  0x00007f216e51667b in _int_memalign (av=0x7f20c0000020, alignment=64, bytes=<optimized out>) at malloc.c:4683
#4  0x00007f216e51c0fa in _mid_memalign (address=<optimized out>, bytes=163533880, alignment=<optimized out>) at malloc.c:3324
#5  __posix_memalign (memptr=0x7f20c7ffdeb0, alignment=<optimized out>, size=163533880) at malloc.c:5361
#6  0x000056192c8c9dd3 in av_malloc ()
#7  0x000056192bcd0ccb in ffio_ensure_seekback ()
#8  0x000056192bdcced5 in ?? ()
#9  0x000056192bdcd815 in ?? ()
#10 0x000056192bdcef16 in ?? ()
#11 0x000056192bce509a in ff_read_packet ()
#12 0x000056192bce5dc3 in ?? ()
#13 0x000056192bce6b38 in av_read_frame ()
#14 0x000056192af3f8ba in CDVDDemuxFFmpeg::Read() ()
#15 0x000056192bc3b8ea in VideoPlayerCodec::ReadPCM(unsigned char*, int, int*) ()
#16 0x000056192bc343ed in CAudioDecoder::ReadSamples(int) ()
#17 0x000056192bc3ac71 in PAPlayer:rocessStreams(double&) ()
#18 0x000056192bc39a23 in PAPlayer:rocess() ()
#19 0x000056192b26cf45 in CThread::Action() ()
#20 0x000056192b26ec3a in ?? ()
#21 0x000056192b26f90a in ?? ()
#22 0x00007f216d67b6df in ?? () from /opt/Kodi19L/lib_kodi/libstdc++.so.6
#23 0x00007f21738136db in start_thread () from /opt/Kodi19L/lib_kodi/libpthread.so.0
#24 0x00007f216e5a271f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

strange malloc with the help of memalign (ensures that memory is really aligend) but why is this done .. reference https://ffmpeg.org/doxygen/0.6/mem_8c.html

boole kodi/Upstream(Matrix)> grep -rs av_free xbmc | grep -v av_freep | wc -l
41

OK 41 times not called av_freep as mentioned in the reference:

void av_free(void * ptr ) 
          Frees a memory block which has been allocated with av_malloc(z)() or av_realloc().
Parameters:
          ptr Pointer to the memory block which should be freed.
Note:
          ptr = NULL is explicitly allowed.
          It is recommended that you use av_freep() instead.

[bitstreamout]

Reading the backtrace (and this piece of code repeats during receiving the oga flac stream) shows that round about 159Mb are memory mapped .... OK in the kodi starter script there is exported

# Fix wasting RAM due to fragmentation
export MALLOC_MMAP_THRESHOLD_=131072

to reduce as explained in the comment the physical memory fragementation. Seems not to work well. Seeking for the code using `av_read_frame()` I found as shown in the backtrace

 DemuxPacket* CDVDDemuxFFmpeg::Read()
{
[...]
      // timeout reads after 100ms
      m_timeout.Set(20000);
      m_pkt.result = av_read_frame(m_pFormatContext, &m_pkt.pkt);
      m_timeout.SetInfinite();

looks like in the `m_pFormatContext` there is stored thise huge size ... the only problem is that I do not have debug infos on the QNAP in the Ubuntu Container ...
now I'm plaing around with cgroups v1 with cgcreate and cgexec

if type -p cgcreate > /dev/null 2>&1
then
  if ! test -e /sys/fs/cgroup/kodi/memory.swappiness
  then
    mount -t cgroup -o memory cgroup /sys/fs/cgroup
    cgcreate -g memory:kodi
  fi
  if test -e /sys/fs/cgroup/kodi/memory.swappiness
  then
    read swap < /sys/fs/cgroup/kodi/memory.swappiness
    if test $swap -gt 0
    then
      echo 0 > /sys/fs/cgroup/kodi/memory.swappiness
    fi
  fi
fi

[...]

  if type -p cgexec > /dev/null 2>&1 && test -e /sys/fs/cgroup/kodi/memory.swappiness
  then
    cgexec -g memory:kodi ${KODI_BINARY} $SAVED_ARGS &
    CHILD=$!
  else
    ${KODI_BINARY} $SAVED_ARGS &
    CHILD=$!
  fi

as well as with

# Fix wasting RAM due to fragmentation
#export MALLOC_MMAP_THRESHOLD_=131072
unset ${!MALLOC_*}
export MALLOC_MMAP_THRESHOLD_=1703936
export MALLOC_TRIM_THRESHOLD_=65536
export MALLOC_MMAP_MAX_=131072
export MALLOC_ARENA_MAX=16

[bitstreamout]

This is what ffplay found (and it does never ever use this huge amount of memory) see https://paste.kodi.tv/ranekunasa

The highest memory consumption for local music files is

21038 root      20   0 1185448 212932  80420 R 120.7 11.1   3:44.04 kodi-x11                                                                                                                  

for a dsf files from SACD (5.1 channels 352.8kHz for 16934kbps).

[bitstreamout]

Catch an other break at mmap which shows a strange ffio_ensure_seekback ... and I found out that this kodi binary uses static ffmpeg 4.3.2 which provides libavformat 58.45.100 ... at least a string call shows this.

#5  0x00007f672492b098 in _mid_memalign (address=<optimized out>, bytes=47707931, alignment=<optimized out>) at malloc.c:3278
#6  __posix_memalign (memptr=0x7f66f86a3eb0, alignment=<optimized out>, size=47707931) at malloc.c:5361
#7  0x0000556323335dd3 in av_malloc ()
#8  0x000055632273cccb in ffio_ensure_seekback ()

(here we have 47707931 bytes aka 45Mb).  Cause could be a timeshift configuration or a bug in the older ffmpeg tar ball.

[bitstreamout]

Now I've build a kodi for QNAP HD-Station and debugged this in an container. IMHO it is a BUG in ffmpeg ffio_ensure_seekback() function which should accordingly to https://ffmpeg.org/doxygen/4.1/avio__int...27333eb284 do

Ensures that the requested seekback buffer size will be available. 
Will ensure that when reading sequentially up to buf_size, seeking within the current pos and pos+buf_size is possible. Once the stream position moves outside this window this guarantee is lost. 

but

(gdb) down
#8  0x000055ebc54dd7e5 in ogg_read_page (s=s@entry=0x7f79980e9d40, sid=sid@entry=0x7f79b57f92fc, probing=probing@entry=0) at src/libavformat/oggdec.c:367
367         ffio_ensure_seekback(bc, MAX_PAGE_SIZE);
(gdb) print bc->buffer_size
$6 = 168115605

this leads as seen above and here to exterma large buffer allocations

0  0x00007f7a3151b6d0 in __GI___mmap64 (addr=addr@entry=0x0, len=len@entry=168161280, prot=prot@entry=3, flags=flags@entry=34, fd=fd@entry=-1, offset=offset@entry=0)
    at ../sysdeps/unix/sysv/linux/mmap64.c:51
#1  0x00007f7a3149430f in sysmalloc (nb=nb@entry=168159584, av=av@entry=0x7f7998000020) at malloc.c:2329
#2  0x00007f7a31495060 in _int_malloc (av=av@entry=0x7f7998000020, bytes=bytes@entry=168159568) at malloc.c:4135
#3  0x00007f7a3149555b in _int_memalign (av=0x7f7998000020, alignment=64, bytes=<optimized out>) at malloc.c:4683
#4  0x00007f7a3149afda in _mid_memalign (address=<optimized out>, bytes=168159462, alignment=64) at malloc.c:3324
#5  0x00007f7a3149afda in __posix_memalign (memptr=memptr@entry=0x7f79b57f9090, alignment=alignment@entry=64, size=size@entry=168159462) at malloc.c:5361
#6  0x000055ebc5ec1463 in av_malloc (size=size@entry=168159462) at src/libavutil/mem.c:86
#7  0x000055ebc53e1fdb in ffio_ensure_seekback (s=s@entry=0x7f79980ea3c0, buf_size=168159462, buf_size@entry=65307) at src/libavformat/aviobuf.c:996
#8  0x000055ebc54dd7e5 in ogg_read_page (s=s@entry=0x7f79980e9d40, sid=sid@entry=0x7f79b57f92fc, probing=probing@entry=0) at src/libavformat/oggdec.c:367

reading ./build/build/ffmpeg/src/ffmpeg/libavformat/aviobuf.c

int ffio_ensure_seekback(AVIOContext *s, int64_t buf_size)
{
    uint8_t *buffer;
    int max_buffer_size = s->max_packet_size ?
                          s->max_packet_size : IO_BUFFER_SIZE;
    int filled = s->buf_end - s->buffer;
    ptrdiff_t checksum_ptr_offset = s->checksum_ptr ? s->checksum_ptr - s->buffer : -1;

    buf_size += s->buf_ptr - s->buffer + max_buffer_size;  // <<< IMHO this is a BUG

    if (buf_size < filled || s->seekable || !s->read_packet)
        return 0;
    av_assert0(!s->write_flag);

    buffer = av_malloc(buf_size);
    if (!buffer)
        return AVERROR(ENOMEM);

    memcpy(buffer, s->buffer, filled);
    av_free(s->buffer);
    s->buf_ptr = buffer + (s->buf_ptr - s->buffer);
    s->buf_end = buffer + (s->buf_end - s->buffer);
    s->buffer = buffer;
    s->buffer_size = buf_size;
    if (checksum_ptr_offset >= 0)
        s->checksum_ptr = s->buffer + checksum_ptr_offset;
    return 0;
}

and this is also discussed at https://patchwork.ffmpeg.org/project/ffm...gmail.com/

[bitstreamout]

Yep actual code shows no malloc anymore https://git.ffmpeg.org/gitweb/ffmpeg.git...uf.c#l1055

[bitstreamout]

I'm now using Kodi Matrix build with ffmpeg 4.4.2 for an Ubuntu HDStation Container and it works absolute flawless. Kodi stays now exactyl at 8.8% memory playing https://maggie.torontocast.com:8076/flac

Still at 8.8% memory

Guess what 8.8% memory