[smf007]

Just wanted you guys to know the crash/low resolution pictures are related to what appears to be a memory leak. The leak is somewhere in CXImage class. Possibly a couple spots it is leaking. I've been looking back and forth with the libraw, jpegio and freeimage implementation. I believe in my test environment I've squashed some, but I see the thumbnail generation in CXImage is leaking a little.

[Robotica]

Hopefully, Jmarshall can have a look..

[smf007]

I have not been able to figure out the memory leak. My C++ knowledge is not enough to figure this one out.

Where I left off at was looking at the memset usage. I'm left wondering if CXImage is leaking structures of ImageInfo (i.e. m_image). When reading up on the use of memset, there seemed to be a fair amount of caution on it's usage.

[Ace]

Do you have a git repo, or something similar, with the things you've found so far?

[spiff]

look, if this is a regression, and you care, go through the git bisect and tell us what broke it.

if you don't know git bisect, 'git help bisect' or ask.

[Robotica]

@Ace and @smf007: any progress on this?

[smf007]

Sorry Robotica, I haven't had any time to continue looking. I got to a spot where I needed to understand C++ a little better.

I did get a chance yesterday to look into what other media centre applications were doing. MythTV calls the dcraw executable directly. MediaPortal only handles RAW pictures through a plugin which calls the imagemagic executable directly. MeediOS uses Microsoft's WIC. Plex lists RAW picture extensions in their importer code, but I haven't been able to get it to import.

So none of those were any help. Additionally IF (big IF) there was a memory leak in the original dcraw code, we couldn't users others experience to determine that (the memory would be released once the executable closes).

Looking at the latest cximage, it's beyond my current skills (and time) to upgrade to using it. Numerous changes to the argument lists for the routines.


As for what changes I have made to the source code, it doesn't look like any make a difference.

[smf007]

Can someone try running through RAW pictures using Eden? I just did a quick test (didn't run it to the point of crashing) and I see the memory usage just take off similar to Frodo and the nightlies. I'm not seeing this as a regression as was implied earlier in this thread.

[smf007]

Was just trying to compile the libraw branches from above, but both are crashing. ace20022, does yours have all the changes committed? I'm getting a stack trace from RawPicture:ecode.

[Robotica]

Now I think I understand what's happening when crashinh happens with some .RAW files:

I think the problem is when there is an embedded JPG file in the .arw (or any other .raw file). This would also explain the inconsistent behaviour with raw files since not all raw files have embedded jpg's.

See http://trac.xbmc.org/ticket/14575 and https://www.dropbox.com/sh/2rdalgdiy4zammk/2ZqPz2fk1P for an example pics.

(If more examples are needed:
http://www.rawsamples.ch/index_en.php

This website organised all .raw files by their respective brandnames, without embedded .jpg.)

[Memphiz]

none of those 2 examples from the dropbox does crash xbmc on osx with current mainline here...

[Robotica]

Thanks for looking into this.....

Damn, I hoped this embedded JPEG was the problem with RAW-files.

I just tried out a nighlty (32 bit win7) with the same raw-files + an extra which I know works. I don't see a thumbnail for the 2 files (the working one has one) and after opening the file I know works, XBMC hangs endless in a black screen when I try to view a non working file. Then I checked this log.

Back to XBMC, I could escape from the black screen and then logging and XBMC continued. I then choose "regenerate thumbnails" to see this log.

After doing all this, even the correct raw-file isn't showed anymore (not even the thumb). It says "not able to reallocate the buffer", see this log.

Hopefully this helps.

(BTW: I also read tickets about 64 bit OS handles RAW correct but 32 bit fails so that also might indicated to some memory overflow. But it doesn't explain the failing in creating thumbs and loading images on my end of those other 2 examples.)

ps. Notice the loglines repeating theirselves in sets of 2, spawning the log.

[Robotica]

Could you make any sense out of those logs?

[Robotica]

Some memleaks in cximage by wsoltys: https://github.com/xbmc/xbmc/pull/3265

[smf007]

Some memleaks in cximage by wsoltys: https://github.com/xbmc/xbmc/pull/3265

They are appreciated. Unfortunately for Nikon NEFs (D90), there is still a heavy memory leak somewhere in cximage.