20 Nov 2019 - TVDB Scraper v3.2.0 is now available which reinstates scraping. TVDB are still in the process of fixing a number of bugs so there may still be further breakages. See this post. 2901570 (post)

  • 1(current)
  • 2
  • 3
  • 4
  • 5
  • 7
  •   
Req Better RAW-support (pictures)
#1
**Historical topic about RAW-files for reference**

While using RAW files in XBMC isn't a large feature but many photographers keep their RAW files in the same directory as the .JPEG equivalant. Why?
Quote:The advantage of the RAW format is that, like the old film negative, it contains the truest record of the data captured by the camera’s image sensor. That data can be processed to suit what the photographer wants as the final image. In traditional photography, this is equivalent to processing the negative into the final positive print

So RAW-files are listed as supported but they are surrounded with troubles. First, like you can see in the WIKI, it's not 100% clear which formats are supported.

Supported Vendor specific RAW extensions:
*.orf (Olympus), *.crw(Canon), *.ARW (Sony), *.NEF (Nikon), *.dcr (Kodak), *.raw, *.cr2(Canon), *.EEF (Epson), *.mrw (Minolta), *.pef (Pentax), *.raf (Fuji), *.srf (Sony), *.x3f (Sigma), *.rc2.

Then there also is *.DNG (Adobe prop. RAW format adopted as industry standard).

To better support all those vendor specific RAW extensions, XBMC (Wiso) already made the imagefactory to easily integrate other image libraries.

Since a lot of development happened around pictures like the imagefactory, the new jpeg-encoding and Ulion's rewrite (and in XBMC in generel like threading, caching, etc) is not strange that some regression pops up.

Some bugs related with raw-support:
http://trac.xbmc.org/ticket/14419
http://trac.xbmc.org/ticket/14184
http://trac.xbmc.org/ticket/13974
http://forum.xbmc.org/showthread.php?tid=157742
NEF problems: http://forum.xbmc.org/showthread.php?tid=140941
http://forum.xbmc.org/showthread.php?tid=13484&page=7

*.ARW has 2 bugs.
1> Caching .ARW over network: jpegparser is unable to load .arw files and then the threading crashes.
2> Displaying .ARW: Texture manager can not handle .ARW

*.dng fails creating thumbs
1> http://trac.xbmc.org/ticket/13947
(and wrongly created on .rc2 (http://trac.xbmc.org/ticket/14375))
http://forum.xbmc.org/showthread.php?tid...pid1499149

Making pictures threat-safe solved some problem with corrupted thumbs. To fix remaining bugs could be simply updating libdcr 8.91 (which ships with current CXimage 6.x) to the newest version (I've seen 8.93 in CxImage 7.x and 8.91 by the creator of libdcr). But the more final solution would be using the image factory to use libRAW and to use Ffmpeg's tag loader to do all CXimage's work.

Edit: This is confirmed with current nightlies.
Reply
#2
The wish is to get rid of cximage and use system libs instead (libpng, libtiff, libraw) etc. We just need to find someone who wants it and has the time to develop it.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
Reply
#3
Thnx. What about a GUI option to "don't show RAW-files in listings"? At least for the time being this seems a sensible "solution". Would fix playback from the native GUI but would leave addons and the like.
Reply
#4
This option "Disable RAW-files" means:
- don't extract thumbs/EXIF info. from RAW-files
- don't show RAW (extracted) thumbs in listings
- don't use raw files in screensaver/slideshow

Even with proper raw-support (libRAW) this is a usefully feature, given that many people like to store their raw files in the same directory as their jpeg counterparts but don't want them to show up double in their listings/slideshow/screensaver.

Enabling this option (default to Disable) for Gotham would improve the general user experience untill libRAW.
Reply
#5
(2013-06-14, 16:10)WiSo Wrote: The wish is to get rid of cximage and use system libs instead (libpng, libtiff, libraw) etc. We just need to find someone who wants it and has the time to develop it.

Like Ace is doing for animated GIF's but this needs to be done for all format detection an gaining access to all needed file info.
Reply
#6
I disagree with the evaluation that no one in his right mind uses raw files in XBMC. For "normal", i.e. brand specific raw files I would agree but I would argue DNGs are the exception. To me DNG is just the same as any other format like tiff or jpg and I do not generate jpgs in addition to DNG files (too labour intensive). Unfortunately, if XBMC Frodo so much as smells a dng file on a MacMini it crashes right away. So we have regressed quite a lot since Eden where the format was at least supported.

For another thread on this topic which has elicited no useful response at all from developers but which does contain log files see http://forum.xbmc.org/showthread.php?tid=151268
Reply
#7
I updated the op to beter reflect your comments. Hopefully you file a proper debug report for your .Dng issue.
Reply
#8
@TeamXBMC:
Are there any plans on improving RAW support for Gotham? Or is it waiting for patches? Or would you like bugs to be better tested against master? Anything that can be done to improve RAW-support besides switching to libRAW? Stopgap solutions?
Reply
#9
Hi, just out of curiosity I've added libraw for win32 and pushed the code to my github repo. It's just a quick hack, e.g. colours don't seem to be correct there's an issue with rotation/flip and it's terrible slow, but the crashes are gone.
Due to lack of time and because I'm really no expert in image processing I won't develop it any further. So I encourage everyone to take my code and do it properlyWink

The code can be found there: https://github.com/ace20022/xbmc/commits/add_libraw
Reply
#10
thanks for doing and sharing the work and this initial testing. So no more crashes in listings, generating thumbs and so on? Which raw's did you try? And do you mean pink colors? If so, I've seen those problems a lot so that could be something else..

The rotating is done based on exif-info I believe.. Hopefully someone develops this further..
Reply
#11
(2013-06-25, 16:50)Robotica Wrote: thanks for doing and sharing the work and this initial testing. So no more crashes in listings, generating thumbs and so on? Which raw's did you try? And do you mean pink colors? If so, I've seen those problems a lot so that could be something else..

The rotating is done based on exif-info I believe.. Hopefully someone develops this further..

I used samples from http://www.photographyblog.com. I haven't encountered crashes. The colors were (slightly) different from the ones rendered with irfanview, which probably isn't an defect.
Reply
#12
I've given it a shot yesterday.

1. added libraw to the unified depends buildsystem (glad it had a proper autotools buildsystem at first look) and compiled it for osx
2. added proper libraw detection support to our configure
3. fixed your source pitch calculation and removed the flip altering (why did you alter it to 3?)

I only tested it with one random raw cr2 i've found on google (5010x3336 50MB).

There are multiple problems here.

1. Libraw only decodes the image. If i am correct it doesn't apply any corrections based on the raw metadata (whitebalance, color matrix, color profile). So basically you get flat decoded rgb data. (i might be wrong - but it looked flat)
2. Libraw only decodes the image in full resolution. Based on texture size limitations - current cximage implementation also scales the image down for getting a texture fit. There is no such possibility in libraw (at least i didn't find one).
3. Not sure how to solve 2. - but i guess thats what makes libraw so slow in comparison to cximage (we are copying all pixels but the texture then only shows the upper left square).

I am not into the picture stuff and how to handle the limited texture sizes (imo scaling down is only an option for getting the full screen overview - when zooming in there should be a new more detailed texture from the source data).

Well i feel not up to the task to tackle this (also i am not very interested in it). If it jus was about getting libraw rolling on all our platforms i would be able to do that. But the deeper integration and correction of its usage in xbmc is nothing for me.
AppleTV4/iPhone/iPod/iPad: HowTo find debug logs and everything else which the devs like so much: click here
HowTo setup NFS for Kodi: NFS (wiki)
HowTo configure avahi (zeroconf): Avahi_Zeroconf (wiki)
READ THE IOS FAQ!: iOS FAQ (wiki)
Reply
#13
Thnx Memphiz. At least, xbmc is few steps closer. Do you have a link to your work so the community can continue, if possible?

Maybe your detection stuff can also be used for cximage? Since most logs shows a error about jpeg-parsing. Example overhere: http://forum.xbmc.org/showthread.php?tid...pid1450424
Reply
#14
The "detection stuff" is for our buildsystem (proper detection of libraw and activating rawpicturesupport in case it is found during compile).

https://github.com/Memphiz/xbmc/commits/libraw
AppleTV4/iPhone/iPod/iPad: HowTo find debug logs and everything else which the devs like so much: click here
HowTo setup NFS for Kodi: NFS (wiki)
HowTo configure avahi (zeroconf): Avahi_Zeroconf (wiki)
READ THE IOS FAQ!: iOS FAQ (wiki)
Reply
#15
So wat do you guys think?
1) Leave Gotham with broken Raws.
2) update cximage to version 7 and try fixing open bugs.
3) wait for someone who feel like fixing xbmc to properly use libraw.

Since it worked it in eden I guess it's regression due to all development done.
I think those are the 3 options.
Reply
  • 1(current)
  • 2
  • 3
  • 4
  • 5
  • 7
  •   
 
Thread Rating:
  • 3 Vote(s) - 4 Average



Logout Mark Read Team Forum Stats Members Help
Better RAW-support (pictures)43