[henke66]

Is that a good or bad thing? Should i be concerned?
Maybe he will do magic when he sees red (or writes red).

[davilla]

I found an extensive document about shared libraries (aka dynamic linked libraries), which I haven't had time to read yet.
If you wan't to understand the implications about dynamic linking this seems to be a great reference.
How To Write Shared Libraries

what was that thing about grandmothers and sucking eggs and by the way, darwin is not ELF.

[henke66]

what was that thing about grandmothers and sucking eggs and by the way, darwin is not ELF.

First, I was not trying to be rude! And when I wrote 'you' in the post about shared libraries, it was not adressed to you personnaly but to the community, there might be others reading as well.

I really don't understand if you rather be left alone here or if it is allowed to come up with ideas on how to solve the problem.

If we are still discussing the problem here, my thinking (wthout knowing) is that the assembler code preventing the ffmpeg to build as a dylib, is the same that will cause the kernel panic when it is statically linked.
So what I was suggesting earlier is that the ffmpeg team need to make changes to their code so that it will be able to use as a dylib. If they manage with that it would probably work also for the ATV2. If you are of another opinion, please explain...

So if darwin is not elf, would you care to explain the difference?

[davilla]

lighten up, cowboy. I tend to joke a lot

Darwin uses Mach object file format, Linux and some others use ELF. The difference is huge and cannot be explained in a few simple sentences.

As an example, darwin (Mach object file format) can support multiple archs in the same file and the loader will make sure the correct arch is loaded. ELF cannot support multiple archs in the same file.

[linusyang]

lighten up, cowboy. I tend to joke a lot

Darwin uses Mach object file format, Linux and some others use ELF. The difference is huge and cannot be explained in a few simple sentences.

As an example, darwin (Mach object file format) can support multiple archs in the same file and the loader will make sure the correct arch is loaded. ELF cannot support multiple archs in the same file.

Hi, davilla. Is there any chance to make a branch with the old ffmpeg library so that we could get the build to work on the device temporarily?

[Cranial]

lighten up, cowboy. I tend to joke a lot

Darwin uses Mach object file format, Linux and some others use ELF. The difference is huge and cannot be explained in a few simple sentences.

As an example, darwin (Mach object file format) can support multiple archs in the same file and the loader will make sure the correct arch is loaded. ELF cannot support multiple archs in the same file.

Hi, davilla. Is there any chance to make a branch with the old ffmpeg library so that we could get the build to work on the device temporarily?

Not a coder, but I think this would be more trouble than it's worth. I assume a lot of code has changed with the implementation of the new ffmpeg, so time spent getting it working with the old version is time not spent trying to fix mainline to work with the ATV2.

Eden still works, and works well....

[henke66]

1st, on an real iOS device does not have this problem as XBMC is a real app and as such it nor ffmpeg is dyloaded.

dlopen() crashes the system when loading library that contains references to global symbols from the assembler code (neon). Starting from iOS 5 dlopen() is no longer able to write the symbol addresses to the .text segment during the library load (required to offset symbol address). We used a patch that modified the neon files in FFMpeg to use relative symbol addresses that do not require modification of the .text segment during load. That patch no longer applies and there are more areas to fix up.

Generally if you grep ffmpeg's arm asm code, I think (limited arm-asm-fu) anything that does movrel with an X(xxxx) var would need patching if the X(xxxx) var references a global.

This is very similar to a ppc patch that went into ffmpeg about two years ago to handle an Apple bork under OSX 10.5.

See https://github.com/xbmc/xbmc/blob/Eden/l...iOS5.patch

I have a test app for testing this, much quicker than building xbmc, uploading, test. specially since xbmc's frapp gets loaded each time on boot and you generally have to re-jailbreak to fix. With the test app, the kernel will panic and you just reboot.

I have spent some time looking at this...I guess the relocation is not a problem for c-files, right?
When looking at a disassembled c-file the instructions for loading a global variable is using a pair of movw and movt like :

Code:

00000064    e3018c2c    movw    r8, :lower16:0x1ca0-0x6c+0xfffffff8
00000068    e3408000    movt    r8, :upper16:0x1ca0-0x6c+0xfffffff8

If this works fine we could use the same instructions in the assembler code. Since the problem is caused in the movrel macro, I beleive we could use the following implementation for it in ffmpeg/libavcodec/arm/asm.S

Code:

.macro movrel rd, val
        movw \rd, #:lower16:\val
        movt \rd, #:upper16:\val
.endm

This should take care of the problem in all locations. What is your opinion? Am I missing something? Is this worth trying?

[davilla]

val is still wrong because the loader cannot fix up the address, infact you will kernel panic miles before that code even runs. you die in loading the dyload, not running the asm code.

[henke66]

val is still wrong because the loader cannot fix up the address, infact you will kernel panic miles before that code even runs. you die in loading the dyload, not running the asm code.

So you are saying that the kernel panic occurs when dlopen is trying to patch the code according to the relocation entries.
Is it known what kind of relocations that will fail?
Is the change in ios5/dlopen documented? Bbecause there is one, right?

Relocation seems to be working well for c-files, so my theory is that we need to look at the difference between the relocation tables for a typical c-file and a neon-assembler one.

[davilla]

yes.
global in .text sections. see my original post
no and iOS apps are not permitted to dlopen, unless the change might be in darwin sources.
yes, c/c++/objc, the compiler knows what to do.

[henke66]

yes.
global in .text sections. see my original post
no and iOS apps are not permitted to dlopen, unless the change might be in darwin sources.
yes, c/c++/objc, the compiler knows what to do.

"global in .text sections." doesn't seem to be the hole thing. C-files also references global varibales in text-segments and you are saying that this is working fine, since the compiler "knows what to do".
So I beleive by applying the patch in arm/asm.s I proposed earlier we are very close to the assembler that the c-compiler will generate.

Code:

.macro movrel rd, val
        movw \rd, #:lower16:\val
        movt \rd, #:upper16:\val
.endm

This fairly simple c-code:

Code:

const int myConstVar = 47;
int myVar;

int myFunc(void)
{
   myVar = myConstVar;
   return myVar;
}

will generate the following assembler code

Code:

(__TEXT,__text) section
_myFunc:
00000000    e30002c4    movw    r0, :lower16:0x2d8-0xc+0xfffffff8
00000004    e3a0102f    mov    r1, #47    @ 0x2f
00000008    e3400000    movt    r0, :upper16:0x2d8-0xc+0xfffffff8
0000000c    e79f0000    ldr    r0, [pc, r0]
00000010    e5801000    str    r1, [r0]
00000014    e3a0002f    mov    r0, #47    @ 0x2f
00000018    e12fff1e    bx    lr

So it will use the same instructions as the proposed patched movrel macro. Below are resulting assembler code for the following assembler code in fft_neon.s

Code:

movrel          r2,  X(ff_cos_16)

Code:

00000190    e3002000    movw    r2, :lower16:_ff_cos_16
00000194    e3402000    movt    r2, :upper16:_ff_cos_16

If we compare the relocation tables for these instructions they are also quite similar (but not identical).
fft_neon.s

Code:

00000194 False hi/arm True   HALF    False     _ff_cos_16
         False hi/arm False  PAIR    False     other_half = 0x0000
00000190 False lo/arm True   HALF    False     _ff_cos_16
         False lo/arm False  PAIR    False     other_half = 0x0000

c-file

Code:

00000008 False hi/arm n/a    HALFDIF True      0x000002d8
         False hi/arm n/a    PAIR    True      0x0000000c other_half = 0x02c4
00000000 False lo/arm n/a    HALFDIF True      0x000002d8
         False lo/arm n/a    PAIR    True      0x0000000c other_half = 0x0000

The original code had relocation entries like this, they are gone with the patch. Maybe these are the ones that causes trouble.

Code:

00000758 False long   True   VANILLA False     _ff_cos_16

If this haven't been tested before, I think it is worth a shot.

[gumba]

So, in conclusion, the problem is trying to execute code in the text segment?

I too work as a commercial iOS/OSX developer. We had issues with old PowerPC/x86 code on OSX. When apple introduced the Sandbox to protect people from exploits.

Basically, we had made our assembly dynamically loadable once upon a time by putting the global names in the text segment, which then linked to the actual code.

When Sandbox came out it made it an error (requiring app termination) to jump to the text segment. So, if you load a dylib and jump to a function which is defined in the .text segment then your process will get terminated.

The solution is of course to put the function in the .code segment. The problem then is you have to handle the setup/tear down dynamic stuff properly, which was of course the reason for using the .text stuff in the first place.

This means writing some macros and modifying EVERY assembly function that there is. We did this for our code... someone will probably have to do this for libav's code.

Re: XBMC and iOS assuming static linking... as Apple won't allow dynamic linking for app store distirbuted apps, its a fair assumption to make that most iOS developers only care about static builds. All iOS developers... except ATV developers that is!

So, if the issue is this .text vs .code stuff... the first thing to do is make a simply dylib (helloworld.dylib), get that to load... and crash... work out how to fix that simple library. Then someone has to apply the fix to all of libav...

[Memphiz]

This has to be fixed by the neon-asm dev of libav who did all that assembler stuff. I'm pretty sure he won't accept a fix from another dev because its most likly that its not done in the way he wants it to be done (i talked to him on linuxtage). So imho the goal is not to fix this by ourselfs (and maybe get a patch which will never be applied upstream), but to get the dev to work on the problem. We have to be a bit diplomatic here and don't do to hard pressure because you all know that we really use ffmpeg and libav + ffmpeg != best friends.

So all we will do (speaking of Team XBMC here) is to try to steer the dev towards the problem once in a while and hope that he will pick it up.

Sounds lazy? Pfff....

[blazespinnaker]

nt

[blazespinnaker]

Looks like some people are dyloading code. Check out this vid:

http://www.consumerizeit.com/blogs/consu...ement.aspx

Check at 2 minutes. "We pull in a lot of our code as a dynamic library"