[doveman2]

I believe(*) the ondemand governor is the default on OpenELEC. So the first few seconds of boot will be at 700MHz, then the ondemand governor is enabled and the clock frequency becomes dynamic and will switch up to (e.g.) 900MHz whenever its busy. This is during kernel boot, and well before xbmc is launched.

This is different from raspian which defaults to the conservative governor and switches to the ondemand governor (assuming shift key is not pressed) from an init script.

OK but that still doesn't explain what initial_turbo does. You seem to be saying that OpenELEC switches to ondemand a few seconds after boot, regardless of what initial_turbo is set to.

[popcornmix]

OK but that still doesn't explain what initial_turbo does. You seem to be saying that OpenELEC switches to ondemand a few seconds after boot, regardless of what initial_turbo is set to.

initial_turbo forces turbo mode (from the GPU) side for the first <n> seconds. Doesn't matter what frequency arm requests, you will get turbo.

[Milhouse]

I must say I'm not so keen on writing a file to detect filsystem corruption. Although you may not care whether the contents of the file are corrupted, it seems quite possible that the FAT will get corrupted rendering start.elf/kernel.img unreadable.

Me neither, but it's the lesser of two evils. If there were a more reliable, and less destructive way of detecting the state of the SD card I'd use it, but other than enabling dmesg, and waiting for the timeout messages (which may or may not appear) I'm not aware of a better solution.

From the log, which message guarantees you are in corrupting state? We seem to have:
<6>[ 4.428548] mmc0: missed completion of cmd 18 DMA (512/512 [1]/[1]) - ignoring it
<6>[ 4.428592] mmc0: DMA IRQ 6 ignored - results were reset
<3>[ 4.913573] mmc0: final write to SD card still running
<3>[ 14.920858] mmc0: Timeout waiting for hardware interrupt - cmd12.
<3>[ 14.922006] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x900

The real solution is finding what part of rebooting gets it into a good state. Presumably some sort of sdhost or sdcard reset. Can the kernel perform this reset without a reboot? (or even with a reboot)

Not to my knowledge, which is limited to basically hacking the init script.

The problem remains one of determining the SD card status sufficiently early - dmesg only reveals there is a critical issue with the SD card at 14.9 seconds into the boot sequence, by which time you'd normally be well on the way to loading XBMC. Is there a way to query the SD card subsystem and determine if it is usable or experiencing a problem that maybe solved by a reboot (or reset, if that is possible)?

---

As described in the update procedure I put the SYSTEM file in the Update folder manually.
This file contains the third party binaries..!?? So why putting files from 3rdparty/bootloader files also to the SD card??

If you are using OpenELEC to update itself (by dropping the SYSTEM/KERNEL files into the Update folder) then that's all there is to do - but this process, even when performed manually, can still lead to corruption of your SD card (unless you are not overclocking the Pi, but even then...)

If you are using a PC to update the SD card, ie. copying files by hand, so that there is no Raspberry Pi involvement and no chance of SD corruption, then you also need to copy the 3rdparty files to the SD card.

[popcornmix]

I must say I'm not so keen on writing a file to detect filsystem corruption. Although you may not care whether the contents of the file are corrupted, it seems quite possible that the FAT will get corrupted rendering start.elf/kernel.img unreadable.

Me neither, but it's the lesser of two evils. If there were a more reliable, and less destructive way of detecting the state of the SD card I'd use it, but other than enabling dmesg, and waiting for the timeout messages (which may or may not appear) I'm not aware of a better solution.

From the log, which message guarantees you are in corrupting state? We seem to have:
<6>[ 4.428548] mmc0: missed completion of cmd 18 DMA (512/512 [1]/[1]) - ignoring it
<6>[ 4.428592] mmc0: DMA IRQ 6 ignored - results were reset
<3>[ 4.913573] mmc0: final write to SD card still running
<3>[ 14.920858] mmc0: Timeout waiting for hardware interrupt - cmd12.
<3>[ 14.922006] mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x900

The real solution is finding what part of rebooting gets it into a good state. Presumably some sort of sdhost or sdcard reset. Can the kernel perform this reset without a reboot? (or even with a reboot)

Not to my knowledge, which is limited to basically hacking the init script.

But if you use your init script reboot hack - with dmesg log enabled, and it successfully boots, are there no mmc errors on boot?
I've a suspicion I've seen the first two messages (the DMA ones) but with no ill effect. Whereas the final three are unfamiliar to me and sound more worrying (and hence a better choice for rebooting).

[Milhouse]

But if you use your init script reboot hack - with dmesg log enabled, and it successfully boots, are there no mmc errors on boot?
I've a suspicion I've seen the first two messages (the DMA ones) but with no ill effect. Whereas the final three are unfamiliar to me and sound more worrying (and hence a better choice for rebooting).

It seems quite normal to have DMA messages (maybe they should be suppressed) but its the last three (the timeout errors) that seem to be related to the slow performance, and thus the eventual file/partition corruption. It's just that it's quicker to determine slow performance (or the lack of it) by writing a file than waiting for dmesg to report that there is (or isn't?) a problem. An SD card that is behaving itself doesn't generate these timeout messages.

[popcornmix]

It seems quite normal to have DMA messages (maybe they should be suppressed) but its the last three (the timeout errors) that seem to be related to the slow performance, and thus the eventual file/partition corruption. It's just that it's quicker to determine slow performance (or the lack of it) by writing a file than waiting for dmesg to report that there is (or isn't?) a problem. An SD card that is behaving itself doesn't generate these timeout messages.

I'm thinking of a command line option for the kernel that means it deliberately reboots when it sees
"mmc0: final write to SD card still running"

That should kick in 5 seconds into boot, so will be quicker that writing a 1M file (after kernel has booted). Also avoids the corruption possibility of writing a file.

[popcornmix]

@MilhouseVH
Can you build a new kernel?
Apply this patch:
https://dl.dropbox.com/u/3669512/temp/sd...atal.patch
and add to cmdline.txt:
sdhci-bcm2708.reboot_fatal=Y

[Milhouse]

@MilhouseVH
Can you build a new kernel?
Apply this patch:
https://dl.dropbox.com/u/3669512/temp/sd...atal.patch
and add to cmdline.txt:
sdhci-bcm2708.reboot_fatal.patch=Y

Sure, will do. I just built a kernel that checked for "final write still running" and it didn't appear in dmesg until 12 seconds - so it wasn't detected early (but the write check noticed it the SD was performing slowly) so the early appearance of these "final write" messages is possibly not something that can be relied upon.

Will try the patch and get back to you.... presumably I should disable my own checks?

[popcornmix]

Sure, will do. I just built a kernel that checked for "final write still running" and it didn't appear in dmesg until 12 seconds - so it wasn't detected early (but the write check noticed it the SD was performing slowly) so the early appearance of these "final write" messages is possibly not something that can be relied upon.

Were there any mmc0 errors at (or before) your write test failed?

Will try the patch and get back to you.... presumably I should disable my own checks?

Yes.

[Milhouse]

Were there any mmc0 errors at (or before) your write test failed?

Not that I recall, the mmc messages all kicked in at about 12 seconds. I plugged in a USB stick to grab the dmesg but this caused the Pi to spontaneously reboot. ;-)

[Milhouse]

and add to cmdline.txt:
sdhci-bcm2708.reboot_fatal.patch=Y

Should that be:

sdhci-bcm2708.reboot_fatal=Y

?

Might explain why it's not working...

[popcornmix]

and add to cmdline.txt:
sdhci-bcm2708.reboot_fatal.patch=Y

Should that be:

sdhci-bcm2708.reboot_fatal=Y

?

Might explain why it's not working...

Ooops. Yes, the .patch only belongs to the patch file, not the command line option...

[doveman2]

OK but that still doesn't explain what initial_turbo does. You seem to be saying that OpenELEC switches to ondemand a few seconds after boot, regardless of what initial_turbo is set to.

initial_turbo forces turbo mode (from the GPU) side for the first <n> seconds. Doesn't matter what frequency arm requests, you will get turbo.

OK, thanks but does it matter then whether intial_turbo is used or not, as the corruption occurs when writing to the SD with the Pi overclocked and I presume the writing won't occur in the first few seconds and after that OpenElec will have switched to ondemand (i.e. overclocked) anyway, so when overclocking the writing will be done in an overclocked state, whether intial_turbo is used or not?

[popcornmix]

OK, thanks but does it matter then whether intial_turbo is used or not, as the corruption occurs when writing to the SD with the Pi overclocked and I presume the writing won't occur in the first few seconds and after that OpenElec will have switched to ondemand (i.e. overclocked) anyway, so when overclocking the writing will be done in an overclocked state, whether intial_turbo is used or not?

I don't believe initial_turbo makes corruption more or less likely.

[Milhouse]

@MilhouseVH
Can you build a new kernel?
Apply this patch:
https://dl.dropbox.com/u/3669512/temp/sd...atal.patch
and add to cmdline.txt:
sdhci-bcm2708.reboot_fatal.patch=Y

Unfortunately, it appears that a write to the SD card is required to trigger the timeout.

I've applied your patch (and disabled my hack), although I modified the patch slightly to use a kernel_halt rather than emergency_reboot so that I could analyse any info on the screen at the time of the halt.

Having primed the system for an auto-update (ie. new SYSTEM and KERNEL plus md5's in /storage/.update), I rebooted and saw the following sequence of events:

Code:

### Loading kernel modules ###
### Checking disks ###
### Mounting flash ###
### mount filesystem /dev/mmcblk0p1 ... ###
### Loading bootsplash ###
### Mounting storage ###
### mount filesystem /dev/mmcblk0p2 ... ###
### Checking for updates ###
Checking KERNEL.md5...
/storage/.update/KERNEL: OK
Checking SYSTEM.md5
/storage/.update/SYSTEM: OK
Updating Kernel on local...
[    8.183597] System halted.

As you can see, it got as far as mounting the partitions, and started writing to kernel.img, before the timeout occurred and the system then halted.

Subsequently inserting this SD card into a Ubuntu PC, I can only see the primary FAT partition - the ext4 partition is hosed and can't be mounted.

Code:

[1737049.219613] sdd: detected capacity change from 4029677568 to 0
[1737276.808598] sd 20:0:0:0: [sdd] 7870464 512-byte logical blocks: (4.02 GB/3.75 GiB)
[1737276.814077] sd 20:0:0:0: [sdd] No Caching mode page present
[1737276.814080] sd 20:0:0:0: [sdd] Assuming drive cache: write through
[1737276.819980] sd 20:0:0:0: [sdd] No Caching mode page present
[1737276.819982] sd 20:0:0:0: [sdd] Assuming drive cache: write through
[1737276.820743]  sdd: sdd1 sdd2
[1737277.176731] EXT4-fs (sdd2): ext4_check_descriptors: Checksum for group 0 failed (4739!=0)
[1737277.176734] EXT4-fs (sdd2): group descriptors corrupted!

kernel.img is also corrupt, as it has been partially written to.

So it looks like too much damage will already have been done before the timeout triggers and halts/reboots the system.