[fmafma]

Hi,

Just to say that yesterday night, XBMC filled my entire /home partition with its log file, which grew up to 115GB!!!! Yes, 115 Giga-Bytes...

Is there a logrotate mecanism coming with XBMC?

[Martijn]

Sounds more like there is something wrong there as no one ever mentioned something like that so i doubt it will justify adding it.
Unless you can provide the needed code patches

[nickr]

Yes it happened to me once too. Can't recall how big it got, but it filled the system up! It became too unwieldy to even open it and see what had gone wrong.

Is there any reason why logrotate can't work on the log file anyway? Although in my case it happened very quickly, like within a few hours, and logrotate is generally set to work on a daily or weekly basis.

I guess a workaround might be to mount ~/.xbmc/temp to it's own partition, or some virtual ram filesystem?

[fmafma]

As nickr, I was unable to open the file, so I can't even tell what happened! Too bad for a log file

Anyway, I guess there are already some libs which manage logs in a more efficient way; no need to write them from scratch. But if not, a simple system to prevent logs happening too quickly is not really complicated. It just needs a buffer which takes, say, up to 100 messages and write them at a low rate on disk. If messages comes faster than the write-on-disk rate, messages are just discarded. This way, it can handle fast messages up to 100, which is enough for some events.

I think that what has filled my log file was the same message coming over and over, at a very fast rate.

[Kib]

I assume you had debugging turned on ?

Several identical messages in rapid succession are already being suppressed in the log by the way.
I am interested to see what was in the log.

[nickr]

I didn't have debugging turned on when I had the problem. Just reading a file that big is difficult, particularly when the filesystem is full. Deletion without seeing what is there was the only solution for me (Believe me I have been using linux for years [redhat 4.x] and know my way around)

The problem didn't repeat. But it could.

logrotate might not be the correct tool for something that grows very quickly when a problem arises. You could run it, say, every 30 minutes using the size criteria:

Code:

size size
    Log files are rotated only if they grow bigger then size bytes. If size is followed by k, the size is assumed to be in kilobytes. If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100Gare all valid.

However there are plenty of file monitoring tools that could be used. Hell a daemon that did nothing but (pseudo code)

while true
if size (~use/.xbmc/temp/xbmc.log) > 2GB then (take some action)
repeat

would be pretty easy to write.

But harder to make cross platform.

[Martijn]

But harder to make cross platform.

and there is the whole problem

[leechguy]

I think in the past there's also been a feature request from someone who was running XBMC on Windows (no logrotate deamon on Windows as far as I know).

If there is interest, I can take a look at implementing rolling logfiles. Once the xbmc.log grows bigger then e.g. 10MB we roll it; xbmc.log.9 is deleted, xbmc.log.8 becomes xbmc.log.9, xbmc.log.7 becomes xbmc.log.8 and so on and finally xbmc.log becomes xbmc.log.1 and a new xbmc.log is created.

I would like to keep that XBMC starts with a fresh log on every startup. So it would also be rolling the logs on startup.

[fmafma]

Debug was not turned on for me neither... XBMC was just playing a web radio when it generated the huge file.

[fmafma]

I would like to keep that XBMC starts with a fresh log on every startup. So it would also be rolling the logs on startup.

BTW, the current xbmc.log needs to remains the same file over and over during rotations, and even at XBMC startup. I don't know how it is done technically¹, but this is mandatory for allowing applications to keep reading the log file whithout the need to re-open it at each rotation.

Currently, when you use tail -f xbmc.log, you need to relaunch this command each time you restart XBMC, which is annoying when debuging...

¹ I think you need to copy the file content, then empty the current one. But I don't know how to do this.

[nickr]

Try running tail -F xbmc.log

[leechguy]

I don't think that there is any application that actually copies over the contents of the file when it rolls it to *.1 (neither does XBMC when it rolls the xbmc.log to xbmc.log.old on startup). That is too time consuming and you don't want to block your application on logging. Logging should be as lightweight as possible.

What you generally see is that the application tailing the log file supports the opening of new log files after they have rolled. In your specific case of tail, instead of 'tail -f xbmc.log' use 'tail -F xbmc.log'. That will cause tail to re-open the file when XBMC at startup renames the existing xbmc.log to xbmc.log.old and opens a new xbmc.log (if xbmc.log does not exist yet, then tail will simply retry).

[fmafma]

Thanks for the tips with tail!

But I'm pretty sure I have read the mecanism I described, about not deleting the current log file. I ill try to retreive the paper... I think it was done by an external stuff, not by the application itself. So, no blocking issue.