[Bug 656197] New: Unable to log in due to disk full because unbootable because syslog registered one message many thousands of times
https://bugzilla.novell.com/show_bug.cgi?id=656197 https://bugzilla.novell.com/show_bug.cgi?id=656197#c0 Summary: Unable to log in due to disk full because unbootable because syslog registered one message many thousands of times Classification: openSUSE Product: openSUSE 11.2 Version: Final Platform: 32bit OS/Version: openSUSE 11.2 Status: NEW Severity: Normal Priority: P5 - None Component: Printing AssignedTo: jsmeix@novell.com ReportedBy: dmp5@uwm.edu QAContact: jsmeix@novell.com Found By: --- Blocker: --- Created an attachment (id=402380) --> (http://bugzilla.novell.com/attachment.cgi?id=402380) The log file for messages and wern (same as paste) User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101026 SUSE/3.6.12-0.7.1 Firefox/3.6.12 Here is the type of printer used: Description: HP Deskjet 6940 series Location: Office Printer Driver: HP Deskjet 6940 Series, hpcups 3.9.8 Printer State: idle, accepting jobs, published. Printer works fine, wireless The problem was I was out of black ink (I didn't realize this) and I had 6 jobs waiting to print, 5 were test pages because my 1 document wouldn't print, so I kept trying to see if it was a wireless issue with each new test page, eventually I gave up. I'm not sure when but a few days laster I could not log in. I would get to my splash screen. I would type my username and password and the process would keep up and never log in. I used a a live disc to learn than my /var/log was at 27g. Messages and wern held the majority 13.8g each. As going through the forums process in opensuse, Here is my post http://forums.opensuse.org/english/get-help-here/install-boot-login/450335-c... I learned, "syslog registered one message many thousands of times, several gigabytes of logs, instead of filtering the repeated message. Here are the two files that were the trouble http://susepaste.org/860249 After learning the two files were able to be deleted, I am back up and running as usual. Reproducible: Didn't try Steps to Reproduce: 1. 2. 3. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c1
Johannes Meixner
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c2
Carlos Robinson
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c3
Johannes Meixner
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c
Johannes Meixner
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c4
Johannes Meixner
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c6
Carlos Robinson
By default we use syslog-ng (I don't know since which openSUSE version) but syslog-ng does not filter out duplicates (what the old syslogd did).
That is not entirely correct. 11.2 and 11.3 use rsyslog by default (I just checked three installs of 11.2 and one of 11.3). 11.0 used syslog-ng. 11.1 I don't know. Please, lets wait till David can confirm what he has before closing this one. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c7
Carlos Robinson
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c8
Marius Tomaschewski
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c9
--- Comment #9 from peter czanik
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c10
--- Comment #10 from Marius Tomaschewski
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c11
--- Comment #11 from David Pulkowski
David:
Please add the result of this command:
grep SYSLOG_DAEMON /etc/sysconfig/syslog
I'm not sure if you got this or not, The return was this
SYSLOG_DAEMON="rsyslogd"
-- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c
Xinli Niu
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c12
Carlos Robinson
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c13
Marius Tomaschewski
This is a bug either in rsyslog or in its openSUSE configuration
It is neither a bug in rsyslog, nor in openSUSE, but as already written in comment 8, it is a configration option and openSUSE is using the rsyslogd default setting of "$RepeatedMsgReduction off". I'm quite sure, somebody will report the change as bug [that messages dups are dropped], but I've changed to explicit "on" in Base:System / Factory, because it useful to catch amok running programs logging exactly same log message... But this IMO does not change the initial resolution as UPSTREAM: the bug is in hpijs/services.cpp. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c14
Marius Tomaschewski
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c15
--- Comment #15 from Carlos Robinson
It is neither a bug in rsyslog, nor in openSUSE, but as already written in comment 8, it is a configration option and openSUSE is using the rsyslogd default setting of "$RepeatedMsgReduction off".
Question: When a message is repeated many times and filtered out, doesn't rsyslog print a message of the type "skipped 100 identical messages", or something like that? I have seen that on openSUSE logs, time ago... If that is not so, I think it would be useful perhaps to leave messages on in factory, and filter them in the stable releases. As to the bug being solely the fault of hpijs/services.cpp, I mildly object: a faulty program should not crash a system (disk full). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c16
--- Comment #16 from Johannes Meixner
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c17
--- Comment #17 from Marius Tomaschewski
It is neither a bug in rsyslog, nor in openSUSE, but as already written in comment 8, it is a configration option and openSUSE is using the rsyslogd default setting of "$RepeatedMsgReduction off".
Question:
When a message is repeated many times and filtered out, doesn't rsyslog print a message of the type "skipped 100 identical messages", or something like that? I have seen that on openSUSE logs, time ago...
As I already wrote: - syslogd does this by default. - for syslog-ng-1.6 I wrote a patch to do it by default as well. in syslog-ng-2.0.9, there is no suppress() option doing it as in syslog-ng-3.1. - in rsyslogd it is a option with built-in default 'off'. For openSUSE-11.4 I've enabled the RepeatedMsgReduction option and I'll enable the suppress() option for syslog-ng as well in the next submit to catch this special case. Because it is an option in rsyslog (=default), I've asked in comment 14, whether I have to provide an update package with new /etc/rsyslog.conf, that enables this option or not.
If that is not so, I think it would be useful perhaps to leave messages on in factory, and filter them in the stable releases.
As to the bug being solely the fault of hpijs/services.cpp, I mildly object: a faulty program should not crash a system (disk full).
It is not possible to prohibit this. The option catches a special case only: programs that log *exactly* same message. As soon as a message differs even one bit, the message gets logged. So a user or program just need to run e.g. the following to fill up the log-disk: i=0 ; while true ; do logger "======== $((i++)) ==========" ; done ==>> Best is to always put /var/log to a separate disk. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c18
--- Comment #18 from Marius Tomaschewski
- for syslog-ng-1.6 I wrote a patch to do it by default as well. in syslog-ng-2.0.9, there is no suppress() option doing it as in syslog-ng-3.1.
BTW: Also here: suppress() is an option in 3.1 and 'off' by default. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c19
Christian Dengler
Because it is an option in rsyslog (=default), I've asked in comment 14, whether I have to provide an update package with new /etc/rsyslog.conf, that enables this option or not.
IMHO we should only change the behavior in the next openSUSE release (-1). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c20
Christian Dengler
https://bugzilla.novell.com/show_bug.cgi?id=656197
https://bugzilla.novell.com/show_bug.cgi?id=656197#c21
Marius Tomaschewski
participants (1)
-
bugzilla_noreply@novell.com