Ok, yesterday happend another crash... This is the free result: Sun Oct 16 22:05:00 CEST 2005 total used free shared buffers cached Mem: 1003 963 40 0 71 691 -/+ buffers/cache: 200 803 Swap: 1027 19 1008 Total: 2031 982 1048 Sun Oct 16 22:10:00 CEST 2005 total used free shared buffers cached Mem: 1003 966 37 0 71 691 -/+ buffers/cache: 203 800 Swap: 1027 19 1008 Total: 2031 985 1045 Mon Oct 17 08:20:00 CEST 2005 total used free shared buffers cached Mem: 1003 454 549 0 63 247 -/+ buffers/cache: 143 860 Swap: 1027 0 1027 Total: 2031 454 1577 I restarted the server now with ACPI=off. Untill this in HTOP I had 4 CPU's listed (I have a dual CPU server) and now the CPU 3 and 4 are blue and almost always at 100% or nan. The CPU 1 and 2 are normal. It this normal??? I hope both Cpu's are working.... Regards, Andy. ----- Original Message ----- From: "Andy" To: Sent: Thursday, October 13, 2005 9:35 PM Subject: Re: [suse-security] Strange server crash

Ok, I log the free memory in a log file to see what happens. I'll keep you in touch.

...

The next peice is kernel panics. Type: cat /proc/sys/kernel/panic What's the output? cat /proc/sys/kernel/panic 0

Andy.

----- Original Message ----- From: To: Sent: Thursday, October 13, 2005 6:54 PM Subject: Re: [suse-security] Strange server crash

...

Could it be a DOS attack from somewhere ?

Not nit-picking but if you do 'free -m -s5'

that will return the memory usage in MB's - a bit easier to understand I think ;-).

Keith

On Thu, 13 Oct 2005, Andy wrote:

...

To: suse@karsites.net, suse-security@suse.com From: Andy Subject: Re: [suse-security] Strange server crash

I will respond to all:

To restart the server we had to do a hard reboot.

We have 1G Ram, 1G Swap space. There is plenty(more GBs) of space on the hard disks. On all partitions(except /boot).

The hang-up occurs randomly... sometimes at 3 weeks, sometimes at 2 months, sometimes at 1 month.

free -s -m5: total used free shared buffers cached Mem: 1027964 958260 69704 0 45416 729952 -/+ buffers/cache: 182892 845072 Swap: 1052248 90976 961272

-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here

-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here

On /tmp I have more than 5GB. The system is a dual CPU. This is the listing from /var/log/allmessages and /var/log/warn after 22:12 was the "dead".... ... Oct 16 21:50:00 fileserver /USR/SBIN/CRON[18272]: (root) CMD (/home/scripts/photo_ftp_downloader.sh) Oct 16 21:52:00 fileserver /USR/SBIN/CRON[18375]: (root) CMD (/home/scripts/photo_temp_parse.sh) Oct 16 21:59:00 fileserver /USR/SBIN/CRON[18635]: (root) CMD ( rm -f /var/spool/cron/lastrun/cron.hourly) Oct 16 22:02:00 fileserver /USR/SBIN/CRON[18780]: (root) CMD (/home/scripts/photo_temp_parse.sh) Oct 16 22:10:00 fileserver /USR/SBIN/CRON[19079]: (root) CMD (/home/scripts/db_vacuum_all) Oct 16 22:10:00 fileserver /USR/SBIN/CRON[19080]: (root) CMD (/home/scripts/dfu_imsmca_in) Oct 16 22:12:00 fileserver /USR/SBIN/CRON[19166]: (root) CMD (/home/scripts/photo_temp_parse.sh) Oct 17 08:11:10 fileserver syslogd 1.4.1: restart. Oct 17 08:11:11 fileserver webmin[2015]: Webmin starting Oct 17 08:11:14 fileserver modprobe: FATAL: Error inserting hw_random (/lib/modules/2.6.4-52-smp/kernel/drivers/char/hw_random.ko): No such device Oct 17 08:11:14 fileserver /etc/hotplug/pci.agent[1490]: ... can't load module hw_random Oct 17 08:11:14 fileserver /etc/hotplug/pci.agent[1490]: missing kernel or user mode driver hw_random Oct 17 08:11:15 fileserver kernel: klogd 1.4.1, log source = /proc/kmsg started. Oct 17 08:11:15 fileserver kernel: Inspecting /boot/System.map-2.6.4-52-smp Oct 17 08:11:15 fileserver kernel: Loaded 23810 symbols from /boot/System.map-2.6.4-52-smp. Oct 17 08:11:15 fileserver kernel: Symbols match kernel version 2.6.4. Oct 17 08:11:15 fileserver kernel: No module symbols loaded - kernel modules not enabled. Oct 17 08:11:15 fileserver kernel: drivers/usb/core/usb.c: registered new driver usbfs Oct 17 08:11:15 fileserver kernel: drivers/usb/core/usb.c: registered new driver hub Oct 17 08:11:15 fileserver kernel: tg3.c:v2.9 (March 8, 2004) Oct 17 08:11:15 fileserver kernel: eth0: Tigon3 [partno(BCM95705A50) rev 3003 PHY(5705)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:11:d8:33:3c:0e Oct 17 08:11:15 fileserver kernel: tg3: eth0: Link is up at 100 Mbps, full duplex. Oct 17 08:11:15 fileserver kernel: tg3: eth0: Flow control is on for TX and on for RX. Oct 17 08:11:15 fileserver kernel: hw_random: RNG not detected Oct 17 08:11:15 fileserver kernel: agpgart: Unsupported Intel chipset (device id: 3592) Oct 17 08:11:17 fileserver kernel: NET: Registered protocol family 10 Oct 17 08:11:17 fileserver kernel: Disabled Privacy Extensions on device ffffffff803cd7c0(lo) Oct 17 08:11:17 fileserver kernel: IPv6 over IPv4 tunneling driver Oct 17 08:11:17 fileserver sshd[2400]: Server listening on :: port 22. Oct 17 08:11:17 fileserver kernel: Disabled Privacy Extensions on device 000001003cac7800(sit0) Oct 17 08:11:17 fileserver kernel: ACPI: Power Button (FF) [PWRF] Oct 17 08:11:17 fileserver kernel: ACPI: Processor [CPU1] (supports C1, 8 throttling states) Oct 17 08:11:17 fileserver kernel: ACPI: Processor [CPU2] (supports C1) Oct 17 08:11:17 fileserver kernel: ACPI: Processor [CPU3] (supports C1) Oct 17 08:11:17 fileserver kernel: ACPI: Processor [CPU4] (supports C1) Oct 17 08:11:17 fileserver [powersaved][2452]: WARNING in Function ACPI_Interface; line 26: could not set battery alarm, you may need to force polling mode. Oct 17 08:11:17 fileserver [powersaved][2452]: resmgr: server response code 200 Oct 17 08:11:17 fileserver su: (to postgres) root on /dev/pts/27 Oct 17 08:11:17 fileserver su: pam_unix2: session started for user postgres, service su Oct 17 08:11:22 fileserver ifup: No configuration found for sit0 Oct 17 08:11:28 fileserver kernel: eth0: no IPv6 routers present Oct 17 08:11:30 fileserver su: pam_unix2: session finished for user postgres, service su Oct 17 08:11:35 fileserver kernel: lp: driver loaded but no devices found Oct 17 08:11:35 fileserver kernel: drivers/usb/serial/usb-serial.c: USB Serial support registered for Generic Oct 17 08:11:35 fileserver kernel: drivers/usb/core/usb.c: registered new driver usbserial Oct 17 08:11:35 fileserver kernel: drivers/usb/serial/usb-serial.c: USB Serial Driver core v2.0 Oct 17 08:11:45 fileserver /usr/sbin/cron[3172]: (CRON) STARTUP (fork ok) Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/chargen [file=/etc/xinetd.conf] [line=26] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/chargen-udp [file=/etc/xinetd.d/chargen-udp] [line=13] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/cups-lpd [file=/etc/xinetd.d/cups-lpd] [line=14] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/daytime [file=/etc/xinetd.d/daytime] [line=11] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/daytime-udp [file=/etc/xinetd.d/daytime-udp] [line=13] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/echo [file=/etc/xinetd.d/echo] [line=14] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/echo-udp [file=/etc/xinetd.d/echo-udp] [line=13] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/netstat [file=/etc/xinetd.d/netstat] [line=14] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/rsync [file=/etc/xinetd.d/rsync] [line=16] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/servers [file=/etc/xinetd.d/servers] [line=12] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/services [file=/etc/xinetd.d/services] [line=13] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/swat [file=/etc/xinetd.d/swat] [line=13] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/systat [file=/etc/xinetd.d/systat] [line=12] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/time [file=/etc/xinetd.d/time] [line=17] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/time-udp [file=/etc/xinetd.d/time-udp] [line=14] Oct 17 08:11:45 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/vnc [file=/etc/xinetd.d/vnc] [line=14] Oct 17 08:11:45 fileserver kernel: inserting floppy driver for 2.6.4-52-smp Oct 17 08:11:45 fileserver kernel: Floppy drive(s): fd0 is 1.44M Oct 17 08:11:45 fileserver kernel: FDC 0 is a post-1991 82077 Oct 17 08:11:46 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:11:46 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:11:46 fileserver kernel: st: Version 20040318, fixed bufsize 32768, s/g segs 256 Oct 17 08:11:46 fileserver xinetd[3191]: Reading included configuration file: /etc/xinetd.d/vsftpd [file=/etc/xinetd.d/vsftpd] [line=90] Oct 17 08:11:46 fileserver /etc/hotplug/block.agent[3198]: try 1 while waiting for /block/fd0's bus_id Oct 17 08:11:46 fileserver xinetd[3191]: removing chargen Oct 17 08:11:46 fileserver xinetd[3191]: removing chargen Oct 17 08:11:46 fileserver xinetd[3191]: removing printer Oct 17 08:11:46 fileserver xinetd[3191]: removing daytime Oct 17 08:11:46 fileserver xinetd[3191]: removing daytime Oct 17 08:11:46 fileserver xinetd[3191]: removing echo Oct 17 08:11:46 fileserver xinetd[3191]: removing echo Oct 17 08:11:46 fileserver xinetd[3191]: removing netstat Oct 17 08:11:46 fileserver xinetd[3191]: removing rsync Oct 17 08:11:46 fileserver xinetd[3191]: removing servers Oct 17 08:11:46 fileserver xinetd[3191]: removing services Oct 17 08:11:46 fileserver xinetd[3191]: removing swat Oct 17 08:11:47 fileserver /etc/hotplug/block.agent[3198]: try 2 while waiting for /block/fd0's bus_id Oct 17 08:11:46 fileserver xinetd[3191]: removing systat Oct 17 08:11:47 fileserver xinetd[3191]: removing time Oct 17 08:11:47 fileserver xinetd[3191]: removing time Oct 17 08:11:47 fileserver xinetd[3191]: removing vnc1 Oct 17 08:11:47 fileserver xinetd[3191]: removing vnc2 Oct 17 08:11:47 fileserver xinetd[3191]: removing vnc3 Oct 17 08:11:47 fileserver xinetd[3191]: removing vnchttpd1 Oct 17 08:11:47 fileserver xinetd[3191]: removing vnchttpd2 Oct 17 08:11:47 fileserver xinetd[3191]: removing vnchttpd3 Oct 17 08:11:47 fileserver xinetd[3191]: xinetd Version 2.3.13 started with libwrap loadavg options compiled in. Oct 17 08:11:47 fileserver xinetd[3191]: Started working: 1 available service Oct 17 08:11:47 fileserver kernel: Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0 Oct 17 08:11:47 fileserver kernel: Attached scsi generic sg1 at scsi1, channel 0, id 0, lun 0, type 0 Oct 17 08:11:48 fileserver /etc/hotplug/block.agent[3198]: try 3 while waiting for /block/fd0's bus_id Oct 17 08:11:49 fileserver /etc/hotplug/block.agent[3198]: try 4 while waiting for /block/fd0's bus_id Oct 17 08:11:50 fileserver /etc/hotplug/block.agent[3198]: try 5 while waiting for /block/fd0's bus_id Oct 17 08:11:51 fileserver /etc/hotplug/block.agent[3198]: new block device /block/fd0 Oct 17 08:12:00 fileserver /USR/SBIN/CRON[3543]: (root) CMD (/home/scripts/photo_temp_parse.sh) Oct 17 08:15:46 fileserver syslogd 1.4.1: restart. Oct 17 08:15:47 fileserver webmin[1931]: Webmin starting Oct 17 08:15:49 fileserver [powersaved][2193]: WARNING in Function ACPI_Interface; line 26: could not set battery alarm, you may need to force polling mode. Oct 17 08:15:49 fileserver sshd[2232]: Server listening on :: port 22. Oct 17 08:15:49 fileserver su: (to postgres) root on /dev/pts/27 Oct 17 08:15:49 fileserver su: pam_unix2: session started for user postgres, service su Oct 17 08:15:49 fileserver [powersaved][2193]: resmgr: server response code 200 Oct 17 08:15:50 fileserver modprobe: FATAL: Error inserting hw_random (/lib/modules/2.6.4-52-smp/kernel/drivers/char/hw_random.ko): No such device Oct 17 08:15:50 fileserver /etc/hotplug/pci.agent[1491]: ... can't load module hw_random Oct 17 08:15:50 fileserver /etc/hotplug/pci.agent[1491]: missing kernel or user mode driver hw_random Oct 17 08:15:51 fileserver kernel: klogd 1.4.1, log source = /proc/kmsg started. Oct 17 08:15:51 fileserver kernel: Inspecting /boot/System.map-2.6.4-52-smp Oct 17 08:15:51 fileserver kernel: Loaded 23810 symbols from /boot/System.map-2.6.4-52-smp. Oct 17 08:15:51 fileserver kernel: Symbols match kernel version 2.6.4. Oct 17 08:15:51 fileserver kernel: No module symbols loaded - kernel modules not enabled. Oct 17 08:15:51 fileserver kernel: drivers/usb/core/usb.c: registered new driver usbfs Oct 17 08:15:51 fileserver kernel: drivers/usb/core/usb.c: registered new driver hub Oct 17 08:15:51 fileserver kernel: tg3.c:v2.9 (March 8, 2004) Oct 17 08:15:51 fileserver kernel: eth0: Tigon3 [partno(BCM95705A50) rev 3003 PHY(5705)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:11:d8:33:3c:0e Oct 17 08:15:51 fileserver kernel: tg3: eth0: Link is up at 100 Mbps, full duplex. Oct 17 08:15:51 fileserver kernel: tg3: eth0: Flow control is on for TX and on for RX. Oct 17 08:15:51 fileserver kernel: ACPI: Power Button (FF) [PWRF] Oct 17 08:15:51 fileserver kernel: ACPI: Processor [CPU1] (supports C1, 8 throttling states) Oct 17 08:15:51 fileserver kernel: ACPI: Processor [CPU2] (supports C1) Oct 17 08:15:51 fileserver kernel: ACPI: Processor [CPU3] (supports C1) Oct 17 08:15:51 fileserver kernel: ACPI: Processor [CPU4] (supports C1) Oct 17 08:15:51 fileserver kernel: NET: Registered protocol family 10 Oct 17 08:15:51 fileserver kernel: Disabled Privacy Extensions on device ffffffff803cd7c0(lo) Oct 17 08:15:51 fileserver kernel: IPv6 over IPv4 tunneling driver Oct 17 08:15:51 fileserver kernel: Disabled Privacy Extensions on device 0000010038628000(sit0) Oct 17 08:15:51 fileserver kernel: hw_random: RNG not detected Oct 17 08:15:51 fileserver kernel: agpgart: Unsupported Intel chipset (device id: 3592) Oct 17 08:15:54 fileserver ifup: No configuration found for sit0 Oct 17 08:15:56 fileserver su: pam_unix2: session finished for user postgres, service su Oct 17 08:15:58 fileserver kernel: lp: driver loaded but no devices found Oct 17 08:15:59 fileserver kernel: drivers/usb/serial/usb-serial.c: USB Serial support registered for Generic Oct 17 08:15:59 fileserver kernel: drivers/usb/core/usb.c: registered new driver usbserial Oct 17 08:15:59 fileserver kernel: drivers/usb/serial/usb-serial.c: USB Serial Driver core v2.0 Oct 17 08:16:00 fileserver kernel: eth0: no IPv6 routers present Oct 17 08:16:07 fileserver /usr/sbin/cron[3135]: (CRON) STARTUP (fork ok) Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/chargen [file=/etc/xinetd.conf] [line=26] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/chargen-udp [file=/etc/xinetd.d/chargen-udp] [line=13] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/cups-lpd [file=/etc/xinetd.d/cups-lpd] [line=14] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/daytime [file=/etc/xinetd.d/daytime] [line=11] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/daytime-udp [file=/etc/xinetd.d/daytime-udp] [line=13] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/echo [file=/etc/xinetd.d/echo] [line=14] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/echo-udp [file=/etc/xinetd.d/echo-udp] [line=13] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/netstat [file=/etc/xinetd.d/netstat] [line=14] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/rsync [file=/etc/xinetd.d/rsync] [line=16] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/servers [file=/etc/xinetd.d/servers] [line=12] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/services [file=/etc/xinetd.d/services] [line=13] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/swat [file=/etc/xinetd.d/swat] [line=13] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/systat [file=/etc/xinetd.d/systat] [line=12] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/time [file=/etc/xinetd.d/time] [line=17] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/time-udp [file=/etc/xinetd.d/time-udp] [line=14] Oct 17 08:16:07 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/vnc [file=/etc/xinetd.d/vnc] [line=14] Oct 17 08:16:08 fileserver xinetd[3140]: Reading included configuration file: /etc/xinetd.d/vsftpd [file=/etc/xinetd.d/vsftpd] [line=90] Oct 17 08:16:08 fileserver xinetd[3140]: removing chargen Oct 17 08:16:08 fileserver xinetd[3140]: removing chargen Oct 17 08:16:08 fileserver xinetd[3140]: removing printer Oct 17 08:16:08 fileserver xinetd[3140]: removing daytime Oct 17 08:16:08 fileserver xinetd[3140]: removing daytime Oct 17 08:16:08 fileserver xinetd[3140]: removing echo Oct 17 08:16:08 fileserver xinetd[3140]: removing echo Oct 17 08:16:08 fileserver xinetd[3140]: removing netstat Oct 17 08:16:08 fileserver xinetd[3140]: removing rsync Oct 17 08:16:08 fileserver xinetd[3140]: removing servers Oct 17 08:16:08 fileserver xinetd[3140]: removing services Oct 17 08:16:08 fileserver xinetd[3140]: removing swat Oct 17 08:16:08 fileserver xinetd[3140]: removing systat Oct 17 08:16:08 fileserver xinetd[3140]: removing time Oct 17 08:16:08 fileserver xinetd[3140]: removing time Oct 17 08:16:08 fileserver xinetd[3140]: removing vnc1 Oct 17 08:16:08 fileserver xinetd[3140]: removing vnc2 Oct 17 08:16:08 fileserver xinetd[3140]: removing vnc3 Oct 17 08:16:08 fileserver xinetd[3140]: removing vnchttpd1 Oct 17 08:16:08 fileserver xinetd[3140]: removing vnchttpd2 Oct 17 08:16:08 fileserver xinetd[3140]: removing vnchttpd3 Oct 17 08:16:08 fileserver xinetd[3140]: xinetd Version 2.3.13 started with libwrap loadavg options compiled in. Oct 17 08:16:08 fileserver xinetd[3140]: Started working: 1 available service Oct 17 08:16:08 fileserver kernel: inserting floppy driver for 2.6.4-52-smp Oct 17 08:16:08 fileserver kernel: Floppy drive(s): fd0 is 1.44M Oct 17 08:16:08 fileserver kernel: FDC 0 is a post-1991 82077 Oct 17 08:16:08 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:16:08 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:16:08 fileserver /etc/hotplug/block.agent[3161]: try 1 while waiting for /block/fd0's bus_id Oct 17 08:16:08 fileserver kernel: st: Version 20040318, fixed bufsize 32768, s/g segs 256 Oct 17 08:16:08 fileserver kernel: Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0 Oct 17 08:16:08 fileserver kernel: Attached scsi generic sg1 at scsi1, channel 0, id 0, lun 0, type 0 Oct 17 08:16:09 fileserver /etc/hotplug/block.agent[3161]: try 2 while waiting for /block/fd0's bus_id Oct 17 08:16:10 fileserver /etc/hotplug/block.agent[3161]: try 3 while waiting for /block/fd0's bus_id Oct 17 08:16:11 fileserver /etc/hotplug/block.agent[3161]: try 4 while waiting for /block/fd0's bus_id Oct 17 08:16:12 fileserver /etc/hotplug/block.agent[3161]: try 5 while waiting for /block/fd0's bus_id Oct 17 08:16:13 fileserver /etc/hotplug/block.agent[3161]: new block device /block/fd0 Oct 17 08:20:55 fileserver sshd[3857]: reverse mapping checking getaddrinfo for ar-sd.net.155.77.82.in-addr.arpa failed - POSSIBLE BREAKIN ATTEMPT! Oct 17 08:20:58 fileserver sshd[3857]: Accepted keyboard-interactive/pam for root from ::ffff:82.77.155.72 port 44704 ssh2 Oct 17 08:22:00 fileserver /USR/SBIN/CRON[3977]: (root) CMD (/home/scripts/photo_temp_parse.sh) /var/log/warn ... Oct 14 16:02:05 fileserver sshd[17256]: error: PAM: Authentication failure Oct 14 16:13:47 fileserver sshd[17828]: error: PAM: Authentication failure Oct 14 16:13:54 fileserver last message repeated 2 times Oct 15 23:15:01 fileserver sshd[27917]: error: PAM: Authentication failure Oct 17 08:11:15 fileserver kernel: hw_random: RNG not detected Oct 17 08:11:15 fileserver kernel: agpgart: Unsupported Intel chipset (device id: 3592) Oct 17 08:11:17 fileserver [powersaved][2452]: WARNING in Function ACPI_Interface; line 26: could not set battery alarm, you may need to force polling mode. Oct 17 08:11:22 fileserver ifup: No configuration found for sit0 Oct 17 08:11:46 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:11:46 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:15:49 fileserver [powersaved][2193]: WARNING in Function ACPI_Interface; line 26: could not set battery alarm, you may need to force polling mode. Oct 17 08:15:51 fileserver kernel: hw_random: RNG not detected Oct 17 08:15:51 fileserver kernel: agpgart: Unsupported Intel chipset (device id: 3592) Oct 17 08:15:54 fileserver ifup: No configuration found for sit0 Oct 17 08:16:08 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 08:16:08 fileserver kernel: end_request: I/O error, dev fd0, sector 0 Oct 17 09:27:10 fileserver kernel: RAID1 conf printout: Oct 17 09:27:10 fileserver kernel: --- wd:2 rd:2 Oct 17 09:27:10 fileserver kernel: disk 0, wo:0, o:1, dev:sdb3 Oct 17 09:27:10 fileserver kernel: disk 1, wo:0, o:1, dev:sda3 Andy.

I noticed this in the logs as well. Oct 17 08:20:55 fileserver sshd[3857]: reverse mapping checking getaddrinfo for ar-sd.net.155.77.82.in-addr.arpa failed - POSSIBLE BREAKIN ATTEMPT! This is one of the locations from where I connect remotely. Tried to fix this... but still got the error.

From your logs I see that powersaved fails to start. There are also some missing kernel modules that fail to load. This morning after the latest crash I disabled ACPI. This could be also a problem.

Would it be feasable to disable a CPU, and run the server on a single CPU only, using a plain vanilla non-SMP kernel? Uh... the people that are using the application from the server won't be too happy ...but it remains an option.

Andy. ----- Original Message ----- From: To: Sent: Monday, October 17, 2005 7:07 PM Subject: Re: [suse-security] Strange server crash

From your logs I see that powersaved fails to start. There are also some missing kernel modules that fail to load.

Does this problem happen when the server is idle?

If so, can it be a problem when the CPU's try to go into power saving mode?

My other thought was is it a problem with the dual processors?

Would it be feasable to disable a CPU, and run the server on a single CPU only, using a plain vanilla non-SMP kernel?

I noticed this in the logs as well. Oct 17 08:20:55 fileserver sshd[3857]: reverse mapping checking getaddrinfo for ar-sd.net.155.77.82.in-addr.arpa failed - POSSIBLE BREAKIN ATTEMPT!

could that be of relevance?

Keith

-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here

/var/log/slony1/node2/mps_kd.log & Slon successfully started for cluster replic_loco, node node2 PID [2555] Start the watchdog process as well... done Starting mail service (Postfix)done Loading keymap qwerty/us.map.gz doneLoading compose table winkeys shiftctrl latin1.adddone Stop Unicode mode doneLoading console font lat9w-16.psfu -m trivial (K done<notice>exit status of (splash replication postfix kbd cups) is (0 0 0 0

And the boot.msg: Inspecting /boot/System.map-2.6.4-52-smp Loaded 23810 symbols from /boot/System.map-2.6.4-52-smp. Symbols match kernel version 2.6.4. No module symbols loaded - kernel modules not enabled. klogd 1.4.1, log source = ksyslog started. <4>Bootdata ok (command line is auto BOOT_IMAGE=Linux ro root=900 resume=/dev/sda2 acpi=off splash=silent console=tty0 desktop) <4>Linux version 2.6.4-52-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Wed Apr 7 01:58:54 UTC 2004 <6>BIOS-provided physical RAM map: <4> BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) <4> BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) <4> BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved) <4> BIOS-e820: 0000000000100000 - 000000003ffe0000 (usable) <4> BIOS-e820: 000000003ffe0000 - 000000003ffef000 (ACPI data) <4> BIOS-e820: 000000003ffef000 - 000000003fff0000 (ACPI NVS) <4> BIOS-e820: 000000003fff0000 - 0000000040000000 (reserved) <4> BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved) <4> BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved) <6>No NUMA configuration found <6>Faking a node at 0000000000000000-000000003ffe0000 <4>Bootmem setup node 0 0000000000000000-000000003ffe0000 <7>ACPI: have wakeup address 0x10000002000 <6>No mptable found. <6>No mptable found. <6>setting up node 0 0-3ffe0 <4>On node 0 totalpages: 262112 <4> DMA zone: 4096 pages, LIFO batch:1 <4> Normal zone: 258016 pages, LIFO batch:16 <4> HighMem zone: 0 pages, LIFO batch:1 <4>Intel MultiProcessor Specification v1.4 <6> Virtual Wire compatibility mode. <6>OEM ID: ASUSTek <6>Product ID: PRIMEGRY <6>APIC at: 0xFEE00000 <6>Processor #0 15:4 APIC version 20 <6>Processor #6 15:4 APIC version 20 <4>I/O APIC #8 Version 32 at 0xFEC00000. <4>I/O APIC #9 Version 32 at 0xFEC10000. <6>Processors: 2 <4>Checking aperture... <4>Built 1 zonelists <4>Kernel command line: auto BOOT_IMAGE=Linux ro root=900 resume=/dev/sda2 acpi=off splash=silent console=tty0 desktop <6>bootsplash: silent mode. <4>Initializing CPU#0 <4>PID hash table entries: 16 (order 4: 256 bytes) <4>CKRM Initialized <6>time.c: Using 1.193182 MHz PIT timer. <6>time.c: Detected 2800.212 MHz processor. <4>Console: colour dummy device 80x25 <4>Memory: 1027360k/1048448k available (2308k kernel code, 0k reserved, 866k data, 204k init) <4>Calibrating delay loop... 5521.40 BogoMIPS <6>Security Scaffold v1.0.0 initialized <6>SELinux: Initializing. <6>SELinux: Starting in permissive mode <6>There is already a security framework initialized, register_security failed. <6>Failure registering capabilities with the kernel <6>selinux_register_security: Registering secondary module capability <6>Capability LSM initialized <6>Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) <4>Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) <4>Mount-cache hash table entries: 256 (order: 0, 4096 bytes) <6>checking if image is initramfs...it isn't (no cpio magic); looks like an initrd <6>Looking for DSDT in initrd ...No customized DSDT found in initrd! <4>using mwait in idle threads. <6>CPU: Trace cache: 12K uops <6>CPU: L2 cache: 1024K <6>CPU: Physical Processor ID: 0 <4>POSIX conformance testing by UNIFIX <6>Using IO APIC NMI watchdog <6>CPU: Trace cache: 12K uops <6>CPU: L2 cache: 1024K <6>CPU: Physical Processor ID: 0 <6>CPU0: Intel(R) Xeon(TM) CPU 2.80GHz stepping 01 <6>per-CPU timeslice cutoff: 1023.85 usecs. <6>task migration cache decay timeout: 2 msecs. <6>Booting processor 1/6 rip 6000 rsp 10001f37f58 <4>Initializing CPU#1 <4>Calibrating delay loop... 5586.94 BogoMIPS <6>CPU: Trace cache: 12K uops <6>CPU: L2 cache: 1024K <6>CPU: Physical Processor ID: 3 <4> Intel(R) Xeon(TM) CPU 2.80GHz stepping 01 <6>Total of 2 processors activated (11108.35 BogoMIPS). <4>WARNING: No sibling found for CPU 0. <4>WARNING: No sibling found for CPU 1. <4>ENABLING IO-APIC IRQs <6>Using IO-APIC 8 <6>...changing IO-APIC physical APIC ID to 8 ... ok. <6>Using IO-APIC 9 <6>...changing IO-APIC physical APIC ID to 9 ... ok. <7>init IO_APIC IRQs <7> IO-APIC (apicid-pin) 8-0, 8-9, 8-10, 8-11, 8-19, 8-20, 8-22, 8-23, 9-0, 9-1, 9-2, 9-3, 9-4, 9-5, 9-6, 9-7, 9-8, 9-9, 9-10, 9-11, 9-12, 9-13, 9-14, 9-15, 9-16, 9-17, 9-18, 9-19, 9-20, 9-21, 9-22, 9-23 not connected. <6>..TIMER: vector=0x31 pin1=2 pin2=0 <6>activating NMI Watchdog ... done. <6>testing NMI watchdog ... OK. <7>number of MP IRQ sources: 19. <7>number of IO-APIC #8 registers: 24. <7>number of IO-APIC #9 registers: 24. <6>testing the IO APIC....................... <4> <7>IO APIC #8...... <7>.... register #00: 08000000 <7>....... : physical APIC id: 08 <7>.... register #01: 00178020 <7>....... : max redirection entries: 0017 <7>....... : PRQ implemented: 1 <7>....... : IO APIC version: 0020 <7>.... register #02: 00178020 <7>....... : arbitration: 00 <7>.... IRQ redirection table: <7> NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: <7> 00 000 00 1 0 0 0 0 0 0 00 <7> 01 001 01 0 0 0 0 0 1 1 39 <7> 02 001 01 0 0 0 0 0 1 1 31 <7> 03 001 01 0 0 0 0 0 1 1 41 <7> 04 001 01 0 0 0 0 0 1 1 49 <7> 05 001 01 0 0 0 0 0 1 1 51 <7> 06 001 01 0 0 0 0 0 1 1 59 <7> 07 001 01 0 0 0 0 0 1 1 61 <7> 08 001 01 0 0 0 0 0 1 1 69 <7> 09 000 00 1 0 0 0 0 0 0 00 <7> 0a 000 00 1 0 0 0 0 0 0 00 <7> 0b 000 00 1 0 0 0 0 0 0 00 <7> 0c 001 01 0 0 0 0 0 1 1 71 <7> 0d 001 01 0 0 0 0 0 1 1 79 <7> 0e 001 01 0 0 0 0 0 1 1 81 <7> 0f 001 01 0 0 0 0 0 1 1 89 <7> 10 001 01 1 1 0 1 0 1 1 91 <7> 11 001 01 1 1 0 1 0 1 1 99 <7> 12 001 01 1 1 0 1 0 1 1 A1 <7> 13 000 00 1 0 0 0 0 0 0 00 <7> 14 000 00 1 0 0 0 0 0 0 00 <7> 15 001 01 1 1 0 1 0 1 1 A9 <7> 16 000 00 1 0 0 0 0 0 0 00 <7> 17 000 00 1 0 0 0 0 0 0 00 <4> <7>IO APIC #9...... <7>.... register #00: 09000000 <7>....... : physical APIC id: 09 <7>.... register #01: 00178020 <7>....... : max redirection entries: 0017 <7>....... : PRQ implemented: 1 <7>....... : IO APIC version: 0020 <7>.... register #02: 00178020 <7>....... : arbitration: 00 <7>.... IRQ redirection table: <7> NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: <7> 00 000 00 1 0 0 0 0 0 0 00 <7> 01 000 00 1 0 0 0 0 0 0 00 <7> 02 000 00 1 0 0 0 0 0 0 00 <7> 03 000 00 1 0 0 0 0 0 0 00 <7> 04 000 00 1 0 0 0 0 0 0 00 <7> 05 000 00 1 0 0 0 0 0 0 00 <7> 06 000 00 1 0 0 0 0 0 0 00 <7> 07 000 00 1 0 0 0 0 0 0 00 <7> 08 000 00 1 0 0 0 0 0 0 00 <7> 09 000 00 1 0 0 0 0 0 0 00 <7> 0a 000 00 1 0 0 0 0 0 0 00 <7> 0b 000 00 1 0 0 0 0 0 0 00 <7> 0c 000 00 1 0 0 0 0 0 0 00 <7> 0d 000 00 1 0 0 0 0 0 0 00 <7> 0e 000 00 1 0 0 0 0 0 0 00 <7> 0f 000 00 1 0 0 0 0 0 0 00 <7> 10 000 00 1 0 0 0 0 0 0 00 <7> 11 000 00 1 0 0 0 0 0 0 00 <7> 12 000 00 1 0 0 0 0 0 0 00 <7> 13 000 00 1 0 0 0 0 0 0 00 <7> 14 000 00 1 0 0 0 0 0 0 00 <7> 15 000 00 1 0 0 0 0 0 0 00 <7> 16 000 00 1 0 0 0 0 0 0 00 <7> 17 000 00 1 0 0 0 0 0 0 00 <7>IRQ to pin mappings: <7>IRQ0 -> 0:2 <7>IRQ1 -> 0:1 <7>IRQ3 -> 0:3 <7>IRQ4 -> 0:4 <7>IRQ5 -> 0:5 <7>IRQ6 -> 0:6 <7>IRQ7 -> 0:7 <7>IRQ8 -> 0:8 <7>IRQ12 -> 0:12 <7>IRQ13 -> 0:13 <7>IRQ14 -> 0:14 <7>IRQ15 -> 0:15 <7>IRQ16 -> 0:16 <7>IRQ17 -> 0:17 <7>IRQ18 -> 0:18 <7>IRQ21 -> 0:21 <6>.................................... done. <6>Using local APIC timer interrupts. <6>Detected 12.500 MHz APIC timer. <6>checking TSC synchronization across 2 CPUs: passed. <6>time.c: Using PIT/TSC based timekeeping. <4>Brought up 2 CPUs <4>do_initcalls <4>init_elf32_binfmt <4>init_elf_binfmt <6>NET: Registered protocol family 16 <6>PCI: Using configuration type 1 <6>mtrr: v2.0 (20020519) <6>ACPI: Subsystem revision 20040326 <6>ACPI: Interpreter disabled. <4>ACPI: ACPI tables contain no PCI IRQ routing entries <4>PCI: Invalid ACPI-PCI IRQ routing table <4>PCI: Probing PCI hardware <4>PCI: Probing PCI hardware (bus 00) <4> ... the first call_usermodehelper: pci_bus <6>PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1 <4>Transparent bridge - 0000:00:1e.0 <6>PCI: Using IRQ router default [8086/25a1] at 0000:00:1f.0 <6>PCI->APIC IRQ transform: (B0,I2,P0) -> 16 <6>PCI->APIC IRQ transform: (B0,I3,P0) -> 16 <6>PCI->APIC IRQ transform: (B0,I31,P0) -> 18 <6>PCI->APIC IRQ transform: (B0,I31,P0) -> 18 <6>PCI->APIC IRQ transform: (B0,I31,P1) -> 17 <6>PCI->APIC IRQ transform: (B4,I3,P0) -> 17 <6>PCI->APIC IRQ transform: (B4,I4,P0) -> 21 <6>PCI-DMA: Disabling IOMMU. <6>vesafb: framebuffer at 0xfb000000, mapped to 0xffffff0000000000, size 8128k <6>vesafb: mode is 1024x768x16, linelength=2048, pages=4 <6>vesafb: scrolling: redraw <6>vesafb: directcolor: size=0:5:6:5, shift=0:11:5:0 <6>fb0: VESA VGA frame buffer device <4>Initial HugeTLB pages allocated: 0 <4>IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $ <6>ikconfig 0.7 with /proc/config* <5>VFS: Disk quotas dquot_6.5.1 <6>SELinux: Registering netfilter hooks <6>Initializing Cryptographic API <6>bootsplash 3.1.6-2004/03/31: looking for picture.... silentjpeg size 37245 bytes, found (1024x768, 17142 bytes, v3). <4>Console: switching to colour frame buffer device 118x41 <6>Real Time Clock Driver v1.12 <6>Non-volatile memory driver v1.2 <6>Linux agpgart interface v0.100 (c) Dave Jones <6>Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled <4>ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A <4>RAMDISK driver initialized: 16 RAM disks of 128000K size 1024 blocksize <6>loop: loaded (max 8 devices) <6>Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 <6>ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx <6>ICH5: IDE controller at PCI slot 0000:00:1f.1 <6>ICH5: chipset revision 2 <6>ICH5: not 100%% native mode: will probe irqs later <6> ide0: BM-DMA at 0xfc00-0xfc07, BIOS settings: hda:pio, hdb:pio <6> ide1: BM-DMA at 0xfc08-0xfc0f, BIOS settings: hdc:DMA, hdd:pio <4>hdc: IDE-DVD ROM 6116, ATAPI CD/DVD-ROM drive <4>Using anticipatory io scheduler <4>ide1 at 0x170-0x177,0x376 on irq 15 <4>hdc: ATAPI 48X DVD-ROM drive, 512kB Cache <6>Uniform CD-ROM driver Revision: 3.20 <4>ide-floppy driver 0.99.newide <6>mice: PS/2 mouse device common for all mice <6>input: PC Speaker <6>serio: i8042 AUX port at 0x60,0x64 irq 12 <6>serio: i8042 KBD port at 0x60,0x64 irq 1 <6>input: AT Translated Set 2 keyboard on isa0060/serio0 <6>md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27 <6>NET: Registered protocol family 2 <6>IP: routing cache hash table of 4096 buckets, 64Kbytes <6>TCP: Hash tables configured (established 131072 bind 65536) <6>NET: Registered protocol family 1 <6>md: Autodetecting RAID arrays. <6>md: autorun ... <6>md: ... autorun DONE. <5>RAMDISK: Compressed image found at block 0 <4>VFS: Mounted root (ext2 filesystem). <5>SCSI subsystem initialized <7>libata version 1.02 loaded. <7>ata_piix version 1.02 <7>PCI: Setting latency timer of device 0000:00:1f.2 to 64 <6>ata1: SATA max UDMA/133 cmd 0xAC00 ctl 0xA882 bmdma 0xA400 irq 18 <6>ata2: SATA max UDMA/133 cmd 0xA800 ctl 0xA482 bmdma 0xA408 irq 18 <7>ata1: dev 0 cfg 49:2f00 82:346b 83:7d01 84:4003 85:3449 86:3c01 87:4003 88:207f <6>ata1: dev 0 ATA, max UDMA/133, 312581808 sectors (lba48) <6>ata1: dev 0 configured for UDMA/133 <6>scsi0 : ata_piix <7>ata2: dev 0 cfg 49:2f00 82:346b 83:7d01 84:4003 85:3449 86:3c01 87:4003 88:207f <6>ata2: dev 0 ATA, max UDMA/133, 312581808 sectors (lba48) <6>ata2: dev 0 configured for UDMA/133 <6>scsi1 : ata_piix <5> Vendor: ATA Model: ST3160023AS Rev: 1.02 <5> Type: Direct-Access ANSI SCSI revision: 05 <6>ata1: dev 0 max request 512KB (lba48) <5>SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB) <5>SCSI device sda: drive cache: write through <6> sda: sda1 sda2 sda3 sda4 < sda5 > <5>Attached scsi disk sda at scsi0, channel 0, id 0, lun 0 <5> Vendor: ATA Model: ST3160023AS Rev: 1.02 <5> Type: Direct-Access ANSI SCSI revision: 05 <6>ata2: dev 0 max request 512KB (lba48) <5>SCSI device sdb: 312581808 512-byte hdwr sectors (160042 MB) <5>SCSI device sdb: drive cache: write through <6> sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 > <5>Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0 <6>md: raid1 personality registered as nr 3 <6>md: Autodetecting RAID arrays. <3>md: invalid raid superblock magic on sdb1 <4>md: sdb1 has invalid sb, not importing! <1>md: could not import sdb1! <3>md: invalid raid superblock magic on sdb2 <4>md: sdb2 has invalid sb, not importing! <1>md: could not import sdb2! <6>md: autorun ... <6>md: considering sdb5 ... <6>md: adding sdb5 ... <4>md: sdb3 has different UUID to sdb5 <6>md: adding sda5 ... <4>md: sda3 has different UUID to sdb5 <6>md: created md2 <6>md: bind<sda5> <6>md: bind<sdb5> <6>md: running: <sdb5><sda5> <6>raid1: raid set md2 active with 2 out of 2 mirrors <6>md: considering sdb3 ... <6>md: adding sdb3 ... <6>md: adding sda3 ... <6>md: created md0 <6>md: bind<sda3> <6>md: bind<sdb3> <6>md: running: <sdb3><sda3> <6>raid1: raid set md0 active with 2 out of 2 mirrors <6>md: ... autorun DONE. <5>ReiserFS: md0: found reiserfs format "3.6" with standard journal <5>ReiserFS: md0: using ordered data mode <5>ReiserFS: md0: journal params: device md0, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 <5>ReiserFS: md0: checking transaction log (md0) <5>ReiserFS: md0: Using r5 hash to sort names <4>VFS: Mounted root (reiserfs filesystem) readonly. <5>Trying to move old root to /initrd ... failed <5>Unmounting old root <5>Trying to free ramdisk memory ... okay <4>Freeing unused kernel memory: 204k freed <6>Adding 1052248k swap on /dev/sda2. Priority:42 extents:1 <6>md: Autodetecting RAID arrays. <6>md: autorun ... <6>md: ... autorun DONE. <6>device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com <5>ReiserFS: sda1: found reiserfs format "3.6" with standard journal <5>ReiserFS: sda1: using ordered data mode <5>ReiserFS: sda1: journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 <5>ReiserFS: sda1: checking transaction log (sda1) <5>ReiserFS: sda1: Using r5 hash to sort names <5>ReiserFS: md2: found reiserfs format "3.6" with standard journal <5>ReiserFS: md2: using ordered data mode <5>ReiserFS: md2: journal params: device md2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 <5>ReiserFS: md2: checking transaction log (md2) <5>ReiserFS: md2: Using r5 hash to sort names <6>subfs 0.9 Kernel logging (ksyslog) stopped. Kernel log daemon terminating. Boot logging started on /dev/tty1(/dev/console) at Mon Oct 17 14:56:06 2005 <notice>run boot scripts (setserial boot.proc) Configuring serial ports... /dev/ttyS0 at 0x03f8 (irq = 4) is a 16550A Configured serial ports done<notice>exit status of (setserial boot.proc) is (0 0) <notice>run boot scripts (boot.shm) Mounting shared memory FS on /dev/shmdone <notice>exit status of (boot.shm) is (0) <notice>run boot scripts (boot.rootfsck) Activating swap-devices in /etc/fstab... doneChecking root file system... fsck 1.34 (25-Jul-2003) Reiserfs super block in block 16 on 0x900 of format 3.6 with standard journal Blocks (total/free): 20970832/6881696 by 4096 bytes Filesystem is clean Filesystem seems mounted read-only. Skipping journal replay. Checking internal tree..finished done<notice>exit status of (boot.rootfsck) is (0) <notice>run boot scripts (boot.md boot.device-mapper) Activating device mapper... Creating /dev/mapper/control character device with major:10 minor:63. done <notice>exit status of (boot.md boot.device-mapper) is (0 0) <notice>run boot scripts (boot.localfs) Checking file systems... fsck 1.34 (25-Jul-2003) Reiserfs super block in block 16 on 0x801 of format 3.6 with standard journal Blocks (total/free): 26096/16329 by 4096 bytes Filesystem is clean Replaying journal.. Reiserfs journal '/dev/sda1' in blocks [18..8211]: 0 transactions replayed Checking internal tree..finished Reiserfs super block in block 16 on 0x902 of format 3.6 with standard journal Blocks (total/free): 17810032/14153841 by 4096 bytes Filesystem is clean Replaying journal.. Reiserfs journal '/dev/md2' in blocks [18..8211]: 0 transactions replayed Checking internal tree..finished doneSetting updone Mounting local file systems... proc on /proc type proc (rw) tmpfs on /dev/shm type tmpfs (rw) devpts on /dev/pts type devpts (rw,mode=0620,gid=5) /dev/sda1 on /boot type reiserfs (rw,acl,user_xattr) /dev/md2 on /utils type reiserfs (rw,acl,user_xattr) /dev/hdc on /media/dvd type subfs (ro,nosuid,nodev,fs=cdfss,procuid,iocharset=utf8) /dev/fd0 on /media/floppy type subfs (rw,nosuid,nodev,sync,fs=floppyfss,procuid) done<notice>exit status of (boot.localfs) is (0) <notice>run boot scripts (boot.crypto) <notice>exit status of (boot.crypto) is (0) <notice>run boot scripts (boot.swap boot.scpm boot.restore_permissions boot.loadmodules) Loading required kernel modules doneActivating remaining swap-devices in /etc/fstab... doneRestore device permissionsdone <notice>exit status of (boot.swap boot.scpm boot.restore_permissions boot.loadmodules) is (0 0 0 0) <notice>run boot scripts (boot.idedma boot.clock) Setting up the CMOS clockdone <notice>exit status of (boot.idedma boot.clock) is (0 0) <notice>run boot scripts (boot.ldconfig) Setting up timezone datadone <notice>exit status of (boot.ldconfig) is (0) <notice>run boot scripts (boot.sched boot.localnet boot.isapnp) Setting scheduling timeslices unused Setting up hostname 'fileserver'done Setting up loopback interface lo lo IP address: 127.0.0.1/8 done find: /tmp/ssh-*: No such file or directory <notice>exit status of (boot.sched boot.localnet boot.isapnp) is (6 0 0) <notice>run boot scripts (boot.klog boot.ipconfig) Enabling syn flood protectiondone Disabling IP forwardingdone done Creating /var/log/boot.msg done<notice>killproc: kill(386,29) <notice>exit status of (boot.klog boot.ipconfig) is (0 0) System Boot Control: The system has been set up Skipped features: boot.sched System Boot Control: Running /etc/init.d/boot.local done<notice>killproc: kill(386,3) INIT: Entering runlevel: 3 Boot logging started on /dev/tty1(/dev/console) at Mon Oct 17 12:56:24 2005 Master Resource Control: previous runlevel: N, switching to runlevel: 3 <notice>start services (random isdn hotplug boot.sysctl)start services (random isdn hotplug boot.sysctl) Hotplug is already active (disable with NOHOTPLUG=1 at the boot prompt)done Initializing random number generatordone Setting current sysctl status from /etc/sysctl.conf kernel.shmmax = 629145600 done <notice>exit status of (random isdn hotplug boot.sysctl) is (0 0 0 0) <notice>start services (coldplug) coldplug scanning input: **done scanning pci: *W*****..*W*done scanning usb: done . . . . . . . . . . . . . . . . . . . . . . . .done <notice>exit status of (coldplug) is (0) <notice>start services (network) Setting up network interfaces: lo lo IP address: 127.0.0.1/8 done eth0 device: Broadcom Corporation NetXtreme BCM5705 Gigabit Ethernet (rev 03) eth0 configuration: eth-id-00:11:d8:33:3c:0e eth0 IP address: 10.200.0.50/24 doneSetting up service network . . . . . . . . . . . . . . . .done <notice>exit status of (network) is (0) <notice>start services (syslog) Starting syslog services<notice>startproc: execve (/sbin/syslogd) [ /sbin/syslogd -a /var/lib/ntp/dev/log ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=26 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/sbin/syslogd ] <notice>startproc: execve (/sbin/klogd) [ /sbin/klogd -c 1 -2 ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=26 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/sbin/klogd ] done <notice>exit status of (syslog) is (0) <notice>start services (nmb) Starting Samba NMB daemon <notice>startproc: execve (/usr/sbin/nmbd) [ /usr/sbin/nmbd -D -s /etc/samba/smb.conf ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=27 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/nmbd ] done <notice>exit status of (nmb) is (0) <notice>start services (webmin splash_early resmgr portmap) <notice>startproc: execve (/sbin/resmgrd) [ /sbin/resmgrd ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=28 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/sbin/resmgrd ] <notice>startproc: execve (/sbin/portmap) [ /sbin/portmap ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=28 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/sbin/portmap ] Starting RPC portmap daemondone Starting resource managerdone <notice>exit status of (webmin splash_early resmgr portmap) is (0 0 0 0) <notice>start services (nfsboot) Starting nfsboot (sm-notify) done <notice>exit status of (nfsboot) is (0) <notice>start services (sshd powersaved postgresql fbset alsasound) <notice>startproc: execve (/usr/sbin/sshd) [ /usr/sbin/sshd -o PidFile=/var/run/sshd.init.pid ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=33 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/sshd ] Starting sound driverdone Starting SSH daemondone <notice>startproc: execve (/usr/sbin/powersaved) [ /usr/sbin/powersaved -d -e /etc/powersave.conf -n -v 3 ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=33 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/powersaved ] skipped Starting powersaved failed Starting PostgreSQLdone <notice>exit status of (sshd powersaved postgresql fbset alsasound) is (0 7 0 0 0) <notice>start services (splash replication postfix kbd cups) <notice>startproc: execve (/usr/sbin/cupsd) [ /usr/sbin/cupsd ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=38 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/cupsd ] Starting cupsddone Starting Replication Invoke slon for node 1 - /usr/local/pgsql/bin/slon -s 1000 -d2 replic_loco 'host=localhost dbname=mps user=slony port=5432 password=SLDataPass' 2>&1 > /var/log/slony1/node1/mps.log & Slon successfully started for cluster replic_loco, node node1 PID [2533] Start the watchdog process as well... Invoke slon for node 2 - /usr/local/pgsql/bin/slon -s 1000 -d2 replic_loco 'host=localhost dbname=mps_kd user=slony port=5432 password=SLDataPass' 2>&1 0) <notice>start services (apache2) Starting httpd2 (prefork) <notice>startproc: execve (/usr/sbin/httpd2-prefork) [ /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh get_module_list_done=true progress=43 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 get_includes_done=true PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/httpd2-prefork ] done <notice>exit status of (apache2) is (0) <notice>start services (xinetd smb nscd mdadmd hwscan cron clamd) <notice>startproc: execve (/sbin/mdadm) [ /sbin/mdadm -F -d 60 -m root@localhost -s -c /etc/mdadm.conf ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/sbin/mdadm ] Starting hardware scan on boot<notice>startproc: execve (/usr/sbin/clamd) [ /usr/sbin/clamd ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/clamd ] <notice>startproc: execve (/usr/sbin/xinetd) [ /usr/sbin/xinetd ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/xinetd ] <notice>startproc: execve (/usr/sbin/cron) [ /usr/sbin/cron ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/cron ] <notice>startproc: execve (/usr/sbin/nscd) [ /usr/sbin/nscd ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/nscd ] Starting Name Service Cache Daemondone Starting mdadmd done Starting CRON daemondone Starting Clam AntiVirus daemon done Starting INET services. (xinetd)done <notice>startproc: execve (/usr/sbin/smbd) [ /usr/sbin/smbd -D -s /etc/samba/smb.conf ], [ CONSOLE=/dev/console TERM=linux SHELL=/bin/sh TMPDIR=/var/tmp progress=44 INIT_VERSION=sysvinit-2.85 REDIRECT=/dev/tty1 AUTOBOOT=YES COLUMNS=118 PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin RUNLEVEL=3 resume=/dev/sda2 PWD=/ PREVLEVEL=N acpi=off LINES=41 SHLVL=2 BOOT_IMAGE=Linux splash=silent sscripts=52 _=/sbin/startproc DAEMON=/usr/sbin/smbd ] Samba SMB: Waiting for cupsd to get readydone Starting Samba SMB daemon done <notice>exit status of (xinetd smb nscd mdadmd hwscan cron clamd) is (0 0 0 0 0 0 0) <notice>start services (splash_late smbfs) Mount SMB/ CIFS File Systems unused <notice>exit status of (splash_late smbfs) is (0 6) Master Resource Control: runlevel 3 has been reached Failed services in runlevel 3: powersaved Skipped services in runlevel 3: smbfs <notice>killproc: kill(1127,3)