Sandy Drobic wrote:

This is the version where you wish you WERE dreaming or at least leave all the mess behind you after exiting the cinema...

The actors:

- Fujitsu-Siemens Primergy 470 (Dual P3-800, Mylex Raidcontroller (DAC960PRL) with Hardware RAID5 on three SCSI hdd, currently running with Suse Linux 10.0, file system reiser 3)

- OpenSUSE 10.3 DVD 32bit

- one happy sysadmin who successfully tested that the installation dvd recognised the raid and didn't scream any warnings up to the point where the installation starts

Prologue: The machine is rather old being from the last century, but as it is still reliable and sufficient for the small environment, I would like to keep it. I already decided to skip the installation of Opensuse 10.1 and 10.2 as there were ominous warnings from gparted during the update praparation and the installation crashed mostly during install anyway.

So I was very happy, when Opensuse 10.3 didn't raise any fuss like warnings "the partitions can not be changed, do you want to use them unchanged?" or something like it.

Main story: Overall I was rather impressed with the speed and ease of the upgrade. It detected some unmaintained and non-upgradable packages which I expected anyway. Some of the packages were not from the distro or even compiled from source. The actual package installation went without any problem, the sun was shining, it was sunday and everything promised to turn into a great day.

That was when the nightmare slowly started. At the end of the installation the system is prepared for the first start of the new OS. Suddenly a warning appeared "mkinitrd failed" (or something like that). The installation procedure wasn't impressed though, and the countdown for the automatic reboot began. 10..9..8..7..6..

My eyes that were glued to the screen resetted and I hit the stop button. The countdown was stopped, phew. So, what to do now? No combination of keys gave me a shell to intervene.

Finally, with a glum premonition of approaching desaster, I allowed the system to reboot, lit a few incense sticks and did a little voodoo dance to invoke the gods of luck. It didn't work, the booting system didn't see any disks, panicked and finally crashed. The nightmare had arrived and had the previously happy sysadmin firmly in his grip.

Was all data/configuration lost and had to be installed from backup? I booted once again from the installation dvd, it seemed to recognise the partition, so all data apparently wasn't lost. Then I tried to use the rescue system to repair the boot configuration. The login prompt appeared and I logged in as root. Well, at least I attempted to login. The only reaction of the system was that it replied with a service error. So, no login via rescue system.

Then I tried to boot from dvd and run the installed system. That resulted in a nice little crash but no usable shell.

At that point I decided to take a timeout, eat something and think about any further steps.

Half an hour later I began to investigate in earnest, meaning I grabbed my Knoppix 5.2 dvd to see in what state the partitions currently were. Booting from the Knoppix dvd went without a hitch. It showed all partitions on the raid, and I could mount them without any problem. The worst of the nightmare slowly began to fade.

/boot resided on its own partition, and I immediately saw that indeed no initrd had been installed.

So I installed 10.3 (minimal instalation) as a VM to get a working initrd. Five minutes later I copied the initrd on my server and rebooted. Lucky, the initrd was accepted, unfortunately it didn't find my raid.

Okay, I thought, let's configure the initrd to include the necessary modules. That was when I discovered that the minimal installation did not have vi nor any other editor, not even less. Even a manual execution of mkinitrd with the list of modules to include only gave a list of missing modules errors.

I scrapped the minimal installation and installed a standard KDE version instead. 20 minutes later I had a working 10.3 in vmware with all the niceties I was accustomed to.

Once again I booted the Knoppix dvd and looked up the exact list of required modules in /etc/sysconfig/kernel. This time the new initrd was created without error message. I copied the new initrd to the server (with an usb stick) and rebooted.

Finally, the boot process showed my raid again, the installed system continued the installation. Yatta! Happy little sysadmin is doing the victory dance! The nightmare was over!

Or so I thought at least. When I began to check the various server installations on the system I quickly noticed that the logfiles showed no events after the reboot. What the heck is going on here? What use is a server installation without the means to debug the services?

A restart of the syslog server only gave back that /var/run/syslogd.pid was empty and the service had started. Of course it didn't start. :-/

Then I recalled that the newer versions of Opensuse had a new security feature: AppArmor. So I deactivated Apparmor and rebooted the entire system. And finally the logfiles were showing events again, also this little message appeared:

type=APPARMOR_DENIED msg=audit(1191788249.506:39): type=1503 operation="file_lock" requested_mask="k" denied_mask="k" name="/var/run/syslogd.pid" pid=8055 profile="/sbin/syslogd"

Great, AppArmor hits again. :-((

So finally the real work could start. I will look into apparmor when I've got time (translation: next year earliest).

Epilogue

The rest was business as usual after an upgrade: Apache2 didn't start because a php.ini value in a vhost config wasn't allowed. MySQL didn't start because of failed dependencies, apcupsd didn't work, the policyserver had to be reinstalled, Squirrelmail wasnt installed yadda yadda yadda.

Well, that is to be expected after an upgrade of a system with source packages and a leap from 10.0 to 10.3. But that only took about two hours to reinstall and debug those problems.

All in all about 7-8 hours of downtime before the system was up again and the most important services were running. A lot of time for a system update.

At least the installation didn't ruined the file system with all the data. Still, it provided me with some blood pressure elevation for some hours.

I can only recommend to have a Knoppix dvd present and better yet a test installation on a less important system before you jump on the band wagon of 10.3. It seems as if some raid controllers are still not correctly supported by newer Opensuse installers, even though they are working beautifully in older versions.

Oh NO! Sandy, if this happened to you, what can the rest of us expect? Just don't let your postfix knowledge become corrupt with the "blood pressure elevation." Good rule of thumb (thou shall not Upgrade). Back up home and data, fresh install, then reload data -- "The only way to fly." -- David C. Rankin, J.D., P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 (936) 715-9333 (936) 715-9339 fax www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org