Okay, after following all of these I'm still hosed up. I THINK my problem is that I don't have Yaboot installed (and can't figure out a way to do so) and I can't boot it normally to get it installed. Lilo is there, however, and I can run it. Unfortunately because root is on a software raid drive, the script it runs pukes out errors (what I've gathered is it can't create the yaboot.conf file because the kernel is located where the firmware can't see it). SO, I've tried copying the yaboot chrp file to the PReP partition (which it WILL boot to), but naturally it can't see the kernel. I've copied /boot to a fat partition but firmware can't read it... where am I going. In all of the following - and from what you all know - what's the answer? If I had it to do over again I'd probably avoid the software RAID - but it's too late for that now and I KNOW I can get this to work if I can figure out the right magic to perform on the boot process! Using yaboot on IBM hardware On IBM hardware you cannot run yabootconfig or ybin. You must proceed with the following steps: * Install yaboot-static * Run 'dd if=/usr/lib/yaboot/yaboot.chrp of=/dev/sdXX' (fill in XX with your disk and partition for the PReP partition; this was in our example /dev/sda1) * Next construct your own yaboot.conf file and place into /etc. (Take a look at the config above, look into the man page of yaboot.conf or look at the below yaboot.conf example) * Assuming your boot device in OF is pointing to the harddrive you prep boot partition is on then it'll just work, otherwise at IPL time, go into the multiboot menu and set the boot device to the one with your prep boot partition. * That's it! Code Listing 4: yaboot.conf for IBM hardware device=disk: partition=2 root=/dev/sda2 default=2.6.12-gentoo-r10 timeout=50 image=/boot/kernel-2.6.12-gentoo-r10 label=Linux append="console=ttyS0,9600" read-only For POWER4, POWER5, and blade-based hardware where the PReP disk partition and the disk partition that contains your kernel are on the same physical disk, you can use a simplified yaboot.conf. The following should be sufficient: Code Listing 5: yaboot.conf for PReP hardware default = linux timeout = 100 image=/boot/kernel-2.6.12-gentoo-r10 label=linux read-only root = /dev/sda2 append="root=/dev/sda2" To verify that yaboot has been copied to the PReP partition: Code Listing 6: Verifying the yaboot install on PReP # dd if=/dev/sda1 count=10 | grep ELF Binary file (standard input) matches 10+0 records in 10+0 records out A match signifies that yaboot was installed correctly. Bootloader Installation (lilo or yaboot) There is currently no support to install the bootloader automatically during installation, nor is there anything to repartition your hard disk (within YaST). To workaround this, first you need to do the installation (with some special steps), then you need to reboot the installation kernel and choose to boot the installed system, and finally you need to configure the installed system. First, during the install at the main YaST menu choose to Change Booting. Then select Boot Loader Installation and then Do not Install any Boot Loader. Continue the rest of the install until it reboots. Once the packages are copied, the system will reboot. Since you don't have a bootloader, you need to boot off your installation kernel again (either your CD drive or network source). Once in there, boot to the linuxrc. Inside linuxrc you can specify to boot the installed system - select option 4 (Start Installation or System) and then option 2 (Boot Installed System). Select the correct root partition and it'll boot up and continue the second stage of the install. Note: if you previously had something installed on this system you might end up with a (to Open Firmware) bootable partition, but then no config file for yaboot. In that case you might need to select in Open Firmware where to boot. Once you're in the installed system, create a /etc/lilo.conf. This is a simple one on a pSeries: boot=/dev/sda1 timeout=100 default=linux activate image=/boot/vmlinux initrd=/boot/initrd label=linux append="root=/dev/sda3 quiet sysrq=1" My system has a PReP partition on sda1, swap on sda2, and root on sda3. The just run 'lilo' - a script that will setup yaboot for you - and reboot to test. Here's a more involved Mac example lilo.conf: boot=/dev/hda9 timeout=100 default=linux activate image=/boot/vmlinux initrd=/boot/initrd label=linux append="root=/dev/hda11 resume=/dev/hda10 quiet sysrq=1" other=/dev/hda8 label=macos Adjust the example for your system, use pdisk -l to see what partition layout exists: pdisk -l /dev/hda Partition map (with 512 byte blocks) on '/dev/hda' #: type name length base ( size ) 1: Apple_partition_map 'Apple ' 63 @ 1 2: Apple_Driver43*'Macintosh ' 54 @ 64 3: Apple_Driver43*'Macintosh ' 74 @ 118 4: Apple_Driver_ATA*'Macintosh ' 54 @ 192 5: Apple_Driver_ATA*'Macintosh ' 74 @ 246 6: Apple_Driver_IOKit 'Macintosh ' 512 @ 320 7: Apple_Patches 'Patch Partition' 512 @ 832 8: Apple_HFS 'Macintosh HD ' 13365743 @ 1344 ( 6.4G) 9: Apple_HFS 'Ohne Titel 2 ' 345453 @ 13367087 (168.7M) 10: Apple_UNIX_SVR2 'Swap ' 489513 @ 13712540 (239.0M) 11: Apple_UNIX_SVR2 'A/UX Root ' 39262257 @ 14202053 ( 18.7G) 12: Apple_Free 'Extra ' 10 @ 53464310 In this example, hda9 is a small HFS partition (not HFS+), hda8 is your MacOS installation. Run /sbin/lilo as root to write the bootloader to disk and adjust the boot-device variable in OpenFirmware NVRAM Boot Loader Configuration in the Installed System pSeries use the yaboot boot loader to provide a list of configured kernels during start-up and to load the kernel and initrd from the hard disk or via network. To reduce the hassle of configuration, a wrapper script 'lilo' exists to generate a yaboot.conf file suitable for your setup. lilo uses a simple config file /etc/lilo.conf that contains a list of kernels and the boot partition. It supports also root on jfs, RAID, and LVM, in this case, a small FAT partition is created because yaboot cannot read the kernel from such a setup. An example config: # wait 10 seconds for user input timeout=100 # kernel to boot when no user input was given default=linux # update the firmware boot-device variable activate # small 0x41 PReP Boot partition boot=/dev/sda # # kernel ELF binary image=/boot/vmlinux label=linux initrd=/boot/initrd # kernel cmdline options # quiet to surpress the useless kernel messages # sysrq= to allow backtrace when things go downhill append="root=/dev/sda3 quiet sysrq=1" HOWTO-Booting with Yaboot on PowerPC Chapter 4 - Creating the Bootstrap Partition Be sure you have reviewed the mac-fdisk Basics page at http://penguinppc.org/bootloaders/yaboot/doc/mac-fdisk-basics.shtml. If you are creating a new Linux installation, the first partition you create with mac-fdisk should be the bootstrap partition. Just use the mac-fdisk b command to automatically create a bootstrap partition of the proper size and type. If your version of mac-fdisk doesn't include the b command, use: Command (? for help): C xxxx 800k bootstrap Apple_Bootstrap (Replace xxxx with the starting block number.) A working tarball of a bootstrap-capable version of mac-fdisk (Debian users already have this version) is also distributed at http://penguinppc.org/~eb/files/. To install this, use su - cd / gzip -dc mac-fdisk.tar.gz | tar -xvp The bootstrap partition only takes 800k of space. If you are adding a bootstrap partition to your existing setup, you may have enough free space on your existing drive without changing other partitions. If not, you could avoid re-partitioning the entire disk by stealing a little space from your swap partition. Or parted would allow you to resize partitions. The bootstrap partition should end up being partition #2, after the partition map itself but before any other partitions on the disk. You can easily change the order of the partitions in the partition map using the r command (see the mac-fdisk tutorial). In other words, it's not the physical placement of the bootstrap partition that counts, it's the logical order within the partition map. Note, though, if you have an existing system and you shift the partition numbers around, you will need to make the appropriate changes in /etc/fstab. The reason you want the bootstrap partition as partition #2, is so that it precedes all other partitions in the boot sequence. This helps tremendously to make the entire system more stable, since some OS's are very intrusive in their use of partitions. When you're finished partitioning, use the p command and make note of the final partition numbers. You will need to know the partition number for the bootstrap partition, the location of your kernel, and if you intend to set up yaboot for multiple OS booting, partition numbers for your alternate OS's. While it's possible to install yaboot on a mountable HFS /boot partition, that configuration is deprecated, discouraged and completely unsupported. 4.1 Installing yaboot onto a PReP partition If you are creating an installation CD, you will probably be interested in how to install yaboot onto a PReP partition so you can boot from a hard drive. Here are three simple steps to get you started: 1. Use fdisk to create an 8M PReP partition (type 0x41) 2. Make the partition bootable 3. Copy the yaboot binary onto the PReP partition dd if=/your/path/yaboot of=/dev/sdaX where /dev/sdaX is the PReP partition 4. Create a valid yaboot.conf file and put it in /etc You should now be able to boot from the PReP partition with yaboot installed on it. And then there's this: http://www-128.ibm.com/developerworks/eserver/library/es-conf-swraid/

The only thing this doesn't show is mount points, sda/b3 are md0 which is / - the other two raid partitions are /var and /home. sdb2 is /tmp. Disk /dev/sda: 73.4 GB, 73407488000 bytes 128 heads, 32 sectors/track, 35003 cylinders Units = cylinders of 4096 * 512 = 2097152 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 5 10239+ 41 PPC PReP Boot /dev/sda2 * 72 403 679936 6 FAT16 /dev/sda3 404 10645 20972857+ fd Linux raid autodetect /dev/sda4 10645 35001 49881825 f W95 Ext'd (LBA) /dev/sda5 10645 15767 10490413+ fd Linux raid autodetect /dev/sda6 15767 35001 39391348+ fd Linux raid autodetect Disk /dev/sdb: 73.4 GB, 73407488000 bytes 255 heads, 63 sectors/track, 8924 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 131 1052226 82 Linux swap / Solaris /dev/sdb2 132 654 4200997+ 83 Linux /dev/sdb3 655 3265 20972857+ fd Linux raid autodetect /dev/sdb4 3266 8924 45455917+ f W95 Ext'd (LBA) /dev/sdb5 3266 4571 10490413+ fd Linux raid autodetect /dev/sdb6 4572 8924 34965441 fd Linux raid autodetect Disk /dev/md0: 21.4 GB, 21476081664 bytes 2 heads, 4 sectors/track, 5243184 cylinders Units = cylinders of 8 * 512 = 4096 bytes Device Boot Start End Blocks Id System Disk /dev/md2: 35.8 GB, 35804545024 bytes 2 heads, 4 sectors/track, 8741344 cylinders Units = cylinders of 8 * 512 = 4096 bytes Disk /dev/md1: 10.7 GB, 10742071296 bytes 2 heads, 4 sectors/track, 2622576 cylinders Units = cylinders of 8 * 512 = 4096 bytes On 12/14/05, Olaf Hering <olh@suse.de> wrote:

On Tue, Dec 13, Brian Loe wrote:

...

And then there's this: http://www-128.ibm.com/developerworks/eserver/library/es-conf-swraid/

This size is probably a good start to understand the issues involved with root on software raid. Can you paste the fdisk -l output from your system?

-- short story of a lazy sysadmin: alias appserv=wotan

On Wed, Dec 14, Brian Loe wrote:

The only thing this doesn't show is mount points, sda/b3 are md0 which is / - the other two raid partitions are /var and /home. sdb2 is /tmp.

Please run these commands and create a /etc/lilo.conf as shown below: parted /dev/sda rm 1 parted /dev/sda rm 2 parted /dev/sda mkpart primary 0M 16M parted /dev/sda set 1 type 0x41 /etc/lilo.conf: boot=/dev/sda activate default=linux timeout=123 image=/boot/vmlinux label=linux initrd=/boot/initrd append="root=/dev/md" run '/sbin/lilo' now, it should convert sda1 from 0x41 'PPC PReP Boot' to 0x6 FAT and set the OpenFirmware 'boot-device' variable. Verify this by running 'fdisk -l /dev/sda' and 'nvsetenv' -- short story of a lazy sysadmin: alias appserv=wotan

On Wed, Dec 14, Brian Loe wrote:

Do I do this from the rescue prompt or do I boot to the installed system and mount / and /var rw manually?

You can do this from the installed system. Make sure that sda2 is not mounted and part of /etc/fstab. parted (unlike fdisk) can edit partitions which are not used by the system. -- short story of a lazy sysadmin: alias appserv=wotan

On Wed, Dec 14, Brian Loe wrote:

ReiserFS: warning: is_tree_node: node level 0 does not match to the expected one 1 ReiserFS: md0: warning: vs-5150: search_by_key: invalid format found in block 4947981. Fsck? ReiserFS: md0: warning: vs-13070: reiserfs_read_locked_inode: i/o failure occured trying to find stat data of [11 81164 0x0 SD] INIT: version 2.85 booting

and then the rest is a lof of the reiserfs run with --rebuild-tree errors... and loads a "maintenance mode" - which I guess I'm suppose to use to remount md0, run fsck with --rebuild-tree and reboot. Before I do that I want to make sure its not still some kind of bootloader issue.

I dont know how the reiserfs warnings can happen. The bootloader part is working ok. -- short story of a lazy sysadmin: alias appserv=wotan

On Thu, Dec 15, Brian Loe wrote:

Well, here's an update - and I'm not real happy about this but I'm glad to share my pain. If you have root on a software RAID you can not boot to an install CD and jump out at the first option to boot installed system - you'll get the errors I saw. HOWEVER, if you stay in the install (linuxrc?), you'll get a SECOND option to jump out and boot the installed system and at that point the system CAN see the software RAID devices!

Always use yast to boot into an already installed system. -- short story of a lazy sysadmin: alias appserv=wotan

Well, I'm right back where I was except that I know how to get the system back (although crippled due to kernel updates) and I have an idea of what is supposed to work: arg-ns02n:~ # lilo running on chrp Boot target is /dev/sda ERROR: show_of_path.sh returned error 1 while analysing path /etc/yaboot.conf ERROR: show_of_path.sh returned error 1 while analysing path /boot/vmlinux ERROR: show_of_path.sh returned error 1 while analysing path /boot/initrd ERROR: Unknown IBM machine IBM,9123-710/SF235_160. ************************************************************ * We cannot guarantee a proper reboot with your current * * non trivial boot configuration which relies on a working * * FAT firmware driver from IBM * * * * Please consult your local IBM representative so that he * * can take the appropriate steps to help you out. * ************************************************************ arg-ns02n:~ # cat /etc/yaboot.conf arg-ns02n:~ # cat /etc/lilo.conf force boot=/dev/sda1 activate default=linux timeout=123 image=/boot/vmlinux label=linux initrd=/boot/initrd append="root=/dev/md0" arg-ns02n:~ # fdisk -l Disk /dev/sda: 73.4 GB, 73407488000 bytes 255 heads, 63 sectors/track, 8924 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 2 16033+ 41 PPC PReP Boot /dev/sda2 3 2432 19518975 fd Linux raid autodetect /dev/sda3 2433 8924 52146990 fd Linux raid autodetect Disk /dev/sdb: 73.4 GB, 73407488000 bytes 255 heads, 63 sectors/track, 8924 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 * 1 2 16033+ 6 FAT16 /dev/sdb2 3 2432 19518975 fd Linux raid autodetect /dev/sdb3 2433 8924 52146990 fd Linux raid autodetect Disk /dev/md0: 19.9 GB, 19987300352 bytes 2 heads, 4 sectors/track, 4879712 cylinders Units = cylinders of 8 * 512 = 4096 bytes Disk /dev/md0 doesn't contain a valid partition table Disk /dev/md1: 53.3 GB, 53398405120 bytes 2 heads, 4 sectors/track, 13036720 cylinders Units = cylinders of 8 * 512 = 4096 bytes Disk /dev/md1 doesn't contain a valid partition table

That was it. I now have a manually mirrored boot sector, a software raid mirror for root and the rest of my logical volumes - AND IT BOOTS! So, couple more questions: what did deleting the firmware_status.chrp file do for me? Why doesn't OSS 10 do as good a job at this as 9 ES does? And finally, is it safe to tell SMS to boot to that first partition on drive? Once again, thanks for putting up with me, tomorrow when I get back in the office I'll try to put all of this together as best I can and, if possible, blend it with what you already had. I believe IBM is selling this 710 as a linux box (in fact, you CAN'T install AIX on this box - tells you your not licensed for it) so there might be more and more people trying to get it to work and not use one of the commercial distributions you usually get with the box (RHES in our case). On 12/15/05, Olaf Hering <olh@suse.de> wrote:

On Thu, Dec 15, Brian Loe wrote:

...

Well, I'm right back where I was except that I know how to get the system back (although crippled due to kernel updates) and I have an idea of what is supposed to work:

arg-ns02n:~ # lilo running on chrp Boot target is /dev/sda ERROR: show_of_path.sh returned error 1 while analysing path /etc/yaboot.conf ERROR: show_of_path.sh returned error 1 while analysing path /boot/vmlinux ERROR: show_of_path.sh returned error 1 while analysing path /boot/initrd ERROR: Unknown IBM machine IBM,9123-710/SF235_160.

rm /lib/lilo/chrp/firmware_status.chrp lilo

this should fix it.

-- short story of a lazy sysadmin: alias appserv=wotan

On Thu, Dec 15, Brian Loe wrote:

So, couple more questions: what did deleting the firmware_status.chrp file do for me?

It declared your firmware level as tested, see /lib/lilo/lilo-chrp.lib There are a few bugs in the firmware FAT filesystem driver. Some of them are already fixed. If your box boots ok, then everything is fine.

Why doesn't OSS 10 do as good a job at this as 9 ES does? And finally, is it safe to tell SMS to boot to that first partition on drive?

It did not receive the required amount of testing that SLES9 received. Also not all fixes for SLES9 were applied in time. If the SMS offers you the boot partition, it should work ok. -- short story of a lazy sysadmin: alias appserv=wotan