Notice: This Note entry was modified. 1345 non-ascii character(s) were
replaced with question marks.
Contact Information:
====================
FSP:
====
Machine Type:............8286-42A
Card Type:...............FSP2_P8LE
Current Boot Side:.......T
Next Boot Side:..........T
PT_Swap:.................0
Current Side Driver:.....fips861/b1213a_1652.861
Non-Current Side Driver:.fips861/b1202a_1650.861
P Side FipS Valid Flag:..1
T Side FipS Valid Flag:..1
Update Policy:...........Out of Band Update
Current FSP Position.....A
Current FSP Role:........Primary
Sibling FSP:.............Not Present
Sibling FSP IP...........Unknown
Image Type...............ship
H
(0) root @ gp1p01: /etc/microcode
# cat /proc/version
Oct 10 2016
19:11:03
1640D_61k
@(#) _kdb_buildinfo unix_64 Oct 10 2016 19:11:03 1640D_61k
Problem Description:
===================
On my Tuleta PowerVM system, I created a new LPAR and assigned 2 PCIe3 NVMe
adapters from slots P1-C8 and P-C11 to the new LPAR. I then NIM installed
SLES12 SP2. During installation, I created software RAID 1 to install OS. The
OS installation completed successfully and reboot, however, it could not boot
OS from either NVME drives.
Please see below the steps I did:
=================================
1) I went into the "Expert Partitioner..." to setup software RAID.
Before:
=======
YaST2 - installation @ install
Expert Partitioner [Release Notes...]
?System View??????????Hard Disks
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?
? ???RAID ??/dev/nvme0n1 ? 1.46 TiB? ? ?HGST, Inc.-Ultrast?
? ???Volume Managemen??/dev/nvme1n1 ? 2.91 TiB? ? ?HGST, Inc.-Ultrast?
? ???Crypt Files ??/dev/nvme1n1p1?62.75 MiB? ? ?GPT PReP ?
? ???Device Mapper ??/dev/nvme1n1p2? 2.02 GiB?F? ?Linux swap ?
? ???NFS ??/dev/nvme1n1p3?40.02 GiB?F? ?Linux native ?
? ???Btrfs ??/dev/nvme1n1p4? 2.87 TiB?F? ?Linux native ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ????????????????????????????????????????????????????
? ?[Add Partition...][Edit...]
? ?[Move...][Resize...]
??????????????????????[Delete...]
[ Help ] [ Back ] [Abort] [Accept]
2) I cleaned out the old partitions. I then created new partitions like below.
After:
======
YaST2 - installation @ install
Expert Partitioner [Release Notes...]
?System View??????????Hard Disks
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?
? ???RAID ??/dev/nvme0n1 ? 1.46 TiB? ? ?HGST, Inc.-Ultrast?
? ???Volume Managemen??/dev/nvme0n1p1?40.02 GiB? ? ?Linux RAID ?
? ???Crypt Files ??/dev/nvme0n1p2?60.00 GiB? ? ?Linux RAID ?
? ???Device Mapper ??/dev/nvme0n1p3? 3.98 GiB? ? ?GPT PReP ?
? ???NFS ??/dev/nvme1n1 ? 2.91 TiB? ? ?HGST, Inc.-Ultrast?
? ???Btrfs ??/dev/nvme1n1p1?40.02 GiB? ? ?Linux RAID ?
? ???tmpfs ??/dev/nvme1n1p2?60.00 GiB? ? ?Linux RAID ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ????????????????????????????????????????????????????
? ?[Add Partition...][Edit...]
? ?[Move...][Resize...]
??????????????????????[Delete...]
3) I then created software RAID 1 for swap (md0) and root (md1). See below.
YaST2 - installation @ install
Expert Partitioner [Release Notes...]
?System View??????????RAID
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?FS Type?Label?Mo?
? ?+?RAID ??/dev/md0?40.02 GiB?F? ?MD RAID?Swap ? ?sw?
? ???Volume Managemen??/dev/md1?60.00 GiB?F? ?MD RAID?BtrFS ? ?/ ?
? ???Crypt Files ?? ?
? ???Device Mapper ?? ?
? ???NFS ?? ?
? ???Btrfs ?? ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ?? ?
? ?? ?
? ????????????????????????????????????????????????????
??????????????????????[Add RAID...][Edit...][Resize...][Delete...]
[ Help ] [ Back ] [Abort] [Accept]
4) I then accepted the changes and procceeded with OS installation.
....
YaST2 - installation @ install
Suggested Partitioning [Release Notes...]
????????????????????????????????????????????????????????????????????????
? * Create volume /dev/nvme0n1p1 (40.02 GiB) ?
? * Create volume /dev/nvme0n1p2 (60.00 GiB) ?
? * Create GPT PReP volume /dev/nvme0n1p3 (3.98 GiB) ?
? * Create volume /dev/nvme1n1p1 (40.02 GiB) ?
? * Create volume /dev/nvme1n1p2 (60.00 GiB) ?
? * Create software RAID /dev/md0 (40.02 GiB) from /dev/nvme0n1p1 ?
? /dev/nvme1n1p1 for swap with swap ?
? * Create root volume /dev/md1 (60.00 GiB) with btrfs ?
? * Create subvolume @/boot/grub2/powerpc-ieee1275 on device /dev/md1 ?
????????????????????????????????????????????????????????????????????????
[Edit Proposal Settings]
[Create Partition Setup...]
[Expert Partitioner...]
[Help] [Back] [Abort] [Next]
...
YaST2 - installation @ install
Performing Installation
?Details??SLES12-SP2 Release Notes??????????????????????????????????????????
? ?????????????????????????????????????????????????????????????????????? ?
? ?Media ?Remaining?Packages?Time ? ?
? ?Total ?2.503 GiB? 1428? ? ?
? ?SLES12-SP2-12.2-0? ? ? ? ?
? ?Medium 1 ?2.503 GiB? 1428? ? ?
? ?????????????????????????????????????????????????????????????????????? ?
? ?Actions performed:??????????????????????????????????????????????????? ?
? ?~ze 6.42 MiB) ? ?
? ?Installing sles-installquick_en-pdf-12.2-20.7.noarch.rpm (installed ? ?
? ?~size 1.42 MiB) ? ?
? ?Installing sles-security_en-pdf-12.2-20.7.noarch.rpm (installed size? ?
? ?????????????????????????????????????????????????????????????????????? ?
? Installing sles-security_en-pdf-12.2-20.7.noarch.rpm (installed size 3 ?
? 27% ?
? ?
? Installing Packages... (Remaining: 2.503 GiB, 1428 packages) ?
? 13% ?
????????????????????????????????????????????????????????????????????????????
F1 Help F8 Back F9 Abort F10 Next
....
[ Help ] [ Back ] [Abort] [Accept]
5) The installation did complete successfully. It then prompted to reboot at
the end of installation.
6) However, when the LPAR rebooted to SMS menu, it could not boot OS from
either NVME drives.
Version FW860.20 (SV860_067)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
Select Media Adapter
1. U78C9.001.WZS00AJ-P1-C11 /pci@800000020000013/pci1014,4f5@0
2. U78C9.001.WZS00AJ-P1-C8 /pci@80000002000001c/pci1014,4f6@0
3. List all devices
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main Menu
ESC key = return to previous screen X = eXit System Management
Services
-------------------------------------------------------------------------------
Type menu item number and press Enter or select Navigation key:
...
7) It could not boot from either NVME drives.
.----------------------------------------------------------.
| The selected devices were not detected in the system ! |
| Press any key to continue. |
`----------------------------------------------------------'
== Comment: #2 - SEETEENA THOUFEEK <sthoufee@in.ibm.com> - 2016-12-22 01:17:56
==
Have you installed GRUB after the installation
https://www.suse.com/communities/blog/installing-sles-software-raid1/
== Comment: #3 - Paul Nguyen <nguyenp@us.ibm.com> - 2016-12-22 17:03:33 ==
Hi Seeteena,
I attempted to recreate the problem today and confirmed that from YasT
installation menu, it does not provide a way to install grub like you suggested
in previous comment.
I found that right after YasT finished installation, it provide two options
"ok" to reboot or "stop".
- If I chose the "ok" option to reboot, then machine will reboot to SMS menu
but then it could not find the NVMe drive to boot.
- If I chose the "stop" option, then it just sit there at YasT installation
menu. See below. Right now it is at "stop" option.
So please take over system to see if you could figure how to do it. Thanks
==========================================================
YaST2 - installation @ install
Performing Installation
?Details??SLES12-SP2 Release Notes??????????????????????????????????????????
? ?Ac??????????????????????????????????????????????????????????????????? ?
? ? *? ???????????????????????????????????????????????????????????? ? ? ?
? ? *? ?The system will reboot now... ? ? ? ?
? ? *? ? ? ? ? ?
? ?Pr? ? ? ? ? ?
? ? *? ? ? ? ? ?
? ? *? ? ? ? ? ?
? ? *? ? ? ? ? ?
? ? *? ? ? ? ? ?
? ? *? ? ? ? ? ?
? ?Fi? ???????????????????????????????????????????????????????????? ? ? ?
? ???? 4 ? ?? ?
? Unm? ? ?
? ? [OK] [Stop] ? ?
? Pre???????????????????????????????????????????????????????????????? ?
? ?
????????????????????????????????????????????????????????????????????????????
[Help] [Back] [Abort] [Next]
F9 Stop F10 OK
== Comment: #4 - Paul Nguyen <nguyenp@us.ibm.com> - 2016-12-22 17:08:00 ==
Here is the system information again. So you can log into it and look at the
problem.
FSP:
====
gp1fp1.aus.stglabs.ibm.com
Machine Type:............8286-42A
Card Type:...............FSP2_P8LE
Current Boot Side:.......T
Next Boot Side:..........T
PT_Swap:.................0
Current Side Driver:.....fips861/b1213a_1652.861
Non-Current Side Driver:.fips861/b1202a_1650.861
P Side FipS Valid Flag:..1
T Side FipS Valid Flag:..1
Update Policy:...........Out of Band Update
Current FSP Position.....A
Current FSP Role:........Primary
Sibling FSP:.............Not Present
Sibling FSP IP...........Unknown
Image Type...............ship
HMC:
====
brhmc10.aus.stglabs.ibm.com
HMC V8R8.6.0 (11-01-2016)
VIOS server:
==========
gp1p01.aus.stglabs.ibm.com
(0) root @ gp1p01: /etc/microcode
# cat /proc/version
Oct 10 2016
19:11:03
1640D_61k
@(#) _kdb_buildinfo unix_64 Oct 10 2016 19:11:03 1640D_61k
SLES12 SP2 LPAR:
================
gp1p06.aus.stglabs.ibm.com
=> This VIOS client is the one we were able to create software RAID 1 and
installed SLES 12 SP2 but not able to boot.
== Comment: #8 - Paul Nguyen <nguyenp@us.ibm.com> - 2016-12-28 13:11:14 ==
SLES12 installation successfully completed and at the prompt for reboot. So
because it's not failed during installation so it does not provide a way get to
shell command prompt to get the y2logs.
- The issue here is that after SLES12 installation, It could not boot from
either NVME drives. When select any of the NVMe drives to boot, it displays
error message below.
.----------------------------------------------------------.
| The selected devices were not detected in the system ! |
| Press any key to continue. |
`----------------------------------------------------------'
- So is it a partition firmware issue in this case? If it at least boot from
any of the NVMe drive and end up in grub> then we can go install grub. But in
this case, it did not get to that point.
Any idea of what the next step for this problem?
== Comment: #12 - SEETEENA THOUFEEK <sthoufee@in.ibm.com> - 2017-01-12 03:57:38
==
When I tried boot it shows below failure.
BOOTP Parameters:
----------------
chosen-network-type = ethernet,auto,rj45,auto
server IP = 9.3.80.16
client IP = 9.3.165.24
gateway IP = 9.3.165.1
device = /vdevice/l-lan@30000002
MAC address = 46 c1 03 01 52 02
loc-code = U8286.42A.101127A-V6-C2-T1
BOOTP request retry attempt: 1
BOOTP request retry attempt: 2
and shows below
Version FW860.20 (SV860_069)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
Main Menu
1. Select Language
2. Setup Remote IPL (Initial Program Load)
3. I/O Device Information
4. Select Console
5. Select Boot Options
if I select console it shows below and hence I am not able to collect yast
logs.
ok
0 > ok
brhmc10.aus.stglabs.ibm.com - hscpe/abc1234
gp1p06 - padmin/padmin
== Comment: #13 - Douglas Miller <dougmill@us.ibm.com> - 2017-01-12 09:17:25 ==
I think your next steps need to be to determine if the firmware can ever boot
from the NVMe drives. If you install without RAID can you boot? If you install
something other than SUSE can you boot?
== Comment: #17 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-07 18:49:29 ==
When I opened this defect, I created raid 1 for /swap and / and chunk size of 4
KiB during SLES12 installation and it failed to boot after OS finished
installation. See my previous comments.
Today, I re-attempted software raid 1 on NVMe (leaf adapters) during SLES12
installation again.
- This time, I created raid 1 for /swap and /home (instead of /swap and /).
Expert Partitioner [Release Notes...]
?System View??????????RAID
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?FS Type?Label?Mo?
? ?+?RAID ??/dev/md0?50.01 GiB?F? ?MD RAID?XFS ? ?/h?
? ???Volume Managemen??/dev/md1?50.01 GiB?F? ?MD RAID?Swap ? ?sw?
? ???Crypt Files ?? ?
? ???Device Mapper ?? ?
? ???NFS ?? ?
? ???Btrfs ?? ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ?? ?
? ?? ?
? ????????????????????????????????????????????????????
??????????????????????[Add RAID...][Edit...][Resize...][Delete...]
[ Help ] [ Back ] [Abort] [Accept]
- I also selected raid chunk size of 64KiB (instead of 4KiB like before).
- The RAID 1 creation worked. The SLES12 installation also worked.
- At the end of SLES12 installation, it prompted for system reboot.
- Firmware able to boot the LPAR to login prompt.
So I can come to conclusion that:
- When created RAID 1 for /swap and / it installed but could not boot after
installation.
- When created RAID 1 for /swap and /home then it installed and after
installation it boot all the way to login prompt.
At this point, I need development team to verify that why it failed to boot
when create raid 1 for /swap and / during installation. Now, when created raid
1 for /swap and /home then it works fine.
== Comment: #18 - Douglas Miller <dougmill@us.ibm.com> - 2017-02-08 09:56:55 ==
I suspect the reason it worked for "/home" and not for "/" (root) is that SMS
does not need to find "/home" (or "swap") - only "/". If SMS cannot access the
volume to boot, then you cannot boot. If "/" is a volume or device that SMS can
access, then you can boot. Once the OS boots, the OS will easily access md
volumes of "/home" and "swap" because it has the necessary code.
Does SMS support Linux software RAID? If so, what special setup is required? My
guess would be that grub provides that support and without a grub bootloader
you cannot boot software RAID. But I am just guessing based on the little
information in this bug report.
== Comment: #19 - Douglas Miller <dougmill@us.ibm.com> - 2017-02-08 10:04:45 ==
It also sounds like you changed TWO variables between your first attempt and
this latest one. One change was the chunk size (4KiB to 64KiB) and the second
was the RAID partition ("/" to "/home"). So, we can't really tell which one of
those changes allowed progress. Since we know there are problems with RAID
chunk size not being 64KiB, you probably need to try again for "/" but use the
64KiB chunk size.
== Comment: #20 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-08 15:22:45 ==
I re-ran the test like Doug M. mentioned in the previous comments. However,
with software RAID1 of /swap and / and RAID chunk size of 64KiB configuration,
the install boot manager hit a grub error during boot manager install. Then
after installation, system was prompted for reboot but it could not boot from
any NVMe drives.
So the software RAID 1 configuration for /swap and / seems to be a valid
configuration. But because of the grub error during boot manager install and
that lead to boot problem.
?Error
?
?Execution of command "[["/usr/sbin/grub2-install",
"--target=powerpc-ieee1275"?
?Exit code: 1
?
?Error output: Installing for powerpc-ieee1275 platform.
?
?/usr/sbin/grub2-install: error: disk `mduuid/1cf2427b96e808347abf48968d56c72e'
See my steps below:
===================
1) I created software RAID 1 for /swap and /
2) I selected chunk size is 64KiB for RAID
Expert Partitioner [Release Notes...]
?System View??????????RAID
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?FS Type?Label?Mo?
? ?+?RAID ??/dev/md0?40.02 GiB?F? ?MD RAID?BtrFS ? ?/ ?
? ???Volume Managemen??/dev/md1?60.00 GiB?F? ?MD RAID?Swap ? ?sw?
? ???Crypt Files ?? ?
? ???Device Mapper ?? ?
? ???NFS ?? ?
? ???Btrfs ?? ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ?? ?
? ?? ?
? ????????????????????????????????????????????????????
??????????????????????[Add RAID...][Edit...][Resize...][Delete...]
[ Help ] [ Back ] [Abort] [Accept]
===
Suggested Partitioning [Release Notes...]
????????????????????????????????????????????????????????????????????????
? * Create volume /dev/nvme0n1p1 (40.02 GiB) ?
? * Create volume /dev/nvme0n1p2 (60.00 GiB) ?
? * Create GPT PReP volume /dev/nvme1n1p1 (62.75 MiB) ?
? * Create volume /dev/nvme1n1p2 (40.02 GiB) ?
? * Create volume /dev/nvme1n1p3 (60.00 GiB) ?
? * Create root volume /dev/md0 (40.02 GiB) with btrfs ?
? * Create software RAID /dev/md1 (60.00 GiB) from /dev/nvme0n1p2 ?
? /dev/nvme1n1p3 for swap with swap ?
? * Create subvolume @/boot/grub2/powerpc-ieee1275 on device /dev/md0 ?
????????????????????????????????????????????????????????????????????????
[Edit Proposal Settings]
[Create Partition Setup...]
[Expert Partitioner...]
[Help] [Back] [Abort] [Next]
===
3) I proceed with SLES12 installation, However, with sofware RAID1 of /swap and
/ and RAID chunk size of 64KiB configuration, the install boot manager hit this
error.
?Error
?
?Execution of command "[["/usr/sbin/grub2-install",
"--target=powerpc-ieee1275"?
?Exit code: 1
?
?Error output: Installing for powerpc-ieee1275 platform.
?
?/usr/sbin/grub2-install: error: disk `mduuid/1cf2427b96e808347abf48968d56c72e'
- Then after installation, system was prompted for reboot but it could not boot
from any NVMe drives.
== Comment: #22 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-08 17:58:46 ==
I got YaST2 directory, /var/log/messages and y2log. I am attaching those files
for this problem.
For a complete picture, see below what I configured.
1 - Before I do any thing:
Expert Partitioner [Release Notes...]
?System View??????????Available Storage on linux
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?
? ???RAID ??/dev/nvme0n1 ? 1.46 TiB? ? ?HGST, Inc.-Ultrast?
? ???Volume Managemen??/dev/nvme1n1 ? 2.91 TiB? ? ?HGST, Inc.-Ultrast?
? ???Crypt Files ??/dev/nvme1n1p1?62.75 MiB? ? ?GPT PReP ?
? ???Device Mapper ??/dev/nvme1n1p2? 2.02 GiB?F? ?Linux swap ?
? ???NFS ??/dev/nvme1n1p3?40.02 GiB?F? ?Linux native ?
? ???Btrfs ??/dev/nvme1n1p4? 2.87 TiB?F? ?Linux native ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ?? ?
? ????????????????????????????????????????????????????
? ?[Rescan Devices][Import Mount Points...]
?????????????????????? [Configure...?]
[ Help ] [ Back ] [Abort] [Accept]
2 - I created partitions on the 2 NVMe drives
Expert Partitioner [Release Notes...]
?System View??????????Hard Disks
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?
? ???RAID ??/dev/nvme0n1 ? 1.46 TiB? ? ?HGST, Inc.-Ultrast?
? ???Volume Managemen??/dev/nvme0n1p1?40.02 GiB? ? ?Linux RAID ?
? ???Crypt Files ??/dev/nvme0n1p2?60.00 GiB? ? ?Linux RAID ?
? ???Device Mapper ??/dev/nvme1n1 ? 2.91 TiB? ? ?HGST, Inc.-Ultrast?
? ???NFS ??/dev/nvme1n1p1?62.75 MiB? ? ?GPT PReP ?
? ???Btrfs ??/dev/nvme1n1p2?40.02 GiB? ? ?Linux RAID ?
? ???tmpfs ??/dev/nvme1n1p3?60.00 GiB? ? ?Linux RAID ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ????????????????????????????????????????????????????
? ?[Add Partition...][Edit...]
? ?[Move...][Resize...]
??????????????????????[Delete...]
[ Help ] [ Back ] [Abort] [Accept]
3 - I created software RAID 1 (dev/md0) for / (with RAID chunk size of 64KiB)
Add RAID /dev/md0 [Release Notes...]
?RAID Type??????????????????????????????? Raid Name (optional)
? (x) RAID 1 (Mirroring) ? ????????????????????
? ( ) RAID 5 (Redundant Striping) ?
? ( ) RAID 6 (Dual Redundant Striping) ?
? ( ) RAID 10 (Mirroring and Striping) ?
?????????????????????????????????????????
Available Devices: Selected Devices:
????????????????????????????? ??????????????????? [Top]
? ?Device ? Siz? ? ?Device ?
?[ ]?/dev/nvme0n1p2?60.00 Gi? [ Add -> ] ?[ ]?/dev/nvme0n1p? [Up]
?[ ]?/dev/nvme1n1p3?60.00 Gi? [ Add All -> ] ?[ ]?/dev/nvme1n1p?
? ? ? ? [Down]
? ? [ <- Remove ] ? ?
? ? [<- Remove All] ? ? [Bottom]
? ? ? ?
? ? ? ?
????????????????????????????? ???????????????????[Classify]
Total size: 119.99 GiB Resulting size: 40.02 GiB
===
Add RAID /dev/md0 [Release Notes...]
?Mounting Options?????????????
?Formatting Options???????????? ? (x) Mount partition ?
? (x) Format partition ? ? Mount Point ?
? File System ? ? /????????????????????? ?
? BtrFS?????????????????? ? ? [ Fstab Options... ] ?
? [ Options... ] ? ? ( ) Do not mount partition ?
? ( ) Do not format partition ? ??????????????????????????????
? [ ] Encrypt Device ? ??????????????????????????????
??????????????????????????????? ? [ Subvolume Handling ] ?
??????????????????????????????
[ Help ] [ Back ] [Abort] [Finish]
4 - I created software RAID 1 (dev/md1) for /swap (with RAID chunk size of
64KiB)
Add RAID /dev/md1 [Release Notes...]
?RAID Type??????????????????????????????? Raid Name (optional)
? (x) RAID 1 (Mirroring) ? ????????????????????
? ( ) RAID 5 (Redundant Striping) ?
? ( ) RAID 6 (Dual Redundant Striping) ?
? ( ) RAID 10 (Mirroring and Striping) ?
?????????????????????????????????????????
Available Devices: Selected Devices:
????????????????????????????? ??????????????????? [Top]
??Device?Size?Enc?Type ? ? ?Device ?
? ? [ Add -> ] ?[ ]?/dev/nvme0n1p? [Up]
? ? [ Add All -> ] ?[ ]?/dev/nvme1n1p?
? ? ? ? [Down]
? ? [ <- Remove ] ? ?
? ? [<- Remove All] ? ? [Bottom]
? ? ? ?
? ? ? ?
????????????????????????????? ???????????????????[Classify]
Total size: 0 B Resulting size: 60.00 GiB
==
Add RAID /dev/md1 [Release Notes...]
?Formatting Options???????????? ?Mounting Options?????????????
? (x) Format partition ? ? (x) Mount partition ?
? File System ? ? Mount Point ?
? Swap??????????????????? ? ? swap?????????????????? ?
? [ Options... ] ? ? [ Fstab Options... ] ?
? ( ) Do not format partition ? ? ( ) Do not mount partition ?
? [ ] Encrypt Device ? ??????????????????????????????
???????????????????????????????
[ Help ] [ Back ] [Abort] [Finish]
===
5 - I accepted the new change.
Expert Partitioner [Release Notes...]
?System View??????????RAID
????linux ????????????????????????????????????????????????????
? ?+?Hard Disks ??Device ? Size?F?Enc?Type ?FS Type?Label?Mo?
? ?+?RAID ??/dev/md0?40.02 GiB?F? ?MD RAID?BtrFS ? ?/ ?
? ???Volume Managemen??/dev/md1?60.00 GiB?F? ?MD RAID?Swap ? ?sw?
? ???Crypt Files ?? ?
? ???Device Mapper ?? ?
? ???NFS ?? ?
? ???Btrfs ?? ?
? ???tmpfs ?? ?
? ???Unused Devices ?? ?
????Installation Summ?? ?
????Settings ?? ?
? ?? ?
? ????????????????????????????????????????????????????
??????????????????????[Add RAID...][Edit...][Resize...][Delete...]
[ Help ] [ Back ] [Abort] [Accept]
6 - I proceed with SLES12 installation
Suggested Partitioning [Release Notes...]
????????????????????????????????????????????????????????????????????????
? * Create volume /dev/nvme0n1p1 (40.02 GiB) ?
? * Create volume /dev/nvme0n1p2 (60.00 GiB) ?
? * Create GPT PReP volume /dev/nvme1n1p1 (62.75 MiB) ?
? * Create volume /dev/nvme1n1p2 (40.02 GiB) ?
? * Create volume /dev/nvme1n1p3 (60.00 GiB) ?
? * Create root volume /dev/md0 (40.02 GiB) with btrfs ?
? * Create software RAID /dev/md1 (60.00 GiB) from /dev/nvme0n1p2 ?
? /dev/nvme1n1p3 for swap with swap ?
? * Create subvolume @/boot/grub2/powerpc-ieee1275 on device /dev/md0 ?
????????????????????????????????????????????????????????????????????????
[Edit Proposal Settings]
[Create Partition Setup...]
[Expert Partitioner...]
[Help] [Back] [Abort] [Next]
===
Installation Settings [Release Notes...]
Click a headline to make changes or use the "Change..." menu below.
????????????????????????????????????????????????????????????????????????????
?Software ?
? ?
? * Product: SUSE Linux Enterprise Server 12 SP2 ?
? * Patterns: ?
? + Help and Support Documentation ?
? + Base System ?
? + AppArmor ?
? + Minimal System (Appliances) ?
? + GNOME Desktop Environment ?
? + X Window System ?
? * Size of Packages to Install: 2.6 GiB ?
? ?
?Booting ?
? ?
????????????????????????????????????????????????????????????????????????????
[Change...?]
[ Help ] [ Back ] [Abort] [Install]
7 - Hit grub error during Install boot manager
YaST2 - installation @ install
Performing Installation
?Details??SLES12-SP2 Release Notes??????????????????????????????????????????
? ?Actions performed:??????????????????????????????????????????????????? ?
? ? * Writing YaST configuration... ? ?
? ? * Saving network configuration... ? ?
????????????????????????????????????????????????????????????????????????????????
?Error
?
?Execution of command "[["/usr/sbin/grub2-install",
"--target=powerpc-ieee1275"?
?Exit code: 1
?
?Error output: Installing for powerpc-ieee1275 platform.
?
?/usr/sbin/grub2-install: error: disk
`mduuid/fff1756a990ab69d60c97f79ce9d493b'?
?
?
? [OK]
?
????????????????????????????????????????????????????????????????????????????????
? 66% ?
? Install boot manager ?
? 97% ?
????????????????????????????????????????????????????????????????????????????
[Help] [Back] [Abort] [Next]
hvc0:linux-o40q:/var/log/YaST2 # cat y2log |grep grub2-install
2017-02-08 17:23:12 <1> linux-o40q(3777) [Ruby] lib/cheetah.rb:158 Executing
"/usr/sbin/grub2-install --target\=powerpc-ieee1275 --force --skip-fs-probe
/dev/disk/by-id/nvme-2000cca00600bf180-part1".
2017-02-08 17:23:13 <3> linux-o40q(3777) [Ruby] lib/cheetah.rb:206 Error
output: /usr/sbin/grub2-install: error: disk
`mduuid/fff1756a990ab69d60c97f79ce9d493b' not found.
2017-02-08 17:23:13 <3> linux-o40q(3777) [Ruby] yast2/execute.rb:66 Execution
of command "[["/usr/sbin/grub2-install", "--target=powerpc-ieee1275",
"--force", "--skip-fs-probe", "/dev/disk/by-id/nvme-2000cca00600bf180-part1"]]"
failed.
/usr/sbin/grub2-install: error: disk `mduuid/fff1756a990ab69d60c97f79ce9d493b'
not found.
hvc0:linux-o40q:/var/log/YaST2 #
== Comment: #23 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-08 17:59:42 ==
== Comment: #24 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-08 18:00:20 ==
== Comment: #25 - Paul Nguyen <nguyenp@us.ibm.com> - 2017-02-08 18:02:53 ==
== Comment: #27 - Douglas Miller <dougmill@us.ibm.com> - 2017-02-09 14:30:25 ==
What I see in y2log is that md0 gets created and is
MD_UUID=fff1756a:990ab69d:60c97f79:ce9d493b
Later, grub install tries to access disk
"mduuid/fff1756a990ab69d60c97f79ce9d493b" which does not exist. What I can't
tell is whether the installer has scrambled the UUID (e.g. should it have
removed the ":"s?) or if something has removed the disk (volume), such as
shutdown by mdadm. I see no evidence that any mdadm commands are run after
setting up the two RAID volumes, so if RAID got shutdown it must have been
through some other mechanism.
It might help to know what these volumes look like on a running system. Might
have to setup volumes on some other booted OS instance and look at how they are
presented. An "ls" of /dev/mduuid would be helpful.
.......................................
s