[opensuse] md stealing disks!
Hi Folks, Here's an odd one. I pulled a "zypper dup" yesterday on a 12.3 x86-64 without any obvious issues. But today, I connected a brand new 3.5-in SATA disk to the system with one of those SATA/USB converters. I do this all the time and have never had any issues. So, I "tail -f /var/log/messages" to get the /dev/sdx id and then run fdisk /dev/sdf, in this case. I create one Linux partition using the whole disk. No problems noted. But now, when I run "mkfs.xfs /dev/sdf1" I get this error: "mkfs.xfs: cannot open /dev/sdf1: Device or resource busy" df doesn't show anything, and the Device Notifier is quiet. I then notice this in /var/log/messages: "[ 1122.179127] md: bind<sdf>" It looks like md is snatching the disk! But I'm not running md, to the best of my knowledge. A "ps ax |grep -i md" shows: 90 ? S< 0:00 [md] 95 ? SN 0:00 [ksmd] Is systemd quietly starting md which is then trying to be helpful? "kill -9 90" doesn't touch the process. "cat /proc/mdstat" shows: Personalities : md126 : inactive sdf[0] 976224256 blocks super external:/md127/0 md127 : inactive sdf[0](S) 538328 blocks super external:ddf unused devices: <none> This process of adding/formatting disks has worked for decades before yesterday's zypper dup. Are other folks seeing this? Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
В Wed, 17 Jul 2013 10:53:58 -0700 Lew Wolfgang <wolfgang@sweet-haven.com> пишет:
Hi Folks,
Here's an odd one. I pulled a "zypper dup" yesterday on a 12.3 x86-64 without any obvious issues.
But today, I connected a brand new 3.5-in SATA disk to the system with one of those SATA/USB converters. I do this all the time and have never had any issues.
So, I "tail -f /var/log/messages" to get the /dev/sdx id and then run fdisk /dev/sdf, in this case. I create one Linux partition using the whole disk. No problems noted.
But now, when I run "mkfs.xfs /dev/sdf1" I get this error:
"mkfs.xfs: cannot open /dev/sdf1: Device or resource busy"
df doesn't show anything, and the Device Notifier is quiet.
I then notice this in /var/log/messages:
"[ 1122.179127] md: bind<sdf>"
It looks like md is snatching the disk! But I'm not running md, to the best of my knowledge. A "ps ax |grep -i md" shows:
90 ? S< 0:00 [md] 95 ? SN 0:00 [ksmd]
Is systemd quietly starting md which is then trying to be helpful?
Of course. It also ate your kittens.
"kill -9 90" doesn't touch the process.
"cat /proc/mdstat" shows:
Personalities : md126 : inactive sdf[0] 976224256 blocks super external:/md127/0
md127 : inactive sdf[0](S) 538328 blocks super external:ddf
Zero out last couple of megabytes of this disk.
unused devices: <none>
This process of adding/formatting disks has worked for decades before yesterday's zypper dup. Are other folks seeing this?
Regards, Lew
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/17/2013 11:22 AM, Andrey Borzenkov wrote:
В Wed, 17 Jul 2013 10:53:58 -0700 Lew Wolfgang <wolfgang@sweet-haven.com> пишет:
Hi Folks,
Here's an odd one. I pulled a "zypper dup" yesterday on a 12.3 x86-64 without any obvious issues.
But today, I connected a brand new 3.5-in SATA disk to the system with one of those SATA/USB converters. I do this all the time and have never had any issues.
So, I "tail -f /var/log/messages" to get the /dev/sdx id and then run fdisk /dev/sdf, in this case. I create one Linux partition using the whole disk. No problems noted.
But now, when I run "mkfs.xfs /dev/sdf1" I get this error:
"mkfs.xfs: cannot open /dev/sdf1: Device or resource busy"
df doesn't show anything, and the Device Notifier is quiet.
I then notice this in /var/log/messages:
"[ 1122.179127] md: bind<sdf>"
It looks like md is snatching the disk! But I'm not running md, to the best of my knowledge. A "ps ax |grep -i md" shows:
90 ? S< 0:00 [md] 95 ? SN 0:00 [ksmd]
Is systemd quietly starting md which is then trying to be helpful? Of course. It also ate your kittens.
Of course! That's where they went!
"kill -9 90" doesn't touch the process.
"cat /proc/mdstat" shows:
Personalities : md126 : inactive sdf[0] 976224256 blocks super external:/md127/0
md127 : inactive sdf[0](S) 538328 blocks super external:ddf Zero out last couple of megabytes of this disk.
Huh?
unused devices: <none>
This process of adding/formatting disks has worked for decades before yesterday's zypper dup. Are other folks seeing this?
Regards, Lew
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
(top posting to close the issue) Hi Folks, I figured out what was going on. The disk, that I thought was new, apparently wasn't. It must have been a member of an md array at one point, and the kernel was grabbing it and trying to use it. So this sequence cleared out the md superblock and allowed me to configure the disk. 1. cat /proc/mdstat to determine the md devices. In this case they were md126 and md127 living on /dev/sdg. They were also inactive. 2. mdadm --stop /dev/md126; mdadm --stop /dev/md127 3. mdadm --zero-superblock /dev/sdg That was it. The zypper dup was just happenstance and not related to the issue. Regards, Lew On 07/17/2013 11:31 AM, Lew Wolfgang wrote:
On 07/17/2013 11:22 AM, Andrey Borzenkov wrote:
В Wed, 17 Jul 2013 10:53:58 -0700 Lew Wolfgang <wolfgang@sweet-haven.com> пишет:
Hi Folks,
Here's an odd one. I pulled a "zypper dup" yesterday on a 12.3 x86-64 without any obvious issues.
But today, I connected a brand new 3.5-in SATA disk to the system with one of those SATA/USB converters. I do this all the time and have never had any issues.
So, I "tail -f /var/log/messages" to get the /dev/sdx id and then run fdisk /dev/sdf, in this case. I create one Linux partition using the whole disk. No problems noted.
But now, when I run "mkfs.xfs /dev/sdf1" I get this error:
"mkfs.xfs: cannot open /dev/sdf1: Device or resource busy"
df doesn't show anything, and the Device Notifier is quiet.
I then notice this in /var/log/messages:
"[ 1122.179127] md: bind<sdf>"
It looks like md is snatching the disk! But I'm not running md, to the best of my knowledge. A "ps ax |grep -i md" shows:
90 ? S< 0:00 [md] 95 ? SN 0:00 [ksmd]
Is systemd quietly starting md which is then trying to be helpful? Of course. It also ate your kittens.
Of course! That's where they went!
"kill -9 90" doesn't touch the process.
"cat /proc/mdstat" shows:
Personalities : md126 : inactive sdf[0] 976224256 blocks super external:/md127/0
md127 : inactive sdf[0](S) 538328 blocks super external:ddf Zero out last couple of megabytes of this disk.
Huh?
unused devices: <none>
This process of adding/formatting disks has worked for decades before yesterday's zypper dup. Are other folks seeing this?
Regards, Lew
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/17/2013 2:23 PM, Lew Wolfgang wrote:
Hi Folks,
I figured out what was going on. The disk, that I thought was new, apparently wasn't. It must have been a member of an md array at one point, and the kernel was grabbing it and trying to use it.
LOL... You know I had an email all typed up asking how sure you were that the disk was in fact NEW as you stated, but I thought it might be pretty insulting to ask something so obvious. So I deleted it instead of sending it. Next time I'll take the risk. ;-) I did notice that md does appear to be running on my laptop which has never had an array in it. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/17/2013 02:27 PM, John Andersen wrote:
On 7/17/2013 2:23 PM, Lew Wolfgang wrote:
Hi Folks,
I figured out what was going on. The disk, that I thought was new, apparently wasn't. It must have been a member of an md array at one point, and the kernel was grabbing it and trying to use it. LOL...
You know I had an email all typed up asking how sure you were that the disk was in fact NEW as you stated, but I thought it might be pretty insulting to ask something so obvious. So I deleted it instead of sending it.
Next time I'll take the risk. ;-)
I did notice that md does appear to be running on my laptop which has never had an array in it.
Har! It seems that the more experienced we get, the more spectacular our foul-ups can be! I'm glad that none of my management reads this list. (or do they?) Thanks for your help, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
El 17/07/13 17:45, Lew Wolfgang escribió:
Har! It seems that the more experienced we get, the more spectacular our foul-ups can be! I'm glad that none of my management reads this list. (or do they?)
They probably do :-) ..but don't worry.. they EXPECT failure, epic and total :-P It is the way and the time it takes you to recover from a fail what usually matters to them. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2013-07-17 at 14:23 -0700, Lew Wolfgang wrote:
I figured out what was going on. The disk, that I thought was new, apparently wasn't. It must have been a member of an md array
You can find out how old the disk is. smartctl -a /dev/sdX The value "Power_On_Hours" - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlHnHokACgkQtTMYHG2NR9U5UACgksGutfjjkrP/4GWF1dAP/jC9 FFwAnRDgpx6VTQ6cBPJk/8tIeUhsbTdR =v7Yi -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/17/2013 5:23 PM, Lew Wolfgang wrote:
1. cat /proc/mdstat to determine the md devices. In this case they hmmm, did cat get back the kittens mentioned previously?
Damon Register -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2013-07-18 at 06:23 -0400, Damon Register wrote:
On 7/17/2013 5:23 PM, Lew Wolfgang wrote:
1. cat /proc/mdstat to determine the md devices. In this case they hmmm, did cat get back the kittens mentioned previously?
No, because the dog scared the cat. <http://software.opensuse.org/package/dog> :-) - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlHoFfIACgkQtTMYHG2NR9UQlQCdFyi9jnuF4sufxWj3cto9FxUi BgwAn2i2Mn+L0+Es7UzqgW0q7UAnINJp =672g -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/18/2013 12:20 PM, Carlos E. R. wrote:
No, because the dog scared the cat.
<http://software.opensuse.org/package/dog> and I thought I was bad. Only the open source community would come up with a package like that :-)
Damon Register -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/18/2013 09:20 AM, Carlos E. R. wrote:
On Thursday, 2013-07-18 at 06:23 -0400, Damon Register wrote:
hmmm, did cat get back the kittens mentioned previously?
No, because the dog scared the cat.
Not to worry. A tom-cat up the alley has offered his services to remedy the kitten situation. Tom -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2013-07-17 at 10:53 -0700, Lew Wolfgang wrote:
Hi Folks,
Here's an odd one. I pulled a "zypper dup" yesterday on a 12.3 x86-64 without any obvious issues.
But today, I connected a brand new 3.5-in SATA disk to the system with one of those SATA/USB converters. I do this all the time and have never had any issues.
So, I "tail -f /var/log/messages" to get the /dev/sdx id and then run fdisk /dev/sdf, in this case. I create one Linux partition using the whole disk. No problems noted.
But now, when I run "mkfs.xfs /dev/sdf1" I get this error:
Notice that you could also create the filesystem on the whole disk, no partitions. What type number did you use for the partition? There is one for raid, if the system sees it, it will automatically try to use md on the partition.
It looks like md is snatching the disk! But I'm not running md, to the best of my knowledge. A "ps ax |grep -i md" shows:
90 ? S< 0:00 [md] 95 ? SN 0:00 [ksmd]
Is systemd quietly starting md which is then trying to be helpful? "kill -9 90" doesn't touch the process.
Because it is a kernel thread, not a program. Not killable.
This process of adding/formatting disks has worked for decades before yesterday's zypper dup. Are other folks seeing this?
It might have other reasons :-) - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlHnHS0ACgkQtTMYHG2NR9WsWQCglG90pF4Zmp04OzeDSpmspV8r 8wMAniBGNSIae68CcTgiNfH7lKFWrM3+ =cPiG -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
Notice that you could also create the filesystem on the whole disk, no partitions.
You can, but IMHO it is a bad decision to do so. It exposes you to several more ways you can shoot yourself in the foot, and can confuse anybody else who looks at the disk (of course, that might be a bonus in some circumstances). There is another opinion at: https://raid.wiki.kernel.org/index.php/Partition_Types
What type number did you use for the partition? There is one for raid, if the system sees it, it will automatically try to use md on the partition.
Indeed, the same URL discusses the issue briefly. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, Jul 18, 2013 at 2:57 PM, Dave Howorth <dhoworth@mrc-lmb.cam.ac.uk> wrote:
Carlos E. R. wrote:
Notice that you could also create the filesystem on the whole disk, no partitions.
You can, but IMHO it is a bad decision to do so. It exposes you to several more ways you can shoot yourself in the foot, and can confuse anybody else who looks at the disk (of course, that might be a bonus in some circumstances).
That's right, but it makes it really easy to resize (at least, increase size of) your filesystem when it is on external storage array LUN. Just grow a LUN and resize filesystem. Changing partition size is rather more involving (not sure if it can be done online at all). -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2013-07-18 at 11:57 +0100, Dave Howorth wrote:
Carlos E. R. wrote:
Notice that you could also create the filesystem on the whole disk, no partitions.
You can, but IMHO it is a bad decision to do so. It exposes you to several more ways you can shoot yourself in the foot,
Why? :-? I can only think that the disk may appear unformatted to Windows >:-p
and can confuse anybody else who looks at the disk (of course, that might be a bonus in some circumstances).
Indeed :-)
There is another opinion at:
The pro or against are trivial. Another one, against: a redo of a partition table made some years ago was impossible because the current tool insisisted on aligning on megabytes instead of tracks (for the same model of disk), resulting on slightly smaller sizes, making the dd of the partitions impossible. :-) Again, trivial :-) - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlHoGAEACgkQtTMYHG2NR9VePQCgkQyU7zGvHd9GPt/WaBwafUvh i6MAnjSMJRepzcMAb3cErLiGj/Ydyfqj =ePjZ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (8)
-
Andrey Borzenkov -
Carlos E. R. -
Cristian Rodríguez -
Damon Register -
Dave Howorth -
John Andersen -
Lew Wolfgang -
Tom Wekell