[Bug 838705] New: mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

bugzilla_noreply＠novell.com

5 Sep 2013 5 Sep '13

15:34

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c0 Summary: mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock Classification: openSUSE Product: openSUSE Factory Version: 13.1 Milestone 4 Platform: x86-64 OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: High Availability AssignedTo: lzhong@suse.com ReportedBy: rgoldwyn@suse.com QAContact: qa-bugs@suse.de CC: lmb@suse.com, ygao@suse.com Found By: Development Blocker: --- While dealing with existing partitions mkfs.ocfs2/tunefs.ocfs2 hang on dlm_unlock. Surprisingly, it is dlm_unlock which hangs rather than dlm_lock. I have tried producing this outside of the ocfs2-tools, but have failed to narrow it down as yet. In order to reproduce this bug, start the cluster services and run mkfs.ocfs2 twice. It will hang on the second instance (if you started with a zeroed device) or on the first one if it was an existing ocfs2 partition. Explanation of procedure: The tools creates a lockspace, and locks and unlocks each journal file on the ocfs2 partition to ensure the device is not mounted on another partition. It hangs on the first unlock. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

Show replies by date

bugzilla_noreply＠novell.com

5 Sep 5 Sep

15:46

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c1 Lars Marowsky-Bree changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - None |P3 - Medium --- Comment #1 from Lars Marowsky-Bree 2013-09-05 15:46:14 UTC --- Can you provide an strace of both mkfs.ocfs2 runs and a dlm_tool dump / lockdebug for the hang? Thanks! -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

19:14

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c2 --- Comment #2 from Goldwyn Rodrigues 2013-09-05 19:14:12 UTC --- Created an attachment (id=556050) --> (http://bugzilla.novell.com/attachment.cgi?id=556050) mkfs.ocfs2 strace -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

19:16

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c3 --- Comment #3 from Goldwyn Rodrigues 2013-09-05 19:16:32 UTC --- mkfs.ocfs2 1.8.2 Cluster stack: pcmk Cluster name: mycluster Stack Flags: 0x0 NOTE: Feature extended slot map may be enabled Overwriting existing ocfs2 partition. ^C Program received signal SIGINT, Interrupt. 0x00007ffff6797924 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff6797924 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ffff77ccc04 in sync_write_v6 () from /usr/lib64/libdlm.so.3 #2 0x00007ffff77cdada in dlm_ls_unlock () from /usr/lib64/libdlm.so.3 #3 0x000000000042533b in o2dlm_unlock_lock_res_fsdlm (ctxt=0x642010, lockres=0x63e700) at o2dlm.c:923 #4 o2dlm_unlock_lock_res (ctxt=ctxt@entry=0x642010, lockres=lockres@entry=0x63e700) at o2dlm.c:1266 #5 0x00000000004259a3 in o2dlm_unlock (ctxt=0x642010, lockid=lockid@entry=0x7fffffffae10 "M", '0' , "24f7fa2b9d") at o2dlm.c:1307 #6 0x000000000040da9f in ocfs2_meta_unlock (fs=fs@entry=0x63cbb0, ci=<optimized out>) at dlm.c:385 #7 0x000000000040dc05 in ocfs2_lock_down_cluster (fs=0x63cbb0) at dlm.c:83 #8 0x00000000004085e7 in ocfs2_check_volume (s=s@entry=0x63c070) at check.c:372 #9 0x0000000000403ba2 in main (argc=<optimized out>, argv=0x7fffffffe518) at mkfs.c:581 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

19:17

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c4 --- Comment #4 from Goldwyn Rodrigues 2013-09-05 19:17:38 UTC --- Created an attachment (id=556051) --> (http://bugzilla.novell.com/attachment.cgi?id=556051) lockdebug B529A67B098D4BDCA792129F7307DD69 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

19:18

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c5 --- Comment #5 from Goldwyn Rodrigues 2013-09-05 19:18:32 UTC --- Created an attachment (id=556052) --> (http://bugzilla.novell.com/attachment.cgi?id=556052) dlm_tool dump -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

6 Sep 6 Sep

02:18

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c6 --- Comment #6 from Goldwyn Rodrigues 2013-09-06 02:18:27 UTC --- Created an attachment (id=556090) --> (http://bugzilla.novell.com/attachment.cgi?id=556090) mkfs.ocfs2 strace of a zeroed device This problem happens only when using tools on existing ocfs2 partitions because it checks if the device is mounted by locking journals. So, if there is no filesystem present, it does not need to lock. Here is the strace of a working mkfs.ocfs2 from a zeroed device. Note there are no pread("INODE01..." in this case. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

9 Sep 9 Sep

03:40

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c7 --- Comment #7 from Lidong Zhong 2013-09-09 03:39:59 UTC --- (In reply to comment #3)

...

mkfs.ocfs2 1.8.2 Cluster stack: pcmk Cluster name: mycluster Stack Flags: 0x0 NOTE: Feature extended slot map may be enabled Overwriting existing ocfs2 partition. ^C Program received signal SIGINT, Interrupt. 0x00007ffff6797924 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007ffff6797924 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ffff77ccc04 in sync_write_v6 () from /usr/lib64/libdlm.so.3 #2 0x00007ffff77cdada in dlm_ls_unlock () from /usr/lib64/libdlm.so.3 #3 0x000000000042533b in o2dlm_unlock_lock_res_fsdlm (ctxt=0x642010, lockres=0x63e700) at o2dlm.c:923

Seems strange here. The function on line 923 in o2dlm.c is fsdlm_ls_unlock_wait(). It should be dlm_ls_unlock_wait from libdlm_lt.so.3. I don't know why the stack here shows it is dlm_ls_unlock () from /usr/lib64/libdlm.so.3. The code I checked is the latest branch from network:ha-clustering:Factory. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

10 Sep 10 Sep

07:30

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c8 --- Comment #8 from Lidong Zhong 2013-09-10 07:30:14 UTC --- Hi Goldwyn, I couldn't reproduce this in your test machines. I booted the machine with kernel 3.11.0-rc7-1.g99e1318-desktop and removed the gfs2 RA. Then I run mkfs.ocfs2 -F --cluster-name mycluster --cluster-stack pcmk /dev/sdb1 and there are no hang at all. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

12:45

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c9 --- Comment #9 from Goldwyn Rodrigues 2013-09-10 12:45:04 UTC --- (In reply to comment #7)

...

Seems strange here. The function on line 923 in o2dlm.c is fsdlm_ls_unlock_wait(). It should be dlm_ls_unlock_wait from libdlm_lt.so.3. I don't know why the stack here shows it is dlm_ls_unlock () from /usr/lib64/libdlm.so.3. The code I checked is the latest branch from network:ha-clustering:Factory.

fs_dlm_ls_unlock maps to dlm_ls_unlock because the library is opened dynamically using dlopen(). Check the code next to dlopen() -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

12:50

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c10 Goldwyn Rodrigues changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rgoldwyn@suse.com --- Comment #10 from Goldwyn Rodrigues 2013-09-10 12:50:44 UTC --- (In reply to comment #8)

...

Hi Goldwyn, I couldn't reproduce this in your test machines. I booted the machine with kernel 3.11.0-rc7-1.g99e1318-desktop and removed the gfs2 RA. Then I run mkfs.ocfs2 -F --cluster-name mycluster --cluster-stack pcmk /dev/sdb1 and there are no hang at all.

-F disables cluster checks. You have to run without -F. Also ensure ocfs2-kmp is installed. With the dlm service running. Try the following steps: # modprobe ocfs2_stack_user # modprobe ocfs2 # echo "pcmk" > /sys/fs/ocfs2/cluster_stack # mkfs.ocfs2 /dev/sdb1 You may have to execute mkfs.ocfs2 twice if the device is zeroed. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

12 Sep 12 Sep

09:34

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c11 --- Comment #11 from Lidong Zhong 2013-09-12 09:34:16 UTC --- Hi Goldwyn, I have made a patch and submmitted to the repo.(req#198636) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

bugzilla_noreply＠novell.com

8 Nov 8 Nov

01:56

New subject: [Bug 838705] mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

https://bugzilla.novell.com/show_bug.cgi?id=838705 https://bugzilla.novell.com/show_bug.cgi?id=838705#c12 Lidong Zhong changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CLOSED Resolution| |FIXED --- Comment #12 from Lidong Zhong 2013-11-08 01:56:03 UTC --- close it now -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

3830

Age (days ago)

3894

Last active (days ago)

List overview

Download

12 comments

1 participants

participants (1)

bugzilla_noreply＠novell.com

[Bug 838705] New: mkfs.ocfs2/tunefs.ocfs2 hangs on dlm_unlock

tags

participants (1)