http://bugzilla.suse.com/show_bug.cgi?id=1019792 Gang He changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|lmb@suse.com |ghe@suse.com -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1019792 http://bugzilla.suse.com/show_bug.cgi?id=1019792#c1 --- Comment #1 from Gang He --- The first node back trace, ├─sudo───runTest.sh───discontig_runne───activate_discon───mpirun─┬─fillup_contig_b───gen_extents │ ├─2*[ssh] │ └─{mpirun} ocfs2dev1:~ # cat /proc/20337/stack [<ffffffff8c263e67>] __wait_on_buffer+0x27/0x30 [<ffffffffc06ce143>] ocfs2_read_blocks+0x3f3/0x630 [ocfs2] [<ffffffffc070c4f3>] ocfs2_read_group_descriptor+0x43/0x130 [ocfs2] [<ffffffffc070cfb3>] ocfs2_search_chain+0xd3/0x8e0 [ocfs2] [<ffffffffc070dc14>] ocfs2_claim_suballoc_bits+0x454/0x720 [ocfs2] [<ffffffffc070eae4>] __ocfs2_claim_clusters+0xa4/0x290 [ocfs2] [<ffffffffc07104cc>] ocfs2_claim_clusters+0x1c/0x20 [ocfs2] [<ffffffffc06f08a1>] ocfs2_local_alloc_slide_window+0x281/0x8f0 [ocfs2] [<ffffffffc06f2216>] ocfs2_reserve_local_alloc_bits+0x1b6/0x2b0 [ocfs2] [<ffffffffc07101e0>] ocfs2_reserve_clusters_with_limit+0x110/0x3c0 [ocfs2] [<ffffffffc0710705>] ocfs2_lock_allocators+0x185/0x200 [ocfs2] [<ffffffffc06cb50e>] ocfs2_write_begin_nolock+0xb9e/0x18d0 [ocfs2] [<ffffffffc06cc342>] ocfs2_write_begin+0x102/0x1c0 [ocfs2] [<ffffffff8c1a086a>] generic_perform_write+0xba/0x190 [<ffffffff8c1a3185>] __generic_file_write_iter+0x185/0x1d0 [<ffffffffc06e5c2a>] ocfs2_file_write_iter+0x43a/0xaa0 [ocfs2] [<ffffffff8c22be37>] __vfs_write+0xc7/0x140 [<ffffffff8c22cfc5>] vfs_write+0xb5/0x1a0 [<ffffffff8c22e466>] SyS_pwrite64+0x76/0x90 [<ffffffff8c71937b>] entry_SYSCALL_64_fastpath+0x1e/0xad [<ffffffffffffffff>] 0xffffffffffffffff -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1019792 Gang He changed: What |Removed |Added ---------------------------------------------------------------------------- Hardware|Other |x86-64 -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1019792 http://bugzilla.suse.com/show_bug.cgi?id=1019792#c4 --- Comment #4 from Gang He --- The kernel version is 4.9.0-rc8-1.g59795cb-vanilla -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1019792 http://bugzilla.suse.com/show_bug.cgi?id=1019792#c5 --- Comment #5 from Gang He --- Assign to Lei, try to reproduce this issue with the latest kernel, to see if we can encounter this problem again since some related fixes have been in it. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1019792 http://bugzilla.suse.com/show_bug.cgi?id=1019792#c6 Larry Chen changed: What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED --- Comment #6 from Larry Chen --- This is NOT a ocfs2 caused bug. It really looks like a dead lock problem, but actually that's not the truth. The ultimate cause of this problem is that block device blocks the node1 process before it could finish reading a block, but unfortunately, meanwhile, the process also holds the lock of global bitmap. And when node1 is blocked, node2 and node3 also need to lock global bitmap, so, they all hangs. Code path of node1's locking global bitmap: --->ocfs2_local_alloc_slide_window ------->ocfs2_local_alloc_reserve_for_window ----------->ocfs2_reserve_cluster_bitmap_bits ---------------->ocfs2_reserve_suballoc_bits -------------------->ocfs2_inode_lock(alloc_inode, &bh, 1); //here, alloc_inode represents the global bitmap. -- You are receiving this mail because: You are on the CC list for the bug.