https://bugzilla.novell.com/show_bug.cgi?id=775694 https://bugzilla.novell.com/show_bug.cgi?id=775694#c1 --- Comment #1 from Mel Gorman 2012-08-14 16:00:20 UTC --- (In reply to comment #0)

When one process is doing significant I/O, no other process can read or write data and is suspended until the first process finally finishes.

1. The bug is against openSUSE 12.1 but what kernel version are you using exactly? 2. What filesystem? 3. Is the filesystem you are doing the significant IO on and the music the same filesystem? 4. Are barriers enabled? If so, what is the experience if they are disabled? 5. What is the value of /proc/sys/vm/dirty_ratio? If it's high (like 40), what is the experience if it's lower like 10?

How to reproduce:

Play some music with your favorite music player. Then in some shell, type something like:

cp -a hugefile.in hugefile.out

after some seconds, music will stop playing and only continue after the cp has finished. I'm currently copying a 53 GB file and music stopped more than one minute ago!

I do not see the same problem as such but I do know that IO interactivity for the 3.1 kernel was particularly bad. If there is a lot of different IO requests then I would sometimes see a process stall for a long period of time waiting on log space for ext4. This was later fixed but not all the fixes were not backported. I recently backported a second round of patches related to excessive reclaim and stalls while copying to USB. I do not believe a kernel has been released with them yet but I could build a test kernel if you wanted to try it. However, it's not exactly the same problem On a similar note, have you tried the tumbleweed kernel? It should be 3.4 based.

<SNIP> Meanwhile the "cp" has finished and music continues to play. After 2 (TWO!!!) minutes. On a system with 8 GB RAM and core-7 processor.

The number of cores is not as important if they are stalled waiting on access to storage. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=775694 https://bugzilla.novell.com/show_bug.cgi?id=775694#c3 --- Comment #3 from Hubert Mantel 2012-08-15 14:57:45 UTC --- Thanks for taking care. I will try to give some additional information: The big copy happened on the local hard disk where all filesystems are ext4: Mandelbrot:~ # mount |grep sda /dev/sda1 on / type ext4 (rw,relatime,user_xattr,acl,barrier=1,data=ordered) /dev/sda3 on /old type ext4 (rw,relatime,user_xattr,acl,barrier=1,data=ordered) /dev/sda4 on /scratch type ext4 (rw,relatime,user_xattr,acl,barrier=1,data=ordered) The music currently comes from an USB disk with ext3 filesystem. The problem is not restricted to listening to music; some colleagues report that switching virtual screens or to firefox just yields a white window when some I/O intensive process is running in the background. As to the exact kernel version: I included the output of "uname -a" in my initial report, so I do not know which additional information is needed. Btw, the problem is NOT 100% reproducible! Yesterday I had to copy a huge file again (I'm restoring snapshots of VMs quite often), and I did not encounter any problem. I also just want to make sure this issue will not be forgotten. Starting tomorrow I'm in vacation anyway. In Praha :) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=775694 https://bugzilla.novell.com/show_bug.cgi?id=775694#c5 --- Comment #5 from Mel Gorman 2012-08-16 09:50:05 UTC --- (In reply to comment #4)

OK, then it gets more interesting and my patch isn't going to help. So the music is on a separate, otherwise unused disk.

And the information on the USB disk being read so it's not like we're waiting on data to be written to slow storage. Is there any possibility that a barrier due to the write on ext4 can cause a read on a ext3 USB stick to be delayed? Hubert, what are the mount options used for the USB stick? In particular, is it mounted with atime because that would mean that a write is necessary when accessing a file which could be delayed for a long time.

That means that the problem shouldn't be caused by disk being busy due to other IO. The only idea I have is that reclaim is too eager and reclaims some of the loaded music data / player executable (due to dirty pages from large copy putting stress to reclaim).

That seems unlikely, the amount of data needed by the music player should be so low that I find it hard to believe it's getting reclaimed that quickly. That said the most recent round of backports for the 3.1 kernel included a fix for keeping mapped file pages resident but I don't think it'll fix this particular problem but if you build a test kernel with your ext3 latency fixes then it'll also include those reclaim fixes.

But to confirm this we'd need some back traces of blocked processes when audio stalls happen I guess. Mel, do you have other ideas?

If Huburt could capture vmstat -n 1 during the whole test it would be useful to see if swapping is actually happening. Second, capturing a log of the following substituting MUSICPLAYER for whatever the music player process is called should tell us where it is stalling while [ 1 ]; do date; cat /proc/`pidof MUSICPLAYER`/stack; sleep 1; done We also know from https://lkml.org/lkml/2012/7/10/132 that read latencies for ext3 can be very bad. Hubert, to see if this is ext3 specific would it be possible for you to test with a similarly sized USB stick formatted with ext4 instead of ext3 please? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=775694 https://bugzilla.novell.com/show_bug.cgi?id=775694#c7 --- Comment #7 from Jan Kara 2012-10-25 19:48:08 UTC --- Hubert, do you still see the problems? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.