[opensuse-kernel] Resume take a long time to bring in the images to main memory
Hi, I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time. I noticed the following in my top (3 outputs taken at different times) after 5 mins-------------------------------- Cpu(s): 1.2%us, 2.0%sy, 0.0%ni, 56.7%id, 32.2%wa, 0.1%hi, 7.9%si, 0.0%st Mem: 2075132k total, 299748k used, 1775384k free, 264k buffers Swap: 2097140k total, 567580k used, 1529560k free, 26856k cached after 10 mins-------------------------------- Cpu(s): 1.3%us, 2.4%sy, 0.0%ni, 64.8%id, 26.5%wa, 0.1%hi, 4.8%si, 0.0%st Mem: 2075132k total, 256348k used, 1818784k free, 300k buffers Swap: 2097140k total, 581164k used, 1515976k free, 28740k cached after 12 mins-------------------------------- top - 19:29:29 up 17:53, 6 users, load average: 4.63, 5.56, 5.86 Tasks: 163 total, 1 running, 160 sleeping, 0 stopped, 2 zombie Cpu(s): 0.9%us, 1.5%sy, 0.0%ni, 57.8%id, 36.6%wa, 0.1%hi, 3.1%si, 0.0%st Mem: 2075132k total, 187348k used, 1887784k free, 300k buffers Swap: 2097140k total, 609680k used, 1487460k free, 18480k cached -------------------------------- It shows that I/O is very high during this period. Though I have 2GB memory, only 200-300MB is used where as 550-600MB swap is used. Why is it this way? Also, only one task is running out of 163 total tasks in the system. The kernel I have is 2.6.22.9-0.4-default #1 SMP Here is another output that I took 15 minutes after resuming. Again, RES is very low for all processes and it looks like VM is still thinking that the main memory is not available. A VM bug? I can still see my harddisk grinding continuously. top - 23:34:43 up 21:58, 6 users, load average: 3.16, 3.37, 4.73 Tasks: 165 total, 1 running, 162 sleeping, 0 stopped, 2 zombie Cpu(s): 1.4%us, 2.4%sy, 0.0%ni, 45.2%id, 44.2%wa, 0.3%hi, 6.5%si, 0.0%st Mem: 2075132k total, 467476k used, 1607656k free, 600k buffers Swap: 2097140k total, 680644k used, 1416496k free, 128576k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3081 root 15 0 564m 74m 14m S 2 3.7 43:26.70 X 4272 jjaimon 15 0 260m 68m 13m S 0 3.4 8:02.26 thunderbird-bin 20011 root 5 -10 591m 52m 50m S 3 2.6 22:47.90 vmware-vmx 4607 jjaimon 15 0 330m 42m 7584 S 2 2.1 19:34.48 firefox-bin 3967 jjaimon 15 0 114m 18m 2632 S 3 0.9 23:13.73 main-menu Load average is shown high. But, all that I have is 3-4 regular applications (firefox, gnome-terminal and thunderbird) and one VMWare instance. Regards, --jaimon -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Hello, On Sun 21-10-07 22:43:16, Jaimon Jose wrote:
I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time. Yes, that's the problem with swapping in large applications - you get mostly random-access load on the swap partition and so it takes horribly long to swap in anything larger...
I noticed the following in my top (3 outputs taken at different times) after 5 mins-------------------------------- Cpu(s): 1.2%us, 2.0%sy, 0.0%ni, 56.7%id, 32.2%wa, 0.1%hi, 7.9%si, 0.0%st Mem: 2075132k total, 299748k used, 1775384k free, 264k buffers Swap: 2097140k total, 567580k used, 1529560k free, 26856k cached after 10 mins-------------------------------- Cpu(s): 1.3%us, 2.4%sy, 0.0%ni, 64.8%id, 26.5%wa, 0.1%hi, 4.8%si, 0.0%st Mem: 2075132k total, 256348k used, 1818784k free, 300k buffers Swap: 2097140k total, 581164k used, 1515976k free, 28740k cached after 12 mins-------------------------------- top - 19:29:29 up 17:53, 6 users, load average: 4.63, 5.56, 5.86 Tasks: 163 total, 1 running, 160 sleeping, 0 stopped, 2 zombie Cpu(s): 0.9%us, 1.5%sy, 0.0%ni, 57.8%id, 36.6%wa, 0.1%hi, 3.1%si, 0.0%st Mem: 2075132k total, 187348k used, 1887784k free, 300k buffers Swap: 2097140k total, 609680k used, 1487460k free, 18480k cached -------------------------------- It shows that I/O is very high during this period. Though I have 2GB memory, only 200-300MB is used where as 550-600MB swap is used. Why is it this way? Also, only one task is running out of 163 total tasks in the system. The kernel I have is 2.6.22.9-0.4-default #1 SMP
Here is another output that I took 15 minutes after resuming. Again, RES is very low for all processes and it looks like VM is still thinking that the main memory is not available. A VM bug? I can still see my harddisk grinding continuously.
top - 23:34:43 up 21:58, 6 users, load average: 3.16, 3.37, 4.73 Tasks: 165 total, 1 running, 162 sleeping, 0 stopped, 2 zombie Cpu(s): 1.4%us, 2.4%sy, 0.0%ni, 45.2%id, 44.2%wa, 0.3%hi, 6.5%si, 0.0%st Mem: 2075132k total, 467476k used, 1607656k free, 600k buffers Swap: 2097140k total, 680644k used, 1416496k free, 128576k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3081 root 15 0 564m 74m 14m S 2 3.7 43:26.70 X
4272 jjaimon 15 0 260m 68m 13m S 0 3.4 8:02.26 thunderbird-bin 20011 root 5 -10 591m 52m 50m S 3 2.6 22:47.90 vmware-vmx
4607 jjaimon 15 0 330m 42m 7584 S 2 2.1 19:34.48 firefox-bin
3967 jjaimon 15 0 114m 18m 2632 S 3 0.9 23:13.73 main-menu
Load average is shown high. But, all that I have is 3-4 regular applications (firefox, gnome-terminal and thunderbird) and one VMWare instance. You can try using swap-prefetch patches from Con Kolivas from http://ck.kolivas.org/patches/swap-prefetch/mm-swap_prefetch-41.patch (they are also merged in -mm kernel). That should help you.
Honza
--
Jan Kara
Jan Kara wrote, On 10/24/2007 08:51 PM:
Hello,
On Sun 21-10-07 22:43:16, Jaimon Jose wrote:
I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time. Yes, that's the problem with swapping in large applications - you get mostly random-access load on the swap partition and so it takes horribly long to swap in anything larger...
As I mentioned below, I don't have any large applications. All that I have is firefox, thunderbird and VMWare. One more correction is that my laptop never recovers from this state. I noticed that, if I restart X, it comes back to normal. For eg. Cpu(s): 1.8%us, 2.8%sy, 0.3%ni, 78.7%id, 2.5%wa, 0.0%hi, 13.8%si, 0.0%st Mem: 2075132k total, 1954952k used, 120180k free, 349592k buffers Swap: 2097140k total, 14392k used, 2082748k free, 919952k cached You can see that most of the memory is used and hardly any swap area is used. This is by just restarting the X. --jaimon -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Wed 24-10-07 23:27:41, Jaimon Jose wrote:
Jan Kara wrote, On 10/24/2007 08:51 PM:
Hello,
On Sun 21-10-07 22:43:16, Jaimon Jose wrote:
I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time. Yes, that's the problem with swapping in large applications - you get mostly random-access load on the swap partition and so it takes horribly long to swap in anything larger...
As I mentioned below, I don't have any large applications. All that I have is firefox, thunderbird and VMWare. One more correction is that my laptop never recovers from this state. I noticed that, if I restart X, it comes back to normal. For eg. Cpu(s): 1.8%us, 2.8%sy, 0.3%ni, 78.7%id, 2.5%wa, 0.0%hi, 13.8%si, 0.0%st Mem: 2075132k total, 1954952k used, 120180k free, 349592k buffers Swap: 2097140k total, 14392k used, 2082748k free, 919952k cached
You can see that most of the memory is used and hardly any swap area is used. This is by just restarting the X. Yes, but when you suspend do disk, almost all the memory is first "swapped out". So after you resume, you are in a situation as if all the applications were in swap. So in your case you'll be something like 1.5GB to swap in after resume... How have you found out that the laptop never recovers? Or better, how long have you waited? :) Still, I think that swap-prefetch patches are worth trying...
Honza
--
Jan Kara
Jan Kara wrote, On 10/25/2007 01:08 AM:
How have you found out that the laptop never recovers? Or better, how long have you waited? :) I waited upto 30mins. The memory usage never increased beyond 200-300MB and the system was swapping always (vmstat) though I have over 1.5GB free memory
Still, I think that swap-prefetch patches are worth trying...
sure. I will try this out... Thanks --jaimon -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Thu 25-10-07 01:29:57, Jaimon Jose wrote:
Jan Kara wrote, On 10/25/2007 01:08 AM:
How have you found out that the laptop never recovers? Or better, how long have you waited? :) I waited upto 30mins. The memory usage never increased beyond 200-300MB and the system was swapping always (vmstat) though I have over 1.5GB free memory I see. Then this sounds like a bug. Maybe some of our VM guys will have a better clue what to look at... Nick?
Honza
--
Jan Kara
On Thu, Oct 25, 2007 at 02:05:15PM +0200, Jan Kara wrote:
On Thu 25-10-07 01:29:57, Jaimon Jose wrote:
Jan Kara wrote, On 10/25/2007 01:08 AM:
How have you found out that the laptop never recovers? Or better, how long have you waited? :) I waited upto 30mins. The memory usage never increased beyond 200-300MB and the system was swapping always (vmstat) though I have over 1.5GB free memory I see. Then this sounds like a bug. Maybe some of our VM guys will have a better clue what to look at... Nick?
That's strange. Can you cat /proc/meminfo and /proc/zoneinfo, then post a few lines of vmstat 1? Also, try swapoff -a ; swapon -a and then post some more lines of vmstat 1. Thanks, Nick -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Sun 2007-10-21 22:43:16, Jaimon Jose wrote:
Hi, I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time.
Hmmm, can you check dmesg for hints? Like ide driver timeouting continuously...? -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Pavel Machek wrote, On 10/27/2007 12:24 PM:
On Sun 2007-10-21 22:43:16, Jaimon Jose wrote:
I have been observing that laptop is too slow when it resumes from suspend (T61p with 2GB ram), it takes more than 10 minutes before the system starts responding normally. There will be continuous read from the hard disk during this time.
Hmmm, can you check dmesg for hints? Like ide driver timeouting continuously...?
I checked that. Nothing suspicious. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Nick Piggin wrote, On 10/26/2007 04:15 AM:
On Thu, Oct 25, 2007 at 02:05:15PM +0200, Jan Kara wrote:
On Thu 25-10-07 01:29:57, Jaimon Jose wrote:
Jan Kara wrote, On 10/25/2007 01:08 AM:
How have you found out that the laptop never recovers? Or better, how long have you waited? :) I waited upto 30mins. The memory usage never increased beyond 200-300MB and the system was swapping always (vmstat) though I have over 1.5GB free memory I see. Then this sounds like a bug. Maybe some of our VM guys will have a better clue what to look at... Nick?
That's strange. Can you cat /proc/meminfo and /proc/zoneinfo, then post a few lines of vmstat 1?
Attached file has all the information requested.
Also, try swapoff -a ; swapon -a and then post some more lines of vmstat 1.
Yes. the system came back to the normal state (You can see this in the resume.txt attached.). However, it went back to the original state after a few minutes. I could see the swap increasing. (see the top output in the attached file). Thanks --jaimon jjaimon-ibm(jjaimon)::~>cat /proc/meminfo MemTotal: 2075132 kB MemFree: 1913872 kB Buffers: 408 kB Cached: 17396 kB SwapCached: 41660 kB Active: 78480 kB Inactive: 2452 kB HighTotal: 1178432 kB HighFree: 1085204 kB LowTotal: 896700 kB LowFree: 828668 kB SwapTotal: 2097140 kB SwapFree: 1461432 kB Dirty: 76 kB Writeback: 4 kB AnonPages: 61372 kB Mapped: 19944 kB Slab: 38236 kB SReclaimable: 22460 kB SUnreclaim: 15776 kB PageTables: 4716 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 3134704 kB Committed_AS: 1556764 kB VmallocTotal: 114680 kB VmallocUsed: 30660 kB VmallocChunk: 83636 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 Hugepagesize: 4096 kB jjaimon-ibm(jjaimon)::~>cat /proc/zoneinfo Node 0, zone DMA pages free 3195 min 17 low 21 high 25 scanned 0 (a: 9 i: 9) spanned 4096 present 4064 nr_free_pages 3195 nr_inactive 0 nr_active 0 nr_anon_pages 0 nr_mapped 1 nr_file_pages 0 nr_dirty 0 nr_writeback 0 nr_slab_reclaimable 10 nr_slab_unreclaimable 4 nr_page_table_pages 2 nr_unstable 0 nr_bounce 0 nr_vmscan_write 178 protection: (0, 873, 2014) pagesets cpu: 0 pcp: 0 count: 0 high: 0 batch: 1 cpu: 0 pcp: 1 count: 0 high: 0 batch: 1 vm stats threshold: 2 cpu: 1 pcp: 0 count: 0 high: 0 batch: 1 cpu: 1 pcp: 1 count: 0 high: 0 batch: 1 vm stats threshold: 4 all_unreclaimable: 1 prev_priority: 12 start_pfn: 0 Node 0, zone Normal pages free 203929 min 936 low 1170 high 1404 scanned 108 (a: 21 i: 3) spanned 225280 present 223520 nr_free_pages 203929 nr_inactive 27 nr_active 234 nr_anon_pages 142 nr_mapped 1231 nr_file_pages 241 nr_dirty 15 nr_writeback 0 nr_slab_reclaimable 5604 nr_slab_unreclaimable 3987 nr_page_table_pages 57 nr_unstable 0 nr_bounce 0 nr_vmscan_write 33638 protection: (0, 0, 9134) pagesets cpu: 0 pcp: 0 count: 16 high: 186 batch: 31 cpu: 0 pcp: 1 count: 55 high: 62 batch: 15 vm stats threshold: 8 cpu: 1 pcp: 0 count: 13 high: 186 batch: 31 cpu: 1 pcp: 1 count: 47 high: 62 batch: 15 vm stats threshold: 16 all_unreclaimable: 0 prev_priority: 4 start_pfn: 4096 Node 0, zone HighMem pages free 264747 min 128 low 434 high 740 scanned 1931 (a: 0 i: 13) spanned 294608 present 292307 nr_free_pages 264747 nr_inactive 568 nr_active 26053 nr_anon_pages 20705 nr_mapped 4934 nr_file_pages 20276 nr_dirty 6 nr_writeback 0 nr_slab_reclaimable 0 nr_slab_unreclaimable 0 nr_page_table_pages 1130 nr_unstable 0 nr_bounce 0 nr_vmscan_write 217566 protection: (0, 0, 0) pagesets cpu: 0 pcp: 0 count: 15 high: 186 batch: 31 cpu: 0 pcp: 1 count: 1 high: 62 batch: 15 vm stats threshold: 10 cpu: 1 pcp: 0 count: 98 high: 186 batch: 31 cpu: 1 pcp: 1 count: 9 high: 62 batch: 15 vm stats threshold: 20 all_unreclaimable: 0 prev_priority: 4 start_pfn: 229376 jjaimon-ibm(jjaimon)::~>vmstat 1 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 10 634920 1892892 696 23392 3 2 13 16 83 13 1 22 76 1 1 7 634936 1896472 708 23212 308 648 660 648 1215 1357 2 9 70 19 2 12 635476 1896540 764 23652 672 1056 1428 1068 859 2054 2 13 10 75 0 4 635596 1896984 752 24496 900 592 1860 680 1393 1493 1 10 27 62 0 4 635656 1897248 804 24824 608 716 1396 716 986 1378 1 10 65 25 3 6 635836 1899420 536 24600 264 1532 448 1540 1344 1066 2 10 61 28 1 2 635896 1905060 340 20668 212 812 308 936 824 1254 1 12 17 69 0 7 635896 1906840 356 19276 504 1192 560 1200 1070 1652 2 10 46 42 2 5 635900 1916276 304 15560 376 1372 460 1496 862 1882 2 14 53 30 0 9 636092 1920712 320 14372 392 1424 732 1436 1329 749 1 8 6 85 0 11 636128 1922144 336 14268 332 792 572 792 561 393 1 13 0 86 0 11 636164 1919272 368 15816 1824 332 3476 332 1770 1346 2 13 0 86 ------------------------------------------------------------------------------------ top - 00:50:14 up 15:54, 5 users, load average: 6.38, 6.20, 6.67 Tasks: 158 total, 3 running, 154 sleeping, 0 stopped, 1 zombie Cpu(s): 1.3%us, 3.3%sy, 0.0%ni, 43.8%id, 45.5%wa, 0.2%hi, 5.9%si, 0.0%st Mem: 2075132k total, 201884k used, 1873248k free, 744k buffers Swap: 2097140k total, 636840k used, 1460300k free, 33760k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12682 jjaimon 15 0 25228 4248 1944 S 11 0.2 5:56.29 compiz 3009 root 15 0 580m 32m 7112 S 3 1.6 53:13.87 X 5357 jjaimon 15 0 235m 13m 5872 S 2 0.7 8:27.31 thunderbird-bin 4025 jjaimon 16 0 98.0m 14m 11m D 1 0.7 0:00.32 gedit 12706 jjaimon 15 0 98200 8344 2536 S 1 0.4 2:04.79 main-menu 4568 jjaimon 15 0 234m 7380 3672 R 1 0.4 42:56.24 firefox-bin -------------------------------------------------------------------------------------- jjaimon-ibm(jjaimon)::~>sudo swapoff -a; sudo swapon -a
/var/log/message Oct 29 00:56:52 jjaimon-ibm sudo: jjaimon : TTY=pts/2 ; PWD=/home/jjaimon ; USER=root ; COMMAND=/sbin/swapon -a Oct 29 00:56:52 jjaimon-ibm kernel: Adding 2097140k swap on /dev/sda3. Priority:-2 extents:1 across:2097140k
top output after swapoff & swapon
top - 00:57:26 up 16:01, 5 users, load average: 3.04, 5.08, 6.05 Tasks: 158 total, 2 running, 155 sleeping, 0 stopped, 1 zombie Cpu(s): 0.6%us, 1.9%sy, 0.0%ni, 66.6%id, 25.1%wa, 0.1%hi, 5.8%si, 0.0%st Mem: 2075132k total, 627160k used, 1447972k free, 232k buffers Swap: 2097140k total, 146624k used, 1950516k free, 22072k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12682 jjaimon 15 0 25228 7856 1560 S 4 0.4 6:09.24 compiz 3009 root 15 0 590m 155m 7484 R 2 7.7 53:42.22 X 5357 jjaimon 15 0 243m 60m 5164 S 2 3.0 8:34.07 thunderbird-bin 28308 jjaimon 15 0 178m 17m 3200 S 1 0.9 0:16.40 soffice.bin 4025 jjaimon 15 0 126m 6816 2172 S 0 0.3 0:03.37 gedit --------------------------------------------------------------------------------------
after 5 minutes
top - 01:01:23 up 16:05, 5 users, load average: 2.21, 3.47, 5.15 Tasks: 158 total, 4 running, 153 sleeping, 0 stopped, 1 zombie Cpu(s): 2.4%us, 5.4%sy, 0.0%ni, 78.2%id, 6.7%wa, 0.0%hi, 7.3%si, 0.0%st Mem: 2075132k total, 282508k used, 1792624k free, 1780k buffers Swap: 2097140k total, 630636k used, 1466504k free, 50264k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3009 root 15 0 588m 28m 7508 S 22 1.4 54:03.66 X 8536 root 15 0 157m 3408 2100 D 2 0.2 3:16.01 vmware 12682 jjaimon 15 0 25228 3664 1456 S 1 0.2 6:16.60 compiz 4568 jjaimon 15 0 234m 17m 6348 S 1 0.9 43:39.26 firefox-bin 12693 jjaimon 15 0 126m 5984 3072 S 1 0.3 1:14.05 gnome-panel jjaimon-ibm(jjaimon)::~>vmstat 1 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 1 0 630208 1762232 1896 52460 5 4 17 18 84 16 1 22 76 2 1 0 630208 1761544 1896 52500 64 0 64 0 726 1431 2 12 85 0 1 1 630208 1761500 1896 52568 92 0 184 0 1037 1221 1 10 88 1 1 1 630208 1758712 1924 53528 2480 0 3436 100 984 2013 1 12 62 25 0 0 630208 1757396 1924 53716 612 0 800 0 746 1411 1 13 82 5 1 0 630208 1757396 1924 53716 0 0 0 20 967 1353 1 13 87 0 1 0 630208 1757272 1932 53708 32 0 32 44 892 1165 1 9 89 1 0 0 630208 1757524 1932 53784 64 0 132 0 521 1581 2 11 86 1 cat /proc/meminfo MemTotal: 2075132 kB MemFree: 1714624 kB Buffers: 2188 kB Cached: 64356 kB SwapCached: 175916 kB Active: 207584 kB Inactive: 73828 kB HighTotal: 1178432 kB HighFree: 886628 kB LowTotal: 896700 kB LowFree: 827996 kB SwapTotal: 2097140 kB SwapFree: 1467368 kB Dirty: 524 kB Writeback: 0 kB AnonPages: 132408 kB Mapped: 30880 kB Slab: 37560 kB SReclaimable: 22496 kB SUnreclaim: 15064 kB PageTables: 4800 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 3134704 kB Committed_AS: 1620108 kB VmallocTotal: 114680 kB VmallocUsed: 30660 kB VmallocChunk: 83636 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 Hugepagesize: 4096 kB
participants (4)
-
Jaimon Jose -
Jan Kara -
Nick Piggin -
Pavel Machek