[opensuse] How to reclaim performance lost over gradual slowdown?
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 08/10/14 00:26, Roger Luedecke wrote:
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed?
Never tried it myself, but would it be worth trying a tool like Bleachbit? http://bleachbit.sourceforge.net/ Peter -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/7/2014 3:26 PM, Roger Luedecke wrote:
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed?
I'm not actually seeing this myself. I just don't see any accumulating slowdown. Having more programs in the program menu has nothing to do with it, as long as they aren't started all at once. This isn't windows. You can have a boat load of software installed with out that causing any slowdowns. If you carefully look in TOP to see what is running under your account, and get rid of anything you don't use often, so that you don't have a bunch of stuff started automatically this shouldn't happen. In KDE its Start-icon, configure desktop, Start-up and Shutdown, and look at the autostart items and also on that same page look in Session Management to make sure you aren't restarting everything that was running when you logged off. (Unless you WANT that behavior). Make every program running in your tray justify itself to you. If you haven't used it in months, then don't autostart it. Other than that, sometimes Opensuse will hide a failing disk drive so well that you things are just getting slow, when it it really fighting for survival. You might run smartctl and see what it has to say about your disk. I can't tell from you're using Gnome or KDE. -- _____________________________________ ---This space for rent--- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown.
I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot. jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 10/07/2014 10:55 PM, jdd wrote:
Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown.
I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot.
jdd
I started to see sudden slowdowns, but it was a failing drive. But the machine was not slow prior to that. I just don't think most linux file systems are all that prone to getting fragmented, unlike some windows file systems. Your file tends to be all in one place in linux, and it only starts fragmenting when it can't find ANY block big enough to hold the whole thing. -- Explain again the part about rm -rf / -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
John Andersen wrote on 2014-10-07 23:08 (UTC-0700):
I just don't think most linux file systems are all that prone to getting fragmented, unlike some windows file systems. Your file tends to be all in one place in linux, and it only starts fragmenting when it can't find ANY block big enough to hold the whole thing.
For file copying, that I can understand, but what about when creating new files out of 30 minute or 120 minute long video transport streams, files what wind up in the 2g-30G range, on a 1TB+ filesystem upwards of half filled? 80% filled? 95% filled? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2014-10-08 08:36, Felix Miata wrote:
John Andersen wrote on 2014-10-07 23:08 (UTC-0700):
I just don't think most linux file systems are all that prone to getting fragmented, unlike some windows file systems. Your file tends to be all in one place in linux, and it only starts fragmenting when it can't find ANY block big enough to hold the whole thing.
For file copying, that I can understand, but what about when creating new files out of 30 minute or 120 minute long video transport streams, files what wind up in the 2g-30G range, on a 1TB+ filesystem upwards of half filled? 80% filled? 95% filled?
Fragmented, of course. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Le 08/10/2014 11:31, Carlos E. R. a écrit :
On 2014-10-08 08:36, Felix Miata wrote:
For file copying, that I can understand, but what about when creating new files out of 30 minute or 120 minute long video transport streams, files what wind up in the 2g-30G range, on a 1TB+ filesystem upwards of half filled? 80% filled? 95% filled?
Fragmented, of course.
not si sure. Such files are only written once (and read manu times) and even 30Gb is not that much compared to a 1 (or more) Tb disk. Even 10% free is still 100Gb :-) fsck.ext4 -fn /dev/sda9(...) /dev/sda9 : 125950/31129600 fichiers (1.4% non contigüs), 83950792/124517806 blocs for a 450Gb partition on a 1Tb disk, very often used for video editing (61% full) 1.4% frag is not that much jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, 08 Oct 2014 07:55:27 +0200
jdd
Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown.
I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot.
I see this too, after just a little swappage, as well as doing copies to/from USB devices and working with large media files. The more swappage that occurs, the slower it gets. Very shortly I have to reboot but that always ends with a kernel panic. Can the cache be swapped out/in now? IIRC, it wasn't allowed before. jd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdebert wrote:
On Wed, 08 Oct 2014 07:55:27 +0200 jdd
wrote: Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown. I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot.
I see this too, after just a little swappage, as well as doing copies to/from USB devices and working with large media files.
The more swappage that occurs, the slower it gets. Very shortly I have to reboot but that always ends with a kernel panic.
Can the cache be swapped out/in now? IIRC, it wasn't allowed before.
just for testing, you could try dropping your cache once in a while. --dropcaches--- #!/bin/bash function dropcaches () { echo -n "3"|sudo dd status=none of=/proc/sys/vm/drop_caches } #if [[ ${BASH_LINE[@]} == 0 ]]; then time dropcaches #fi ==== Throw that in a script-bin and see if it changes things for a little while. It's NOT a solution, but a diagnostic... From <kernroot>/Documentation/sysctl/vm.txt drop_caches Writing to this will cause the kernel to drop clean caches, as well as reclaimable slab objects like dentries and inodes. Once dropped, their memory becomes free. To free pagecache: echo 1 > /proc/sys/vm/drop_caches To free reclaimable slab objects (includes dentries and inodes): echo 2 > /proc/sys/vm/drop_caches To free slab objects and pagecache: echo 3 > /proc/sys/vm/drop_caches This is a non-destructive operation and will not free any dirty objects. To increase the number of objects freed by this operation, the user may run `sync' prior to writing to /proc/sys/vm/drop_caches. This will minimize the number of dirty objects on the system and create more candidates to be dropped. This file is not a means to control the growth of the various kernel caches (inodes, dentries, pagecache, etc...) These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system. Use of this file can cause performance problems. Since it discards cached objects, it may cost a significant amount of I/O and CPU to recreate the dropped objects, especially if they were under heavy use. Because of this, use outside of a testing or debugging environment is not recommended. --- Same place you can read about other knobs and buttons to play er, test and study... grep and a kernel-doc tree are helpful .... You could put memory intensive stuff in its own cgroup -- memgroup and constrain the maximum amount the bunch could have for memory. Usually a matter of creating a subdir under /sys/fs/cgroup/memory and changing values in the subgroup. There are some tools to help do that stuff in "libcgroup-tools", but I have yet to really do anything other play a bit w/network priorities... but I couldn't measure any effective differences so I moved on...;-) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 09 Oct 2014 00:38:52 -0700
Linda Walsh
jdebert wrote:
On Wed, 08 Oct 2014 07:55:27 +0200 jdd
wrote: Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown. I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot.
I see this too, after just a little swappage, as well as doing copies to/from USB devices and working with large media files.
The more swappage that occurs, the slower it gets. Very shortly I have to reboot but that always ends with a kernel panic.
Can the cache be swapped out/in now? IIRC, it wasn't allowed before.
just for testing, you could try dropping your cache once in a while.
Thanks! I'll try that out. I had completely forgotten the kernel docs. I had written them off a long time ago because when I needed them they had been incorrect, too vague or contradictory. I'll fetch a current copy. jd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, Oct 09, 2014 at 12:38:52AM -0700, Linda Walsh wrote:
jdebert wrote:
On Wed, 08 Oct 2014 07:55:27 +0200 jdd
wrote: Le 08/10/2014 00:58, John Andersen a écrit :
I'm not actually seeing this myself. I just don't see any accumulating slowdown. I see sometime dramatic slowdown, probably after USB mass copy, may be related to the cache problem discussed recently, but this do not stay after the next reboot.
I see this too, after just a little swappage, as well as doing copies to/from USB devices and working with large media files.
The more swappage that occurs, the slower it gets. Very shortly I have to reboot but that always ends with a kernel panic.
Can the cache be swapped out/in now? IIRC, it wasn't allowed before.
just for testing, you could try dropping your cache once in a while.
--dropcaches--- #!/bin/bash
function dropcaches () { echo -n "3"|sudo dd status=none of=/proc/sys/vm/drop_caches }
#if [[ ${BASH_LINE[@]} == 0 ]]; then time dropcaches #fi ==== Throw that in a script-bin and see if it changes things for a little while. It's NOT a solution, but a diagnostic...
nice
From <kernroot>/Documentation/sysctl/vm.txt
drop_caches
Writing to this will cause the kernel to drop clean caches, as well as reclaimable slab objects like dentries and inodes. Once dropped, their memory becomes free.
To free pagecache: echo 1 > /proc/sys/vm/drop_caches To free reclaimable slab objects (includes dentries and inodes): echo 2 > /proc/sys/vm/drop_caches To free slab objects and pagecache: echo 3 > /proc/sys/vm/drop_caches
This is a non-destructive operation and will not free any dirty objects. To increase the number of objects freed by this operation, the user may run `sync' prior to writing to /proc/sys/vm/drop_caches. This will minimize the number of dirty objects on the system and create more candidates to be dropped.
This file is not a means to control the growth of the various kernel caches (inodes, dentries, pagecache, etc...) These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system.
Use of this file can cause performance problems. Since it discards cached objects, it may cost a significant amount of I/O and CPU to recreate the dropped objects, especially if they were under heavy use. Because of this, use outside of a testing or debugging environment is not recommended.
--- Same place you can read about other knobs and buttons to play er, test and study... grep and a kernel-doc tree are helpful ....
You could put memory intensive stuff in its own cgroup -- memgroup and constrain the maximum amount the bunch could have for memory. Usually a matter of creating a subdir under /sys/fs/cgroup/memory and changing values in the subgroup. There are some tools to help do that stuff in "libcgroup-tools", but I have yet to really do anything other play a bit w/network priorities... but I couldn't measure any effective differences so I moved on...;-)
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Roger Luedecke wrote on 2014-10-07 15:26 (UTC-0700):
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed?
What percentage of total filesystem space is remaining unused by the installations on your systems? If not copious, maybe the following could play a part, a parallel to file fragmentation that often exists, and defraggers were made for, on some old filesystem types: On a new installation, files from packages surely get installed into contiguous space, with a lot of related packages getting installed together. The resulting seek efficiency surely gets destroyed over time as updates replace packages, and the files within them, and related packages, get a reduced opportunity for contiguity. Not a solution, but maybe a real impact explained. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Roger Luedecke wrote:
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed? ==== It might be your computer is growing short on resources. Like John A. said, you want to monitor your computer to find out what is sluggish and where.
Try " xlatencytop" to see what is really taking time in your system: LatencyTOP version 0.5 (C) 2008 Intel Corporation Cause Maximum Percentage Reading from file 19.0 msec 1.7 % [down] 9.3 msec 0.2 % [ep_poll] 5.0 msec 91.1 % Waiting for event (select) 4.9 msec 6.2 % Waiting for event (poll) 3.2 msec 0.8 % Userspace lock contention 0.6 msec 0.0 % Reading from a pipe 0.1 msec 0.0 % --------------------------------------------------------------------------- Process imap (6286) Total: 107.5 msec fsync() on a file (type 'F' for details) 19.2 msec 18.0 % [xfs_buf_iowait] 17.6 msec 47.1 % [ep_poll] 1.1 msec 35.0 % Will show you by system call, and by program. Are you running lower on memory? Are you using swap? What does 'free' tell you?
free -h total used free Mem: 94G 15G 79G -/+ buffers/cache: 2.6G 91G Swap: 8.0G 0B 8.0G << any in the 'used' category?
What type of disk do you have and what type of file system do you have on it? File systems get fragmented over time, that's why it's good to keep your OS data that doesn't get written much consolidated in separate partitions from "var", "home" "tmp" ... Do you ever defragment? Likely easiest fixes: 1 - more memory -- fastest memory your computer can use (and no faster unless you plan on reusing the memory very soon). (memory is not just about MHz/GHz but also about wait states). and # channels and "N-way" memory. (I find it rather confusing, honestly, but seems like fewer 'N-way's, the faster GHz the memory can run... but do LOTS of research in this area (try ComputerPowerUser/CPU , http://www.computerpoweruser.com/ -- and remember they are being paid to place and review products, BUT it can give you a good idea of what is in the field... toms hardware is another good site). 2) faster hard disk(s)... RAID's using 1+0 if you have the dedication; RAID-0 if you keep good daily backups and don't mind losing a day if a disk goes bad, RAID5 if you have the HW for it (usually a dedicated RAID card will offload the RAID functions, leaving your CPU's for other stuff). 3) Better dedicated graphics card (free's cpu, faster response with effects -- will 'feel better', though throughput may not increase that much. 4) More Core's on your CPU... especially with more onboard cache. ---- Most likely culprit(s) -- increased disk fragmentation... get a 2nd disk and install / copy your current install onto it (not with 'DD', but with cp or tar or a backup/restore) -- that will lay out files contiguously on disk again (for a while)... or use a file system with a defragmenter (xfs has an 'ok' one ... should do better after 3.16.2 kernel as they are adding better freespace consolidators which should help pave the way for 'freespace' defragmentation... (i.e. All your files may look unfragmented, but if free space is fragmented, it becomes slower for the OS to look for space.). Start benchmarking things on your computer so you can tell what is good and what is not good -- and what helps and what doesn't. With no numbers you won't know what helps... (I should take some of this advice more seriously as well -- especially with broader benchmarks and metrics)... Anyway, probably wrote too much, .. but it's a very large topic... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Linda Walsh wrote on 2014-10-07 17:51 (UTC-0700):
Try " xlatencytop" to see what is really taking time in your system: LatencyTOP version 0.5 (C) 2008 Intel Corporation
Not in standard repos or in buildservice: http://software.opensuse.org/search?utf8=%E2%9C%93&q=xlatencytop&search_devel=false&search_devel=true&search_unsupported=false&baseproject=ALL Looks like maybe something several years ago orphaned: http://lists.opensuse.org/opensuse-commit/2010-04/msg00213.html :-( Is latencytop what you meant to type? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2014-10-08 03:24, Felix Miata wrote:
Is latencytop what you meant to type?
The package contains both: cer@Telcontar:~/tmp/date> rpm -ql latencytop | grep bin /usr/sbin/latencytop /usr/sbin/xlatencytop cer@Telcontar:~/tmp/date> -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Felix Miata wrote:
Linda Walsh wrote on 2014-10-07 17:51 (UTC-0700):
Try " xlatencytop" to see what is really taking time in your system: LatencyTOP version 0.5 (C) 2008 Intel Corporation
Is latencytop what you meant to type?
What Carlos said.... It's in 13.1 in the latencytop package. yeah, it's old, so is powertop -- but both fill a niche... There's also the "getdelays" package but I haven't found it as useful... but might be more suitable for monitoring an app over time. In 13.1 as delayacct-utils-3.11.1-23.1.2.x86_64 Description : Delay accounting allows the administrator to track the time an application spends waiting on disk I/O, swap I/O and CPU scheduling. This can help pin-point resource shortages in a system configuration. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Linda Walsh wrote on 2014-10-07 22:40 (UTC-0700):
Felix Miata wrote:
Linda Walsh wrote on 2014-10-07 17:51 (UTC-0700):
Try " xlatencytop" to see what is really taking time in your system: LatencyTOP version 0.5 (C) 2008 Intel Corporation
Is latencytop what you meant to type?
What Carlos said.... It's in 13.1 in the latencytop package. yeah, it's old, so is powertop -- but both fill a niche...
Earlier you wrote: [quote] Try " xlatencytop" to see what is really taking time in your system: LatencyTOP version 0.5 (C) 2008 Intel Corporation Cause Maximum Percentage ... fsync() on a file (type 'F' for details) 19.2 msec 18.0 % [/quote] How were you able to copy and paste that? (In the past few days, where I remember not, maybe IRC, maybe a Fedora or Mageia list if not an openSUSE list, I saw mention of Firefox getting sluggard at the same time as fsync calls from IIRC 2 plugins were causing dependent looping.) I get (11.4 on ICH8 SATA2 HDs RAID 1 EXT3; latencytop-0.5-10.1.i586) from SeaMonkey in xlatencytop: Cause Maximum Percentage ... fsync() on a file (type 'F' for details) 361.4 msec 14.1 % (highest) fsync() on a file (type 'F' for details) 180.5 msec 7.0 % (typical) fsync() on a file (type 'F' for details) 42.2 msec 3.2 % (lowest non-0) If I try to get that in copy and pasteable form from latencytop, it shows the wrong SeaMonkey session (I run 2). It seems only able to find the earliest started, whereas in xlatencytop I see and can access all Firefox and SeaMonkey sessions' data. What is it actually telling me? It says 'type "F" for details', but typing an "F" doesn't do anything I can notice. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Felix Miata wrote:
... fsync() on a file (type 'F' for details) 19.2 msec 18.0 % [/quote] How were you able to copy and paste that?
duplicated it on the ascii version. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Roger Luedecke wrote:
So, I've noticed how nice a new installation is. Then it gets progressively sluggish over time. Some of it is obvious such as when the programs menu has more programs in it. Other than that though, not so obvious. Do you know any techniques for cleaning up the system to regain performance closer to when it's first installed?
Uninstall <strike>windows</strike> systemd. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (10)
-
Banga Gong
-
Carlos E. R.
-
Felix Miata
-
jdd
-
jdebert
-
John Andersen
-
Linda Walsh
-
Peter
-
Roger Luedecke
-
Ruben Safir