On Mon, 21 Mar 2011 15:02:22 +0100 Stefan Seyfried wrote:

* slabtop shows there is quite some amount of space in use: Active / Total Objects (% used) : 160024 / 14304077 (1.1%) Active / Total Slabs (% used) : 11111 / 778760 (1.4%) Active / Total Caches (% used) : 117 / 220 (53.2%) Active / Total Size (% used) : 104352.71K / 3073816.64K (3.4%) Minimum / Average / Maximum Object : 0.02K / 0.21K / 4096.00K

OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 13434482 0 0% 0.20K 707078 19 2828312K xfs_btree_cur 507270 0 0% 0.39K 50727 10 202908K xfs_efi_item 167297 8 0% 0.22K 9841 17 39364K xfs_buf_item 24752 21011 84% 0.03K 221 112 884K size-32

after unmounting all xfs, the situation is the same, after "rmmod xfs" it looks like this: Active / Total Objects (% used) : 164213 / 195172 (84.1%) Active / Total Slabs (% used) : 11174 / 11187 (99.9%) Active / Total Caches (% used) : 109 / 203 (53.7%) Active / Total Size (% used) : 105308.29K / 110456.62K (95.3%) Minimum / Average / Maximum Object : 0.02K / 0.57K / 4096.00K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 24304 20402 83% 0.03K 217 112 868K size-32 20736 20601 99% 0.08K 432 48 1728K sysfs_dir_cache 20034 19445 97% 0.18K 954 21 3816K vm_area_struct 18800 14065 74% 0.19K 940 20 3760K dentry 17169 10284 59% 0.06K 291 59 1164K size-64 Unfortunately, "unmount all xfs and rmmod xfs" is amost equivalent to rebooting the machine, so it is not really an option :-) -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org

  Hello,

On Mon 21-03-11 15:02:22, Stefan Seyfried wrote:
> I have seen the following repeatedly, from time to time during the last
> years, but only today I have probably found a way to trigger it:
> 
> * create some heavy FS workload on XFS, probably pushing the machine
>   heavily into swap
> * stop the workload (probably that "make -j" in a big C++ project is not
>   such a good idea after all...)
> * machine stays slow, as if it had no memory, starts swapping even though
>   there are apparently gigabytes of free memory
> * free shows there is lots of memory free:
>   susi:~ # free
>             total       used     free  shared buffers  cached
>   Mem:     3949728    3583132   366596      0     348  199180
>   -/+ buffers/cache:  3383604   566124
>   Swap:    2093052     317460  1775592
  Well, 366 MB free isn't that much given you have 317 MB in swap and 3.9
GB of memory.

> * swapoff often fails, even though there should be enough memory according
>   to "free"
> * slabtop shows there is quite some amount of space in use:
>   Active / Total Objects (% used)    : 160024 / 14304077 (1.1%)
>   Active / Total Slabs (% used)      : 11111 / 778760 (1.4%)
>   Active / Total Caches (% used)     : 117 / 220 (53.2%)
>   Active / Total Size (% used)       : 104352.71K / 3073816.64K (3.4%)
>   Minimum / Average / Maximum Object : 0.02K / 0.21K / 4096.00K
> 
>     OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
>   13434482    0   0%    0.20K 707078       19   2828312K xfs_btree_cur
>   507270      0   0%    0.39K  50727       10    202908K xfs_efi_item
>   167297      8   0%    0.22K   9841       17     39364K xfs_buf_item
>    24752  21011  84%    0.03K    221      112       884K size-32
  Hmm, so you have about 3 GB of memory in unused slabs. That's indeed a
bug. This reminds me of one swap-over-NFS bug which was causing a similar
effect for journal_handle slab. And after some digging - it was bug 554081
- I see that openSUSE 11.4 and master branches don't have the fix.  Nick
forgot to commit the fix to master branch and I didn't realize that
either. So I'd bet on this patch... I'm also going to push it to master
branch.

> * there is no way to get rid of this short of unloading the XFS kernel
>   module. Only unmounting the FS's does not help.
> 
> I guess this is not really a SUSE kernel bug. Where to report this? And
> has maybe someone else seen something similar?
> 
> Running FACTORY, 2.6.38-24-desktop (Kernel:HEAD), Thinkpad X200s with 4GB
> RAM, but I have seen similar things at least two years ago on a then
> current openSUSE server at home, also x86_64, also XFS.
> 
> Until today I had guessed it had something to do with sparse files
> (running lots of kvm guests from images stored on XFS seemed to trigger it
> sometimes), but today it was clearly triggered by "overload the machine
> heavily by unbounded make -j", no sparse files involved and no kvm guests
> running at all.

								Honza
-- 
Jan Kara 
SUSE Labs, CR
-- 
To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse-kernel+help@opensuse.org