[Bug 237770] New: beagled is doing too much work
https://bugzilla.novell.com/show_bug.cgi?id=237770 Summary: beagled is doing too much work Product: openSUSE 10.2 Version: Final Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: GNOME AssignedTo: bnc-team-gnome@forge.provo.novell.com ReportedBy: Ulrich.Windl@rz.uni-regensburg.de QAContact: qa@suse.de Since update from 10.1 to 10.2, I felt the system is less responsive. Also when I was doing nothing, the harddisk made a sound like continuously self-testing. I had also suspected some trojan, but it turned out that beagled is the top CPU process (regarding cumulative CPU): PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4097 user 21 0 97924 18m 4708 S 7.0 4.8 211:18.46 beagled I agree that I have a lot of files in my homedirectory, but I always had them. It's my computer after all. Other symptoms found while examining the logs: 2007-01-23-12-57-23-IndexHelper: 070123 1257283649 25787 IndexH WARN: Unable to set IO priority for process to idle 2007-01-19-13-31-40-Beagle: 070123 1603259703 04097 Beagle WARN: KMail backend: /home/user/Mail contains a maildir directory but no corresponding index file. Probably not a KMail mail directory. Ignoring this location! (this message repeating over and over; I never used KMail on this machine) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #1 from Ulrich.Windl@rz.uni-regensburg.de 2007-01-26 07:08 MST ------- As I'm going to reboot soon, here's the current CPU usage of beagled: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4097 user 20 0 97988 14m 3088 S 5.3 3.8 1822:02 beagled System uptime is a little over 7 days now. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 jpr@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|bnc-team- |joeshaw@novell.com |gnome@forge.provo.novell.com| -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #2 from Ulrich.Windl@rz.uni-regensburg.de 2007-01-30 00:39 MST ------- It seems that after a reboot beagled has calmed down: Uptime is 1 day 40 minutes, and beagled consumed 2 minutes CPU so far. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 joeshaw@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Info Provider| |Ulrich.Windl@rz.uni-regensburg.de ------- Comment #3 from joeshaw@novell.com 2007-01-30 10:14 MST ------- (In reply to comment #0)
Other symptoms found while examining the logs: 2007-01-23-12-57-23-IndexHelper: 070123 1257283649 25787 IndexH WARN: Unable to set IO priority for process to idle
This is typical. Newer kernels don't allow unprivileged processes to set their IO priority to idle.
2007-01-19-13-31-40-Beagle: 070123 1603259703 04097 Beagle WARN: KMail backend: /home/user/Mail contains a maildir directory but no corresponding index file. Probably not a KMail mail directory. Ignoring this location! (this message repeating over and over; I never used KMail on this machine)
Roughly how many times is this repeated? Does it go on forever, or just many times and then stop? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 Ulrich.Windl@rz.uni-regensburg.de changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|Ulrich.Windl@rz.uni- | |regensburg.de | ------- Comment #4 from Ulrich.Windl@rz.uni-regensburg.de 2007-01-31 00:27 MST ------- The message repeated 1037 times between 2007-01-29 and 2007-01-31, and the file "2007-01-29-08-00-51-Beagle" is the logest one within the Log directory. The file ist still updated with those messages. However the CPU usage of beagled is back to normal at the moment. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 mfreitas@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mfreitas@gmail.com ------- Comment #5 from mfreitas@gmail.com 2007-02-10 10:28 MST ------- I can confirm this bug: beagled runs forever, eating all cpu (actually just 1 cpu) whenever desktop is locked. This computer is turned on about all the time, i lock screen when i leave the university and i unlock it on the next day. so whatever it is doing i'm sure it should have completed it already by now. currently (saturday - i'm accessing by ssh and X is idle there) i have this: top - 15:17:18 up 1 day, 5:27, 29 users, load average: 1.27, 1.21, 1.18 Tasks: 143 total, 5 running, 135 sleeping, 1 stopped, 2 zombie Cpu0 : 54.8%us, 15.3%sy, 0.0%ni, 2.0%id, 26.6%wa, 0.3%hi, 1.0%si, 0.0%st Cpu1 : 3.3%us, 0.7%sy, 0.0%ni, 95.0%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 2060508k total, 2043868k used, 16640k free, 691892k buffers Swap: 4241152k total, 272k used, 4240880k free, 175120k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4380 miguel 15 0 1057m 163m 8488 S 70 8.1 812:12.44 beagled 4292 miguel 15 0 264m 141m 21m S 3 7.0 39:20.48 firefox-bin 28458 miguel 15 0 8744 1276 900 R 0 0.1 0:00.04 top 1 root 18 0 804 304 244 S 0 0.0 0:00.87 init I have nothing suspicious at ~/.beagle/Log. All files have the "WARN: Unable to set IO priority for process to idle" thing so i guess the KMail stuff above could be ruled out. Note beagled uses 1GB virtual, is this considered normal behaviour? I'm aware Joe has new rpms at http://repos.opensuse.org/home:/joeshaw/openSUSE_10.2. However i would expect some advice if i should update it right now or try to obtain additional information. Since we have a couple of suse computers like this one, i have interest in having this fixed through normal update channels as well... -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #6 from mfreitas@gmail.com 2007-02-11 11:12 MST ------- one day later, beagled is still running and eating cpu cycles. 4380 miguel 34 19 1057m 163m 8488 S 59 8.1 1678:34 beagled just occurred me to check what files beagled is accessing, which might provide some hint. i will create it as an attachment. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #7 from mfreitas@gmail.com 2007-02-11 11:13 MST ------- Created an attachment (id=118515) --> (https://bugzilla.novell.com/attachment.cgi?id=118515&action=view) lsof | grep beagled -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 mfreitas@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #118515|application/octet-stream |text/plain mime type| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #8 from joeshaw@novell.com 2007-02-12 09:33 MST ------- Total CPU usage on screensaver is the intended behavior (assuming you're not on laptop battery). The idea is that since you're away from your computer, Beagle can index your data as quickly as possible without interfering with your usage of the machine. We should probably make that an option, but that should be filed as a separate enhancement request bug. If the CPU usage extends beyond returning to the machine and turning the screensaver off, that would be a bug. Since a number of these have been fixed in the newer versions, I'd ask that you update first and see if you still see the problem. There have also been substantial fixes to memory usage so the 163 megs RSS should decrease quite a bit (and presumably virtual as well, although that's a harder number to analyze). The default log level as shipped is low and won't give you any useful information about CPU usage or what it's indexing. You can either send the beagled process SIGUSR1 to bump it up, or restart beagled with the --debug option (beagled --replace --debug). That will hopefully help things. Newer versions also allow you to send SIGUSR2 to the beagled-helper process to report what is presently being indexed and how long it's been running. That would only be useful if it were the beagled-helper process that was pegging the CPU, not beagled. The only way to get useful information from the log files would be to run beagled with the --debug option. You can send SIGUSR1 to the running beagled -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #9 from mfreitas@gmail.com 2007-02-12 13:15 MST ------- 1) this is a desktop computer (no battery) 2) total CPU usage on screensaver, no CPU usage with screensaver off - so no bug here the problem is about beagled never finishing his work... i will update to the newer version and try again. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #10 from joeshaw@novell.com 2007-02-12 13:38 MST ------- (In reply to comment #9)
the problem is about beagled never finishing his work...
Ah, ok. It would be good to bump up the debug level and monitor what is going on in the logs. There were issues recently with Beagle continuously recrawling one directory. Those should be fixed with the update. Another thing would be to check to see how many directories you have under your home directory, and if this number is greater than /proc/sys/fs/inotify/max_user_watches. If so, Beagle can't watch all of your files and has to recrawl those directories for changes periodically. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #11 from mfreitas@gmail.com 2007-02-14 08:51 MST ------- (In reply to comment #10)
Another thing would be to check to see how many directories you have under your home directory, and if this number is greater than /proc/sys/fs/inotify/max_user_watches.
this is definitely the case: find -type d | wc -l 50941 cat /proc/sys/fs/inotify/max_user_watches 8192 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #12 from joeshaw@novell.com 2007-02-14 09:22 MST ------- Ok, so that's why Beagle never stops crawling. It shouldn't peg the CPU though. I'm still interested in seeing your logs. Maybe Beagle should ship an init script to bump the max_user_watches value up higher. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #13 from Ulrich.Windl@rz.uni-regensburg.de 2007-02-15 00:48 MST ------- At the moment I have 1742 directories with 8192 max watches, and beagled is "tame" (uses little CPU). Could it be a problem for the first boot or log-in only? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 james@usr-local-bin.org changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |james@usr-local-bin.org ------- Comment #14 from james@usr-local-bin.org 2007-02-22 08:56 MST ------- (In reply to comment #13)
At the moment I have 1742 directories with 8192 max watches, and beagled is "tame" (uses little CPU). Could it be a problem for the first boot or log-in only?
Don't think it's just a first-time log-in issue as I can kill beagled-helper and then at some point later it is respawned and consuming my CPU like Mr Creosote. The top of top on my system: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10956 ogley 37 12 52200 21m 7616 S 78.1 4.3 334:11.19 beagled-helper Joe, what information do you need, I'll provide what I can. (Info pertinent to my system that's already been provided for other systems follows) ogley@riggwelter:~> rpm -q beagle beagle-0.2.16-4 (current Factory version) ogley@riggwelter:~> find -type d | wc -l 10658 ogley@riggwelter:~> cat /proc/sys/fs/inotify/max_user_watches 8192 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 joeshaw@novell.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Info Provider| |james@usr-local-bin.org ------- Comment #15 from joeshaw@novell.com 2007-02-22 09:31 MST ------- James: You're probably seeing a different issue. But in any case, send the beagled-helper process SIGUSR2 and it should print out in the log file what file it's been working on. If it's been a while, it's probably a filter bug. I think 0.2.16 has a problem with SVG files; there's a 0.2.16.1 release I did for SLED... I'm going to put together a 0.2.16.2 to fix the /tmp filling issue in another bug, and I'll submit that for both SLED and 10.3. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 james@usr-local-bin.org changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|james@usr-local-bin.org | ------- Comment #16 from james@usr-local-bin.org 2007-02-22 09:55 MST ------- confirmed, it seems to be the SVG issue that I'm seeing. 20070222 16:38:15.2064 15113 IndexH WARN: Filtering status (1m29s ago): determining filter for file:///home/install/gnome-themes-extras/gnome-themes-extras.orig/Wasp/icons/scalable/emblems/emblem-mail.svg -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #17 from joeshaw@novell.com 2007-02-22 12:04 MST ------- James: I just checked in 0.2.16.2 to SLED and STABLE, and checked it into the Beagle OBS project as well. So what's the status of this bug, are the issues fixed, are people still having problems? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770 ------- Comment #18 from james@usr-local-bin.org 2007-02-22 15:28 MST ------- A quick spec file change and the OBS version build on Factory and I can confirm that the SVG filter issue seems fixed now. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug, or are watching someone who is.
https://bugzilla.novell.com/show_bug.cgi?id=237770#c19
--- Comment #19 from Ulrich Windl
https://bugzilla.novell.com/show_bug.cgi?id=237770#c20
--- Comment #20 from Joe Shaw
https://bugzilla.novell.com/show_bug.cgi?id=237770#c22
Mark Gordon
https://bugzilla.novell.com/show_bug.cgi?id=237770#c23
Ulrich Windl
https://bugzilla.novell.com/show_bug.cgi?id=237770
User captain.magnus@opensuse.org added comment
https://bugzilla.novell.com/show_bug.cgi?id=237770#c25
Magnus Boman
participants (1)
-
bugzilla_noreply@novell.com