Mailinglist Archive: opensuse-factory (807 mails)

< Previous Next >
[opensuse-factory] Re: "Retrieving Folders..." hang and a possible solution
  • From: Martin Steigerwald <martin@xxxxxxxxxxxx>
  • Date: Sun, 05 Feb 2017 11:27:06 +0100
  • Message-id: <5128306.sxne9jhjEu@merkaba>
Adding in kde-pim, so KDEPIM devs have a chance to notice this and give
feedback if they wish to.

Am Sonntag, 5. Februar 2017, 09:30:21 CET schrieb ianseeks:
Hi

I've posted this to 3 lists in case it helps someone and stop blaming kmail,
maybe there is a case for akonadi to identify a resource issue or a decent
default should be in the base install.

Stop blaming KMail for delays is often a good idea.

There is mainly two kinds of delays that are actually caused by KMail:

1) Display (not retrieval!) of the contents of large folders, especially when
threading view mode is enabled. Dan, but I also think other PIM developers
*heavily* optimized this. KMail is much, much, much here faster since about
KDEPIM 16.04.

2) Selecting all mails in one folder in order to move tham or do something
else with them. It already takes several seconds of KMail hogging one CPU core
when selecting all mails of a folder with about 15000 mails on this
Sandybridge ThinkPad here. I think when dragging those mails to a different
folder there is another delay. There are further delays then caused by Akonadi
which are much higher. This is way I do moving folders with lots of mails
differently meanwhile: Stop Akonadi, mv the folder, start Akonadi and let it
update the database. This is still not optimal but lots faster than during it
through KMail. (I also reported this one.)

To my experience all or almost all other delays are caused by either Akonadi
or in case of IMAP accounts slow access to IMAP server, which may partly also
be caused by Akonadi hammering the IMAP server during (probably needless)
synchronisation runs.

I originally reduced the size of my mail folders by deleting everything
prior to 2016 then deleted the akonadi database in the hope of fixing the
problem but no go. So i created a new user with a brand new mail database
and still the problem persisted.

Martin Steigerwald suggested i try this and it seems to be working for me,
i've not had the hanging "Retrieving Folders..." since i did it yesterday
afternoon.

I stopped akonadiserver and mysqld (if it doesn't do it by itself).
I increased the innodb_buffer_pool_size value from 80M to 128M in ~/.local/
share/akonadi/mysql.conf. (it was set to 80M which is 48 below the default -
not sure who set below the mysql default). This new value may not be
enough for your circumstances.

Start akonadi again.

Akonadi still crashes from time to time, especially on log out, but i think
its for other reasons now.

I brought this up quite a time ago already, but didn´t find any easily
measurable difference back then:

[Akonadi] [Bug 332626] MySQL tuning: adaption of MySQL tuning options for
larger accounts
https://bugs.kde.org/show_bug.cgi?id=332626

I still didn´t measure, but meanwhile I have the firm impression that it really
helps with larger accounts to raise that size. Even to 1 GiB.

So my recommendation would give it a good increase. I never systematically
tested different sizes… it may well be that for my usecase 512 MiB would or
even be enough.

My oppinion still is: It is a bug that an user would have to care for
approbiate database tuning. Akonadi serves desktop applications like KMail and
its unreasonable for to expect that every KMail user also is a database
administrator.

Just raising the setup by default does not serve those systems with little
amounts of RAM. I´d personally for the time going would go with a moderate
increase of the default, since from all I know about database performance, 80
MiB of InnoDB buffer pool size is ridiculously low.

Also this is and quite other bug reports are mentioning Retrieving folder
contents issue:

https://bugs.kde.org/show_bug.cgi?id=338571

However AFAIK there is a combination of several issues at place:

1) If Akonadi does a folder synchronisation, it doesn´t do anything else, i.e.
blocks out mail access:

[Akonadi] [Bug 367892] New: During folder synchronisation Akonadi blocks out
other operations like deleting or viewing mails

[kmail2] [Bug 331848] displaying, moving, deleting mails takes 10-20 seconds
when Akonadi synchronizes in background

Hmmm, I think these are duplicates. Will look and mark so.

I think this is a major issue: A background operation is not supposed to block
out interactive user requests *ever*. I think you know how frustrating it is
to wait for Akonadi to talk with KMail again.

2) IMHO it also needlessly does folder synchronisation:

[Akonadi] [Bug 334216] New: synchronizes folder with filesystem after
downloading and filtering mails needlessly

(note, that exactly with hogging one core for minutes usually is no more due
to other optimization in database access, and its folder synchronisation
causes less I/O and thus is faster due to high innodb buffer size I think)

This is the maildir case. I think I also reported it for IMAP. I just do not
find this bug report anymore, but it was very easy to reproduce: Have a fast
IMAP server like dovecot. Have a folder where you can safely delete from mails
from. Delete mails one by one using delete key. Watch KMail synchronizing the
folder after about 5-10 delete operations, consequently blocking out other
operations.

3) Too many and too complex database accesses.

Dan did some great work there, so by all means use at least KDEPIM + Akonadi
16.04. But its still doing queries it shouldn´t do according to what I read.

4) Non optimal database configuration for large accounts.


Of all of these issues the user and to some extent distros can directly only
influence 4. I.e. mainly raise InnoDB buffer pool size. So this is what I
recommend users to try. However: It cannot and will not solve all performance
issues. But in my experience it can give a much better experience.

Unfortunately performance in Akonadi is a quite complex topic, which requires
further serious work on it. Already for just diagnosing the actual bottle
necks. Database admins offered their help before and maybe Randa Meetings
would be a good place to meet up regarding this. I consider attending and
bring with me my insane Akonadi testcase (and in my case: its really insane,
with a working set of more than 940000 mails and at least another 1500000
mails I somehow tricked Akonadi into not synchronizing unless I click on these
archival folders.)

Thanks,
--
Martin
--
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
This Thread
References