[opensuse-kde] Re: Issue with deletions
Dear Dave, On Dienstag, 9. Juni 2015 14:23:16 Dave McMurtrie wrote:
Apologies for the top-post (and for what is likely an html-formatted message). Unless you didn't include all of the protocol, the problem is that the client never sent an EXPUNGE command. It only set the deleted flag on the message, so the message still exists.
Thanks for that quick and detailed answer. This is exactly, what I was after. So I can further investigate, why akonadi doesn't expunge the MB. Will copy the kdepim people. Cheers, Pete
Here's some example protocol showing what I mean with a bunch of stuff snipped for brevity.
1 SELECT INBOX * 797 EXISTS 1 OK [READ-WRITE] Completed
2 UID STORE 95578 +FLAGS (\Deleted) * 784 FETCH (FLAGS (\Deleted NonJunk) UID 95578) 2 OK Completed
3 SELECT INBOX * OK [CLOSED] Ok * 797 EXISTS 3 OK [READ-WRITE] Completed
Notice that there are still 797 existing messages. Now if I send an expunge command:
4 EXPUNGE * 784 EXPUNGE 4 OK [HIGHESTMODSEQ 104051] Completed
5 SELECT INBOX * OK [CLOSED] Ok * 796 EXISTS 5 OK [READ-WRITE] Completed
hth,
Dave
________________________________________ From: info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu [info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu] on behalf of Hans-Peter Jansen [hpj@urpla.net] Sent: Tuesday, June 09, 2015 9:55 AM To: info-cyrus@lists.andrew.cmu.edu Subject: Issue with deletions
Hi,
I'm innvestigating a pretty nagging issue from kmail, and seek some advice from an cyrus imap expert, if this log is consistent.
Env: openSUSE 13.2/x86_64 on server and client, cyrus-imapd 2.4.17, kdepim 4.14.{5,6,8,9}
The failing operation is: moving a mail to trash.
The net effect is, that kmail removes the mail from the list, notices an inconsisteny, and refetches all mails from this folder.
C: Client S: Server #: Comment above command
# client copies the mail 32602 to the trash folder (german version) C: A000024 UID COPY 32602 "INBOX.M&APw-lleimer" # server successfully created 9933 in trash S: A000024 OK Completed [ COPYUID 1432637812 32602 9933 ] # client flags mail as deleted C: A000025 UID STORE 32602 +FLAGS (\Deleted) # server flagged it as deleted, seen was set before S: * 13712 FETCH ( FLAGS (\Deleted \Seen) UID 32602 ) S: A000025 OK Completed # client investigates folder settings C: A000026 GETANNOTATION "INBOX" "*" "value.shared" S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/duplicatedeliver ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/sharedseen ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/pop3newuidl ( value.shared true ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastpop ( value.shared ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastupdate ( value.shared 9-Jun-2015 14:57:56 +0200 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/size ( value.shared 2371801752 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/partition ( value.shared default ) S: A000026 OK Completed C: A000027 GETACL "INBOX" S: * ACL INBOX hp lrswipcda cyrus lrswipkxtecda S: A000027 OK Completed C: A000028 MYRIGHTS "INBOX" S: * MYRIGHTS INBOX lrswipkxtecda S: A000028 OK Completed C: A000029 GETQUOTAROOT "INBOX" S: * QUOTAROOT INBOX user.hp S: * QUOTA user.hp ( STORAGE 54982759 100000000 ) S: A000029 OK Completed # client investigates folder state C: A000030 SELECT "INBOX" (CONDSTORE) S: * OK Ok [ CLOSED ] S: * 13724 EXISTS S: * 0 RECENT S: * FLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK ) S: * OK Ok [ PERMANENTFLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK \* ) ] S: * OK Ok [ UNSEEN 13670 ] S: * OK Ok [ UIDVALIDITY 1430161627 ] S: * OK Ok [ UIDNEXT 32615 ] S: * OK Ok [ HIGHESTMODSEQ 1235 ] S: * OK Ok [ URLMECH INTERNAL ] S: A000030 OK Completed [ READ-WRITE ]
Server responded with 13724 mails in this folder. This is the problem, since akonadi expects 13723 mails only:
akonadi_imap_resource_1(25099) RetrieveItemsTask::onFinalSelectDone: Detected inconsistency in local cache, we're missing some messages. Server: 13724 Local: 13723
Resulting in a full refetch. Although missing in the log, be assured, that there were 13724 mails in that folder before this operation.
Is that result to be expected? Shouldn't it be 13723 on the server side, too? Could this be some kind of server side race?
Thanks in advance, Pete ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
-- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kde+owner@opensuse.org
On Tuesday 09 June 2015 17:48:57 Hans-Peter Jansen wrote:
Dear Dave,
On Dienstag, 9. Juni 2015 14:23:16 Dave McMurtrie wrote:
Apologies for the top-post (and for what is likely an html-formatted message). Unless you didn't include all of the protocol, the problem is that the client never sent an EXPUNGE command. It only set the deleted flag on the message, so the message still exists.
EXPUNGE is always run before messages are downloaded. If the message for some reason was not expunged (e.g. because of missing expunge rights), the resulting message should be greyed out in kmail and be read-only (which means you can't delete it again). Not that is necessarily what you want, but it's also not the symptom you're seeing, is it? ... I just noticed that you don't have an expunge command. The corresponding code is in kdepim-runtime/resources/imap/retrieveitemstask.cpp RetrieveItemsTask::startRetrievalTasks I guess if you either don't have the automatic expunge enabled or not sufficient rights, you'd end up with these "ghost" messages. But they should still be greyed out in kmail. Cheers, Christian
Thanks for that quick and detailed answer. This is exactly, what I was after. So I can further investigate, why akonadi doesn't expunge the MB.
Will copy the kdepim people.
Cheers, Pete
Here's some example protocol showing what I mean with a bunch of stuff snipped for brevity.
1 SELECT INBOX * 797 EXISTS 1 OK [READ-WRITE] Completed
2 UID STORE 95578 +FLAGS (\Deleted) * 784 FETCH (FLAGS (\Deleted NonJunk) UID 95578) 2 OK Completed
3 SELECT INBOX * OK [CLOSED] Ok * 797 EXISTS 3 OK [READ-WRITE] Completed
Notice that there are still 797 existing messages. Now if I send an expunge command:
4 EXPUNGE * 784 EXPUNGE 4 OK [HIGHESTMODSEQ 104051] Completed
5 SELECT INBOX * OK [CLOSED] Ok * 796 EXISTS 5 OK [READ-WRITE] Completed
hth,
Dave
________________________________________ From: info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu [info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu] on behalf of Hans-Peter Jansen [hpj@urpla.net] Sent: Tuesday, June 09, 2015 9:55 AM To: info-cyrus@lists.andrew.cmu.edu Subject: Issue with deletions
Hi,
I'm innvestigating a pretty nagging issue from kmail, and seek some advice from an cyrus imap expert, if this log is consistent.
Env: openSUSE 13.2/x86_64 on server and client, cyrus-imapd 2.4.17, kdepim 4.14.{5,6,8,9}
The failing operation is: moving a mail to trash.
The net effect is, that kmail removes the mail from the list, notices an inconsisteny, and refetches all mails from this folder.
C: Client S: Server #: Comment above command
# client copies the mail 32602 to the trash folder (german version) C: A000024 UID COPY 32602 "INBOX.M&APw-lleimer" # server successfully created 9933 in trash S: A000024 OK Completed [ COPYUID 1432637812 32602 9933 ] # client flags mail as deleted C: A000025 UID STORE 32602 +FLAGS (\Deleted) # server flagged it as deleted, seen was set before S: * 13712 FETCH ( FLAGS (\Deleted \Seen) UID 32602 ) S: A000025 OK Completed # client investigates folder settings C: A000026 GETANNOTATION "INBOX" "*" "value.shared" S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/duplicatedeliver ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/sharedseen ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/pop3newuidl ( value.shared true ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastpop ( value.shared ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastupdate ( value.shared 9-Jun-2015 14:57:56 +0200 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/size ( value.shared 2371801752 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/partition ( value.shared default ) S: A000026 OK Completed C: A000027 GETACL "INBOX" S: * ACL INBOX hp lrswipcda cyrus lrswipkxtecda S: A000027 OK Completed C: A000028 MYRIGHTS "INBOX" S: * MYRIGHTS INBOX lrswipkxtecda S: A000028 OK Completed C: A000029 GETQUOTAROOT "INBOX" S: * QUOTAROOT INBOX user.hp S: * QUOTA user.hp ( STORAGE 54982759 100000000 ) S: A000029 OK Completed # client investigates folder state C: A000030 SELECT "INBOX" (CONDSTORE) S: * OK Ok [ CLOSED ] S: * 13724 EXISTS S: * 0 RECENT S: * FLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK ) S: * OK Ok [ PERMANENTFLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK \* ) ] S: * OK Ok [ UNSEEN 13670 ] S: * OK Ok [ UIDVALIDITY 1430161627 ] S: * OK Ok [ UIDNEXT 32615 ] S: * OK Ok [ HIGHESTMODSEQ 1235 ] S: * OK Ok [ URLMECH INTERNAL ] S: A000030 OK Completed [ READ-WRITE ]
Server responded with 13724 mails in this folder. This is the problem, since akonadi expects 13723 mails only:
akonadi_imap_resource_1(25099) RetrieveItemsTask::onFinalSelectDone:
Detected inconsistency in local cache, we're missing some messages. Server: 13724 Local: 13723
Resulting in a full refetch. Although missing in the log, be assured, that there were 13724 mails in that folder before this operation.
Is that result to be expected? Shouldn't it be 13723 on the server side, too? Could this be some kind of server side race?
Thanks in advance, Pete ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
_______________________________________________ KDE PIM mailing list kde-pim@kde.org https://mail.kde.org/mailman/listinfo/kde-pim KDE PIM home page at http://pim.kde.org/
-- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kde+owner@opensuse.org
On Dienstag, 9. Juni 2015 18:42:32 Christian Mollekopf wrote:
On Tuesday 09 June 2015 17:48:57 Hans-Peter Jansen wrote:
Dear Dave,
On Dienstag, 9. Juni 2015 14:23:16 Dave McMurtrie wrote:
Apologies for the top-post (and for what is likely an html-formatted message). Unless you didn't include all of the protocol, the problem is that the client never sent an EXPUNGE command. It only set the deleted flag on the message, so the message still exists.
EXPUNGE is always run before messages are downloaded. If the message for some reason was not expunged (e.g. because of missing expunge rights), the resulting message should be greyed out in kmail and be read-only (which means you can't delete it again). Not that is necessarily what you want, but it's also not the symptom you're seeing, is it?
Excuse me, Christian, but this is exactly the behaviour, I'm seeing. Sorry guys, but in the end this boils down to a PEBCAK. For some (performance related) reasons, I disabled this option (translated from german): Compress folder automatically (removes messages flagged as deleted) Needless to say, that the resulting behaviour is looking like a bug (as there is no warning similar to): "Be aware, that, if you disable this option, you cannot move or delete any mails successfully anymore, and if you attempt to do so, your mails are COPIED to trash/other folder AND left in place. While at it, expect more silly behaviour from akonadi, since it is going to refetch all messages of the affected source folder immediately. IOW: if you disable this option, you're going to suffer!". Beg my pardon for traces of sarcasm, please. Seriously, this option should really trigger a warning dialog when disabled EVERY time before the IMAP account settings dialog is closed. What do you think? Will update the bug report accordingly. Cheers, Pete
... I just noticed that you don't have an expunge command. The corresponding code is in
kdepim-runtime/resources/imap/retrieveitemstask.cpp RetrieveItemsTask::startRetrievalTasks
I guess if you either don't have the automatic expunge enabled or not sufficient rights, you'd end up with these "ghost" messages. But they should still be greyed out in kmail.
Cheers, Christian
Thanks for that quick and detailed answer. This is exactly, what I was after. So I can further investigate, why akonadi doesn't expunge the MB.
Will copy the kdepim people.
Cheers, Pete
Here's some example protocol showing what I mean with a bunch of stuff snipped for brevity.
1 SELECT INBOX * 797 EXISTS 1 OK [READ-WRITE] Completed
2 UID STORE 95578 +FLAGS (\Deleted) * 784 FETCH (FLAGS (\Deleted NonJunk) UID 95578) 2 OK Completed
3 SELECT INBOX * OK [CLOSED] Ok * 797 EXISTS 3 OK [READ-WRITE] Completed
Notice that there are still 797 existing messages. Now if I send an expunge command:
4 EXPUNGE * 784 EXPUNGE 4 OK [HIGHESTMODSEQ 104051] Completed
5 SELECT INBOX * OK [CLOSED] Ok * 796 EXISTS 5 OK [READ-WRITE] Completed
hth,
Dave
________________________________________ From: info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu [info-cyrus-bounces+dave64=andrew.cmu.edu@lists.andrew.cmu.edu] on behalf of Hans-Peter Jansen [hpj@urpla.net] Sent: Tuesday, June 09, 2015 9:55 AM To: info-cyrus@lists.andrew.cmu.edu Subject: Issue with deletions
Hi,
I'm innvestigating a pretty nagging issue from kmail, and seek some advice from an cyrus imap expert, if this log is consistent.
Env: openSUSE 13.2/x86_64 on server and client, cyrus-imapd 2.4.17, kdepim 4.14.{5,6,8,9}
The failing operation is: moving a mail to trash.
The net effect is, that kmail removes the mail from the list, notices an inconsisteny, and refetches all mails from this folder.
C: Client S: Server #: Comment above command
# client copies the mail 32602 to the trash folder (german version) C: A000024 UID COPY 32602 "INBOX.M&APw-lleimer" # server successfully created 9933 in trash S: A000024 OK Completed [ COPYUID 1432637812 32602 9933 ] # client flags mail as deleted C: A000025 UID STORE 32602 +FLAGS (\Deleted) # server flagged it as deleted, seen was set before S: * 13712 FETCH ( FLAGS (\Deleted \Seen) UID 32602 ) S: A000025 OK Completed # client investigates folder settings C: A000026 GETANNOTATION "INBOX" "*" "value.shared" S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/duplicatedeliver ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/sharedseen ( value.shared false ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/pop3newuidl ( value.shared true ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastpop ( value.shared ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/lastupdate ( value.shared 9-Jun-2015 14:57:56 +0200 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/size ( value.shared 2371801752 ) S: * ANNOTATION INBOX /vendor/cmu/cyrus-imapd/partition ( value.shared default ) S: A000026 OK Completed C: A000027 GETACL "INBOX" S: * ACL INBOX hp lrswipcda cyrus lrswipkxtecda S: A000027 OK Completed C: A000028 MYRIGHTS "INBOX" S: * MYRIGHTS INBOX lrswipkxtecda S: A000028 OK Completed C: A000029 GETQUOTAROOT "INBOX" S: * QUOTAROOT INBOX user.hp S: * QUOTA user.hp ( STORAGE 54982759 100000000 ) S: A000029 OK Completed # client investigates folder state C: A000030 SELECT "INBOX" (CONDSTORE) S: * OK Ok [ CLOSED ] S: * 13724 EXISTS S: * 0 RECENT S: * FLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK ) S: * OK Ok [ PERMANENTFLAGS ( \Answered \Flagged \Draft \Deleted \Seen $TODO $NOTJUNK $JUNK \* ) ] S: * OK Ok [ UNSEEN 13670 ] S: * OK Ok [ UIDVALIDITY 1430161627 ] S: * OK Ok [ UIDNEXT 32615 ] S: * OK Ok [ HIGHESTMODSEQ 1235 ] S: * OK Ok [ URLMECH INTERNAL ] S: A000030 OK Completed [ READ-WRITE ]
Server responded with 13724 mails in this folder. This is the problem, since akonadi expects 13723 mails only:
akonadi_imap_resource_1(25099) RetrieveItemsTask::onFinalSelectDone:
Detected inconsistency in local cache, we're missing some messages.
Server:
13724 Local: 13723
Resulting in a full refetch. Although missing in the log, be assured, that there were 13724 mails in that folder before this operation.
Is that result to be expected? Shouldn't it be 13723 on the server side, too? Could this be some kind of server side race?
Thanks in advance, Pete ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
_______________________________________________ KDE PIM mailing list kde-pim@kde.org https://mail.kde.org/mailman/listinfo/kde-pim KDE PIM home page at http://pim.kde.org/
-- To unsubscribe, e-mail: opensuse-kde+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kde+owner@opensuse.org
participants (2)
-
Christian Mollekopf -
Hans-Peter Jansen