[opensuse] dovecot full-text index not working?
Carlos E. R. wrote:
I have attempted a few times searches using my local dovecot imap server. I had to kill Th.
I'm not sure what you have done. My Thunderbird can search via IMAP pretty fine.
Last time I tried was long ago. Ok, so I try a search in my local dovecot server for messages containing "ldap" in the subject line, and I get results very fast. Then I repeat the same search but in the body... I'm still waiting. The process "imap" is at 100% cpu. Disk activity is nil.
I know this is a silly question, but do you have a full-text indexing engine installed? dovecot has the support for several engines, but afaik none is installed by default. -- Per Jessen, Zürich (23.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 14:48, Per Jessen wrote:
Carlos E. R. wrote:
Last time I tried was long ago. Ok, so I try a search in my local dovecot server for messages containing "ldap" in the subject line, and I get results very fast. Then I repeat the same search but in the body... I'm still waiting. The process "imap" is at 100% cpu. Disk activity is nil.
(still searching, after 50 minutes)
I know this is a silly question, but do you have a full-text indexing engine installed? dovecot has the support for several engines, but afaik none is installed by default.
I have no idea, but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way. Or does the _user_ have to know if the server has that capability? - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAedjoACgkQIvFNjefEBxp8pwCgqFNJ7tmx8gJtVrPnnDDZxvTv +0wAoJbYJ/haTiCWl4wcSMPP5fyxdCxq =gzyX -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-08-05 14:48, Per Jessen wrote:
Carlos E. R. wrote:
Last time I tried was long ago. Ok, so I try a search in my local dovecot server for messages containing "ldap" in the subject line, and I get results very fast. Then I repeat the same search but in the body... I'm still waiting. The process "imap" is at 100% cpu. Disk activity is nil.
(still searching, after 50 minutes)
I know this is a silly question, but do you have a full-text indexing engine installed? dovecot has the support for several engines, but afaik none is installed by default.
I have no idea,
Then you don't have a full-text index.
but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way.
Not having a full-text index doesn't mean there is no search capability, it just means it is a lot slower.
Or does the _user_ have to know if the server has that capability?
The server always has the search capability, but it may not have a full-text index. -- Per Jessen, Zürich (24.2°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:38, Per Jessen wrote:
Carlos E. R. wrote:
but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way.
Not having a full-text index doesn't mean there is no search capability, it just means it is a lot slower.
Mmm. After about one hour it has found 835 hits (still running). Even Thunderbird is faster finding text.
Or does the _user_ have to know if the server has that capability?
The server always has the search capability, but it may not have a full-text index.
Good grief! I would consider that a bug, it's a waste of cpu doing the search. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAeeKIACgkQIvFNjefEBxoX3wCgv04mnTy8CMXGJYkOpeJ2Wztn NTEAoNNdYLAAj5dqDFM8Pdt+Zl+oel/w =w3I9 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:44, Carlos E. R. wrote:
On 2012-08-05 15:38, Per Jessen wrote:
Carlos E. R. wrote:
The server always has the search capability, but it may not have a full-text index.
Good grief!
I would consider that a bug, it's a waste of cpu doing the search.
I closed the search window, "imap" process kept running. Stopped Th., same thing. Stopped dovecot, same. Killed "imap", refused to die. Had to kill 9 it. Indeed buggy! :-( - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAeeZYACgkQIvFNjefEBxoCZQCfT83Bhe9WuHpmiWa+YfZmuOF9 ohEAnRl3GD1cdvR9w/rJJmx3Xvi6OKsn =SBY7 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
On 2012-08-05 15:38, Per Jessen wrote:
Carlos E. R. wrote:
but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way.
Not having a full-text index doesn't mean there is no search capability, it just means it is a lot slower.
Mmm. After about one hour it has found 835 hits (still running). Even Thunderbird is faster finding text.
TB maintains it's own index afaik.
Or does the _user_ have to know if the server has that capability?
The server always has the search capability, but it may not have a full-text index.
Good grief!
I would consider that a bug, it's a waste of cpu doing the search.
It works as designed, it's up to you configure to suit your requirements. If you prefer fast searching and/or saving CPU cycles, you should configure the full-text indexing feature. Having or not having a full-text index is a trade-off - without it, you save disk-space and have to pay with more CPU cycles. With it, you save CPU cycles, but pay with disk-space. -- Per Jessen, Zürich (24.1°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:57, Per Jessen wrote:
Having or not having a full-text index is a trade-off - without it, you save disk-space and have to pay with more CPU cycles. With it, you save CPU cycles, but pay with disk-space.
The way it is you can kill a server remotely. No server should allow a process like that. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAefPoACgkQIvFNjefEBxqRgwCdHGk0sPBzxKdaRwBnAl1bI/NX LucAoMxY6MfKMS7DaB0GGQ52Olu0koMt =v0S2 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-08-05 15:57, Per Jessen wrote:
Having or not having a full-text index is a trade-off - without it, you save disk-space and have to pay with more CPU cycles. With it, you save CPU cycles, but pay with disk-space.
The way it is you can kill a server remotely. No server should allow a process like that.
Nor should the admin :-) Maybe your dovecot needs some tuning? My local dovecot doesn't have full-text index either, but a search for 'ldap' in 20000 emails took about 2m20s. (old mail-server, pentium III 1.3GHz, 784M RAM). -- Per Jessen, Zürich (24.4°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen wrote:
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Having or not having a full-text index is a trade-off - without it, you save disk-space and have to pay with more CPU cycles. With it, you save CPU cycles, but pay with disk-space. The way it is you can kill a server remotely. No server should allow a
On 2012-08-05 15:57, Per Jessen wrote: process like that. Nor should the admin :-)
Maybe your dovecot needs some tuning? My local dovecot doesn't have full-text index either, but a search for 'ldap' in 20000 emails took about 2m20s. (old mail-server, pentium III 1.3GHz, 784M RAM).
From Yast Software Management: dovecot20-fts-solr - Fulltext search support via solr Dovecot is an IMAP and POP3 server for Linux and UNIX-like systems, written primarily with security in mind. Although it is written in C, it uses several coding techniques to avoid most of the common pitfalls. Dovecot can work with standard mbox and maildir formats and is fully compatible with UW-IMAP and Courier IMAP servers as well as mail clients accessing the mailboxes directly. This package holds the files needed for fulltext search support via solr. There is also a version for Dovecot 1.2 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:57, Per Jessen wrote:
Carlos E. R. wrote:
Mmm. After about one hour it has found 835 hits (still running). Even Thunderbird is faster finding text.
TB maintains it's own index afaik.
which proves that the decision of TB devs to keep that index was correct. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAefYEACgkQIvFNjefEBxoVigCg2MKEnLmKbIRaikhZno9dI02i 6lgAoJaJkzCirsXrv7XiK1G4MGs+y1bn =27pJ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2012-08-05 15:57, Per Jessen wrote:
Carlos E. R. wrote:
Mmm. After about one hour it has found 835 hits (still running). Even Thunderbird is faster finding text.
TB maintains it's own index afaik.
which proves that the decision of TB devs to keep that index was correct.
I honestly don't see how that proves anything at all, but let's stop that discussion here, it won't lead anywhere. -- Per Jessen, Zürich (24.5°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:38, Per Jessen wrote:
Then you don't have a full-text index.
And how does one enable it? Grepping for "search" in the sample configuration files doesn't hit anything. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAefnQACgkQIvFNjefEBxq9mQCeNB7NJCQTc07rZUqK0kE/k1Ef 3skAnRhoKqaLjnyOtJYRnGOUlf70l+1n =ZieN -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
On 2012-08-05 15:38, Per Jessen wrote:
Then you don't have a full-text index.
And how does one enable it?
Grepping for "search" in the sample configuration files doesn't hit anything.
Start here: http://wiki2.dovecot.org/Plugins/FTS http://wiki2.dovecot.org/Plugins/FTS/Squat -- Per Jessen, Zürich (24.5°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am 05.08.2012 15:33, schrieb Carlos E. R.:
On 2012-08-05 14:48, Per Jessen wrote:
Carlos E. R. wrote:
Last time I tried was long ago. Ok, so I try a search in my local dovecot server for messages containing "ldap" in the subject line, and I get results very fast. Then I repeat the same search but in the body... I'm still waiting. The process "imap" is at 100% cpu. Disk activity is nil.
(still searching, after 50 minutes)
I know this is a silly question, but do you have a full-text indexing engine installed? dovecot has the support for several engines, but afaik none is installed by default.
I have no idea, but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way. Or does the _user_ have to know if the server has that capability?
No, the user does not have to know. You can search nevertheless but probably you get no answer in reasonable time. In IMAP protocol this is mainly transparent. It's not obvious to the client which search capabilities are available (there are some special search capas though). Wolfgang -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-08-05 15:39, Wolfgang Rosenauer wrote:
Am 05.08.2012 15:33, schrieb Carlos E. R.:
I have no idea, but I assume that the server would tell Thunderbird that there is no such capability and Th. would then not do the search that way. Or does the _user_ have to know if the server has that capability?
No, the user does not have to know. You can search nevertheless but probably you get no answer in reasonable time.
Dovecot should refuse to do the search, it is a waste of cpu. Can you imagine a production server? You can cause a successful DOS attack with just one search per CPU core of the server. I had to kill 9 the imap process! - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlAeeioACgkQIvFNjefEBxpopgCcCiGliA+M5KB9c9RVnMIdziAy 4yYAoIpdk01dvFf36KmzCrlisEyQoCxq =I1nD -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Carlos E. R. -
James Knott -
Per Jessen -
Wolfgang Rosenauer