[opensuse-programming] pcre match_limit
All, I'm using PCRE (the C-library) to extract base64 encoded attachments from emails. Needless to say, such attachments can get pretty big. I've been getting segfaults, so I have been playing with the match_limit (as far as I can tell, it's not a stacksize problem). However, according to the pcreapi man page, the default match_limit is 10million - where as I'm having to keep it at 15000 or thereabouts. I'm right now trying to find the optimal level. Can anyone explain why I would have to set it so low? /Per Jessen, Zürich --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-programming+help@opensuse.org
On Tue, 03 Jul 2007 08:53:43 +0200 Per Jessen <per@computer.org> wrote:
All,
I'm using PCRE (the C-library) to extract base64 encoded attachments from emails. Needless to say, such attachments can get pretty big. I've been getting segfaults, so I have been playing with the match_limit (as far as I can tell, it's not a stacksize problem). However, according to the pcreapi man page, the default match_limit is 10million - where as I'm having to keep it at 15000 or thereabouts. I'm right now trying to find the optimal level. Can anyone explain why I would have to set it so low?
Have you tried to use a debugger? There are many reasons that code will segfault. PCRE seems a bit of an overkill for simply extracting base-64. There are a number of other base-64 decoders available. -- Jerry Feldman <gaf@blu.org> Boston Linux and Unix user group http://www.blu.org PGP key id:C5061EA9 PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
Jerry Feldman wrote:
I'm using PCRE (the C-library) to extract base64 encoded attachments from emails. Needless to say, such attachments can get pretty big. I've been getting segfaults, so I have been playing with the match_limit (as far as I can tell, it's not a stacksize problem). However, according to the pcreapi man page, the default match_limit is 10million - where as I'm having to keep it at 15000 or thereabouts. I'm right now trying to find the optimal level. Can anyone explain why I would have to set it so low?
Have you tried to use a debugger? There are many reasons that code will segfault.
Hi Jerry, yeah, I've used the debugger and I've checked the core dumps, but this happens in the PCRE code, not in mine. And it's working fine right till I shift the match_limit up to 16000.
PCRE seems a bit of an overkill for simply extracting base-64. There are a number of other base-64 decoders available.
I'm not decoding, that bit is easy I agree. I'm using the regex to spot and extract a properly encoded and attached document. It's both fast and easy. /Per Jessen, Zürich --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-programming+help@opensuse.org
Per Jessen wrote:
yeah, I've used the debugger and I've checked the core dumps, but this happens in the PCRE code, not in mine. And it's working fine right till I shift the match_limit up to 16000.
I need to add - I've also tried using match_limit_recursion set to 10.000 which my 8M stack should be able to deal with, but it's match_limit that triggers first. /Per Jessen, Zürich --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-programming+help@opensuse.org
Per Jessen wrote:
I'm using PCRE (the C-library) to extract base64 encoded attachments from emails. Needless to say, such attachments can get pretty big. I've been getting segfaults, so I have been playing with the match_limit (as far as I can tell, it's not a stacksize problem). However, according to the pcreapi man page, the default match_limit is 10million - where as I'm having to keep it at 15000 or thereabouts. I'm right now trying to find the optimal level. Can anyone explain why I would have to set it so low?
I'm not 100% certain yet, but this looks like it might have been due to a back-level PCRE library. The production was 6.2, my development system 6.7, but I've now staticly linked in version 7.2 and the problem has yet to reappear. /Per Jessen, Zürich --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse-programming+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-programming+help@opensuse.org
participants (2)
-
Jerry Feldman -
Per Jessen