[opensuse-buildservice] Spurious "401 unauthorized" errors in osc with 2.8.3 / LDAP backend
Hi, I'm a long time happy user of the OBS with LDAP auth, even though it was not officially supported. Since the update from 2.8.2 to 2.8.3 (which brought official LDAP support), my users and I am seeing spurious "401 Unauthorized" messages on osc commands. Usually, a retry of the command then succeeds. There is nothing suspicious at that time in production.log. This is the LDAP config on my box (slightly edited to protect the guilty) obs:/srv/www/obs/api # grep ^ldap config/options.yml ldap_mode: :on ldap_servers: ad0301.my.do.main ad0302.my.do.main ad0300.my.do.main ldap_max_attempts: 10 ldap_user_memberof_attr: memberof ldap_group_member_attr: member ldap_ssl: :off ldap_start_tls: :on ldap_port: 389 ldap_referrals: :on ldap_search_base: DC=my,DC=do,DC=main ldap_search_attr: SAMAccountName ldap_name_attr: displayName ldap_mail_attr: mail ldap_search_user: AD2LDAP@my.do.main ldap_search_auth: "V3rYS3Cr37P@ssw0rd" ldap_user_filter: "(memberof=cn=540d57e4fd84a07798000002,ou=DL,ou=MSX,ou=Resources,dc=my,dc=do,dc=main)" ldap_authenticate: :ldap ldap_auth_mech: :md5 ldap_auth_attr: userPassword ldap_update_support: :off ldap_object_class: inetOrgPerson ldap_entry_base: ou=OBSUSERS,dc=EXAMPLE,dc=COM ldap_sn_attr_required: :on ldap_group_support: :off ldap_group_search_base: ou=OBSGROUPS,dc=EXAMPLE,dc=COM ldap_group_title_attr: cn ldap_group_objectclass_attr: groupOfNames LDAP server is Microsoft Active Directory. My *guess* is, that the AD servers sometimes answer with some kind of "busy, please try again" or "busy, please wait" response and OBS treats this as "auth failed". Any hints on where to look (or where to put debug code? ;-) Best regards, Stefan -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Dienstag, 19. September 2017, 08:30:04 CEST wrote Stefan Seyfried:
Hi,
I'm a long time happy user of the OBS with LDAP auth, even though it was not officially supported. Since the update from 2.8.2 to 2.8.3 (which brought official LDAP support), my users and I am seeing spurious "401 Unauthorized" messages on osc commands.
Usually, a retry of the command then succeeds.
There is nothing suspicious at that time in production.log.
This is the LDAP config on my box (slightly edited to protect the guilty)
obs:/srv/www/obs/api # grep ^ldap config/options.yml ldap_mode: :on ldap_servers: ad0301.my.do.main ad0302.my.do.main ad0300.my.do.main ldap_max_attempts: 10 ldap_user_memberof_attr: memberof ldap_group_member_attr: member ldap_ssl: :off ldap_start_tls: :on ldap_port: 389 ldap_referrals: :on ldap_search_base: DC=my,DC=do,DC=main ldap_search_attr: SAMAccountName ldap_name_attr: displayName ldap_mail_attr: mail ldap_search_user: AD2LDAP@my.do.main ldap_search_auth: "V3rYS3Cr37P@ssw0rd" ldap_user_filter: "(memberof=cn=540d57e4fd84a07798000002,ou=DL,ou=MSX,ou=Resources,dc=my,dc=do,dc=main)" ldap_authenticate: :ldap ldap_auth_mech: :md5 ldap_auth_attr: userPassword ldap_update_support: :off ldap_object_class: inetOrgPerson ldap_entry_base: ou=OBSUSERS,dc=EXAMPLE,dc=COM ldap_sn_attr_required: :on ldap_group_support: :off ldap_group_search_base: ou=OBSGROUPS,dc=EXAMPLE,dc=COM ldap_group_title_attr: cn ldap_group_objectclass_attr: groupOfNames
LDAP server is Microsoft Active Directory.
My *guess* is, that the AD servers sometimes answer with some kind of "busy, please try again" or "busy, please wait" response and OBS treats this as "auth failed".
Any hints on where to look (or where to put debug code? ;-)
You have set # Authentication with Windows 2003 AD requires ldap_referrals: :on in config/options.yml ? -- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 19.09.2017 08:50, Adrian Schröter wrote:
On Dienstag, 19. September 2017, 08:30:04 CEST wrote Stefan Seyfried:
ldap_mode: :on ldap_servers: ad0301.my.do.main ad0302.my.do.main ad0300.my.do.main ldap_max_attempts: 10 ldap_user_memberof_attr: memberof ldap_group_member_attr: member ldap_ssl: :off ldap_start_tls: :on ldap_port: 389 ldap_referrals: :on
LDAP server is Microsoft Active Directory.
My *guess* is, that the AD servers sometimes answer with some kind of "busy, please try again" or "busy, please wait" response and OBS treats this as "auth failed".
Any hints on where to look (or where to put debug code? ;-)
You have set
# Authentication with Windows 2003 AD requires ldap_referrals: :on
in config/options.yml ?
yes. Also note it is a spurious failure, and usually we cannot really reproduce it. Hence my guess that a strange "try again" answer from AD is treated as "this failed" by the OBS code. Best regards, -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 19/09/17 07:30, Stefan Seyfried wrote:
Hi,
I'm a long time happy user of the OBS with LDAP auth, even though it was not officially supported. Since the update from 2.8.2 to 2.8.3 (which brought official LDAP support), my users and I am seeing spurious "401 Unauthorized" messages on osc commands.
Usually, a retry of the command then succeeds.
There is nothing suspicious at that time in production.log.
Hi Stefan, the code that attempts to authenticate a user via ldap occurs in this method: https://github.com/opensuse/open-build-service/blob/2.8/src/api/app/models/u... Although you say there is nothing suspicious in the logs, could you provide a copy of the logs around the time when one of these spurious 401s occur? It would still be really helpful for us to debug this.. Thanks, Evan. -- Evan Rolfe Full Stack Web Developer SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nürnberg Tel: +49-911-74053-0; Fax: +49-911-7417755; https://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 19.09.2017 12:08, Evan Rolfe wrote:
On 19/09/17 07:30, Stefan Seyfried wrote:
Hi,
I'm a long time happy user of the OBS with LDAP auth, even though it was not officially supported. Since the update from 2.8.2 to 2.8.3 (which brought official LDAP support), my users and I am seeing spurious "401 Unauthorized" messages on osc commands.
Usually, a retry of the command then succeeds.
There is nothing suspicious at that time in production.log.
Hi Stefan, the code that attempts to authenticate a user via ldap occurs in this method:
https://github.com/opensuse/open-build-service/blob/2.8/src/api/app/models/u...
Yes, I found it (and that you probably broek it with 44df33c0 ;-)
Although you say there is nothing suspicious in the logs, could you provide a copy of the logs around the time when one of these spurious 401s occur? It would still be really helpful for us to debug this..
The message basically is I, [2017-09-19T10:42:25.796267 #15652] INFO -- : [c6a0df55-e0e2-4d55-8bae-c36d15eab44d] [15652:6936.13] Search failed: error -1: Can't contact LDAP server I, [2017-09-19T11:04:18.648923 #15660] INFO -- : [9de903f8-2aca-401d-9c45-41587d2c094b] [15660:8248.87] Search failed: error -1: Can't contact LDAP server I, [2017-09-19T12:30:04.264456 #3787] INFO -- : [4984a994-ef51-468d-a66c-b829c3e9a23b] [3787:4483.55] Search failed: error -1: Can't contact LDAP server And reading the comments in https://github.com/opensuse/open-build-service/blob/2.8/src/api/app/models/u... there should be a retry in this case (and was, before 44df33c0 ;-) to catch this. I had seen similar things years ago with OBS 2.3, which were fixed by updating to 2.6 (or something like that, might be even 2.4 did already fix it). It actually looks the original fix was in 2.4, commit 920a731d96. Would be nice to get this back in 2.8.4 :-) Thanks, Stefan -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 19.09.2017 14:49, Stefan Seyfried wrote:
On 19.09.2017 12:08, Evan Rolfe wrote:
On 19/09/17 07:30, Stefan Seyfried wrote:
Hi,
I'm a long time happy user of the OBS with LDAP auth, even though it was not officially supported. Since the update from 2.8.2 to 2.8.3 (which brought official LDAP support), my users and I am seeing spurious "401 Unauthorized" messages on osc commands.
Usually, a retry of the command then succeeds.
There is nothing suspicious at that time in production.log.
Hi Stefan, the code that attempts to authenticate a user via ldap occurs in this method:
https://github.com/opensuse/open-build-service/blob/2.8/src/api/app/models/u...
Yes, I found it (and that you probably broek it with 44df33c0 ;-)
It wasn't you, it was Björn. please revert db44312746793b8dc46651af48de231f827d0cc6 (and backport to 2.8 ;) Björn: please read up on what "redo" does :-P I'm testing a locally patched version right now. Thanks, Stefan -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Is your patch working? I've open a bug report issue for this so we should have this fixed soon in 2.8.4 and master. https://github.com/openSUSE/open-build-service/issues/3861 On 19/09/17 14:10, Stefan Seyfried wrote:
It wasn't you, it was Björn.
please revert db44312746793b8dc46651af48de231f827d0cc6 (and backport to 2.8 ;) Björn: please read up on what "redo" does :-P
I'm testing a locally patched version right now.
Thanks,
Stefan
-- Evan Rolfe Full Stack Web Developer SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nürnberg Tel: +49-911-74053-0; Fax: +49-911-7417755; https://www.suse.com/ SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On 20.09.2017 09:54, Evan Rolfe wrote:
Is your patch working? I've open a bug report issue for this so we should have this fixed soon in 2.8.4 and master.
I got no complaints today ;-), it seems that it helps but I cannot yet say for sure.
I also created an issue: https://github.com/openSUSE/open-build-service/issues/3853 -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (3)
-
Adrian Schröter -
Evan Rolfe -
Stefan Seyfried