Hi,
I looked in the last few months of the archive but couldn't see anything regarding this. My apologies if I missed it somehow.
We're having strange behaviour in the GUI where trying to delete systems is failing. We've had this problem with different admin accounts: organization admins, Uyuni admins, admin accounts that are AD-integrated through PAM, the initial local admin account. We've enabled Duo multi-factor authentication but also had this problem before adding MFA.
In all cases Uyuni fails to delete the system. After Confirming System Profile deletion, we first get a popup indicating Cleanup timed out. Please check if the machine is reachable. in the case of one system that had already been taken offline, but after indicating the profile should be deleted without cleanup (or without that first popup if we try to delete a system that is online) we get a follow-up error: Server error, please check log files..
Other operations work fine against systems: updating packages, rebooting systems, etc. We just can't delete systems.
If we switch to the Admin menu to go look at the tomcat log, we see a notice: System profile 1000010001 could not be deleted.
In the "tomcat" /var/log/rhn/rhn_web_ui.log log file, we see the errors:
2022-04-08 12:30:48,726 [ajp-nio-0:0:0:0:0:0:0:1-8009-exec-14] WARN com.suse.manager.webui.services.impl.SaltService - Got no result for state.apply on minion CARMD-EX-DWsbx4.sierrawireless.local (minion did not respond in time)
2022-04-08 12:30:50,542 [ajp-nio-0:0:0:0:0:0:0:1-8009-exec-9] ERROR com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand - XmlRpcFault while logging in. most likely user doesn't have permissions.
redstone.xmlrpc.XmlRpcFault: :'login failed (<myaccount>)'
at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:444)
at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:51)
at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:113)
at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:66)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:66)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerSystemRemoveCommand.<init>(CobblerSystemRemoveCommand.java:37)
at com.redhat.rhn.manager.system.SystemManager.deleteServer(SystemManager.java:725)
at com.redhat.rhn.manager.system.SystemManager.deleteServer(SystemManager.java:704)
at com.suse.manager.webui.controllers.SystemsController.delete(SystemsController.java:218)
at com.suse.manager.webui.utils.SparkApplicationHelper.lambda$withUser$1(SparkApplicationHelper.java:88)
at spark.RouteImpl$1.handle(RouteImpl.java:72)
at spark.http.matching.Routes.execute(Routes.java:61)
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:130)
at spark.servlet.SparkFilter.doFilter(SparkFilter.java:173)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:98)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:69)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:99)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:432)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1591)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-04-08 12:30:50,543 [ajp-nio-0:0:0:0:0:0:0:1-8009-exec-9] ERROR com.redhat.rhn.frontend.servlets.SessionFilter - Error during transaction. Rolling back
com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.
at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:56)
at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:113)
at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:66)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:66)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerSystemRemoveCommand.<init>(CobblerSystemRemoveCommand.java:37)
at com.redhat.rhn.manager.system.SystemManager.deleteServer(SystemManager.java:725)
at com.redhat.rhn.manager.system.SystemManager.deleteServer(SystemManager.java:704)
at com.suse.manager.webui.controllers.SystemsController.delete(SystemsController.java:218)
at com.suse.manager.webui.utils.SparkApplicationHelper.lambda$withUser$1(SparkApplicationHelper.java:88)
at spark.RouteImpl$1.handle(RouteImpl.java:72)
at spark.http.matching.Routes.execute(Routes.java:61)
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:130)
at spark.servlet.SparkFilter.doFilter(SparkFilter.java:173)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:98)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:69)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:99)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:55)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:432)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1591)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: redstone.xmlrpc.XmlRpcFault: :'login failed (<myaccount>)'
at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:444)
at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:51)
... 50 more
It's not clear to me why trying to remove a system would be causing cobbler exceptions. Any ideas or suggestions?
Thanks
Paul-Andre Panon
Paul-Andre Panon