[uyuni-users] Bootstrapped salt minions never appear as systems
Hi, so this was working initially, but something has broke. I am running Uyuni 4.0.1 stable on openSUSE 42.3 host. A new openSUSE Leap 15.0 minion is bootstrapped using bootstrap.sh which was generated using mgr-bootstrap. It joins Salt master and the key shows using command: salt-key -L But the minion never appears in Systems > Systems List Viewing the Salt message bus using salt-run state.event pretty=True I see this: salt/job/20190411153502494691/new { "_stamp": "2019-04-11T14:35:02.495370", "arg": [], "fun": "mgractionchains.resume", "jid": "20190411153502494691", "minions": [ "minion.domain.co.uk" ], "missing": [], "tgt": "minion.domain.co.uk", "tgt_type": "glob", "user": "salt" } salt/job/20190411153502494691/ret/minion.domain.co.uk { "_stamp": "2019-04-11T14:35:06.850239", "cmd": "_return", "fun": "mgractionchains.resume", "fun_args": [], "id": "minion.domain.co.uk", "jid": "20190411153502494691", "metadata": { "suma-action-chain": true }, "out": "nested", "retcode": 254, "return": "'mgractionchains.resume' is not available.", "success": false }
From the minion if I do a salt-call state.apply I get this error:
minion.domain.co.uk:~ # salt-call state.apply [WARNING ] /var/cache/salt/minion/extmods/grains/__pycache__/cpuinfo.cpython- 36.py:20: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon. [ERROR ] Error encountered during module reload. Modules were not reloaded. [WARNING ] /var/cache/salt/minion/extmods/modules/udevdb.py:24: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon. [ERROR ] Template was specified incorrectly: False [ERROR ] Template was specified incorrectly: False local: Data failed to compile: ---------- Specified SLS packages.packages_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver ---------- Specified SLS custom.custom_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver Not sure where else to look for errors? -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
Hi David, I also have an openSuSE 15 minion and it joined properly. Here is what I did . 1. Checked what is the expected channel name so the bootstrap repo will be successfully created. -> /usr/share/susemgr/mgr_bootstrap_data.py 2. Created the channel with the same name and added several repos Mines are: oss/nonoss base oss/nonoss updates SLE15-uyuni-client-tools 3. Created the bootstrap repo mgr-create-bootstrap-repo 4. Created bootstrap script via web and edited it manually via cli to add the sle15 key 5. Bootstraped via web filling everything necessary. For bootstraping I have used the ip of the machine , but most probably it will work via hostname. Best Regards, Strahil Nikolov В петък, 12 април 2019 г., 6:37:09 ч. Гринуич-4, David Mace <david@macemail.co.uk> написа: Hi, so this was working initially, but something has broke. I am running Uyuni 4.0.1 stable on openSUSE 42.3 host. A new openSUSE Leap 15.0 minion is bootstrapped using bootstrap.sh which was generated using mgr-bootstrap. It joins Salt master and the key shows using command: salt-key -L But the minion never appears in Systems > Systems List Viewing the Salt message bus using salt-run state.event pretty=True I see this: salt/job/20190411153502494691/new { "_stamp": "2019-04-11T14:35:02.495370", "arg": [], "fun": "mgractionchains.resume", "jid": "20190411153502494691", "minions": [ "minion.domain.co.uk" ], "missing": [], "tgt": "minion.domain.co.uk", "tgt_type": "glob", "user": "salt" } salt/job/20190411153502494691/ret/minion.domain.co.uk { "_stamp": "2019-04-11T14:35:06.850239", "cmd": "_return", "fun": "mgractionchains.resume", "fun_args": [], "id": "minion.domain.co.uk", "jid": "20190411153502494691", "metadata": { "suma-action-chain": true }, "out": "nested", "retcode": 254, "return": "'mgractionchains.resume' is not available.", "success": false }
From the minion if I do a salt-call state.apply I get this error:
minion.domain.co.uk:~ # salt-call state.apply [WARNING ] /var/cache/salt/minion/extmods/grains/__pycache__/cpuinfo.cpython- 36.py:20: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon. [ERROR ] Error encountered during module reload. Modules were not reloaded. [WARNING ] /var/cache/salt/minion/extmods/modules/udevdb.py:24: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon. [ERROR ] Template was specified incorrectly: False [ERROR ] Template was specified incorrectly: False local: Data failed to compile: ---------- Specified SLS packages.packages_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver ---------- Specified SLS custom.custom_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver Not sure where else to look for errors? -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
Keep in mind that when you onboard a minion using the bootstrap script, you need to acept the key. So the first obvious question (most probably you will answer "yes") is: Is the key marked as accepted? If not, you can accept it from the WebUI. On viernes, 12 de abril de 2019 12:37:02 (CEST) David Mace wrote:
Hi, so this was working initially, but something has broke.
I am running Uyuni 4.0.1 stable on openSUSE 42.3 host.
A new openSUSE Leap 15.0 minion is bootstrapped using bootstrap.sh which was generated using mgr-bootstrap.
It joins Salt master and the key shows using command:
salt-key -L
But the minion never appears in Systems > Systems List
Viewing the Salt message bus using
salt-run state.event pretty=True
I see this:
salt/job/20190411153502494691/new { "_stamp": "2019-04-11T14:35:02.495370", "arg": [], "fun": "mgractionchains.resume", "jid": "20190411153502494691", "minions": [ "minion.domain.co.uk" ], "missing": [], "tgt": "minion.domain.co.uk", "tgt_type": "glob", "user": "salt" }
salt/job/20190411153502494691/ret/minion.domain.co.uk { "_stamp": "2019-04-11T14:35:06.850239", "cmd": "_return", "fun": "mgractionchains.resume", "fun_args": [], "id": "minion.domain.co.uk", "jid": "20190411153502494691", "metadata": { "suma-action-chain": true }, "out": "nested", "retcode": 254, "return": "'mgractionchains.resume' is not available.", "success": false }
From the minion if I do a salt-call state.apply I get this error:
minion.domain.co.uk:~ # salt-call state.apply [WARNING ] /var/cache/salt/minion/extmods/grains/__pycache__/cpuinfo.cpython- 36.py:20: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Error encountered during module reload. Modules were not reloaded. [WARNING ] /var/cache/salt/minion/extmods/modules/udevdb.py:24: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Template was specified incorrectly: False [ERROR ] Template was specified incorrectly: False local: Data failed to compile: ---------- Specified SLS packages.packages_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver ---------- Specified SLS custom.custom_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver
Not sure where else to look for errors?
-- Julio González Gil Release Engineer, SUSE Manager and Uyuni jgonzalez@suse.com
Hi Julio, David already mentioned that it is there. I have seen such behaviour on our SUMA 3.2 and sometimes it takes up to 5 min to get into the system list. Best Regards, Strahil Nikolov В петък, 12 април 2019 г., 7:54:46 ч. Гринуич-4, Julio González Gil <jgonzalez@suse.de> написа: Keep in mind that when you onboard a minion using the bootstrap script, you need to acept the key. So the first obvious question (most probably you will answer "yes") is: Is the key marked as accepted? If not, you can accept it from the WebUI. On viernes, 12 de abril de 2019 12:37:02 (CEST) David Mace wrote:
Hi, so this was working initially, but something has broke.
I am running Uyuni 4.0.1 stable on openSUSE 42.3 host.
A new openSUSE Leap 15.0 minion is bootstrapped using bootstrap.sh which was generated using mgr-bootstrap.
It joins Salt master and the key shows using command:
salt-key -L
But the minion never appears in Systems > Systems List
Viewing the Salt message bus using
salt-run state.event pretty=True
I see this:
salt/job/20190411153502494691/new { "_stamp": "2019-04-11T14:35:02.495370", "arg": [], "fun": "mgractionchains.resume", "jid": "20190411153502494691", "minions": [ "minion.domain.co.uk" ], "missing": [], "tgt": "minion.domain.co.uk", "tgt_type": "glob", "user": "salt" }
salt/job/20190411153502494691/ret/minion.domain.co.uk { "_stamp": "2019-04-11T14:35:06.850239", "cmd": "_return", "fun": "mgractionchains.resume", "fun_args": [], "id": "minion.domain.co.uk", "jid": "20190411153502494691", "metadata": { "suma-action-chain": true }, "out": "nested", "retcode": 254, "return": "'mgractionchains.resume' is not available.", "success": false }
From the minion if I do a salt-call state.apply I get this error:
minion.domain.co.uk:~ # salt-call state.apply [WARNING ] /var/cache/salt/minion/extmods/grains/__pycache__/cpuinfo.cpython- 36.py:20: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Error encountered during module reload. Modules were not reloaded. [WARNING ] /var/cache/salt/minion/extmods/modules/udevdb.py:24: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Template was specified incorrectly: False [ERROR ] Template was specified incorrectly: False local: Data failed to compile: ---------- Specified SLS packages.packages_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver ---------- Specified SLS custom.custom_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver
Not sure where else to look for errors?
-- Julio González Gil Release Engineer, SUSE Manager and Uyuni jgonzalez@suse.com -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
Hi, yes we accept the key and it joins salt master successfully. But I think there is some salt reactor process that hooks into the Java side of spacewalk to make the system show in Systems but that is not working. I also realised there have been no tasks scheduled or run in Schedule > Pending Actions or Schedule > Completed Actions since yesterday 10:25. I have restarted taskomatic since then. Is there something wrong with my database? Cheers -----Original Message----- Date: Fri, 12 Apr 2019 13:54:42 +0200 Subject: Re: [uyuni-users] Bootstrapped salt minions never appear as systems Cc: David Mace <david@macemail.co.uk> To: uyuni-users@opensuse.org From: Julio González Gil <jgonzalez@suse.de> Keep in mind that when you onboard a minion using the bootstrap script, you need to acept the key. So the first obvious question (most probably you will answer "yes") is: Is the key marked as accepted? If not, you can accept it from the WebUI. On viernes, 12 de abril de 2019 12:37:02 (CEST) David Mace wrote:
Hi, so this was working initially, but something has broke.
I am running Uyuni 4.0.1 stable on openSUSE 42.3 host.
A new openSUSE Leap 15.0 minion is bootstrapped using bootstrap.sh which was generated using mgr-bootstrap.
It joins Salt master and the key shows using command:
salt-key -L
But the minion never appears in Systems > Systems List
Viewing the Salt message bus using
salt-run state.event pretty=True
I see this:
salt/job/20190411153502494691/new { "_stamp": "2019-04-11T14:35:02.495370", "arg": [], "fun": "mgractionchains.resume", "jid": "20190411153502494691", "minions": [ "minion.domain.co.uk" ], "missing": [], "tgt": "minion.domain.co.uk", "tgt_type": "glob", "user": "salt" }
salt/job/20190411153502494691/ret/minion.domain.co.uk { "_stamp": "2019-04-11T14:35:06.850239", "cmd": "_return", "fun": "mgractionchains.resume", "fun_args": [], "id": "minion.domain.co.uk", "jid": "20190411153502494691", "metadata": { "suma-action-chain": true }, "out": "nested", "retcode": 254, "return": "'mgractionchains.resume' is not available.", "success": false }
From the minion if I do a salt-call state.apply I get this error:
minion.domain.co.uk:~ # salt-call state.apply [WARNING ] /var/cache/salt/minion/extmods/grains/__pycache__/cpuinfo.cpython- 36.py:20: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Error encountered during module reload. Modules were not reloaded. [WARNING ] /var/cache/salt/minion/extmods/modules/udevdb.py:24: DeprecationWarning: Use of 'salt.utils.which_bin' detected. This function has been moved to 'salt.utils.path.which_bin' as of Salt 2018.3.0. This warning will be removed in Salt Neon.
[ERROR ] Template was specified incorrectly: False [ERROR ] Template was specified incorrectly: False local: Data failed to compile: ---------- Specified SLS packages.packages_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver ---------- Specified SLS custom.custom_7a5b9695fb48e3a146f6b8d35caf5e2a in saltenv base is not available on the salt master or through a configured fileserver
Not sure where else to look for errors?
-- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
On 4/12/19 1:59 PM, David Mace wrote:
Hi, yes we accept the key and it joins salt master successfully. But I think there is some salt reactor process that hooks into the Java side of spacewalk to make the system show in Systems but that is not working.
Please check for clues in /etc/rhn/rhn_web_ui.log around the time keys were accepted. Is there any relevant error? Regards, -- Silvio Moioli SUSE Manager Development Team -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
Hi, in our salt master config we have in /etc/salt/master.d/extra.conf auto_accept: True So we don't manually go to the UI to accept keys. However in the rhn_web_ui.log I am seeing this error (not sure if it's related though) 2019-04-15 12:11:40,184 [pool-7-thread-1] ERROR com.suse.manager.webui.websocket.Notification - Notification scheduledExecutorService exception java.lang.NullPointerException: Deflater has been closed at java.util.zip.Deflater.ensureOpen(Deflater.java:559) at java.util.zip.Deflater.deflate(Deflater.java:440) at org.apache.tomcat.websocket.PerMessageDeflate.sendMessagePart(PerMessag eDeflate.java:348) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.startMessage(WsRem oteEndpointImplBase.java:300) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase$TextMessageSendHan dler.write(WsRemoteEndpointImplBase.java:759) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendPartialString( WsRemoteEndpointImplBase.java:252) at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemot eEndpointImplBase.java:195) at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndp ointBasic.java:37) at com.suse.manager.webui.websocket.Notification.sendMessage(Notification. java:136) at com.suse.manager.webui.websocket.Notification.lambda$spreadUpdate$0(Not ification.java:161) at java.util.HashMap.forEach(HashMap.java:1289) at com.suse.manager.webui.websocket.Notification.spreadUpdate(Notification .java:160) at com.suse.manager.webui.websocket.Notification.lambda$static$3(Notificat ion.java:221) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac cess$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru n(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja va:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j ava:624) at java.lang.Thread.run(Thread.java:748) Cheers -----Original Message----- Date: Mon, 15 Apr 2019 07:52:52 +0200 Subject: Re: [uyuni-users] Bootstrapped salt minions never appear as systems To: uyuni-users@opensuse.org From: Silvio Moioli <moio@suse.de> On 4/12/19 1:59 PM, David Mace wrote:
Hi, yes we accept the key and it joins salt master successfully. But I think there is some salt reactor process that hooks into the Java side of spacewalk to make the system show in Systems but that is not working.
Please check for clues in /etc/rhn/rhn_web_ui.log around the time keys were accepted. Is there any relevant error? Regards, -- Silvio Moioli SUSE Manager Development Team -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
On 4/15/19 1:19 PM, David Mace wrote:
Hi, in our salt master config we have in /etc/salt/master.d/extra.conf
auto_accept: True
So we don't manually go to the UI to accept keys.
However in the rhn_web_ui.log I am seeing this error (not sure if it's related though)
2019-04-15 12:11:40,184 [pool-7-thread-1] ERROR com.suse.manager.webui.websocket.Notification - Notification scheduledExecutorService exception java.lang.NullPointerException: Deflater has been closed at java.util.zip.Deflater.ensureOpen(Deflater.java:559) at java.util.zip.Deflater.deflate(Deflater.java:440) at
It is not related, and looks like this error is raised because a browser was closed during the transfer of notification data. We should probably not error out this loudly, but other than that I do not foresee problems. David, can you reliably reproduce this? How? Are there any other symptoms? Dario, opinions? Regards, -- Silvio Moioli SUSE Manager Development Team -- To unsubscribe, e-mail: uyuni-users+unsubscribe@opensuse.org To contact the owner, e-mail: uyuni-users+owner@opensuse.org
participants (4)
-
David Mace -
Julio González Gil -
Silvio Moioli -
Strahil Nikolov