Hi,

We’ve got a weird behaviour that happened in the last day or so. The Uyuni salt master stopped being able to communicate with nearly all minions except one.

salt -vvv ServerX test.ping       Executing job with jid 20230118190311229937

-------------------------------------------

ServerX:

    Minion did not return. [Not connected]

ERROR: Minions returned with non-zero exit code

The venv-salt-minion service seems to be working on the minion. The service has the error salt-minion[952]: [ERROR] File /etc/venv-salt-minion/minion.d/_mgractionchains.conf not found, but that appears to be cosmetic based on the outstanding issue, and the working minion has the same error.

However, I restarted one of the salt-minions and the salt master appears to be able to connect afterwards. And yet the minion system was rebooted the previous day and hasn’t been patched in the interim so there should have been no version change. Is there possibly a bug in the latest salt minion version? The Ubuntu 22.04 minion version is
venv-salt-minion                      3004-11.3.uyuni                         amd64        The venvjailed client component for Salt

although we’re also getting this across the board on CentOS 7, openSUSE, and multiple versions of Ubuntu. Could the salt master have issued something that put all the salt minions in a wedged state due to some bug? Is there anything we can gather, information wise before restarting those venv-salt-minion services across the board?

Cheers,

Paul-Andre