[Bug 1203393] New: microOS cloud-init (OpenStack variant) fails due to attempting systemd-network over NetworkManager
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 Bug ID: 1203393 Summary: microOS cloud-init (OpenStack variant) fails due to attempting systemd-network over NetworkManager Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: aarch64 OS: openSUSE Tumbleweed Status: NEW Severity: Major Priority: P5 - None Component: MicroOS Assignee: kubic-bugs@opensuse.org Reporter: matt@traverse.com.au QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:104.0) Gecko/20100101 Firefox/104.0 Build Identifier: When using an OpenStack-ContainerHost [aarch64] microOS Tumbleweed build (from https://en.opensuse.org/Portal:MicroOS), cloud-init will fail as it tries to create a file with a systemd-network UID. Despite NetworkManager being the microOS 'default', and systemd-network not being installed, cloud-init decides to create for systemd-networkd You do not even need to have a valid cloud-init source for this, the issue will occur on a 'fresh' system, verified by examining the log files after first boot ---------------------------------------------- 2022-09-14 06:54:40,414 - util.py[DEBUG]: Writing to /etc/systemd/network/10-cloud-init-eth0.network - wb: [644] 69 bytes 2022-09-14 06:54:40,416 - util.py[WARNING]: failed stage init-local 2022-09-14 06:54:40,417 - util.py[DEBUG]: failed stage init-local Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/cloudinit/util.py", line 1430, in chownbyname uid = pwd.getpwnam(user).pw_uid KeyError: "getpwnam(): name not found: 'systemd-network'" The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/cloudinit/cmd/main.py", line 689, in status_wrapper ret = functor(name, args) File "/usr/lib/python3.10/site-packages/cloudinit/cmd/main.py", line 398, in main_init init.apply_network_config(bring_up=bring_up_interfaces) [edited for brevity, I will attach the full log separately] OSError: Unknown user or group: "getpwnam(): name not found: 'systemd-network'" ---------------------------------------------- I built a custom microOS image (see https://build.opensuse.org/package/revisions/home:mcbridematt:branches:devel... ) with systemd-network installed and verified this "resolves" the issue. However, I haven't tried to diagnose why cloud-init wants to use networkd over NetworkManager. Reproducible: Always Steps to Reproduce: 1. Boot a microOS Tumbleweed VM with a cloud-init file (e.g cidata image) 2. Attempt to use system once it reaches the login prompt (for example, using credentials or SSH keys set in the supplied cloud-config) Actual Results: Login fails because cloud-init did not complete. No mention of the cloud-init failure is provided to the user at the console (except maybe the hostname not being configured by cloud-init) , it can only be discovered by examining the /var partition offline. The specific image I am using is: openSUSE-MicroOS.aarch64-16.0.0-OpenStack-Cloud-Snapshot20220911.qcow2 -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c1 --- Comment #1 from Mathew McBride <matt@traverse.com.au> --- Created attachment 861456 --> http://bugzilla.opensuse.org/attachment.cgi?id=861456&action=edit cloud-init verbose log showing failure -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c2 --- Comment #2 from Mathew McBride <matt@traverse.com.au> --- Created attachment 861457 --> http://bugzilla.opensuse.org/attachment.cgi?id=861457&action=edit cloud-init output log showing failure and system state afterwards -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c3 Martin Petersen <martin@petersen20.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin@petersen20.de --- Comment #3 from Martin Petersen <martin@petersen20.de> --- The problem belongs to an old cloud-init version used by MicroOS. This old version does not have support for NetworkManager. I build an actual version of cloud-init (22.3.4) and all works fine for me. The version can be downloaded from the opensuse build system: https://build.opensuse.org/package/show/home%3Amartux69/cloud-init -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c4 Hans Rakers <hans@shoq.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hans@shoq.com --- Comment #4 from Hans Rakers <hans@shoq.com> --- There are more issues with the current cloud-init package. It still depends on wicked-service so wicked gets pulled back in to your NetworkManager based system. Also the systemd unit for cloud-init.service still says 'After=wicked.service' instead of 'After=NetworkManager.service'. Seems like cloud-init didn't get much love since the switch from wicked to NetworkManager. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c5 --- Comment #5 from Hans Rakers <hans@shoq.com> --- (In reply to Martin Petersen from comment #3)
The problem belongs to an old cloud-init version used by MicroOS. This old version does not have support for NetworkManager. I build an actual version of cloud-init (22.3.4) and all works fine for me. The version can be downloaded from the opensuse build system: https://build.opensuse.org/package/show/home%3Amartux69/cloud-init
I just tested this package and it instantly solved all my networking problems. Thank you! I've been struggling with the network intermittently not coming up, and thus being unable to contact the metadata service of my cloud, resulting in a dead instance. I've tried everything from systemd drop-ins to creating a /etc/NetworkManager/NetworkManager.conf to convince cloud-init to look at the sysconfig renderer, to no avail. This newer cloud-init package is the answer. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c6 --- Comment #6 from Hans Rakers <hans@shoq.com> --- (In reply to Hans Rakers from comment #5)
(In reply to Martin Petersen from comment #3)
The problem belongs to an old cloud-init version used by MicroOS. This old version does not have support for NetworkManager. I build an actual version of cloud-init (22.3.4) and all works fine for me. The version can be downloaded from the opensuse build system: https://build.opensuse.org/package/show/home%3Amartux69/cloud-init
I just tested this package and it instantly solved all my networking problems. Thank you!
I've been struggling with the network intermittently not coming up, and thus being unable to contact the metadata service of my cloud, resulting in a dead instance. I've tried everything from systemd drop-ins to creating a /etc/NetworkManager/NetworkManager.conf to convince cloud-init to look at the sysconfig renderer, to no avail. This newer cloud-init package is the answer.
This newer package still has 'After=wicked.service' in `/usr/lib/systemd/system/cloud-init.service` so i still need a drop-in that has the following: # cat /etc/systemd/system/cloud-init.service.d/boot-order.conf [Unit] After=NetworkManager.service After=NetworkManager-wait-online.service -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1203393 http://bugzilla.opensuse.org/show_bug.cgi?id=1203393#c7 Martin Petersen <martin@petersen20.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS --- Comment #7 from Martin Petersen <martin@petersen20.de> --- Thanks for the hint @Hans I build a new package, also updating cloud-init to version 22.4. It can be downloaded from https://build.opensuse.org/project/show/home:martux69:branches:Cloud:Tools:N... . The main branch from the obs core maintainers under https://build.opensuse.org/package/show/Cloud:Tools:Next/cloud-init is also in progress to a new version. I try to submit my changes to it. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com