Hi list, I'm trying to build a new base image for our OpenStack cloud (Ocata). I use cloud-init version leap15:~ # rpm -qa | grep cloud-init cloud-init-config-suse-18.2-lp150.2.6.1.x86_64 cloud-init-18.2-lp150.2.6.1.x86_64 I didn't provide any datasources (intentionally) to test the fallback, we have two use cases: either user-data is provided via config drive (for external networks) or via meta-data (169.254.169.254 for internal networks). Attaching a config drive works fine, cloud-init detects a local datasource and configures the network successfully. But without config-drive it's supposed to fallback to dhcp, and I see those attempts, but for some reason the DHCP lease takes quite long and times out. Cloud-init tries it's config stage too early and fails to read any meta-data. A couple of seconds later the instance gets its IP address and I can manually apply the user-data, but that's not useful, of course. Here's some output from cloud-init.log (only relevant parts): ---cut here--- 2018-10-02 07:55:10,800 - main.py[DEBUG]: No local datasource found [...] 2018-10-02 07:55:10,803 - stages.py[DEBUG]: applying net config names for {'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 'fa:16:3e:e1:d0:f5', 'subnets': [{'type': 'dhcp'}]}], 'version': 1} [...] 2018-10-02 07:55:12,300 - stages.py[DEBUG]: no cache found 2018-10-02 07:55:12,300 - handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found 2018-10-02 07:55:12,301 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance 2018-10-02 07:55:12,306 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.opensuse.Distro'> 2018-10-02 07:55:12,307 - __init__.py[DEBUG]: Looking for data source in: ['OpenStack', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK'] 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Searching for network data source in: ['DataSourceOpenStack', 'DataSourceNone'] 2018-10-02 07:55:12,310 - handlers.py[DEBUG]: start: init-network/search-OpenStack: searching for network data from DataSourceOpenStack 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceOpenStack.DataSourceOpenStack'> 2018-10-02 07:55:12,314 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/18.2'}} configuration 2018-10-02 07:55:12,330 - url_helper.py[DEBUG]: Calling 'http://169.254.169.254/openstack' failed [0/-1s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe01999d240>: Failed to establish a new connection: [Errno 101] Network is unreachable',))] 2018-10-02 07:55:12,330 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack'] after 0 seconds 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: finish: init-network/search-OpenStack: SUCCESS: no network data found from DataSourceOpenStack 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: start: init-network/search-None: searching for network data from DataSourceNone ---cut here--- and here are the timestamps from /var/log/messages for bringing up eth0: ---cut here--- 2018-10-02T09:55:11.621416+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:11.692718+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto [...] 2018-10-02T09:55:12.390001+02:00 leap15-install cloud-init[956]: Cloud-init v. 18.2 running 'init' at Tue, 02 Oct [...] <<<<<< This is where cloud-init fails to configure network >>>>>> [...] 2018-10-02T09:55:13.717296+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) [...] 2018-10-02T09:55:17.821272+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:17.822502+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto 2018-10-02T09:55:18.040147+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) 2018-10-02T09:55:18.118851+02:00 leap15-install wickedd[839]: route ipv4 0.0.0.0/0 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease 2018-10-02T09:55:18.119191+02:00 leap15-install wickedd[839]: route ipv4 169.254.169.254/32 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease [...] 2018-10-02T09:55:32.844273+02:00 leap15-install wicked[955]: lo up 2018-10-02T09:55:32.844718+02:00 leap15-install wicked[955]: eth0 up 2018-10-02T09:55:32.852411+02:00 leap15-install systemd[1]: Started wicked managed network interfaces. 2018-10-02T09:55:32.852661+02:00 leap15-install systemd[1]: Reached target Network. ---cut here--- According to this output the network is configured after cloud-init has finished (at least the relevant stage). This seems to be only a problem at initial boot of instances. If I reboot an existing instance the network is configured way faster and cloud-init successfully runs through its stages. Is this a known issue? I have filed multiple bugs for cloud-init in the past but I'm not sure if this is one. Is there a workaround to delay the cloud-init run until the instance has an IP address? I would appreciate any pointers on how to proceed. If there's more information needed, please let me know. Best regards, Eugen -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org