[opensuse-cloud] Networking provided via meta-data does not work

newer
[opensuse-cloud] Devstack support...

older
[opensuse-cloud] Queens packages...

Ben Howard

2 Nov 2017 2 Nov '17

01:22

Hello from DigtialOcean! I am the engineer at DigitalOcean that is responsible for making distributions work on our cloud. I have been attempting to work with community members to try and get OpenSUSE on DigitalOcean, but those efforts have failed. The root of the problem is that DigitalOcean (like a few others clouds) does not have DHCP. To get around this we have implemented both ConfigDrive and our DataSource that handles how to configure a network in Cloud-init. On other distributions, this method works just fine. However the OpenSUSE Cloud-init distribution module does NOT handle configuring networking when the network configuration comes from either the ConfigDrive or DigitalOcean Cloud-init datasources. To whit: CONSOLE: 2017-11-01 14:47:22,468 - __init__.py[WARNING]: apply_network_config is not currently implemented for distribution ''. Attempting to use apply_network /var/log/cloud-init.log Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 346, in _bring_up_interface (_out, err) = util.subp(cmd) File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1849, in subp cmd=args) ProcessExecutionError: Unexpected error while running command. Command: ['ifup', 'eth0'] Exit code: 162 Reason: - Stdout: eth0 setup-in-progress Stderr: - (That was taken from a OpenSUSE LEAP 42.3 image booted on DigitalOcean). Anyhow, this is blocking bringing OpenSUSE to DigitalOcean. If someone would like to work with me to produce upstream images that work -- and can configure networking, I would love the help. I can give free instances (just ping me off list with your DigitalOcean account). Thanks, Ben -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

Show replies by date

Eugen Block

2 Nov 2 Nov

06:47

Hi Ben, which version of cloud-init do you use? Our cloud is running on OpenSUSE and until now there's only one cloud-init version we are able to use, it's 0.7.8. Just last week I reported a bug for the current version 17.1 because the network config was only applied if the instance (Leap 42.3) was launched in an internal network with active DHCP. I hope this will be fixed soon. Regards, Eugen Zitat von Ben Howard :

...

Hello from DigtialOcean!

I am the engineer at DigitalOcean that is responsible for making distributions work on our cloud. I have been attempting to work with community members to try and get OpenSUSE on DigitalOcean, but those efforts have failed.

The root of the problem is that DigitalOcean (like a few others clouds) does not have DHCP. To get around this we have implemented both ConfigDrive and our DataSource that handles how to configure a network in Cloud-init. On other distributions, this method works just fine.

However the OpenSUSE Cloud-init distribution module does NOT handle configuring networking when the network configuration comes from either the ConfigDrive or DigitalOcean Cloud-init datasources.

To whit:

CONSOLE: 2017-11-01 14:47:22,468 - __init__.py[WARNING]: apply_network_config is not currently implemented for distribution ''. Attempting to use apply_network

/var/log/cloud-init.log Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 346, in _bring_up_interface (_out, err) = util.subp(cmd) File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1849, in subp cmd=args) ProcessExecutionError: Unexpected error while running command. Command: ['ifup', 'eth0'] Exit code: 162 Reason: - Stdout: eth0 setup-in-progress Stderr: -

(That was taken from a OpenSUSE LEAP 42.3 image booted on DigitalOcean).

Anyhow, this is blocking bringing OpenSUSE to DigitalOcean. If someone would like to work with me to produce upstream images that work -- and can configure networking, I would love the help. I can give free instances (just ping me off list with your DigitalOcean account).

Thanks, Ben -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Eugen Block voice : +49-40-559 51 75 NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 Postfach 61 03 15 D-22423 Hamburg e-mail : eblock@nde.ag Vorsitzende des Aufsichtsrates: Angelika Mozdzen Sitz und Registergericht: Hamburg, HRB 90934 Vorstand: Jens-U. Mozdzen USt-IdNr. DE 814 013 983 -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

Ben Howard

14:38

Specifically, I am hitting this on OpenSUSE LEAP 42.3 which tracks 17.1. Do you have the bug that's been filed? The problem is specific to the OpenSUSE datasource. I don't know enough about Wicked and the networking stack on OpenSUSE. ~Ben On Thu, Nov 2, 2017 at 12:47 AM, Eugen Block wrote:

...

Hi Ben,

which version of cloud-init do you use? Our cloud is running on OpenSUSE and until now there's only one cloud-init version we are able to use, it's 0.7.8. Just last week I reported a bug for the current version 17.1 because the network config was only applied if the instance (Leap 42.3) was launched in an internal network with active DHCP. I hope this will be fixed soon.

Regards, Eugen

Zitat von Ben Howard :

...
Hello from DigtialOcean!

I am the engineer at DigitalOcean that is responsible for making distributions work on our cloud. I have been attempting to work with community members to try and get OpenSUSE on DigitalOcean, but those efforts have failed.

The root of the problem is that DigitalOcean (like a few others clouds) does not have DHCP. To get around this we have implemented both ConfigDrive and our DataSource that handles how to configure a network in Cloud-init. On other distributions, this method works just fine.

However the OpenSUSE Cloud-init distribution module does NOT handle configuring networking when the network configuration comes from either the ConfigDrive or DigitalOcean Cloud-init datasources.

To whit:

CONSOLE: 2017-11-01 14:47:22,468 - __init__.py[WARNING]: apply_network_config is not currently implemented for distribution ''. Attempting to use apply_network

/var/log/cloud-init.log Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 346, in _bring_up_interface (_out, err) = util.subp(cmd) File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1849, in subp cmd=args) ProcessExecutionError: Unexpected error while running command. Command: ['ifup', 'eth0'] Exit code: 162 Reason: - Stdout: eth0 setup-in-progress Stderr: -

(That was taken from a OpenSUSE LEAP 42.3 image booted on DigitalOcean).

Anyhow, this is blocking bringing OpenSUSE to DigitalOcean. If someone would like to work with me to produce upstream images that work -- and can configure networking, I would love the help. I can give free instances (just ping me off list with your DigitalOcean account).

Thanks, Ben -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Eugen Block voice : +49-40-559 51 75 NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 Postfach 61 03 15 D-22423 Hamburg e-mail : eblock@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen Sitz und Registergericht: Hamburg, HRB 90934 Vorstand: Jens-U. Mozdzen USt-IdNr. DE 814 013 983

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Ben Howard Senior Engineer (720) 507.4209 bh@digitalocean.com ________________________________ We're Hiring! | @digitalocean | GPG Key | linkedin | keybase -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

Eugen Block

3 Nov 3 Nov

07:55

Sure: https://bugzilla.opensuse.org/show_bug.cgi?id=1064854 I updated it yesterday and added this thread to get more attention. Regards, Eugen Zitat von Ben Howard :

...

Specifically, I am hitting this on OpenSUSE LEAP 42.3 which tracks 17.1. Do you have the bug that's been filed?

The problem is specific to the OpenSUSE datasource. I don't know enough about Wicked and the networking stack on OpenSUSE.

~Ben

On Thu, Nov 2, 2017 at 12:47 AM, Eugen Block wrote:

...
Hi Ben,

which version of cloud-init do you use? Our cloud is running on OpenSUSE and until now there's only one cloud-init version we are able to use, it's 0.7.8. Just last week I reported a bug for the current version 17.1 because the network config was only applied if the instance (Leap 42.3) was launched in an internal network with active DHCP. I hope this will be fixed soon.

Regards, Eugen

Zitat von Ben Howard :

...
Hello from DigtialOcean!

I am the engineer at DigitalOcean that is responsible for making distributions work on our cloud. I have been attempting to work with community members to try and get OpenSUSE on DigitalOcean, but those efforts have failed.

The root of the problem is that DigitalOcean (like a few others clouds) does not have DHCP. To get around this we have implemented both ConfigDrive and our DataSource that handles how to configure a network in Cloud-init. On other distributions, this method works just fine.

However the OpenSUSE Cloud-init distribution module does NOT handle configuring networking when the network configuration comes from either the ConfigDrive or DigitalOcean Cloud-init datasources.

To whit:

CONSOLE: 2017-11-01 14:47:22,468 - __init__.py[WARNING]: apply_network_config is not currently implemented for distribution ''. Attempting to use apply_network

/var/log/cloud-init.log Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 346, in _bring_up_interface (_out, err) = util.subp(cmd) File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1849, in subp cmd=args) ProcessExecutionError: Unexpected error while running command. Command: ['ifup', 'eth0'] Exit code: 162 Reason: - Stdout: eth0 setup-in-progress Stderr: -

(That was taken from a OpenSUSE LEAP 42.3 image booted on DigitalOcean).

Anyhow, this is blocking bringing OpenSUSE to DigitalOcean. If someone would like to work with me to produce upstream images that work -- and can configure networking, I would love the help. I can give free instances (just ping me off list with your DigitalOcean account).

Thanks, Ben -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Eugen Block voice : +49-40-559 51 75 NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 Postfach 61 03 15 D-22423 Hamburg e-mail : eblock@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen Sitz und Registergericht: Hamburg, HRB 90934 Vorstand: Jens-U. Mozdzen USt-IdNr. DE 814 013 983

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Ben Howard Senior Engineer (720) 507.4209 bh@digitalocean.com ________________________________ We're Hiring! | @digitalocean | GPG Key | linkedin | keybase

Eugen Block

16 Jan 16 Jan

16:03

Hi, this is just an update on the cloud-init datasources. I filed another bug report [1] for cloud-init because using multiple datasources in the config file does not work as expected. Just in case you're hitting that issue, too. Regards, Eugen [1] https://bugzilla.opensuse.org/show_bug.cgi?id=1068971 Zitat von Eugen Block :

...

Sure: https://bugzilla.opensuse.org/show_bug.cgi?id=1064854

I updated it yesterday and added this thread to get more attention.

Regards, Eugen

Zitat von Ben Howard :

...
Specifically, I am hitting this on OpenSUSE LEAP 42.3 which tracks 17.1. Do you have the bug that's been filed?

The problem is specific to the OpenSUSE datasource. I don't know enough about Wicked and the networking stack on OpenSUSE.

~Ben

On Thu, Nov 2, 2017 at 12:47 AM, Eugen Block wrote:

...
Hi Ben,

which version of cloud-init do you use? Our cloud is running on OpenSUSE and until now there's only one cloud-init version we are able to use, it's 0.7.8. Just last week I reported a bug for the current version 17.1 because the network config was only applied if the instance (Leap 42.3) was launched in an internal network with active DHCP. I hope this will be fixed soon.

Regards, Eugen

Zitat von Ben Howard :

...
Hello from DigtialOcean!

I am the engineer at DigitalOcean that is responsible for making distributions work on our cloud. I have been attempting to work with community members to try and get OpenSUSE on DigitalOcean, but those efforts have failed.

The root of the problem is that DigitalOcean (like a few others clouds) does not have DHCP. To get around this we have implemented both ConfigDrive and our DataSource that handles how to configure a network in Cloud-init. On other distributions, this method works just fine.

However the OpenSUSE Cloud-init distribution module does NOT handle configuring networking when the network configuration comes from either the ConfigDrive or DigitalOcean Cloud-init datasources.

To whit:

CONSOLE: 2017-11-01 14:47:22,468 - __init__.py[WARNING]: apply_network_config is not currently implemented for distribution ''. Attempting to use apply_network

/var/log/cloud-init.log Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cloudinit/distros/__init__.py", line 346, in _bring_up_interface (_out, err) = util.subp(cmd) File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1849, in subp cmd=args) ProcessExecutionError: Unexpected error while running command. Command: ['ifup', 'eth0'] Exit code: 162 Reason: - Stdout: eth0 setup-in-progress Stderr: -

(That was taken from a OpenSUSE LEAP 42.3 image booted on DigitalOcean).

Anyhow, this is blocking bringing OpenSUSE to DigitalOcean. If someone would like to work with me to produce upstream images that work -- and can configure networking, I would love the help. I can give free instances (just ping me off list with your DigitalOcean account).

Thanks, Ben -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Eugen Block voice : +49-40-559 51 75 NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 Postfach 61 03 15 D-22423 Hamburg e-mail : eblock@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen Sitz und Registergericht: Hamburg, HRB 90934 Vorstand: Jens-U. Mozdzen USt-IdNr. DE 814 013 983

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- Ben Howard Senior Engineer (720) 507.4209 bh@digitalocean.com ________________________________ We're Hiring! | @digitalocean | GPG Key | linkedin | keybase

-- Eugen Block voice : +49-40-559 51 75 NDE Netzdesign und -entwicklung AG fax : +49-40-559 51 77 Postfach 61 03 15 D-22423 Hamburg e-mail : eblock@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen Sitz und Registergericht: Hamburg, HRB 90934 Vorstand: Jens-U. Mozdzen USt-IdNr. DE 814 013 983

Eugen Block

2 Oct 2 Oct

08:30

New subject: [opensuse-cloud] Cloud-init 18.2 initial run fails with datasource OpenStack

Hi list, I'm trying to build a new base image for our OpenStack cloud (Ocata). I use cloud-init version leap15:~ # rpm -qa | grep cloud-init cloud-init-config-suse-18.2-lp150.2.6.1.x86_64 cloud-init-18.2-lp150.2.6.1.x86_64 I didn't provide any datasources (intentionally) to test the fallback, we have two use cases: either user-data is provided via config drive (for external networks) or via meta-data (169.254.169.254 for internal networks). Attaching a config drive works fine, cloud-init detects a local datasource and configures the network successfully. But without config-drive it's supposed to fallback to dhcp, and I see those attempts, but for some reason the DHCP lease takes quite long and times out. Cloud-init tries it's config stage too early and fails to read any meta-data. A couple of seconds later the instance gets its IP address and I can manually apply the user-data, but that's not useful, of course. Here's some output from cloud-init.log (only relevant parts): ---cut here--- 2018-10-02 07:55:10,800 - main.py[DEBUG]: No local datasource found [...] 2018-10-02 07:55:10,803 - stages.py[DEBUG]: applying net config names for {'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 'fa:16:3e:e1:d0:f5', 'subnets': [{'type': 'dhcp'}]}], 'version': 1} [...] 2018-10-02 07:55:12,300 - stages.py[DEBUG]: no cache found 2018-10-02 07:55:12,300 - handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found 2018-10-02 07:55:12,301 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance 2018-10-02 07:55:12,306 - stages.py[DEBUG]: Using distro class 2018-10-02 07:55:12,307 - __init__.py[DEBUG]: Looking for data source in: ['OpenStack', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK'] 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Searching for network data source in: ['DataSourceOpenStack', 'DataSourceNone'] 2018-10-02 07:55:12,310 - handlers.py[DEBUG]: start: init-network/search-OpenStack: searching for network data from DataSourceOpenStack 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Seeing if we can get any data from 2018-10-02 07:55:12,314 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/18.2'}} configuration 2018-10-02 07:55:12,330 - url_helper.py[DEBUG]: Calling 'http://169.254.169.254/openstack' failed [0/-1s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable',))] 2018-10-02 07:55:12,330 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack'] after 0 seconds 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: finish: init-network/search-OpenStack: SUCCESS: no network data found from DataSourceOpenStack 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: start: init-network/search-None: searching for network data from DataSourceNone ---cut here--- and here are the timestamps from /var/log/messages for bringing up eth0: ---cut here--- 2018-10-02T09:55:11.621416+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:11.692718+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto [...] 2018-10-02T09:55:12.390001+02:00 leap15-install cloud-init[956]: Cloud-init v. 18.2 running 'init' at Tue, 02 Oct [...] <<<<<< This is where cloud-init fails to configure network >>>>>> [...] 2018-10-02T09:55:13.717296+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) [...] 2018-10-02T09:55:17.821272+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:17.822502+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto 2018-10-02T09:55:18.040147+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) 2018-10-02T09:55:18.118851+02:00 leap15-install wickedd[839]: route ipv4 0.0.0.0/0 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease 2018-10-02T09:55:18.119191+02:00 leap15-install wickedd[839]: route ipv4 169.254.169.254/32 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease [...] 2018-10-02T09:55:32.844273+02:00 leap15-install wicked[955]: lo up 2018-10-02T09:55:32.844718+02:00 leap15-install wicked[955]: eth0 up 2018-10-02T09:55:32.852411+02:00 leap15-install systemd[1]: Started wicked managed network interfaces. 2018-10-02T09:55:32.852661+02:00 leap15-install systemd[1]: Reached target Network. ---cut here--- According to this output the network is configured after cloud-init has finished (at least the relevant stage). This seems to be only a problem at initial boot of instances. If I reboot an existing instance the network is configured way faster and cloud-init successfully runs through its stages. Is this a known issue? I have filed multiple bugs for cloud-init in the past but I'm not sure if this is one. Is there a workaround to delay the cloud-init run until the instance has an IP address? I would appreciate any pointers on how to proceed. If there's more information needed, please let me know. Best regards, Eugen -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

Sumit Jamgade

09:35

New subject: [opensuse-cloud] Cloud-init 18.2 initial run fails with datasource OpenStack

Hello, This is a known issue: https://bugzilla.opensuse.org/show_bug.cgi?id=1097388 there is workaround mentioned on the bug, if you are building a custom image, you could install a custom cloud-init too. Hope that is helpful Thanks Sumit On 10/02/2018 10:30 AM, Eugen Block wrote:

...

Hi list,

I'm trying to build a new base image for our OpenStack cloud (Ocata). I use cloud-init version

leap15:~ # rpm -qa | grep cloud-init cloud-init-config-suse-18.2-lp150.2.6.1.x86_64 cloud-init-18.2-lp150.2.6.1.x86_64

I didn't provide any datasources (intentionally) to test the fallback, we have two use cases: either user-data is provided via config drive (for external networks) or via meta-data (169.254.169.254 for internal networks). Attaching a config drive works fine, cloud-init detects a local datasource and configures the network successfully.

But without config-drive it's supposed to fallback to dhcp, and I see those attempts, but for some reason the DHCP lease takes quite long and times out. Cloud-init tries it's config stage too early and fails to read any meta-data. A couple of seconds later the instance gets its IP address and I can manually apply the user-data, but that's not useful, of course.

Here's some output from cloud-init.log (only relevant parts):

---cut here--- 2018-10-02 07:55:10,800 - main.py[DEBUG]: No local datasource found [...] 2018-10-02 07:55:10,803 - stages.py[DEBUG]: applying net config names for {'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 'fa:16:3e:e1:d0:f5', 'subnets': [{'type': 'dhcp'}]}], 'version': 1} [...] 2018-10-02 07:55:12,300 - stages.py[DEBUG]: no cache found 2018-10-02 07:55:12,300 - handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found 2018-10-02 07:55:12,301 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance 2018-10-02 07:55:12,306 - stages.py[DEBUG]: Using distro class 2018-10-02 07:55:12,307 - __init__.py[DEBUG]: Looking for data source in: ['OpenStack', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK'] 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Searching for network data source in: ['DataSourceOpenStack', 'DataSourceNone'] 2018-10-02 07:55:12,310 - handlers.py[DEBUG]: start: init-network/search-OpenStack: searching for network data from DataSourceOpenStack 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Seeing if we can get any data from 2018-10-02 07:55:12,314 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/18.2'}} configuration 2018-10-02 07:55:12,330 - url_helper.py[DEBUG]: Calling 'http://169.254.169.254/openstack' failed [0/-1s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable',))] 2018-10-02 07:55:12,330 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack'] after 0 seconds 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: finish: init-network/search-OpenStack: SUCCESS: no network data found from DataSourceOpenStack 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: start: init-network/search-None: searching for network data from DataSourceNone ---cut here---

and here are the timestamps from /var/log/messages for bringing up eth0:

---cut here--- 2018-10-02T09:55:11.621416+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:11.692718+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto [...] 2018-10-02T09:55:12.390001+02:00 leap15-install cloud-init[956]: Cloud-init v. 18.2 running 'init' at Tue, 02 Oct [...]

<<<<<< This is where cloud-init fails to configure network >>>>>>

[...] 2018-10-02T09:55:13.717296+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) [...] 2018-10-02T09:55:17.821272+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:17.822502+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto 2018-10-02T09:55:18.040147+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) 2018-10-02T09:55:18.118851+02:00 leap15-install wickedd[839]: route ipv4 0.0.0.0/0 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease 2018-10-02T09:55:18.119191+02:00 leap15-install wickedd[839]: route ipv4 169.254.169.254/32 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease [...] 2018-10-02T09:55:32.844273+02:00 leap15-install wicked[955]: lo up 2018-10-02T09:55:32.844718+02:00 leap15-install wicked[955]: eth0 up 2018-10-02T09:55:32.852411+02:00 leap15-install systemd[1]: Started wicked managed network interfaces. 2018-10-02T09:55:32.852661+02:00 leap15-install systemd[1]: Reached target Network. ---cut here---

According to this output the network is configured after cloud-init has finished (at least the relevant stage). This seems to be only a problem at initial boot of instances. If I reboot an existing instance the network is configured way faster and cloud-init successfully runs through its stages.

Is this a known issue? I have filed multiple bugs for cloud-init in the past but I'm not sure if this is one. Is there a workaround to delay the cloud-init run until the instance has an IP address?

I would appreciate any pointers on how to proceed. If there's more information needed, please let me know.

Best regards, Eugen

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

Eugen Block

10:47

New subject: [opensuse-cloud] Cloud-init 18.2 initial run fails with datasource OpenStack

Hi Sumit, thanks for the pointer. I didn't take the time to compare with previous versions, replacing "Before=network-online.target" with "After=..." did the trick. Thank you very much for the quick response and the helpful answer! Regards, Eugen Zitat von Sumit Jamgade :

...

Hello,

This is a known issue:

https://bugzilla.opensuse.org/show_bug.cgi?id=1097388

there is workaround mentioned on the bug, if you are building a custom image, you could install a custom cloud-init too.

Hope that is helpful

Thanks Sumit

On 10/02/2018 10:30 AM, Eugen Block wrote:

...
Hi list,

I'm trying to build a new base image for our OpenStack cloud (Ocata). I use cloud-init version

leap15:~ # rpm -qa | grep cloud-init cloud-init-config-suse-18.2-lp150.2.6.1.x86_64 cloud-init-18.2-lp150.2.6.1.x86_64

I didn't provide any datasources (intentionally) to test the fallback, we have two use cases: either user-data is provided via config drive (for external networks) or via meta-data (169.254.169.254 for internal networks). Attaching a config drive works fine, cloud-init detects a local datasource and configures the network successfully.

But without config-drive it's supposed to fallback to dhcp, and I see those attempts, but for some reason the DHCP lease takes quite long and times out. Cloud-init tries it's config stage too early and fails to read any meta-data. A couple of seconds later the instance gets its IP address and I can manually apply the user-data, but that's not useful, of course.

Here's some output from cloud-init.log (only relevant parts):

---cut here--- 2018-10-02 07:55:10,800 - main.py[DEBUG]: No local datasource found [...] 2018-10-02 07:55:10,803 - stages.py[DEBUG]: applying net config names for {'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 'fa:16:3e:e1:d0:f5', 'subnets': [{'type': 'dhcp'}]}], 'version': 1} [...] 2018-10-02 07:55:12,300 - stages.py[DEBUG]: no cache found 2018-10-02 07:55:12,300 - handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found 2018-10-02 07:55:12,301 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance 2018-10-02 07:55:12,306 - stages.py[DEBUG]: Using distro class 2018-10-02 07:55:12,307 - __init__.py[DEBUG]: Looking for data source in: ['OpenStack', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK'] 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Searching for network data source in: ['DataSourceOpenStack', 'DataSourceNone'] 2018-10-02 07:55:12,310 - handlers.py[DEBUG]: start: init-network/search-OpenStack: searching for network data from DataSourceOpenStack 2018-10-02 07:55:12,310 - __init__.py[DEBUG]: Seeing if we can get any data from 2018-10-02 07:55:12,314 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0, 'headers': {'User-Agent': 'Cloud-Init/18.2'}} configuration 2018-10-02 07:55:12,330 - url_helper.py[DEBUG]: Calling 'http://169.254.169.254/openstack' failed [0/-1s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /openstack (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable',))] 2018-10-02 07:55:12,330 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack'] after 0 seconds 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: finish: init-network/search-OpenStack: SUCCESS: no network data found from DataSourceOpenStack 2018-10-02 07:55:12,330 - handlers.py[DEBUG]: start: init-network/search-None: searching for network data from DataSourceNone ---cut here---

and here are the timestamps from /var/log/messages for bringing up eth0:

---cut here--- 2018-10-02T09:55:11.621416+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:11.692718+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto [...] 2018-10-02T09:55:12.390001+02:00 leap15-install cloud-init[956]: Cloud-init v. 18.2 running 'init' at Tue, 02 Oct [...]

<<<<<< This is where cloud-init fails to configure network >>>>>>

[...] 2018-10-02T09:55:13.717296+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) [...] 2018-10-02T09:55:17.821272+02:00 leap15-install wickedd-dhcp4[808]: eth0: Request to acquire DHCPv4 lease with UUID 5f24b35b-c028-0900-4703-000004000000 2018-10-02T09:55:17.822502+02:00 leap15-install wickedd-dhcp6[810]: eth0: Request to acquire DHCPv6 lease with UUID 5f24b35b-c028-0900-4703-000005000000 in mode auto 2018-10-02T09:55:18.040147+02:00 leap15-install wickedd-dhcp4[808]: eth0: Committed DHCPv4 lease with address 172.17.2.15 (lease time 86400 sec, renew in 43200 sec, rebind in 75600 sec) 2018-10-02T09:55:18.118851+02:00 leap15-install wickedd[839]: route ipv4 0.0.0.0/0 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease 2018-10-02T09:55:18.119191+02:00 leap15-install wickedd[839]: route ipv4 169.254.169.254/32 via 172.17.2.1 dev eth0#2 type unicast table main scope universe protocol dhcp covered by a ipv4:dhcp lease [...] 2018-10-02T09:55:32.844273+02:00 leap15-install wicked[955]: lo up 2018-10-02T09:55:32.844718+02:00 leap15-install wicked[955]: eth0 up 2018-10-02T09:55:32.852411+02:00 leap15-install systemd[1]: Started wicked managed network interfaces. 2018-10-02T09:55:32.852661+02:00 leap15-install systemd[1]: Reached target Network. ---cut here---

According to this output the network is configured after cloud-init has finished (at least the relevant stage). This seems to be only a problem at initial boot of instances. If I reboot an existing instance the network is configured way faster and cloud-init successfully runs through its stages.

Is this a known issue? I have filed multiple bugs for cloud-init in the past but I'm not sure if this is one. Is there a workaround to delay the cloud-init run until the instance has an IP address?

I would appreciate any pointers on how to proceed. If there's more information needed, please let me know.

Best regards, Eugen

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

-- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org

2034

Age (days ago)

2368

Last active (days ago)

List overview

Download

7 comments

3 participants

participants (3)

Ben Howard
Eugen Block
Sumit Jamgade