Hello all, 16.06.2016 13:52, Marius Tomaschewski:

Am 18.05.2016 um 18:16 schrieb Jason Schultz:

...

I hope I am not starting a new thread with this, I am attempting to reply to Nikolai Zhubr's last message from his thread: https://lists.opensuse.org/wicked-devel/2016-03/msg00004.html

I had started a similar thread a couple weeks before that here, which also did not get a response:

https://lists.opensuse.org/wicked-devel/2016-03/msg00000.html

The reason for replying again is to agree with the conclusion that "LINK_REQUIRED=no" should be the default in order to prevent this type of issue when there's no physical link.

Definitely not. Many, many things (in the kernel and elsewhere) starts after the carrier detection / cannot start without carrier, e.g. ipv6, bond + bridge protocols, any kind of link authentication (wpa), any autoconfiguration incl. dhcp, duplicate address detection [required (except in obsolete RFCs) and enabled also for ipv4] ...

Ok, then you should disallow BOOTPROTO='static' and also all those tons of software that are not designed to bind dynamically to randomly appearing interfaces, so as to be truly consistent. No, there is nothing wrong with this (fully-dynamic) behaviour in itself. It may appear handy and fit in many cases, e.g. for mobiles. However problem is: - compatibility, - expectations, - consistency. (-and still not all of us run Suse on mobiles but some do on servers and desktops!) If you force a beautifull new behaviour that is horribly incompatible with (a lot of regular) existing software, incompatible with people's expectations, and introduces inconsistencies like silently treating BOOTPROTO='static' as kind of bogus or obsolete, then you should really really make it clear that this new shiny solution is still NOT something that can be relied upon as a proper REPLACEMENT of old buggy scriptware. But better yet... make it reasonably compatilble, please. Don't take me wrong, I understand and appreciate the amount of hard thinking and effort wickedd developers have invested into it already, and I'd really like it success and good acceptance. I seriously hope wickedd helps to avoid some ancient and annoying network issues (that I'm quite aware of and suffer from time to time on non-wickedd installations). But still again: unexpectidly broken BOOTPROTO='static' does not add much joy, especially when you suddenly find your server inaccessible. I really see no reason for not defaulting to LINK_REQUIRED=no when BOOTPROTO=static. In such case dhcp does not need to even be mentioned. Regarding duplicate address detection I'm not exactly sure how it should work in all detail, but I'd suppose it would need to somehow deal with link going physically up and down repratedly anyway? So what is the benefit of bothering so much at some particular moment then? Thank you, Regards, Nikolai

See also "man ifcfg", where it is described already until some extent and also https://github.com/openSUSE/wicked/blob/master/doc/FAQ.txt.

Make sure, nanny use is enabled (<use-nanny>true</use-nanny> in the /etc/wicked/common.xml) -- it will set it up once a carrier is there.

...

As I explained in my previous post, I thought there would already be a solution to this issue that would require less manual configuration, such as setting LINK_REQUIRED=no in /etc/sysconfig/network/config but this did not work for me. I spent some time looking at the source code to see if I could find an option in the XML files that would work to change the default behavior, but I was unsuccessful.

Currently, there is none. Same as there is no possibility to set e.g. BOOTPROTO=dhcp by default.

But... https://github.com/openSUSE/wicked/pull/657 adds it now to make some people that like to break rules happy :-)

Gruesse / Regards, Marius Tomaschewski,

-- To unsubscribe, e-mail: wicked-devel+unsubscribe@opensuse.org To contact the owner, e-mail: wicked-devel+owner@opensuse.org

Am 16.06.2016 um 17:59 schrieb Marius Tomaschewski:

Am 16.06.2016 um 14:42 schrieb Nikolai Zhubr: [...] But LINK_REQUIRED=no default is definitely the wrong way to go.

But now, you'll be able to set such a default for you: https://github.com/openSUSE/wicked/pull/657 Gruesse / Regards, Marius Tomaschewski , -- SUSE LINUX GmbH, GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg), Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: wicked-devel+unsubscribe@opensuse.org To contact the owner, e-mail: wicked-devel+owner@opensuse.org

Am 16.06.2016 um 20:51 schrieb Nikolai Zhubr:

Hello Marius, 16.06.2016 19:04, Marius Tomaschewski:

...

Am 16.06.2016 um 17:59 schrieb Marius Tomaschewski:

...

Am 16.06.2016 um 14:42 schrieb Nikolai Zhubr: [...] But LINK_REQUIRED=no default is definitely the wrong way to go.

But now, you'll be able to set such a default for you:

https://github.com/openSUSE/wicked/pull/657

Good to know, I'll definitely add it to my emergency reminder. Although it still feels somewhat hacky...

Because it is :-) The short version of all this is: LINK_REQUIRED=no is not possible as default, because it breaks requested + approved features and reverts to obsolete RFCs [that not require duplicate address detection on ipv4], which were written before many of the protocols/mechanisms existing today.

I appreciate your detailed explantion of why changing the default is inacceptable and I tend to agree that your reasons are valid.

I explicitly spend my time here to explain in hope it will reach more people. Perhaps even somebody willing to help. Yes, we definitely have to improve the documentation -- any help here would be very appreciated. e.g. using https://en.opensuse.org/Portal:Wicked https://github.com/openSUSE/wicked/wiki https://github.com/openSUSE/wicked/tree/master/doc[/FAQ.txt] [merge them somehow?]

But then question is, how to properly set up a most trivial ipv4-only configuration with a static (fixed forever) ip address and some trivial tcp service (say httpd) bound to this fixed ip address only?

As I see it now, there are 2 choices currently:

1. Leave all per default. Works fine until one day ocasionally link happens to be temporary down at the time of booting up. The service will not be able to bind to the address it expects and will fail, which is unwise because we know our address is fixed forever anyway. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Really? :-) Usually, you don't install 1000 routers + 1000 servers for 1 workstation in the network. It is exactly the another way around: you install ~1..2 router and few servers for 2000 workstations. In case of the routers, you can disable e.g. duplicate address detection and define "anybody using this address is broken, I'm not". In case of some servers, you can do the same. But be prepared to manually verify it and fix machines using an address, before you install a new one -- and unintentionally have a typo in the IP addr. In case of workstations -- using a static address configuration (as with any dynamic addresses, dad is simply mandatory), you would be very unhappy when the person effectively doing this in your advise mixed the gateway or server address with the ip address and your advise was to disable duplicate address detection/set LINK_REQUIRE=no. Even worser with mac address duplicates (e.g. in some virtualization setup where) -- ipv6 covers this, ipv4 not really. Let's go through the duplicate address detection only -- just because it is the most trivial variant to test/see, it's not the only one. In ipv6 this is more clear/visible. For IPv4, the kernel unfortunately does not provide such functionality/handling, but it is quite similar: there are also diverse L2 protocols which all start with/need a carrier (e.g. bond lacp, bridge stp, link authentication) and have to finish before the address can be really used. Instead of arp, the kernel is using ipv6 multicasts (fe80 link local <-> ff multicast addresses). When the kernel is applying an IPv6 address (regardless if static or not), it sets a tentative flag on it. [Yes, you can disable it and run e.g. into MAC collisions, ... -> have fun]. It is visible in the interface, but is not ready yet/pending and it is not possible to bind it in a service, thus a service start will simply fail, e.g. netcat here [the link _will_ get carrier later]: # ip link set down eth1 ; echo ; ip a s dev eth1 ; echo ; ip link set up dev eth1 ; ip a s dev eth1 ; echo ; ip a a 2001:db8:a::b/64 dev eth1 ; echo ; ip a s dev eth1 ; echo ; netcat -v -6 -l 2001:db8:a::b 8888 ; echo ; ip a s dev eth1 ; echo ; sleep 2 ; ip a s dev eth1 ; echo ; sleep 5 ; ip a s dev eth1 3: eth1: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff 3: eth1: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff 3: eth1: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff inet6 2001:db8:a::b/64 scope global tentative valid_lft forever preferred_lft forever netcat: Cannot assign requested address 3: eth1: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff inet6 2001:db8:a::b/64 scope global tentative valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff inet6 2001:db8:a::b/64 scope global tentative valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 68:05:ca:0a:39:e7 brd ff:ff:ff:ff:ff:ff inet6 2001:db8:a::b/64 scope global valid_lft forever preferred_lft forever inet6 fe80::6a05:caff:fe0a:39e7/64 scope link valid_lft forever preferred_lft forever Once the link carrier has been detected, ... ipv6 assigns a tentative link local address (you can think of it as "ipv6 MAC address", it is quite often an EUI64 mapped MAC address). Then it starts duplicate address detection of the link local address as well as all other addresses you may have assigned before. When all is fine, it will remove the tentative flag _and_ emit a NEWADDR event -> programs are able to bind to the addresses. In case of failure (duplicate has been detected), the kernel: - just deletes addresses dynamic with a lifetime + emits DELADDR - removes the tentative flag and sets dadfailed flag on persistent (infinite lifetime) addresses + emits NEWADDR [broken one]. In ipv4, you simply send to /dev/null and don't see all this. Of course, it is possible to monitor much more things [e.g. bridge traffic on switches, neighbour discovery ...].

2. Manually tweak LINK_REQUIRED. This will allow our service to successfully start no matter what. But feels hacky, and you might not know about this LINK_REQUIRED thing until you get a surprise.

On a router, I'd do it or at least disable duplicate address detection (accept_dad=0, CHECK_DUPLICATE_IP=no); otherwise definitely not as a default but per interface if _really_ required.

I'm not sure, both variants feel somewhat flawed to me (And this is a remarkably trivial use case!). I can live with the second one, but it looks like there is still room for improvement.

There is simply no "golden way" that works in any case, just a bunch of config tweaks / configuration scenarios permitting to address the issue. "I just need to start after the network service started/network-online is reached" is a false conclusion. It is not possible to block the system at boot time forever -> there is always a timeout. In our case, it is the WAIT_FOR_INTERFACES setting. Regardless of the state, we have return & report after. About services: a service which binds to a specific address (or device in case of BindDevice) without to monitor the addresses/handle this accordingly, is just broken or misconfigured. You have to assume that the devices are detected in random order and may reach carrier (or not) also in random order and all this at a unspecified point of time. Anything else is opportunistic (hack :). So: - Bind to 0.0.0.0 || :: + set ACLs as neeeded - Start the service once the address is available (e.g. via POST_UP_SCRIPT="systemd:....") - Fix the service (report a bug) to bind using a free bind, from "man 7 ip": IP_FREEBIND (since Linux 2.4) If enabled, this boolean option allows binding to an IP address that is nonlocal or does not (yet) exist. This permits listening on a socket, without requiring the underlying network interface or the specified dynamic IP address to be up at the time that the application is trying to bind to it. This option is the per-socket equivalent of the ip_nonlocal_bind /proc interface described below. See e.g. [SLES Bug #958728 that caused to apply the patch]: https://github.com/rsyslog/rsyslog/blob/v8-stable/ChangeLog where we've implemented setting of the option for rsyslog. See also: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ which basically says the same. Further, try to configure explicitly. Quite often, the config does not say what it exactly wants and we use some defaults (e.g. kernel sysctl where default is to autoconf ipv6 even on loopback, even the kernel will never do it there ;-) where we think, it will fit and cause the lowest number of bug reports / complains / regressions. And: Do _not_ disable nanny use in wicked (set use-nanny true in /etc/wicked/common.xml or local.xml): nanny is responsible to "ifup" an interface when it appears and to continue with ifup when the carrier arrives (or other prerequires) and call POST_UP scripts. A wicked with disabled nanny makes a direct/inline one-shot "ifup" for 30sec (by default) and exits+reports -- it will _not_ continue later when the carrier is there. It is kind of test mode allowing us to debug what the finite state machine is doing without to debug a daemon (which may also handle stuff we are not relevant, ...). A wicked without nanny will not reapply the config on restart e.g. on update (not from /etc which may have changed in the meantime; nanny keeps a copy of the last applied config) or renew leases. Gruesse / Regards, Marius Tomaschewski , -- SUSE LINUX GmbH, GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg), Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: wicked-devel+unsubscribe@opensuse.org To contact the owner, e-mail: wicked-devel+owner@opensuse.org

Hello Marius, 17.06.2016 12:10, Marius Tomaschewski: [...]

...

2. Manually tweak LINK_REQUIRED. This will allow our service to successfully start no matter what. But feels hacky, and you might not know about this LINK_REQUIRED thing until you get a surprise.

On a router, I'd do it or at least disable duplicate address detection (accept_dad=0, CHECK_DUPLICATE_IP=no); otherwise definitely not as a default but per interface if _really_ required.

Ok, then maybe it would be usefull to equip yast network settings UI with some checkbox to set LINK_REQUIRED in a confortable way? There is some unused space in GUI to the right of dhcp menu so an additional checkbox would fit perfectly (At least in 13.2 it looks so). I don't think it would cause too much abuse. And as a bonus, it would then allow to add some brief introduction/warnings and/or add links to wicked wiki into context help linked to the GUI so user would more easily get at least some starting point when in trouble or in doubt. It could save people some time and also educate them about wickedd and all this.

...

I'm not sure, both variants feel somewhat flawed to me (And this is a remarkably trivial use case!). I can live with the second one, but it looks like there is still room for improvement.

There is simply no "golden way" that works in any case, just a bunch of config tweaks / configuration scenarios permitting to address the issue.

Probably the root cause of problems is mixing dynamic and static approaches. If everything is static it works (well, used to work...) reliably. If everything is dynamic it should be fine too. But in reality most systems are probably still far from being purely dynamic because of tons of legacy software. [...]

- Bind to 0.0.0.0 || :: + set ACLs as neeeded

- Start the service once the address is available (e.g. via POST_UP_SCRIPT="systemd:....")

I like this one. I'll try to use it somewhere.

- Fix the service (report a bug) to bind using a free bind, from "man 7 ip":

IP_FREEBIND (since Linux 2.4)

I have to confess I didn't know about this - thanks for pointing out! Thank you, Regards, Nikolai -- To unsubscribe, e-mail: wicked-devel+unsubscribe@opensuse.org To contact the owner, e-mail: wicked-devel+owner@opensuse.org