Am 23.09.2016 um 09:59 schrieb Bolesław Tokarski:

Hello,

I know OVS support in wickedd is limited, and I might have some setup that is more than supported, but I hope we can get somewhere with this. I am still running the 42.1 wicked version 0.6.31.

I have two OVS bridges, I'll skip the 'tunnels' bridge, as I don't configure any interfaces on it through wickedd. It's just for reference, if wicked behaves differently if it has more than one bridge.

Then, I have the 'vms' OVS bridge that I use for KVM - I specify the VLANs the VMs are to join. The VMs connect fine, so also out of scope.

I configure the 'vms' bridge: ovs-vsctl add-br vms

On the host side, I only want connect to one of these VLANs - vlan100, so I configure only that:

# cat /etc/sysconfig/network/ifcfg-vlan100 MTU='1394' STARTMODE='auto' BOOTPROTO='static' OVS_BRIDGE='yes' OVS_BRIDGE_VLAN_PARENT='vms' OVS_BRIDGE_VLAN_TAG='100'

BROADCAST='10.100.0.255' IPADDR='10.100.0.4' NETMASK='255.255.255.0' NETWORK='10.100.0.0'

Let wicked create the vms ovs bridge if not there, e.g.: /etc/sysconfig/network/ifcfg-vms STARTMODE='auto' ... OVS_BRIDGE='yes' OVS_BRIDGE_PORT_DEVICE_1='foo' When you need some special tweaks, use PRE_UP_SCRIPT (man ifcfg) to apply them [before "ip link set up"].

Then, I run an ifup vlan100 and everything seems fine... but.

When ovs starts before wickedd and set ups the vms bridge...

After reboot I get (I am skipping most logs for now, there's nothing interesting in them but this): vlan100: <parent> element references unknown device vms vlan100: cannot resolve reference ovs-bridge/vlan/parent to subordinate device yet

... and it seems the vms bridge disappeared! The 'tunnels' bridge is there after a reboot, they are persistently stored in OVS's DB, so it must have been removed by... wicked!

Yes. Wicked deletes all virtual interfaces that have a deleteDevice call implemented -- basically all which it is able to create. As I wrote in irc, we need something like "OVS_CONFIG=ovsdb", which disables the ovs-bridge factory createDevice and deleteDevice [maybe even more] or at least a global flag, so wicked does not nuke the bridges on ifdown but is using them as "unspec ovs"... or something like this. At the time we've implemented the ovs support, using ifcfg files was the only choice as ovs was unable to setup things from db properly [missed hotplugging, ...]. So we create the briges from ifcfg files and delete them on ifdown. To avoid the ifdown (completely), you could use STARTMODE=nfsroot (aka "wicked ifup --persistent foo", then normal "wicked ifdown foo" does not do any ifdown on it ("wicked ifdown --delete foo" forces it).

Unfortunately, although I set journald.conf's Storage=persistent, I wasn't able to get logs from before a reboot. I don't see why wicked would remove any bridges, but well... that reproduces on every reboot. I found some code that removes OVS bridges in the wicked code... Why is that there? And even so, I did not tell wicked to configure 'vms' interface nor bridge, just a vlan100 port inside it, so it being removed is somewhat unexpected.

Another pecularity: After the boot I re-create the vms bridge, ifup vlan100. Then I remove the bridge and re-create it yet again, without doing an ifdown vlan100 first.

Then: # ifup vlan100 vlan100 no-device

# ifdown vlan100 wicked: ifdown: no matching interfaces

From the logs you provided separately (as the list does not accept attachments):

Without ifcfg-vms, the vlan cannot find the parent and it seems, it generates some crap: Sep 22 20:39:05.536639 phost4 wickedd-nanny[1330]: vms: process event signal deviceReady from /org/opensuse/Network/Interface/10; uuid=<> Sep 22 20:39:05.536748 phost4 wickedd[1309]: __ni_dbus_object_message(path=/org/opensuse/Network/Interface/10, interface=org.freedesktop.DBus.ObjectManager, method=GetManagedObjects) called Sep 22 20:39:05.536890 phost4 wickedd[1309]: __ni_dbus_object_manager_get_managed_objects(path=/org/opensuse/Network/Interface/10, method=GetManagedObjects) Sep 22 20:39:05.537157 phost4 wickedd-nanny[1330]: received refresh for ready device vms (/org/opensuse/Network/Interface/10) Sep 22 20:39:05.537268 phost4 wickedd-nanny[1330]: vlan100: <parent> identified device as "vms" Sep 22 20:39:05.537341 phost4 wickedd-nanny[1330]: vlan100: resolved shared reference ovs-bridge/vlan/parent to subordinate device vms Sep 22 20:39:05.537462 phost4 wickedd-nanny[1330]: vlan100: generating default config for vms child ... Sep 22 20:39:05.539918 phost4 wickedd-nanny[1330]: vms: using device config Sep 22 20:39:05.540037 phost4 wickedd-nanny[1330]: <interface> Sep 22 20:39:05.540162 phost4 wickedd-nanny[1330]: <name>vms</name> Sep 22 20:39:05.540286 phost4 wickedd-nanny[1330]: <link> Sep 22 20:39:05.540409 phost4 wickedd-nanny[1330]: <master>ovs-system</master> Sep 22 20:39:05.540533 phost4 wickedd-nanny[1330]: <port type="ovs-bridge"> Sep 22 20:39:05.540655 phost4 wickedd-nanny[1330]: <bridge>vlan100</bridge> Sep 22 20:39:05.540758 phost4 wickedd-nanny[1330]: </port> .... This is slave/port config to enslave port vms into bridge vlan100: <master>ovs-system</master> <port type="ovs-bridge"> <bridge>vlan100</bridge> </port> This place needs some fix/cleanup in general (bug #954329. When you remove a port from a bridge config, you have to ifup the port + bridge to get rid of the "enslave into bridge" advise on the port. An ifdown of the port can't be done, as it can be e.g. a bond with 1000 vlan's on top of the port which an ifdown would "kill". More close to "real life" perhaps [using normal linux bridge + dummy]: You define a bridge br0 with port dummy0 + ifup them: - on the bridge side: ip link add br0 type bridge # creates the bridge ip link set up dev br0 - on the port side: ip link add dummy0 type dummy # creates the dummy ip link set master br0 dev dummy0 ip link set up dev dummy0 The above "<master>..." and "<port type=x>" is an advise to call ip link set master br0 dev dummy0 which currently remains (is not automatically cleaned up) once the port has been removed from bridge config + bridge ifup. An "wicked ifup br0 dummy0" would push the dummy0 config without the master relation, but unfortunately, there is some crap to take the master from device then :-/ In ovs case, some similar nonsense seems to happen -- and as we see, without ifcfg-vms even worser things. OVS has the problem, that it breaks the usual master & lower device dependencies (master is always ovs-system and config needs a further bridge attr) and without config, it is sometimes hard to deal with it. We will address this bug (there is also some L3 bug which needs it too) and cleanup this as soon as possible...

Subsequent bringup of the interface works: # ifup vlan100 vlan100 up

ok. We'll put all this on the "verify list" when we start cleaning up to fix the bug mentioned above (soon!) and look what we can do here. Gruesse / Regards, Marius Tomaschewski , -- SUSE LINUX GmbH, GF: Felix Imendörffer, Jane Smithard, Dilip Upmanyu, Graham Norton, HRB 21284 (AG Nürnberg), Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: wicked-devel+unsubscribe@opensuse.org To contact the owner, e-mail: wicked-devel+owner@opensuse.org