[opensuse-factory] 'wicked' refuses prior tun1 interface config: "wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0" ?
on boot of an upgrade 13.2 system, with a working tun1 interface config migrated from 13.1,
cat /etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' IPADDR="0.0.0.0"
the system boots to state
tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP POINTOPOINT NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
but I no longer have my usual tun1 connection to remote vpns.
checking interface toggle returns
ifdown tun1 tun1 device-down
ifup tun1 wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 wicked: device tun1 failed: operation timed out tun1 setup-in-progress
and
/var/log/messages Nov 10 13:22:24 edge03 wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 13:22:32 edge03 wicked: device tun1 failed: operation timed out Nov 10 13:22:39 edge03 wicked: device tun1 failed: operation timed out
"0.0.0.0" has been clearly appropriate prior to wicked/13.2.
How's this interface to be properly configured? Or, is this likely a bug?
On Monday 2014-11-10 22:29, grantksupport@operamail.com wrote:
on boot of an upgrade 13.2 system, with a working tun1 interface config migrated from 13.1,
cat /etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' IPADDR="0.0.0.0"
0.0.0.0 is not a valid address to be used. You probably should remove the line (even in case of non-wicked 13.1).
On Mon, Nov 10, 2014, at 01:43 PM, Jan Engelhardt wrote:
On Monday 2014-11-10 22:29, grantksupport@operamail.com wrote:
on boot of an upgrade 13.2 system, with a working tun1 interface config migrated from 13.1,
cat /etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' IPADDR="0.0.0.0"
0.0.0.0 is not a valid address to be used. You probably should remove the line (even in case of non-wicked 13.1).
0.0.0.0 is widely referenced, and widely used for initialization and config of tun interfaces by OS on system boot. On init of the VPN, app up-scripts assign the actual IP.
Can you provide a documentation reference for your claim that it's neither legitimate nor allowed use?
On Monday 2014-11-10 23:14, grantksupport@operamail.com wrote:
On Mon, Nov 10, 2014, at 01:43 PM, Jan Engelhardt wrote:
On Monday 2014-11-10 22:29, grantksupport@operamail.com wrote:
IPADDR="0.0.0.0"0.0.0.0 is not a valid address to be used. You probably should remove the line (even in case of non-wicked 13.1).
0.0.0.0 is widely referenced, and widely used for initialization and config of tun interfaces by OS on system boot. On init of the VPN, app up-scripts assign the actual IP.
Can you provide a documentation reference for your claim that it's neither legitimate nor allowed use?
"0.0.0.0 is a non-routable meta-address used to designate an invalid, unknown [...] target" (from the so-called experts at Wikipedia).
IPADDR= is generally for assigning regular addresses, of which 0.0.0.0 is not one, because of its special meaning.
In Linux and other systems, 0.0.0.0 is not assigned. You will never see it in `ip addr`. To specify "unset", the empty set rather than the 1-element set {0.0.0.0}, in both display and configuration files, because that makes a lot more sense.
The only place 0.0.0.0 is normally encountered is with fields of fixed size (i.e. an implementation detail of programs) and programs with bad user interface design which expose this fixedness, this "there must be exactly one address" thing (like the Windows network config dialog).
On 11/11/2014 01:29 AM, Jan Engelhardt wrote:
The only place 0.0.0.0 is normally encountered [...]
... and of course as the address for a listening port:
$ netstat -tln | grep :22 tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 :::22 :::* LISTEN
Some servers like e.g. tomcat have configuration setting with "0.0.0.0" to "listen on any interface". As I don't know wicked, this may not help the OP, unfortunately.
Have a nice day, Berny
On Mon, Nov 10, 2014, at 04:29 PM, Jan Engelhardt wrote:
In Linux and other systems, 0.0.0.0 is not assigned. You will never see it in `ip addr`. To specify "unset", the empty set rather than the 1-element set {0.0.0.0}, in both display and configuration files, because that makes a lot more sense.
Changing to unset
/etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' --- IPADDR="0.0.0.0" +++ IPADDR=''
does eliminate the error (Still, odd that 0.0.0.0's so widely referenced; it's been in use as the initial IP 'address' for pre-vpn-up-state on dozens of machines here, in use with no issues for years ...)
Nov 10 16:41:17 net03 systemd[1]: Starting wicked AutoIPv4 supplicant service... Nov 10 16:41:17 net03 systemd[1]: Starting wicked DHCPv6 supplicant service... Nov 10 16:41:17 net03 systemd[1]: Starting wicked DHCPv4 supplicant service... Nov 10 16:41:18 net03 systemd[1]: Started wicked AutoIPv4 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Started wicked DHCPv6 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Started wicked DHCPv4 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Starting wicked network management service daemon... Nov 10 16:41:18 net03 systemd[1]: Started wicked network management service daemon. Nov 10 16:41:18 net03 systemd[1]: Starting wicked network nanny service... Nov 10 16:41:19 net03 systemd[1]: Started wicked network nanny service. Nov 10 16:41:19 net03 systemd[1]: Starting wicked managed network interfaces... Nov 10 16:41:20 net03 wickedd[927]: lo: address 127.0.0.1 covered by a static lease Nov 10 16:41:20 net03 wickedd[927]: lo: address ::1 covered by a static lease Nov 10 16:41:34 net03 wicked[967]: device tun1 failed: operation timed out Nov 10 16:41:34 net03 wicked[967]: lo up Nov 10 16:41:34 net03 wicked[967]: eth0 up Nov 10 16:41:34 net03 wicked[967]: eth1 setup-in-progress Nov 10 16:41:34 net03 wicked[967]: tun1 setup-in-progress Nov 10 16:41:34 net03 systemd[1]: Started wicked managed network interfaces. Nov 10 16:41:56 net03 wicked[1653]: device tun1 failed: operation timed out
but reports the "device tun1 failed".
checking
wicked ifstatus tun1 tun1 setup-in-progress link: #7, state device-up, mtu 1500 type: tun config: compat:/etc/sysconfig/network/ifcfg-tun1
and
ifconfig tun1 tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP POINTOPOINT NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
but, if I (re)start openvpn as I always have pre 13.2,
systemctl restart openvpn-custom Job for openvpn-custom.service failed. See "systemctl status openvpn-custom.service" and "journalctl -xn" for details.
/var/log/messages Nov 10 16:50:30 net03 systemd[1]: Stopping OpenVPN Server... Nov 10 16:50:30 net03 systemd[1]: Starting OpenVPN Server... Nov 10 16:50:45 net03 up.script[3578]: wicked: device tun1 failed: operation timed out Nov 10 16:50:46 net03 up.script[3578]: tun1 setup-in-progress Nov 10 16:50:46 net03 systemd[1]: openvpn-custom.service: control process exited, code=exited status=162 Nov 10 16:50:46 net03 systemd[1]: Failed to start OpenVPN Server. Nov 10 16:50:46 net03 systemd[1]: Unit openvpn-custom.service entered failed state.
This used to work flawlessly prior to the 13.1 -> 13.2 upgrade. Something's changed as a result ... tun1 is not being seen :-/
On Tuesday 2014-11-11 01:54, grantksupport@operamail.com wrote:
Nov 10 16:41:34 net03 wicked[967]: device tun1 failed: operation timed out Nov 10 16:41:34 net03 wicked[967]: lo up Nov 10 16:41:34 net03 wicked[967]: eth0 up Nov 10 16:41:34 net03 wicked[967]: eth1 setup-in-progress Nov 10 16:41:34 net03 wicked[967]: tun1 setup-in-progress
This used to work flawlessly prior to the 13.1 -> 13.2 upgrade. Something's changed as a result ... tun1 is not being seen :-/
wicked presumes that all the interfaces exist. Which obviously is not true for tuns and taps. If networkd was not disabled in SUSE's systemd, I'd probably be trying to use that instead of wicked.
On Mon, Nov 10, 2014, at 05:07 PM, Jan Engelhardt wrote:
wicked presumes that all the interfaces exist. Which obviously is not true for tuns and taps.
If networkd was not disabled in SUSE's systemd, I'd probably be trying to use that instead of wicked.
That's news -- & a problem. What alternatives DO exist for use in this case, specifically for server use where the tunX needs to exist early?
wicked apparently won't work, and networkd is not available ...
grantksupport@operamail.com wrote:
On Mon, Nov 10, 2014, at 05:07 PM, Jan Engelhardt wrote:
wicked presumes that all the interfaces exist. Which obviously is not true for tuns and taps.
If networkd was not disabled in SUSE's systemd, I'd probably be trying to use that instead of wicked.
That's news -- & a problem. What alternatives DO exist for use in this case, specifically for server use where the tunX needs to exist early?
wicked apparently won't work, and networkd is not available ...
Please open a bugreport, others may want to track this issue too. I know I do.
On Mon, Nov 10, 2014, at 11:52 PM, Per Jessen wrote:
Please open a bugreport, others may want to track this issue too. I know I do.
fyi,
Bug 904903 - 13.2/wicked tun interface setup, using migrated/prior working 13.1 config(s), FAILs https://bugzilla.suse.com/show_bug.cgi?id=904903
On Tue, 11 Nov 2014 02:07:19 +0100 (CET) Jan Engelhardt jengelh@inai.de wrote:
On Tuesday 2014-11-11 01:54, grantksupport@operamail.com wrote:
Nov 10 16:41:34 net03 wicked[967]: device tun1 failed: operation timed out Nov 10 16:41:34 net03 wicked[967]: lo up Nov 10 16:41:34 net03 wicked[967]: eth0 up Nov 10 16:41:34 net03 wicked[967]: eth1 setup-in-progress Nov 10 16:41:34 net03 wicked[967]: tun1 setup-in-progress
This used to work flawlessly prior to the 13.1 -> 13.2 upgrade. Something's changed as a result ... tun1 is not being seen :-/
wicked presumes that all the interfaces exist. Which obviously is not true for tuns and taps.
This is wrong. What makes you think it does? Wicked is supposed to create virtual interfaces, but at the moment it does not start openvpn and thus there could be no link on the interface (final state devcie-up) and setup-in-progress is reported.
If networkd was not disabled in SUSE's systemd, I'd probably be trying to use that instead of wicked.
Good luck with that beast ;-).
On Mon, 10 Nov 2014 16:54:08 -0800 grantksupport@operamail.com wrote:
On Mon, Nov 10, 2014, at 04:29 PM, Jan Engelhardt wrote:
In Linux and other systems, 0.0.0.0 is not assigned. You will never see it in `ip addr`. To specify "unset", the empty set rather than the 1-element set {0.0.0.0}, in both display and configuration files, because that makes a lot more sense.
Changing to unset
/etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' --- IPADDR="0.0.0.0" +++ IPADDR=''
does eliminate the error (Still, odd that 0.0.0.0's so widely referenced; it's been in use as the initial IP 'address' for pre-vpn-up-state on dozens of machines here, in use with no issues for years ...)
Nov 10 16:41:17 net03 systemd[1]: Starting wicked AutoIPv4 supplicant service... Nov 10 16:41:17 net03 systemd[1]: Starting wicked DHCPv6 supplicant service... Nov 10 16:41:17 net03 systemd[1]: Starting wicked DHCPv4 supplicant service... Nov 10 16:41:18 net03 systemd[1]: Started wicked AutoIPv4 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Started wicked DHCPv6 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Started wicked DHCPv4 supplicant service. Nov 10 16:41:18 net03 systemd[1]: Starting wicked network management service daemon... Nov 10 16:41:18 net03 systemd[1]: Started wicked network management service daemon. Nov 10 16:41:18 net03 systemd[1]: Starting wicked network nanny service... Nov 10 16:41:19 net03 systemd[1]: Started wicked network nanny service. Nov 10 16:41:19 net03 systemd[1]: Starting wicked managed network interfaces... Nov 10 16:41:20 net03 wickedd[927]: lo: address 127.0.0.1 covered by a static lease Nov 10 16:41:20 net03 wickedd[927]: lo: address ::1 covered by a static lease Nov 10 16:41:34 net03 wicked[967]: device tun1 failed: operation timed out Nov 10 16:41:34 net03 wicked[967]: lo up Nov 10 16:41:34 net03 wicked[967]: eth0 up Nov 10 16:41:34 net03 wicked[967]: eth1 setup-in-progress Nov 10 16:41:34 net03 wicked[967]: tun1 setup-in-progress Nov 10 16:41:34 net03 systemd[1]: Started wicked managed network interfaces. Nov 10 16:41:56 net03 wicked[1653]: device tun1 failed: operation timed out
but reports the "device tun1 failed".
checking
wicked ifstatus tun1 tun1 setup-in-progress link: #7, state device-up, mtu 1500 type: tun config: compat:/etc/sysconfig/network/ifcfg-tun1
and
ifconfig tun1 tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP POINTOPOINT NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
but, if I (re)start openvpn as I always have pre 13.2,
systemctl restart openvpn-custom Job for openvpn-custom.service failed. See "systemctl status openvpn-custom.service" and "journalctl -xn" for details.
/var/log/messages Nov 10 16:50:30 net03 systemd[1]: Stopping OpenVPN Server... Nov 10 16:50:30 net03 systemd[1]: Starting OpenVPN Server... Nov 10 16:50:45 net03 up.script[3578]: wicked: device tun1 failed: operation timed out Nov 10 16:50:46 net03 up.script[3578]: tun1 setup-in-progress Nov 10 16:50:46 net03 systemd[1]: openvpn-custom.service: control process exited, code=exited status=162 Nov 10 16:50:46 net03 systemd[1]: Failed to start OpenVPN Server. Nov 10 16:50:46 net03 systemd[1]: Unit openvpn-custom.service entered failed state.
This used to work flawlessly prior to the 13.1 -> 13.2 upgrade. Something's changed as a result ... tun1 is not being seen :-/
The upper problem appears because on the openvpn service call you some extra ifup is being called trying to brought up tun1 device and then to start openvpn daemon to handle the fd of the tun and establish the link, connection etc. This does not happen because wicked is unable to finish the configuration before openvpn daemon is started. There is not link on the tunnel, and wicked reports setup-in-progress (162) status code. This status code is rejected by systemd failing with openvpn start up.
But...
You do not need explicit ifcfg-tun* files and starting ifup scripts in order to set up openvpn tunnel. All you need is to call systemctl openvpn@XXX.service
Please open an enhancement bug so we could improve that.
Thanks.
On Mon, Nov 10, 2014 at 04:54:08PM -0800, grantksupport@operamail.com wrote:
Still, odd that 0.0.0.0's so widely referenced; it's been in use as the initial IP 'address' for pre-vpn-up-state on dozens of machines here, in use with no issues for years ...
IMHO it's a relict of pre-1999 times when interface had one IPv4 address which had to be set to something so that setting it to 0.0.0.0 was the way to set "no address". Since kernel 2.2, interface has a list of addresses and this list can be empty. But some old habits die really hard.
Michal Kubeček
On 11/13/2014 05:31 AM, Michal Kubecek wrote:
IMHO it's a relict of pre-1999 times when interface had one IPv4 address which had to be set to something so that setting it to 0.0.0.0 was the way to set "no address". Since kernel 2.2, interface has a list of addresses and this list can be empty. But some old habits die really hard.
It's still part of the DHCP spec. A device requesting an address uses it as the source address, until it's been assigned one.
"UDP https://en.wikipedia.org/wiki/User_Datagram_Protocol Src=0.0.0.0 sPort=68"
https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol#DHCP_disco...
On 11/13/2014 07:46 AM, James Knott wrote:
"UDP https://en.wikipedia.org/wiki/User_Datagram_Protocol Src=0.0.0.0 sPort=68"
I guess I should have used "Paste Without Formating" on that line.
UDP Src=0.0.0.0 sPort=68
On Thursday 13 of November 2014 07:46:15 James Knott wrote:
On 11/13/2014 05:31 AM, Michal Kubecek wrote:
IMHO it's a relict of pre-1999 times when interface had one IPv4 address which had to be set to something so that setting it to 0.0.0.0 was the way to set "no address". Since kernel 2.2, interface has a list of addresses and this list can be empty. But some old habits die really hard.
It's still part of the DHCP spec. A device requesting an address uses it as the source address, until it's been assigned one.
That's something completely different from what I was talking about. Using zero address as source address of a specific packet is something else than setting interface address to zero (or pretending to).
Michal Kubeček
On 11/10/2014 04:43 PM, Jan Engelhardt wrote:
0.0.0.0 is not a valid address to be used.
Actually, it's the address a device uses when it sends a DHCP request. Since it doesn't yet have an assigned address, it uses 0.0.0.0.
On Mon, 10 Nov 2014 22:43:09 +0100 (CET) Jan Engelhardt jengelh@inai.de wrote:
On Monday 2014-11-10 22:29, grantksupport@operamail.com wrote:
on boot of an upgrade 13.2 system, with a working tun1 interface config migrated from 13.1,
cat /etc/sysconfig/network/ifcfg-tun1 BOOTPROTO='static' STARTMODE='auto' TUNNEL='tun' TUNNEL_SET_GROUP='openvpn' TUNNEL_SET_OWNER='openvpn' TUNNEL_SET_PERSISTENT='yes' IPV6INIT='no' IPADDR="0.0.0.0"
0.0.0.0 is not a valid address to be used. You probably should remove the line (even in case of non-wicked 13.1).
There was also a bug with proper setting TUNNEL_SET_GROUP (http://bugzilla.opensuse.org/show_bug.cgi?id=904380). Fixed in current master you can get from repo: https://build.opensuse.org/package/show/network:wicked:master/wicked
HTH
On Mon, Nov 10, 2014, at 02:17 PM, Pawel Wieczorkiewicz wrote:
There was also a bug with proper setting TUNNEL_SET_GROUP (http://bugzilla.opensuse.org/show_bug.cgi?id=904380). Fixed in current master you can get from repo: https://build.opensuse.org/package/show/network:wicked:master/wicked
I'll take a look, thanks.
Could network:wicked:master repo/builds be enabled for 13.2 ? Atm, it's enabled for everything _but_ ...
THanks.
On Mon, Nov 10, 2014, at 02:17 PM, Pawel Wieczorkiewicz wrote: fixed in current master you can get from repo: https://build.opensuse.org/package/show/network:wicked:master/wicked
HTH
unfortunately, no.
upgraded to network:wicked:master's
after reboot, still no tun1 connectivity
checking boot logs,
journalctl -xb | grep -i wick Nov 10 15:45:49 net03 systemd[1]: Starting wicked AutoIPv4 supplicant service... Nov 10 15:45:49 net03 systemd[1]: Starting wicked DHCPv6 supplicant service... Nov 10 15:45:49 net03 systemd[1]: Starting wicked DHCPv4 supplicant service... Nov 10 15:45:49 net03 systemd[1]: Started wicked DHCPv6 supplicant service. Nov 10 15:45:50 net03 systemd[1]: Started wicked AutoIPv4 supplicant service. Nov 10 15:45:50 net03 systemd[1]: Started wicked DHCPv4 supplicant service. Nov 10 15:45:50 net03 systemd[1]: Starting wicked network management service daemon... Nov 10 15:45:50 net03 systemd[1]: Started wicked network management service daemon. Nov 10 15:45:50 net03 systemd[1]: Starting wicked network nanny service... Nov 10 15:45:50 net03 systemd[1]: Started wicked network nanny service. Nov 10 15:45:50 net03 systemd[1]: Starting wicked managed network interfaces... Nov 10 15:45:50 net03 wicked[939]: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:45:51 net03 wickedd[915]: lo: address 127.0.0.1 covered by a static lease Nov 10 15:45:51 net03 wickedd[915]: lo: address ::1 covered by a static lease Nov 10 15:46:05 net03 wicked[939]: device tun1 failed: operation timed out Nov 10 15:46:05 net03 wicked[939]: lo up Nov 10 15:46:05 net03 wicked[939]: eth0 up Nov 10 15:46:05 net03 wicked[939]: eth1 setup-in-progress Nov 10 15:46:05 net03 wicked[939]: tun1 setup-in-progress Nov 10 15:46:05 net03 systemd[1]: Started wicked managed network interfaces. Nov 10 15:46:13 net03 up.script[1633]: wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:46:13 net03 wicked[1640]: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:46:28 net03 up.script[1633]: wicked: device tun1 failed: operation timed out Nov 10 15:46:28 net03 wicked[1640]: device tun1 failed: operation timed out Nov 10 15:46:58 net03 up.script[2066]: wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:46:58 net03 wicked[2073]: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:47:13 net03 wicked[2073]: device tun1 failed: operation timed out Nov 10 15:47:13 net03 up.script[2066]: wicked: device tun1 failed: operation timed out Nov 10 15:47:44 net03 up.script[3208]: wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 Nov 10 15:47:44 net03 wicked[3215]: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0
and at console, still,
ifdown tun1 tun1 device-down ifup tun1 wicked: ifcfg-tun1: ignoring unspecified ip address 0.0.0.0 wicked: device tun1 failed: operation timed out
-
Bernhard Voelker -
grantksupport＠operamail.com -
James Knott -
Jan Engelhardt -
Michal Kubecek -
Pawel Wieczorkiewicz -
Per Jessen