12.07.2019 21:10, Andrei Borzenkov пишет:
12.07.2019 6:56, Andrei Borzenkov пишет:
11.07.2019 15:53, Per Jessen пишет:
Andrei, you wouldn't happen to know how to look at the hash value used? Afaict, it doesn't change - if I do e.g. 10 or 20 new sessions one after one, they all pick the same route.
Displaying hash itself which is return value of kernel function is relatively easy - function return is well defined trace point. Displaying its arguments or internal variables seems to be more complicated and requires real programming in whatever trace language is used.
...
Here "telnet" is called twice in different terminals with the same destination address. I am not sure where two other calls to hash come from, may be some internal library resolver or similar.
This happens in kernel, kernel calls routing code several times when application did not bind socket to local address (it needs to compute outgoing device to select local address to use - chicken and egg).
I did become curious and tried to capture input to hash function.
bor@tw:~> sudo ./fibhashsnoop PID COMM SADDR DADDR HASH 3882 telnet 00000000:0 4a7d8363:23 20767294 3882 telnet 0a00020f:0 4a7d8363:23 736f1ef2 3882 telnet 0a00020f:56546 4a7d8363:23 396252f9 3887 telnet 00000000:0 4a7d8363:23 20767294 3887 telnet 0a00020f:0 4a7d8363:23 736f1ef2 3887 telnet 0a00020f:56548 4a7d8363:23 34d6e578 3892 telnet 00000000:0 4a7d8363:23 20767294 3892 telnet 0a00020f:0 4a7d8363:23 736f1ef2 3892 telnet 0a00020f:56550 4a7d8363:23 7d96dbcf 3901 telnet 00000000:0 4a7d8363:23 20767294 3901 telnet 0a00020f:0 4a7d8363:23 736f1ef2 3901 telnet 0a00020f:56552 4a7d8363:23 783155e1 3906 telnet 00000000:0 4a7d8363:23 20767294 3906 telnet 0a00020f:0 4a7d8363:23 736f1ef2 3906 telnet 0a00020f:56554 4a7d8363:23 33cb2d99 ^Cbor@tw:~>
All numbers are hex (I am lazy). So it appears kernel calls routing code several times, assuming the last one (with actual port numbers) determines the final route - two consecutive invocations select the same gateway. So in the worst case you may have bad luck.
Max hash value ix 0x7ffffff, with equal weights each gateway gets half of all values, i.e. hash below 0x3ffffff selects one gateway, above - another gateway. So in the worst case all your traffic is indeed hashed to one gateway only.
It seems to work for me - kernel actually attempts to send traffic via different gateways and it correlates with hash value. tw:/home/bor # ./fibhashsnoop PID COMM SADDR DADDR HASH 4812 telnet 0a00020f:60414 4a7d8363:23 38aa371d 4817 telnet 0a00020f:60416 4a7d8363:23 465d2b13 4822 telnet 0a00020f:60418 4a7d8363:23 4f222ac6 4827 telnet 0a00020f:60420 4a7d8363:23 ff99190 4832 telnet 0a00020f:60422 4a7d8363:23 145c021e ^Ctw:/home/bor # tw:/usr/share/bcc/tools # tshark -i enp0s4 -o column.format:'"No.","%m","Time","%t","Source","%s","Destination","%d","Protocol","%p","Info","%i","src","%uhs","dst","%uhd"' Running as user "root" and group "root". This could be dangerous. ** (process:4792): WARNING **: 10:01:55.457: Preference "column.format" has been converted to "(null).format" Save your preferences to make this change permanent. Capturing on 'enp0s4' 1 0.000000000 10.0.2.15 → 74.125.131.99 TCP 60414 → 23 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1605324308 TSecr=0 WS=128 52:54:00:12:34:56 → 52:55:0a:00:02:02 2 1.502272623 10.0.2.15 → 74.125.131.99 TCP 60416 → 23 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1605325810 TSecr=0 WS=128 52:54:00:12:34:56 → 52:55:0a:00:02:03 3 2.960769688 10.0.2.15 → 74.125.131.99 TCP 60418 → 23 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1605327269 TSecr=0 WS=128 52:54:00:12:34:56 → 52:55:0a:00:02:03 4 4.473304376 10.0.2.15 → 74.125.131.99 TCP 60420 → 23 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1605328781 TSecr=0 WS=128 52:54:00:12:34:56 → 52:55:0a:00:02:02 7 5.851979498 10.0.2.15 → 74.125.131.99 TCP 60422 → 23 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=1605330160 TSecr=0 WS=128 52:54:00:12:34:56 → 52:55:0a:00:02:02 ^C10 packets captured tw:/usr/share/bcc/tools # As you see connections go either to gateway with MAC 52:55:0a:00:02:02 or 52:55:0a:00:02:03 according to hash value. Disclaimer - this is TW, so may be Leap kernel issue.
Anyway, resulting hashes seem to be always the same. I am afraid I'm not as deep into kernel routing code to make any useful comment here.
Still true :) In particular I have no idea why routing decision is apparently performed three times.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org