tcp mss clamping or ... ?
I use a 9k mtu on my 10gb network @ home -- only exists between my desktop & my server where it can be worthwhile. Bridged to that NW is a 1gb, 1.5k mtu network use for my home IOT (Internet of Things) This works for administrative traffic, for the most part, but in gaining some new 'things', some wish to talk to a media/file server on the higher BW net. Someone referred me to possibly using tcp mss clamping, though they were using it in the context of a VPN tunnel which I'm not using. I'm wondering if it is as simple as installing routes with mtu clamping so hosts wanting(needing) to goto a smaller mtu would take the appropriate network and talk with smaller packets. Seems there maybe something missing there since some traffic can be ip-level w/no tcp in play (ex: dns). Doesn't seem that would really work too well... Has anyone had any experience w/this type of setup? tnx!
Hi, Normally you would see ICMP "message too big" errors on the network. Sometimes those ICMP messages get blocked on a firewall and fail to reach their destination. Make sure that RELATED connections are allowed by any stateful firewalls in the network. I know TCP MSS can be set per route, but I usually stick to TCP MSS clamping via netfilter. https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.mtu-mss.html gives a good example, but use POSTROUTING instead of FORWARDING when using those rules on the endpoints. If all else fails, the --set-mss option can be used to test even lower values and stick a hard coded value on it to validate your assumptions. Kind regards, Erwin On Tue, Nov 29, 2022 at 1:47 AM L A Walsh <suse@tlinx.org> wrote:
I use a 9k mtu on my 10gb network @ home -- only exists between my desktop & my server where it can be worthwhile.
Bridged to that NW is a 1gb, 1.5k mtu network use for my home IOT (Internet of Things) This works for administrative traffic, for the most part, but in gaining some new 'things', some wish to talk to a media/file server on the higher BW net.
Someone referred me to possibly using tcp mss clamping, though they were using it in the context of a VPN tunnel which I'm not using.
I'm wondering if it is as simple as installing routes with mtu clamping so hosts wanting(needing) to goto a smaller mtu would take the appropriate network and talk with smaller packets. Seems there maybe something missing there since some traffic can be ip-level w/no tcp in play (ex: dns). Doesn't seem that would really work too well...
Has anyone had any experience w/this type of setup?
tnx!
On 2022-11-29 07:19, Erwin Van de Velde wrote:
Normally you would see ICMP "message too big" errors on the network. Sometimes those ICMP messages get blocked on a firewall and fail to reach their destination. Make sure that RELATED connections are allowed by any stateful firewalls in the network.
I know TCP MSS can be set per route, but I usually stick to TCP MSS clamping via netfilter. https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.mtu-mss.html gives a good example, but use POSTROUTING instead of FORWARDING when using those rules on the endpoints. If all else fails, the --set-mss option can be used to test even lower values and stick a hard coded value on it to validate your assumptions.
She's trying to mix different size MTUs with 9000 on her main LAN to the server, but 1500 for IoT. Mixing MTUs will likely cause problems.
I don't have an instance of a firewall or netfilter between local routes within my local network. I have, sorta, the opposite -- a 'bridge' to bind them to look like 1 local network, which pretty much ignores the mtu size. It seems inefficient thought it might become necessary to use a firewall between 2 internal networks just to manage the mtu sizes between the two networks, arg!, but it is looking like that may be the only way. On 2022/11/29 04:19, Erwin Van de Velde wrote:
Hi,
Normally you would see ICMP "message too big" errors on the network. Sometimes those ICMP messages get blocked on a firewall and fail to reach their destination. Make sure that RELATED connections are allowed by any stateful firewalls in the network.
I know TCP MSS can be set per route, but I usually stick to TCP MSS clamping via netfilter. https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.mtu-mss.html gives a good example, but use POSTROUTING instead of FORWARDING when using those rules on the endpoints. If all else fails, the --set-mss option can be used to test even lower values and stick a hard coded value on it to validate your assumptions.
Kind regards, Erwin
On Tue, Nov 29, 2022 at 1:47 AM L A Walsh <suse@tlinx.org <mailto:suse@tlinx.org>> wrote:
I use a 9k mtu on my 10gb network @ home -- only exists between my desktop & my server where it can be worthwhile.
Bridged to that NW is a 1gb, 1.5k mtu network use for my home IOT (Internet of Things) This works for administrative traffic, for the most part, but in gaining some new 'things', some wish to talk to a media/file server on the higher BW net.
Someone referred me to possibly using tcp mss clamping, though they were using it in the context of a VPN tunnel which I'm not using.
I'm wondering if it is as simple as installing routes with mtu clamping so hosts wanting(needing) to goto a smaller mtu would take the appropriate network and talk with smaller packets. Seems there maybe something missing there since some traffic can be ip-level w/no tcp in play (ex: dns). Doesn't seem that would really work too well...
Has anyone had any experience w/this type of setup?
tnx!
On 2022-11-29 15:09, L A Walsh wrote:
I don't have an instance of a firewall or netfilter between local routes within my local network. I have, sorta, the opposite -- a 'bridge' to bind them to look like 1 local network, which pretty much ignores the mtu size. What do you mean by "bridge"? A bridge is essentially a 2 port switch.
It seems inefficient thought it might become necessary to use a firewall between 2 internal networks just to manage the mtu sizes between the two networks, arg!, but it is looking like that may be the only way.
You don't need a full firewall, just a router, something that Linux does very well. For example, you could set up a VLAN for your IoT and have that VLAN on your main LAN connection to your computer and then enable routing. That's it. What are you using for your firewall/router? Linux or some commercial box?
On 2022-11-28 19:47, L A Walsh wrote:
I use a 9k mtu on my 10gb network @ home -- only exists between my desktop & my server where it can be worthwhile.
Bridged to that NW is a 1gb, 1.5k mtu network use for my home IOT (Internet of Things) This works for administrative traffic, for the most part, but in gaining some new 'things', some wish to talk to a media/file server on the higher BW net.
Someone referred me to possibly using tcp mss clamping, though they were using it in the context of a VPN tunnel which I'm not using.
I'm wondering if it is as simple as installing routes with mtu clamping so hosts wanting(needing) to goto a smaller mtu would take the appropriate network and talk with smaller packets. Seems there maybe something missing there since some traffic can be ip-level w/no tcp in play (ex: dns). Doesn't seem that would really work too well...
Are you talking about IPv4 or IPv6? Path MTU discovery is mandatory with IPv6, but IPv4 might still use fragmenting. Linux uses PMTUD for almost everything, IIRC, including UDP, but Windows uses PMTUD only for TCP. Regardless, IPv4 on either can fall back to fragmenting when PMTUD fails. With it, you should get an ICMP too big message coming back from whatever router is choking on that packet. However, I see you're bridging a network with a smaller MTU to your main network. That's a no no. Different size MTUs must be separated by a router, as a bridge (switch) has no way of dealing with an oversize packet. In fact, some switches can handle much larger packets than you're using.
On Mon, Nov 28, 2022 at 9:47 PM L A Walsh <suse@tlinx.org> wrote:
I use a 9k mtu on my 10gb network @ home -- only exists between my desktop & my server where it can be worthwhile.
Bridged to that NW is a 1gb, 1.5k mtu network use for my home IOT (Internet of Things) This works for administrative traffic, for the most part, but in gaining some new 'things', some wish to talk to a media/file server on the higher BW net.
Someone referred me to possibly using tcp mss clamping, though they were using it in the context of a VPN tunnel which I'm not using.
I'm wondering if it is as simple as installing routes with mtu clamping so hosts wanting(needing) to goto a smaller mtu would take the appropriate network and talk with smaller packets. Seems there maybe something missing there since some traffic can be ip-level w/no tcp in play (ex: dns). Doesn't seem that would really work too well...
Has anyone had any experience w/this type of setup?
tnx!
This is not the right place to post this, neither you state an actual, concrete issue with the facts of the matter. Unless you see many icmp errors there is nothing to fix or try. unless your 10g equipment is broken or wrongly configured, routers will fragment the packets accordingly and business will continue as usual.
On 2022-11-29 11:27, Cristian Rodríguez wrote:
unless your 10g equipment is broken or wrongly configured, routers will fragment the packets accordingly and business will continue as usual.
Actually, path MTU discovery is used these days. On IPv4 Linux uses it for everything and Windows for TCP. It's also mandatory for IPv6. https://en.wikipedia.org/wiki/Path_MTU_Discovery
participants (4)
-
Cristian Rodríguez -
Erwin Van de Velde -
James Knott -
L A Walsh