[opensuse] iptables MASQUERADE problems with oss 11.2
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2. Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is: $IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces). This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working? Thanks in advance for any help, -- --Moby They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, Dec 15, 2009 at 07:29:00PM -0600, Moby wrote:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
I have virtually the same problem. First, where do you put the scrript becasue is it almost impossible to drop into /etc/init.d/network Secondly, port forwarding has to be ON and the system, even when it says it is on, doesn't have it on. #/bin/sh PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome2/bin:/opt/gnome/bin:/opt/kde3/bin:/opt/kde2/bin:/usr/openwin/bin:/usr/lib/java/bin:/opt/gnome/bin /usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE echo 1 > /proc/sys/net/ipv4/ip_forward echo "Forwarding Enabled"
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
-- --Moby
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-- http://www.mrbrklyn.com - Interesting Stuff http://www.nylxs.com - Leadership Development in Free Software So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://fairuse.nylxs.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 "Yeah - I write Free Software...so SUE ME" "The tremendous problem we face is that we are becoming sharecroppers to our own cultural heritage -- we need the ability to participate in our own society." "> I'm an engineer. I choose the best tool for the job, politics be damned.< You must be a stupid engineer then, because politcs and technology have been attached at the hip since the 1st dynasty in Ancient Egypt. I guess you missed that one." © Copyright for the Digital Millennium -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 12/15/2009 10:09 PM, Ruben Safir wrote:
On Tue, Dec 15, 2009 at 07:29:00PM -0600, Moby wrote:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
I have virtually the same problem. First, where do you put the scrript becasue is it almost impossible to drop into /etc/init.d/network
Secondly, port forwarding has to be ON and the system, even when it says it is on, doesn't have it on.
#/bin/sh
PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome2/bin:/opt/gnome/bin:/opt/kde3/bin:/opt/kde2/bin:/usr/openwin/bin:/usr/lib/java/bin:/opt/gnome/bin
/usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE echo 1 > /proc/sys/net/ipv4/ip_forward
echo "Forwarding Enabled"
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
-- --Moby
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks for the response Rubin. I created my own init script that calls /etc/rc.local (one of the things I like in RH!). I call whatever I need executed at boot-time from this /etc/rc.local. I can send that init script to you if you like. Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference. Do you still have the problem or have you resolved yours? The fact that this very same script has been working fine until the upgrade to 11.2 makes me thing something in 11.2 is breaking it - I just wish I could find out what. Thanks again. -- --Moby They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, Dec 15, 2009 at 10:50:36PM -0600, Moby wrote:
On 12/15/2009 10:09 PM, Ruben Safir wrote:
On Tue, Dec 15, 2009 at 07:29:00PM -0600, Moby wrote:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
I have virtually the same problem. First, where do you put the scrript becasue is it almost impossible to drop into /etc/init.d/network
Secondly, port forwarding has to be ON and the system, even when it says it is on, doesn't have it on.
#/bin/sh
PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome2/bin:/opt/gnome/bin:/opt/kde3/bin:/opt/kde2/bin:/usr/openwin/bin:/usr/lib/java/bin:/opt/gnome/bin
/usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE echo 1 > /proc/sys/net/ipv4/ip_forward
echo "Forwarding Enabled"
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
After i ran this, everything was fine echo 1 > /proc/sys/net/ipv4/ip_forward Of course, routing has to be correct. Ruben
Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference.
Do you still have the problem or have you resolved yours? The fact that this very same script has been working fine until the upgrade to 11.2 makes me thing something in 11.2 is breaking it - I just wish I could find out what.
Thanks again.
-- http://www.mrbrklyn.com - Interesting Stuff http://www.nylxs.com - Leadership Development in Free Software So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://fairuse.nylxs.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 "Yeah - I write Free Software...so SUE ME" "The tremendous problem we face is that we are becoming sharecroppers to our own cultural heritage -- we need the ability to participate in our own society." "> I'm an engineer. I choose the best tool for the job, politics be damned.< You must be a stupid engineer then, because politcs and technology have been attached at the hip since the 1st dynasty in Ancient Egypt. I guess you missed that one." © Copyright for the Digital Millennium -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Thanks for the response Rubin.
I created my own init script that calls /etc/rc.local (one of the things I like in RH!). I call whatever I need executed at boot-time from this /etc/rc.local. I can send that init script to you if you like.
Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference.
Do you still have the problem or have you resolved yours? The fact that this very same script has been working fine until the upgrade to 11.2 makes me thing something in 11.2 is breaking it - I just wish I could find out what.
Turn off the firewall...did I mention that? Ruben
Thanks again.
-- --Moby
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-- http://www.mrbrklyn.com - Interesting Stuff http://www.nylxs.com - Leadership Development in Free Software So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://fairuse.nylxs.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 "Yeah - I write Free Software...so SUE ME" "The tremendous problem we face is that we are becoming sharecroppers to our own cultural heritage -- we need the ability to participate in our own society." "> I'm an engineer. I choose the best tool for the job, politics be damned.< You must be a stupid engineer then, because politcs and technology have been attached at the hip since the 1st dynasty in Ancient Egypt. I guess you missed that one." © Copyright for the Digital Millennium -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Moby wrote:
Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference.
Uh, ip_forward, that is "/proc/sys/net/ipv4/ip_forward", has got nothing to do with port forwarding. The ip_forward setting is about forwarding of packets from one network to another. /Per -- Per Jessen, Zürich (0.0°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 12/16/2009 01:59 AM, Per Jessen wrote:
Moby wrote:
Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference.
Uh, ip_forward, that is "/proc/sys/net/ipv4/ip_forward", has got nothing to do with port forwarding. The ip_forward setting is about forwarding of packets from one network to another.
/Per
My fault, I meant to say packet forwarding was indeed on, not port forwarding. Built-in SuSE firewall stuff is all off. Any other hints or ideas of diagnostic steps? -- --Moby They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Moby wrote:
On 12/16/2009 01:59 AM, Per Jessen wrote:
Moby wrote:
Port forwarding is indeed on - I verified both by cat'ing ip_forward, running sysctl, and for good measure I echoed 1 into ip_forward. That makes no difference.
Uh, ip_forward, that is "/proc/sys/net/ipv4/ip_forward", has got nothing to do with port forwarding. The ip_forward setting is about forwarding of packets from one network to another.
/Per
My fault, I meant to say packet forwarding was indeed on, not port forwarding. Built-in SuSE firewall stuff is all off. Any other hints or ideas of diagnostic steps?
No, I can't really think of what to try next. I don't use the MASQUERADE myself, I use DNAT'ing, but that shouldn't make a big difference. /Per -- Per Jessen, Zürich (0.0°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
2009/12/16 Moby <moby@mobsternet.com>:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
-- --Moby
What you see in your /var/log/firewall? (check "yast2 firewall" that "Log critical packets"... is enabled). -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 12/16/2009 09:20 AM, Alexander R wrote:
2009/12/16 Moby <moby@mobsternet.com>:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
-- --Moby
What you see in your /var/log/firewall? (check "yast2 firewall" that "Log critical packets"... is enabled).
After much more googling and oogling, it appears that I am running into a 2.6.31 kernel bug as described at http://www.linuxforums.org/forum/linux-security/155418-2-6-31-iptables-ignor... and http://bugs.gentoo.org/show_bug.cgi?id=295934 So it appears the issue is not so much with masquerade as with fwmark based routing. I plan on running more diagnostics when I am physically at the machine and will post the results. --Moby They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 12/16/2009 04:12 PM, Moby wrote:
On 12/16/2009 09:20 AM, Alexander R wrote:
2009/12/16 Moby <moby@mobsternet.com>:
The setup involved in this issue was working perfectly fine with oss 10.X and 11.1. The problem arose after an in-place upgrade to oss 11.2.
Basically the machine (call it machine A with internal IP address 192.168.10.10) is configured to do masquerade via iptables (with a custom script). The relevant line is:
$IPTABLES -t mangle -A PREROUTING -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t mangle -A FORWARD -m iprange --src-range 192.168.10.50-192.168.10.200 -j MARK --set-mark 2 $IPTABLES -t nat -A POSTROUTING -m mark --mark 2 -o comcast1 -j MASQUERADE
Once this is in place, initiating (for example) ping www.yahoo.com from a machine with ip address 192.168.10.52 shows that traffic is making it's way to machine A on A's internal interface, then the traffic is getting "masqueraded" (packets source ip gets changed to the valid public address of machine A) and sent out the interface called comcast1, reply comes back from www.yahoo.com and enters machine A properly on interface comcast1 and that's it ... the packet never gets mapped back to the inside host that sent it. I see this sequence of events in a packet capture btw (Wireshark with capture on pseudo-interface to capture all interfaces).
This exact same script worked fine with oss11.1 and earlier versions - is anyone here aware of any changes in oss11.2 that could prevent something like this from working?
Thanks in advance for any help,
-- --Moby
What you see in your /var/log/firewall? (check "yast2 firewall" that "Log critical packets"... is enabled).
After much more googling and oogling, it appears that I am running into a 2.6.31 kernel bug as described at http://www.linuxforums.org/forum/linux-security/155418-2-6-31-iptables-ignor... and http://bugs.gentoo.org/show_bug.cgi?id=295934 So it appears the issue is not so much with masquerade as with fwmark based routing. I plan on running more diagnostics when I am physically at the machine and will post the results.
The issue is resolved. The problem was indeed with policy based routing (marking packets with iptables, then using ip rule to send differently marked packets through different routing tables). Apparently the role or interpretation of /proc/sys/net/ipv4/conf/x/rp_filter file (where x=all, default, or any interface name) has changed in a rather significant way in kernels 2.6.31 and higher. Echoing a 0 into this said file resolved the issue. Thanks to everyone for their help. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (4)
-
Alexander R -
Moby -
Per Jessen -
Ruben Safir