What the #@(*$% is going on with SSH and my firewall?!?!?

older
RE: [SLE] Microsoft flushes out...

David Krider

11 May 2003 11 May '03

22:07

I have stared at this for HOURS now. I can log into my firewall with ssh. Repeatedly. I can even then scp a file over. But if I try to log into the server again after the scp, I can't. All of a sudden, the SuSEfirewall2 setup starts blocking SSH packets! WTF!? How can the firewall allow, allow, allow connections, then STOP allowing them? What in the bloody blue blazes is that SuSEfirewall2 script doing in the bowels of my system to suddenly start blocking SSH packets? As a bonus, if I wait, like, 5 minutes, it suddenly STARTS WORKING AGAIN! Here's one of the blocked packets: May 11 16:45:57 reliant kernel: SuSE-FW-ILLEGAL-TARGET IN=eth0 OUT= MAC=00:10:b5:0d:c3:0c:00:02:b3:03:68:05:08:00 SRC=192.168.4.200 DST=192.168.1.254 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=36723 DF PROTO=TCP SPT=34137 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A01390E8F0000000001030300) "ILLEGAL-TARGET"!? It was legal two seconds ago. I've tried turning on and off various options in /etc/sysconfig/SuSEfirewall2 rerunning `SuSEfirewall2', but nothing I do anymore fixes this. Not "protecting" my firewall from internal, or not NOT protecting it. Not the advanced "kernel" features. Nothing. I even tried downgrading my ssh to the version that came on the CD's (since it was just upgraded last night), but that didn't fix it either. It's like I'm running afoul of some rate-limiter, but I'll be dipped if I can find it in the setup script. I don't even see it in the debug output. PLEASE SOMEBODY HELP ME! I reran my old tried-and-true hand-written firewall script, and everything works just fine. I am just going out of my mind with this. I even wrote the guy who wrote the firewall script. I hardly EVER bother the source like that, but I am just stuck. As a recent convert, I want to use the "SuSE way," but to do that, apparently some kind SuSE expert is going to have to take pity on me. Regards, dk

Show replies by date

Thomas Jones

12 May 12 May

01:10

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Sunday 11 May 2003 17:07, David Krider wrote:

...

I have stared at this for HOURS now. I can log into my firewall with ssh. Repeatedly. I can even then scp a file over. But if I try to log into the server again after the scp, I can't. All of a sudden, the SuSEfirewall2 setup starts blocking SSH packets! WTF!? How can the firewall allow, allow, allow connections, then STOP allowing them? What in the bloody blue blazes is that SuSEfirewall2 script doing in the bowels of my system to suddenly start blocking SSH packets? As a bonus, if I wait, like, 5 minutes, it suddenly STARTS WORKING AGAIN!

Here's one of the blocked packets: May 11 16:45:57 reliant kernel: SuSE-FW-ILLEGAL-TARGET IN=eth0 OUT= MAC=00:10:b5:0d:c3:0c:00:02:b3:03:68:05:08:00 SRC=192.168.4.200 DST=192.168.1.254 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=36723 DF PROTO=TCP SPT=34137 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A01390E8F0000000001030300)

"ILLEGAL-TARGET"!? It was legal two seconds ago. I've tried turning on and off various options in /etc/sysconfig/SuSEfirewall2 rerunning `SuSEfirewall2', but nothing I do anymore fixes this. Not "protecting" my firewall from internal, or not NOT protecting it. Not the advanced "kernel" features. Nothing. I even tried downgrading my ssh to the version that came on the CD's (since it was just upgraded last night), but that didn't fix it either. It's like I'm running afoul of some rate-limiter, but I'll be dipped if I can find it in the setup script. I don't even see it in the debug output. PLEASE SOMEBODY HELP ME!

I reran my old tried-and-true hand-written firewall script, and everything works just fine. I am just going out of my mind with this. I even wrote the guy who wrote the firewall script. I hardly EVER bother the source like that, but I am just stuck. As a recent convert, I want to use the "SuSE way," but to do that, apparently some kind SuSE expert is going to have to take pity on me.

Regards, dk

Curiously, are you using dhcp? Do you have the following set: DHCLIENT_SET_HOSTNAME=YES in /etc/sysconfig/network/dhcp. I have seen others have problems with the same failure, but different scenarios. -- Thomas Jones Linux-Howtos Administrator

David Krider

02:46

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Sun, 2003-05-11 at 20:10, Thomas Jones wrote:

...

Curiously, are you using dhcp?

Do you have the following set:

DHCLIENT_SET_HOSTNAME=YES in /etc/sysconfig/network/dhcp. I have seen others have problems with the same failure, but different scenarios.

Nope. All 3 of my interfaces on the firewall are static, so I haven't fooled around with that file. I wish I had; at least it'd be something to try. Thanks, dk

Togan Muftuoglu

10:07

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

* David Krider; on 11 May, 2003 wrote:

...

in the bloody blue blazes is that SuSEfirewall2 script doing in the bowels of my system to suddenly start blocking SSH packets? As a bonus, if I wait, like, 5 minutes, it suddenly STARTS WORKING AGAIN!

weird

...

"ILLEGAL-TARGET"!? It was legal two seconds ago. I've tried turning on and off various options in /etc/sysconfig/SuSEfirewall2 rerunning `SuSEfirewall2', but nothing I do anymore fixes this. Not "protecting" my firewall from internal, or not NOT protecting it. Not the advanced "kernel" features. Nothing. I even tried downgrading my ssh to the

If you change the advanced kernel features ( ie from yes to no the only way to make them actually happen is a reboot as far as I know )

...

version that came on the CD's (since it was just upgraded last night), but that didn't fix it either. It's like I'm running afoul of some rate-limiter, but I'll be dipped if I can find it in the setup script. I

Do you have the HTB part configured in the firewall and/or do you have the wondershaper also set ? These are the two reagdring traffic shapping

...

don't even see it in the debug output. PLEASE SOMEBODY HELP ME!

Which SuSEfirewall2 are you running IIRC there was an update for 8.2. Also can you set post the firewall configuration -- Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

David Krider

12:07

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Mon, 2003-05-12 at 05:07, Togan Muftuoglu wrote:

...

If you change the advanced kernel features ( ie from yes to no the only way to make them actually happen is a reboot as far as I know )

Good thought. I rebooted to check this, but I still have the problem.

...

Do you have the HTB part configured in the firewall and/or do you have the wondershaper also set ? These are the two reagdring traffic shapping

Nope. Neither.

...

Which SuSEfirewall2 are you running IIRC there was an update for 8.2. Also can you set post the firewall configuration

I updated to the latest: SuSEfirewall2-3.1-95. And here's the config. (192.168.1.0 is my DMZ, 192.168.4.0 is my internal. There really ought to be a facility to encapsulate networks like there is the interfaces, but that's another kettle of fish.) FW_QUICKMODE="no" FW_DEV_EXT="eth2" FW_DEV_INT="eth0" FW_DEV_DMZ="eth1" FW_ROUTE="yes" FW_MASQUERADE="yes" FW_MASQ_DEV="$FW_DEV_EXT" FW_MASQ_NETS="0/0" FW_PROTECT_FROM_INTERNAL="yes" FW_AUTOPROTECT_SERVICES="yes" FW_SERVICES_EXT_TCP="ssh domain" FW_SERVICES_EXT_UDP="domain" FW_SERVICES_EXT_IP="" FW_SERVICES_DMZ_TCP="ssh domain" FW_SERVICES_DMZ_UDP="domain" FW_SERVICES_DMZ_IP="" FW_SERVICES_INT_TCP="ssh domain" FW_SERVICES_INT_UDP="domain" FW_SERVICES_INT_IP="" FW_SERVICES_QUICK_TCP="" FW_SERVICES_QUICK_UDP="" FW_SERVICES_QUICK_IP="" FW_TRUSTED_NETS="" FW_ALLOW_INCOMING_HIGHPORTS_TCP="" FW_ALLOW_INCOMING_HIGHPORTS_UDP="domain ntp" FW_SERVICE_AUTODETECT="yes" FW_SERVICE_DNS="yes" FW_SERVICE_DHCLIENT="no" FW_SERVICE_DHCPD="yes" FW_SERVICE_SQUID="no" FW_SERVICE_SAMBA="no" FW_FORWARD="192.168.4.0/24,192.168.1.1,tcp,25 192.168.4.0/24,192.168.1.1,tcp,993 192.168.4.0/24,192.168.1.1,tcp,465, 192.168.4.0/24,192.168.1.1,tcp,80 192.168.4.0/24,192.168.1.1,tcp,443 192.168.4.0/24,192.168.1.0/24,tcp,22 192.168.4.0/24,192.168.1.2,udp,2049 192.168.4.0/24,192.168.1.2,tcp,631" FW_FORWARD_MASQ="0/0,192.168.1.1,tcp,25 0/0,192.168.1.1,tcp,465 0/0,192.168.1.1,tcp,993 0/0,192.168.1.1,tcp,80 0/0,192.168.1.1,tcp,443" FW_REDIRECT="" FW_LOG_DROP_CRIT="yes" FW_LOG_DROP_ALL="yes" FW_LOG_ACCEPT_CRIT="no" FW_LOG_ACCEPT_ALL="no" FW_LOG="--log-level warning --log-tcp-options --log-ip-option --log-prefix SuSE-FW" FW_KERNEL_SECURITY="no" FW_STOP_KEEP_ROUTING_STATE="no" FW_ALLOW_PING_FW="yes" FW_ALLOW_PING_DMZ="no" FW_ALLOW_PING_EXT="no" FW_ALLOW_FW_TRACEROUTE="yes" FW_ALLOW_FW_SOURCEQUENCH="yes" FW_ALLOW_FW_BROADCAST="yes" FW_IGNORE_FW_BROADCAST="yes" FW_ALLOW_CLASS_ROUTING="no" FW_CUSTOMRULES="" FW_REJECT="no" FW_HTB_TUNE_DEV=""

...

Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

I really appreciate your response. The SuSEfirewall PDF has really helped me to understand what (apparently) little I do know about it. Thanks! dk

Togan Muftuoglu

13:00

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

* David Krider; on 12 May, 2003 wrote:

...

...
Which SuSEfirewall2 are you running IIRC there was an update for 8.2. Also can you set post the firewall configuration

I updated to the latest: SuSEfirewall2-3.1-95. And here's the config.

...

(192.168.1.0 is my DMZ, 192.168.4.0 is my internal. There really ought to be a facility to encapsulate networks like there is the interfaces, but that's another kettle of fish.)

...

FW_MASQ_NETS="0/0"

I would have placed the actual networks as 0/0 means the whole internet

...

FW_ALLOW_INCOMING_HIGHPORTS_UDP="domain ntp"

So you do not have xntp running on the firewall machine as I have not made xntp to work above UDP:1023- Yet other then these two non major things I do not see anything wrong with the setup. Hence it is even more weird than before. Have you tried to capture the traffic when this weird thing happens. Maybe it would give a clue

...

I really appreciate your response. The SuSEfirewall PDF has really helped me to understand what (apparently) little I do know about it.

Glad to hear it helped. The next version will include the new features that are available with the latest one yet I have not had time to finish it up as time is what I am lacking nowadays. -- Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

David Krider

15:53

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Mon, 2003-05-12 at 08:00, Togan Muftuoglu wrote:

...

...
FW_MASQ_NETS="0/0"

I would have placed the actual networks as 0/0 means the whole internet

I tried setting this like it "should" be set, with my internal and DMZ networks. It didn't make a difference.

...

...
FW_ALLOW_INCOMING_HIGHPORTS_UDP="domain ntp"

So you do not have xntp running on the firewall machine as I have not made xntp to work above UDP:1023-

You're right. I haven't even tried to get ntpd running. I was just setting that in anticipation of doing it.

...

Yet other then these two non major things I do not see anything wrong with the setup. Hence it is even more weird than before.

I suppose what might be most important thing I've discovered is that it doesn't do this on the external interface. It's only doing it on the internal and DMZ interfaces. (Nor does it happen between the internal network and the DMZ.) To sum up: this is only happening when I try to SSH into my firewall from my internal network or my DMZ.

...

Have you tried to capture the traffic when this weird thing happens. Maybe it would give a clue

I tried capturing the output from `iptables -L' when it was happening and when it wasn't, then diff'ing them, but the rules aren't changing on me (as expected). So I guess this is the only thing left. Regards, dk -- David "Dunkirk" Krider, http://www.davidkrider.com Acts 17:28, "For in Him we live, and move, and have our being." Linux: Will you use the power for good... or for AWESOME?

zentara

13:19

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On 12 May 2003 07:07:22 -0500 David Krider wrote:

...

I updated to the latest: SuSEfirewall2-3.1-95. And here's the config. (192.168.1.0 is my DMZ, 192.168.4.0 is my internal. There really ought to be a facility to encapsulate networks like there is the interfaces, but that's another kettle of fish.)

FW_QUICKMODE="no" FW_DEV_EXT="eth2"

This is a "long shot", but maybe try setting FW_QUICKMODE="yes" That may be a source of delay. I remember when I first setup FW2 on my 8.1 dialup box, there was about a 1 minute delay, before the firewall would allow traffic to pass. Changing QUICKMODE="yes" was the solution. -- use Perl; #powerful programmable prestidigitation

Togan Muftuoglu

13:34

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

* zentara; on 12 May, 2003 wrote:

...

On 12 May 2003 07:07:22 -0500 David Krider wrote:

...
I updated to the latest: SuSEfirewall2-3.1-95. And here's the config. (192.168.1.0 is my DMZ, 192.168.4.0 is my internal. There really ought to be a facility to encapsulate networks like there is the interfaces, but that's another kettle of fish.)

FW_QUICKMODE="no" FW_DEV_EXT="eth2"

This is a "long shot", but maybe try setting FW_QUICKMODE="yes" That may be a source of delay.

I do not think that is the case as FW_QUICKMODE is not what David is trying to achieve. FW_QUOCKMODE is a better personal_firewall that what it used to be yet it is not suitable for complex things like FW_FORWARD and FW_FORWARD_MASQ which are used by David. The problem lies somewhere else otherwise why should the firewall let first then hold and later on let again the same specific type of traffic -- Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

David Krider

14 May 14 May

00:54

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Mon, 2003-05-12 at 08:34, Togan Muftuoglu wrote:

...

The problem lies somewhere else otherwise why should the firewall let first then hold and later on let again the same specific type of traffic

Well, there's a little "egg on my face" now. It was my DNS server. See, my firewall, is, naturally, multihomed, but I only have one name server process running, and it resolves everything, for both the internal networks, and for the internet. I was getting random replies for the IP address attached to the name of the firewall, and it seems to respond with the "closer" address 3 times out of 4. So, I would get in 3 times, then get stuck. Anyway, adding a "sortlist 192.168.4.0" to my client's resolv.conf file fixed it. One of these days, I should really learn how to do a proper DNS server. Kudos to the person who wrote the SuSEfirewall2 script. Ultimately, I backtracked from `SuSEfirewall2 debug' and determined that the packets *had* to be getting dropped because they didn't fit into the rule that directed them to "input_int". That meant only one thing, and that led me immediately to the DNS problem. Now I just need to figure out how to get NFS working through the firewall... Thanks for the responses, and please forgive my stupidity, dk

Togan Muftuoglu

05:32

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

* David Krider; on 13 May, 2003 wrote:

...

Now I just need to figure out how to get NFS working through the firewall...

Well although I would not recommended having NFS on your firewall machine here are couple of things ( if you search the archieves you can find them in detail) that I am planning to have in the upcoming version of the unofficial SuSEfirewall2 document. All credit goes to the people how submitted the answers. <options> Option I. Under the following URL you find one description how to tunnel nfs in ssh. This is afaik the easiest approach to use nfs in a secure way (not only if you have a firewall, it might even be a good replacement for unencrypted nfs in your lan - as long as you are not 100% sure who has access to the network): http://www.math.ualberta.ca/imaging/snfs/ Option 2. What about tunnelling NFS-over-IP-over-PPP-over-SSH-through-the-firewall, as shown in http://www.jfranken.de/homepages/johannes/vortraege/ssh2.en.html#ToC12 As a nice side-effect, your nfs traffic would be compressed and enciphered, and ssh itself can easily tunnel through other protocols like https (see http://www.jfranken.de/homepages/johannes/vortraege/ssh3.en.html#ToC6 ). Option 3. use it from /etc/sysconfig/scripts/SuSEfirewall2-custom FW_ALLOW_NFS="" # These ports will be opened for access by the given host # (showmount -e seems to use tcp ports around 1200 damn... allow_nfs_ports_in() { echo " $1,tcp,111 $1,udp,111 $1,udp,2049 $1,udp,600:1399 $1,udp,2100:2499 " } if [ -n "$FW_ALLOW_NFS" -a "$FW_ALLOW_NFS" != no ]; then for host in $FW_ALLOW_NFS; do addnet=( `allow_nfs_ports_in $host` ) FW_TRUSTED_NETS="$FW_TRUSTED_NETS ${addnet[@]}" done echo "FW_TRUSTED_NETS=$FW_TRUSTED_NETS" fi Issues: It allows those ports on all interfaces, not just the one you want - if you only have one, fine. Those udp ports are a guess - security won't be much worse by just allowing 600:6000. If your mounts suddenlyhang (or the mount times out) check this. It doesn't allow for your MAC address checking. If you want finer control, you have to generate iptables rules yourself, at the correct point in the SuSEfirewall2 script. You'll probably find that you need to edit the script itself. </options> -- Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

David Krider

11:41

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

On Wed, 2003-05-14 at 00:32, Togan Muftuoglu wrote:

...

Option 3.

use it from /etc/sysconfig/scripts/SuSEfirewall2-custom

FW_ALLOW_NFS=""

# These ports will be opened for access by the given host # (showmount -e seems to use tcp ports around 1200 damn... allow_nfs_ports_in() { echo " $1,tcp,111 $1,udp,111 $1,udp,2049 $1,udp,600:1399 $1,udp,2100:2499 " }

if [ -n "$FW_ALLOW_NFS" -a "$FW_ALLOW_NFS" != no ]; then for host in $FW_ALLOW_NFS; do addnet=( `allow_nfs_ports_in $host` ) FW_TRUSTED_NETS="$FW_TRUSTED_NETS ${addnet[@]}" done echo "FW_TRUSTED_NETS=$FW_TRUSTED_NETS" fi

Issues: It allows those ports on all interfaces, not just the one you want - if you only have one, fine. Those udp ports are a guess - security won't be much worse by just allowing 600:6000. If your mounts suddenlyhang (or the mount times out) check this. It doesn't allow for your MAC address checking.

...

From what I'm seeing in my messages log file, NFS is acting very "behaved." My NFS server sits on the DMZ (because it used to host a Quake 3 server), and my firewall is a client. It would seem that no matter what happens, the NFS server sits on port 2049, and all clients always use 800. I've never seen NFS clients always use the same port

This is VERY interesting, and I may stick that in my script. What I have done so far is follow this advice: # Note that you can't use rpc requests (e.g. rpcinfo, showmount) as root # from a firewall using this script (well, you can if you include range # 600:1023 in FW_SERVICES_EXT_UDP ...). like that. I *like* that, don't get me wrong, but I don't understand why it's happening. The bottom line is that this (excerpt) is enough to make it work on my network: FW_FORWARD="0/0,0/0,udp,2049 0/0,0/0,udp,800" I suppose I might be a little nervous opening that up to ALL networks, but I didn't want to write 4 separate rules when the fact is that the firewall script is going to make sure that someone's going to have to be *on* the machine -- and not just touch those ports from external -- before they can access the service, and by then, I'm toast anyway. The weird thing is that if I take out the 600:1023 range from FW_SERVICES_EXT_UDP, it stops working, but it does NOT show any dropped packets in the messages log file (and I'm logging all dropped packets). So I don't understand the failure there. In fact, if I do a tcpdump, I can see packets going back and forth, so they are, in fact, not getting dropped, but it still doesn't work. Very strange. The other weird thing is that I can leave off the "0/0,0/0,udp,800" above and everything will still work. I will get messages in the log about those dropped packets, but everything acts fine. I have no idea what this means either. I suppose I should read a book about NFS. I hate NFS. It's the only thing I've seen that can consistently lock up a machine so hard as to need a reboot (because of misconfiguration, I'll admit). The same machine also has samba on it. Perhaps I should just be doing all of this over samba. It's much more resilient to lockups than NFS... Thanks once again! dk

Togan Muftuoglu

12 May 12 May

17:30

New subject: [SLE] What the #@(*$% is going on with SSH and my firewall?!?!?

* David Krider; on 12 May, 2003 wrote: On anotehr mail related to this topic you said " Nope. All 3 of my interfaces on the firewall are static, so I haven't fooled around with that file. I wish I had; at least it'd be something to try" And then when you send the config you have a server running

...

FW_SERVICE_DHCPD="yes"

and again on this thread you mentioned the following "I suppose what might be most important thing I've discovered is that it doesn't do this on the external interface. It's only doing it on the internal and DMZ interfaces. (Nor does it happen between the internal network and the DMZ.) To sum up: this is only happening when I try to SSH into my firewall from my internal network or my DMZ." Could there be something related have you checked the DHCPD server conmfiguration or does it happen when the server is trying to give an address to a lachine in your LAN etc. -- Togan Muftuoglu Unofficial SuSE FAQ Maintainer http://dinamizm.ath.cx

7661

Age (days ago)

7664

Last active (days ago)

List overview

Download

12 comments

4 participants

participants (4)

David Krider
Thomas Jones
Togan Muftuoglu
zentara