fail2ban stopped working?
Hi list, I just noticed that since my latest update 3 days ago fail2ban is no longer working! It properly runs, but doesn't find any failed logins, so the block list of sshd is empty, and the other jails (I have a staged fbloop setup with increasing bantime) only perform unbans. The update was to 20240829, IIRC that was also the first one using kernel 6.10 - could that be related? Or was it some systemd update? Anyone else noticed this?
On 9/3/24 6:52 AM, Pit Suetterlin via openSUSE Users wrote:
The update was to 20240829, IIRC that was also the first one using kernel 6.10 - could that be related? Or was it some systemd update?
Anyone else noticed this?
Well, no, but I would suspect python before any of the others. Here's why. I have ipset, fail2ban and iptables running on Arch with systemd 256 and Linux 6.10.7 kernel and haven't had any fail2ban issues (other than getting the notification e-mails to include the log lines from the systemd journal -- use matches instead of lines -- or provide a custom action= for each jail) No hiccups at all across all 6.10 kernels. Does a manual ban work from the command line, any errors? # fail2ban-client set <jail-name> banip <ip-address> -- David C. Rankin, J.D.,P.E.
David C. Rankin wrote:
On 9/3/24 6:52 AM, Pit Suetterlin via openSUSE Users wrote:
The update was to 20240829, IIRC that was also the first one using kernel 6.10 - could that be related? Or was it some systemd update?
Anyone else noticed this?
Well, no, but I would suspect python before any of the others.
There was no change in python though, I think :o
Here's why. I have ipset, fail2ban and iptables running on Arch with systemd 256 and Linux 6.10.7 kernel and haven't had any fail2ban issues (other than getting the notification e-mails to include the log lines from the systemd journal -- use matches instead of lines -- or provide a custom action= for each jail)
Interesting, Which sub-version of systemd is it? The update I mentioned did upgrade 256.4->256.5
No hiccups at all across all 6.10 kernels.
Does a manual ban work from the command line, any errors?
# fail2ban-client set <jail-name> banip <ip-address>
Yes, it does, and fail2ban-client banned then shows the IP in the sshd section (which is otherwise empty). And it gets unbanned after the bantime expires. So the whole mechanism works properly, just nothing is triggering it anymore. The long-banned IPs in f2b-loop3 (bantime 1 week) are still there, f2b-loop (bantime 1 day) is empty, everything in there has been unbanned in the meantime The thing is, I usually had entries in /var/log/fail2ban.log that listed detected fails: 2024-08-31 18:51:20,585 fail2ban.filter [1791]: INFO [sshd] Found 164.132.56.147 - 2024-08-31 18:51:20 None of those appear anymore, although there's permanent login attempts. That's why I assumed there was some change either in log syntax or access method (though that would probably cause an error?) Hmm, what I just see is that also openssh was updated to 9.8p1 .....
-- David C. Rankin, J.D.,P.E.
-- Dr. Peter "Pit" Suetterlin http://www.isf.astro.su.se/~pit Institute for Solar Physics Tel.: +34 922 405 590 (Spain) P.Suetterlin@sst.iac.es +46 8 5537 8559 (Sweden) Peter.Suetterlin@astro.su.se
Pit Suetterlin via openSUSE Users wrote:
Hmm, what I just see is that also openssh was updated to 9.8p1 .....
Bingo: https://linuxthings.co.uk/blog/fail2ban-stopped-working-with-openssh-9-8
On 9/3/24 4:16 PM, Pit Suetterlin via openSUSE Users wrote:
Pit Suetterlin via openSUSE Users wrote:
Hmm, what I just see is that also openssh was updated to 9.8p1 .....
Bingo: https://linuxthings.co.uk/blog/fail2ban-stopped-working-with-openssh-9-8
Good catch, Yes, Arch has openssh 9.8, but also patched the build file to pull in a fix: # openssh 9.8 compatibility git cherry-pick -n 2fed408c05ac5206b490368d94599869bd6a056d -- David C. Rankin, J.D.,P.E.
Hi, I submitted a fix to fail2ban. Ciao, Marcus On Tue, Sep 03, 2024 at 09:18:03PM -0500, David C. Rankin wrote:
On 9/3/24 4:16 PM, Pit Suetterlin via openSUSE Users wrote:
Pit Suetterlin via openSUSE Users wrote:
Hmm, what I just see is that also openssh was updated to 9.8p1 .....
Bingo: https://linuxthings.co.uk/blog/fail2ban-stopped-working-with-openssh-9-8
Good catch,
Yes, Arch has openssh 9.8, but also patched the build file to pull in a fix:
# openssh 9.8 compatibility git cherry-pick -n 2fed408c05ac5206b490368d94599869bd6a056d
-- David C. Rankin, J.D.,P.E.
-- Marcus Meissner (he/him), Distinguished Engineer / Senior Project Manager Security SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew McDonald, Werner Knoblich, HRB 36809, AG Nuernberg
On 9/3/24 4:10 PM, Pit Suetterlin via openSUSE Users wrote:
David C. Rankin wrote:
On 9/3/24 6:52 AM, Pit Suetterlin via openSUSE Users wrote:
The update was to 20240829, IIRC that was also the first one using kernel 6.10 - could that be related? Or was it some systemd update?
Anyone else noticed this?
Well, no, but I would suspect python before any of the others.
There was no change in python though, I think :o
Here's why. I have ipset, fail2ban and iptables running on Arch with systemd 256 and Linux 6.10.7 kernel and haven't had any fail2ban issues (other than getting the notification e-mails to include the log lines from the systemd journal -- use matches instead of lines -- or provide a custom action= for each jail)
Interesting, Which sub-version of systemd is it? The update I mentioned did upgrade 256.4->256.5
systemd 256.5-1
The thing is, I usually had entries in /var/log/fail2ban.log that listed detected fails: 2024-08-31 18:51:20,585 fail2ban.filter [1791]: INFO [sshd] Found 164.132.56.147 - 2024-08-31 18:51:20
None of those appear anymore, although there's permanent login attempts. That's why I assumed there was some change either in log syntax or access method (though that would probably cause an error?)
Here is a short write-up on how I got log lines working from the journal https://bbs.archlinux.org/viewtopic.php?id=298572 -- David C. Rankin, J.D.,P.E.
participants (3)
-
David C. Rankin -
Marcus Meissner -
Pit Suetterlin