Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting. Here is some info: linux:~ # ps aux|grep test test 3887 0.0 0.0 72236 7956 ? Ss 09:02 0:00 /usr/lib/systemd/systemd --user test 3888 0.0 0.0 265004 3220 ? S 09:02 0:00 (sd-pam) test 3909 416 7.3 2439200 2401880 ? SNsl 09:02 5:35 /home/test/.dhpcd root 5532 0.0 0.0 7432 920 pts/2 S+ 09:03 0:00 grep --color=auto test linux:~ # cd ~test linux:/home/test # ls .Xauthority .config .emacs .gtkrc-2.0 .local .profile .xinitrc.template Downloads Templates .bash_history .dbus .esd_auth .i18n .mozilla .qt .xsession-errors-:1 Music Videos .bashrc .dhpcd .fonts .inputrc .nx .ssh Desktop Pictures bin .cache .dmrc .gnupg .kde4 .pki .xim.template Documents Public public_html linux:/home/test # ls -l .dhpcd -rwxr-xr-x 1 test users 3458848 Mar 1 18:38 .dhpcd linux:/home/test # kill -9 3909 linux:/home/test # ps aux|grep test root 7884 0.0 0.0 7432 820 pts/2 S+ 09:06 0:00 grep --color=auto test linux:/home/test # more .dhpcd ******** .dhpcd: Not a text file ******** linux:/home/test # ls -a .dhpcd .dhpcd linux:/home/test # ls -al .dhpcd -rwxr-xr-x 1 test users 3458848 Mar 1 18:38 .dhpcd linux:/home/test # There were some updates yesterday, so I wonder if one of them is infected with something. Also, this computer is configured with a static IPv4 address and uses SLAAC on IPv6, so it shouldn't be using dhcpd.
On 02.03.21 15:17, James Knott wrote:
Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting. You can have a look at what it's actually doing with "strace": strace -f -p $(pidof .dhcpd)
Also, have a look if it has any suspicious strings (attached ;-) ): strings .dhcpd Josef -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
On Tue, Mar 02, 2021 at 03:27:23PM +0100, Josef Moellers wrote:
On 02.03.21 15:17, James Knott wrote:
Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting. You can have a look at what it's actually doing with "strace": strace -f -p $(pidof .dhcpd)
Also, have a look if it has any suspicious strings (attached ;-) ): strings .dhcpd
Sounds definitely malware if you did not put it there. Let me guess the test user has password "test"? Ciao, Marcus
On 2021-03-02 9:27 a.m., Josef Moellers wrote:
On 02.03.21 15:17, James Knott wrote:
Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting. You can have a look at what it's actually doing with "strace": strace -f -p $(pidof .dhcpd)
Also, have a look if it has any suspicious strings (attached;-) ): strings .dhcpd
I have already removed the execute bits from it and allowed only root rw access. If someone wants to examine it, I can provide the file. It's about 3.5 MB, so I'll leave it on Google drive. This could have only come from one of the updates that arrived yesterday, as I haven't installed any other software recently, other than OpenVPN 2.5.0, which came from https://download.opensuse.org/repositories/home:/stroeder:/network/openSUSE_... last week.
On 02/03/2021 15.38, James Knott wrote:
On 2021-03-02 9:27 a.m., Josef Moellers wrote:
On 02.03.21 15:17, James Knott wrote:
Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting. You can have a look at what it's actually doing with "strace": strace -f -p $(pidof .dhcpd)
Also, have a look if it has any suspicious strings (attached;-) ): strings .dhcpd
I have already removed the execute bits from it and allowed only root rw access. If someone wants to examine it, I can provide the file. It's about 3.5 MB, so I'll leave it on Google drive.
This could have only come from one of the updates that arrived yesterday, as I haven't installed any other software recently, other than OpenVPN 2.5.0, which came from https://download.opensuse.org/repositories/home:/stroeder:/network/openSUSE_... last week.
Try rpm -f /home/test/.dhcpd to find out if it came in an rpm. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02.03.21 16:44, James Knott wrote:
On 2021-03-02 10:41 a.m., Carlos E. R. wrote:
rpm -f/home/test/.dhcpd
to find out if it came in an rpm.
That didn't work. I just got a usage display.
Put a space between the '-f' and the pathname Josef -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
On 02.03.21 16:53, James Knott wrote:
On 2021-03-02 10:46 a.m., Josef Moellers wrote:
That didn't work. I just got a usage display. Put a space between the '-f' and the pathname
Same thing. I've never had to worry about a space with the options before.
Fired but forgot ... it should be "rpm -qf <pathname>" <redface> Josef -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
James Knott wrote:
On 2021-03-02 10:41 a.m., Carlos E. R. wrote:
rpm -f/home/test/.dhcpd
to find out if it came in an rpm.
That didn't work. I just got a usage display.
Try "rpm -qf /home/test/.dhcpd" -- Per Jessen, Zürich (14.8°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland.
On 2021-03-02 10:49 a.m., Per Jessen wrote:
That didn't work. I just got a usage display. Try "rpm -qf/home/test/.dhcpd"
rpm -qf /home/test/.dhcpd error: file /home/test/.dhcpd: No such file or directory However, I noticed it seems to be looking in whatever directory I run it from.
James Knott wrote:
On 2021-03-02 10:49 a.m., Per Jessen wrote:
That didn't work. I just got a usage display. Try "rpm -qf/home/test/.dhcpd"
rpm -qf /home/test/.dhcpd error: file /home/test/.dhcpd: No such file or directory
My typo - rpm -qf /home/test/.dhpcd -- Per Jessen, Zürich (14.0°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
Per Jessen wrote:
James Knott wrote:
On 2021-03-02 10:49 a.m., Per Jessen wrote:
That didn't work. I just got a usage display. Try "rpm -qf/home/test/.dhcpd"
rpm -qf /home/test/.dhcpd error: file /home/test/.dhcpd: No such file or directory
My typo -
rpm -qf /home/test/.dhpcd
Try googling ".dhpcd", there are mentions going back to 2019. Also with user 'test'. -- Per Jessen, Zürich (14.1°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland.
James Knott wrote:
On 2021-03-02 11:01 a.m., Per Jessen wrote:
Try googling ".dhpcd", there are mentions going back to 2019. Also with user 'test'.
I didn't see anything relevant. I even tried .dhcpd virus.
it has to be ".dhpcd", not ".dhcpd". at askubuntu, I saw people discussing malware. -- Per Jessen, Zürich (11.8°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland.
Op dinsdag 2 maart 2021 17:34:44 CET schreef Per Jessen:
it has to be ".dhpcd", not ".dhcpd". at askubuntu, I saw people discussing malware. https://askubuntu.com/questions/1100467/re-what-is-dhpcd-command-and-how-to-...
-- Gertjan Lettink a.k.a. Knurpht openSUSE Board openSUSE Forums Team
On 02/03/2021 17.37, Knurpht-openSUSE wrote:
Op dinsdag 2 maart 2021 17:34:44 CET schreef Per Jessen:
it has to be ".dhpcd", not ".dhcpd". at askubuntu, I saw people discussing malware. https://askubuntu.com/questions/1100467/re-what-is-dhpcd-command-and-how-to-...
But they don't know how it got installed. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/2/21 8:05 AM, James Knott wrote:
On 2021-03-02 11:01 a.m., Per Jessen wrote:
Try googling ".dhpcd", there are mentions going back to 2019. Also with user 'test'.
I didn't see anything relevant. I even tried .dhcpd virus.
Did you try to upload the file to virustotal.com? Regards, Lew
On Tue, Mar 02, 2021 at 12:11:56PM -0500, James Knott wrote:
On 2021-03-02 11:53 a.m., Lew Wolfgang wrote:
I didn't see anything relevant. I even tried .dhcpd virus.
Did you try to upload the file to virustotal.com?
Marcus Meissner did.
https://www.virustotal.com/gui/file/364c6b2dcd55840f9c121c4131024965ef44dd18... It reports as some generic Bitcoin Miner malware. Ciao, Marcus
On 3/2/21 9:33 AM, Marcus Meissner wrote:
On Tue, Mar 02, 2021 at 12:11:56PM -0500, James Knott wrote:
On 2021-03-02 11:53 a.m., Lew Wolfgang wrote:
I didn't see anything relevant. I even tried .dhcpd virus. Did you try to upload the file to virustotal.com? Marcus Meissner did. https://www.virustotal.com/gui/file/364c6b2dcd55840f9c121c4131024965ef44dd18...
It reports as some generic Bitcoin Miner malware.
Of course everyone knows that the infected host can never be fully trusted again, right? James: have you tried running rkhunter on the infected host? It can be loaded via YaST if necessary. Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch. Regards, Lew
On 2021-03-02 12:50 p.m., Lew Wolfgang wrote:
On 3/2/21 9:33 AM, Marcus Meissner wrote:
On Tue, Mar 02, 2021 at 12:11:56PM -0500, James Knott wrote:
On 2021-03-02 11:53 a.m., Lew Wolfgang wrote:
I didn't see anything relevant. I even tried .dhcpd virus. Did you try to upload the file to virustotal.com? Marcus Meissner did. https://www.virustotal.com/gui/file/364c6b2dcd55840f9c121c4131024965ef44dd18...
It reports as some generic Bitcoin Miner malware.
Of course everyone knows that the infected host can never be fully trusted again, right?
Yep.
James: have you tried running rkhunter on the infected host? It can be loaded via YaST if necessary.
Done that.
Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch.
I was already thinking about that. Any chance 15.3 will be available soon? btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely.
James Knott wrote:
James: have you tried running rkhunter on the infected host? It can be loaded via YaST if necessary.
Done that.
and it found nothing ?
Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch.
I was already thinking about that. Any chance 15.3 will be available soon?
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely.
sometimes just grepping for 'ssh' in /var/log/messages will show you a login from an unknown ip address. -- Per Jessen, Zürich (9.0°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland.
On 2021-03-02 1:55 p.m., Per Jessen wrote:
James Knott wrote:
James: have you tried running rkhunter on the infected host? It can be loaded via YaST if necessary. Done that. and it found nothing ?
I found several warnings. I'm checking them now.
Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch. I was already thinking about that. Any chance 15.3 will be available soon?
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely. sometimes just grepping for 'ssh' in /var/log/messages will show you a login from an unknown ip address.
That appears to be the way they got in. I just realized I had ssh opened in my firewall for testing recently and forgot to close it. I have changed the password and disabled login for that account. I will be removing the account entirely. It came from 61.177.173.3, which is in China.
On 2021-03-02 20:21, James Knott wrote:
I just realized I had ssh opened in my firewall for testing recently and forgot to close it. I have changed the password and disabled login for that account. I will be removing the account entirely.
in /etc/ssh/sshd_config PasswordAuthentication no But first put an authorized_keys in your ~/.ssh/ Works like a charm. No scumbags for over a decade on our servers. All our servers have sshd port (on a different port than 22) open on public internet. -- /bengan
On 2021-03-02 2:57 p.m., Bengt Gördén wrote:
I just realized I had ssh opened in my firewall for testing recently and forgot to close it. I have changed the password and disabled login for that account. I will be removing the account entirely.
in /etc/ssh/sshd_config PasswordAuthentication no But first put an authorized_keys in your ~/.ssh/
Works like a charm. No scumbags for over a decade on our servers. All our servers have sshd port (on a different port than 22) open on public internet.
I used to do that, but got out of the habit.
On 02/03/2021 20.21, James Knott wrote:
On 2021-03-02 1:55 p.m., Per Jessen wrote:
James Knott wrote:
sometimes just grepping for 'ssh' in /var/log/messages will show you a login from an unknown ip address.
That appears to be the way they got in. I just realized I had ssh opened in my firewall for testing recently and forgot to close it. I have changed the password and disabled login for that account. I will be removing the account entirely.
It came from 61.177.173.3, which is in China.
He guessed the login/pass? -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02/03/2021 21.52, James Knott wrote:
On 2021-03-02 3:50 p.m., Carlos E. R. wrote:
He guessed the login/pass?
Apparently. According to what I saw about that site, they try lots of passwords and there were lots of attemps.
And what user did they get in, root or test? -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
James Knott wrote:
On 2021-03-02 3:54 p.m., Carlos E. R. wrote:
Apparently. According to what I saw about that site, they try lots of passwords and there were lots of attemps. And what user did they get in, root or test?
Test.
Don't feel too comfortable yet, but that is a good(!) sign. They may have been after nothing but your cpu cycles. -- Per Jessen, Zürich (5.1°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.
On 02/03/2021 21.56, James Knott wrote:
On 2021-03-02 3:54 p.m., Carlos E. R. wrote:
Apparently. According to what I saw about that site, they try lots of passwords and there were lots of attemps. And what user did they get in, root or test?
Test.
Ok, so the user did exist and had actually a password and they guessed it, via ssh and many attempts. Ok, that's the important information to know. They did not make "root", so the machine probably was not compromised. Check "crontab -e" (uses vi by default) to see if he created a cronjob. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 4:12 p.m., Carlos E. R. wrote:
Ok, so the user did exist and had actually a password and they guessed it, via ssh and many attempts. Ok, that's the important information to know. They did not make "root", so the machine probably was not compromised.
Check "crontab -e" (uses vi by default) to see if he created a cronjob.
Nothing there.
On 02/03/2021 22.14, James Knott wrote:
On 2021-03-02 4:12 p.m., Carlos E. R. wrote:
Ok, so the user did exist and had actually a password and they guessed it, via ssh and many attempts. Ok, that's the important information to know. They did not make "root", so the machine probably was not compromised.
Check "crontab -e" (uses vi by default) to see if he created a cronjob.
Nothing there.
A systemd user timer, then? Something in /etc/cron.d/? -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02/03/2021 22.18, James Knott wrote:
On 2021-03-02 4:17 p.m., Carlos E. R. wrote:
Nothing there. A systemd user timer, then?
Something in/etc/cron.d/?
No.
Then maybe they connected via ssh to start it again? Doesn't make sense. A user systemd service? A desktop autostart? No, that would be a different user. Oh, I meant "crontab -e - u test" (you must be root). -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 4:24 p.m., Carlos E. R. wrote:
Then maybe they connected via ssh to start it again? Doesn't make sense.
A user systemd service?
A desktop autostart? No, that would be a different user.
Oh, I meant "crontab -e - u test" (you must be root).
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null 2>/dev/null However, I cleared the executable bits for that file. btw, no space between - and u.
On 02/03/2021 22.41, James Knott wrote:
On 2021-03-02 4:24 p.m., Carlos E. R. wrote:
Then maybe they connected via ssh to start it again? Doesn't make sense.
A user systemd service?
A desktop autostart? No, that would be a different user.
Oh, I meant "crontab -e - u test" (you must be root).
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null 2>/dev/null
However, I cleared the executable bits for that file.
Yes, sure, but it was important for us to know that they set a crontab entry and which.
btw, no space between - and u.
Yeah, finger error :-) You could also firewall ca.minexmr.com or in your dns setup. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02/03/2021 23.04, James Knott wrote:
On 2021-03-02 4:57 p.m., Carlos E. R. wrote:
You could also firewall ca.minexmr.com or in your dns setup.
I blocked the entire /8 network that it came from. Also, I closed port 22, which I had open for testing.
All that do not have effect on ca.minexmr.com, because they would be outgoing connections from your computer. In fact, it would be good for everybody. Even the FBI. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/2/21 1:57 PM, Carlos E. R. wrote:
On 02/03/2021 22.41, James Knott wrote:
On 2021-03-02 4:24 p.m., Carlos E. R. wrote:
Then maybe they connected via ssh to start it again? Doesn't make sense.
A user systemd service?
A desktop autostart? No, that would be a different user.
Oh, I meant "crontab -e - u test" (you must be root). 2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null 2>/dev/null
However, I cleared the executable bits for that file. Yes, sure, but it was important for us to know that they set a crontab entry and which.
btw, no space between - and u. Yeah, finger error :-)
You could also firewall ca.minexmr.com or in your dns setup.
minexmr.com is a Monero mining pool site. They claim to have 8990 active miners. Regards, Lew
On 3/2/21 1:21 PM, James Knott wrote:
On 2021-03-02 1:55 p.m., Per Jessen wrote:
James Knott wrote:
James: have you tried running rkhunter on the infected host? It can be loaded via YaST if necessary. Done that. and it found nothing ?
I found several warnings. I'm checking them now.
Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch. I was already thinking about that. Any chance 15.3 will be available soon?
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely. sometimes just grepping for 'ssh' in /var/log/messages will show you a login from an unknown ip address.
That appears to be the way they got in. I just realized I had ssh opened in my firewall for testing recently and forgot to close it. I have changed the password and disabled login for that account. I will be removing the account entirely.
It came from 61.177.173.3, which is in China.
Would have never gotten it... iptables is your friend: -A INPUT -s 61.0.0.0/8 -j DROP -- David C. Rankin, J.D.,P.E.
"JK" == James Knott <james.knott@jknott.net> writes:
JK> On 2021-03-02 11:36 p.m., David C. Rankin wrote:
Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
JK> My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network. Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
On 03/03/2021 13.16, Togan Muftuoglu wrote:
"JK" == James Knott <james.knott@jknott.net> writes:
JK> On 2021-03-02 11:36 p.m., David C. Rankin wrote:
Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
JK> My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network.
Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
China is not the only country with hackers. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
"Carlos" == Carlos E R <robin.listas@telefonica.net> writes:
Carlos> On 03/03/2021 13.16, Togan Muftuoglu wrote:
> "JK" == James Knott <james.knott@jknott.net> writes:
JK> On 2021-03-02 11:36 p.m., David C. Rankin wrote:
Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
JK> My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network.
Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
Carlos> China is not the only country with hackers. When one understands the concept then one can block wtf countries. Try not so hard to read everything literally. ;)
On 03/03/2021 13.24, Togan Muftuoglu wrote:
"Carlos" == Carlos E R <robin.listas@telefonica.net> writes: Carlos> On 03/03/2021 13.16, Togan Muftuoglu wrote: >> "JK" == James Knott <james.knott@jknott.net> writes: JK> On 2021-03-02 11:36 p.m., David C. Rankin wrote: Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
JK> My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network.
Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
Carlos> China is not the only country with hackers.
When one understands the concept then one can block wtf countries. Try not so hard to read everything literally. ;)
It is not that. It is that you have to block way too many countries. maybe even the USA :-p -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-03 7:24 a.m., Togan Muftuoglu wrote:
Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
Carlos> China is not the only country with hackers.
When one understands the concept then one can block wtf countries. Try not so hard to read everything literally.;)
Maybe I can allow only Canada or perhaps just Ontario.
On 3/3/21 6:16 AM, Togan Muftuoglu wrote:
"JK" == James Knott <james.knott@jknott.net> writes:
JK> On 2021-03-02 11:36 p.m., David C. Rankin wrote:
Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
JK> My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network.
Maybe better to use geoip based filtering and block the whole China, unless you have something to do.
I keep track of the top 50 offending IP blocks from which my servers have to block malicious intrusion attempts. While over-inclusive, I haven't suffered a successful intrusions since 2013 (MediaWiki was breached back then -- mess). Here are the current reports ( Server 1: (up 9 days) 1 5019 240K 51.0.0.0/8 2 2270 120K 185.0.0.0/8 3 1350 66256 5.0.0.0/8 4 1327 56164 217.0.0.0/8 5 1179 47160 131.161.0.0/16 6 1090 63412 194.0.0.0/8 7 908 65624 81.0.0.0/8 8 860 46500 89.0.0.0/8 9 718 31600 116.0.0.0/8 10 687 33044 78.0.0.0/8 11 611 30052 93.0.0.0/8 12 610 33576 103.0.0.0/8 13 573 26952 91.0.0.0/8 14 572 24380 95.0.0.0/8 15 561 27532 195.0.0.0/8 16 545 22808 45.128.0.0/11 17 544 30796 88.0.0.0/8 18 540 23652 2.0.0.0/8 19 483 23648 188.0.0.0/8 20 475 20192 120.0.0.0/8 21 460 22132 92.0.0.0/8 22 459 26428 77.0.0.0/8 23 424 16960 139.162.0.0/16 24 399 21912 80.0.0.0/8 25 385 18892 79.0.0.0/8 26 346 19972 58.0.0.0/8 27 330 18736 87.0.0.0/8 28 315 16744 94.0.0.0/8 29 314 15784 37.0.0.0/8 30 271 11953 203.0.0.0/8 31 260 13140 178.0.0.0/8 32 254 13428 114.0.0.0/8 33 242 11008 220.0.0.0/8 34 234 13420 85.0.0.0/8 35 230 13744 111.0.0.0/8 36 223 10984 82.0.0.0/8 37 218 8744 49.0.0.0/8 38 199 11072 109.0.0.0/8 39 194 11144 31.0.0.0/8 40 184 9444 212.0.0.0/8 41 183 8912 121.0.0.0/8 42 179 9503 193.0.0.0/8 43 179 10496 83.0.0.0/8 44 169 7940 46.0.0.0/8 45 168 9356 176.0.0.0/8 46 167 9232 62.0.0.0/8 47 165 8500 86.0.0.0/8 48 161 9236 84.0.0.0/8 49 150 10750 151.0.0.0/8 50 144 6860 1.0.0.0/8 Server 2: (up 3 days) 1 1122 61028 185.0.0.0/8 2 741 101K 203.0.0.0/8 3 683 40220 37.0.0.0/8 4 567 38300 103.0.0.0/8 5 507 25992 5.0.0.0/8 6 477 24796 123.0.0.0/8 7 366 21840 51.0.0.0/8 8 281 16540 46.0.0.0/8 9 248 12712 178.0.0.0/8 10 212 17152 193.0.0.0/8 11 188 14136 111.0.0.0/8 12 187 10060 125.0.0.0/8 13 159 8268 176.0.0.0/8 14 150 8940 93.0.0.0/8 15 122 6395 221.0.0.0/8 16 113 7515 196.0.0.0/8 17 112 6220 62.0.0.0/8 18 105 6300 116.0.0.0/8 19 100 5072 89.0.0.0/8 20 99 5520 91.0.0.0/8 21 80 4752 188.0.0.0/8 22 67 3420 81.0.0.0/8 23 61 3660 182.0.0.0/8 24 56 3360 154.0.0.0/8 25 55 3292 77.0.0.0/8 26 54 2808 114.0.0.0/8 27 48 2844 85.0.0.0/8 28 47 2444 84.0.0.0/8 29 46 2600 101.0.0.0/8 30 43 2580 42.0.0.0/8 31 37 2204 212.0.0.0/8 32 36 2160 207.245.103.0/24 33 35 2373 202.0.0.0/8 34 29 1468 87.0.0.0/8 35 24 1248 78.0.0.0/8 36 23 1380 144.76.0.0/16 37 22 1304 180.0.0.0/8 38 21 1260 27.0.0.0/8 39 18 896 217.0.0.0/8 40 17 908 194.0.0.0/8 41 17 848 222.0.0.0/8 42 16 960 49.0.0.0/8 43 16 904 83.0.0.0/8 44 14 784 151.0.0.0/8 45 13 807 195.0.0.0/8 46 13 732 82.0.0.0/8 47 13 716 118.0.0.0/8 48 12 720 79.0.0.0/8 49 10 504 61.0.0.0/8 50 10 468 121.0.0.0/8 -- David C. Rankin, J.D.,P.E.
On 3/3/21 4:04 AM, James Knott wrote:
On 2021-03-02 11:36 p.m., David C. Rankin wrote:
Would have never gotten it... iptables is your friend:
-A INPUT -s 61.0.0.0/8 -j DROP
My firewall is pfsense and I have blocked the entire 61.0.0.0 /8 network.
But China has many networks. Then you have the proxy issue... Why not block everything by default if you're not running a pubic server? Just open needed ports to specific addresses, not even subnets. Regards, Lew
On 2021-03-03 12:04 p.m., Lew Wolfgang wrote:
Why not block everything by default if you're not running a pubic server? Just open needed ports to specific addresses, not even subnets.
Normally, I only allow VPN access to my network and just had ssh open for some testing. Putting that in will block them from trying anything, not just ssh and that network was where all the hits were coming from.
On 02/03/2021 19.46, James Knott wrote:
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely.
Correlate that point in time with all logs. What were you doing at the time. It is very important to find out how that thing got in. To create /home/test the first time they need root access of some kind, so anything you do to disable that account is pointless. Then you say that the thing would restart itself. I have read on google that it restarts every halfhour, so there will be some cronjob entry somewhere. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 2:02 p.m., Carlos E. R. wrote:
Correlate that point in time with all logs. What were you doing at the time.
Having dinner.
To create /home/test the first time they need root access of some kind, so anything you do to disable that account is pointless.
It was already there, as I created it years ago.
Then you say that the thing would restart itself. I have read on google that it restarts every halfhour, so there will be some cronjob entry
It remained through a reboot. I haven't seen it since I killed it and cleared the execute bits. I still have to see how it started. I have also identified someone connecting with ssh at about the time it appeared on my system. I have created a firewall rule to block the entire 61.0.0.0 /8 network, as the attack came from 61.177.173.3, which is in China.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 10:46 AM, James Knott <james.knott@jknott.net> wrote:
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed.
Game over. He's rooted you. Wish you'd respond to my questions so it may help the rest of us.
On 02/03/2021 20.07, colony.three@protonmail.ch wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 10:46 AM, James Knott <james.knott@jknott.net> wrote:
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed.
Game over. He's rooted you.
Maybe not, if he directly loged in as "test". Should be in the log. If he were root, he would have deleted the traces.
Wish you'd respond to my questions so it may help the rest of us.
-- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02/03/2021 21.58, James Knott wrote:
On 2021-03-02 3:53 p.m., Carlos E. R. wrote:
Maybe not, if he directly loged in as "test". Should be in the log. If he were root, he would have deleted the traces.
It was test. It's been over an hour since I rebooted and no sign of .dhcpd yet.
Check the log to see if something tried to start it and failed (because it is not executable). -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 4:18 p.m., Carlos E. R. wrote:
It was test. It's been over an hour since I rebooted and no sign of .dhcpd yet. Check the log to see if something tried to start it and failed (because it is not executable).
It's not executable now, because I cleared the execute bits. It was running this morning.
On 02/03/2021 22.19, James Knott wrote:
On 2021-03-02 4:18 p.m., Carlos E. R. wrote:
It was test. It's been over an hour since I rebooted and no sign of .dhcpd yet. Check the log to see if something tried to start it and failed (because it is not executable).
It's not executable now, because I cleared the execute bits. It was running this morning.
I know you did, which is precisely why I ask you to scan the logs after that instant to see if something tried and failed. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 4:25 p.m., Carlos E. R. wrote:
It's not executable now, because I cleared the execute bits. It was running this morning. I know you did, which is precisely why I ask you to scan the logs after that instant to see if something tried and failed.
Well, I have removed that crontab line, so it won't be trying to start again.
On 02/03/2021 22.43, James Knott wrote:
On 2021-03-02 4:25 p.m., Carlos E. R. wrote:
It's not executable now, because I cleared the execute bits. It was running this morning. I know you did, which is precisely why I ask you to scan the logs after that instant to see if something tried and failed.
Well, I have removed that crontab line, so it won't be trying to start again.
Ah, you removed a crontab line! You did not tell us that. What line, where exactly? -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 4:46 p.m., Carlos E.R. wrote:
Well, I have removed that crontab line, so it won't be trying to start again. Ah, you removed a crontab line! You did not tell us that. What line, where exactly?
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null 2>/dev/null I mentioned it in another message to the list. It was in the crontab for the user test.
James Knott wrote:
On 2021-03-02 4:46 p.m., Carlos E.R. wrote:
Well, I have removed that crontab line, so it won't be trying to start again. Ah, you removed a crontab line! You did not tell us that. What line, where exactly?
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null ^^^^^^^^^^^^^^^^^^^
That is a useful hint - it led me to this: https://blogs.akamai.com/sitr/2020/04/a-brief-history-of-a-rootable-docker-i... As also expected when this project first started, the docker image was a prime target for crypto-mining malware. An installation script is run via SSH that monitors the process tree with "top" - a process monitoring tool - checking for when the malware has successfully been executed. Once the process is detected, in this case it is "dhcpcd", the malware adds two new users to the system - test and test1. It changes the root password to ""(blank) and creates entries in cron to start the mining software up after a reboot. It also attempts to set the immutable bit (+i) on various files including /etc/shadow and the malware binary in /sbin/dhcpcd. The malware also adds a .ssh key to /root/.ssh/authorized_keys: ssh-rsa .......... The mining software reports as XMRig v5.2.0. -- Per Jessen, Zürich (5.5°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland.
On 03/03/2021 07.48, Per Jessen wrote:
James Knott wrote:
On 2021-03-02 4:46 p.m., Carlos E.R. wrote:
Well, I have removed that crontab line, so it won't be trying to start again. Ah, you removed a crontab line! You did not tell us that. What line, where exactly?
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null ^^^^^^^^^^^^^^^^^^^
That is a useful hint - it led me to this:
https://blogs.akamai.com/sitr/2020/04/a-brief-history-of-a-rootable-docker-i...
As also expected when this project first started, the docker image was a prime target for crypto-mining malware. An installation script is run via SSH that monitors the process tree with "top" - a process monitoring tool - checking for when the malware has successfully been executed. Once the process is detected, in this case it is "dhcpcd", the malware adds two new users to the system - test and test1. It changes the root password to ""(blank) and creates entries in cron to start the mining software up after a reboot. It also attempts to set the immutable bit (+i) on various files including /etc/shadow and the malware binary in /sbin/dhcpcd.
The malware also adds a .ssh key to /root/.ssh/authorized_keys:
ssh-rsa ..........
The mining software reports as XMRig v5.2.0.
That attack is different, they start with root powers. Much more dangerous. Interesting read that link, thanks :-) -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-03 1:48 a.m., Per Jessen wrote:
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null ^^^^^^^^^^^^^^^^^^^
That is a useful hint - it led me to this:
https://blogs.akamai.com/sitr/2020/04/a-brief-history-of-a-rootable-docker-i...
As also expected when this project first started, the docker image was a prime target for crypto-mining malware. An installation script is run via SSH that monitors the process tree with "top" - a process monitoring tool - checking for when the malware has successfully been executed. Once the process is detected, in this case it is "dhcpcd", the malware adds two new users to the system - test and test1. It changes the root password to ""(blank) and creates entries in cron to start the mining software up after a reboot. It also attempts to set the immutable bit (+i) on various files including /etc/shadow and the malware binary in /sbin/dhcpcd.
I haven't seen that. For example: -rw-r----- 1 root shadow 1060 Mar 2 17:00 /etc/shadow
The malware also adds a .ssh key to /root/.ssh/authorized_keys: Nothing there.
On 03/03/2021 14.42, James Knott wrote:
On 2021-03-03 1:48 a.m., Per Jessen wrote:
2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null ^^^^^^^^^^^^^^^^^^^
That is a useful hint - it led me to this:
https://blogs.akamai.com/sitr/2020/04/a-brief-history-of-a-rootable-docker-i...
As also expected when this project first started, the docker image was a prime target for crypto-mining malware. An installation script is run via SSH that monitors the process tree with "top" - a process monitoring tool - checking for when the malware has successfully been executed. Once the process is detected, in this case it is "dhcpcd", the malware adds two new users to the system - test and test1. It changes the root password to ""(blank) and creates entries in cron to start the mining software up after a reboot. It also attempts to set the immutable bit (+i) on various files including /etc/shadow and the malware binary in /sbin/dhcpcd.
I haven't seen that. For example: -rw-r----- 1 root shadow 1060 Mar 2 17:00 /etc/shadow
The malware also adds a .ssh key to /root/.ssh/authorized_keys: Nothing there.
Do you allow root login via ssh? Did they attempt to login as root (check your logs)? If "yes" then perhaps your root's password is harder to guess. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 10:48 PM, Per Jessen > > 2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null
^^^^^^^^^^^^^^^^^^^
That is a useful hint - it led me to this:
https://blogs.akamai.com/sitr/2020/04/a-brief-history-of-a-rootable-docker-i...
As also expected when this project first started, the docker image was a prime target for crypto-mining malware. An installation script is run via SSH that monitors the process tree with "top" - a process monitoring tool - checking for when the malware has successfully been executed. Once the process is detected, in this case it is "dhcpcd", the malware adds two new users to the system - test and test1. It changes the root password to ""(blank) and creates entries in cron to start the mining software up after a reboot. It also attempts to set the immutable bit (+i) on various files including /etc/shadow and the malware binary in /sbin/dhcpcd.
The malware also adds a .ssh key to /root/.ssh/authorized_keys:
ssh-rsa ..........
The mining software reports as XMRig v5.2.0.
Per Jessen, Zürich (5.5°C)
This is all why I have switched over to an immutable OS. (MicroOS) Every security layer counts.
Le 02/03/2021 à 20:07, colony.three@protonmail.ch a écrit :
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 10:46 AM, James Knott <james.knott@jknott.net> wrote:
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed.
Game over. He's rooted you.
Wish you'd respond to my questions so it may help the rest of us.
user test can change his own passwd jdd -- http://dodin.org
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 1:42 PM, jdd@dodin.org <jdd@dodin.org> wrote:
Le 02/03/2021 à 20:07, colony.three@protonmail.ch a écrit :
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 10:46 AM, James Knott james.knott@jknott.net wrote:
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed.
Game over. He's rooted you. Wish you'd respond to my questions so it may help the rest of us.
user test can change his own passwd
jdd
Sheesh. I give up.
On 2021-03-03 06:29, colony.three@protonmail.ch wrote:
Sheesh. I give up.
You have written three messages. Different recipients each time. It's not easy to figure out who you ask. In your first message to Jonathan Arsenault you ask three questions. In your second message to James Knott you ask him to answer your question to Jonathan. In your third message to Jean-Daniel Dodin you give up. Are you a bot? -- /bengan
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, March 3, 2021 2:00 AM, Bengt Gördén <bengan@bag.org> wrote:
On 2021-03-03 06:29, colony.three@protonmail.ch wrote:
Sheesh. I give up.
You have written three messages. Different recipients each time. It's not easy to figure out who you ask. In your first message to Jonathan Arsenault you ask three questions. In your second message to James Knott you ask him to answer your question to Jonathan. In your third message to Jean-Daniel Dodin you give up. Are you a bot?
/bengan
'Am I a 'bot'... That's an amusing conclusion. Do I sound like a 'bot? If so, you have not been understanding my questions. However your most fundamental misunderstanding... would. a. 'bot. give. up.... Maybe something to think about. No, not different recipients. The listserv each time (unless one time I slipped up). No, I have just been asking questions which get to the heart of this, but no one seems to understand them. Everyone has zeroed on on SSH being the vector, which I think is nonsense unless the OP is an idiot, which he professes not to be. Now we are left -without- the vital information needed to fully investigate this... and instead a pile-on game of who is smarter. Another dreary race to the bottom. I am just a refugee from CentOS. But my experiences on #suse and here have been disappointing.
On 3/2/21 12:46 PM, James Knott wrote:
On 2021-03-02 12:50 p.m., Lew Wolfgang wrote:
Hopefully you have off-line backups. If I were you, after trying to determine how this happened, I'd slick the disk and reinstall everything from scratch.
I was already thinking about that. Any chance 15.3 will be available soon?
btw, the shadow password file was changed at the same time as that .dhcpd file was created. Since there were only 3 login password in it and I know 2 of them still work, I assume the test account password was changed. I changed the password and set the account to not allow login. My next step is to remove the test user entirely.
The link Gertjan provide shows the final solution for the Ubuntu user: sudo deluser --system --remove-all-files test -- David C. Rankin, J.D.,P.E.
On 3/3/21 6:03 AM, James Knott wrote:
On 2021-03-02 11:33 p.m., David C. Rankin wrote:
The link Gertjan provide shows the final solution for the Ubuntu user:
sudo deluser --system --remove-all-files test
I used the Yast User and Group Management to remove the user and files. Is that any different?
Late-reply, No, but make sure you look for a check box that says "remove all user file" or something similar. -- David C. Rankin, J.D.,P.E.
On 05/03/2021 05.26, David C. Rankin wrote:
On 3/3/21 6:03 AM, James Knott wrote:
On 2021-03-02 11:33 p.m., David C. Rankin wrote:
The link Gertjan provide shows the final solution for the Ubuntu user:
sudo deluser --system --remove-all-files test
I used the Yast User and Group Management to remove the user and files. Is that any different?
Late-reply,
No, but make sure you look for a check box that says "remove all user file" or something similar.
Notice that this means "/home/[user]", not any other file in the filesystem that may belong to the user. It doesn't do a search. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-05 7:47 a.m., Carlos E. R. wrote:
No, but make sure you look for a check box that says "remove all user file" or something similar. Notice that this means "/home/[user]", not any other file in the filesystem that may belong to the user. It doesn't do a search.
Where else is a user allowed to write?
On 05/03/2021 14.45, James Knott wrote:
On 2021-03-05 7:47 a.m., Carlos E. R. wrote:
No, but make sure you look for a check box that says "remove all user file" or something similar. Notice that this means "/home/[user]", not any other file in the filesystem that may belong to the user. It doesn't do a search.
Where else is a user allowed to write?
/tmp, for instance. There is a "find" concoction that will find any file belonging to a certain user, but I don't have it handy -- meaning I did not find the note :-) -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Carlos E. R. wrote:
On 05/03/2021 14.45, James Knott wrote:
On 2021-03-05 7:47 a.m., Carlos E. R. wrote:
No, but make sure you look for a check box that says "remove all user file" or something similar. Notice that this means "/home/[user]", not any other file in the filesystem that may belong to the user. It doesn't do a search.
Where else is a user allowed to write?
/tmp, for instance.
There is a "find" concoction that will find any file belonging to a certain user, but I don't have it handy -- meaning I did not find the note :-)
Try this: find / -type f -user carlos -- Per Jessen, Zürich (5.6°C)
On 2021-03-05 9:02 a.m., Per Jessen wrote:
There is a "find" concoction that will find any file belonging to a certain user, but I don't have it handy -- meaning I did not find the note:-) Try this: find / -type f -user carlos
-user carlos didn't work. ;-) Actually, since I have deleted the user, I had to search on UID and came up with this: /tmp/sora.arm5 /tmp/sora.x86 /tmp/sora.m68k /tmp/sora.mips /tmp/sora.ppc /tmp/sora.sh4 /tmp/sora.arm4 /tmp/uYtea.x86 /tmp/sora.arm6 /tmp/sora.mpsl /tmp/uYtea.x86.3 /tmp/uYtea.x86.1 /tmp/uYtea.x86.4 /tmp/uYtea.x86.2 /tmp/sora.arm7 linux:~ # ls -l /tmp/sora* -rwxr-xr-x 1 1001 users 212 Feb 28 15:29 /tmp/sora.arm4 -rwxr-xr-x 1 1001 users 22140 Feb 28 15:29 /tmp/sora.arm5 -rwxr-xr-x 1 1001 users 29448 Feb 28 15:29 /tmp/sora.arm6 -rwxr-xr-x 1 1001 users 48664 Feb 28 15:29 /tmp/sora.arm7 -rwxr-xr-x 1 1001 users 53072 Feb 28 15:29 /tmp/sora.m68k -rwxr-xr-x 1 1001 users 26204 Feb 28 15:29 /tmp/sora.mips -rwxr-xr-x 1 1001 users 27220 Feb 28 15:29 /tmp/sora.mpsl -rwxr-xr-x 1 1001 users 23928 Feb 28 15:29 /tmp/sora.ppc -rwxr-xr-x 1 1001 users 51616 Feb 28 15:29 /tmp/sora.sh4 -rwxr-xr-x 1 1001 users 24760 Feb 28 15:29 /tmp/sora.x86 linux:~ # ls -l /tmp/uYtea* -rwxr-xr-x 1 1001 users 63540 Mar 1 16:44 /tmp/uYtea.x86 -rwxr-xr-x 1 1001 users 63540 Feb 28 21:18 /tmp/uYtea.x86.1 -rwxr-xr-x 1 1001 users 63540 Feb 28 21:18 /tmp/uYtea.x86.2 -rwxr-xr-x 1 1001 users 63540 Feb 28 21:18 /tmp/uYtea.x86.3 -rwxr-xr-x 1 1001 users 63540 Feb 28 21:18 /tmp/uYtea.x86.4 I have deleted all those.
On 05.03.21 14:45, James Knott wrote:
On 2021-03-05 7:47 a.m., Carlos E. R. wrote:
No, but make sure you look for a check box that says "remove all user file" or something similar. Notice that this means "/home/[user]", not any other file in the filesystem that may belong to the user. It doesn't do a search.
Where else is a user allowed to write?
The user doesn't have to write the file himself, it can also be created on his behalf, eg mailboxes, crontabs, atjobs, to name but a few. Josef -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
"JK" == James Knott <james.knott@jknott.net> writes:
JK> On 2021-03-05 9:39 a.m., Josef Moellers wrote:
Where else is a user allowed to write? The user doesn't have to write the file himself, it can also be created on his behalf, eg mailboxes, crontabs, atjobs, to name but a few.
JK> I have deleted all files with that user's UID. I would have preferred to wipe the disk and make a clean install to have a peaceful mind. But again that is me.
Having read through this long conversation, and, yes, coming in late (sorry for that), I seem to be missing something. How did the hacker get in? Was it simply a brute force at the SSH login? On March 5, 2021 2:41:35 PM UTC, James Knott <james.knott@jknott.net> wrote:
On 2021-03-05 9:39 a.m., Josef Moellers wrote:
Where else is a user allowed to write? The user doesn't have to write the file himself, it can also be created on his behalf, eg mailboxes, crontabs, atjobs, to name but a few.
I have deleted all files with that user's UID.
On 14/03/2021 15.43, ken wrote:
Having read through this long conversation, and, yes, coming in late (sorry for that), I seem to be missing something. How did the hacker get in? Was it simply a brute force at the SSH login?
Yes. They guessed the password, dictionary attack, of user "test". -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-14 10:43 a.m., ken wrote:
Having read through this long conversation, and, yes, coming in late (sorry for that), I seem to be missing something. How did the hacker get in? Was it simply a brute force at the SSH login?
It appears to be. I had been doing some testing and accidentally left ssh open through my firewall. I normally use OpenVPN to connect to my system.
It's interesting that that is a user ID that a hacker would try. Do the logs show other interesting user IDs that the hackers have tried? On March 14, 2021 3:13:38 PM UTC, James Knott <james.knott@jknott.net> wrote:
On 2021-03-14 10:43 a.m., ken wrote:
Having read through this long conversation, and, yes, coming in late (sorry for that), I seem to be missing something. How did the hacker get in? Was it simply a brute force at the SSH login?
It appears to be. I had been doing some testing and accidentally left ssh open through my firewall. I normally use OpenVPN to connect to my system.
Drop the link I want to throw that thing at IDA, sounds like cryptominer. Do you have bind/smb enabled with no master? On Tue, Mar 2, 2021 at 10:59 AM Per Jessen <per@computer.org> wrote:
James Knott wrote:
On 2021-03-02 10:49 a.m., Per Jessen wrote:
That didn't work. I just got a usage display. Try "rpm -qf/home/test/.dhcpd"
rpm -qf /home/test/.dhcpd error: file /home/test/.dhcpd: No such file or directory
My typo -
rpm -qf /home/test/.dhpcd
-- Per Jessen, Zürich (14.0°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
On 2021-03-02 11:04 a.m., Jonathan Arsenault wrote:
Drop the link I want to throw that thing at IDA, sounds like cryptominer. Do you have bind/smb enabled with no master?
The link doesn't work. The file gets identified as a virus, which only the owner can download. If you have some other safe way to provide the file I can try that.
On 02.03.2021 19:07, James Knott wrote:
On 2021-03-02 11:04 a.m., Jonathan Arsenault wrote:
Drop the link I want to throw that thing at IDA, sounds like cryptominer. Do you have bind/smb enabled with no master?
The link doesn't work. The file gets identified as a virus, which only the owner can download. If you have some other safe way to provide the file I can try that.
Password protected zip or any other archive format.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, March 2, 2021 8:04 AM, Jonathan Arsenault <jonharson@gmail.com> wrote:
Drop the link I want to throw that thing at IDA, sounds like cryptominer. Do you have bind/smb enabled with no master?
"rpm -f"? Gad. Did you notice the 'strings' suggestion? Or Per's question above? I think it's in everyone's interest to know how you caught this. Please post the results of: # lsof -i -n -w -P
On 02/03/2021 16.57, James Knott wrote:
On 2021-03-02 10:49 a.m., Per Jessen wrote:
That didn't work. I just got a usage display. Try "rpm -qf/home/test/.dhcpd"
rpm -qf /home/test/.dhcpd error: file /home/test/.dhcpd: No such file or directory
Is that the actual path to the malware? Or have you deleted it? If you have deleted it, create it again with touch, then repeat the rpm query.
However, I noticed it seems to be looking in whatever directory I run it from.
-- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 02/03/2021 17.58, James Knott wrote:
On 2021-03-02 11:55 a.m., Carlos E. R. wrote:
error: file/home/test/.dhcpd: No such file or directory Is that the actual path to the malware? Or have you deleted it? If you have deleted it, create it again with touch, then repeat the rpm query.
That is where I found it.
Weird. I would have expected this: cer@Telcontar:~> touch .dhcpd cer@Telcontar:~> rpm -qf /home/cer/.dhcpd file /home/cer/.dhcpd is not owned by any package cer@Telcontar:~> Are you running rpm as user or as root? Perhaps you do not have permissions to read the file. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 12:03 p.m., Carlos E.R. wrote:
re you running rpm as user or as root? Perhaps you do not have permissions to read the file.
I did it as root, as it was in a different user account and I didn't want to log in as that user, in case it would run something. I used su - from my main account.
On 02/03/2021 16.45, James Knott wrote:
On 2021-03-02 10:41 a.m., Carlos E. R. wrote:
rpm -f/home/test/.dhcpd
to find out if it came in an rpm.
BTW, I have determined it is a virus.
It is malware. To be a virus it has to match certain criteria that I don't see here. I think it is a trojan. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-02 11:58 a.m., Carlos E.R. wrote:
BTW, I have determined it is a virus. It is malware. To be a virus it has to match certain criteria that I don't see here. I think it is a trojan.
Whichever. That's the message I got when I tried to download it from Google drive onto Windows. Also, I purposely used malware in the message title, as I didn't know what it was.
On 2021-03-02 9:27 a.m., Josef Moellers wrote:
You can have a look at what it's actually doing with "strace": strace -f -p $(pidof .dhcpd)
Also, have a look if it has any suspicious strings (attached;-) ): strings .dhcpd
First off, I killed .dhcpd before I posted my first message, so there isn't much to trace. Also, here's my result from strace: # strace -f -p $(pidof .dhcpd) strace: option requires an argument -- 'p' Try 'strace -h' for more information.
On 2021-03-02 08:17:09 James Knott wrote:
|Today, I noticed my computer performance was very poor. Top showed |something called .dhcpd running under user test and using almost 400% of |my CPU. In ~/test, I found an executable .dhcpd. I have not used the |test account for a long time. That .dhcpd also survived a reboot, so |I'll have to determine how it's starting. : |There were some updates yesterday, so I wonder if one of them is |infected with something. Also, this computer is configured with a |static IPv4 address and uses SLAAC on IPv6, so it shouldn't be using | dhcpd.
Wow, giant thread. Could you possibly post the resolution of this at the first level? Leslie -- openSUSE Leap 15.2 x86_64
On 02/03/2021 22.59, J Leslie Turriff wrote:
On 2021-03-02 08:17:09 James Knott wrote:
|Today, I noticed my computer performance was very poor. Top showed |something called .dhcpd running under user test and using almost 400% of |my CPU. In ~/test, I found an executable .dhcpd. I have not used the |test account for a long time. That .dhcpd also survived a reboot, so |I'll have to determine how it's starting. : |There were some updates yesterday, so I wonder if one of them is |infected with something. Also, this computer is configured with a |static IPv4 address and uses SLAAC on IPv6, so it shouldn't be using | dhcpd.
Wow, giant thread. Could you possibly post the resolution of this at the first level?
They guessed the password for user "test" and logged in via ssh, after multiple attempts (dictionary attack, probably). The user "test" was configured in the system long ago, and forgotten. They copied executable malware to "/home/test/.dhpcd" and added a cronjob: 2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B \
/dev/null 2>/dev/null
Thus, no vulnerability in the system, no compromised update. The root account was not compromised. Mitigations for the future: - Don't open ssh port 22, use a high port on a strange number (not 50000, for instance). - in /etc/ssh/sshd_config set "PasswordAuthentication no" after adding appropriate keys to ~/.ssh/ -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/2/21 4:03 PM, Carlos E.R. wrote:
Thus, no vulnerability in the system, no compromised update. The root account was not compromised.
Fortunately! I wonder if they tried to get root?
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily. Consider shodan.io for example. They have crawlers that map all ports on all servers.
- in /etc/ssh/sshd_config set "PasswordAuthentication no" after adding appropriate keys to ~/.ssh/
The file is ~/.ssh/authorized_keys by default. Setting up something like fail2ban would also be a good thing. Regards, Lew
On 03/03/2021 01.59, Lew Wolfgang wrote:
On 3/2/21 4:03 PM, Carlos E.R. wrote:
Thus, no vulnerability in the system, no compromised update. The root account was not compromised.
Fortunately! I wonder if they tried to get root?
Well, they had ssh in, so they could have tried to su. But it was not their objective, so not probable, but not impossible.
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily. Consider shodan.io for example. They have crawlers that map all ports on all servers.
Takes time. No one has hit my ssh yet. But the idea is to use all possible mitigations.
- in /etc/ssh/sshd_config set "PasswordAuthentication no" after adding appropriate keys to ~/.ssh/
The file is ~/.ssh/authorized_keys by default.
Whatever :-D Actual instructions are longer, you have to do things on both computers. Any one that wants to can ask for the complete instructions, or google ;-)
Setting up something like fail2ban would also be a good thing.
Yep. Or a rule I forgot about in the firewall: after three attempts with bad password, the IP is automatically blocked for some time. An iptables rule does it. I don't know how to do it with firewald, though. Only with SuSEfirewall2. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Lew Wolfgang wrote:
On 3/2/21 4:03 PM, Carlos E.R. wrote:
Thus, no vulnerability in the system, no compromised update. The root account was not compromised.
Fortunately! I wonder if they tried to get root?
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time. If ssh is to be open publicly, setting up public key authentication is the way to go.
Setting up something like fail2ban would also be a good thing.
Possibly - for 14-15 years, our firewall has automatically blocked new external accesses after a certain rate per minute, both for ssh, sip, and ftp traffic. That has worked well, but hackers have become very patient - brute force attacks was yesterday. Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban. -- Per Jessen, Zürich (5.8°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
On 03/03/2021 08.05, Per Jessen wrote:
Lew Wolfgang wrote:
On 3/2/21 4:03 PM, Carlos E.R. wrote:
Thus, no vulnerability in the system, no compromised update. The root account was not compromised.
Fortunately! I wonder if they tried to get root?
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time.
They have not found mine... :-)
If ssh is to be open publicly, setting up public key authentication is the way to go.
Setting up something like fail2ban would also be a good thing.
Possibly - for 14-15 years, our firewall has automatically blocked new external accesses after a certain rate per minute, both for ssh, sip, and ftp traffic. That has worked well, but hackers have become very patient - brute force attacks was yesterday. Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban.
Well, security is not really about doing only one thing, we have to do several. One thing alone doesn't stop them. Use everything you can: change the port, set up public keys, dynamic firewalling, etc. Some of the measures will not stop a determined attack, but it does a script kiddie. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/3/21 3:52 AM, Carlos E. R. wrote:
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance). Changing ports might help a bit, but dedicated hackers can discover moved ports easily. Absolutely. It is only a matter of time. They have not found mine...:-)
How are you so sure? Maybe they're using an nmap "stealth" scan and know about your open ports? Maybe they just haven't tried to connect yet? Regards, Lew
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2021-03-03 at 08:04 -0800, Lew Wolfgang wrote:
On 3/3/21 3:52 AM, Carlos E. R. wrote:
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time.
They have not found mine... :-)
How are you so sure? Maybe they're using an nmap "stealth" scan and know about your open ports? Maybe they just haven't tried to connect yet?
Because an ssh attempt is logged. grep shd /var/log/messages*z | grep port | egrep -v "192.168" | less The only strange entries are some like this: <4.6> 2020-03-07T09:39:57.547577+01:00 Isengard sshd 6958 - - Bad protocol version identification '\003' from 45.148.121.4 port 63970 <4.6> 2020-03-07T11:48:20.709325+01:00 Isengard sshd 10492 - - Bad protocol version identification '\003' from 45.148.121.4 port 63187 <4.6> 2020-03-07T15:21:02.069546+01:00 Isengard sshd 16327 - - Bad protocol version identification '\003' from 195.54.166.115 port 63442 <4.6> 2020-03-07T15:36:13.889905+01:00 Isengard sshd 16744 - - Bad protocol version identification '\003' from 195.54.166.115 port 62613 <4.6> 2020-03-07T15:49:21.625882+01:00 Isengard sshd 17100 - - Bad protocol version identification '\003' from 195.54.166.115 port 63678 <4.6> 2020-03-21T17:05:21.639712+01:00 Isengard sshd 25854 - - Bad protocol version identification '\003' from 194.61.26.163 port 1145 <4.6> 2020-04-07T14:35:53.062798+02:00 Isengard sshd 31626 - - Bad protocol version identification '\003' from 80.89.234.58 port 913 I have no idea what that is. These are the last: <4.6> 2021-01-02T02:58:24.470179+01:00 Isengard sshd 6839 - - Did not receive identification string from 194.36.85.126 port 12770 <4.6> 2021-01-05T19:19:46.729422+01:00 Isengard sshd 14602 - - Bad protocol version identification '\003' from 89.248.165.25 port 63861 (all are from Europe, not China ;-) ) - -- Cheers, Carlos E. R. (from openSUSE 15.2 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCYD+8cRwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfVx2cAnj7XdbQPH+HNCzIwk96s mqQn3SkzAKCRqeqnoBwlQ8cLTZxDR3fmwWGJ+w== =p3yQ -----END PGP SIGNATURE-----
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wednesday, 2021-03-03 at 08:04 -0800, Lew Wolfgang wrote:
On 3/3/21 3:52 AM, Carlos E. R. wrote:
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time.
They have not found mine... :-)
How are you so sure? Maybe they're using an nmap "stealth" scan and know about your open ports? Maybe they just haven't tried to connect yet?
Because an ssh attempt is logged.
Maybe re-read what Lew wrote :-)
The only strange entries are some like this:
<4.6> 2020-03-07T09:39:57.547577+01:00 Isengard sshd 6958 - - Bad protocol version identification '\003' from 45.148.121.4 port 63970 [snip]
Someone knows about an open port. -- Per Jessen, Zürich (8.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland.
On 3/3/21 8:42 AM, Carlos E. R. wrote:
On Wednesday, 2021-03-03 at 08:04 -0800, Lew Wolfgang wrote:
On 3/3/21 3:52 AM, Carlos E. R. wrote:
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time. They have not found mine... :-)
How are you so sure? Maybe they're using an nmap "stealth" scan and know about your open ports? Maybe they just haven't tried to connect yet?
Because an ssh attempt is logged.
grep shd /var/log/messages*z | grep port | egrep -v "192.168" | less
But nmap stealth scans don't leave log entries. I'm sure that someone has swept all your ports without you knowing about it.
The only strange entries are some like this:
<4.6> 2020-03-21T17:05:21.639712+01:00 Isengard sshd 25854 - - Bad protocol version identification '\003' from 194.61.26.163 port 1145 <4.6> 2020-04-07T14:35:53.062798+02:00 Isengard sshd 31626 - - Bad protocol version identification '\003' from 80.89.234.58 port 913
I have no idea what that is.
These are the last:
<4.6> 2021-01-02T02:58:24.470179+01:00 Isengard sshd 6839 - - Did not receive identification string from 194.36.85.126 port 12770 <4.6> 2021-01-05T19:19:46.729422+01:00 Isengard sshd 14602 - - Bad protocol version identification '\003' from 89.248.165.25 port 63861
(all are from Europe, not China ;-) )
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right? Maybe these are shodan scans? Another thing you might be able to do is use the "AllowUsers" parameter in /etc/ssh/sshd_config. Any login attempts to users not listed are rejected by sshd. You can even list an IP address, as in "wolfgang@w.x.t.z", or even "wolfgang@w.x.y.0/24" as examples. I just checked and I've got 942 entries like the following in the past four days, most from different IP's. "sshd[6236]: User root from 220.248.95.178 not allowed because not listed in AllowUsers" This was from China, btw. Regards, Lew
Lew Wolfgang wrote:
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right?
I would say so.
I just checked and I've got 942 entries like the following in the past four days, most from different IP's.
Oh yeah, the threat is certainly real. In February, our external (rented) servers had about 4-5000 such attempts, each. All configured for public key only, even so listening on a high port. Always with big time gaps, 5-10-15 minutes - true brute force was yesteryear. 44'244 attempts over twelve specific servers, 14'963 unique ipv4 addresses from 141 countries. -- Per Jessen, Zürich (6.2°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
On 03/03/2021 20.24, Per Jessen wrote:
Lew Wolfgang wrote:
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right?
I would say so.
I just checked and I've got 942 entries like the following in the past four days, most from different IP's.
Oh yeah, the threat is certainly real. In February, our external (rented) servers had about 4-5000 such attempts, each. All configured for public key only, even so listening on a high port. Always with big time gaps, 5-10-15 minutes - true brute force was yesteryear.
44'244 attempts over twelve specific servers, 14'963 unique ipv4 addresses from 141 countries.
What do you use to analyze that? Ie, how do you know? -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Carlos E. R. wrote:
On 03/03/2021 20.24, Per Jessen wrote:
Lew Wolfgang wrote:
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right?
I would say so.
I just checked and I've got 942 entries like the following in the past four days, most from different IP's.
Oh yeah, the threat is certainly real. In February, our external (rented) servers had about 4-5000 such attempts, each. All configured for public key only, even so listening on a high port. Always with big time gaps, 5-10-15 minutes - true brute force was yesteryear.
44'244 attempts over twelve specific servers, 14'963 unique ipv4 addresses from 141 countries.
What do you use to analyze that? Ie, how do you know?
The plain old logfile - /var/log/messages. sshd will log failed login attempts. To analyze it, I grep through the files, then load the data into a database, makes it easier to query. -- Per Jessen, Zürich (5.6°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.
On 03/03/2021 23.24, Per Jessen wrote:
Carlos E. R. wrote:
On 03/03/2021 20.24, Per Jessen wrote:
Lew Wolfgang wrote:
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right?
I would say so.
I just checked and I've got 942 entries like the following in the past four days, most from different IP's.
Oh yeah, the threat is certainly real. In February, our external (rented) servers had about 4-5000 such attempts, each. All configured for public key only, even so listening on a high port. Always with big time gaps, 5-10-15 minutes - true brute force was yesteryear.
44'244 attempts over twelve specific servers, 14'963 unique ipv4 addresses from 141 countries.
What do you use to analyze that? Ie, how do you know?
The plain old logfile - /var/log/messages. sshd will log failed login attempts. To analyze it, I grep through the files, then load the data into a database, makes it easier to query.
Ah, ok, you load a database. Ok. My job at Lucent involved a lot of analyzing logs, we had specific tools to do it, but often I used grep. In fact, Lucent had their own specific version of grep, called cgrep (there is(was?) a packaged version of it on home:rbos) that made much easier to analyze their reports. A tool that would collect the failed connection attempts, then would automatically search for those IPs in the logs, would be nice. Cross referencing. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Carlos E.R. wrote:
A tool that would collect the failed connection attempts, then would automatically search for those IPs in the logs, would be nice. Cross referencing.
I don't think I quite understand. Finding the failed connection attempts and the origin IP-addresses is easy, but which logs do you then want to search? -- Per Jessen, Zürich (12.2°C)
On 04/03/2021 10.53, Per Jessen wrote:
Carlos E.R. wrote:
A tool that would collect the failed connection attempts, then would automatically search for those IPs in the logs, would be nice. Cross referencing.
I don't think I quite understand. Finding the failed connection attempts and the origin IP-addresses is easy, but which logs do you then want to search?
/var/log/messages, for instance. Locate any other occurrence of the same IP, possibly in a time range, to find out if that IP that tried to connect to something else. If you have running services that use other logs, like apache, then read those, too. If the machine is external, not NATted like mine, it will receive scans directly. Thus, suppose you get a failed connect attempt at some service. The tool registers that IP, and finds out that he is trying other services at one hour intervals. Well, the tool would then detect slow scans. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Carlos E. R. wrote:
On 04/03/2021 10.53, Per Jessen wrote:
Carlos E.R. wrote:
A tool that would collect the failed connection attempts, then would automatically search for those IPs in the logs, would be nice. Cross referencing.
I don't think I quite understand. Finding the failed connection attempts and the origin IP-addresses is easy, but which logs do you then want to search?
/var/log/messages, for instance. Locate any other occurrence of the same IP, possibly in a time range, to find out if that IP that tried to connect to something else. If you have running services that use other logs, like apache, then read those, too.
Oh okay. I guess it might be worth a try, it's easily done if you load the extracted data into a database.
If the machine is external, not NATted like mine, it will receive scans directly. Thus, suppose you get a failed connect attempt at some service. The tool registers that IP, and finds out that he is trying other services at one hour intervals. Well, the tool would then detect slow scans.
Well, the analysis already does, but after the fact. I don't know what other services might be open to an attack though. -- Per Jessen, Zürich (13.2°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland.
On 04/03/2021 13.42, Per Jessen wrote:
Carlos E. R. wrote:
On 04/03/2021 10.53, Per Jessen wrote:
Carlos E.R. wrote:
A tool that would collect the failed connection attempts, then would automatically search for those IPs in the logs, would be nice. Cross referencing.
I don't think I quite understand. Finding the failed connection attempts and the origin IP-addresses is easy, but which logs do you then want to search?
/var/log/messages, for instance. Locate any other occurrence of the same IP, possibly in a time range, to find out if that IP that tried to connect to something else. If you have running services that use other logs, like apache, then read those, too.
Oh okay. I guess it might be worth a try, it's easily done if you load the extracted data into a database.
Several syslog tools directly support databases. Maybe it is possible to log to a database any entry that contains an IP address. A later process would purge from the database those entries that have no interest and the filter be refined.
If the machine is external, not NATted like mine, it will receive scans directly. Thus, suppose you get a failed connect attempt at some service. The tool registers that IP, and finds out that he is trying other services at one hour intervals. Well, the tool would then detect slow scans.
Well, the analysis already does, but after the fact. I don't know what other services might be open to an attack though.
Yes, of course, after the fact. At best, some hours later. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 03/03/2021 18.33, Lew Wolfgang wrote:
On 3/3/21 8:42 AM, Carlos E. R. wrote:
On Wednesday, 2021-03-03 at 08:04 -0800, Lew Wolfgang wrote:
On 3/3/21 3:52 AM, Carlos E. R. wrote:
Mitigations for the future:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Changing ports might help a bit, but dedicated hackers can discover moved ports easily.
Absolutely. It is only a matter of time. They have not found mine... :-)
How are you so sure? Maybe they're using an nmap "stealth" scan and know about your open ports? Maybe they just haven't tried to connect yet?
Because an ssh attempt is logged.
grep shd /var/log/messages*z | grep port | egrep -v "192.168" | less
But nmap stealth scans don't leave log entries. I'm sure that someone has swept all your ports without you knowing about it.
All the ports of the external router, it is possible and I will not know because my router says nothing. But that is not where ssh is. If something attempts to actually connect to my ssh machine at the correct port, the ssh machine logs the attempt and the failure or success. And that (at the time I last looked) had not happened. attempts on other ports will not reach the ssh daemon on another machine...
The only strange entries are some like this:
<4.6> 2020-03-21T17:05:21.639712+01:00 Isengard sshd 25854 - - Bad protocol version identification '\003' from 194.61.26.163 port 1145 <4.6> 2020-04-07T14:35:53.062798+02:00 Isengard sshd 31626 - - Bad protocol version identification '\003' from 80.89.234.58 port 913
I have no idea what that is.
These are the last:
<4.6> 2021-01-02T02:58:24.470179+01:00 Isengard sshd 6839 - - Did not receive identification string from 194.36.85.126 port 12770 <4.6> 2021-01-05T19:19:46.729422+01:00 Isengard sshd 14602 - - Bad protocol version identification '\003' from 89.248.165.25 port 63861
(all are from Europe, not China ;-) )
Well, that you have any entries at all from sshd means that something is poking at the port it's listening on, right? Maybe these are shodan scans?
I don't know what they are. The entries list an IP and a port. At that port there is nothing listening on my machine, so maybe it is the egress port. This is just the thing: <https://unix.stackexchange.com/questions/573275/what-kind-of-an-sshd-attack-is-this> Basically, they are saying that they are scanning hhtp servers, and when they hit by chance the sshd NATted port, the sshd server sees it and complains. The other one: "Did not receive identification string from" seems to happen from providers of virtual servers checking if you are alive - but this is not a virtual server. It can be simply an aborted connection attempt (nmap) or an attempt at a known vulnerability, documented in "/etc/fail2ban/filter.d/sshd-ddos.conf" <https://serverfault.com/questions/483852/sshd-log-full-of-did-not-receive-identification-string-from>
Another thing you might be able to do is use the "AllowUsers" parameter in /etc/ssh/sshd_config. Any login attempts to users not listed are rejected by sshd. You can even list an IP address, as in "wolfgang@w.x.t.z", or even "wolfgang@w.x.y.0/24" as examples.
No, I can not list an IP address, as I do not know in advance what address I would use to connect from. The daemon is using publickkey, so I don't think they can guess easily the correct user, but yes, you are right, I could limit them. Problem is limiting from outside and not from inside. Ah, of course, connection is impossible if the attempted user has no publickey defined, which in effect is the same as listing the users.
I just checked and I've got 942 entries like the following in the past four days, most from different IP's.
"sshd[6236]: User root from 220.248.95.178 not allowed because not listed in AllowUsers"
This was from China, btw.
Regards, Lew
-- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/3/21 11:47 AM, Carlos E. R. wrote:
All the ports of the external router, it is possible and I will not know because my router says nothing. But that is not where ssh is. If something attempts to actually connect to my ssh machine at the correct port, the ssh machine logs the attempt and the failure or success. And that (at the time I last looked) had not happened.
But something woke up your ssh daemon, you showed us the log entries. I'd guess that you're seeing aborted login attempts that don't get to the point of supplying a userid. Maybe our Chinese friends are just sniffing around? Regards, Lew
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2021-03-03 at 12:46 -0800, Lew Wolfgang wrote:
On 3/3/21 11:47 AM, Carlos E. R. wrote:
All the ports of the external router, it is possible and I will not know because my router says nothing. But that is not where ssh is. If something attempts to actually connect to my ssh machine at the correct port, the ssh machine logs the attempt and the failure or success. And that (at the time I last looked) had not happened.
But something woke up your ssh daemon, you showed us the log entries. I'd guess that you're seeing aborted login attempts that don't get to the point of supplying a userid. Maybe our Chinese friends are just sniffing around?
No, not really. The "Bad protocol version identification" are, apparently, attempts to find an http server on other ports, and hitting instead the ssh port. There is no attempt later to login on ssh. None of those comes from China: several come from the Netherlands :-p This is the one that tries ssh: Did not receive identification string from 194.36.85.126 port 12770 and does not try to login because that error is produced earlier in the negotiation. It is rather a probe of the existence and alivenes of an ssh server (and my IP is not fixed, so it doesn't matter). There are few in the entire log: cer@Isengard:~> zgrep sshd /var/log/messages*z | grep "Did not receive identification string from" /var/log/messages-20200909.xz:<4.6> 2020-07-31T16:16:13.521585+02:00 Isengard sshd 1137 - - Did not receive identification string from 185.176.27.190 port 62000 /var/log/messages-20200909.xz:<4.6> 2020-08-01T19:27:06.749153+02:00 Isengard sshd 11473 - - Did not receive identification string from 185.176.27.86 port 62000 /var/log/messages-20200909.xz:<4.6> 2020-08-15T18:50:07.476866+02:00 Isengard sshd 4601 - - Did not receive identification string from 142.93.116.72 port 51906 /var/log/messages-20201025.xz:<4.6> 2020-10-06T10:56:30.354699+02:00 Isengard sshd 30444 - - Did not receive identification string from 185.193.90.86 port 62000 /var/log/messages-20201025.xz:<4.6> 2020-10-10T19:09:05.197476+02:00 Isengard sshd 30599 - - Did not receive identification string from 80.82.65.74 port 34970 /var/log/messages-20201025.xz:<4.6> 2020-10-10T19:09:05.415243+02:00 Isengard sshd 30601 - - Did not receive identification string from 80.82.65.74 port 37092 /var/log/messages-20201025.xz:<4.6> 2020-10-10T19:09:05.637049+02:00 Isengard sshd 30602 - - Did not receive identification string from 80.82.65.74 port 39174 /var/log/messages-20210106.xz:<4.6> 2020-12-08T00:57:49.171469+01:00 Isengard sshd 20188 - - Did not receive identification string from 80.82.65.74 port 35436 /var/log/messages-20210106.xz:<4.6> 2020-12-08T00:57:49.392584+01:00 Isengard sshd 20189 - - Did not receive identification string from 80.82.65.74 port 38132 /var/log/messages-20210106.xz:<4.6> 2020-12-08T00:57:49.623665+01:00 Isengard sshd 20190 - - Did not receive identification string from 80.82.65.74 port 40424 /var/log/messages-20210106.xz:<4.6> 2021-01-02T02:58:24.470179+01:00 Isengard sshd 6839 - - Did not receive identification string from 194.36.85.126 port 12770 cer@Isengard:~> Two from Russia, one from the USA, another from Turkey. And this one is interesting: cer@Isengard:~> host 80.82.65.74 74.65.82.80.in-addr.arpa domain name pointer scanner.openportstats.com. cer@Isengard:~> That one might be me requesting a scanner on myself. Possibly there might be a reference to this in some mail list at those days, but I have not looked :-) - -- Cheers, Carlos E. R. (from openSUSE 15.2 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHIEARECADIWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCYEAHChQccm9iaW4ubGlz dGFzQGdteC5lcwAKCRC1MxgcbY1H1b0OAKCQJGog5j3/cz6D3YA3UnIlwo2UrwCg iJvf12XibDF/y+xMciZTk0BOZIM= =cG58 -----END PGP SIGNATURE-----
On 3/3/21 2:00 PM, Carlos E. R. wrote:
cer@Isengard:~> host 80.82.65.74 74.65.82.80.in-addr.arpa domain name pointer scanner.openportstats.com. cer@Isengard:~>
That one might be me requesting a scanner on myself. Possibly there might be a reference to this in some mail list at those days, but I have not looked :-)
Try going to openportstats.com and enter your router's wan IP address, then let us know what you see. They couldn't find anything on my home router, but they have lots of info on the open ports of my server. BTW, they don't need permission to scan you. Regards, Lew
On 03/03/2021 23.52, Lew Wolfgang wrote:
On 3/3/21 2:00 PM, Carlos E. R. wrote:
cer@Isengard:~> host 80.82.65.74 74.65.82.80.in-addr.arpa domain name pointer scanner.openportstats.com. cer@Isengard:~>
That one might be me requesting a scanner on myself. Possibly there might be a reference to this in some mail list at those days, but I have not looked :-)
Try going to openportstats.com and enter your router's wan IP address, then let us know what you see.
"No results" They don't do an actual scan when I ask, but look on their database - that is useless, the IP is dynamic, so they could have scanned me or any other Telefónica client.
They couldn't find anything on my home router, but they have lots of info on the open ports of my server.
BTW, they don't need permission to scan you.
Well... I'm not sure about that. Maybe not in the country they are based, but they are trespassing on other countries and jurisdictions. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/3/21 5:05 PM, Carlos E.R. wrote:
On 03/03/2021 23.52, Lew Wolfgang wrote:
cer@Isengard:~> host 80.82.65.74 74.65.82.80.in-addr.arpa domain name pointer scanner.openportstats.com. cer@Isengard:~>
That one might be me requesting a scanner on myself. Possibly there might be a reference to this in some mail list at those days, but I have not looked :-) Try going to openportstats.com and enter your router's wan IP address,
On 3/3/21 2:00 PM, Carlos E. R. wrote: then let us know what you see. "No results"
They don't do an actual scan when I ask, but look on their database - that is useless, the IP is dynamic, so they could have scanned me or any other Telefónica client.
How often does your IP change? I've seen mine not change for years.
They couldn't find anything on my home router, but they have lots of info on the open ports of my server.
BTW, they don't need permission to scan you. Well... I'm not sure about that. Maybe not in the country they are based, but they are trespassing on other countries and jurisdictions.
The commercial scan services collect only publicly available data, like which ports are open, and banners from open ports. You or me scanning would probably violate our use agreements with our ISP's, but that wouldn't be a problem for a business who's their own ISP. Even if a jurisdiction had a specific law regarding scanning, I doubt if it could, or would, be enforced. I could fat-finger a url and accidentally scan someone. Would an arrest warrant for my IP address be issued? Shodan.io never asked me for permission. And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed. Regards, Lew
On 04/03/2021 05.22, Lew Wolfgang wrote:
On 3/3/21 5:05 PM, Carlos E.R. wrote:
On 03/03/2021 23.52, Lew Wolfgang wrote:
cer@Isengard:~> host 80.82.65.74 74.65.82.80.in-addr.arpa domain name pointer scanner.openportstats.com. cer@Isengard:~>
That one might be me requesting a scanner on myself. Possibly there might be a reference to this in some mail list at those days, but I have
not looked :-) Try going to openportstats.com and enter your router's wan IP address,
On 3/3/21 2:00 PM, Carlos E. R. wrote: then let us know what you see. "No results"
They don't do an actual scan when I ask, but look on their database - that is useless, the IP is dynamic, so they could have scanned me or any other Telefónica client.
How often does your IP change? I've seen mine not change for years.
Any time the router restarts. The other day the router was working but the connection was down, so I had to power cycle the entire setup, and the IP changed again. For instance, this year it has changed four times. By the way. The other day I could not reach my own apache server (on a dynamic name). After many tests, I found that it failed with Firefox, but worked with Chrome. Whaaat? Apparently, Firefox (on phone) was using its own dns over http service and hitting the IP I had a week ago, while Chrome was hitting the current IP. It was a scare, because my testing revealed open ports in "my" router that shouldn't be. Well, they were somebody else's router (in the same ISP, same router brand). Meaning that the the DNS over http service of Firefox was using stale DNS information.
They couldn't find anything on my home router, but they have lots of info on the open ports of my server.
BTW, they don't need permission to scan you. Well... I'm not sure about that. Maybe not in the country they are based, but they are trespassing on other countries and jurisdictions.
The commercial scan services collect only publicly available data, like which ports are open, and banners from open ports. You or me scanning would probably violate our use agreements with our ISP's, but that wouldn't be a problem for a business who's their own ISP. Even if a jurisdiction had a specific law regarding scanning, I doubt if it could, or would, be enforced. I could fat-finger a url and accidentally scan someone. Would an arrest warrant for my IP address be issued? Shodan.io never asked me for permission.
An arrest no... a reprimand more likely. And very difficult to enforce, practically impossible.
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed.
It can, certainly. Which is why I also use publickey. I can not detect scans, my router is too dumb for that. I can only see them if they hit directly the right port, as you saw in my logs. And it happens sparsely. I thought it was never, because the last time I studied the logs I did not see them. Having a dynamic IP helps. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021-03-04 7:31 a.m., Carlos E. R. wrote:
How often does your IP change? I've seen mine not change for years. Any time the router restarts. The other day the router was working but the connection was down, so I had to power cycle the entire setup, and the IP changed again.
WOW! Mine changes so seldom it's virtually static. Also, my host name is based on my firewall & modem MAC addresses, so it doesn't change unless I change hardware. My IPv6 prefix is also consistent. It took replacing my firewall computer & fresh pfsense install to make it change.
On 04/03/2021 14.36, James Knott wrote:
On 2021-03-04 7:31 a.m., Carlos E. R. wrote:
How often does your IP change? I've seen mine not change for years. Any time the router restarts. The other day the router was working but the connection was down, so I had to power cycle the entire setup, and the IP changed again.
WOW! Mine changes so seldom it's virtually static. Also, my host name is based on my firewall & modem MAC addresses, so it doesn't change unless I change hardware. My IPv6 prefix is also consistent. It took replacing my firewall computer & fresh pfsense install to make it change.
My ISP is capable of claiming they do it for backward compatibility with ADSL and modems :-p I suspect they will do the same when they implement IPv6. I would even bet on it, except that I never bet :-) -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed.
Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts. On another, also installed in 2015, attempts started 1 November 2020, since then 136680. -- Per Jessen, Zürich (12.9°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.
On 04/03/2021 14.12, Per Jessen wrote:
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed.
Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts.
On another, also installed in 2015, attempts started 1 November 2020, since then 136680.
Static IPs? We might have to go one step further: migrate the port randomly and periodically. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Carlos E. R. wrote:
On 04/03/2021 14.12, Per Jessen wrote:
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed.
Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts.
On another, also installed in 2015, attempts started 1 November 2020, since then 136680.
Static IPs?
They are servers, so yes.
We might have to go one step further: migrate the port randomly and periodically.
As I have been trying to explain, it does not add anything. Just use public key auth. -- Per Jessen, Zürich (14.2°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.
On 04/03/2021 14.33, Per Jessen wrote:
Carlos E. R. wrote:
On 04/03/2021 14.12, Per Jessen wrote:
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed.
Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts.
On another, also installed in 2015, attempts started 1 November 2020, since then 136680.
Static IPs?
They are servers, so yes.
We might have to go one step further: migrate the port randomly and periodically.
As I have been trying to explain, it does not add anything. Just use public key auth.
On my case, it does help. Hits are very rare. Yes, of course I use public key auth, since day one. Changing the port periodically just reduces noise in the logs. Would be akin to changing the password periodically :-) -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/4/21 5:26 AM, Carlos E. R. wrote:
On 04/03/2021 14.12, Per Jessen wrote:
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed. Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts.
On another, also installed in 2015, attempts started 1 November 2020, since then 136680. Static IPs?
We might have to go one step further: migrate the port randomly and periodically.
If you're really that paranoid, take a look at port knocking: https://www.howtogeek.com/442733/how-to-use-port-knocking-on-linux-and-why-y... Even that is security through obscurity, but an argument could be made that multiple levels of obscurity can only help. Regards, Lew
On 04/03/2021 17.49, Lew Wolfgang wrote:
On 3/4/21 5:26 AM, Carlos E. R. wrote:
On 04/03/2021 14.12, Per Jessen wrote:
Lew Wolfgang wrote:
And of course botnets also scan for open ports, and not just the obvious ones. I just checked on my underused server with a direct and static connection, and over a 75-minute period I had 869 ports scanned, 637 of them were unique ephemeral high-numbered ports. Botnets don't care about laws, and if your IP is static for very long, your hidden ssh port could be exposed. Yep, I can assure you they will be. Even if we use public key authentication, we still run ssh on a high port - on one server installed in 2015, there were no ssh attacks until 7 February 2021. Since 2907 login attempts.
On another, also installed in 2015, attempts started 1 November 2020, since then 136680. Static IPs?
We might have to go one step further: migrate the port randomly and periodically.
If you're really that paranoid, take a look at port knocking:
https://www.howtogeek.com/442733/how-to-use-port-knocking-on-linux-and-why-y...
Interesting. sequence: The sequence of ports someone must access to open or close port 22. The default ports are 7000, 8000, and 9000 to open it, and 9000, 8000, and 7000 to close it. You can change these or add more ports to the list. For our purposes, we’ll stick with the defaults. However, there is a router in front of my server machine. The current configuration is that, from internet I connect using ssh to some high port, and the router changes that incoming connection to another on port 22 to the correct machine on the LAN. To do what that article suggests, I would have to add three more rules to my router, which has a limited and arbitrarily small number of rules I can write. Not very feasible, but something to consider if the situation worsens.
Even that is security through obscurity, but an argument could be made that multiple levels of obscurity can only help.
Sure. :-) Another design could be sending innocuous looking pings to some predefined port and in some time sequence, perhaps with certain payload. If things get bad, we could change altogether the entry system to the ssh to some type of challenge response method, where what is sent changes every time, and doesn't rely on encryption (but can use it). Or combine with a phone. Two factor security. One method used long ago was to call (via phone) the server, the server would hang and call you back on a predefined number. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021/03/02 23:05, Per Jessen wrote:
Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban.
Today or rather from about five or more years ago, most sensitive institutions give about 3 invalid password attempts against an account and then lock the account, requiring a call to support with your personally identifying info. Seems like that would stop such attacks as the trigger is any 3 bad attempts and then the account is locked, no?
L A Walsh wrote:
On 2021/03/02 23:05, Per Jessen wrote:
Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban.
Today or rather from about five or more years ago, most sensitive institutions give about 3 invalid password attempts against an account and then lock the account, requiring a call to support with your personally identifying info.
Seems like that would stop such attacks as the trigger is any 3 bad attempts and then the account is locked, no?
I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts. -- Per Jessen, Zürich (3.9°C)
On 3/5/21 8:36 AM, Per Jessen wrote:
L A Walsh wrote:
On 2021/03/02 23:05, Per Jessen wrote:
Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban.
Today or rather from about five or more years ago, most sensitive institutions give about 3 invalid password attempts against an account and then lock the account, requiring a call to support with your personally identifying info.
Seems like that would stop such attacks as the trigger is any 3 bad attempts and then the account is locked, no? I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts.
Others do. Also, Smartcards will lock themselves after three failed pin attempts, requiring that the card be brought to a facility to unlock it after the bearer proves who they are. Regards, Lew
Lew Wolfgang wrote:
On 3/5/21 8:36 AM, Per Jessen wrote:
L A Walsh wrote:
On 2021/03/02 23:05, Per Jessen wrote:
Today it is slow, thorough, distributed - maybe 50 machines slowly trying out passwords, once a minute, one machine after the other. Such slow, patient attacks usually don't trigger any traps or fail2ban.
Today or rather from about five or more years ago, most sensitive institutions give about 3 invalid password attempts against an account and then lock the account, requiring a call to support with your personally identifying info.
Seems like that would stop such attacks as the trigger is any 3 bad attempts and then the account is locked, no? I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts.
Others do.
I just foresee the situation where and 'admin' account is locked out and support has gone home for the weekend.
Also, Smartcards will lock themselves after three failed pin attempts, requiring that the card be brought to a facility to unlock it after the bearer proves who they are.
Yes, much the same with mobile phones, but that is another level of security. Besides, with public key auth, why bother? -- Per Jessen, Zürich (4.0°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
Le 05/03/2021 à 17:36, Per Jessen a écrit :
I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts.
possible if the login is a private long thing, where it's unlikely it can be found, else any attempt by a non authorized user lock out the real user jdd -- http://dodin.org
On 2021/03/05 08:58, jdd@dodin.org wrote:
Le 05/03/2021 à 17:36, Per Jessen a écrit :
I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts.
possible if the login is a private long thing, where it's unlikely it can be found, else any attempt by a non authorized user lock out the real user
---- The financial institutions really don't seem to have a problem with some random hacker trying to DOS all their customers. Maybe they have other policies in place to detect and block such attempts. Or maybe their customers prefer having to call in to support rather than losing a few million dollars. Note these are web-logins/password, not ones where they can hammer a specific port. It's a bit harder to automate a web-account password when you have to remember your secure picture and answer one of your secret questions in addition to the password. Instead of locking you out, it might simply force you to re-enter your some account information -- like 3 attempts on an ssh password, and you have to login to the web-access page and identify your secure picture after receiving a 1-time code at your local email address. Just having to switch modalities from host+port to web and email to re-enable host+port login is sufficiently messy to dissuade very many attempts against a specific account. ======== On 2021/03/05 09:10, Per Jessen wrote: I just foresee the situation where and 'admin' account is locked out and support has gone home for the weekend. --- First thing -- the institutions/organizations that have such security measures have teams answering phones 24/7, holidays included. There are no weekends. Second thing -- as for someone having their admin account open to password cracking facing the web -- that shouldn't happen to begin with. With a minimal security policy, you couldn't access an admin account directly from the external web. You'd have to first login to a internet access account to a bastion machine, then try for a user login to an admin-capable machine and from there elevate to an admin account. The admin account would never be exposed to the outside internet such that it would be able to be probed from the outside net -- actually not likely from the inside net either -- only an internal user account would be able to acquire admin privs on some specific machine.
L A Walsh wrote:
On 2021/03/05 08:58, jdd@dodin.org wrote:
Le 05/03/2021 à 17:36, Per Jessen a écrit :
I'm not sure if I would even contemplate disabling a Linux user account based on 3 bad attempts.
possible if the login is a private long thing, where it's unlikely it can be found, else any attempt by a non authorized user lock out the real user
----
The financial institutions really don't seem to have a problem with some random hacker trying to DOS all their customers. Maybe they have other policies in place to detect and block such attempts.
A DOS attack is something else ?
Note these are web-logins/password, not ones where they can hammer a specific port.
It's a bit harder to automate a web-account password when you have to remember your secure picture and answer one of your secret questions in addition to the password.
Instead of locking you out, it might simply force you to re-enter your some account information -- like 3 attempts on an ssh password, and you have to login to the web-access page and identify your secure picture after receiving a 1-time code at your local email address.
Is there a reason public key auth isn't good enough?
======== On 2021/03/05 09:10, Per Jessen wrote: I just foresee the situation where and 'admin' account is locked out and support has gone home for the weekend. ---
First thing -- the institutions/organizations that have such security measures have teams answering phones 24/7, holidays included.
So why are we talking about here on this list?
There are no weekends. Second thing -- as for someone having their admin account open to password cracking facing the web -- that shouldn't happen to begin
We were not talking about the web, but about ssh. -- Per Jessen, Zürich (4.1°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland.
On 2021/03/05 10:03, Per Jessen wrote:
Is there a reason public key auth isn't good enough? Dunno, I didn't design it. Just describing security systems used as an adjunct to ssh access.
======== On 2021/03/05 09:10, Per Jessen wrote: I just foresee the situation where and 'admin' account is locked out and support has gone home for the weekend. ---
First thing -- the institutions/organizations that have such security measures have teams answering phones 24/7, holidays included.
So why are we talking about here on this list?
---- Why are you taking about support going home for the weekend?
There are no weekends. Second thing -- as for someone having their admin account open to password cracking facing the web -- that shouldn't happen to begin
We were not talking about the web, but about ssh.
Same same. In this context, facing open web = facing public internet. Talking about disabling accounts after 3 broken password attempts and why admin accounts wouldn't be vulnerable -- mainly, they aren't accessible via a public facing internet port.
On 06/03/2021 21.05, L A Walsh wrote:
Talking about disabling accounts after 3 broken password attempts and why admin accounts wouldn't be vulnerable -- mainly, they aren't accessible via a public facing internet port.
Think of the team that do the remote maintenance of the openSUSE infrastructure, like the mail list, the forum, the repository servers. They are admins. If they can not get in on a Saturday, the mail list might stay out for the weekend :-p -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
L A Walsh wrote:
On 2021/03/05 10:03, Per Jessen wrote:
Is there a reason public key auth isn't good enough?
Dunno, I didn't design it.
Me neither, but for some fifteeen years, it has been very good at preventing unauthorized access to my systems. If you don't know if it is good enough, maybe you need to explore it?
======== On 2021/03/05 09:10, Per Jessen wrote: I just foresee the situation where and 'admin' account is locked out and support has gone home for the weekend. ---
First thing -- the institutions/organizations that have such security measures have teams answering phones 24/7, holidays included.
So why are we talking about here on this list?
---- why are you taking about support going home for the weekend?
Well, because they do - I don't pay them to work on weekends. If a user account were to be locked due to third parties trying to gain access, that would become a simple support case. For Monday morning.
There are no weekends. Second thing -- as for someone having their admin account open to password cracking facing the web -- that shouldn't happen to begin
We were not talking about the web, but about ssh.
Same same. In this context, facing open web = facing public internet.
Plonk. -- Per Jessen, Zürich (0.7°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland.
On 3/3/21 1:03 AM, Carlos E.R. wrote:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Security through poor obscurity. You have only 16bits there. Easy to find. They scan entire 32bit internet already and the 16 bits is easy too. Even my little VM has attempts on high port. If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet. - Adam
On 2021-03-04 11:53 a.m., Adam Majer wrote:
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet.
I don't know what you mean by unique for ssh but, with privacy addresses, the address used for outgoing connections is not the same as the one used for incoming. There is one consistent address, often based on the MAC address, and up to seven temporary privacy addresses, with a new one every day. There's no way anyone could determine the ssh server address from any privacy address. The best they could do is narrow(?) it down to 1 in 18.4 billion, billion.
On 3/4/21 6:00 PM, James Knott wrote:
On 2021-03-04 11:53 a.m., Adam Majer wrote:
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet.
I don't know what you mean by unique for ssh but, with privacy addresses, the address used for outgoing connections is not the same as the one used for incoming. There is one consistent address, often based
Yeah, obviously you need an incoming address which is static. So, if you have some /64 assigned, then pick something in the range, assign to your interface and make it listening address for SSH-only. Privacy addresses are for outgoing connections only, though they are not really private (you are as identifiable as with current ipv4 address). And legacy programs have problems handling these addresses in the first place so people tend to turn them off. It was a nice idea but with little benefit. - Adam
it should be possible to use random port, based on algorithm like the one used by banks and other dealers that send us a random number by sms: listen to a port with a given number, only at a given date&time if the key is common between client and server, one can know what port it is, and as this port is used only once in a while, can't be guessed, and the rest of the time ssh is not even listening... jdd -- http://dodin.org
On 2021-03-04 12:07 p.m., Adam Majer wrote:
Privacy addresses are for outgoing connections only, though they are not really private (you are as identifiable as with current ipv4 address). And legacy programs have problems handling these addresses in the first place so people tend to turn them off. It was a nice idea but with little benefit.
I have been running IPv6 for almost 11 years and don't recall any mention of such issues. The point of the privacy addresses is they're not tied to a MAC address, as the consistent addresses often are. This means if you take your device to different locations, etc. Your address, as used by browsers, etc., is not traceable to your device. The only "issue" I can think of would be if you had some connection up for more than a week and the privacy address you had been using for that week is no longer available.
On 2021-03-04 12:00 p.m., James Knott wrote:
On 2021-03-04 11:53 a.m., Adam Majer wrote:
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet.
I don't know what you mean by unique for ssh but, with privacy addresses, the address used for outgoing connections is not the same as the one used for incoming. There is one consistent address, often based on the MAC address, and up to seven temporary privacy addresses, with a new one every day. There's no way anyone could determine the ssh server address from any privacy address. The best they could do is narrow(?) it down to 1 in 18.4 billion, billion
Another thing you could do is use a different /64 for your servers. For example, I get a /56 prefix from my ISP. This means I have 256 /64s. Others provide a /48 for 65536 /64s. Just pick one different from your outgoing connections.
Adam Majer wrote:
On 3/3/21 1:03 AM, Carlos E.R. wrote:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Security through poor obscurity. You have only 16bits there. Easy to find. They scan entire 32bit internet already and the 16 bits is easy too. Even my little VM has attempts on high port.
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet.
Yup, I have not seen any ssh login attempts on IPv6 for at least ten years. -- Per Jessen, Zürich (9.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland.
On 2021-03-04 12:46 p.m., Per Jessen wrote:
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet. Yup, I have not seen any ssh login attempts on IPv6 for at least ten years.
One reason for that is the address space is so huge. A single /64 has as many addresses as the entire IPv4 address space squared!
On 04/03/2021 17.53, Adam Majer wrote:
On 3/3/21 1:03 AM, Carlos E.R. wrote:
- Don't open ssh port 22, use a high port on a strange number (not 50000, for instance).
Security through poor obscurity. You have only 16bits there. Easy to find. They scan entire 32bit internet already and the 16 bits is easy too. Even my little VM has attempts on high port.
The idea is not to rely on using a high port alone, but on doing several things. A high port reduces the noise. A firewall that blocks after three incoming failed attempts further reduces the danger. And then using public keys finally does it.
If you want to have a quiet life for SSH admin only connections, limit to some IPv6 address unique for SSH. Then you don't have to worry about high ports. It will be quiet.
Unfortunately, IPv6 is not available to me. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 04/03/2021 20.51, James Knott wrote:
On 2021-03-04 2:49 p.m., Carlos E.R. wrote:
Unfortunately, IPv6 is not available to me.
You should be able to get it through tunnelling, as I did for the first six years. Hurricane Electric (he.net) is popular.
I tried, did not work out. And anyway, it is not a solution. I need the address to be known to me when I'm not home, reliable, and to have Ipv6 connectivity at my "outside" location, which presents similar problems. No, I will not use IPv6 till the providers here do it by routine. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
The problem occurred when someone hacked ssh to log in as my test user. They installed an executable file .dhcpd and started it in crontab. I killed the app and cleared the executable bits. I then removed the crontab line. 2 * * * * /home/test/.dhpcd -o ca.minexmr.com:4444 -B >/dev/null 2>/dev/null On 2021-03-02 9:17 a.m., James Knott wrote:
Today, I noticed my computer performance was very poor. Top showed something called .dhcpd running under user test and using almost 400% of my CPU. In ~/test, I found an executable .dhcpd. I have not used the test account for a long time. That .dhcpd also survived a reboot, so I'll have to determine how it's starting.
Here is some info:
linux:~ # ps aux|grep test test 3887 0.0 0.0 72236 7956 ? Ss 09:02 0:00 /usr/lib/systemd/systemd --user test 3888 0.0 0.0 265004 3220 ? S 09:02 0:00 (sd-pam) test 3909 416 7.3 2439200 2401880 ? SNsl 09:02 5:35 /home/test/.dhpcd root 5532 0.0 0.0 7432 920 pts/2 S+ 09:03 0:00 grep --color=auto test linux:~ # cd ~test linux:/home/test # ls .Xauthority .config .emacs .gtkrc-2.0 .local .profile .xinitrc.template Downloads Templates .bash_history .dbus .esd_auth .i18n .mozilla .qt .xsession-errors-:1 Music Videos .bashrc .dhpcd .fonts .inputrc .nx .ssh Desktop Pictures bin .cache .dmrc .gnupg .kde4 .pki .xim.template Documents Public public_html linux:/home/test # ls -l .dhpcd -rwxr-xr-x 1 test users 3458848 Mar 1 18:38 .dhpcd linux:/home/test # kill -9 3909 linux:/home/test # ps aux|grep test root 7884 0.0 0.0 7432 820 pts/2 S+ 09:06 0:00 grep --color=auto test linux:/home/test # more .dhpcd
******** .dhpcd: Not a text file ********
linux:/home/test # ls -a .dhpcd .dhpcd linux:/home/test # ls -al .dhpcd -rwxr-xr-x 1 test users 3458848 Mar 1 18:38 .dhpcd linux:/home/test #
There were some updates yesterday, so I wonder if one of them is infected with something. Also, this computer is configured with a static IPv4 address and uses SLAAC on IPv6, so it shouldn't be using dhcpd.
participants (20)
-
Adam Majer -
Andrei Borzenkov -
Bengt Gördén -
Carlos E. R. -
Carlos E. R. -
Carlos E.R.
-
colony.three＠protonmail.ch -
David C. Rankin -
J Leslie Turriff -
James Knott -
jdd＠dodin.org -
Jonathan Arsenault -
Josef Moellers -
ken -
Knurpht-openSUSE -
L A Walsh -
Lew Wolfgang -
Marcus Meissner -
Per Jessen -
Togan Muftuoglu