selinux? New install sshd can't open authorized_keys - permission denied?
(sorry if you get 2 copies, after sending to the user-list, I thought this really should go to factory since this relates to the new selinux setup in the Tumbleweed install) All, I think the new Tumbleweed has growing-pains associated with this new selinux stuff. sshd was broken after install, no firewall was opened despite the installer telling me it was after I explicitly turned it on. Now, I can't use public/private keys to access the new Tumbleweed install. I can enter a password, but public/private key auth fails with "Permission denied" ?? : Feb 25 15:57:02 niflheim sshd-session[2750]: Could not open user 'david' authorized keys '/home/david/.ssh/authorized_keys': Permission denied Feb 25 15:57:02 niflheim sshd-session[2750]: Could not open user 'david' authorized keys '/home/david/.ssh/authorized_keys': Permission denied Feb 25 15:57:02 niflheim sshd-session[2750]: Could not open user 'david' authorized keys '/home/david/.ssh/authorized_keys': Permission denied Feb 25 15:57:02 niflheim sshd-session[2750]: Postponed keyboard-interactive for david from 192.168.6.104 port 41652 ssh2 [preauth] Feb 25 15:57:10 niflheim sshd-session[2750]: Postponed keyboard-interactive/pam for david from 192.168.6.104 port 41652 ssh2 [preauth] Feb 25 15:57:10 niflheim sshd-session[2750]: Accepted keyboard-interactive/pam for david from 192.168.6.104 port 41652 ssh2 Feb 25 15:57:10 niflheim systemd-logind[940]: New session 15 of user david. Feb 25 15:57:10 niflheim systemd[1]: Started Session 15 of User david. The authorized_keys file is proper at 0644 and that is indeed what it is, e.g.: $ l total 32 drwx------. 3 david david 4096 Feb 25 16:10 . drwx------. 17 david david 4096 Feb 25 15:44 .. drwxr-xr-x. 3 david david 4096 Feb 17 03:35 keys -rw-r--r--. 1 david david 956 Feb 25 15:44 authorized_keys -rw-r--r--. 1 david david 233 Feb 19 03:44 config -rw-------. 1 david david 3381 Feb 17 02:28 id_rsa -rw-r--r--. 1 david david 738 Feb 17 02:28 id_rsa.pub -rw-------. 1 david david 2295 Feb 24 20:03 known_hosts The known_hosts and the private key are 0600 as they should be, so why is sshd-session confused about the permissions of authorized_keys -- and where and how do I fix it? Is there an easy way to ditch the selinux and go back to the aa? The selinux config still seems to be 1/2 baked. I can't set perms on /usr/local without them be switched back and I can't ssh into the new install to author the i915 bug to get the LSPCON/dpms/reboot issue fixed. Help? -- David C. Rankin, J.D.,P.E.
On 2/25/25 5:24 PM, David C. Rankin wrote:
Hi David, Sounds like the selinux policy for sshd has an issue. I am dealing with a similar issues for the xrdp service. I would submit a bug. While you wait for it to be fixed, you should be able to do the following as root: setenforce 0 This will switch selinux to permissive mode which will load the things denied but allow them to continue. It is recommended to get selinux policy updated for this and then you can setenforce 1 Which will switch back to enforcing mode. NOTE: Those changes are temporary so if you reboot you have to do it again. -- Regards, Joe
On Tue 25 Feb 2025 05:43:16 PM CST, Joe Salmeri wrote: <snip>
Hi No issues here with ssh to a fresh Tumbleweed install with selinux and sd boot running, no firewall though. Is the remote ~/.ssh directory created? The authorized_keys file should be 0600. No SELinux alerts reported from Cockpit either. -- Cheers Malcolm °¿° (Linux Counter #276890) Tumbleweed 20250224 | GNOME Shell 47.4 | 6.13.4-1-default HP Z440 | Xeon E5-2695 V4 X36 @ 2.10GHz | ARC A380 & RTX4000 up 1 day 6:09, 3 users, load average: 0.91, 0.63, 0.64
On 2/25/25 5:50 PM, Malcolm wrote:
Hi Malcolm, David is the one with the problem, I was showing him how to temporarily disable selinux to determine if it is what is blocking. FYI, with TW 20250216 I have an issue with the selinux policy for xrdp which goes away when selinux switched to permissive mode and there are NO audit messages logged about it. -- Regards, Joe
On 2/25/25 4:50 PM, Malcolm wrote:
That's likely what I'll do
That's not normally the case. I've used public/private key auth for 20+ years with openssh and the private key and "known_hosts" files are the only ones that require 0600 permissions. I went ahead and change authorized_keys to 0600 to test. No difference, same "Permission denied" in the journal :(
No SELinux alerts reported from Cockpit either.
Can't speak to your setup, but this one is not working. My other Tumbleweed box that uses AppArmor has authorized_keys at 0644 and it works just fine. So I strongly suspect this is due to the recent selinux change. This is a "minimal X install" to iceWM in prep for installing KDE3. So if the minimal X install is missing something, that too is a possibility. Thank you both for your help! We will get this sorted sooner or later :) -- David C. Rankin, J.D.,P.E.
Hi David, SELinux has nothing to do with the unix file permissions (e.g. 0600). It adds another label to the file, on which access control rules are applied from a policy. You can check which SELinux label is set by: ls -alZ ~/.ssh To me it sounds like your system is incorrectly labelled. As Neal said, it would be helpful in this specific scenario to check if this gives you any output: restorecon -Rvn / If yes, please try if restorecon -Rv / fixes your issue. If not, could you please include in your bug the information we need to check what happened as described here: https://en.opensuse.org/openSUSE:Bugreport_SELinux Also for this case, please add in the bug: - How did you switch to SELinux? Was it an installation where it was the default? - Output of: ls -alZ ~/.ssh Thanks :) Kind regards, Cathy On 26.02.25 00:03, David C. Rankin wrote:
-- Cathy Hu <cahu@suse.de> SELinux Security Engineer GPG: 5873 CFD1 8C0E A6D4 9CBB F6C4 062A 1016 1505 A08A SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Geschäftsführer: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On 2/26/25 3:03 AM, Cathy Hu wrote:
Thanks Cathy, I think I'm getting a handle on it, but it is literally like drinking from a firehose. This is no small change. So far I have been plagued by breakage: -- ssh not serving key, solution: restorecon -R /home/david/.ssh -- smb will not server [homes], solution: setsebool -P samba_enable_home_dirs=1 -- nginx will not server files, solution: chcon -R -t httpd_sys_rw_content_t /srv/http -- su broken in xterm /bin/bash Permission denied solution: remove session include common-session (there is a better solution for this, no time to chase it down) So much of this is just so different than anything suse/opensuse has had before, but I can only speak for the last 23 years or so... There was one of the articles that captured this quite well: <quote> I hasten to stress that you should not adopt SELinux just because the extra-robust security functionality exists. Yes, I know how much security matters to your business, but SELinux isn’t for everyone. Running SELinux takes extra time, requires far more hands-on involvement than other Linux distributions, and demands deep knowledge of Linux architecture. ... SELinux's fundamental security approach (restrict everything, unless explicitly permitted) is the exact opposite of Linux's (permit everything, unless explicitly forbidden). https://opensourcewatch.beehiiv.com/p/everything-wanted-know-selinux-afraid-... </quote> I've found that to be true. I'll get through this, but it has left me wondering, is selinux really appropriate for a community distribution? I see the value for SLE and commercial offerings running in the cloud or server farms that battle the bad guys every minute of the day, but my first impression is this is way way over the top for what a normal opensuse user needs and way way over the top for what realistic expectation for learning for a home user is. In my case, yes, I use samba, I use web-servers, I use remote access, I use postgres and mariadb, etc.. so I am likely running into more issue right off the bat during a new install than a lot will, but wow, I never expected so much just wouldn't work right out of the box. Apparmor never caused near this much problem (yes there were are few issues, but very few). We will see where it goes, I'll keep learning, but there is a lot to learn to wrap your head around this monster -- and that's the part that causes me concern for the normal community user. -- David C. Rankin, J.D.,P.E.
28.02.2025 23:01, David C. Rankin wrote:
-- ssh not serving key, solution: restorecon -R /home/david/.ssh
I wonder if https://bugzilla.opensuse.org/show_bug.cgi?id=1238190 is related.
Which could be related too.
On 2/28/25 2:20 PM, Andrei Borzenkov wrote:
Now we are talking, If that would eliminate those issues, it would go a long way towards softening to jolt selinux has posed - at least for me. I'm slowly digesting this, but it's not a quick man-page worth of learning to do. The ~/.ssh issue seems like something users shouldn't have to deal with. public/private key exchange should just work. The ~/.ssh directory isn't present during install, so if something isn't coming along later to fix that up, then the user is just going to be stuck. /srv/http is my directory, but a common directory for serving files from. If relabel can be smart enough to handle anything under /srv, that would also be a plus. We will get there, it is just take a little pain and frustration and a lot more butt-in-chair time reading... -- David C. Rankin, J.D.,P.E.
On 2/28/25 3:01 PM, David C. Rankin wrote:
Hi Dave and Cathy, I just tested smb on my test TW selinux machine and I can CONFIRM that selinux is block smb and that polices will need to be updated for it to work. Switching to permissive mode ( setenforce 0 ) and smb is working again. I don't have a problem with su working with selinux in enforcing mode but I use KDE and konsole. xterm is installed so I just tried that and I don't have an issue with it working in enforcing mode.
My fedora machine is only a "test" machine but it seems like they are more on top of creating the selinux policies for the packages they have which is probably because they switched to selinux a while ago. So far I haven't run into any selinux issues on that test fedora machine, but I have also not tested all the various packages that I use on my TW machines.
If you are using xrdp for remote access then I have an open bug report on that. We got it working but I think there is one more change which needs to work its way into factory. I just tried my apache website and it appears to work fine in enforcing mode which surprised me because I do a few none standard things there. Since I started using TW years ago I only had 1 issues with apparmor after a TW update and someone on here quickly helped me fix it.
-- Regards, Joe
Hi David, On Fri, Feb 28, 2025 at 02:01:15PM -0600, David C. Rankin wrote:
$ sudo semanage fcontext --add --equal "/var/www" "/srv/http" This would add a permanent equivalence between the two folders on your system. Note that you need to relabel after this. Please see the full steps on: https://en.opensuse.org/Portal:SELinux/Common_issues#Non-standard_file_locat...
_______________________________________________________________________________ Please note that we can only fix the bugs we're aware of. If you report your issues on Bugzilla providing the required information, we can help: https://en.opensuse.org/openSUSE:Bugreport_SELinux Half-reported issues on mailing lists do not help fixing the bugs for you or for others. Cheers, Filippo -- Filippo Bonazzi Security Engineer suse.com 8257 4398 947A 2DBE F21D 76E6 937A 63F0 5B36 46D9
On Mon, Mar 03, 2025 at 09:17:40AM +0100, Filippo Bonazzi wrote:
and seeing if that is enough? /srv/http *is* a custom path, but there is some allowance for it in the policy. Note I'm not an expert in web servers or the SELinux policy for them. Filippo -- Filippo Bonazzi Security Engineer suse.com 8257 4398 947A 2DBE F21D 76E6 937A 63F0 5B36 46D9
On 3/3/25 2:30 AM, Filippo Bonazzi wrote:
Luckily so far I've only done the chcon -R -t httpd_sys_rw_content_t /srv/http and that seems to have survived so far. I'm with SELinux for now having to run in permissive mode due to something breaking the properties on /var/log/php.log which prevents php-fpm from starting when SELinux is enforcing: type=AVC msg=audit(1741223800.854:322): avc: denied { open } for pid=13651 comm="php-fpm" path="/var/log/php-fpm.log" dev="nvme0n1p5" ino=19138140 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=file permissive=1 So pretty much everything works except for some strange jpg issue with splash screens and greeter backgrounds. (I'm not even worried about those at the moment) The last zypper dup broke a lot of SELinux permissions and there is currently a bug open on it: Bug 1238403 - https://bugzilla.opensuse.org/show_bug.cgi?id=1238403 We will wait until that bug is resolved to restore SELinux to enforcing mode. -- David C. Rankin, J.D.,P.E.
On Fri, Feb 28, 2025 at 11:02 PM David C. Rankin <drankinatty@gmail.com> wrote:
IIRC you start the display manager manually from within the normal run level 3 console login session. Which means you have an entirely different starting SELinux context than is expected. Normally the display manager is started on boot by the init and SELinux policy is created for this case. And it works for this case. If you want to do it differently, do not blame openSUSE for the result. You will need to add a SELinux policy for your use case.
On 2/25/25 4:43 PM, Joe Salmeri wrote:
You nailed it Joe! Confirmed, the problem is selinux related. I set # setenforce 0 Then magically, public-private key auth start working just fine! 17:08 wizard:~/.mozilla/firefox/xpkdr71o.default-release> ssh niflheim Last login: Tue Feb 25 16:54:03 CST 2025 from 192.168.6.104 on ssh Have a lot of fun... 17:08 niflheim:~> It's just like magic :) I have another longer bug to file against the i915 graphic issue that is causing a reboot when dpms is triggered. There are a number of tangential bugs related to LSPCON (hdmi stuff), but apparently it has gotten worse in newer kernels. I've installed both Arch and a new Tumbleweed and both show the same reboot on dpms activation. The only reason I'm overly concerned about this issue for this box, is it is an all-in-one (monitor is part of computer) and there is no way to power the display down (otherwise I'd just turn the monitor off and use it via ssh) I'll file this selinux bug (and the related bug from the install not opening the firewall for sshd -- even though the installer said it had) after I write up the LSPCON/i915/dpms/reboot issue -- which looks to be a long one. Thanks again for your help. -- David C. Rankin, J.D.,P.E.
To be honest, SELinux was just a bad idea for openSUSE to switch to. I already told them before they made the change, but no one listened. Now we're here, scratching our heads, clueless. Tumbleweed wasn't ready for this switch, and neither was anyone else. This decision was way too hasty and ill-considered. As for OpenQA, it seems to be just for show at this point. Pete
On 2/25/25 5:27 PM, Oyakawa Ichiro via openSUSE Factory wrote:
There are dependency issues that remain to. I just tried to install plocate on this selinux Tumbleweed and it tried to install: apparmor-abstractions apparmor-parser as dependencies. SELinux needs to go back in the oven and bake a little longer, it's not quite done. Personally, whether it is apparmor or selinux, it doesn't make much difference to me as long as it works. There doesn't seems to be a way to install plocate without pulling in those dependencies. I don't think it would be a good idea to mix apparmor and selinux parts together. I may just not know enough to know, but it seems like a "you got your peanutbutter in my chocolate" issue... -- David C. Rankin, J.D.,P.E.
On Tue 25 Feb 2025 06:13:39 PM CST, David C. Rankin wrote:
Likely something is not being installed on a minimal setup? I've been using Aeon and MicroOS with SELinux for some time now, a few little hiccups every now and again, but no show stoppers. I use the flatpak Cockpit client on one machine and can talk to all machines with it, those running SELinux (needed to install setroubleshoot-server) provide solutions for most issues. -- Cheers Malcolm °¿° (Linux Counter #276890) Tumbleweed 20250224 | GNOME Shell 47.4 | 6.13.4-1-default HP Z440 | Xeon E5-2695 V4 X36 @ 2.10GHz | ARC A380 & RTX4000 up 1 day 7:53, 3 users, load average: 0.24, 0.26, 0.25
Am 26.02.25 um 01:13 schrieb David C. Rankin:
JFYI: https://bugzilla.suse.com/show_bug.cgi?id=1237689 Regards, Frank
* On 2/26/25 01:13, David C. Rankin wrote:
Far from it. There is absolutely no harm in installing both AA and SELinux policies, other than using a tiny bit of additional disk space. If a framework is disabled in the kernel, it just won't be used - notwithstanding any installed policy files. Mihai
On Tue, Feb 25, 2025 at 11:27:15PM -0000, Oyakawa Ichiro via openSUSE Factory wrote:
This was a many month process which was discussed here several times. You might disagree with the outcome, but it for sure wasn't hasty. Some growing pains are to be expected. Also please consider you communication style, it is pretty rude. Johannes -- GPG Key EE16 6BCE AD56 E034 BFB3 3ADD 7BF7 29D5 E7C8 1FA0 Subkey fingerprint: 250F 43F5 F7CE 6F1E 9C59 4F95 BC27 DD9D 2CC4 FD66 SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany Geschäftsführer: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
Johannes Segitz wrote:
Look, if you think I was rude, then you should see me when I actually am. This was anything but rude. Don’t mistake constructive criticism or feedback for rudeness. And as far as I remember, I didn’t ask for a lecture or a condescending tone on how to improve my communication style. Yes, I’m aware that my language is strong because I speak the truth—the truth that others don’t dare to say, but without it, there would be no change. And yes, I strike where it hurts, but I do so in a diplomatic and cultured manner. I resent the condescending and lecturing tone about my communication style. Pete
On 2/25/25 5:24 PM, David C. Rankin wrote:
Hi David, Sounds like the selinux policy for sshd has an issue. I am dealing with a similar issues for the xrdp service. I would submit a bug. While you wait for it to be fixed, you should be able to do the following as root: setenforce 0 This will switch selinux to permissive mode which will load the things denied but allow them to continue. It is recommended to get selinux policy updated for this and then you can setenforce 1 Which will switch back to enforcing mode. NOTE: Those changes are temporary so if you reboot you have to do it again. -- Regards, Joe
On Tue 25 Feb 2025 05:43:16 PM CST, Joe Salmeri wrote: <snip>
Hi No issues here with ssh to a fresh Tumbleweed install with selinux and sd boot running, no firewall though. Is the remote ~/.ssh directory created? The authorized_keys file should be 0600. No SELinux alerts reported from Cockpit either. -- Cheers Malcolm °¿° (Linux Counter #276890) Tumbleweed 20250224 | GNOME Shell 47.4 | 6.13.4-1-default HP Z440 | Xeon E5-2695 V4 X36 @ 2.10GHz | ARC A380 & RTX4000 up 1 day 6:09, 3 users, load average: 0.91, 0.63, 0.64
On 2/25/25 5:50 PM, Malcolm wrote:
Hi Malcolm, David is the one with the problem, I was showing him how to temporarily disable selinux to determine if it is what is blocking. FYI, with TW 20250216 I have an issue with the selinux policy for xrdp which goes away when selinux switched to permissive mode and there are NO audit messages logged about it. -- Regards, Joe
On 2/25/25 4:50 PM, Malcolm wrote:
That's likely what I'll do
That's not normally the case. I've used public/private key auth for 20+ years with openssh and the private key and "known_hosts" files are the only ones that require 0600 permissions. I went ahead and change authorized_keys to 0600 to test. No difference, same "Permission denied" in the journal :(
No SELinux alerts reported from Cockpit either.
Can't speak to your setup, but this one is not working. My other Tumbleweed box that uses AppArmor has authorized_keys at 0644 and it works just fine. So I strongly suspect this is due to the recent selinux change. This is a "minimal X install" to iceWM in prep for installing KDE3. So if the minimal X install is missing something, that too is a possibility. Thank you both for your help! We will get this sorted sooner or later :) -- David C. Rankin, J.D.,P.E.
Hi David, SELinux has nothing to do with the unix file permissions (e.g. 0600). It adds another label to the file, on which access control rules are applied from a policy. You can check which SELinux label is set by: ls -alZ ~/.ssh To me it sounds like your system is incorrectly labelled. As Neal said, it would be helpful in this specific scenario to check if this gives you any output: restorecon -Rvn / If yes, please try if restorecon -Rv / fixes your issue. If not, could you please include in your bug the information we need to check what happened as described here: https://en.opensuse.org/openSUSE:Bugreport_SELinux Also for this case, please add in the bug: - How did you switch to SELinux? Was it an installation where it was the default? - Output of: ls -alZ ~/.ssh Thanks :) Kind regards, Cathy On 26.02.25 00:03, David C. Rankin wrote:
-- Cathy Hu <cahu@suse.de> SELinux Security Engineer GPG: 5873 CFD1 8C0E A6D4 9CBB F6C4 062A 1016 1505 A08A SUSE Software Solutions Germany GmbH Frankenstrasse 146 90461 Nürnberg Geschäftsführer: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
participants (11)
-
Andrei Borzenkov -
Cathy Hu -
David C. Rankin -
Filippo Bonazzi -
Frank Krüger -
Joe Salmeri -
Johannes Segitz -
Malcolm -
Mihai Moldovan -
Neal Gompa -
Oyakawa Ichiro