Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
cu, Rudi
On Fri, Dec 23, 2016 at 11:27 AM, Ruediger Meier sweet_f_a@gmx.de wrote:
Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
This was always a best-effort thing and there is no ETA for its completion.
On Friday 23 December 2016, Cristian Rodríguez wrote:
On Fri, Dec 23, 2016 at 11:27 AM, Ruediger Meier sweet_f_a@gmx.de
wrote:
Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
This was always a best-effort thing and there is no ETA for its completion.
This "best effort" looks like it was quickly messed up 5 years ago and nothing happened since then.
I think we are waiting long enough. I'm going to remove all the mess from util-linux.spec file again.
BTW util-linux's original make install would build fine if /bin and /sbin would be finally linked to /usr/bin. Now with all these "#UsrMerge links" it would fail anyways. Don't understand why this was changed this way at all.
cu, Rudi
On Fri, Dec 23, 2016 at 6:27 AM, Ruediger Meier sweet_f_a@gmx.de wrote:
Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
cu, Rudi
Highly motivating message, Rudeli.
diff -srq /bin/ /usr/bin | grep identical DIY rather than waiting for The Help.
bin and usr/bin total <700MiB. Maybe get a higher capacity HD.
Happy Holidays Musketeer in the U.S. War on Christmas
On 12/23/2016 10:20 AM, Carl Symons wrote:
Highly motivating message, Rudeli.
diff -srq /bin/ /usr/bin | grep identical
Hmmmm
# diff -srq /bin/ /usr/bin | grep identical | wc -l 113
so, how many of those are symlinks?
# ls -l /bin | grep -e "->" | wc -l 112
So, what's the exception?
Now, if as recommended, the /usr part of the file system is on the ROOTFS volume, those could be hard links. In which case they are truly redundant.
It seems, superficially at lest, that this is a fob for those who don't have /usr/bin on the same FS as /bin.
And given all that, if you have both /bin and /usr/bin on your SEARCH path, the why have the symlinks at all? Yes, I realise that that if there were no symlinks then the execl code would search /bin first, not find it, then search /usr/bin, but then again, at some level, the execl code, either at the user level or the kernel level, is going to find that the version found in /bin is a symlink and have to go find the version in /usr/bin by step and repeat. Or maybe its faster with a Btree-FS.
Surely a script that converts symlinks into hard links if they are on the same FS isn't that difficult, and why couldn't that script be in the RPM as a post-install?
On Friday 2016-12-23 16:56, Anton Aylward wrote:
Surely a script that converts symlinks into hard links if they are on the same FS isn't that difficult, and why couldn't that script be in the RPM as a post-install?
No, because then you would be out of sync with what the rpm database recorded.
Please: no need to cc me, I am on the list and don't want duplicates.
On 12/23/2016 11:02 AM, Jan Engelhardt wrote:
On Friday 2016-12-23 16:56, Anton Aylward wrote:
Surely a script that converts symlinks into hard links if they are on the same FS isn't that difficult, and why couldn't that script be in the RPM as a post-install?
No, because then you would be out of sync with what the rpm database recorded.
Are you saying no the idea of doing this manually, which would, yes, make things out of sync, or are you saying no to having it as part of a script that makes all the necessary adjustment, and that would include adjustment s to the RPM database?
On Friday 2016-12-23 17:05, Anton Aylward wrote:
Please: no need to cc me, I am on the list and don't want duplicates.
Use a local deduplication filter then.
Surely a script that converts symlinks into hard links if they are on the same FS isn't that difficult, and why couldn't that script be in the RPM as a post-install?
No, because then you would be out of sync with what the rpm database recorded.
Are you saying no the idea of doing this manually, which would, yes, make things out of sync, or are you saying no to having it as part of a script that makes all the necessary adjustment, and that would include adjustment s to the RPM database?
"no" to modifying any files in %post.
On 12/23/2016 01:29 PM, Jan Engelhardt wrote:
"no" to modifying any files in %post.
Reason/Justification?
Question for the list in general:
Are there any RPMs where files *are* modified on %post?
On Friday 2016-12-23 20:19, Anton Aylward wrote:
On 12/23/2016 01:29 PM, Jan Engelhardt wrote:
"no" to modifying any files in %post.
Reason/Justification?
A package already allows you, the packager, to place arbitrary files in arbitrary locations via %files. In the context of the discussion, if you want /bin/* to be hardlinks (not that I think that it is a sensible thing), make that happen when the package is built, not bolting it on after the fact.
Are there any RPMs where files *are* modified on %post?
Quite so, like - /var caches (untracked; regeneratable anytime) - archaic merged config files like those in /etc/sysconfig/* and /etc/passwd (not regeneratable! and therefore a point to mess up badly. You would need a VERY good reason to edit around in /etc in %post directly)
Hi,
Happy Holidays
On 12/23/2016 09:27 AM, Ruediger Meier wrote:
Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
It appears that you are misunderstanding some fundamental concepts.
1.) Calling people lazy will most likely not motivate them to do more voluntary work that may benefit you
2.) Links will have to remain more or less indefinitely. If an executable moves from /{s,}bin to /usr/{s,}bin a link in /{s,}bin must remain for backwards compatibility in order to not break existing code outside of the distribution that may expect executables in /{s,}bin
3.) This is a community of volunteers and contributions are welcome, so if it bothers you, scratch your own itch and pitch in rather than degrading the effort that others have already made.
4.) Although a reasonably compelling argument for the merge has been made not everyone is convinced that this is the approach to take. In the end it is up to those maintaining specific packages if the executables should be moved or not. There is not technical steering committee to tel people otherwise.
Later, Robert
On Sunday 2016-12-25 12:34, Robert Schweikert wrote:
2.) Links will have to remain more or less indefinitely. If an executable moves from /{s,}bin to /usr/{s,}bin a link in /{s,}bin must remain for backwards compatibility in order to not break existing code outside of the distribution that may expect executables in /{s,}bin
That is not quite correct - have a look at Fedora. If /sbin itself is a symlink, then /sbin/PROG does not need to be a link.
And that is the end goal, though that has not yet been reached in openSUSE.
On Sun, Dec 25, 2016 at 01:39:35PM +0100, Jan Engelhardt wrote:
On Sunday 2016-12-25 12:34, Robert Schweikert wrote:
2.) Links will have to remain more or less indefinitely. If an executable moves from /{s,}bin to /usr/{s,}bin a link in /{s,}bin must remain for backwards compatibility in order to not break existing code outside of the distribution that may expect executables in /{s,}bin
That is not quite correct - have a look at Fedora. If /sbin itself is a symlink, then /sbin/PROG does not need to be a link.
Fedora is a mess. If you look at their packages you'll see that there are still many of them that have stuff in /bin and rely on rpm following /bin to /usr/bin when doing the install.
And also keep in mind that you'll need to add lots of "Provides: /bin/foo" statements so that 3rd party stuff is still installable.
I don't see why we should have all the pain that a /bin and /sbin links gives us with no gain at all.
Cheers, Michael.
Michael Schroeder wrote:
On Sun, Dec 25, 2016 at 01:39:35PM +0100, Jan Engelhardt wrote:
On Sunday 2016-12-25 12:34, Robert Schweikert wrote:
2.) Links will have to remain more or less indefinitely. If an executable moves from /{s,}bin to /usr/{s,}bin a link in /{s,}bin must remain for backwards compatibility in order to not break existing code outside of the distribution that may expect executables in /{s,}bin
That is not quite correct - have a look at Fedora. If /sbin itself is a symlink, then /sbin/PROG does not need to be a link.
Fedora is a mess. If you look at their packages you'll see that there are still many of them that have stuff in /bin and rely on rpm following /bin to /usr/bin when doing the install.
Sounds like the easiest part to fix though.
And also keep in mind that you'll need to add lots of "Provides: /bin/foo" statements so that 3rd party stuff is still installable.
Assuming that we don't want to generate more such cases in the future it should be possible to generate those legacy file provides automatically. After all we know what exists today in /bin and /sbin by looking at e.g. ARCHIVES.gz once. So an rpm dependency generator that knows the list of legacy "/-files" could therefore automatically add the needed tags when building an affected package.
I don't see why we should have all the pain that a /bin and /sbin links gives us with no gain at all.
Well, the current rather random split isn't exactly a showcase of brilliant architecture either. Leaving everything in %{_prefix} looks like a simpler solution to me.
I'm not sure our current approach of packaging symlinks of files in / that point to /usr is helpful though. It might even hinder a proper migration of the four directories in question to a symlink. What will rpm do if /bin is a symlink and a package has both /bin/foo and /usr/bin/foo?
In any case the usr move needs someone to seriously drive it to completion otherwise it's not going to happen and we have to live with the current mess.
cu Ludwig
On Sun, Dec 25, 2016 at 06:34:50AM -0500, Robert Schweikert wrote:
Hi,
Happy Holidays
On 12/23/2016 09:27 AM, Ruediger Meier wrote:
Hi,
some years ago we've had big discussions about it...
But know the direcories are still not merged and we still have hundreds of duplicate links in /bin and /usr/bin. Some binaries are still only available in /bin. Still many horrible looking spec files.
So lazy usr-merge supporters, please finish your job and cleanup the mess.
It appears that you are misunderstanding some fundamental concepts.
1.) Calling people lazy will most likely not motivate them to do more voluntary work that may benefit you
2.) Links will have to remain more or less indefinitely. If an executable moves from /{s,}bin to /usr/{s,}bin a link in /{s,}bin must remain for backwards compatibility in order to not break existing code outside of the distribution that may expect executables in /{s,}bin
3.) This is a community of volunteers and contributions are welcome, so if it bothers you, scratch your own itch and pitch in rather than degrading the effort that others have already made.
4.) Although a reasonably compelling argument for the merge has been made not everyone is convinced that this is the approach to take. In the end it is up to those maintaining specific packages if the executables should be moved or not. There is not technical steering committee to tel people otherwise.
This discussion is useless ... could we please stay with /bin, /sbin as well with /usr/bin and /usr/sbin. Even upstream of systemd never had said that /bin/sh has to become /usr/bin/sh to avoid POSIX violations. The same holds true for other essential user commands as well as for other system commands.
IMHO this forced merge is a misunderstanding of having /usr as part of the root file system. Wheras having one root file system with /usr make sense as to many libraries and dynamicallly loaded files are located below /usr (e.g. libraries used for nfs4 and also dynamicallly loaded files of the glibc), there is no need to enforce the depopulation of the system basic paths /bin and /sbin
Werner
Dr. Werner Fink wrote:
IMHO this forced merge is a misunderstanding of having /usr as part of the root file system.
--- Which, IMO, was never really needed if the 1st thing you did when booting up was mount /usr.
Wheras having one root file system with /usr make sense as to many libraries and dynamicallly loaded files are located below /usr
--- And below /usr/share -- a 3rd filesystem on some systems when /usr/share grew too big to fit on /usr partition.
(e.g. libraries used for nfs4 and also dynamicallly loaded files of the glibc), there is no need to enforce the depopulation of the system basic paths /bin and /sbin
---- The dependent libs in /usr can easily be moved to /lib{,64}. I don't understand, if they wanted to have them together, they didn't move files from /usr/bin => /bin and put symlinks in /usr (same for /lib, /sbin, /lib64...etc).
That would have been *safe* and accomplished the merge -- but no one in the /usr-merge camp could answer that question. The inability to technically justify the decision made me feel like some other reason was being hidden. That made me more than a little uncomfortable.
I ended up writing a script to verify file system and file dependencies after the first time my system became unbootable due to new libs required by mount being placed in /usr/lib64 (which wasn't mounted yet). That was very impressive [not!]. How would anyone be expected to use the mount command if it required libs on "yet-to-be-mounted" media? Sigh.
After that I wrote a perl script to check mount and binary dependencies and, optionally, correct them -- I try to remember to run that before every boot.
Also wrote a script to *RE*-rename the interfaces as it comes up. I notice in my boot log that the interfaces come up with the right name, but are then renamed by "something" -- maybe udev?, then renamed again by my bash script - weird. Too many times something got tweaked in the boot process and they stopped coming up with the right names, so I decided to stop relying on the normal boot process keeping them correct and wrote my own renamer.
Necessity is the mother of invention... :-)
-l
On 12/25/2016 10:27 PM, L.A. Walsh wrote:
Wheras having one root file system with /usr make sense as to many libraries and dynamicallly loaded files are located below /usr
No it doesn't. Moving them to /bin, /lib and symlinking the entries in /usr/bin and /usr/lib would have achieved the same end. cf what Linda sys.
And below /usr/share -- a 3rd filesystem on some systems when /usr/share grew too big to fit on /usr partition.
And the original idea of "/usr/share" at USG, decades ago, was that it *would* be shared via NFS. One copy on the server. So there shouldn't be anything there essential to booting.
So what binaries *are* under /usr/share ?
I run find for any '.so' files under /usr/share and come up with nothing. Executables? Well lots of scripts. Lots of XML and other config files and templates that applications use. Yes you _could_ think of the icons needed by an GUI and the various images that OpenOffice Impress uses as a 'library'. But these are applications; they are not needed for booting. They can wait.
(e.g. libraries used for nfs4 and also dynamicallly loaded files of the glibc),
... are not under /usr/share
OK. Look at it this way
If all the necessary binaries are in /bin and /lib then there's no need to have /usr available in order to boot. If there are applications that have some executable in /usr/bin hard coded with that path rather than using the search or use fexecve() or execlp(), which searches using PATH rather than hard coding an absolute path and using execl(). Its like using DNS rather than hard coding an IP address.
Or yes, have the names in /usr/bin but they are symlinks to /bin.
As Linda says
The dependent libs in /usr can easily be moved to /lib{,64}. I don't understand, if they wanted to have them together, they didn't move files from /usr/bin => /bin and put symlinks in /usr (same for /lib, /sbin, /lib64...etc).
This isn't crazy. What I find odd is that its the other way round!
Do a ls -l on /bin. Look at all the stuff that's symlinked to /usr/bin.
What's the rational here? Is it that yes, the binaries needed for booting are in /bin and all these symlinks are are for things that are *not* needed for booting? Well that makes sense. But look to see what they are!
That would have been *safe* and accomplished the merge -- but no one in the /usr-merge camp could answer that question. The inability to technically justify the decision made me feel like some other reason was being hidden. That made me more than a little uncomfortable.
Well, yes, there is that.
there is no need to enforce the depopulation of the system basic paths /bin and /sbin
Anton Aylward wrote:
And the original idea of "/usr/share" at USG, decades ago, was that it *would* be shared via NFS. One copy on the server. So there shouldn't be anything there essential to booting.
--- I agree and complained before 13.2 was released. Was told "too bad".
So what binaries *are* under /usr/share ?
--- We are talking necessary for boot. Not binaries. They were scripts called by udev if I remember correctly.
Forgive me, I don't have such instances on my system anymore.
OK. Look at it this way
If all the necessary binaries are in /bin and /lib then there's no need to have /usr available in order to boot.
--- Gee. Really? And asking why that wouldn't work got you ignored.
As Linda says
The dependent libs in /usr can easily be moved to /lib{,64}. I don't understand, if they wanted to have them together, they didn't move files from /usr/bin => /bin and put symlinks in /usr (same for /lib, /sbin, /lib64...etc).
This isn't crazy. What I find odd is that its the other way round!
--- Bingo. But no one was willing to provide an answer.
Do a ls -l on /bin. Look at all the stuff that's symlinked to /usr/bin.
What's the rational here? Is it that yes, the binaries needed for booting are in /bin and all these symlinks are are for things that are *not* needed for booting? Well that makes sense. But look to see what they are!
That would have been *safe* and accomplished the merge -- but no one in the /usr-merge camp could answer that question. The inability to technically justify the decision made me feel like some other reason was being hidden. That made me more than a little uncomfortable.
Well, yes, there is that.
And now, 2 years later or so, we still have naive zeolots claiming you can't boot your system w/o /usr already being mounted. Then you point out that /usr can be mounted at the beginning of boot and such a system can be as fast or faster than a system w/an initrd (7 seconds for a *server* bringing up 55 services on rotating rust). And they never really get it and continue to share their fake-news, world-view on this list.
I'm sure they listen to Fox News... ;-)
On 12/26/2016 03:34 PM, L.A. Walsh wrote:
We are talking necessary for boot. Not binaries. They were scripts called by udev if I remember correctly.
Forgive me, I don't have such instances on my system anymore.
Granted. And I have /usr/share as a seperate file system that gets loaded in the fine systemd manner asynchronously with /home, /opt, /var, /srv and /tmp. Good thing that there isn't a race condition.
Maybe those udev script were simply in /usr/lib/udev or possibly the executable at /usr/lib/udisks2/udisksd
See the man page for udisks.
It says "At start-up and when a drive is connected..." What "at start-up" specifically means, in terms of 'during boot' or after the boot pivot to do something so that the other drives than the ROOTFS can be activated and mounted ... I'm not sure. I don't have a "/etc/udisks2/" on my system.
Anton Aylward wrote:
Maybe those udev script were simply in /usr/lib/udev or possibly the executable ...
--- greping, I get:
grep -i /usr/share *
apport:AGENT=/usr/share/apport/apport cobblerd: sed -i -e "[...]" /usr/share/cobbler/web/settings.py dkimproxy:DKIMPROXYDIR=/usr/share/dkimproxy festival: CHROOT_PREFIX="/usr/share/festival/" ipmi_port:shardir=/usr/share ipmiutil_evt:# /usr/share/ipmiutil/evt.sh ipmiutil_evt:# RUNSCRIPT="-r /usr/share/ipmiutil/evt.sh" kbd:# /usr/share/kbd - for finding keymaps kbd:KBDBASE="/usr/share/kbd" lirc: DEVINPUTCONF='/usr/share/lirc/remotes/devinput/lircd.conf.devinput' monit:MONIT_MODIFY_INITTAB="/usr/share/monit/monit-modifyinittab" named:NAMED_CONF_META_INCLUDE_FILE_SCRIPT="/usr/share/bind/createNamedConfInclude" named: test "${script:0:1}" = "/" || script="/usr/share/bind/${script}" sfcb: /usr/share/sfcb/genSslCert.sh /etc/sfcb
smb:SMB_APPARMOR_UPDATE="/usr/share/samba/update-apparmor-samba-profile" tog-pegasus: if [ -x /usr/share/Pegasus/scripts/genOpenPegasusSSLCerts ]; then tog-pegasus: /usr/share/Pegasus/scripts/genOpenPegasusSSLCerts;
---- A few of those are third party scripts, others may have been moved elsewhere. Anyway, those are ones I have now.
I think the one that caused probs was a virtual-machine sw-package that had usb drivers (or a script for such) in /usr/share/...
It may be that they've been weeded out of post 13.1 or post 13.2 dists.
Sorry, Linda, if this seems preachy, its not targeted at you, its as much clarifying the logical steps in my own mind before my first litre of coffee and sugar jolt.
On 12/26/2016 09:49 PM, L A Walsh wrote:
Anton Aylward wrote:
Maybe those udev script were simply in /usr/lib/udev or possibly the executable ...
greping, I get:
grep -i /usr/share *
[snip] Noted
A few of those are third party scripts, others may have been moved elsewhere. Anyway, those are ones I have now.
Yes, but my point holds. Those are not needed for ...
OK, perhaps we have a terminological difference. Perhaps to some, the term 'boot' means *EVERYTHING* between pressing the power on button and seeing the login appear. 'The "*EVERYTHING*" means every last little thing that includes starting up the user space services like SMB, Proftp ((even if it is still 32-bit), Postfix, Apache, /home and all the file systems that the user may have under his/her home directory all mounted, ready, before login.
In that case you're going to need /srv mounted as well!
On the other hand some of us view 'boot' as very basic. Boot gets you to 'maintenance mode' where you have a very basic system, the shell, a few utilities, perhaps enough to repair your file system. If you've never needed to flutz around in maintenance mode you've been lucky. But its there, the most basic of boots. No file systems other than ROOTFS available.
Believe me, it works even if you have /usr as a separate file system. The 'why' is interesting.
The very basic boot involves having a runable system - just - in the initrd. Go look at what is delivered in an initrd using 'lsinitrd'.
To my mind, the issue with /usr boils down to two things.
1. Is there enough in the initrd to allow systemd to mount /usr?
That means one of two things:
a) is there a custom unit, something that is like "usr.mount", under /etc/systemd in the initrd that will allow the initrd to mount /usr
b) an entry in the /etc/fstab of initrd and the systemd generator that allows it to build the unit to mount /usr
and
2. is there stuff in /usr that is essential to the boot process?
We've been thinking of specific executables and libraries here, but Linda is right in pointing out that scripts matter too. Are there, for example, units in /usr/lib/systemd that are necessary for the boot process?
well OUCH! I've edited this down ... its from my running 13.2 system. YMMV
# ls -l /etc/systemd/system total 32 lrwxrwxrwx 1 root root 44 Nov 6 2013 dbus-org.freedesktop.Avahi.service -> /usr/lib/systemd/system/avahi-daemon.service lrwxrwxrwx 1 root root 45 Jul 10 10:13 dbus-org.opensuse.Network.AUTO4.service -> /usr/lib/systemd/system/wickedd-auto4.service lrwxrwxrwx 1 root root 45 Jul 10 10:13 dbus-org.opensuse.Network.DHCP4.service -> /usr/lib/systemd/system/wickedd-dhcp4.service lrwxrwxrwx 1 root root 45 Jul 10 10:13 dbus-org.opensuse.Network.DHCP6.service -> /usr/lib/systemd/system/wickedd-dhcp6.service lrwxrwxrwx 1 root root 45 Jul 10 10:13 dbus-org.opensuse.Network.Nanny.service -> /usr/lib/systemd/system/wickedd-nanny.service lrwxrwxrwx 1 root root 40 Jul 11 2014 default.target -> /usr/lib/systemd/system/runlevel5.target
lrwxrwxrwx 1 root root 38 Jul 10 10:13 network.service -> /usr/lib/systemd/system/wicked.service
lrwxrwxrwx 1 root root 39 Nov 6 2013 syslog.service -> /usr/lib/systemd/system/rsyslog.service
Which gets back to the issue, how many of those are essential for whatever you view as 'boot'? I ask this because those don't appear in the 'lsinitrd'.
You might also notice when you run 'lsinitrd' that there is a "/usr" subtree there. So the intird has all that's needed to do the boot even if, in the initrd, some things from /bin are symlinked to /usr/bin (rather than the other way round).
So when, exactly does what's in /usr on the disk become essential because its not part of /usr in the initrd?
I see earlier on in the 'lsinitrd' output that Drakut includes some modules. Perhaps the issue is in getting the right modules included. For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
I see, also, that there is a 'usrmount' modules there. Hmm. Need to drill down there.
I think the one that caused probs was a virtual-machine sw-package that had usb drivers (or a script for such) in /usr/share/...
It may be that they've been weeded out of post 13.1 or post 13.2 dists.
On 2016-12-27 15:28, Anton Aylward wrote:
- Is there enough in the initrd to allow systemd to mount /usr?
Yes, it works. I have a separate /usr. YaST sets it up automatically.
However, repair on emergencies becomes more complicated. It is easier to quit and use a repair system.
But Linda doesn't use initrd, if I remember correctly.
I see earlier on in the 'lsinitrd' output that Drakut includes some modules. Perhaps the issue is in getting the right modules included. For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
On 12/27/2016 10:45 PM, Carlos E. R. wrote:
On 2016-12-27 15:28, Anton Aylward wrote:
- Is there enough in the initrd to allow systemd to mount /usr?
Yes, it works. I have a separate /usr. YaST sets it up automatically.
However, repair on emergencies becomes more complicated. It is easier to quit and use a repair system.
But Linda doesn't use initrd, if I remember correctly.
Ah, yes, that's my recollection too, now you come to mention it.
I see earlier on in the 'lsinitrd' output that Drakut includes some modules. Perhaps the issue is in getting the right modules included. For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
Yes, but I absolutely need LVM. The issue really is 'what don't I need?'
Well, there's
* what don't I need if all I'm doing is a regular boot?
and
* what do I need to do a maintenance boot?
I have
dracut modules: bash warpclock -- what's this i18n -- I only speak english convertfs ifcfg drm plymouth -- A possibility for elimination btrfs -- What if I convert ROOTFS to ext4? dm kernel-modules lvm resume rootfs-block terminfo -- ?? udev-rules biosdevname systemd usrmount base fs-lib shutdown suse
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-28 05:22, Anton Aylward wrote:
On 12/27/2016 10:45 PM, Carlos E. R. wrote:
I see earlier on in the 'lsinitrd' output that Drakut includes some modules. Perhaps the issue is in getting the right modules included. For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
Yes, but I absolutely need LVM. The issue really is 'what don't I need?'
Well, there's
- what don't I need if all I'm doing is a regular boot?
and
- what do I need to do a maintenance boot?
More than you have. I found it too lacking, had to reboot to a rescue image on last problem.
I have
plymouth -- A possibility for elimination
This one was half of mine.
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
On 12/27/2016 11:37 PM, Carlos E. R. wrote:
plymouth -- A possibility for elimination
This one was half of mine.
Oh yes! As far as I can tell, plymouth is boot-time eye candy, to hide “the ugly details of boot up” behind a graphical (and possibly animated) splash screen. Some of us like to know what's going on!
The only plus I can see is that it tries also to set some kind of "optimal" (YMMV) settings for the screen. We've had a thread about that recently so I wonder as to how effective it is.
What? its part of Initrd and that was a Linda-initiated thread and Linda doesn't use initrd?
Oh, well, ....
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
On 28.12.2016 16:44, Anton Aylward wrote:
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
zypper rm libply4
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-28 16:44, Anton Aylward wrote:
On 12/27/2016 11:37 PM, Carlos E. R. wrote:
plymouth -- A possibility for elimination
This one was half of mine.
Oh yes! As far as I can tell, plymouth is boot-time eye candy, to hide “the ugly details of boot up” behind a graphical (and possibly animated) splash screen. Some of us like to know what's going on!
Yep.
The only plus I can see is that it tries also to set some kind of "optimal" (YMMV) settings for the screen. We've had a thread about that recently so I wonder as to how effective it is.
I have seen problems where part of the solution is not to have plymouth in the mix.
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
Uninstall the package, blacklist it, then run mkinitrd :-)
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
On 12/28/2016 11:45 AM, Carlos E. R. wrote:
On 2016-12-28 16:44, Anton Aylward wrote:
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
Uninstall the package, blacklist it, then run mkinitrd :-)
Well it complained about the the absence, but so what?
What bothered me was the absence of the following modules
* unix * atkbd * swap * ext2 * LVM2_member
Does that matter?
Initrd now down from 12M to 9M
On 2016-12-28 18:21, Anton Aylward wrote:
On 12/28/2016 11:45 AM, Carlos E. R. wrote:
On 2016-12-28 16:44, Anton Aylward wrote:
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
Uninstall the package, blacklist it, then run mkinitrd :-)
Well it complained about the the absence, but so what?
The new mkinitrdwhatsthename... ah, dracut, it is way to talkative for my liking. I fail to see what is important.
And it writes even more to syslog... tons of pages.
What bothered me was the absence of the following modules
- unix
- atkbd
- swap
- ext2
- LVM2_member
Does that matter?
I don't know. It could also be that they are builtins to the kernel, not modules.
Initrd now down from 12M to 9M
Yes.
Not long ago it was big enough so that three kernels would not fit in my separate /boot partition. And you know that during the update of a kernel there are three in existence: the current, the previous, and the new. After the reboot, a process delete the "was previous", leaving again two only.
After I replaced my hard disk with a bigger one, I resized the partitions:
Telcontar:~ # df -h /boot/ Filesystem Size Used Avail Use% Mounted on /dev/sda5 1011M 44M 917M 5% /boot Telcontar:~ #
Way too big, but I thought the same thing on previous iterations...
On 12/28/2016 12:50 PM, Carlos E. R. wrote:
On 2016-12-28 18:21, Anton Aylward wrote:
On 12/28/2016 11:45 AM, Carlos E. R. wrote:
On 2016-12-28 16:44, Anton Aylward wrote:
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
Uninstall the package, blacklist it, then run mkinitrd :-)
Well it complained about the the absence, but so what?
The new mkinitrdwhatsthename... ah, dracut, it is way to talkative for my liking. I fail to see what is important.
And it writes even more to syslog... tons of pages.
Are you saying that I can rm drakut as well as plymouth?
Way too big, but I thought the same thing on previous iterations...
Hmmmm # df -h /boot Filesystem Size Used Avail Use% Mounted on /dev/sda1 1.9G 98M 1.7G 6% /boot
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-29 00:05, Anton Aylward wrote:
On 12/28/2016 12:50 PM, Carlos E. R. wrote:
On 2016-12-28 18:21, Anton Aylward wrote:
On 12/28/2016 11:45 AM, Carlos E. R. wrote:
On 2016-12-28 16:44, Anton Aylward wrote:
So, Carlos, did you eliminate plymouth from your initrd? How did you do that?
Uninstall the package, blacklist it, then run mkinitrd :-)
Well it complained about the the absence, but so what?
The new mkinitrdwhatsthename... ah, dracut, it is way to talkative for my liking. I fail to see what is important.
And it writes even more to syslog... tons of pages.
Are you saying that I can rm drakut as well as plymouth?
Nonononononononono.
Never on your life. Unless you wish to do what Linda does ;-)
Way too big, but I thought the same thing on previous iterations...
Hmmmm # df -h /boot Filesystem Size Used Avail Use% Mounted on /dev/sda1 1.9G 98M 1.7G 6% /boot
Much bigger than mine! LOL.
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
On Wednesday 2016-12-28 18:21, Anton Aylward wrote:
Well it complained about the the absence, but so what? What bothered me was the absence of the following modules
- unix
- atkbd
- swap
- ext2
- LVM2_member
Does that matter?
Built-in, and for good measure. If UNIX sockets were a kernel module, a user-space loader to load the file from disk might just require UNIX sockets itself. At least if you dbusify everything :D
ext4.ko supports ext2 and ext3 too, if so enabled.
On 12/28/2016 02:06 PM, Jan Engelhardt wrote:
ext4.ko supports ext2 and ext3 too, if so enabled.
There doesn't seem to be an explicit module for ext4
I can see
lsinitrd initrd | grep ext lrwxrwxrwx 1 root root 29 Dec 28 12:15 lib64/libext2fs.so.2 -> ../usr/lib64/libext2fs.so.2.4 lrwxrwxrwx 1 root root 21 Dec 28 12:15 sbin/fsck.ext2 -> ../usr/sbin/fsck.ext2 -rwxr-xr-x 1 root root 285064 Dec 28 12:15 usr/lib64/libext2fs.so.2.4 lrwxrwxrwx 1 root root 16 Dec 28 12:15 usr/lib64/libext2fs.so.2 -> libext2fs.so.2.4 -rwxr-xr-x 2 root root 256992 Dec 28 12:15 usr/sbin/fsck.ext2
I also see I have raid6 although I'm not using that :-/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-29 01:05, Anton Aylward wrote:
On 12/28/2016 02:06 PM, Jan Engelhardt wrote:
ext4.ko supports ext2 and ext3 too, if so enabled.
There doesn't seem to be an explicit module for ext4
Because it doesn't come as a module, it is inside the kernel binary, so that the kernel alone can read the filesystem.
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
29.12.2016 03:53, Carlos E. R. пишет:
On 2016-12-29 01:05, Anton Aylward wrote:
On 12/28/2016 02:06 PM, Jan Engelhardt wrote:
ext4.ko supports ext2 and ext3 too, if so enabled.
There doesn't seem to be an explicit module for ext4
Because it doesn't come as a module, it is inside the kernel binary, so that the kernel alone can read the filesystem.
Interesting that it does on Leap. I miss reasons to build it into kernel on distribution that defaults to btrfs + xfs on new install :)
On 2016-12-29 04:40, Andrei Borzenkov wrote:
29.12.2016 03:53, Carlos E. R. пишет:
On 2016-12-29 01:05, Anton Aylward wrote:
On 12/28/2016 02:06 PM, Jan Engelhardt wrote:
ext4.ko supports ext2 and ext3 too, if so enabled.
There doesn't seem to be an explicit module for ext4
Because it doesn't come as a module, it is inside the kernel binary, so that the kernel alone can read the filesystem.
Interesting that it does on Leap. I miss reasons to build it into kernel on distribution that defaults to btrfs + xfs on new install :)
Not everybody goes with those defaults ;-)
Also, if for some reason you need to have a separate /boot partition, it is typically ext2/3/4
Now, xfs... perhaps it is a module :-?
29.12.2016 06:44, Carlos E. R. пишет:
On 2016-12-29 04:40, Andrei Borzenkov wrote:
29.12.2016 03:53, Carlos E. R. пишет:
On 2016-12-29 01:05, Anton Aylward wrote:
On 12/28/2016 02:06 PM, Jan Engelhardt wrote:
ext4.ko supports ext2 and ext3 too, if so enabled.
There doesn't seem to be an explicit module for ext4
Because it doesn't come as a module, it is inside the kernel binary, so that the kernel alone can read the filesystem.
Interesting that it does on Leap. I miss reasons to build it into kernel on distribution that defaults to btrfs + xfs on new install :)
Not everybody goes with those defaults ;-)
Following this logic everything should be built into kernel.
Also, if for some reason you need to have a separate /boot partition, it is typically ext2/3/4
Which is the last thing kernel needs to access during boot. Actually, kernel does *not* need /boot to boot.
Now, xfs... perhaps it is a module :-?
Anton Aylward wrote:
On 12/27/2016 10:45 PM, Carlos E. R. wrote:
But Linda doesn't use initrd, if I remember correctly.
Correct.
Ah, yes, that's my recollection too, now you come to mention it.
I see earlier on in the 'lsinitrd' output that Drakut includes some modules. Perhaps the issue is in getting the right modules included.
--- Have you ever built your own kernel?
As a start, look at output of: sudo lspci -k|grep kernel|cut -d: -f2|sort|uniq
That will show you modules that you should *likely* build into your kernel as they are drivers for _your_ machine's hardware, though not all of them are needed for boot. For example, probably don't need network to boot unless you boot from net.
"lsmod", will show you what modules have been dynamically loaded, while "ls /sys/module" will show you all modules the kernel has builtin OR dynamically loaded.
For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
Yes, but I absolutely need LVM.
--- Do you need your rootfs to be LVM? FWIW, My 12G root is about half full. I wanted to minimize software complexity involved in booting root. I also wanted to constrain it at the start of the disk to limit seeks for reading the root files, so it is the 1st partition with only the 1st half of the disk used for content (short-stroking), also to limit seek latency on 15K SAS disks.
The rest of my system uses lvm.
Well, there's
- what don't I need if all I'm doing is a regular boot?
--- My initial boot uses 'B'oot mode (i.e.: /etc/rc.d/boot.d, pre-SysD):
sysv -B
tgt: boot.d-start S01boot.usr-mount, S01setserial, S05boot.sysctl, S05boot.udev S05boot.localnet, S05boot.klog, S05boot.swap, S05boot.isapnp S05boot.ldconfig, S10boot.lvm, S10boot.clock, S15boot.localfs S15boot.lvm_monitor, S20boot.cgroup, S20boot.assign_netif_names S20boot.sys_settings, S25boot.sysstat
and
- what do I need to do a maintenance boot?
For emergency repair, boot is replaced by 'S'ingle mode:
sysv -S
tgt: rcS.d-start S05boot.clock, S90single
where single pretty much just sets up keyboard on start.
I have
dracut modules: bash warpclock -- what's this
Time travel? (likely, "jump" clock to some value).
convertfs
--- ???
ifcfg
--- do you want/need networking? I just make sure the network interfaces are named correctly.
drm
--- Setup graphics mode?
btrfs -- What if I convert ROOTFS to ext4?
---- I build my rootfs into my kernel, otherwise how would it get mounted?
dm, kernel-modules
--- kernel builtins
lvm
--- On disk.
resume
Not needed in my config.
rootfs-block
terminfo -- ??
--- I want to be able to run vim in console mode. Do you? The rest are a bit alien to my boot so hard to say which you need, but, at least, should be on your initrd or root...
On 12/28/2016 05:40 PM, L A Walsh wrote:
Have you ever built your own kernel?
Not since I started using openSuse.
Well, maybe. If you 'compile from source", the see above. if you mean buqqer around with mkinitrd to include and omit -- and its a LOT easier with Dracut! -- then yes.
As a start, look at output of: sudo lspci -k|grep kernel|cut -d: -f2|sort|uniq
That needs "grep -i"
That will show you modules that you should *likely* build into your kernel as they are drivers for _your_ machine's hardware, though not all of them are needed for boot. For example, probably don't need network to boot unless you boot from net.
Ah, "probably". If I'm not booting from the net then I cant see why I should. What exception are you thinking of?
By the same reasoning, I should be able to eliminate RAID (md and dm), all of CRYPTO/LUKS, and all of USB. Perhaps the serial port can go too. This is only booting this specific machine, its not a generic, portable kernel such as we get on the DVD. What else don't I need?
I'm reminded of the scene in book/movie "The Martian" where he's stripping the rocket down to reduce weight. oops, there goes the crash couch! its going to be a rough ride without that.
"lsmod", will show you what modules have been dynamically loaded, while "ls /sys/module" will show you all modules the kernel has builtin OR dynamically loaded.
Hmm. I (still) can't see where the ext[234] driver is, but I do note that BtrFS requires a few other modules like XOR and raid6_pq. More incentive to go for a ext4 RootFS, eh?
For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
Yes, but I absolutely need LVM.
Do you need your rootfs to be LVM?
I am @#$$%$#@ sick and tired of having the RootFS grow and having to take down the disk and shuffle partitions around. I use the predecessor to LVM on an IBM AID years go and had great joy rearranging a DB@ database for balance and performance across 96 spindles! Oh, joy! oh. tabulations!
Having /usr as part of the RootFS is ... not nice. It complicates provisioning.
FWIW, My 12G root is about half full.
I think if I could have /usr on a separate FS as of old and this whole stillness of where the binaries live sorted out, then yes my RootFS would be a LOT smaller. But I don't want to follow your approach; I want to have a system that follows an upgrade path as delivered.
I wanted to minimize software complexity involved in booting root. I also wanted to constrain it at the start of the disk to limit seeks for reading the root files, so it is the 1st partition with only the 1st half of the disk used for content (short-stroking), also to limit seek latency on 15K SAS disks.
I recall from the 1970s and 1980s a lot of papers about disk latency, rotational spacing when doing a MKFS, how the Berkeley Fast File System approached that, the idea of inverted FS so that inodes were groups adjacently on single RM02 drives with "/" and "usr" (in the days before we had "/home"). I remember experimenting with a DEC 8-drive controller (but only one data path at a time). It was all quite pointless. Every optimization was promptly outclassed by and advance in the disk technology. When I worked on optical storage, every advance in rewritable magneto-optical was matched by the same advance in plain magnetics.
Now we have SSDs that are dirt cheap. My core system, that is without /home (and all my photographs) and the web stuff on /srv (not the least of which is ownCloud for my home WLAN and mobile devices) will fit on a 60G pci-e insert that I can get for less than router cost. Check eBay for example. If I omit /srv and the ~anton/Photographs and ~anton/Movies and ~anton/music then everything including /usr/share can fit, comfortably, on a 120G SSD.
I might just do that some time next year. Santa didn't deliver on the SSD :-(
So the kind of optimization you talk about becomes irrelevant.
My home system has to live by the whim of my free time and what's left over after mortgage and food; you're in a commercial setting. You can probably make budget submission :-) Goodluck!
Well, there's
- what don't I need if all I'm doing is a regular boot?
My initial boot uses 'B'oot mode (i.e.: /etc/rc.d/boot.d, pre-SysD):
And in this day and age when openSuse ships with systemd, you have to maintain that by hand. Something I'm not willing to do.
[Big snip about using stuff in /etc/rc.d etc etc which I grew up with and to be honest am glad is past. I much prefer systemd.]
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-29 01:51, Anton Aylward wrote:
On 12/28/2016 05:40 PM, L A Walsh wrote:
Have you ever built your own kernel?
Not since I started using openSuse.
I have, but not recently.
By the same reasoning, I should be able to eliminate RAID (md and dm), all of CRYPTO/LUKS, and all of USB.
Keyboards nowdays are on USB, so you need it in the emergency boot. I guess you don't use an encrypted root.
Perhaps the serial port can go too.
It is used for early debug.
This is only booting this specific machine, its not a generic, portable kernel such as we get on the DVD. What else don't I need?
But that is micromanaging. IMHO, not worth the hassle.
I'm reminded of the scene in book/movie "The Martian" where he's stripping the rocket down to reduce weight. oops, there goes the crash couch! its going to be a rough ride without that.
Ho ho. Yes.
I think if I could have /usr on a separate FS
I do, it works out of the box. You simply need more things on the initrd, but it is automatic.
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
On 12/28/2016 08:02 PM, Carlos E. R. wrote:
Keyboards nowdays are on USB, so you need it in the emergency boot.
Ah yes, many mobos simply don't have the jacks for kbd and mouse any more, its all USB.
I guess you don't use an encrypted root.
I don't see the point in encrypting the root. User DATA yes, the programs that are available for download from the repositories and on the DVD - no point in encrypting them.
All my site passwords, email passwords, web account details, anything that valuable, all lives under /home.
Perhaps with mobile devices things are different. Certainly there's a LOT of critical stuff on my cell phone!
On 2016-12-29 04:15, Anton Aylward wrote:
On 12/28/2016 08:02 PM, Carlos E. R. wrote:
Keyboards nowdays are on USB, so you need it in the emergency boot.
Ah yes, many mobos simply don't have the jacks for kbd and mouse any more, its all USB.
Mine does, but the original keyboard died and the replacement is USB. They don't last that long nowdays. ps/2 keyboards are difficult to obtain, and those I found I didn't like.
I guess you don't use an encrypted root.
I don't see the point in encrypting the root. User DATA yes, the programs that are available for download from the repositories and on the DVD - no point in encrypting them.
I didn't see the point initially, either, but there is also data on /etc that can be sensitive. WiFi password, for instance. Then there is /tmp, databases in /var...
I do not encrypt root because it is inconvenient and I do not like the way yast does it, but I do see the point.
Anyway, the distribution has to cater for such a case.
All my site passwords, email passwords, web account details, anything that valuable, all lives under /home.
Perhaps with mobile devices things are different. Certainly there's a LOT of critical stuff on my cell phone!
A laptop is different.
On 12/28/2016 10:22 PM, Carlos E. R. wrote:
On 2016-12-29 04:15, Anton Aylward wrote:
On 12/28/2016 08:02 PM, Carlos E. R. wrote:
Keyboards nowdays are on USB, so you need it in the emergency boot.
Ah yes, many mobos simply don't have the jacks for kbd and mouse any more, its all USB.
Mine does, but the original keyboard died and the replacement is USB. They don't last that long nowdays. ps/2 keyboards are difficult to obtain, and those I found I didn't like.
I guess you don't use an encrypted root.
I don't see the point in encrypting the root. User DATA yes, the programs that are available for download from the repositories and on the DVD - no point in encrypting them.
I didn't see the point initially, either, but there is also data on /etc that can be sensitive. WiFi password, for instance. Then there is /tmp, databases in /var...
While I have every sympathy for put all config in /etc policy, putting unencrypted passwords there is a risk. In general, what's in /etc tends to be world readable.
As far as /tmp and /var goes, they are not the RootFS and I can see the logic in having them encrypted while at rest.
But that's the issue, isn't it, 'while at rest'. I've mentioned before the boot where an uber-hacker's laptop is stolen while powered up. So long as it stays powered up and active it doesn't matter that the partitions are encrypted. So OK, if you leave your laptop on the seat of your supposedly locked car ... or perhaps from beside you when your attention was elsewhere
http://www.dailymail.co.uk/news/article-559178/Military-laptop-stolen-McDona...
https://www.theguardian.com/uk/2008/jan/22/politics.military
http://arstechnica.com/security/2008/01/uk-military-laptop-theft-exposes-tho...
http://www.philly.com/philly/news/20161106_3_laptops_stolen_from_Clinton_cam...
But I don't see the point in encrypting the FS or drives of the always-up machines in a data centre or their SMB equivalents.
Decommissioned drives, you say? Well if you don't have a policy about scrubbing those or physically destroying them, yes i suppose it is a risk, but that risk has nothing to do with encryption and everything to do with your disposal policy.
I do not encrypt root because it is inconvenient and I do not like the way yast does it, but I do see the point.
Anyway, the distribution has to cater for such a case.
All my site passwords, email passwords, web account details, anything that valuable, all lives under /home.
Perhaps with mobile devices things are different. Certainly there's a LOT of critical stuff on my cell phone!
A laptop is different.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-12-29 15:29, Anton Aylward wrote:
On 12/28/2016 10:22 PM, Carlos E. R. wrote:
I guess you don't use an encrypted root.
I don't see the point in encrypting the root. User DATA yes, the programs that are available for download from the repositories and on the DVD - no point in encrypting them.
I didn't see the point initially, either, but there is also data on /etc that can be sensitive. WiFi password, for instance. Then there is /tmp, databases in /var...
While I have every sympathy for put all config in /etc policy, putting unencrypted passwords there is a risk. In general, what's in /etc tends to be world readable.
The people that can read /etc while it is mounted do not bother me. They have permission. The people that steal a (powered down) computer or hard disk do.
As far as /tmp and /var goes, they are not the RootFS and I can see the logic in having them encrypted while at rest.
It is easier to encrypt everything than to go hunting around for areas in the hard disk that may or not hold some sensitive material.
For instance, about a month or ago I worried about Firefox storing downloaded PDF files, like receipts, for display, somewhere under /tmp with limited permissions. I modified setup so that the temporary directory would be encrypted or under /home. Well, after upgrading to 42.2 the modification disappeared and I have to do it again.
If root were encrypted, I would not have to worry about it.
But that's the issue, isn't it, 'while at rest'. I've mentioned before the boot where an uber-hacker's laptop is stolen while powered up. So long as it stays powered up and active it doesn't matter that the partitions are encrypted. So OK, if you leave your laptop on the seat of your supposedly locked car ... or perhaps from beside you when your attention was elsewhere
Yes, I know about that.
But I don't see the point in encrypting the FS or drives of the always-up machines in a data centre or their SMB equivalents.
The point is that hard disks can be stolen; not by chance, but intentionally.
Decommissioned drives, you say? Well if you don't have a policy about scrubbing those or physically destroying them, yes i suppose it is a risk, but that risk has nothing to do with encryption and everything to do with your disposal policy.
Encryption ensures that if by accident the disk is simply disposed, no data can be accessed. Yes, the policy can be "destroy them", but policies can be forgotten. Perhaps the disk develops some failure and can not be accessed, so a rewrite fails, and the person doing it doesn't have a metal scrapper, so when nobody looks just dumps it.
Security is about having several layers; full encryption is just one layer more.
- -- Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" (Minas Tirith))
On 12/29/2016 07:23 PM, Carlos E. R. wrote:
The people that can read /etc while it is mounted do not bother me. They have permission. The people that steal a (powered down) computer or hard disk do.
Reading reports I think that many of the laptop thefts are not targeted, they are opportunistic and for resale. Maybe you too have been offered a laptop or other portable (or something else) that is heavily discounted. As in "bad IMEI", meaning the phone has been blacklisted, most likely because it was stolen.
The most immediate impact is loss of availability, of function. That applies even if the disk was encrypted. It applies even if you have backup. There is a delay until the device is replaced and the data restored.
This leads to a number of mitigation strategies. One salesman I know has a stripped down phone for travel, just in case the authorities (or the secret police, or others) decide to 'confiscate' it or borrow it to take an image.
Part of the problem is that modern laptops are too capable,too much processing power, too much storage. If they were 'slimmer' we wouldn't be tempted to load them up and, as many people I know, use them as their primary device.
Anton Aylward wrote:
Part of the problem is that modern laptops are too capable,too much processing power, too much storage. If they were 'slimmer' we wouldn't be tempted to load them up and, as many people I know, use them as their primary device.
---- laptop and desktop sales are declining and being replaced with lower-powered handheld devices. It's one of the reasons why Win7's desktop with transparency and 3D features may, in retrospect be the pinnacle of user-desktops. With Win8+10, the desktops were dumbed down to give a common experience with low-end devices -- thus a return to a 1980's metro look. It's very disappointing.
My last machine when I still worked for sgi was a laptop 'workstation'. It fit my needs at the time.
My old 'server' (really an old tower-workstation) died @ 10 years so I needed to replace it quickly and went for a 'real' server type system.
I liked the architecture so much, when I needed to replace my laptop, I looked for a similar HW-base for my desktop, but with graphics support. It was "mandatory" as win7 was just coming out -- with full and incremental backups disabled. I didn't trust Win7 w/my data -- good thing too, had to reinstall everything from scratch 2 times due to Win7 eating my disks (the content, actually).
As I mentioned in response to Anton's fear of having a non-LVM root, I haven't had to resize my root partition since I setup the system, so Anton's fear of resizing root,
"I am @#$$%$#@ sick and tired of having the RootFS grow and having to take down the disk and shuffle partitions around."
wasn't really a concern, especially since I can move directories that grow too large off to their own subdir on /home (which is on top of LVM).
On 12/29/2016 07:23 PM, Carlos E. R. wrote:
The point is that hard disks can be stolen; not by chance, but intentionally.
I've heard about incidents where a burglar opened up every desktop machine in a business and made away with a sack of hard drives. Some swag! Very obviously targeted and very obviously a case of industrial espionage.
This was at a security seminar, the speaker was from CSIS, the Canadian Security Intelligence Service (very roughly the Canadian equivalent of the FBI with a bit of the CIA mixed in). He illustrated his presentation with photos of the offices and comments on the physical security. You don't have to be a security droid to see the point he was making, that those people had Piss Poor Physical security.
My biggest objection to encryption with Linux is that it feels clunky. I've used some forms of military and commercial data encryption and by comparison LUKS is .... awkward.
And in reality, there are many threats that encryption does NOT protect against.
The obvious one, as I've pointed out, is an 'attack' on a running system. That's not just the classical over-the-wire attacks, it includes phishing and as I've mentioned the theft of a running laptop as John Stanford described in his novel "The hanged man's song"
Then there's the "Cold boot" attack; the memory retains an image and perhaps they key even after the system has been shut down. Such attacks have been demonstrated to be effective against full disk encryption schemes of various vendors and operating systems, even where a Trusted Platform Module (TPM) secure cryptoprocessor is used. This is because the problem is fundamentally a hardware (insecure memory) and not a software issue. "Obviously" the solution is to encrypt your memory as well :-)
And then there's human shortcoming.
Security is always a cost-benefit exercise. Is the data important enough to warrant the expense, the effort and the inconvenience? This is a business decision, not a technical issue.
On 2016-12-30 03:21, Anton Aylward wrote:
On 12/29/2016 07:23 PM, Carlos E. R. wrote:
The point is that hard disks can be stolen; not by chance, but intentionally.
I've heard about incidents where a burglar opened up every desktop machine in a business and made away with a sack of hard drives. Some swag! Very obviously targeted and very obviously a case of industrial espionage.
Yes...
My biggest objection to encryption with Linux is that it feels clunky. I've used some forms of military and commercial data encryption and by comparison LUKS is .... awkward.
Well, as LUKS is what I have available, I use it.
Then there is disk hardware native encryption, but I don't know how to boot that.
Security is always a cost-benefit exercise. Is the data important enough to warrant the expense, the effort and the inconvenience? This is a business decision, not a technical issue.
I prefer to inconvenience the burglars if I can ;-)
Hello,
Am Freitag, 30. Dezember 2016, 13:06:41 CET schrieb Carlos E. R.:
On 2016-12-30 03:21, Anton Aylward wrote:
My biggest objection to encryption with Linux is that it feels clunky. I've used some forms of military and commercial data encryption and by comparison LUKS is .... awkward.
Well, as LUKS is what I have available, I use it.
Then there is disk hardware native encryption, but I don't know how to boot that.
The more interesting question is how to _trust_ that.
With LUKS, you can read the source code or, if you are not a programmer and/or encryption expert, ask someone with programming and encryption knownledge to audit it. You won't get the audit for free, but it's possible.
With hardware encryption or military / commercial encryption, you can't check what it does. IIRC there was more than one "encrypted" hard drive that was easy to decrypt, and I also wouldn't trust closed source encryption too much. For examples of both, check the [23][0-9]c3 archives ;-)
Security is always a cost-benefit exercise. Is the data important enough to warrant the expense, the effort and the inconvenience? This is a business decision, not a technical issue.
I prefer to inconvenience the burglars if I can ;-)
Same here. If someone steals my hard drive, the only readable thing will be /boot ;-) (AFAIK nowadays even /boot can be encrypted, but I'm too busy or lazy (chose one ;-) to reinstall just for that.)
BTW: Disk encryption has another advantage - besides preventing reading my files, it also prevents that someone modifies something on my disk. Well, random changes aka damage can be done - but doing a targeted change like installing a trojan is impossible.
Regards,
Christian Boltz
On 2016-12-30 21:31, Christian Boltz wrote:
Hello,
Then there is disk hardware native encryption, but I don't know how to boot that.
The more interesting question is how to _trust_ that.
Well, I can store sensitive material under LUKS, and the entire disk under "that" ;-)
BTW: Disk encryption has another advantage - besides preventing reading my files, it also prevents that someone modifies something on my disk. Well, random changes aka damage can be done - but doing a targeted change like installing a trojan is impossible.
Yes. At least on a powered down machine.
Anton Aylward composed on 2016-12-28 22:15 (UTC-0500):
Carlos E. R. wrote:
Keyboards nowdays are on USB, so you need it in the emergency boot.
Ah yes, many mobos simply don't have the jacks for kbd and mouse any more, its all USB.
Not exactly, NAICT. It seems Intel got the idea some time back that USB was sufficient for all input devices, so for a while most new motherboard designs included no PS/2 ports. Apparently there were enough complaints that most more recent ATX and mATX motherboards have a single PS/2 port on the back panel, and some even have two. More compact form factors commonly omit both.
Anton Aylward wrote:
That will show you modules that you should *likely* build into your kernel as they are drivers for _your_ machine's hardware, though not all of them are needed for boot. For example, probably don't need network to boot unless you boot from net.
Ah, "probably". If I'm not booting from the net then I cant see why I should. What exception are you thinking of?
--- It's the exception that I'm *not* thinking of that I was thinking of ... :-)
"lsmod", will show you what modules have been dynamically loaded, while "ls /sys/module" will show you all modules the kernel has builtin OR dynamically loaded.
Hmm. I (still) can't see where the ext[234] driver is,
--- Why do you think you have it loaded? I don't have it loaded in my kernel.
but I do note that BtrFS requires a few other modules like XOR and raid6_pq. More incentive to go for a ext4 RootFS, eh?
---- xfs anyone? :-)
For example, I have everything except /boot and SWAP on LVM. That includes the ROOTFS. So the LVM modules needs to be included in initrd.
Yes, the initrd becomes bigger.
Yes, but I absolutely need LVM.
Do you need your rootfs to be LVM?
I am @#$$%$#@ sick and tired of having the RootFS grow and having to take down the disk and shuffle partitions around.
--- I've yet to resize my root since I got it:
xfs_admin -u /dev/sdc1
UUID = 4a61f0fb-2397-ac7b-2009-07181557473b
July 18, 2009 at 15:57:47.354
I made root 12G (currently using half). (/usr /var /usr/share, /home and /var/cache/squid are all separate in addition to other local filesystems).
Having /usr as part of the RootFS is ... not nice. It complicates provisioning.
--- :-) Agreed!
FWIW, My 12G root is about half full.
I think if I could have /usr on a separate FS as of old and this whole stillness of where the binaries live sorted out, then yes my RootFS would be a LOT smaller. But I don't want to follow your approach; I want to have a system that follows an upgrade path as delivered.
---- If you have an initrd, having a separate /usr is supported.
If you want to boot directly from disk, as suggested by systemd docs, for speed, you aren't suse supported.
I wanted to minimize software complexity involved in booting root. I also wanted to constrain it at the start of the disk to limit seeks for reading the root files, so it is the 1st partition with only the 1st half of the disk used for content (short-stroking), also to limit seek latency on 15K SAS disks.
It was all quite pointless. Every optimization was promptly outclassed by and advance in the disk technology.
--- Disks are advertised w/an average latency -- w/ 15K SAS drives usually averaging under 10ms. 2ms of that is for rotational position on average, rest is inner/outer seeks. If you cut the max track in half, the inner/outer number goes to less than half (because more tracks are are in outer rings which are track 0).
In rotating media, such concerns are still valid and are separate from SW optimizations. Of course one could consider SSD's which have no positional seek, but the do have some positioning latency that is helped by occasional defragmenting, I've noticed.
Now we have SSDs that are dirt cheap. My core system, that is without /home (and all my photographs) and the web stuff on /srv (not the least of which is ownCloud for my home WLAN and mobile devices) will fit on a 60G pci-e insert that I can get for less than router cost. Check eBay for example. If I omit /srv and the ~anton/Photographs and ~anton/Movies and ~anton/music then everything including /usr/share can fit, comfortably, on a 120G SSD.
I might just do that some time next year. Santa didn't deliver on the SSD :-(
So the kind of optimization you talk about becomes irrelevant.
---- It was relevant in 2009, and still is for my system, since it is on the same system my /home is on. My data disks wouldn't fit on an SSD (about 24T for data and 12T dedicated to backup, each of which is running RAID10). Your 120G wouldn't go far if holding video (my living room computer runs Win7 and feeds my video screen, but the content is on my linux machine in a back bedroom).
My home system has to live by the whim of my free time and what's left over after mortgage and food; you're in a commercial setting. You can probably make budget submission :-) Goodluck!
--- My figures are for my home system.
And in this day and age when openSuse ships with systemd, you have to maintain that by hand. Something I'm not willing to do.
---- It's mostly automated with continuing development as something "bugs me".
[Big snip about using stuff in /etc/rc.d etc etc which I grew up with and to be honest am glad is past. I much prefer systemd.]
--- You have a user-system. Sd was focused on that.
Anton Aylward wrote:
Which gets back to the issue, how many of those are essential for whatever you view as 'boot'? I ask this because those don't appear in the 'lsinitrd'.
--- I differentiated between boot and getting to login prompt, but was told that sysd wanted /usr mounted in order to drive everything up to the login prompts in parallel.
I have no problem separating them. Things to boot are in "boot.d", and run-level scripts are in rcX.d..
You might also notice when you run 'lsinitrd'...
--- What would I run lsinitrd on? I boot directly from my root partition on my hard disk. Then it mounts /usr, et al.
25.12.2016 16:53, Dr. Werner Fink пишет:
This discussion is useless ... could we please stay with /bin, /sbin as well with /usr/bin and /usr/sbin. Even upstream of systemd never had said that /bin/sh has to become /usr/bin/sh to avoid POSIX violations. The same holds true for other essential user commands as well as for other system commands.
It said that having /bin -> usr/bin would provide better compatibility with existing programs by ensuring that both /bin/sh and /usr/bin/sh are always present and refer to the same thing.
IMHO this forced merge is a misunderstanding of having /usr as part of the root file system. Wheras having one root file system with /usr make sense as to many libraries and dynamicallly loaded files are located below /usr (e.g. libraries used for nfs4 and also dynamicallly loaded files of the glibc), there is no need to enforce the depopulation of the system basic paths /bin and /sbin
One of reasons behind usr merge was having complete OS binaries in one location and clear separation of this location from local configuration. It's not something that gives you immediate benefits right now, that's true.
Andrei Borzenkov wrote:
One of reasons behind usr merge was having complete OS binaries in one location and clear separation of this location from local configuration. It's not something that gives you immediate benefits right now, that's true.
The were separate: /etc contained config, and /bin, /lib{,64} contains binaries.
"root" contains local config stuff as well, since that's where new, local directories live.
I've seen "arch" (OS binaries) be mounted in root and in /usr, and seen arch-specific binaries in /arch/{bin,lib}...
On 12/26/2016 03:39 PM, L A Walsh wrote:
Andrei Borzenkov wrote:
One of reasons behind usr merge was having complete OS binaries in one location and clear separation of this location from local configuration. It's not something that gives you immediate benefits right now, that's true.
The were separate: /etc contained config, and /bin, /lib{,64} contains binaries.
"root" contains local config stuff as well, since that's where new, local directories live.
I've seen "arch" (OS binaries) be mounted in root and in /usr, and seen arch-specific binaries in /arch/{bin,lib}...
That makes sense to me. We also have long standing /local, and since the 1970s I've deployed & set up systems that have ~/{bin,lib,src,tmp} for each user.
We have an /usr/X11R6/ which has the libraries for X11
We have (on my system) an /opt/kde3
We have a /usr/lib/libreoffice/
I don't see a reason why we shouldn't have a tree for stuff that is specific to Suse.
Anton Aylward wrote:
I don't see a reason why we shouldn't have a tree for stuff that is specific to Suse.
---- I told the perl maintainer ages ago, that if they couldn't deal with the system perl being upgraded, then maybe they should have a dist-perl that is needed for crippled packages that can't be recompiled (like standard perl packages, as in CPAN), that can be recompiled for a new perl. The maintainer basically said that the system perl was for opensuse, and users were not suppose to upgrade it, even for bug-fixes that didn't change the call-API.
I even contacted the perl devs who confirmed that minor version changes (5.20.0, 5.20.1, 5.20.2) in perl were "guaranteed" not to change the API, and modules for one minor version should "just work" in other minor versions. However, they don't on opensuse, because version numbers are hard coded into modules and perl causing wide spread breakage when perl is upgraded.
If they had the modules build to allow recompilation as other perl-cpan modules, it wouldn't be an issue in most cases -- even when major versions changed. But trying to get some to change is like pulling teeth.
Similar issue is the dependency hell, in many or most cases created by hard-linking libraries rather than doing run-time dynamic loading. Gvim has been that way (don't know about latest gvim). On windows (and on my machine at one point (not sure about now unless I test it!)), gvim tries to load python.so, perl.so, etc... and dynamically activates its ability to call those libs if they are present -- Versus -- opensuse just rolling over and dying. I usually tried to keep gvim on root, to help fix problems, but without /usr, (where all the interps were), it wouldn't start.
Thing is, I needed gvim as an editor -- not as a script-host. Yet due to bad-SW-build practices, it wasn't usable. :-(
A LOT of Opensuse products could be made alot more resilient by using run-time lib-loading and dynamic configuration.
It's not a new idea. SGI did it 15-20 years ago, and dynamic GUI configuration was done based on available features on **Plato** back in the 70's!
Do we ever learn? A large number of today's programmers didn't come out of a Computer Science curriculum and were self taught. While they know the most important and most needed stuff to stay employed, they usually don't care about "history" and thus are doomed to repeat mistakes and problems solved 40-50 years ago. :-(
Thus SW doesn't really progress (and I've seen signs of it going *backward*).
While CS used to design computer programs to be "user-friendly", now, because most programmers can't be bothered to learn how to do that, the demand is that *users*, be computer-adapted. Plegh.
Even Gates idea of 3D desktop paradigms, intro'd as Vista and Win7 rolled out, have back-stepped, as Win10 went for a 1980's GUI.
Everything is being dumbed down so users can be more easily controlled and doled out hand-helds w/all their info in the cloud that they pay monthly for -- w/no privacy.
Lovely.
I like the Win7 Aero, but it's not even an option above Win7. Bill G was so happy demonstrating some of its early features, and now...its being all rearchitected to run on the lowest end hardware. Plegh!
... I think I've gone off on tangents again....*sigh*
On 27/12/16 03:21, L A Walsh wrote:
It's not a new idea. SGI did it 15-20 years ago, and dynamic GUI configuration was done based on available features on **Plato** back in the 70's!
Do we ever learn? A large number of today's programmers didn't come out of a Computer Science curriculum and were self taught. While they know the most important and most needed stuff to stay employed, they usually don't care about "history" and thus are doomed to repeat mistakes and problems solved 40-50 years ago. :-(
A lot of yester-year's programmers didn't come out of a Comp-Sci course either - my first boss was a Maths graduate (I don't think they had Comp-Sci back then :-). I'm a Science graduate.
My first computer was the company mini with 256K ram for an entire office of 20 or more people :-)
Thus SW doesn't really progress (and I've seen signs of it going *backward*).
Driven by "oohhh shiny ...." - not helped by Comp Sci !!! I'm well known as being very anti-relational (well, actually, anti first-normal-form) because I simply apply elementary maths to the 12 rules and discover that they are neither complete, nor consistent, and in fact are highly self-contradictory. No wonder modern databases are a nightmare to maintain.
(My favourite challenge. Please STORE a list in an FNF database. Not model it, store it. Oh - and while you're at it, DON'T muddle data and meta-data in the same table. Have fun, it's impossible ... :-)
While CS used to design computer programs to be "user-friendly", now, because most programmers can't be bothered to learn how to do that, the demand is that *users*, be computer-adapted. Plegh.
The problem there is that the definition of "computer adapted" keeps on changing, and people can't adapt that fast :-(
here endeth the follow-on rant :-)
Cheers, Wol
-
Andrei Borzenkov -
Anton Aylward -
Carl Symons -
Carlos E. R. -
Christian Boltz -
Cristian Rodríguez -
Dr. Werner Fink -
Felix Miata -
Jan Engelhardt -
L A Walsh -
L.A. Walsh
-
Ludwig Nussel -
Michael Schroeder -
Robert Schweikert -
Ruediger Meier -
Stefan Seyfried -
Wols Lists