UsrMerge conversion plan
Hi, Soon the last fixes for packages that were incompatible with UsrMerge will enter Factory. Time to think about the conversion plan. I've updated the wiki page¹ to describe how projects in the build service can transition to UsrMerge and how to do the conversion at run time. Instead of triggering it manually in the initrd like Fedora did, the idea here basically is to leverage the renameat2(2) syscall in %pretrans of the file system package. So there is no intermediate, half merged state. Works fine in my test setups at least. It relies on coreutils' cp though. Can we live with that? I'm not a aware of library that could be used instead and obviously don't really want to re-implement cp just for this :-) An open question is file provides. Atm I'm using a provides generator that automatically adds the legacy /bin provides to well known packages. Do we want to do it that way or manually add file provides to packages? cu Ludwig [1] https://en.opensuse.org/openSUSE:Usr_merge -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
Dominique Leuenberger / DimStar wrote:
On a fresh install there is no need to run the usrmerge conversion script. So the lua part would just not call out into the script :-) cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On Mi, Feb 24, 2021 at 10:59, Ludwig Nussel <ludwig.nussel@suse.de> wrote:
How about rpm's embedded lua interpreter? Wouldn't it be the best choice for this kind of a job? LCP [Sasi] https://lcp.world
Sasi Olin wrote:
Abolutely! AFAIK as of now it does not have a recursive cp with all the features we need, nor offers the rename2 syscall. So if calling the external cp isn't good enough we'd have to re-implement the features into rpm and make them callable from lua. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On Wed, Feb 24, Ludwig Nussel wrote:
I think for the majority of file requires, the /usr/ location should be used. So adjust our RPMs requiring something in /bin. I see only problems with third party RPMs, and here the big question is, is there really anything required beside the coreutils utilities, /bin/sh and /bin/bash? I haven't seen anything else in the last time. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
Dominique Leuenberger / DimStar wrote:
For a few core packages I've manually added the provides already. I'd consider /bin/sh as ABI for example, so requiring that must work always no matter what IMO. The provides generator is on top for the rest. So if there's nobody advocating for manually adding more /bin provides, the generator is cheap to have just in case. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On Wednesday 2021-02-24 10:59, Ludwig Nussel wrote:
If you were to make it automatic, then everything in /usr/bin would generate a fileprovides with /bin, which is at least 2160 items for me. That's 1400% more than what is actually needed and such a disproportionate amount calls for the 130-or-so items in my /bin to be just static, especially since we can probably drop some: Though /bin/bash is seen quite often in #! lines, and /bin/chgrp sometimes here or there, I really really doubt anyone has recently used /bin/psfaddtable. Fewer provides, less pressure on the zypp stack.
Jan Engelhardt wrote:
Nah, the list of /bin files in TW at this point is known. The generator would only trigger on those. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On 2021/02/24 01:59, Ludwig Nussel wrote:
==== Why isn't this a /usr/bin => /bin merge? With /usr/bin being a bind-mount to /bin? That seems less disruptive than trying to expunge all reference to /bin. At the very least, /bin/<shell> is embedded in all shell scripts and shouldn't be relying on a symlink to function.
On Thu, Feb 25, 2021 at 4:05 PM L A Walsh <suse@tlinx.org> wrote:
I'm not going to rehash all the arguments from the past decade, but there is an openSUSE specific advantage: A singular /usr tree can be snapshotted trivially with all OS contents together, which simplifies system rollbacks and synchronization of OS content via Btrfs send/recv. With /bin being a symlink to /usr/bin, everything should work the same anyway. It's transparent to scripts. It works this way in Fedora, Ubuntu, macOS, Solaris, and others too. -- 真実はいつも一つ！/ Always, there's only one truth!
On 2021/02/25 13:28, Neal Gompa wrote
I'm not going to rehash all the arguments from the past decade,
The questions asked in 2012 weren't answered then, either. Many of the reasons stemmed from a need and presumption for /initrd, which I would think most would know isn't necessary either. Reasons for UsrMerge included a 33x inflated space needed for /root without it (email continued below this quoted message): -------- Original Message -------- Subject: Why is initrd needed... Date: Thu, 15 Nov 2012 00:17:47 -0800 From: Linda Walsh <suse(at)tlinx.org> To: opensuse-factory@opensuse.org, SuSE Linux <opensuse@opensuse.org> So far no one is able to explain why it is needed other than It's a fad or it's cool.. You have a 20MB RAM disc that you think you have to load and do secret things before you start my OS. I have a 12GB root disk (and RH/Fedora on their "Excuses for UsrMerge page, claims to need 400GB in root?! Talk about a broken model to be following!) Never the less, a 14 GB root with over 50% free.. and What, can't I put on my root hard disk that you have to have initrd? The drivers on my kernel are built-in. You are deliberately making people use a ram disk for no reason. Give me ANY reason. and don't say it's to mount /usr A "toy /usr/ that is not the full /usr partition lives on your ram disk in less than 20MB, I can throw away that much space in a 'dummy /usr' that the real /usr mounts over. It would be preferable to having to read all of it from disk and duplicating it 12 times for the 12 kernels I had in lilo when I last cleaned it out of initrd's that don't get used. Not only are you duplicating the space of all the utils in /usr that you need to put (ONLY because you moved them from /bin.. which if you hadn't done you wouldn't need to to copy them all to /usr). I read the /usrmerge propaganda... They miss a major point: If they want to merge /usr/bin with /bin, why not get rid of /usr/bin? Why not copy all of /usr/bin into /bin? But to boot do you really need 15G of /usr/share? Is /usr/games really required for boot? Do you need the complete linux source to boot in /usr/src? All of these irrelevant directories live in /usr The total of my /usr/bin, lib and lib64 (1.4G, 1.3G 3.2G) can easily live in the 9.7G free of my root. It's the rest of /usr that you are pulling along. If you want /usr/bin/lib/lib64 on root... why not just move them there? I split /usr and root because I don't need 20G of docs or fonts or source on my boot disk. Why would you claim that's the only viable answer you could come up with? Moreover, what is there on initrd that you can't put in root? -------- End Original Message -------- On 2021/02/25 13:28, Neal Gompa wrote:
Except for the actual OS (kernel), and libs in /lib, and /lib64(mostly same as /usr/lib64), and /etc/ and state in /var. As well as things living in /opt (a bind-mount to /home/opt on my system)
Symlinks aren't transparent. Mounts (and bind-mounts) are. /usr/bin can be mounted from /bin on boot but not the other way around. You can't cleanly snapshot /usr if it is part of the rootfs. But I've had 3rd party SW Virtual Box refuse to run with a symlink in the path.
On Thu, Feb 25, 2021 at 6:09 PM L A Walsh <suse@tlinx.org> wrote:
It isn't until it is. If you're doing custom kernels with everything built into a blob, you don't need it. If you don't care about ramfs based rescue environments, you don't need it. And on and on and on... But here's the cold hard truth: you are not operating in the normal case, and doing UsrMerge does not break your special case at all, while improving the quality of experience for the normal case.
With the exception of /var, /opt, and /home, everything is already moving to /usr by default in openSUSE. Even /etc is, with the /usr/etc containing the functioning defaults and /etc containing only host-specific overrides. There's an ongoing effort in both Fedora and openSUSE to empty /etc and eliminate OS-specific state from /var, such that the OS is centralized into /usr. As for the OS kernel and bootloader code, on openSUSE it *is* mostly in /usr and copied into /boot on install/upgrade. While not perfectly there yet, we're on the path to getting to a point where the OS is completely in /usr and everything else is user/host specific and is out of the realm of the package manager. -- 真実はいつも一つ！/ Always, there's only one truth!
On Friday, 26 February 2021 16:18:52 ACDT Andrei Borzenkov wrote:
This seems to me to be getting right away from the separation of privileges/ ownership ethos that has long been a hallmark of *nix-type systems. I thought system-owned, system- or admin-executable packages should be in /bin (with their corresponding libraries in /lib and configs in /etc), and user- executable binares in /usr/bin with corresponding /usr/lib/ and /usr/etc libs/ configs respectively. That makes perfect sense. Moving everyting into /usr/bin, /usr/lib and /usr/etc makes no sense whatsoever to me, but it seems that it's basically a tait accompli (the decision has already been made, for better or worse), so I guess we'll either have to live with it - and the breakages it will initially (and ineveitably) cause, or simply not upgrade past a certain pre-merge release. -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au CCNA #CSCO12880208 ==============================================================
On Friday 2021-02-26 07:48, Rodney Baker wrote:
You misread FHS. The first category of yours maps to (/usr/)sbin, the second category to (/usr/)bin. And since /usr is mounted Really Early(tm) - in comparison to 20 years ago -, I will postulate that we are simply making use of clause 3.15.1 § FHS 2.3 /usr is the second major section of the filesystem./usris shareable, read-only data. That means that/usrshould be shareable between various FHS-compliant hosts and must not be written to.
Despite being a very old standard written for a completely different decade, and the requirements/desires (snapshots, rollback) have certainly changed since, openSUSE's _default_ installation still adheres to FHS 2.3 section 3.15.1: "Programs executed after /usr is known to be mounted (when thereare no problems) are generally placed into /usr/(s)bin."
On Fri, Feb 26, Rodney Baker wrote:
As Jan already wrote, following the UNIX tradition: (/usr)/bin is for user executeable binaries, where /bin is for this binaries, which are required to mount /usr (/usr)/sbin is for system or admin executeable binaries, where /sbin is for this binaries, which are required to mount /usr. Since we mount /usr in the initrd, following your argumentation with UNIX tradition, /bin and /sbin should be empty on openSUSE ;)
From a standard Desktop user view your are right, the usrMerge doesn't make a difference on the first glance. But openSUSE is more than just a Desktop distribution. We provide many more variants like Server, Edge, Embedded. And we provide many features even from interest for normal Desktop users, like snapshot and rollback. And there are also requests and ideas to provide image based updates, not only package based updates. With the current setup not possible. And this other variants and features have requirements, which are best solveable with a system, where everything is strictly separated: /usr for vendor provided code /usr/local for local admin provided code /etc for local system depending configuration files or admin made changes /usr/etc (or something similar) for vendor provided configuration files /opt for the 3rd party provided code /srv should be empty, as the FHS requirements for /srv are not fulfillable with current package managers /var is for storing statefull data of applications. So no application should install anything there, as the admin is allowed to delete it and application needs to be able to scope with this. We have still many packages doing this wrong. usrMerge is one step on the way to cleanup the distribution for new requirements, use-cases and to allow new features. It will not be the last one. The long term goal is, all openSUSE packages only install stuff in /usr, nowhere else. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
On Friday, 26 February 2021 19:57:49 ACDT Thorsten Kukuk wrote:
I remember /sbin having statically-linked binaries that didn't depend on anything in /lib, which meant they could be useful for system recovery, too (but I'm going back a fair way now).
Fair enough, I guess. Snapshot and rollback are only useful when using btrfs, which I'll never use. They can't replace a well-maintained and properly- orchestrated backup regime, though. In my work world, for all the virtual servers and workstations I maintain (both Linux and Windows), VMWare provides snapshot and rollback capability independent of the guest OS or filesystem, without the overhead of something like btrfs for each VM.
Seems like it logically should have been /usr/loca/etc for consistency, but OK...
/usr/etc (or something similar) for vendor provided configuration files /opt for the 3rd party provided code
As it is now - no argument there.
/srv should be empty, as the FHS requirements for /srv are not fulfillable with current package managers
/srv is useful for hosting things like local web pages (/srv/http), ftp repos (/srv/ftp) and tftp repos (/srv/tftp). Nothing should be installed there by the default install, I agree. I assume that's what you mean, because otherwise, where should such content be hosted?
I agree that nothing should be installed in /var, except perhaps for applications creating bespoke logging directories under /var/log, or caching directories under /var/cache (e.g. zypper).
Thanks for your response (and Jan also). I'm still not a fan, but I (at least partly) get the reasoning. Cheers, Rodney. -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au CCNA #CSCO12880208 ==============================================================
On 2/26/21 8:38 PM, Rodney Baker wrote:
I'll disagree here, both are good, on the very odd occasion where a tumbleweed update leaves my system mostly non working with snapper I can get myself back up and running in a matter of minutes rather then having to pull out backups etc. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hello, On 2021-02-26 12:35, Simon Lees wrote:
Perhaps you mix up a standard desktop end-user's point of view with a "well maintained and properly orchestrated backup regime" in an enterprise environment where as much as possible automated remote administration without user interaction is preferred like "machine X has issues" => "recreate it completely from scratch" (of course that is likely an oversimplified extreme example)? Kind Regards Johannes Meixner -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 - 90409 Nuernberg - Germany (HRB 36809, AG Nuernberg) GF: Felix Imendoerffer
On 2/26/21 10:30 PM, Johannes Meixner wrote:
Like I said as my main point, BOTH approaches have there merits and are worth supporting well. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Andrei Borzenkov wrote:
I didn't want to open that topic yet as it's not a problem right now. I do think we should adjust all packages to actually only install into /usr for consistency. That can wait until after we actually implemented the UsrMerge though. As soon as the filesystem package is installed there are symlinks to /usr, so rpm will follow them just fine if a package tries to install to /. So kernel modules end up in /usr/lib/modules even though the package puts them into /lib/modules.
Even so, what with /boot/vmlinuz?
That's another separate topic but also the answer is move to /usr as well. I've tried to explain where this comes from and where it's going in an article I posted earlier: https://lnussel.github.io/2020/12/16/fslayout/ cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On 2021/02/25 13:28, Neal Gompa wrote:
um... Assume I have shell 'mash' in the new location /usr/bin/mash:
ll /usr/bin/mash -rwxr-xr-x 1 1284320 May 18 2018 /usr/bin/mash*
Now I create my transparent symlink:
ok, now I want to delete /bn/mash
safe-rm /bn/mash /usr/bin/safe-rm: no symlinks allowed in the path of /bn/mash
Explain to me how vendors checking for symlinks in file path won't be able to detect such and won't have an issue with ignoring something generally regarded as insecure. That suse-makers believe such a symlink is transparent and secure and that Suse should do it because everyone else is doing it is more than a little worrisome. Certain it lends support to the idea of not having a 'standard' system.
It isn't until it is. If you're doing custom kernels with everything built into a blob, you don't need it.
A blob? My standard vanilla kernel has # zgrep =m$ /proc/config.gz |wc -l 538 538 modules with # wc -l /proc/modules 45 45 of them currently loaded
If you don't care about ramfs based rescue environments, you don't need it.
My entire system is a rescue environment since things break fairly often. I.e. a ramfs can't hold everything needed and even if it did, why copy it into a ram-based block-dev? If you can read your ram-device, then you can read your sys files. Like someone else said, you have your rescue system on root. *Until recently* when I replace my hard disks, 'usr' was on a separate partition. But the system boot/bringup files were in /bin,lib,lib64, etc... one of the first tasks that init did was mount /usr: cd /etc/rc.d/boot.d Ishtar:/etc/rc.d/boot.d# ls -L S* S01boot.sysctl* S01boot.usr-mount* S02boot.udev* S04boot.lvm* S05boot.clock* S06boot.localnet* S11boot.localfs* S13boot.klog* S13boot.lvm_monitor* S13boot.swap* S13setserial* S14boot.isapnp* S14boot.ldconfig* S30boot.assign_netif_names* S50boot.cgroup* S50boot.create_devs* S50boot.sys_settings* S50boot.sysstat* If you evacuate all files to /usr/bin, of course there's a better argument for a ramdisk, but its still not needed as your rescue disk can be a separate partition on your root disk, or a rescue-image in a file on your boot disk. But deciding what limited rescue files go on a ram disk and copying it to ram first, vs. having your entire /bin+/lib dirs be your 'rescue resources' -- why go the extra step of creating a limited rescue disk vs booting native. And the most basic level of rescue: init=/bin/bash Will that work if bash is a symlink? It certainly wouldn't if my /usr was a separate partition like it used to be.
On Thursday 2021-02-25 22:05, L A Walsh wrote:
When you talk bind mounts, the direction does not matter, because the end result would be the same. Given /bin is almost empty anyway, might as well make /bin the bind mount to (the much more populated) /usr/bin.
Dominique Leuenberger / DimStar wrote:
On a fresh install there is no need to run the usrmerge conversion script. So the lua part would just not call out into the script :-) cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On Mi, Feb 24, 2021 at 10:59, Ludwig Nussel <ludwig.nussel@suse.de> wrote:
How about rpm's embedded lua interpreter? Wouldn't it be the best choice for this kind of a job? LCP [Sasi] https://lcp.world
Sasi Olin wrote:
Abolutely! AFAIK as of now it does not have a recursive cp with all the features we need, nor offers the rename2 syscall. So if calling the external cp isn't good enough we'd have to re-implement the features into rpm and make them callable from lua. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On Wed, Feb 24, Ludwig Nussel wrote:
I think for the majority of file requires, the /usr/ location should be used. So adjust our RPMs requiring something in /bin. I see only problems with third party RPMs, and here the big question is, is there really anything required beside the coreutils utilities, /bin/sh and /bin/bash? I haven't seen anything else in the last time. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
participants (11)
-
Andrei Borzenkov -
Dominique Leuenberger / DimStar -
Jan Engelhardt -
Johannes Meixner -
L A Walsh -
Ludwig Nussel -
Neal Gompa -
Rodney Baker -
Sasi Olin -
Simon Lees -
Thorsten Kukuk