Dominique Leuenberger / DimStar wrote:

On Wed, 2021-02-24 at 10:59 +0100, Ludwig Nussel wrote:

...

Hi,

Soon the last fixes for packages that were incompatible with UsrMerge will enter Factory. Time to think about the conversion plan. I've updated the wiki page¹ to describe how projects in the build service can transition to UsrMerge and how to do the conversion at run time.

Instead of triggering it manually in the initrd like Fedora did, the idea here basically is to leverage the renameat2(2) syscall in %pretrans of the file system package. So there is no intermediate, half merged state. Works fine in my test setups at least. It relies on coreutils' cp though. Can we live with that? I'm not a aware of library that could be used instead and obviously don't really want to re-implement cp just for this :-)

%pretrans can only exist as LUA script - as a fresh install will have to run all the stuff before packages are being installed

On a fresh install there is no need to run the usrmerge conversion script. So the lua part would just not call out into the script :-) cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)

Dominique Leuenberger / DimStar wrote:

On Wed, 2021-02-24 at 11:07 +0100, Thorsten Kukuk wrote:

...

On Wed, Feb 24, Ludwig Nussel wrote:

...

An open question is file provides. Atm I'm using a provides generator that automatically adds the legacy /bin provides to well known packages. Do we want to do it that way or manually add file provides to packages?

I think for the majority of file requires, the /usr/ location should be used. So adjust our RPMs requiring something in /bin. I see only problems with third party RPMs, and here the big question is, is there really anything required beside the coreutils utilities, /bin/sh and /bin/bash? I haven't seen anything else in the last time

A very common requires(post) on a lot of packages is /sbin/ldconfig, coming from

%post -p /sbin/ldconfig

So at least that one must be provided (by glibc)

For a few core packages I've manually added the provides already. I'd consider /bin/sh as ABI for example, so requiring that must work always no matter what IMO. The provides generator is on top for the rest. So if there's nobody advocating for manually adding more /bin provides, the generator is cheap to have just in case. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)

Jan Engelhardt wrote:

On Wednesday 2021-02-24 10:59, Ludwig Nussel wrote:

...

An open question is file provides. Atm I'm using a provides generator that automatically adds the legacy /bin provides to well known packages. Do we want to do it that way or manually add file provides to packages?

If you were to make it automatic, then everything in /usr/bin would generate a fileprovides with /bin, which is at least 2160 items for me.

Nah, the list of /bin files in TW at this point is known. The generator would only trigger on those. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)

On Thu, Feb 25, 2021 at 4:05 PM L A Walsh wrote:

On 2021/02/24 01:59, Ludwig Nussel wrote:

...

Hi, Soon the last fixes for packages that were incompatible with UsrMerge will enter Factory. Time to think about the conversion plan. [1] https://en.opensuse.org/openSUSE:Usr_merge

==== Why isn't this a /usr/bin => /bin merge? With /usr/bin being a bind-mount to /bin? That seems less disruptive than trying to expunge all reference to /bin. At the very least, /bin/<shell> is embedded in all shell scripts and shouldn't be relying on a symlink to function.

I'm not going to rehash all the arguments from the past decade, but there is an openSUSE specific advantage: A singular /usr tree can be snapshotted trivially with all OS contents together, which simplifies system rollbacks and synchronization of OS content via Btrfs send/recv. With /bin being a symlink to /usr/bin, everything should work the same anyway. It's transparent to scripts. It works this way in Fedora, Ubuntu, macOS, Solaris, and others too. -- 真実はいつも一つ！/ Always, there's only one truth!

On Thu, Feb 25, 2021 at 6:09 PM L A Walsh wrote:

On 2021/02/25 13:28, Neal Gompa wrote

...

I'm not going to rehash all the arguments from the past decade,

---

The questions asked in 2012 weren't answered then, either. Many of the reasons stemmed from a need and presumption for /initrd, which I would think most would know isn't necessary either.

It isn't until it is. If you're doing custom kernels with everything built into a blob, you don't need it. If you don't care about ramfs based rescue environments, you don't need it. And on and on and on... But here's the cold hard truth: you are not operating in the normal case, and doing UsrMerge does not break your special case at all, while improving the quality of experience for the normal case.

On 2021/02/25 13:28, Neal Gompa wrote:

...

there is an openSUSE specific advantage: A singular /usr tree can be snapshotted trivially with all OS contents together, which simplifies system rollbacks and synchronization of OS content via Btrfs send/recv.

---

Except for the actual OS (kernel), and libs in /lib, and /lib64(mostly same as /usr/lib64), and /etc/ and state in /var. As well as things living in /opt (a bind-mount to /home/opt on my system)

With the exception of /var, /opt, and /home, everything is already moving to /usr by default in openSUSE. Even /etc is, with the /usr/etc containing the functioning defaults and /etc containing only host-specific overrides. There's an ongoing effort in both Fedora and openSUSE to empty /etc and eliminate OS-specific state from /var, such that the OS is centralized into /usr. As for the OS kernel and bootloader code, on openSUSE it *is* mostly in /usr and copied into /boot on install/upgrade. While not perfectly there yet, we're on the path to getting to a point where the OS is completely in /usr and everything else is user/host specific and is out of the realm of the package manager. -- 真実はいつも一つ！/ Always, there's only one truth!

On Friday, 26 February 2021 16:18:52 ACDT Andrei Borzenkov wrote:

26.02.2021 02:17, Neal Gompa пишет:

...

...

----

Except for the actual OS (kernel), and libs in /lib, and

/lib64(mostly same as /usr/lib64), and /etc/ and state in /var. As well as things living in /opt (a bind-mount to /home/opt on my system)

With the exception of /var, /opt, and /home, everything is already moving to /usr by default in openSUSE. Even /etc is, with the /usr/etc containing the functioning defaults and /etc containing only host-specific overrides. There's an ongoing effort in both Fedora and openSUSE to empty /etc and eliminate OS-specific state from /var, such that the OS is centralized into /usr. As for the OS kernel and bootloader code, on openSUSE it *is* mostly in /usr and copied into /boot on install/upgrade.

/boot/vmlinuz* is part of RPM and is installed there. /lib/modules is there - are you aware of any plans to move it into /usr/lib/modules? Even so, what with /boot/vmlinuz?

...

While not perfectly there yet, we're on the path to getting to a point where the OS is completely in /usr and everything else is user/host specific and is out of the realm of the package manager.

This seems to me to be getting right away from the separation of privileges/ ownership ethos that has long been a hallmark of *nix-type systems. I thought system-owned, system- or admin-executable packages should be in /bin (with their corresponding libraries in /lib and configs in /etc), and user- executable binares in /usr/bin with corresponding /usr/lib/ and /usr/etc libs/ configs respectively. That makes perfect sense. Moving everyting into /usr/bin, /usr/lib and /usr/etc makes no sense whatsoever to me, but it seems that it's basically a tait accompli (the decision has already been made, for better or worse), so I guess we'll either have to live with it - and the breakages it will initially (and ineveitably) cause, or simply not upgrade past a certain pre-merge release. -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au CCNA #CSCO12880208 ==============================================================

On Fri, Feb 26, Rodney Baker wrote:

I thought system-owned, system- or admin-executable packages should be in /bin (with their corresponding libraries in /lib and configs in /etc), and user- executable binares in /usr/bin with corresponding /usr/lib/ and /usr/etc libs/ configs respectively. That makes perfect sense.

As Jan already wrote, following the UNIX tradition: (/usr)/bin is for user executeable binaries, where /bin is for this binaries, which are required to mount /usr (/usr)/sbin is for system or admin executeable binaries, where /sbin is for this binaries, which are required to mount /usr. Since we mount /usr in the initrd, following your argumentation with UNIX tradition, /bin and /sbin should be empty on openSUSE ;)

Moving everyting into /usr/bin, /usr/lib and /usr/etc makes no sense whatsoever to me, but it seems that it's basically a tait accompli (the decision has already been made, for better or worse), so I guess we'll either have to live with it - and the breakages it will initially (and ineveitably) cause, or simply not upgrade past a certain pre-merge release.

From a standard Desktop user view your are right, the usrMerge doesn't make a difference on the first glance. But openSUSE is more than just a Desktop distribution. We provide many more variants like Server, Edge, Embedded. And we provide many features even from interest for normal Desktop users, like snapshot and rollback. And there are also requests and ideas to provide image based updates, not only package based updates. With the current setup not possible. And this other variants and features have requirements, which are best solveable with a system, where everything is strictly separated: /usr for vendor provided code /usr/local for local admin provided code /etc for local system depending configuration files or admin made changes /usr/etc (or something similar) for vendor provided configuration files /opt for the 3rd party provided code /srv should be empty, as the FHS requirements for /srv are not fulfillable with current package managers /var is for storing statefull data of applications. So no application should install anything there, as the admin is allowed to delete it and application needs to be able to scope with this. We have still many packages doing this wrong. usrMerge is one step on the way to cleanup the distribution for new requirements, use-cases and to allow new features. It will not be the last one. The long term goal is, all openSUSE packages only install stuff in /usr, nowhere else. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)

On 2/26/21 8:38 PM, Rodney Baker wrote:

On Friday, 26 February 2021 19:57:49 ACDT Thorsten Kukuk wrote:

...

On Fri, Feb 26, Rodney Baker wrote:

...

Moving everyting into /usr/bin, /usr/lib and /usr/etc makes no sense whatsoever to me, but it seems that it's basically a tait accompli (the decision has already been made, for better or worse), so I guess we'll either have to live with it - and the breakages it will initially (and ineveitably) cause, or simply not upgrade past a certain pre-merge release.

From a standard Desktop user view your are right, the usrMerge doesn't make a difference on the first glance. But openSUSE is more than just a Desktop distribution. We provide many more variants like Server, Edge, Embedded. And we provide many features even from interest for normal Desktop users, like snapshot and rollback. And there are also requests and ideas to provide image based updates, not only package based updates. With the current setup not possible.

Fair enough, I guess. Snapshot and rollback are only useful when using btrfs, which I'll never use. They can't replace a well-maintained and properly- orchestrated backup regime, though. In my work world, for all the virtual servers and workstations I maintain (both Linux and Windows), VMWare provides snapshot and rollback capability independent of the guest OS or filesystem, without the overhead of something like btrfs for each VM.

I'll disagree here, both are good, on the very odd occasion where a tumbleweed update leaves my system mostly non working with snapper I can get myself back up and running in a matter of minutes rather then having to pull out backups etc. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 2/26/21 10:30 PM, Johannes Meixner wrote:

Hello,

On 2021-02-26 12:35, Simon Lees wrote:

...

On 2/26/21 8:38 PM, Rodney Baker wrote:

...

On Friday, 26 February 2021 19:57:49 ACDT Thorsten Kukuk wrote:

...

On Fri, Feb 26, Rodney Baker wrote:

...

Moving everyting into /usr/bin, /usr/lib and /usr/etc makes no sense whatsoever to me, but it seems that it's basically a tait accompli (the decision has already been made, for better or worse), so I guess we'll either have to live with it - and the breakages it will initially (and ineveitably) cause, or simply not upgrade past a certain pre-merge release.

From a standard Desktop user view your are right, the usrMerge doesn't make a difference on the first glance. But openSUSE is more than just a Desktop distribution. We provide many more variants like Server, Edge, Embedded. And we provide many features even from interest for normal Desktop users, like snapshot and rollback. And there are also requests and ideas to provide image based updates, not only package based updates. With the current setup not possible.

Fair enough, I guess. Snapshot and rollback are only useful when using btrfs, which I'll never use. They can't replace a well-maintained and properly- orchestrated backup regime, though. In my work world, for all the virtual servers and workstations I maintain (both Linux and Windows), VMWare provides snapshot and rollback capability independent of the guest OS or filesystem, without the overhead of something like btrfs for each VM.

I'll disagree here, both are good, on the very odd occasion where a tumbleweed update leaves my system mostly non working with snapper I can get myself back up and running in a matter of minutes rather then having to pull out backups etc.

Perhaps you mix up a standard desktop end-user's point of view with a "well maintained and properly orchestrated backup regime" in an enterprise environment where as much as possible automated remote administration without user interaction is preferred like "machine X has issues" => "recreate it completely from scratch" (of course that is likely an oversimplified extreme example)?

Like I said as my main point, BOTH approaches have there merits and are worth supporting well. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 2021/02/25 13:28, Neal Gompa wrote:

With /bin being a symlink to /usr/bin, everything should work the same anyway. It's transparent to scripts. It works this way in Fedora, Ubuntu, macOS, Solaris, and others too.

um... Assume I have shell 'mash' in the new location /usr/bin/mash:

ll /usr/bin/mash -rwxr-xr-x 1 1284320 May 18 2018 /usr/bin/mash*

Now I create my transparent symlink:

sudo ln -s /usr/bin bn ll /bn lrwxrwxrwx 1 8 Mar 1 02:50 /bn -> /usr/bin/ ll bn lrwxrwxrwx 1 8 Mar 1 02:50 bn -> /usr/bin/ ll /bn/mash -rwxr-xr-x 1 1284320 May 18 2018 /bn/mash*

ok, now I want to delete /bn/mash

safe-rm /bn/mash /usr/bin/safe-rm: no symlinks allowed in the path of /bn/mash

Explain to me how vendors checking for symlinks in file path won't be able to detect such and won't have an issue with ignoring something generally regarded as insecure. That suse-makers believe such a symlink is transparent and secure and that Suse should do it because everyone else is doing it is more than a little worrisome. Certain it lends support to the idea of not having a 'standard' system.

It isn't until it is. If you're doing custom kernels with everything built into a blob, you don't need it.

A blob? My standard vanilla kernel has # zgrep =m$ /proc/config.gz |wc -l 538 538 modules with # wc -l /proc/modules 45 45 of them currently loaded

lsmod|wc 46 155 167 46 currently loaded

If you don't care about ramfs based rescue environments, you don't need it.

My entire system is a rescue environment since things break fairly often. I.e. a ramfs can't hold everything needed and even if it did, why copy it into a ram-based block-dev? If you can read your ram-device, then you can read your sys files. Like someone else said, you have your rescue system on root. *Until recently* when I replace my hard disks, 'usr' was on a separate partition. But the system boot/bringup files were in /bin,lib,lib64, etc... one of the first tasks that init did was mount /usr: cd /etc/rc.d/boot.d Ishtar:/etc/rc.d/boot.d# ls -L S* S01boot.sysctl* S01boot.usr-mount* S02boot.udev* S04boot.lvm* S05boot.clock* S06boot.localnet* S11boot.localfs* S13boot.klog* S13boot.lvm_monitor* S13boot.swap* S13setserial* S14boot.isapnp* S14boot.ldconfig* S30boot.assign_netif_names* S50boot.cgroup* S50boot.create_devs* S50boot.sys_settings* S50boot.sysstat* If you evacuate all files to /usr/bin, of course there's a better argument for a ramdisk, but its still not needed as your rescue disk can be a separate partition on your root disk, or a rescue-image in a file on your boot disk. But deciding what limited rescue files go on a ram disk and copying it to ram first, vs. having your entire /bin+/lib dirs be your 'rescue resources' -- why go the extra step of creating a limited rescue disk vs booting native. And the most basic level of rescue: init=/bin/bash Will that work if bash is a symlink? It certainly wouldn't if my /usr was a separate partition like it used to be.

26.02.2021 00:05, L A Walsh пишет:

On 2021/02/24 01:59, Ludwig Nussel wrote:

...

Hi, Soon the last fixes for packages that were incompatible with UsrMerge will enter Factory. Time to think about the conversion plan. [1] https://en.opensuse.org/openSUSE:Usr_merge   ====    Why isn't this a /usr/bin => /bin merge?

Explain how it achieves the goals that usrmerge is striving to achieve.