What does this mean? moving rpm-db failed?
The following package is going to be upgraded: rpm 1 package to upgrade. Overall download size: 1.6 MiB. Already cached: 0 B After the operation, additional 2.6 KiB will be used. Continue? [y/n/? shows all options] (y): Checking for file conflicts: .............................................[done] Retrieving package rpm-4.14.1-lp151.13.10.x86_64 (1/1), 1.6 MiB ( 4.6 MiB unpacked) (1/1) Installing: rpm-4.14.1-lp151.13.10 .................................[done] Additional rpm output: Updating /etc/sysconfig/services ... Output of rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script: migrating rpmdb from /var/lib/rpm to /usr/lib/sysimage/rpm... mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty rmdir: failed to remove 'var/lib/rpm': Directory not empty Warning: rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script failed (returned 1) Problem occured during or after installation or removal of packages: Failed to cache rpm database (1). Please see the above error message for a hint. ---------------- Speaking of weird error messages Tried to install rpm from leap 15.1... Rats; yeah, have backups won't travel much...
On 16.03.2021 20:26, L A Walsh wrote:
The following package is going to be upgraded: rpm
1 package to upgrade. Overall download size: 1.6 MiB. Already cached: 0 B After the operation, additional 2.6 KiB will be used. Continue? [y/n/? shows all options] (y): Checking for file conflicts: .............................................[done]
Retrieving package rpm-4.14.1-lp151.13.10.x86_64 (1/1), 1.6 MiB ( 4.6 MiB unpacked) (1/1) Installing: rpm-4.14.1-lp151.13.10 .................................[done] Additional rpm output: Updating /etc/sysconfig/services ...
Output of rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script: migrating rpmdb from /var/lib/rpm to /usr/lib/sysimage/rpm... mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty rmdir: failed to remove 'var/lib/rpm': Directory not empty
And what exactly is not clear in this message? Mon Nov 20 14:50:53 UTC 2017 - mpluskal@suse.com - Move update alternatives database to new location
Warning: rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script failed (returned 1) Problem occured during or after installation or removal of packages: Failed to cache rpm database (1). Please see the above error message for a hint. ----------------
Speaking of weird error messages
Tried to install rpm from leap 15.1... Rats; yeah, have backups won't travel much...
On 2021/03/16 11:40, Andrei Borzenkov wrote:
On 16.03.2021 20:26, L A Walsh wrote:
mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty; rmdir: failed to remove 'var/lib/rpm': Directory not empty
And what exactly is not clear in this message? Mon Nov 20 14:50:53 UTC 2017 - mpluskal@suse.com - Move update alternatives database to new location
---- I was semi-expecting the rpm-db to move at some point, but how that got turned into a failed move of 'alternatives' wasn't exactly clear to me. Also, How can a 'mv' fail when it is moving to another device? If you are moving a dir to a target that already exists, then you either 1) mv the target out of the way, or 2) automatically merge them. if, perchance you wanted to be very cautious, then rename target files that have the same name. But directories w/the same name would just have their contents merged. Usually that has the desired effect. ======================== On 2021/03/16 12:53, Carlos E. R. wrote:
One of the moves (across mounts) failed because target directory was not empty.
---- Um. How would it not being across a mount make it different? Still -- it's related to *nix poorly handling "cp -a x/y/zdir a/b/zdir". You don't get a deterministic result and may just see an install script fail instead of getting it right. More often than not, the better option is to merge them -- certainly offering to do so is "more better", 🤔, (one area where Windows gets it right).
On 17.03.2021 05:41, L A Walsh wrote:
On 2021/03/16 11:40, Andrei Borzenkov wrote:
On 16.03.2021 20:26, L A Walsh wrote:
mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty; rmdir: failed to remove 'var/lib/rpm': Directory not empty
And what exactly is not clear in this message? Mon Nov 20 14:50:53 UTC 2017 - mpluskal@suse.com - Move update alternatives database to new location
I was semi-expecting the rpm-db to move at some point, but how that got turned into a failed move of 'alternatives' wasn't exactly clear to me.
At some point alternatives database was kept under /var/lib/rpm, later it was moved into /var/lib/alternatives. This directory belongs to update-alternatives package and this package was responsible for relocating it. Apparently transition from /var/lib/rpm to /usr/lib/sysimage/rpm happened after update-alternatives database was already in new location, so script that relocates RPM database does not expect directories nor attempts to move them. Although it is strange when we look at change dates, as RPM happened before update-alternatives ... Thu Oct 26 12:10:16 UTC 2017 - rbrown@suse.com - Remove usr-lib-rpmdb.patch - Add usr-lib-sysimage-rpm.patch to locate rpmdb to /usr/lib/sysimage/rpm after discussions with upstream - Migrates existing rpmdb in /var/lib/rpm to /usr/lib/sysimage/rpm My guess is that both changes happened to be released together, probably in one snapshot, and update-alternatives migrates its database in postinstall script and RPM - in posttrans script, so it passed unnoticed.
On 17.03.2021 08:26, Andrei Borzenkov wrote:
On 17.03.2021 05:41, L A Walsh wrote:
On 2021/03/16 11:40, Andrei Borzenkov wrote:
On 16.03.2021 20:26, L A Walsh wrote:
mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty; rmdir: failed to remove 'var/lib/rpm': Directory not empty
This means you have both non-empty /var/lib/rpm/alternatives/ and /usr/lib/sysimage/rpm/alternatives/. Well, that is something that does not happen normally. You are not even supposed to have both *directories* /var/lib/rpm and /usr/lib/sysimage/rpm at the same time.
And what exactly is not clear in this message? Mon Nov 20 14:50:53 UTC 2017 - mpluskal@suse.com - Move update alternatives database to new location
I was semi-expecting the rpm-db to move at some point, but how that got turned into a failed move of 'alternatives' wasn't exactly clear to me.
At some point alternatives database was kept under /var/lib/rpm, later it was moved into /var/lib/alternatives. This directory belongs to update-alternatives package and this package was responsible for relocating it.
Apparently transition from /var/lib/rpm to /usr/lib/sysimage/rpm happened after update-alternatives database was already in new location, so script that relocates RPM database does not expect directories nor attempts to move them. Although it is strange when we look at change dates, as RPM happened before update-alternatives ...
Thu Oct 26 12:10:16 UTC 2017 - rbrown@suse.com
- Remove usr-lib-rpmdb.patch - Add usr-lib-sysimage-rpm.patch to locate rpmdb to /usr/lib/sysimage/rpm after discussions with upstream - Migrates existing rpmdb in /var/lib/rpm to /usr/lib/sysimage/rpm
My guess is that both changes happened to be released together, probably in one snapshot, and update-alternatives migrates its database in postinstall script and RPM - in posttrans script, so it passed unnoticed.
I was wrong, script works correctly as long as /usr/lib/sysimage/rpm does not exist when conversion happens. It also moves /var/lib/rpm/alternatives.
On 17/03/2021 03.41, L A Walsh wrote:
On 2021/03/16 12:53, Carlos E. R. wrote:
One of the moves (across mounts) failed because target directory was not empty.
Um. How would it not being across a mount make it different?
A move in the same mount is done rewriting inodes or whatever, not moving the actual data sectors. It happens fast even with huge files. Across mounts the data has to be copied, then the source deleted. I remember that when rpms first tried moves and it happened to be across mounts, it aborted. After sopme bugzillas, these cases were cared for, and "same mount" was not assumed. But this was not the case, rather more complicated as Andrei explains. Wow, what a thing. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 2021/03/16 11:40, Andrei Borzenkov wrote:
And what exactly is not clear in this message? That my entire rpm database had been wiped and needed restoration from a user-backup to recover from suse-upgrade damage, again. Most users wouldn't have such a backup, so for most users, that would be an entire wipe.
It's one reason why most of my scripts/programs take me so long these days, as 75-80% of the script is checking for error conditions and telling whoever is running the program (often me) how to fix it, cuz I don't remember every little detail when something breaks years later. Now I don't know what I should do about this, since something will likely try to upgrade it again and will likely mess it up again. Why did rpm have to change locations and source dialects and compression encoding? (all at the same time, from my perspective).
Mon Nov 20 14:50:53 UTC 2017 - mpluskal@suse.com
- Move update alternatives database to new location
Warning: rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script failed (returned 1) Problem occured during or after installation or removal of packages: Failed to cache rpm database (1). Please see the above error message for a hint. ----------------
Speaking of weird error messages
Tried to install rpm from leap 15.1... Rats; yeah, have backups won't travel much...
On 19/03/2021 11.09, Olaf Hering wrote:
Am Fri, 19 Mar 2021 02:46:32 -0700 schrieb L A Walsh <suse@tlinx.org>:
Most users
Don't worry, it is just you AFAICS.
A user not doing continuous regular updates on TW for some reason would also have problems, but must users would not be that persistent trying to recover and just install fresh again. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 3/19/21 8:16 PM, L A Walsh wrote:
On 2021/03/16 11:40, Andrei Borzenkov wrote:
And what exactly is not clear in this message? That my entire rpm database had been wiped and needed restoration from a user-backup to recover from suse-upgrade damage, again. Most users wouldn't have such a backup, so for most users, that would be an entire wipe.
Given the change happened a significant time ago and your the first to hit an issue and its regularly tested from a range of versions its hard to see most users being effected although most users are also now on btrfs and would have been able to roll back.
It's one reason why most of my scripts/programs take me so long these days, as 75-80% of the script is checking for error conditions and telling whoever is running the program (often me) how to fix it, cuz I don't remember every little detail when something breaks years later.
Now I don't know what I should do about this, since something will likely try to upgrade it again and will likely mess it up again.
Why did rpm have to change locations and source dialects and compression encoding? (all at the same time, from my perspective).
Compression because it saves time and money, both in package build times on the build farm and storage for mirrors. "Source Dialects" because sometimes RPM adds new features that make packaging easier and us packagers are a rather lazy bunch and like to take full advantage of them. Moving the database is mostly to help the people interested in transactional updates. Under the transactional update design the "Operating System" is placed on a read only filesystem where only the update mechanism can write to that filesystem during a transactional update. So that users / sysadmin's can still make configuration changes /etc becomes an overlay fs. /var also becomes a separate subvolume so that services can write to it. From this perspective the rpm database makes far more sense being on the read only filesystem with all the packages it installs. This is why you will also notice a pattern of packages moving there default config into /usr/share or /usr/etc and then allowing users / sysadmins to overwrite the defaults by writing to files in /etc. This has a side effect of making merging conflict files much easier because only values that have been overwritten need to be merged and unless something has been removed most will just stick to what the user configured. While such things aren't strictly needed in non transactional systems they don't generally hurt them and they are also useful in many cloud setups which is why you are seeing such changes being adopted by a number of distros. Cheers -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On 2021/03/21 05:18, Simon Lees wrote:
On 3/19/21 8:16 PM, L A Walsh wrote:
On 2021/03/16 11:40, Andrei Borzenkov wrote:
And what exactly is not clear in this message?
That my entire rpm database had been wiped and needed restoration from a user-backup to recover from suse-upgrade damage, again. Most users wouldn't have such a backup, so for most users, that would be an entire wipe.
Given the change happened a significant time ago and your the first to hit an issue and its regularly tested from a range of versions its hard to see most users being effected although most users are also now on btrfs and would have been able to roll back.
Most on brtfs?
It's one reason why most of my scripts/programs take me so long these days, as 75-80% of the script is checking for error conditions and telling whoever is running the program (often me) how to fix it, cuz I don't remember every little detail when something breaks years later.
Now I don't know what I should do about this, since something will likely try to upgrade it again and will likely mess it up again.
Why did rpm have to change locations and source dialects and compression encoding? (all at the same time, from my perspective).
Compression because it saves time and money, both in package build times on the build farm and storage for mirrors.
---- Asked why compression encoding was changed, not why compresssion is used. gzip and bzip2 were both supported previously. I might have even expected xz at some point, but I've seen no measurements on speed or %compression for one that went into rpm, nor have I seen anyone using it outside of rpm.
"Source Dialects" because sometimes RPM adds new features that make packaging easier and us packagers are a rather lazy bunch and like to take full advantage of them.
Moving the database is mostly to help the people interested in transactional updates. Under the transactional update design the "Operating System" is placed on a read only filesystem where only the update mechanism can write to that filesystem during a transactional update. So that users / sysadmin's can still make configuration changes /etc becomes an overlay fs.
--- Try, but the new rpm doesn't have to disable the old syntax. OTOH, having new features doesn't need to sacrifice compatibility. The first versions of C++ were written as translators on top of other languages. I even remember one a RATFOR (Rational Fortran) of the first C++ standard. If that can be done, you can't tell me new feature sets have to be done in an incompatible way. I'm surprised the new features weren't implemented in rpm-macros. putting an overlay-fs over a frozen root doesn't mean /etc has to be moved.
/var also becomes a separate subvolume
/var already was a separate subvolume -- it was mean to be written to constantly (as in logs).
so that services can write to it. From this perspective the rpm database makes far more sense being on the read only filesystem with all the packages it installs.
Maybe. I can see some sense in that, however....
This is why you will also notice a pattern of packages moving there default config into /usr/share
/usr/share is already a separate file system in theory and practice. It contained arch-neutral data. That's one of the reasons my upgrade failed -- as the scripts didn't handle copies going to a pre-existing dir and merging -- they tried to 'mv' them.
or /usr/etc and then allowing users / sysadmins to overwrite the defaults by writing to files in /etc.
I'm far from convinced trying to move things out of /etc/, elsewhere is a good idea. I still think it is a bad idea in fact due to all the programs that reference /etc. It would be smarter to arrange some overlay on /etc that can automatically fall through to /usr/etc (or wherever) the base config may lie. I'm not sure the unionfs is there yet, but probably as much as brtfs was when you started using it.
This has a side effect of making merging conflict files much easier
Maybe once in place, but moving established systems to that point -- it would be nice if it was handled a bit more gracefully. But opensuse no longer has a fix-on-boot & continue recovery system which I'm more used to. Others may not notice the difference.
On 3/24/21 3:15 PM, L A Walsh wrote:
On 2021/03/21 05:18, Simon Lees wrote:
On 3/19/21 8:16 PM, L A Walsh wrote:
On 2021/03/16 11:40, Andrei Borzenkov wrote:
And what exactly is not clear in this message?
That my entire rpm database had been wiped and needed restoration from a user-backup to recover from suse-upgrade damage, again. Most users wouldn't have such a backup, so for most users, that would be an entire wipe.
Given the change happened a significant time ago and your the first to hit an issue and its regularly tested from a range of versions its hard to see most users being effected although most users are also now on btrfs and would have been able to roll back. Most on brtfs?
Yeah, its been the system default for ages now and most users don't change the defaults.
It's one reason why most of my scripts/programs take me so long these days, as 75-80% of the script is checking for error conditions and telling whoever is running the program (often me) how to fix it, cuz I don't remember every little detail when something breaks years later.
Now I don't know what I should do about this, since something will likely try to upgrade it again and will likely mess it up again.
Why did rpm have to change locations and source dialects and compression encoding? (all at the same time, from my perspective).
Compression because it saves time and money, both in package build times on the build farm and storage for mirrors.
Asked why compression encoding was changed, not why compresssion is used. gzip and bzip2 were both supported previously. I might have even expected xz at some point, but I've seen no measurements on speed or %compression for one that went into rpm, nor have I seen anyone using it outside of rpm.
Well it was changed for the same reason it's used :-) someone did the measurements and the newer form of compression that are only supported in the latest versions of rpm make smaller binaries and or decompress / compress data faster, at some point there was a thread on factory discussing this and showing some benchmarks so the change to the newer compression form was adopted, unfortunately significantly older forms of rpm don't support this but we made sure it worked in still supported distros.
"Source Dialects" because sometimes RPM adds new features that make packaging easier and us packagers are a rather lazy bunch and like to take full advantage of them.
Try, but the new rpm doesn't have to disable the old syntax. OTOH, having new features doesn't need to sacrifice compatibility. The first versions of C++ were written as translators on top of other languages. I even remember one a RATFOR (Rational Fortran) of the first C++ standard. If that can be done, you can't tell me new feature sets have to be done in an incompatible way. I'm surprised the new features weren't implemented in rpm-macros.
I don't believe we disabled any old syntax rather we added new syntax that older RPM versions don't understand. I'm guessing it can't be macros due to where and how it needs to be expanded for dependency generation (but I am not an expert in these specifics).
Moving the database is mostly to help the people interested in transactional updates. Under the transactional update design the "Operating System" is placed on a read only filesystem where only the update mechanism can write to that filesystem during a transactional update. So that users / sysadmin's can still make configuration changes /etc becomes an overlay fs. putting an overlay-fs over a frozen root doesn't mean /etc has to be moved.
Again i'm not an expert expert here but I know it was a design decision made because it makes life significantly easier in places (otherwise we wouldn't go to the significant effort of moving things).
/var also becomes a separate subvolume
/var already was a separate subvolume -- it was mean to be written to constantly (as in logs).
Yep and from this description you can start to see why it might not be the most appropriate place for storing the persistent state of the operating system which is basically what the rpm db is and why it might have been moved inside /usr which contains most of the persistent state of the OS that the db describes.
so that services can write to it. From this perspective the rpm database makes far more sense being on the read only filesystem with all the packages it installs.
Maybe. I can see some sense in that, however....
This is why you will also notice a pattern of packages moving there default config into /usr/share
/usr/share is already a separate file system in theory and practice. It contained arch-neutral data. That's one of the reasons my upgrade failed -- as the scripts didn't handle copies going to a pre-existing dir and merging -- they tried to 'mv' them.
At a guess because /usr/share should be reserved for system packages thta shouldn't be modified by sysadmins so I guess they have no reason to expect there to already be stuff there.
or /usr/etc and then allowing users / sysadmins to overwrite the defaults by writing to files in /etc.
I'm far from convinced trying to move things out of /etc/, elsewhere is a good idea. I still think it is a bad idea in fact due to all the programs that reference /etc.
It would be smarter to arrange some overlay on /etc that can automatically fall through to /usr/etc (or wherever) the base config may lie. I'm not sure the unionfs is there yet, but probably as much as brtfs was when you started using it.
Many of the common libraries that handle reading config files are now doing this at a library level which saves some work, but yes the people that care about this proposal are doing significant work to slowly adapt applications to handle it.
This has a side effect of making merging conflict files much easier
Maybe once in place, but moving established systems to that point -- it would be nice if it was handled a bit more gracefully.
But opensuse no longer has a fix-on-boot & continue recovery system which I'm more used to. Others may not notice the difference.
I know these days we have a rescue system based off busybox but fortunately I've never had the need to use it maybe because on all the very rare times something has broken on my system i've been lazy and rolled back to the previous snapshot and waited for a fix before upgrading again. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On 24/03/2021 14.47, Simon Lees wrote:
On 3/24/21 3:15 PM, L A Walsh wrote:
...
Given the change happened a significant time ago and your the first to hit an issue and its regularly tested from a range of versions its hard to see most users being effected although most users are also now on btrfs and would have been able to roll back.
Most on brtfs?
Yeah, its been the system default for ages now and most users don't change the defaults.
I suspect that a random survey would surprise you, about that particular default :-D And in theory, users of TW are more learned and would know how to change defaults. ...
Asked why compression encoding was changed, not why compresssion is used. gzip and bzip2 were both supported previously. I might have even expected xz at some point, but I've seen no measurements on speed or %compression for one that went into rpm, nor have I seen anyone using it outside of rpm.
Well it was changed for the same reason it's used :-) someone did the measurements and the newer form of compression that are only supported in the latest versions of rpm make smaller binaries and or decompress / compress data faster, at some point there was a thread on factory discussing this and showing some benchmarks so the change to the newer compression form was adopted, unfortunately significantly older forms of rpm don't support this but we made sure it worked in still supported distros.
Yes, I remember the discussion. I think that there was also a consideration about the CPU time it took to compress them: some algorithms compress a lot but also are very CPU costly. ...
---- Maybe once in place, but moving established systems to that point -- it would be nice if it was handled a bit more gracefully.
But opensuse no longer has a fix-on-boot & continue recovery system which I'm more used to. Others may not notice the difference.
I know these days we have a rescue system based off busybox but fortunately I've never had the need to use it maybe because on all the very rare times something has broken on my system i've been lazy and rolled back to the previous snapshot and waited for a fix before upgrading again.
Me uses an external rescue image to solve issues. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 16/03/2021 18.26, L A Walsh wrote:
The following package is going to be upgraded: rpm
1 package to upgrade. Overall download size: 1.6 MiB. Already cached: 0 B After the operation, additional 2.6 KiB will be used. Continue? [y/n/? shows all options] (y): Checking for file conflicts: .............................................[done]
Retrieving package rpm-4.14.1-lp151.13.10.x86_64 (1/1), 1.6 MiB ( 4.6 MiB unpacked) (1/1) Installing: rpm-4.14.1-lp151.13.10 .................................[done] Additional rpm output: Updating /etc/sysconfig/services ...
Output of rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script: migrating rpmdb from /var/lib/rpm to /usr/lib/sysimage/rpm... mv: inter-device move failed: 'var/lib/rpm/alternatives' to 'usr/lib/sysimage/rpm/alternatives'; unable to remove target: Directory not empty rmdir: failed to remove 'var/lib/rpm': Directory not empty
Warning: rpm-4.14.1-lp151.13.10.x86_64.rpm %posttrans script failed (returned 1) Problem occured during or after installation or removal of packages: Failed to cache rpm database (1). Please see the above error message for a hint. ----------------
Speaking of weird error messages
Tried to install rpm from leap 15.1... Rats; yeah, have backups won't travel much...
Yes, at some point in time the location of the rpm database changed, so one of updates had to move the existing files. One of the moves (across mounts) failed because target directory was not empty. cer@Telcontar:~> l /var/lib/rpm lrwxrwxrwx 1 root root 26 Jun 8 2020 /var/lib/rpm -> ../../usr/lib/sysimage/rpm/ cer@Telcontar:~> -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
participants (5)
-
Andrei Borzenkov -
Carlos E. R. -
L A Walsh -
Olaf Hering -
Simon Lees