[opensuse-factory] Keeping TW installation lean (Tumbleweed snapshot 20190201 released!)
Morning all, as we noticed on the weekend, the above snapshot offered a complete rebuild. While running a zypper dup -dl, at a certain point the system suddenly could not load any packages, and the KDE desktop kinda crashed. No warning, no nothing beforehand. After switching to a terminal, df .h showed that the system partition (43G, as shown by parted) was full! OK, some local OBS build environments, but nevertheless. So, removed some btrfs snapshots with snapper, cleaned /tmp and /var/tmp, reboot and all was fine again. First point for my wishlist: Let zypper check the available space before downloading. Download size plus some 10-20% security maybe? The question remains, why is a simple Laptop installation with KDE taking some 27GB? I had a look into various directories. First of all, the journal is eating up some 2.8G journalctl --vacuum-time=30d deletes old log files, which freed up 2.4G. Makes probably sense to add a systemd-timer for this, as it is not touched by sytemd-tmpfiles-clean. /var/cache/zypp/packages has another 1.8G - may be cleaned as well In /usr is some 3GB in total, mostly libs, partly in different versions, e.g. libLLVM.so.5/6/7. Is there a way to determine old/unused libs and remove them? Another 680 MB is in locale, with many locales installed I will never use. I guess most of them could be safely removed? Any other thoughts? Cheers Axel PS: After some clean-up: T520:/home/docb # btrfs filesystem df / Data, single: total=34.05GiB, used=23.26GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=1.00GiB, used=654.92MiB GlobalReserve, single: total=74.53MiB, used=0.00B -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2/4/19 10:24 AM, Axel Braun wrote:
First point for my wishlist: Let zypper check the available space before downloading. Download size plus some 10-20% security maybe?
That sounds... undesirable. You can't update because you don't have enough room? We have an update that fixes that, but I'm afraid you can't have it because you don't have enough space... It reminds me of: https://dilbert.com/strip/1995-12-29 How about, instead, Zypper checks for free space -- that does sound like a good idea to me -- and if there isn't enough, it offers to make some by running some kind of disk clean-up routine. Some kind of automatic housekeeping tool which clears old stuff from the cache, empties all files older than the last reboot from /tmp, offers to schedule a reboot and fsck, clears old snapshots and kernels, things like that. This might be easier to automate than just failing with an error, no? -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Feb 05 2019, Liam Proven <lproven@suse.cz> wrote:
This might be easier to automate than just failing with an error, no?
Automated file deletion, what can go wrong? Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On mardi, 5 février 2019 18.04:16 h CET Andreas Schwab wrote:
On Feb 05 2019, Liam Proven <lproven@suse.cz> wrote:
This might be easier to automate than just failing with an error, no?
Automated file deletion, what can go wrong?
Andreas.
unbinded variable :-) btw don't try to clean up my /tmp is tmpfs :-) may be using download-in-advance option can be tricky on those sparse free space system, but at least is would have failed before trying to run any update. -- Bruno Friedmann Ioda-Net Sàrl www.ioda-net.ch Bareos Partner, openSUSE Member, fsfe supporter GPG KEY : D5C9B751C4653227 irc: tigerfoot -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hi,
How about, instead, Zypper checks for free space -- that does sound like a good idea to me -- and if there isn't enough
At least, some clever way to warn users for this, even if no automated solution is offered would be extremely nice!
may be using download-in-advance option can be tricky on those sparse free space system, but at least is would have failed before trying to run any update.
Running Tumbleweed on a small btrfs partition (roughly 20Gb), this is not my experience. Usually, the system manages to download all packages, but it's when the installation begins that it usually fails, generally for a big package like Firefox or the kernel. When I'm unsure about free space, I trigger --download-as-needed. For what it's worth (might be useful for others), I had to compress my root filesystem and set up compression for new files to manage this update on my small partition. It seems to work well enough (didn't notice strong sluggishness after multiple reboots) and allowed the update to pass with 20% free space after that, so a good solution from that perspective. (If this is a bad advice, please --gently-- yell!). Cheers, Pierre. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On mardi, 5 février 2019 18.45:55 h CET Pierre de Villemereuil wrote:
Hi,
How about, instead, Zypper checks for free space -- that does sound like a good idea to me -- and if there isn't enough
At least, some clever way to warn users for this, even if no automated solution is offered would be extremely nice!
may be using download-in-advance option can be tricky on those sparse free space system, but at least is would have failed before trying to run any update.
Running Tumbleweed on a small btrfs partition (roughly 20Gb), this is not my experience. Usually, the system manages to download all packages, but it's when the installation begins that it usually fails, generally for a big package like Firefox or the kernel. When I'm unsure about free space, I trigger --download-as-needed.
For what it's worth (might be useful for others), I had to compress my root filesystem and set up compression for new files to manage this update on my small partition. It seems to work well enough (didn't notice strong sluggishness after multiple reboots) and allowed the update to pass with 20% free space after that, so a good solution from that perspective. (If this is a bad advice, please --gently-- yell!).
Cheers, Pierre.
Hi Pierre, for space, as I already experienced bad behaviour of all kind with btrfs filled at 85%, I've decided to keep space usage as low as possible around 50 to 75% max. It would be interesting for you to check how much snapshots you have before the upgrade. In those case of exceptionnal almost full distribution update, I would delete all old snapshots (afterward the running system is working) and only keep one full in case of. Once snapshots are removed, I also rerun btrfs- balance to force reclaim of free space. With those practice, I've been on the safe side since the last 2 years. -- Bruno Friedmann Ioda-Net Sàrl www.ioda-net.ch Bareos Partner, openSUSE Member, fsfe supporter GPG KEY : D5C9B751C4653227 irc: tigerfoot -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hi,
It would be interesting for you to check how much snapshots you have before the upgrade. In those case of exceptionnal almost full distribution update, I would delete all old snapshots (afterward the running system is working) and only keep one full in case of. Once snapshots are removed, I also rerun btrfs- balance to force reclaim of free space.
Did all that. I started the update with only one snapshot that I set up to be my default subvolume for btrfs (so it shouldn't take up more space) and ran the balance process of btrfsmaintenance beforehand. Still, the update was too large to pass without an issue... But compressing root solved it, so I guess there's that. Thanks for your input anyway! Cheers, Pierre. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Pierre de Villemereuil <flyos@mailoo.org> writes:
Hi,
It would be interesting for you to check how much snapshots you have before the upgrade. In those case of exceptionnal almost full distribution update, I would delete all old snapshots (afterward the running system is working) and only keep one full in case of. Once snapshots are removed, I also rerun btrfs- balance to force reclaim of free space.
Did all that. I started the update with only one snapshot that I set up to be my default subvolume for btrfs (so it shouldn't take up more space) and ran the balance process of btrfsmaintenance beforehand.
Still, the update was too large to pass without an issue...
But compressing root solved it, so I guess there's that.
Not sure if this was covered here, but in case you compress root: make absolutely sure that you do *not* compress /boot/ with zstd! GRUB cannot (yet) decompress zstd partitions and you'll end up with a system that cannot boot. It might work if /boot/ is on a subvolume, but I have not tested that, so take that with a grain of salt.
Thanks for your input anyway!
Cheers, Pierre.
-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Not sure if this was covered here, but in case you compress root: make absolutely sure that you do *not* compress /boot/ with zstd! GRUB cannot (yet) decompress zstd partitions and you'll end up with a system that cannot boot. It might work if /boot/ is on a subvolume, but I have not tested that, so take that with a grain of salt.
Makes sense, thanks for mentionning it. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 2/5/19 6:04 PM, Andreas Schwab wrote:
Automated file deletion, what can go wrong?
That is why I linked to the comic. One of the biggest things people seem to like about Win10 is that its housekeeping is better. It lasts longer and keeps working because it is better at cleaning up its own temp files than older versions. Because of btrfs and snapper, openSUSE already takes a lot more disk space than comparable distros. Good housekeeping is more or less _essential_, I'd say. -- Liam Proven - Technical Writer, SUSE Linux s.r.o. Corso II, Křižíkova 148/34, 186-00 Praha 8 - Karlín, Czechia Email: lproven@suse.com - Office telephone: +420 284 241 084 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tuesday 2019-02-05 17:53, Liam Proven wrote:
On 2/4/19 10:24 AM, Axel Braun wrote:
First point for my wishlist: Let zypper check the available space before downloading. Download size plus some 10-20% security maybe?
That sounds... undesirable. You can't update because you don't have enough room? We have an update that fixes that, but I'm afraid you can't have it because you don't have enough space...
How about, instead, Zypper checks for free space -- that does sound like a good idea to me -- and if there isn't enough, it offers to make some by running some kind of disk clean-up routine.
Why but rpm already does that (the freespace check). -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am 5. Februar 2019 21:47:34 MEZ schrieb Jan Engelhardt <jengelh@inai.de>:
On Tuesday 2019-02-05 17:53, Liam Proven wrote:
On 2/4/19 10:24 AM, Axel Braun wrote:
First point for my wishlist: Let zypper check the available space before downloading. Download size plus some 10-20% security maybe?
That sounds... undesirable. You can't update because you don't have enough room? We have an update that fixes that, but I'm afraid you can't have it because you don't have enough space...
How about, instead, Zypper checks for free space -- that does sound like a good idea to me -- and if there isn't enough, it offers to make some by running some kind of disk clean-up routine.
Why but rpm already does that (the freespace check).
Before installation, but not before download. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Why but rpm already does that (the freespace check).
Before installation, but not before download.
A download size of 2 GB does not mean only 2 GB is needed. It also needs space for the uncompressed bits. And, if you are changing something that is running (the old file is in use) disk space can be quite large until the old items are no longer needed (typically a reboot - starting bits here and there is usually a recipe for a different type of problem). I always download all first, and then let the install happen. Nothing worse that having 1/2 the system updated when a needed bit fails to download. My experience is that one should have at least 2x (I usually want 3x) the download file size free before doing an ininstall. Working in the real world here. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am Mittwoch, 6. Februar 2019, 08:09:56 CET schrieb Roger Oberholtzer:
Why but rpm already does that (the freespace check).
Before installation, but not before download.
A download size of 2 GB does not mean only 2 GB is needed. It also needs space for the uncompressed bits. And, if you are changing something that is running (the old file is in use) disk space can be quite large until the old items are no longer needed (typically a reboot - starting bits here and there is usually a recipe for a different type of problem).
Correct - so the upcharge (add-on to the pure download size) must even be larger.
I always download all first, and then let the install happen. Nothing worse that having 1/2 the system updated when a needed bit fails to download.
See my original post, that's what I did as well. There was not even enough space to download all 2900 packages...but better it fails here, than during installation.... Cheers Axel -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (8)
-
Andreas Schwab -
Axel Braun -
Bruno Friedmann -
Dan Čermák -
Jan Engelhardt -
Liam Proven -
Pierre de Villemereuil -
Roger Oberholtzer