[opensuse] preventing 'no space lef ton device' error
Hello: This applies to openSUSE 13.2 system with its default btrfs file system. So far the system has crashed two times because of snapshots occupied all free space. How can I make sure that it will not happen again? I regularly check space availability using df -h command. Is the output of this reliable? It now gives: Filesystem Size Used Avail Use% Mounted on /dev/sda5 21G 17G 4.9G 77% / devtmpfs 1.8G 0 1.8G 0% /dev tmpfs 1.9G 184K 1.9G 1% /dev/shm tmpfs 1.9G 2.1M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/sda5 21G 17G 4.9G 77% /var/tmp /dev/sda5 21G 17G 4.9G 77% /var/spool /dev/sda5 21G 17G 4.9G 77% /.snapshots /dev/sda5 21G 17G 4.9G 77% /var/lib/mailman /dev/sda5 21G 17G 4.9G 77% /var/opt /dev/sda5 21G 17G 4.9G 77% /var/log /dev/sda5 21G 17G 4.9G 77% /var/lib/named /dev/sda5 21G 17G 4.9G 77% /var/lib/pgsql /dev/sda5 21G 17G 4.9G 77% /tmp /dev/sda5 21G 17G 4.9G 77% /var/crash /dev/sda5 21G 17G 4.9G 77% /usr/local /dev/sda5 21G 17G 4.9G 77% /srv /dev/sda5 21G 17G 4.9G 77% /opt /dev/sda5 21G 17G 4.9G 77% /boot/grub2/i386-pc According to this I have 4.9 G free space on the root fs. Is that correct? Thanks, Istvan -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/07/2015 17:09, Istvan Gabor a écrit :
Hello:
This applies to openSUSE 13.2 system with its default btrfs file system. So far the system has crashed two times because of snapshots occupied all free space.
same here. I completely disabled snapshot (and do not anymore use btrfs, but can't remove it when used initially)
How can I make sure that it will not happen again?
I regularly check space availability using df -h command. Is the output of this reliable? It now gives:
no, its not you have to use btrfs fi df / and/or btrfs fi show btrfs fi info / then btrfs balance... and so on
According to this I have 4.9 G free space on the root fs. Is that correct?
I got the impression than below 100Gb for root, btrfs is unusable jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am 15.07.2015 um 17:25 schrieb jdd:
btrfs fi df / and/or
btrfs fi show btrfs fi info /
There is no "info" command :-) Did you mean "usage"? Is there a good document which explains how to determine the "health" of btrfs?
I got the impression than below 100Gb for root, btrfs is unusable
I'm running btrfs with a 40GB root. All the moving parts (.snapshots, /var, /tmp, /opt) are on a second partition. Isn't that the default setup of the SUSE installer? Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://blog.pdark.de/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/07/2015 17:48, Aaron Digulla a écrit :
Am 15.07.2015 um 17:25 schrieb jdd:
btrfs fi df / and/or
btrfs fi show btrfs fi info /
There is no "info" command :-) Did you mean "usage"?
may be, I wrote this by memory
I'm running btrfs with a 40GB root. All the moving parts (.snapshots,
how can you move snapshots to an other partition? I had very large updates that resulted in enormous snapshot. My original / was 40Gb and I filled it. I'm pretty sure packman was mostly responsible. Fore some ffmpeg dependency proble, packman is rebuilt extremely frequently I have now (with all snapshots at zero in config) btrfs fi show Label: none uuid: 05617c4f-cc09-43a4-9fc7-1de36cdf8c30 Total devices 1 FS bytes used 13.00GiB devid 1 size 40.00GiB used 32.77GiB path /dev/sdb1 btrfs fi df / Data, single: total=31.01GiB, used=12.53GiB System, single: total=4.00MiB, used=16.00KiB Metadata, single: total=1.76GiB, used=486.17MiB GlobalReserve, single: total=176.00MiB, used=0.00B and df -h /dev/sdb1 41G 14G 26G 34% / -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Am 15.07.2015 um 17:57 schrieb jdd:
I'm running btrfs with a 40GB root. All the moving parts (.snapshots,
how can you move snapshots to an other partition?
Being unable to read the output from mount correctly helps ... you're right, that 40GiB is swap.
I had very large updates that resulted in enormous snapshot. My original / was 40Gb and I filled it. I'm pretty sure packman was mostly responsible. Fore some ffmpeg dependency proble, packman is rebuilt extremely frequently
I have now (with all snapshots at zero in config)
btrfs fi show Label: none uuid: 05617c4f-cc09-43a4-9fc7-1de36cdf8c30 Total devices 1 FS bytes used 13.00GiB devid 1 size 40.00GiB used 32.77GiB path /dev/sdb1
btrfs fi df / Data, single: total=31.01GiB, used=12.53GiB System, single: total=4.00MiB, used=16.00KiB Metadata, single: total=1.76GiB, used=486.17MiB GlobalReserve, single: total=176.00MiB, used=0.00B
and
df -h /dev/sdb1 41G 14G 26G 34% /
So "btrfs fi df / " reports roughly 13GiB (data+system+metadata used and df -h reports 14G. Is that GB or GiB? Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://blog.pdark.de/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-07-15 17:57, jdd wrote:
Le 15/07/2015 17:48, Aaron Digulla a écrit :
I'm running btrfs with a 40GB root. All the moving parts (.snapshots,
how can you move snapshots to an other partition?
You can not. Same as you can not use hard links across partitions... - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlWm9joACgkQja8UbcUWM1zKZAEAgXdboJzt41KMyHHRos3cNGzV H1pnp2OT1nlbByNjT2QA/3ACggJ6iv49op9gfwDeI9Tj8ziA1oNI3LsURWtazg+D =cdfJ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/15/2015 11:48 AM, Aaron Digulla wrote:
I'm running btrfs with a 40GB root. All the moving parts (.snapshots, /var, /tmp, /opt) are on a second partition.
??? .snapshots on a seperate partition? ???
Isn't that the default setup of the SUSE installer?
Well it can offer that. You can also get it to offer a LVM set up if you tickle it nicely :-) -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd írta:
Le 15/07/2015 17:09, Istvan Gabor a écrit :
Hello:
This applies to openSUSE 13.2 system with its default btrfs file system. So far the system has crashed two times because of snapshots occupied all free space.
same here. I completely disabled snapshot (and do not anymore use btrfs, but can't remove it when used initially)
How can I make sure that it will not happen again?
I regularly check space availability using df -h command. Is the output of this reliable? It now gives:
no, its not
you have to use
btrfs fi df / and/or
btrfs fi show btrfs fi info /
Thanks, I will look at btrfs command.
then btrfs balance... and so on
According to this I have 4.9 G free space on the root fs. Is that correct?
I got the impression than below 100Gb for root, btrfs is unusable
When I installed the system I chose the default fs. Thanks, Istvan -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/15/2015 11:09 AM, Istvan Gabor wrote:
This applies to openSUSE 13.2 system with its default btrfs file system. So far the system has crashed two times because of snapshots occupied all free space.
How can I make sure that it will not happen again?
I can think of a number of things. 1. Configure your snapshots properly. That has been discussed many times on the forum. Edit /etc/snapper/configs/root down to the minimal requirements. Unfortunately I know of no simple way to exclude subvolumes. 2. This is pretty important. Exclude things from the root FS There are good reasons to have the rootFS (and hence quite possibly the initrd) as small as practical. There are EXCELLENT reasons to have /boot on a separate partition, not least of all when you have to performs a recovery the times the rootFS goes haywire. There are (still) pretty good security reasons to have /tmp (and /var/tmp) on a separate partition. Among those good reason are things like ... you really don't want snapshots of what's in /boot and the many, many scratch files /tmp .... I'd also consider putting /var on a separate file system. Again you don't need snapshots of all the caches that live there. Yes, you will need to be more sure of the backups of things like /var/lib. That's BACKUPS not snapshots. The "Why?" Well at the very least you will reduce the amount of stuff that gets in /.snapshots! This isn't just about purging it, its about the 'surges'. You don't NEED to include things like your caches and scratchfiles in your snapshots in the first place. I could question why home users need to have the rootFS snapshotted at all. I could make a good case that the way many people work a single user system should be snapshotting /home/<usr>/Documents I could make a good case that for many home users BtrFS is more trouble than its worth. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-07-15 20:07, Anton Aylward wrote:
On 07/15/2015 11:09 AM, Istvan Gabor wrote:
1. Configure your snapshots properly. ... There are good reasons to have the rootFS (and hence quite possibly the initrd) as small as practical. There are EXCELLENT reasons to have /boot on a separate partition, not least of all when you have to performs a recovery the times the rootFS goes haywire.
You need /boot and / in the same partition to use on of the most useful features of btrfs and snapshots: be able to boot the system as it was before an update that went fubar. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlWm900ACgkQja8UbcUWM1zePwD/S+NYybCTBnTMknP+BxZ4ouwL 6EbRvFAfaHD3hX86nPEBAJFGVfHKxjEKWVhAp6rQV9q2VmAh31LirRsCVtLheO6J =kkmx -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/15/2015 08:14 PM, Carlos E. R. wrote:
On 2015-07-15 20:07, Anton Aylward wrote:
On 07/15/2015 11:09 AM, Istvan Gabor wrote:
1. Configure your snapshots properly. ... There are good reasons to have the rootFS (and hence quite possibly the initrd) as small as practical. There are EXCELLENT reasons to have /boot on a separate partition, not least of all when you have to performs a recovery the times the rootFS goes haywire.
You need /boot and / in the same partition to use on of the most useful features of btrfs and snapshots: be able to boot the system as it was before an update that went fubar.
I see no purpose in keeping /boot as part of the RootFS under a BtrFS file system. I do see a great deal of utility in having it as a small (comparatively speaking on a 1T drive for example) as an ext[234] partition. It makes life a LOT simpler when thing go pear shaped and the BtrFS goes into some unrecoverable state. I *do* understand about snapshots and their use as 'backup'. But the only cse I cn see where having a /boot where you can't get to a previous version is if you configure /etc/zypp/zypp.conf not to store multiversion, that is to store one and only oncer version of the bootable kernel. I have multiversion.kernels = oldest,latest,latest-1,latest-2,running for that entry, which is, perhaps, overkill. However none of this has to do with the very basic matter of "Configure your snapshots properly". As I said. its a pity that snapper can't have exclusion lists like rsync. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Jul 17, 2015 at 4:37 AM, Anton Aylward
You need /boot and / in the same partition to use on of the most useful features of btrfs and snapshots: be able to boot the system as it was before an update that went fubar.
I see no purpose in keeping /boot as part of the RootFS under a BtrFS file system.
Snapshotting becomes immensely more complicated if they are separated because now the kernel is separated from /lib/modules. Not only is it more complicated, but you're limited how far back you can rollback. So to make this work you'd have to tighten up (substantially) the total number of rollbacks to have parity with the available kernels and hence boot menu entries. What can be done is to separate /boot using a subvolume if you want, for some reason but I don't see a major benefit of this.
I do see a great deal of utility in having it as a small (comparatively speaking on a 1T drive for example) as an ext[234] partition. It makes life a LOT simpler when thing go pear shaped and the BtrFS goes into some unrecoverable state.
This is an argument in favor of: - a bootable recovery partition+volume, which is what OS X and Windows have done for a very long time now, which is a totally stateless environment - a read only root fs that is only offline updated to significantly reduce the chances of fs corruption or confusion, and this could still be snapshot.
I *do* understand about snapshots and their use as 'backup'.
But the only cse I cn see where having a /boot where you can't get to a previous version is if you configure /etc/zypp/zypp.conf not to store multiversion, that is to store one and only oncer version of the bootable kernel.
Well if you're advocating a smaller volume for /boot it's very quickly going to run out of room, especially if it's not Btrfs because now there's not even the possibility of block dedup. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/17/2015 11:43 AM, Chris Murphy wrote:
Well if you're advocating a smaller volume for /boot it's very quickly going to run out of room, especially if it's not Btrfs because now there's not even the possibility of block dedup.
I do not follow your reasoning there. My /boot partition is small compared to my One Terabyte drive, yes, but it's still a fairly large partition as the files in it go.
df -h /boot Filesystem Size Used Avail Use% Mounted on /dev/sda1 1.9G 335M 1.5G 19% /boot
ls /boot/initrd-[34]* | wc -l 8
That's both default and desktop. No, Even if I had multiversion.kernels = oldest,latest,latest-1,latest-2,latest-3,\ latest-4,-latest-5running I'd still have room My comments about a separate /boot making recovery easier is based on experience with having /boot as a part of a BtrFS RootFS. Ever since I've adopted the separate /boot I've been able to recover from everything except a messed up MBR. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-07-17 12:37, Anton Aylward wrote:
On 07/15/2015 08:14 PM, Carlos E. R. wrote:
On 2015-07-15 20:07, Anton Aylward wrote:
You need /boot and / in the same partition to use on of the most useful features of btrfs and snapshots: be able to boot the system as it was before an update that went fubar.
I see no purpose in keeping /boot as part of the RootFS under a BtrFS file system.
...
I *do* understand about snapshots and their use as 'backup'.
You are missing one of the main features of snapshots. Suppose you apply an update to the kernel and many things, and your machine fails to boot. Well, on boot you simply choose to boot a previous snapshot... simple. You can, for instance, upgrade from 13.2 to Tumbleweed, not like it, and revert completely in an instant, without losing home files nor logs and some other things I don't know. For this to work it is required to have /boot in the same btrfs partition as /lib/modules and the rest of "/". You may not want this feature (I don't), but it is the reason for doing it the way openSUSE default install is nowdays. And I know of absolute novices that have used the feature, when the experts knew nothing about it. We were trying to guide him on how to recover his system, and he just clicked on an automatic solution without our help. So yes, it works. Can it fail badly? Yes, it can. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlWpNBcACgkQja8UbcUWM1zNkwEAlOWm06PYPQYOTeW/coH/dvOi tkMfwl70c7RGlUDobegA/3rp1btMdHl/JeNRbGt/H75gqFZIiDXByo+BYkSnVVVA =XdKb -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/17/2015 12:57 PM, Carlos E. R. wrote:
Can it fail badly? Yes, it can.
My experience with BtrFS is not coloured by some prejudice against "The New" as I see in some people. I like progress. I like learning the new. But I've learnt this about BtrFS. When it works, within the boundaries it works, its great. The use cases you give for recovery from, for example, experimenting with Evergreen, good ones. I've lived with Big Iron that had similar capabilities to apply patches and then possibly back out of them. I'm aware of the upside. I welcome changes that make Linux more "professional", especially for the professional settings where it is replacing "Big Iron" as the x86 chips become more powerful. But lets face it; BtrFS is still in development. Things can still go wrong. Things still do go wrong. If you subscribe to appropriate mailing lists you'll see a couple of patches every day, sometimes as many as half a dozen. Many of them are to deal with actual 'problems'. I say 'problems' in quotes since they may not manifest as bugs yet, just as tests that fail. Heck there are still typos going on, use of "btrfs_error()" instead of "btrfs_err()". Humans are fallible, don't you know :-) I've had BtrFS go wrong. I'm used to the occasional glitch where I can't boot. That's what the rescue system is for. The one time I did have /boot as part of the RootFS the .snapshots did not help. The problem was with the BtrFS itself. I could mount it but whenever I tried to mkinitrd it flipped into RO. I ended up doing a reinstall. There might have been another avenue but the reinstall was the simplest. That reinstall was when I created the separate /boot. Since then I've had a few repeats of things going wrong, particularly when a new kernel comes along. Something seems to munge the MBR or GRUB. See my earlier postings on this matter. Even when the RootFS is going to go RO, with a separate /boot I can still rebuild a initrd and get a running system to do what amounts to "stage 2" maintenance. This is my experience; this is my problem that I've had to solve. Yes it is my environment, LVM with lots of LVs to handle things that many people put in the RootFS that I choose not to, /usr/share, /srv, /tmp. /var and similar. Why? Well, /tmp could be a tmpFS; its is on many systems and distributions, cleared on every reboot. There are, still, security reasons why it should be on a sperate FS and mounted nosuid. Well, /srv is sort of like /home for web applications. Well, /usr/share doesn't have a lot of relevance until the system is up and running. Well, /var is sort of like /tmp in places, it has /var/tmp. It also has /var/cache, which isn't relevant until the appropriate applications are running. So none of those are actually needed for boot. Leave them out of the RootFS. Heck, level them out of the initrd! What I'm NOT saying is that you shouldn't have snapshots of them. Quite the opposite. For many users I would think having /home as a BtrFS with change-based snapshoting every few minutes would be of more value than having it on the RootFS. I've no argument against using BtrFS for those other file systems other than I think snapshots of caches and scratch files (e.g compiler intermediate files that are promptly discarded after the next phase of the compiler runs) is a waste of time and resources. That should be better addressed by having an 'exclude' mechanism in the snapshot configuration. https://forums.opensuse.org/showthread.php/476706-Snapper-possible-to-exclud... The man page mentions filters but gives no examples. The examples I find on my system have no explanation and do not cover caches or scratch files. Let me also quote <quote src="https://www.suse.com/documentation/sles11/book_sle_admin/data/sec_snapper_limits.html"> Currently SUSE Linux Enterprise Server cannot boot from Btrfs partitions. Therefore a separate partition for /boot is created upon the installation when using Btrfs for the system partition. Since /boot does not support snapshots, the following restrictions apply for YaST/zypper rollbacks: no rollback for any configuration changes on the boot loader ------------------------------------------------------------ The only file that can be rolled back is the boot loader configuration file in /etc. The main configuration files reside under /boot and cannot be rolled back. no complete rollback for Kernel installations --------------------------------------------- The Kernel itself and its initrd are installed in the /boot partition, whereas Kernel modules or sources are installed in /var/lib and /usr/src, respectively. Furthermore, each Kernel installation also changes the boot loader configuration files in /boot. So whenever you do a rollback that involves undoing a Kernel installation, you need to manually remove the Kernel and its initrd from /boot and adjust the boot loader configuration by removing the boot entry for the Kernel. </quote> -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-07-17 20:24, Anton Aylward wrote:
On 07/17/2015 12:57 PM, Carlos E. R. wrote:
Can it fail badly? Yes, it can.
But lets face it; BtrFS is still in development. Things can still go wrong. Things still do go wrong. If you subscribe to appropriate mailing lists you'll see a couple of patches every day, sometimes as many as half a dozen. Many of them are to deal with actual 'problems'.
I don't trust btrfs. I am subscribed to the XFS mail list, and I see patches pretty often - yet XFS is old and mature, so the number of patches in btrfs is not yet an argument. :-) What I say is that in openSUSE 13.2, where the default is btrfs, and where you accept that default, you should not break it by using a separate /boot. That's all. Me, I simply do not use btrfs at all.
Let me also quote
That problem was solved, I understand. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlWppdYACgkQja8UbcUWM1yIpwD/eAe3Liec4Tj+RmAhyp4Yb+DQ Zz/flz1xbIX5yOKOvQoA/RkuAI8HSLAgKwQiZ/GLLeihfEg7+rFNuTmce/Q7h4b6 =LjEG -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (6)
-
Aaron Digulla -
Anton Aylward -
Carlos E. R. -
Chris Murphy -
Istvan Gabor -
jdd