[opensuse] BtrFS progress and question

older
[opensuse] KDE keyboard right click

Anton Aylward

25 Feb 2013 25 Feb '13

13:30

As mentioned previously, I'm experimenting with BtrFS supporting 12.2. It seems quite stable in the limited, single machine, single drive, desktop setting and handling daily use satisfactorily. Over the weekend I converted to KDE x.x.10 and updated all there was to update and am trying out KDE's office suite, Calligra. The disk is partitioned into swap +btrfs. The is no separate /boot. Grub2 is working reliably. I cant comment about running under RAID or across multiple machines; I'm not in a position to test that; this is a desktop and a small 20G disk since all the 'real stuff' comes over via NFS. I'd be happy to experiment with multi-spindle if I could find a brace of small (10G perhaps?) SATA drives in the Closet-of-Anxieties. The one problem I have is with snapshots. Really, the problem is with my understanding, and googling around I can't find the answer to the question I have. The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up. They are even filling up with stuff that comes from the NFS mounted stuff under /home ! I'm concerned because this is eating away at my small disk. Every time I do an zypper update this grows. If I try to delete I get a message telling me that the files are on a write-protected file system! Can anyone help my understanding here? -- "In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri." -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Show replies by date

Greg Freemyer

25 Feb 25 Feb

13:53

...

The one problem I have is with snapshots. Really, the problem is with my understanding, and googling around I can't find the answer to the question I have.

The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up. They are even filling up with stuff that comes from the NFS mounted stuff under /home !

I'm concerned because this is eating away at my small disk. Every time I do an zypper update this grows.

If I try to delete I get a message telling me that the files are on a write-protected file system!

Can anyone help my understanding here?

Anton, I'm not a btrfs expert, but windows ntfs has had conceptually similar "shadow copy" functionality since windows 2003. I am expert on it. With ntfs the shadow copies are maintained until the disk space is needed for existing files. When that happens, the oldest shadow copy is deleted automatically. There is nothing the user needs to do or even be aware of. The exception is if want to delete a file beyond forensic recovery, then you have to be very aware of the backup copies to ensure that backup copies are also destroyed. Fyi: in my day job I often have to implement court orders that call for intellectual property to be permanently destroyed, so I have to be aware of how the backup maintenance process works. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Carlos E. R.

14:21

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2013-02-25 at 08:53 -0500, Greg Freemyer wrote:

...

...
If I try to delete I get a message telling me that the files are on a write-protected file system!

There is a purge command, but I don't know which. I thought it was automatic.

...

...
Can anyone help my understanding here?

Anton,

I'm not a btrfs expert, but windows ntfs has had conceptually similar "shadow copy" functionality since windows 2003. I am expert on it.

With ntfs the shadow copies are maintained until the disk space is needed for existing files. When that happens, the oldest shadow copy is deleted automatically. There is nothing the user needs to do or even be aware of.

I'm curious. Is this different from the trash folder? - -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAlErc3cACgkQtTMYHG2NR9VX0ACfX0A3Bbzmpq1ogPP1JX693PvN 2D4AoIA6nNgY3BbNda5cyiQ4Oqat3TSx =7Kfy -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Greg Freemyer

14:58

"Carlos E. R." wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Monday, 2013-02-25 at 08:53 -0500, Greg Freemyer wrote:

...
...
If I try to delete I get a message telling me that the files are on a write-protected file system!

There is a purge command, but I don't know which. I thought it was automatic.

...
...
Can anyone help my understanding here?

Anton,

I'm not a btrfs expert, but windows ntfs has had conceptually similar

...
"shadow copy" functionality since windows 2003. I am expert on it.

With ntfs the shadow copies are maintained until the disk space is needed for existing files. When that happens, the oldest shadow copy is deleted automatically. There is nothing the user needs to do or even be aware of.

I'm curious. Is this different from the trash folder?

Ntfs shadow copies are totally unrelated to the trash folder. It is much more similar to lvm differential snapshots, but maintained within the filesystem. I don't know how btrfs does it, so I can't compare/contrast. Greg

...

- -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar)

-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux)

iEYEARECAAYFAlErc3cACgkQtTMYHG2NR9VX0ACfX0A3Bbzmpq1ogPP1JX693PvN 2D4AoIA6nNgY3BbNda5cyiQ4Oqat3TSx =7Kfy -----END PGP SIGNATURE-----

-- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrey Borzenkov

14:20

On Mon, Feb 25, 2013 at 5:30 PM, Anton Aylward wrote:

...

The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up.

I believe .snapshot directory is used by snapper. Do you have it installed and enabled?

...

They are even filling up with stuff that comes from the NFS mounted stuff under /home !

Not sure about NFS, but on btrfs snapshots are point in time view of the whole filesystem (subvolume). They do not actually take up as much space as it seems.

...

I'm concerned because this is eating away at my small disk. Every time I do an zypper update this grows.

That supports my idea that you have snapper active.

...

If I try to delete I get a message telling me that the files are on a write-protected file system!

btrfs snapshots can be both read-only and writable. Probably snapper creates RO snapshots (makes sense in this case). You probably should use snapper to manage those snapshots in this case. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

15:02

Andrey Borzenkov said the following on 02/25/2013 09:20 AM:

...

On Mon, Feb 25, 2013 at 5:30 PM, Anton Aylward wrote:

...
The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up.

I believe .snapshot directory is used by snapper. Do you have it installed and enabled?

Its installed, yes, but I don't know what you mean by 'enabled'. I see # grep -R snapper /etc/*cron* /etc/cron.daily/suse.de-snapper:# get information from /etc/sysconfig/snapper /etc/cron.daily/suse.de-snapper:if [ -f /etc/sysconfig/snapper ] ; then /etc/cron.daily/suse.de-snapper: . /etc/sysconfig/snapper /etc/cron.daily/suse.de-snapper:# run snapper for all configs /etc/cron.daily/suse.de-snapper: . /etc/snapper/configs/$CONFIG /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup number /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup timeline /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup empty-pre-post /etc/cron.hourly/suse.de-snapper:# get information from /etc/sysconfig/snapper /etc/cron.hourly/suse.de-snapper:if [ -f /etc/sysconfig/snapper ] ; then /etc/cron.hourly/suse.de-snapper: . /etc/sysconfig/snapper /etc/cron.hourly/suse.de-snapper:# run snapper for all configs /etc/cron.hourly/suse.de-snapper: . /etc/snapper/configs/$CONFIG /etc/cron.hourly/suse.de-snapper: snapper --config=$CONFIG --quiet create --description="timeline" --cleanup-algorithm="timeline" Oh My! I'm ovverwhelmed! I wans't aware of all that! The hourly create ... WOW!

...

...
They are even filling up with stuff that comes from the NFS mounted stuff under /home !

Not sure about NFS, but on btrfs snapshots are point in time view of the whole filesystem (subvolume). They do not actually take up as much space as it seems.

Two points: Yes, I understand COW, but they ARE taking up space! Yes, they are taking images of the NFS mounted. So far as I can read the config, that seems to be what the config allows. But I'm still confused/unenlightened about what some of the config CAN do. I've got some reseaching to do ...

...

...
I'm concerned because this is eating away at my small disk. Every time I do an zypper update this grows.

That supports my idea that you have snapper active.

Agreed. A side note: We have a large AIX setup. I used to administer it. (I still get lookins but its not my full time job any more.) I was glad when Linux acquired the LVM type of management I had grown used to ;-) One thing I liked was the way that the patches and updates could be rolled back so easily or 'committed'. It seems this is now part of what snapper lets us do with Linux ... if I can only figure out how....

...

...
If I try to delete I get a message telling me that the files are on a write-protected file system!

btrfs snapshots can be both read-only and writable. Probably snapper creates RO snapshots (makes sense in this case).

You probably should use snapper to manage those snapshots in this case.

So this isn't really a BtrFS issue as such ... -- Vegetables are not food; vegetables are what food eats. Fruit are vegetables that fool you by tasting good. Fish are fast-moving vegetables. Mushrooms are what grows on vegetables when food's done with them. -- Meat Eater's Credo -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrey Borzenkov

16:23

В Mon, 25 Feb 2013 10:02:33 -0500 Anton Aylward пишет:

...

Andrey Borzenkov said the following on 02/25/2013 09:20 AM:

...
On Mon, Feb 25, 2013 at 5:30 PM, Anton Aylward wrote:

...
The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up.

I believe .snapshot directory is used by snapper. Do you have it installed and enabled?

Its installed, yes, but I don't know what you mean by 'enabled'.

snapper list-configs snapper list -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Werner Flamme

26 Feb 26 Feb

14:48

Anton Aylward [25.02.2013 16:02]:

...

Andrey Borzenkov said the following on 02/25/2013 09:20 AM:

...
On Mon, Feb 25, 2013 at 5:30 PM, Anton Aylward wrote:

...
The articles I find on BtrFS snapshots all tell me how great this mechanism is and how to use it and set it up. But it seems already set up and working even without me doing anything to initiate it. There are hundreds of sub-directories under /.snapshot and they are filling up.

I believe .snapshot directory is used by snapper. Do you have it installed and enabled?

Its installed, yes, but I don't know what you mean by 'enabled'. I see

# grep -R snapper /etc/*cron* /etc/cron.daily/suse.de-snapper:# get information from /etc/sysconfig/snapper /etc/cron.daily/suse.de-snapper:if [ -f /etc/sysconfig/snapper ] ; then /etc/cron.daily/suse.de-snapper: . /etc/sysconfig/snapper /etc/cron.daily/suse.de-snapper:# run snapper for all configs /etc/cron.daily/suse.de-snapper: . /etc/snapper/configs/$CONFIG /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup number /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup timeline /etc/cron.daily/suse.de-snapper: snapper --config=$CONFIG --quiet cleanup empty-pre-post /etc/cron.hourly/suse.de-snapper:# get information from /etc/sysconfig/snapper /etc/cron.hourly/suse.de-snapper:if [ -f /etc/sysconfig/snapper ] ; then /etc/cron.hourly/suse.de-snapper: . /etc/sysconfig/snapper /etc/cron.hourly/suse.de-snapper:# run snapper for all configs /etc/cron.hourly/suse.de-snapper: . /etc/snapper/configs/$CONFIG /etc/cron.hourly/suse.de-snapper: snapper --config=$CONFIG --quiet create --description="timeline" --cleanup-algorithm="timeline"

Oh My! I'm ovverwhelmed! I wans't aware of all that! The hourly create ... WOW!

Have a look in /etc/snapper/configs/, there should be at least one file. You should be able to edit # create hourly snapshots TIMELINE_CREATE="no" inside :-)

...

...
...
I'm concerned because this is eating away at my small disk. Every time I do an zypper update this grows.

That supports my idea that you have snapper active.

Snapper should still be active when you disable the timeline, so changes by zypper still lead to a snapshot.

...

Agreed.

A side note: We have a large AIX setup. I used to administer it. (I still get lookins but its not my full time job any more.) I was glad when Linux acquired the LVM type of management I had grown used to ;-) One thing I liked was the way that the patches and updates could be rolled back so easily or 'committed'. It seems this is now part of what snapper lets us do with Linux ... if I can only figure out how....

"man snapper" and "man btrfs" can provide some help. I can see snapshots with snapper, but I can delete them with btrfs only. Kind of symbiosis, I think ;-) Just my 2¢ Werner -- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

25 Feb 25 Feb

15:25

Andrey Borzenkov said the following on 02/25/2013 09:20 AM:

...

...
...
They are even filling up with stuff that comes from the NFS mounted stuff under /home !

Not sure about NFS, but on btrfs snapshots are point in time view of the whole filesystem (subvolume). They do not actually take up as much space as it seems.

I've just confirmed this. ~/Documents (and others) is a symlink to /mnt/server/anton/Documents which is a mount-on-demand via the automount mechanism in systemd. I've discussed this before. I'm seeing a snapshot of it when I use the yast tool. -- Man does not live by words alone, despite the fact that sometimes he has to eat them. Adlai E. Stevenson Jr., quoted by Human Behavior, May 1978 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Andrey Borzenkov

16:22

В Mon, 25 Feb 2013 10:25:52 -0500 Anton Aylward пишет:

...

Andrey Borzenkov said the following on 02/25/2013 09:20 AM:

...
...
...
They are even filling up with stuff that comes from the NFS mounted stuff under /home !

Not sure about NFS, but on btrfs snapshots are point in time view of the whole filesystem (subvolume). They do not actually take up as much space as it seems.

I've just confirmed this.

~/Documents (and others) is a symlink to /mnt/server/anton/Documents which is a mount-on-demand via the automount mechanism in systemd. I've discussed this before.

I'm seeing a snapshot of it when I use the yast tool.

I cannot reproduce it. I just lightly tested btrfs which had symlink into another filesystem as well as tried to (bind)mount another filesystem onto directory in btrfs. In the first case symlink was snapped and in the second case I got empty mountpoint in snapshot. I used simple "snapper create". May be YaST2 does something different. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

4087

Age (days ago)

4088

Last active (days ago)

List overview

Download

9 comments

5 participants

participants (5)

Andrey Borzenkov
Anton Aylward
Carlos E. R.
Greg Freemyer
Werner Flamme

[opensuse] BtrFS progress and question

tags

participants (5)