On Sunday 2014-11-30 04:45, Cristian Rodríguez wrote:

El 29/11/14 a las 23:19, Roman Bysh escribió:

...

gcc: internal compiler error: Bus error (program as)

Hrmmm.. are your memory modules OK ? looks like they are not.

SIGBUS is indicative of unaligned access and generally appears only outside of {x86,ppc}. Statistically, unaligned access is foremost due to programmers willingly breaking C programming rules. Bad memory is also possible, but much more likely to appear as a SIGSEGV instead, due to bad values being used for (aligned) displacements that lead to an unmapped area, or cause essential data to be overwritten at some location which causes another part of the program to go to an unmapped place. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 11/30/2014 08:08 AM, šumski wrote:

On Sunday 30 of November 2014 03:19:55 Roman Bysh wrote:

...

Hello everybody,

I just ran:

cd /usr/src/linux

then

make oldconfig && make prepare HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as)

Using btrfs? If yes, check http://bugzilla.suse.com/show_bug.cgi?id=905832

Cheers, Hrvoje

Yes. If I type "make" in /usr/src/linux - I get an error: make /bin/sh: line 1: 5471 Bus error ( ar rcD "$TMP" ) > /dev/null 2>&1 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[2]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make[1]: *** [scripts_basic] Error 2 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[1]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make: *** [scripts_basic] Error 2 I'm unable to work with the Linux kernel source. Very serious. Roman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 11/30/2014 03:00 PM, Larry Finger wrote:

On 11/30/2014 01:37 PM, Roman Bysh wrote:

...

On 11/30/2014 08:08 AM, šumski wrote:

...

On Sunday 30 of November 2014 03:19:55 Roman Bysh wrote:

...

Hello everybody,

I just ran:

cd /usr/src/linux

then

make oldconfig && make prepare HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as)

Using btrfs? If yes, check http://bugzilla.suse.com/show_bug.cgi?id=905832

Cheers, Hrvoje

Yes. If I type "make" in /usr/src/linux - I get an error:

make /bin/sh: line 1: 5471 Bus error ( ar rcD "$TMP" ) > /dev/null 2>&1 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[2]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make[1]: *** [scripts_basic] Error 2 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[1]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make: *** [scripts_basic] Error 2

I'm unable to work with the Linux kernel source.

Very serious.

I use ext4 file systems, and I do not see these errors. Kernel builds work perfectly. Perhaps btrfs is the problem.

Larry

I ran dmesg and copied this portion: 46.119244] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.139713] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.266483] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.306124] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.327488] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.393102] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.422136] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.453146] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.535700] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 46.574283] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.140111] btrfs_readpage_end_io_hook: 56 callbacks suppressed [ 51.140120] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.157894] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.218110] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.264555] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.367132] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.392856] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.405927] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.429365] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.451151] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 [ 51.472355] BTRFS info (device sda1): csum failed ino 3743 off 2920448 csum 2650415738 expected csum 1203463532 However, I really don't know how to address this in btrfs. I want to give btrfs a chance. But.. If I can't fix this over the next week I'll have to switch back to ext4. Cheers! Roman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 11/30/2014 03:59 PM, Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 2014-11-30 21:22, Roman Bysh wrote:

...

However, I really don't know how to address this in btrfs. I want to give btrfs a chance. But..

Start the traditional way. Boot from the 13.2 rescue image, and run fsck on that partition.

Then, if it still fails...

If you have the disk space, place /usr/src/ on a different partition. My preference is reiserfs, because builds run way faster. I measured it years ago... apparently, a build sequence creates and destroys many small files, and reiserfs is better at this.

But for testing, any filesystem on "/usr/src/" would do. Do you have /home as xfs? Then just create "/home/temporary_usr_src", copy the files there, rename /usr/src/ to /usr/src_old, then symlink /home/temporary_usr_src to "/usr/src/".

(I'm unsure if you have to tell btrfs or snapshot to remove something)

Then run the 'make' again. If it runs, you found the culprit.

- -- Cheers / Saludos,

Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux)

iEYEARECAAYFAlR7hTQACgkQtTMYHG2NR9XiiACfWdYroYE91gS3SXckifW8wjqw lkUAnjWRIr2QmEIOu4QHn8Q3kIbl2mwQ =7RLZ -----END PGP SIGNATURE-----

I rebooted with the dvd in rescue mode. Then I ran: btrfs check --init-csum-tree /dev/sda1 Message I got: Backrefs don't agree with each other and extent record doesn't agree with anybody, so we can't fix bytenr 592625664 bytes 16384 failed to repair damaged filesystem, aborting. I also ran btrfs check --repair /dev/sda1 Same error showed up. Since this is my main machine I reinstalled using ext4. I decided that I'll wait another two or three years until the btrfs and btrfs patches for the kernel are more mature. For now, I'll test and practice btrfs on VirtualBox. It's much safer. Cheers! Roman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-12-01 02:28, Roman Bysh wrote:

On 11/30/2014 03:59 PM, Carlos E. R. wrote:

Message I got:

Backrefs don't agree with each other and extent record doesn't agree with anybody, so we can't fix bytenr 592625664 bytes 16384 failed to repair damaged filesystem, aborting.

Yiks.

I also ran btrfs check --repair /dev/sda1 Same error showed up. Since this is my main machine I reinstalled using ext4.

LOL. I can't convince you to try again, because I'd probably do the same as you did...

I decided that I'll wait another two or three years until the btrfs and btrfs patches for the kernel are more mature.

For now, I'll test and practice btrfs on VirtualBox. It's much safer.

Understandable. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlR74nYACgkQtTMYHG2NR9U8EACfQuuWCZbh1SXi9BdEmP1NlOZS Ln4AmwS0D7ea4E/RU+aHNNefSWyXof5T =eT6J -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Roman Bysh writes:

Why does the ID number start at 300? Interesting. I've seen examples where it starts at 1.

Mine actually started at 0 for the initial snapshot. But older snapshots get auto-cleaned, so if you've let it run for a while, you're going to see only higher numbers of those snapshots still active.

Tomorrow, I'm going to order a 1TB drive. How much space should I assign for root when using btrfs?

The installer assigned 40G and I've kept it that way. It's on an LVM volume, so I can increase it if I wanted to. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Tue, Dec 2, 2014 at 12:00 AM, Roman Bysh wrote:

The main problem in btfs is when you get "csum" errors in the filesystem. See the errors I posted earlier. When I restored my earliest snapshot the csum errors still existed.

Checksum error means filesystem corruption. btrfs snapshots cannot protect against it - snapshot is frozen state of filesystem, so if filesystem has any corruption it is frozen in snapshot as well. If you have redundancy on *filesystem* level (RAID1 or similar) it can reconstruct failed block from another copy (I wonder, how it plays together with read-only snapshots ...); otherwise there is really nothing you can do short of restoring from backup or deleting corrupted files. http://www.gattis.org/Work-and-Tech/operating-systems-and-applications/unix/... Now *why* you got checksum errors is different question. It could be hardware issue, it could be btrfs bug. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 12/02/2014 01:42 AM, Achim Gratz wrote:

Roman Bysh writes:

...

How soon did you create your initial snapshot? Was it a single snapshot? I want to create a pre after the initial install and a post after the update. So if something goes wrong in the future I can restore it.

The initial snapshot was created when I gave snapper a config for the root filesystem. The install didn't creaqte one, and I started wondering about two weeks into using the machine why I wasn't seeing any snapshots.

...

The main problem in btfs is when you get "csum" errors in the filesystem. See the errors I posted earlier. When I restored my earliest snapshot the csum errors still existed.

I'm not sure a snapshot would help you here. For the snapshots to work I'd think the filesystem must not be corrupt. So, depending on what the error is and where it might take out the snapshots just as well.

A snapshot is nothing more than telling btrfs "if anything writes to this file, make a copy of it and keep the old data around".

Regards, Achim.

Guys, The filesystem is kaput. Other than a running a backup I must reinstall from my DVD. I've never had this problem with ext4. Too bad there is no way to recover using a btrfs tool. Maybe in two or three years. I will try tomorrow. Bye. Roman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On December 2, 2014 5:31:13 AM EST, "Carlos E. R." wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 2014-12-02 07:42, Achim Gratz wrote:

...

A snapshot is nothing more than telling btrfs "if anything writes to this file, make a copy of it and keep the old data around".

Which is not intuitive: typically a snapshot would be made at the instant of the request, occupying the full needed space. Btrfs snapshots are different, compact and fast. Not a backup snapshot.

I don't find snapshot a well-defined term. There are both COW snapshots and clone snapshots. I believe most file systems use COW technology to implement snapshots. Linux LVM uses COW. http://en.m.wikipedia.org/wiki/Snapshot_(computer_storage)#File_systems But there are also lots of systems that implement snapshots as clones. The way I've seen the most is for a mirror to be created, then broken at snapshot time. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Tuesday 2014-12-02 14:03, Carlos E. R. wrote:

I think that xfs implement copy snapshots, to another device or partition.

Source? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-12-02 14:50, Greg Freemyer wrote:

On December 2, 2014 8:11:54 AM EST, Jan Engelhardt <> wrote:

...

On Tuesday 2014-12-02 14:03, Carlos E. R. wrote:

...

I think that xfs implement copy snapshots, to another device or partition.

Source?

He doesn't have one because he's mistaken.

Well, I said "think" because I wasn't sure :-)

Xfs provides xfsfreeze (sp) that flushes all data / metadata to disk, then an underlying layer in the storage stack can instantiate a snapshot by either COW or clone. (LVM snapshots being an example of that.)

After the snapshot is instantiated xfsfreeze -u is called to release the filesystem to do more work.

I wrote the first xfstest implementation for testing xfsfreeze a decade ago so I'm (or was) very familiar with xfsfreeze. (At the time the kernel implementation was buggy and would corrupt open files at times iirc. I was using it and had to write the xfstest to get a reproduced. With that in hand the kernel team fixed it in short order.)

There is also "xfsdump", which can be done "live". In fact, it has to be mounted. It is some kind of snapshot, no? - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlR90vQACgkQtTMYHG2NR9WVagCfRHcIb7OILPHzZW3/TsT73E5j uxMAnRild37MJ4XQProo3NVvzco62JW6 =eenY -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-12-02 16:14, Greg Freemyer wrote:

On Tue, Dec 2, 2014 at 9:55 AM, Carlos E. R. <> wrote:

...

There is also "xfsdump", which can be done "live". In fact, it has to be mounted. It is some kind of snapshot, no?

No idea how xfsdump is implemented. I've never used it personally.

Well, I don't know the details, but it does a complete backup of everything in the filesystem, very fast, that can be used to recreate it. It is incremental. IIRC, it needs that the destination be also an xfs filesystem, so it is not a plain file copy, nor is it an exact image (specially of the metadata). (I know this later part from experience, from dumping a broken filesystem, reformat, restore, that the restored copy was not broken) (that brokenness was unrepairable, it was only detected) Only xfsrestore can read the dump. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlR92+4ACgkQtTMYHG2NR9XM2gCfUc1XhA5eeVdXw3foo6AByklP HdAAnRgr59xnej69UhZWixoKjRN8kHSB =3qSS -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 12/02/2014 03:51 AM, Lars Müller wrote:

On Mon, Dec 01, 2014 at 04:00:11PM -0500, Roman Bysh wrote: [ 8< ]

...

The main problem in btfs is when you get "csum" errors in the filesystem. See the errors I posted earlier. When I restored my earliest snapshot the csum errors still existed.

And you filed a bug report? No you did'n? Please do and report the bug ID back to this thread. Also add to the bug report a pointer to this thread like http://lists.opensuse.org/opensuse-factory/2014-12/msg00018.html

I'm sure you'll find a posting in the thread which explains your issue better.

Please also consder to update/ modify the subject of the thread too so your description might get the attention of a kernel developer. Something like:

Subject: btrfs, all the devs, and openSUSE suck (was: Internal compiler error when running make oldconfig && make prepare)

...

So early in my initial installation, I could no longer compile changes using the "make" command. I could not fix it. I didn't do anything special or extreme for the filesystem to get so corrupted.

Personally, I think the btrfs filesystem and its tools need more time to evolve.

See, that's what you believe. But we need to track what's going on. And therefore we need a bug report.

At the end it anyhow will be your broken memory. ;) It wouldn't be the first time btrfs (or xfs) defaced such an issue.

Cheers,

Lars

Thank you for subject help. I usually submit a bug report. What was consistent with my actions was that I kept deleting all of the unimportant snapshots in snapper each evening. Except my earliest single snapshot. Interesting this happened after two weeks of deleting snapshots. Could this be the cause? Maybe, maybe not. Can this function be added to the script to https://openqa.opensuse.org/ ? I followed an example of creating it from the snapper website. I should have created a pre and post snapshot to restore. Yes. The problem is that I've reinstalled using ext4. I was upset that I could no longer work with the kernel source. And, the snapshot was read-only when booting into it. I will reinstall with btrfs. During the installation, I always remove the grub2-efi subvolume. My Asus P5Q motherboard uses a standard BIOS. I've been using the ext2 filesystem since 2000 with Mandrake 7. Then Mandrake Linux 8 - 9.2 with ext3. Then switched to openSUSE 10 using ext4 and reiser3 filesystems. I really liked reiser3 - and it was fast. IMHO As a user, I never had to worry about the filesystem except the odd fsck until btrfs. That's a fact. Nor should any user. However, I enjoy a challenge. I'll keep on testing and testing. Next time, I'll submit a bug report as soon as it happens. There's a lot of the new information to cram into this old brain of mine `:-) Cheers! Roman ----------------------------------- openSUSE Open Minds Open Sources Open Future ----------------------------------- http://linuxcounter.net/ #179293 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 12/03/2014 06:39 AM, Lars Müller wrote: | Hi Roman, | | On Tue, Dec 02, 2014 at 08:10:17PM -0500, Roman Bysh wrote: | [ 8< ] || Thank you for subject help. I usually submit a bug report. || What was consistent with my actions was that I kept deleting all of the || unimportant snapshots in snapper each evening. Except my earliest single || snapshot. Interesting this happened after two weeks of deleting snapshots. || || Could this be the cause? Maybe, maybe not. | | Yes, but as you're not sure I would not blame btrfs even if other | filesystems work. I would check the hardware, in particular the memory | and the CPU. I ran a lengthy test using memtest and no RAM errors appeared. My CPU is a Q6600 with no overclocking. No overheating. Ran a test with all four cores enabled and no errors. My drive is WD "Black Caviar" 500 GB 7200. I ran an extensive test using System Rescue CD x86_64 with smartd and it passed. | If you have a syslog daemon running check if there are any traces of a | kernel ooops. Maybe you're able to catch a kernel ooops. | | Check the CPU fan in particular for dust. I had been surprised how much | collected inside of my workstation when I had to replace a disk | recently. The desktop mobo, CPU fans and all internals, connectors etc.. were cleaned recently. No dust yet. || Can this function be added to the script to https://openqa.opensuse.org/ ? | | How? Please make a more precise suggestion how to improve openqa in | this regard. Add a script so that it would create a single snapshot when the installation has (including the updates) finished. Next, keep deleting snapshots in snapper up to the single snapshot originally created. Run dmesg | dmesg.log and post log. Today I feel better. I'll be reinstalling using btrfs. -- Cheers! Roman ----------------------------------- openSUSE Open Minds Open Sources Open Future ----------------------------------- http://linuxcounter.net/ #179293 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Mon, Dec 1, 2014 at 3:32 PM, Roman Bysh wrote:

On 12/01/2014 03:12 PM, Roman Bysh wrote:

...

On 11/30/2014 10:37 PM, Carlos E. R. wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

I really wanted to like btrfs. IMHO Snapper is silently taking way too many snapshots. I think that a snapshot message should pop up letting the user know. Otherwise I have to go in and delete a lot of the snapshots.

I cannot see the size of the snapshots in snapper only in console text. It should create a pre after the installation and post after the updates.

Why does the ID number start at 300? Interesting. I've seen examples where it starts at 1.

Tomorrow, I'm going to order a 1TB drive. How much space should I assign for root when using btrfs?

Follow Up

I can create a pre and post. However, most of the settings are in zypper online_update snapshots. Perhaps there is a way to merge them.

I'm wondering how a new desktop user to Linux and btrfs will understand this all.

I know that there is more flexibility while in console text. Right now, there is insufficient documentation for the average user.

We will probably need another year for us to create lots of examples for the openSUSE wiki.

I will be documenting as much as possible while in VirtualBox.

-- Cheers!

Roman

Microsoft Windows NTFS has had snapshots similar to btrfs since Windows 2003 came out. Vista / WIn7 / WIn8 all have it routinely in use. Most users don't even know it is happening. For Microsoft the secret has been to make it all transparent to the user. Btrfs unfortunately is a little too in your face but I hope the goal is to hide most of the details from the user. Greg -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Mon, Dec 1, 2014 at 3:12 PM, Roman Bysh wrote:

I really wanted to like btrfs. IMHO Snapper is silently taking way too many snapshots. I think that a snapshot message should pop up letting the user know. Otherwise I have to go in and delete a lot of the snapshots.

I think snapper is pretty efficient and only files like database files of VM backend storage consumes drastically more space if there are lots of snapshots. As a simple example, assume I have a collection of 3 1 GB files and I delete 1 of them each day for 3 days. Further I assume I make a nightly snapshot. Day 1: Snapshot 1 created - currentlyempty File 1 deleted - behind the scenes it is moved to snapshot 1 Day 2 Snapshot 2 created - currently empty - Snapshot 1 is effectively frozen File 2 deleted - behind the scenes it is moved to snapshot 2 Day 3 Snapshot 3 created - currently empty - Snapshot 2 is effectively frozen File 3 deleted - behind the scenes it is moved to snapshot 3 So at the end of the day, I have 3 snapshots, each 1GB in size. If you delete snapshot 2, then its contents are merged with snapshot 1, so now snapshot 1 is 2GB and snapshot 3 is 1GB. Thus for static file creation / deletion scenarios, btrfs is very efficient and you don't really recover any disk space until you delete the oldest snapshot. Where btrfs falls down with lots of snapshots is when you have a file like a database where the same data blocks routinely have different content. Then each of the snapshots has to maintain the content of the data blocks redundantly. ie. dnapshot 1 has the contents of the changing data blocks as of the morning of Day 1, Snapshot 2 has the day 2 content, etc. I believe therefore that it is recommended to disable COW snapshot functionality on database files. The same logic applies to VM virtual disks, so there too it is recommended to disable COW functionality. Thus if you are following the guidelines to disable COW on problematic files, then there is very little penalty associated with having lots of snapshots. Greg -- Greg Freemyer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Mon, Dec 1, 2014 at 5:12 PM, Roman Bysh wrote:

On 12/01/2014 04:20 PM, Greg Freemyer wrote:

...

On Mon, Dec 1, 2014 at 3:12 PM, Roman Bysh wrote:

...

I really wanted to like btrfs. IMHO Snapper is silently taking way too many snapshots. I think that a snapshot message should pop up letting the user know. Otherwise I have to go in and delete a lot of the snapshots.

I think snapper is pretty efficient and only files like database files of VM backend storage consumes drastically more space if there are lots of snapshots.

As a simple example, assume I have a collection of 3 1 GB files and I delete 1 of them each day for 3 days. Further I assume I make a nightly snapshot.

Day 1: Snapshot 1 created - currentlyempty File 1 deleted - behind the scenes it is moved to snapshot 1

Day 2 Snapshot 2 created - currently empty - Snapshot 1 is effectively frozen File 2 deleted - behind the scenes it is moved to snapshot 2

Day 3 Snapshot 3 created - currently empty - Snapshot 2 is effectively frozen File 3 deleted - behind the scenes it is moved to snapshot 3

So at the end of the day, I have 3 snapshots, each 1GB in size.

If you delete snapshot 2, then its contents are merged with snapshot 1, so now snapshot 1 is 2GB and snapshot 3 is 1GB.

Thus for static file creation / deletion scenarios, btrfs is very efficient and you don't really recover any disk space until you delete the oldest snapshot.

Where btrfs falls down with lots of snapshots is when you have a file like a database where the same data blocks routinely have different content.

Then each of the snapshots has to maintain the content of the data blocks redundantly.

ie. dnapshot 1 has the contents of the changing data blocks as of the morning of Day 1, Snapshot 2 has the day 2 content, etc.

I believe therefore that it is recommended to disable COW snapshot functionality on database files.

The same logic applies to VM virtual disks, so there too it is recommended to disable COW functionality.

Thus if you are following the guidelines to disable COW on problematic files, then there is very little penalty associated with having lots of snapshots.

Greg -- Greg Freemyer

Greg,

You are a great teacher! I was unaware of this! Is it possible for you to add your knowledge to the openSUSE wiki on btrfs?

I do edit the wiki on occasion but not for btrfs. One reason is my true expertise is with Windows NTFS shadow copies. I have to work with them at a detail level in my day job (computer forensics). NTFS shadow copies are very similar to btrfs snapshots. Also since NTFS shadow copies have been in production use for over a decade there is lots of documentation / reverse engineering in place for shadow copies.

There is so much that users do not know about the btrfs.

I also am not expert. I've barely tinkered with btrfs, but I understand COW in great detail from other storage systems / file systems.

I've encountered a csum error problem that corrupted my filesystem.

See below:

<-- snip --------------------------------------------------------- If I type "make" in /usr/src/linux - I get an error:

make /bin/sh: line 1: 5471 Bus error ( ar rcD "$TMP" ) > /dev/null 2>&1 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[2]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make[1]: *** [scripts_basic] Error 2 HOSTCC scripts/basic/fixdep gcc: internal compiler error: Bus error (program as) Please submit a full bug report, with preprocessed source if appropriate. See http://bugs.opensuse.org/ for instructions. scripts/Makefile.host:118: recipe for target 'scripts/basic/fixdep' failed make[1]: *** [scripts/basic/fixdep] Error 4 Makefile:475: recipe for target 'scripts_basic' failed make: *** [scripts_basic] Error 2 <-- snip ----------------------------------------------------------

I have only been running it for one month. I'm unable to work with the Linux kernel source. Please see mhy earlier posts.

Is this a rare occurrence? The tools were not able to fix the csum errors.

Should I reinstall with btrfs. How would you create your first pre and post snapshot after the initial installation using snapper?

Sorry, I'm just mister theory for btrfs, you have transitioned to practice!

Any thoughts?

Roman

Greg -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org