On Tue, 2014-11-04 at 22:19 -0500, Timothy Butterworth wrote:

On Tue, Nov 4, 2014 at 9:15 PM, Anton Aylward wrote:

...

On 11/04/2014 06:53 PM, John Andersen wrote:

...

While installing 13.2 I let it use btrfs for root and xfs for /home partition.

Then you get these nebulous message that says some of the subvolumes of root are shadowed by other filesystems. That message means that you have a subvolume named /home defined in the /root automatically, and also a /home partition. Better language is needed for that error message. The term "Duplicate directory" names comes to mind. "Shadowed" is not very descriptive.

But then I began to wonder why btrfs defines subvolumes instead of letting the system define sub-directories?

Whats the difference in functionality? Are these sub-volume sizes fixed? Is it just part of the snapshot process? What's up with that?

I'm sure the designers had something in mind and its quite possibly documented somewhere. What follows is my thoughts and experience.

I did the installation as you described. Eventually the /home partition corrupted irretrievably. Its not a conventional reiserFS restored from a backup.

I'm glad I stayed with EXT4 for all partitions except /boot/efi with is FAT of course.

...

I think that the BtrFs model is that the FS should encompass the whole of the system as one logical volume possibly across many partitions or platters or spindles. It has that capability.

The subvolume mechanism, the manual says, can be treated like a directory or a moutned FS. Perhaps that's what is confusing about it. You can have a real (as in the XFS) mounted on /home or or ....

Well, in any other other FS if you don't mount the partition on /home you can still create stuff there, and when you do the actual mount it does overlay. "shadowed". So I asked myself why bother and cleared out all the subvolumes.

OUCH! That mean that /usr got DELETED. The subvolumes are not just markers. They really are directories.

Or are they? See https://lwn.net/Articles/579009/

*sigh* Re-install. "Learning experience".

This time I did without subvolumes and I can't say i missed them.

I'm sure that the idea behind BtrFS is that it can optimise the btree and space in some fantastic way. I'm sure that it really wants the whole of my 1T drive to be the BtrFS, including /boot and /swap. And I'm sure it can mirror onto another, and I'm sure that the subvolumes can be snapshotted and can then be used for a disk-to-disk-to-tape backup. I'm certain that just as my LVM partitions can grow and grow across spindles, so can BtrFS.

I am not, however, ready, to try installing a system on a single, encompassing single BtrFS. I'll let someone else try it and report. I'll also let someone else experiment with that BtrFS growing across many spindles.

https://btrfs.wiki.kernel.org/index.php/Main_Page

What I might experiment with, though, is this

<quote src="http://www.oracle.com/technetwork/articles/servers-storage-admin/advanced-btrfs-1734952.html"> Redundant Configuration

With Btrfs, you no longer need to use mdadm to create mirrored volumes or complex RAID configurations. These capabilities are built into the file system. </quote>

What I'm not clear on is how to extend an existing FS to become a BtrFS style RAIDFS.

-- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon?

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

I've been using btrfs for a couple years or so now without incident. Internally it's more complicated than I can comprehend, but in daily use I don't know anything until I have to roll some changes back. As for XFS, my buddy Drew (fellow openSUSE Advocate) has been running it for years and years even before the newer optimizations and fixes. Anomalies happen, but in general I trust our developers; after all, I sure as hell can't do what they do.

-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/04/2014 06:15 PM, Anton Aylward wrote:

On 11/04/2014 06:53 PM, John Andersen wrote:

...

While installing 13.2 I let it use btrfs for root and xfs for /home partition.

Then you get these nebulous message that says some of the subvolumes of root are shadowed by other filesystems. That message means that you have a subvolume named /home defined in the /root automatically, and also a /home partition. Better language is needed for that error message. The term "Duplicate directory" names comes to mind. "Shadowed" is not very descriptive.

But then I began to wonder why btrfs defines subvolumes instead of letting the system define sub-directories?

Whats the difference in functionality? Are these sub-volume sizes fixed? Is it just part of the snapshot process? What's up with that?

I'm sure the designers had something in mind and its quite possibly documented somewhere. What follows is my thoughts and experience.

I did the installation as you described. Eventually the /home partition corrupted irretrievably. Its not a conventional reiserFS restored from a backup.

I think that the BtrFs model is that the FS should encompass the whole of the system as one logical volume possibly across many partitions or platters or spindles. It has that capability.

The subvolume mechanism, the manual says, can be treated like a directory or a moutned FS. Perhaps that's what is confusing about it. You can have a real (as in the XFS) mounted on /home or or ....

Well, in any other other FS if you don't mount the partition on /home you can still create stuff there, and when you do the actual mount it does overlay. "shadowed". So I asked myself why bother and cleared out all the subvolumes.

OUCH! That mean that /usr got DELETED. The subvolumes are not just markers. They really are directories.

Or are they? See https://lwn.net/Articles/579009/

*sigh* Re-install. "Learning experience".

This time I did without subvolumes and I can't say i missed them.

I'm sure that the idea behind BtrFS is that it can optimise the btree and space in some fantastic way. I'm sure that it really wants the whole of my 1T drive to be the BtrFS, including /boot and /swap. And I'm sure it can mirror onto another, and I'm sure that the subvolumes can be snapshotted and can then be used for a disk-to-disk-to-tape backup. I'm certain that just as my LVM partitions can grow and grow across spindles, so can BtrFS.

I am not, however, ready, to try installing a system on a single, encompassing single BtrFS. I'll let someone else try it and report. I'll also let someone else experiment with that BtrFS growing across many spindles.

https://btrfs.wiki.kernel.org/index.php/Main_Page

What I might experiment with, though, is this

<quote src="http://www.oracle.com/technetwork/articles/servers-storage-admin/advanced-btrfs-1734952.html"> Redundant Configuration

With Btrfs, you no longer need to use mdadm to create mirrored volumes or complex RAID configurations. These capabilities are built into the file system. </quote>

What I'm not clear on is how to extend an existing FS to become a BtrFS style RAIDFS.

All very interesting reading. The first link especially is a nice gentle introduction to the concepts. And looking at my mtab I can see it did indeed mount all those sub-volumes separately. /dev/sda2 /.snapshots btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/tmp btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/spool btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/opt btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/log btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/lib/named btrfs rw,relatime,space_cache 0 0 /dev/sda2 /boot/grub2/x86_64-efi btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/crash btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/lib/pgsql btrfs rw,relatime,space_cache 0 0 /dev/sda2 /usr/local btrfs rw,relatime,space_cache 0 0 /dev/sda2 /var/lib/mailman btrfs rw,relatime,space_cache 0 0 /dev/sda2 /tmp btrfs rw,relatime,space_cache 0 0 /dev/sda2 /srv btrfs rw,relatime,space_cache 0 0 /dev/sda2 /opt btrfs rw,relatime,space_cache 0 0 /dev/sda2 /boot/grub2/i386-pc btrfs rw,relatime,space_cache 0 0 Some of those mounts are two levels deep in the file tree! I've got a bit of studying to do. For now I'm using it more or less as the default install. I did a fresh install because I needed to repartition my disk anyway. Since I have a fresh backup from before, I am also trying encrypted volumes. It didn't want to let me encrypt a btrfs partition where I store my source code so I had to forgo the snapshot capabilities and use XFS for that partition. -- Explain again the part about rm -rf / -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/05/2014 06:39 AM, Carlos E. R. wrote:

Different feature set, there is some isolation between subvolumes. It is, I guess, like different partitions, but flexible, like in LVM. I haven't read it up, but I'd expect automatic resizing, or manual, but inside the current media container, which normally would be the entire hard disk.

That is what I've come to believe. The philosophy behind BtrFS seems to be to take the resource management we saw with LVM, where the logical extents, which may be on more than one spindle, are all managed as one pool. The difference between LVM and BtrFS here is that the pool == file system. For those of us who are used to using partitions, even LVM logical volumes, to divide the FS up into manageable chunks for various reasons[1], this is more scary than a 1980s slasher movie. There are advantages that cut in, or will eventually, with the COW and the optimizations that a B-tree system can perform at a global level. The ability to do RAID of some forms within the file system is attractive. What IS NOT is that I see no way where I can add another drive at a later date and make that part of the RAID as opposed to merely part of the FS. Maybe I'm wrong in that, but all the examples I've found for RAID with BtrFS involve it being set up at MKFS time. [1] see the link to /tmp security bug for example. see the use of "-x" in rync and "-xdev" in find to limit/speed things up see the use of partitions to limit how much space a runaway process can consume before its blocked -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/05/2014 08:13 AM, Joseph Loo wrote:

...

I believe there are commands to add and remove disks from the btrfs during run-time.

Examples or references please. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/05/2014 08:49 AM, arvidjaar@gmail.com wrote:

Отправлено с iPhone

...

5 нояб. 2014 г., в 18:24, Anton Aylward написал(а):

On 11/05/2014 08:13 AM, Joseph Loo wrote:

...

I believe there are commands to add and remove disks from the btrfs during run-time.

Examples or references please.

Btrfs device add; btrfs balance start convert=raid1

1. I subscribe to the list; no need to cc me as well. 2. Is that one command or two? The capitalization seems odd. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Thu, Nov 6, 2014 at 7:28 AM, Joseph Loo wrote:

On 11/05/2014 05:24 AM, Anton Aylward wrote:

...

On 11/05/2014 08:13 AM, Joseph Loo wrote:

...

...

I believe there are commands to add and remove disks from the btrfs during run-time.

Examples or references please.

btrfs filesystem resize [devid:][+/-]<newsize>[kKmMgGtTpPeE]|[devid:]max <path> Resize a filesystem

btrfs device add [options] <device> [<device>...] <path> Add a device to a filesystem btrfs device delete <device> [<device>...] <path> Remove a device from a filesystem btrfs scrub start [-BdqrRf] [-c ioprio_class -n ioprio_classdata] <path>|<device> Start a new scrub. If a scrub is already running, the new one fails. btrfs scrub cancel <path>|<device> Cancel a running scrub

Neither of these commands changes raid type of btrfs. You need "btrfs balance" to do it. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/05/2014 11:28 PM, Joseph Loo wrote:

On 11/05/2014 05:24 AM, Anton Aylward wrote:

...

On 11/05/2014 08:13 AM, Joseph Loo wrote:

...

...

I believe there are commands to add and remove disks from the btrfs during run-time.

Examples or references please.

btrfs filesystem resize [devid:][+/-]<newsize>[kKmMgGtTpPeE]|[devid:]max <path> Resize a filesystem

btrfs device add [options] <device> [<device>...] <path> Add a device to a filesystem btrfs device delete <device> [<device>...] <path> Remove a device from a filesystem

Sorry, I didn't make myself clear. It wasn't the specific commands I was asking about, it was the "during run-time" I was questioning. If course, if I boot from the CD in maintenance mode I can do all that, since the disk if really off line, but if by "run time" you mean the live system, while the file system is in use (as I do with ReiserFS), no I haven't done that What I'm asking about is not the commands themselves but a reference to "during run time" in the sense of a live FS in use. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/06/2014 08:05 AM, Joseph Loo wrote:

One of the most important feature of btrfs is the vacuum system where you can check the state of the file system while it is running, that is check if the data is intact or to check for bit rot on the file system.

Tell that to Rodney Baker! He has pointed out that he can't run (conventional) FSCK on his BtrFS. When people report complete disasters, as he has done, I always do wonder what went wrong. In absolute terms no s/w is bug free, but that doesn't mean it is not usable or stable enough to use. The Linux kernel is in consonant flux and the same applies to commercial software. Long ago IBM discovered that there were around 200 clear bugs in /360. After each new release and upgrade they still found around 200 clear bugs were reported :-) Software is like that. All of engineering is like that; it is a human artifice and every bit as imperfect as its creator. But like its creator it can be quite viable. The biggest difference I see between commercial software and FOSS is how they are released. Commercial software can pay for internal testing before releasing the alpha and beta versions to a small group of external testers, probably with specific skills and probably under a non disclosure agreement. FOSS follows the philosophy of "release early, release often" and is indiscriminate in who its testers are, so that incomplete or poorly installed/configured releases happen and often get publicity. And of course the way the internet and google works those bad reviews and commentaries about version 0.01beta don't make it clear that they were for 0.01beta and stick around and influence people even when release 3.5 comes out. Please note, I'm not saying "because suse released it, it must be good". Suse released ext3 and ext4 a while back and I find them more difficult to configure and less reliable than the supposedly unsupported ReiserFS. Oh, and people don't differentiate between ReiserFS and Reiser4FS when they badmouth it. For me, BtrFS as a root file system has proven quite reliable. That's the BtrFS which came with 12.3 and is not on 13.1. Its seen the upgrades. I'm not using factory or evergreen. :~> uname -a Linux Mainbox 3.11.10-101.g966c1cc-desktop #1 SMP PREEMPT Fri Oct 31 17:56:41 UTC 2014 (966c1cc) x86_64 x86_64 x86_64 GNU/Linux -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/06/2014 07:54 AM, Rodney Baker wrote:

I installed a laptop with btrfs - I now have a B$)(*#$ Totally R%%ted file system!

When something that drastic happens to me - and it has a couple of time in the 10.x and 11.x era - when everyone else seems to get by and do successful installs, I question myself and my hardware and my configuration rather than say the s/w is a pile of doggy do-do. I found I'd made I mess of the bios with respect to the options at install. I've also observed that installing on laptops is more prone to weird things happening. I've also observed that the old, slow, small disk systems from the Closet of Anxieties with old, simplistic BIOSes and single core cpus, never seem to have any of these problems. (even when I install BtrFS on them!) Perhaps were getting too sophisticated for our own good?

Never again! / mounts as read-write but as soon as a user is logged on it switches to RO mode, which means /var is now read-only and I cannot install any updates because zypper won't work (nor will anything else that relies on writing files to /var/tmp or /tmp, because they now exist on a read- only file system.

Yes, that happens. That happens even with ext3. I've had it happen when I've foolishly installed / as ext3. I do have a ext3 partition that I mirror my real /boot onto 'just in case'. My real /boot is on my LVM and I don't have any problems with it. It is of course a ReiserFS.

Of course, btrfsck won't check / because it is mounted and can't be unmounted to check it, even in single user mode!

Nothing new about that. The reality is that the ramdisk system in your initrd used to do the fsck before pivoting and mounting the real system. If you needed to do a fsck on root manually you needed to use the CD's maintenance mode. However as Joseph Woo points out there are other 'recovery' tools than fsck. You might also, as he suggests, consult the wiki. You might find https://btrfs.wiki.kernel.org/index.php/Mount_options useful. The https://btrfs.wiki.kernel.org/index.php/Btrfsck page mentions mounting in recovery mode.

Curses and naughty words! This will mean a clean install once 13.2 has settled down for a month or so, back to ext4 partitions. Forget btrfs. It isn't worth the hassle, yet.

I beg to differ. I've been running a stable BtrFS as / for almost a year. I've always had hassle with ext4. ALWAYS. Which is why the stuff I mount under /home is all ReiserFS. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/06/2014 07:53 AM, jdd wrote:

I stopped using it when Reiser has his problems.

Because you did 't want all that money flowing to a murderer? Oh, wait....its free as in beer software. The truth of the matter is that Suse (even before opensuse) was the point distro for reiserfs maintenance, and some guy left for greener pastures, so no more maintenance was done on it. But because it was virtually bullet proof by that time it hasn't needed much/and maintenance. Every reiserfs failure I've had (both of them in like forever) was hardware induced. People say it doesn't scale. I say so what!? But visiting the sins of the developer upon the software is sort of dumb if you ask me. -- Explain again the part about rm -rf / -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2014-11-06 20:52, Anton Aylward wrote:

On 11/06/2014 01:51 PM, John Andersen wrote:

...

The truth of the matter is that Suse (even before opensuse) was the point distro for reiserfs maintenance, and some guy left for greener pastures, so no more maintenance was done on it. But because it was virtually bullet proof by that time it hasn't needed much/and maintenance.

+1 A testimony to good design and good fundamental coding.

I have an unsolved bugzilla on it, IIRC.

...

Every reiserfs failure I've had (both of them in like forever) was hardware induced.

+1 (sadly)

I remember at least one software crash. Two files with different names were considered the same name and clashed, causing filesystem total corruption. Long ago. Caused a bit of uproar at the time.

...

People say it doesn't scale. I say so what!?

It scales enough for me :-)

It uses only one thread and one core, even when having several large disks and dozens of free cpu cores. Thus it can not parallelize writes. Which is unfortunate, because I love that filesystem; only that I don't use it as much as I would like. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2014-11-07 01:57, John Andersen wrote:

On 11/06/2014 04:51 PM, Carlos E. R. wrote:

...

It uses only one thread and one core, even when having several large disks and dozens of free cpu cores. Thus it can not parallelize writes.

Probably in recognition of the fact that disks have exactly one controller, on on board processor, one arm.

But there are several disks. I have a minimum of 4, sometimes 8. Even with one disk, you may have one thread "thinking" and another actually writing or reading, and another stuck on some wait. Linux is good at that. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

On 2014-11-07 02:58, Anton Aylward wrote:

PDP-11 and early Vaxen had a similar problem. The dis controller could support up to 8 spindles. Good, you think. 4K file system, 512 byte sectors, 8 drives, PARALLELISM!

No! Could only transfer to one spindle/arm at a time. best effort, of you hand managed the control queue, and had a controller with dedicated DMA that could fetch the control blocks that you set up and chained was to have

But Linux can parallelize even on PATA hardware. It can send a read chunk command to the disk, and instead of waiting, send another command to another disk - maybe even on the same cable, I'm unsure. This is what allowed us to burn dvds, for instance: reading from one device, writing to another, and not being stuck waiting for the next chunk. You could even do some other "heavy" operations at the same time, on a puny Pentium 1. On SATA hardware this is even more powerful. We no longer are limited by the early far west explorers capabilities. No more wood wagons: we have trucks, locomotives, planes, cars... ;-pp -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)

Bernhard Voelker wrote:

On 11/07/2014 01:57 AM, John Andersen wrote:

...

- -- Explain again the part about rm -rf /

Just saw this in the signature. Actually this is not as scary as one might expect. (The braver ones among you may already have tried it in a snapshotted VM.)

POSIX mandates to skip operands that resolve to the root directory (as well as to "." and ".."). [0]

A good reason not to use POSIX compliant utilities. W/R/T : rm --one-file-system -fr /tmp/.) /tmp/root@ => / /tmp/mroot (mount -o bind / /tmp/mroot). If you use the new posix 'rm' and the suggested workaround: "use *", as in "cd /tmp && rm --one-file-system -fr *" -- in both cases the symlink'ed dir is followed (as "rm" never sees the symlink which is resolved to the root file system before 'rm' ever sees it). And "--one-file-system" won't protect either, as it only stops deletion if a sub-dir is on a different fs from the cmd-line arguments, but since you used "*", that will be pre-expanded to 'mroot' before 'rm' sees it. So again, no protection. Used to be rm -fr dir/. was the safest way to clean out a directory, *w/o* deleting the directory. I used it all the time to delete tmp dir contents (not usually '/tmp', but others). Now, there is no easy way. You need to use some incarnation of 'find', with find / -delete find /tmp/. delete replacing the previous rm based "solutions". Neat thing about find / -delete, is that it *seems* (not going to test it here!) to have no protections -- unlike "rm -fr /" where you had to specify some arg to override the default. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/07/2014 05:13 AM, John Andersen wrote:

I Liked it better when we were discussing systemd [..]

s/systemd/BRTFS/ ? (the original thread was about BTRFS ...) okay, so my little contribution there is: I like ext3/ext4, and things look like I will continue using them for quite some time: it's been very stable on my systems in the past years. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 11/07/2014 08:52 AM, John Andersen wrote:

Still, we'd all be using ext or ext2 if people didn't explore new alternatives.

I don't think so: that's the cool thing in the FOSS/*IX world: you can choose whatever file system you like ... $ 2>/dev/null curl http://git.savannah.gnu.org/cgit/coreutils.git/plain/src/stat.c \ | grep -c '^[ ]*case .*_MAGIC' 103 ... and if none of the existing file systems meets one's expectations, then nothing prevents one to write her own new file system. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org