[opensuse] Paying with the "future": Btrfs Grub2, systemd
Out of curiosity I pulled a scratch machine from 'the closet of anxiety' (it was labelled as having a dab disk) found another disk and some memory and the openSuse 12.2 installation DVD, oh and a DVD drive that worked, and .... Set up an install that used BtrFS, grub2 and systemd I left it to run overnight and came in just now and found a big red/orange error message saying that it hadn't be able to install the user - that's me, 'anton'. I rebooted and it was all there. It booted (using grub2) and systemd. I logged in as root and hand created the 'anton' account and all was merry, merry. I had partitioned the 10G drive as swap plus a single BtrFS partition. No separate /boot. The partitioner complained that I should have a separate /boot that was ext3 or ext4. I've had problems with that before so even if I were to do that I know there would be problems later. So I have one big partition. I've played with BtrFS before in a LVM partition, but it struck me that the design and advantages of BtrFS come into play when you have the big mix of files. I do have some reservations. Classically there have been good reasons for partitioning. I recall one vulnerability that arise if /tmp was on the same fs as the root. Good reason to have a /tmp that is nosetuid, possibly even noexec. In fact the principle of least privileged means you should apply those two to trees that have no reason to have executables or privilege. Certainly /usr/share -- documentation, manuals, fonts, icons, come into that category. There's a good case that /srv/httpd/ should be restricted too, after all the scripts there are to be interpreted (by perl or php or python or ruby or whatever) rather than executed. I'll look into these security concerns when I get to play about with this scratch machine over the coming weeks. Right now all I can say is 'it works'. So far. Next to bring it to the desk and tie it to the LAN... If you have questions about running Suse purely on BtrFS I'll try to answer them. Please don't ask performance questions, this is a crappy old machine. slow disk, slow cpu, slow memory. I'm concerned with functions and hurdles. -- "What you have to do and the way you have to do it is incredibly simple. Whether you are willing to do it, that's another matter." -- Peter Drucker -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
El 27/01/13 14:53, Anton Aylward escribió: The partitioner complained that I should have a
separate /boot that was ext3 or ext4.
That's a bug in the partitioner that I have also seen.
I do have some reservations. Classically there have been good reasons for partitioning. I recall one vulnerability that arise if /tmp was on the same fs as the root. Good reason to have a /tmp that is nosetuid, possibly even noexec.
Daemons using /tmp should be already using systemd's PrivateTemp feature that renders this kind of bugs moot. - daemon start, a new filesystem namespace is created with separate /tmp per process group. - if daemon dies, crashes or is stopped the task goes "out of scope" and the files it created there are deleted. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Cristian Rodríguez wrote:
Daemons using /tmp should be already using systemd's PrivateTemp feature that renders this kind of bugs moot.
- daemon start, a new filesystem namespace is created with separate /tmp per process group. - if daemon dies, crashes or is stopped the task goes "out of scope" and the files it created there are deleted.
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger.
Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 01/28/2013 11:13 AM, Dave Howorth wrote:
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger.
Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept?
If the daemon crashes and the files are wanted as evidence, then it must be run in a debugger or in some other way outside init. This is a feature of the linux kernel and is used by systemd, it is not usually a part of the "crashing daemon" itself -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Monday 28 January 2013 11:22:58 Cristian Rodríguez wrote:
On 01/28/2013 11:13 AM, Dave Howorth wrote:
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger.
Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept?
If the daemon crashes and the files are wanted as evidence, then it must be run in a debugger or in some other way outside init.
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? what a magnificent waste of time, did someone actually design that system or was it the result of a very very wet lunch with gallons of whisky? Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anders Johansson wrote:
On Monday 28 January 2013 11:22:58 Cristian Rodríguez wrote:
On 01/28/2013 11:13 AM, Dave Howorth wrote:
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger.
Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept?
If the daemon crashes and the files are wanted as evidence, then it must be run in a debugger or in some other way outside init.
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted?
what a magnificent waste of time, did someone actually design that system or was it the result of a very very wet lunch with gallons of whisky?
Both? -- Per Jessen, Zürich (3.4°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 29 Jan 2013 08:58:10 Per Jessen wrote:
Anders Johansson wrote:
On Monday 28 January 2013 11:22:58 Cristian Rodríguez wrote:
On 01/28/2013 11:13 AM, Dave Howorth wrote:
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence?
Yes, but the init system is not a debugger.
Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept?
If the daemon crashes and the files are wanted as evidence, then it must be run in a debugger or in some other way outside init.
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted?
what a magnificent waste of time, did someone actually design that system or was it the result of a very very wet lunch with gallons of whisky?
Both?
Isn't that what /var/log/* is for? -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ============================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
El 29/01/13 03:31, Anders Johansson escribió:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted?
FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tuesday 29 January 2013, Cristian Rodríguez wrote:
El 29/01/13 03:31, Anders Johansson escribió:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted?
FHS version 2.3 explains it for you.
"The /tmp directory must be made available for programs that require temporary files.
Programs must not assume that any files or directories in /tmp are preserved between invocations of the program."
This can't be true that you (as aggressive /usr merge supporter) are quoting FHS standard ... Usually you give a f*** on any standard. cu, Rudi -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 2013-01-29 at 14:42 +0100, Ruediger Meier wrote:
On Tuesday 29 January 2013, Cristian Rodríguez wrote:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are
El 29/01/13 03:31, Anders Johansson escribió: preserved between invocations of the program." This can't be true that you (as aggressive /usr merge supporter) are quoting FHS standard ... Usually you give a f*** on any standard.
It is customary and correct for UNIX processes to create temporary file and while holding a handle - delete that file. Meaning the temporary file will disappear along with the process anyway. The kernel feature just helps with poorly written processes that don't do this. So still, attempting to use a processes' debri as forensic information isn't going to work. But you could, via ZFS, LVM, or BRTFS always snapshot a filesystem in-progress and look at that. Or use an alternate temp location - most decent services support that. -- Adam Tauno Williams GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 2013-01-29 at 10:20 -0300, Cristian Rodríguez wrote:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are
El 29/01/13 03:31, Anders Johansson escribió: preserved between invocations of the program."
+1 Assuming you can collect forensic information from /tmp is mis-guided; plain and simple. If a deamon actually 'crashes' then it needs to be debugged, that means gdb or some tool, at least using strace. As it is running one assumes a deaemon will be logging relevant information somewhere - logs are intentionally persistent. If you have processes you really need to audit then they need to be built to support auditing [that is what workflow solutions and workflow engines are for]. -- Adam Tauno Williams GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Adam Tauno Williams wrote:
On Tue, 2013-01-29 at 10:20 -0300, Cristian Rodríguez wrote:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are
El 29/01/13 03:31, Anders Johansson escribió: preserved between invocations of the program."
+1 Assuming you can collect forensic information from /tmp is mis-guided; plain and simple. If a deamon actually 'crashes' then it needs to be debugged, that means gdb or some tool, at least using strace.
When a problem is not immediately reproducable, you can't do much else than wait and see - collect diagnostics, core dumps etc. Debugging a production issue with gdb and strace is for the optimistic. -- Per Jessen, Zürich (5.1°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 2013-01-29 at 17:56 +0100, Per Jessen wrote:
On Tue, 2013-01-29 at 10:20 -0300, Cristian Rodríguez wrote:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are
El 29/01/13 03:31, Anders Johansson escribió: preserved between invocations of the program." +1 Assuming you can collect forensic information from /tmp is mis-guided; plain and simple. If a deamon actually 'crashes' then it needs to be debugged, that means gdb or some tool, at least using strace. When a problem is not immediately reproducable, you can't do much else
Adam Tauno Williams wrote: than wait and see - collect diagnostics, core dumps etc. Debugging a production issue with gdb and strace is for the optimistic.
What? It isn't opportunistic at all. You configure the system to preserve core files; and you can look at them after-the-fact with gdb. UN*X is aware of core files from the bolts and screws on up. There are sysctl and ulimit parameters to control how core files are created [or not created] and managed. If you have a service that is crashing then you MUST grab a core. That is debugging step #2. [Step #1 being looking in the log]. -- Adam Tauno Williams GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Adam Tauno Williams wrote:
On Tue, 2013-01-29 at 17:56 +0100, Per Jessen wrote:
On Tue, 2013-01-29 at 10:20 -0300, Cristian Rodríguez wrote:
So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted? FHS version 2.3 explains it for you. "The /tmp directory must be made available for programs that require temporary files. Programs must not assume that any files or directories in /tmp are
El 29/01/13 03:31, Anders Johansson escribió: preserved between invocations of the program." +1 Assuming you can collect forensic information from /tmp is mis-guided; plain and simple. If a deamon actually 'crashes' then it needs to be debugged, that means gdb or some tool, at least using strace. When a problem is not immediately reproducable, you can't do much else
Adam Tauno Williams wrote: than wait and see - collect diagnostics, core dumps etc. Debugging a production issue with gdb and strace is for the optimistic.
What? It isn't opportunistic at all.
I meant it is optimistic to expect to have access to a production system and to run e.g. a daemon with gdb or strace it. -- Per Jessen, Zürich (10.9°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Thu, 31 Jan 2013 15:23:24 Per Jessen wrote:
[...]
When a problem is not immediately reproducable, you can't do much else than wait and see - collect diagnostics, core dumps etc. Debugging a production issue with gdb and strace is for the optimistic.
What? It isn't opportunistic at all.
I meant it is optimistic to expect to have access to a production system and to run e.g. a daemon with gdb or strace it.
I would agree. It would be silly to try that on a live, production system, which is why, if you must do the debugging yourself, you need test environment where you can duplicate the production system (including simulating the problem transaction(s)), do the debugging and test the fixes before pushing them out into the production environment. Of course, this requires considerable investment in the initial instance but it isn't really optional if you are developing and/or maintaining software in- house (but this is getting outside the scope of the original discussion). -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ============================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Rodney Baker wrote:
On Thu, 31 Jan 2013 15:23:24 Per Jessen wrote:
[...]
When a problem is not immediately reproducable, you can't do much else than wait and see - collect diagnostics, core dumps etc. Debugging a production issue with gdb and strace is for the optimistic.
What? It isn't opportunistic at all.
I meant it is optimistic to expect to have access to a production system and to run e.g. a daemon with gdb or strace it.
I would agree. It would be silly to try that on a live, production system, which is why, if you must do the debugging yourself, you need test environment where you can duplicate the production system (including simulating the problem transaction(s)), do the debugging and test the fixes before pushing them out into the production environment.
As anyone who has been around for long enough will know, it is impossible to completely duplicate a production system. An imitation is usually as good as it gets. When this doesn't work to your or your customer's satisfaction, you have to collect diagnostics as and when they appear. Which might just be files left in /tmp. Of course, daemons which might leave useful diagnostics in /tmp will have to be rewritten.
Of course, this requires considerable investment in the initial instance but it isn't really optional if you are developing and/or maintaining software in- house (but this is getting outside the scope of the original discussion).
Both very true. -- Per Jessen, Zürich (3.5°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
El 29/01/13 13:14, Adam Tauno Williams escribió:
+1 Assuming you can collect forensic information from /tmp is mis-guided; plain and simple. If a deamon actually 'crashes' then it needs to be debugged,
Yes, because crashing leaves the process internal routines in an "undefined state of completion" therefore those attempting to rely on files being available are fooling themselves. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 2013-01-29 at 07:31 +0100, Anders Johansson wrote:
On Monday 28 January 2013 11:22:58 Cristian Rodríguez wrote:
On 01/28/2013 11:13 AM, Dave Howorth wrote:
Cristian Rodríguez wrote:
On Mon 28 Jan 2013 08:33:16 AM CLST, Dave Howorth wrote:
No agenda, just curious. If the daemon crashes, won't the tmp files be wanted as evidence? Yes, but the init system is not a debugger. Sorry, but I don't understand what you mean. What does it have to do with the init system? And if the files are wanted, how are they kept? If the daemon crashes and the files are wanted as evidence, then it must be run in a debugger or in some other way outside init. So you are saying if a daemon crashes, all we can tell people is "sorry, run it again, your files have been deleted?
Your statement is fallacious. The correct statement is "sorry, run it again, your *TEMPORARY* files have been deleted". Yes. If the service does *NOT* work that way, then the service is broken. -- Adam Tauno Williams GPG D95ED383 Systems Administrator, Python Developer, LPI / NCLA -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----Original Message-----
From: Anton Aylward
Hans Witvliet said the following on 01/27/2013 04:38 PM:
Hi Anton,
After Gabor's impressive promotional tour of btrfs quite some time ago, i decided to have a go with brtfs, with my 4*3TB disks
WOW! "Live" and "Production" and "Risk". I'm impressed. I'd be nervous == scared, but ... WOW!
Untill now i've two remarks. I created a number of logical volumes of identical sizes, and formated some with brtfs and others with ext4.
Two years ago (2011) I tried that. Regular readers will recall that I'm keen on using LVM. Putting a single BtrFS in a single LVM partition doesn't really give this a lot of scope to play with. Well OK, it if had been all of /usr with the variety of file sizes and the plethora of dinky files like icons to practice 'packing' with then maybe.
The netto available disk space for btrfs systems is slightly smaller than those with ext4.
I'm not sure a comparison with ext4 is fair. With ReiserFS perhaps. Think about it. I tried ext4 and ... OUCH! I had a mail archive on a ResierFS and moved it to a larger ext4 FS. it wouldn't fit, ran out of inodes. Yes I know, you can determine the inode ratio when you mkfs. But with ReiserFS the issue never comes up; inodes are dynamic. So you "go fish" - tuna. No! Wait! flex_bg and resize_inode have to be set by mkfs. You miss out on that and ... So what's wrong with ReiserFS? Not a lot; the main complaint seems to be that "its unsupported", as in "not being developed any more". But its a great file system. Perhaps I'm not an extremist but I've not had any problems with it. I resizes - up and down - smoothly; performs more than adequately. handles all sizes of files well. If anything, it seems to be an ancestor of BtrFS. Part of my exercise was to see if I could a) have grub boot directly into a non-ext file system b) do away with a separate primary partition for /boot The latter has become a bit of an albatross for me. Fixed in size there is a limit on how many kernels I can retroactively retain. My previous experiment was to have all LVM and boot in resizeable LVM. Well that worked too :-) As I keep telling people who have problems when they used fixed partitioning and run out of space, LVM is the way to go! To be honest, I suspect some of my Windows colleagues might snicker. I've long advocated the separation of "Code and Data", or "C:" and "D:". Have all the system and Windows in C: and the user Data in :D - the equivalent of /home. One got back to me when, after a series of Tuesday patches and installs of applications the C: partition had run out of space but there was still lots of space in D:. "If I hadn't done what you suggested I wouldn't be having this problem, If I'd done what MS suggests, the way then machines are delivered with C: being the whole disk, this wouldn't be happening". In effect having the BtrFS as the whole drive (this was an old Windows disk I overwrote) is like the single C: partition. Sort of. But I keep coming back to the logic of the algorithms of BtrFS for packing and indexing. An some of the future features like snapshots. As to available space, well vanilla BtrFS vs vanilla ext4 ... maybe Hans is right, but it might depend on many things, such as the parameters at mkfs time.
Also, when you completely fill the btrfs-file system, you get a nasty stack trace in syslog.... First time it scared the hell out of me. After a lvextend + btrfs filesystem resize max, blue sky again.
Now that scares me. Isn't there a utility that tells you when your FS is filling up? Oh, right, one of my selling points for partition was that a rogue process that eats up all of /home won't affect the system on /. Heck, even ext4 has a barrier that saves what, the last x% of the disk can only be used by root or members of a designated group.
Other thing is (but perhaps not related to btrfs), since i did a re-install with OS_12.2 i periodically find my system frozen solid. After restart absolutely no shred of evidence what might have caused it. (might very well be hw related though)
I've not had any problems with with 12.2 running on ext4/boot with LVM/ResiserFS so I'll be watching out for this. The hw is old. slow and came out of the closet of anxiety, that is machines that had been rejected with undiagnosed problems, so may well be flakey. We'll see how robust and tolerant 12.2+BtrFS is. -- "Only those with narrow minds fail to see that the definition of Impossible is 'Lack of imagination and incentive." -- Brian Herbert -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (9)
-
Adam Tauno Williams -
Anders Johansson -
Anton Aylward -
Cristian Rodríguez -
Dave Howorth -
Hans Witvliet -
Per Jessen -
Rodney Baker -
Ruediger Meier