-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday, 2013-06-25 at 21:13 +0400, Andrey Borzenkov wrote:

В Tue, 25 Jun 2013 13:02:23 -0400 Felix Miata пишет:

...

On 2013-06-25 18:12 (GMT+0200) Stephan Kulow composed:

...

It looks like I just broke my laptop by updating to factory, I can't get past the initrd with something that sounds like broken storage setup in systemd/mkinitrd/udev. So I advise everyone not to update before I found out it was my fault :)

This is the other thing that routinely annoys me about working initrds getting rebuilt every time any package that ever affects what makes up an initrd gets an update[1]. Why shouldn't working initrds be left alone, and new packages get put into them only either when a new kernel is installed, mkinitrd is called by a user, or an affirmative answer is given to a request to rebuild by an updated package installation? The way initrds get clobbered now, what point is there to enabling multiversion for kernels?

I dare to guess that the reason is to ensure that binaries in initrd match binaries on your system.

But it is an interesting point. If we have multiversion, but initrd is remade for both old and current versions, a failure breaks all kernel versions. We can not be sure that the old version will keep working as a failsafe. - -- Cheers, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlHKV0YACgkQtTMYHG2NR9U55ACfcYrVRXJgx1CRhcjmjmXn0iWz sqwAniLY+jN+l8EJ+80XB3NXx4b7aTpW =xTlT -----END PGP SIGNATURE-----

On 2013-06-26 09:43 (GMT+0200) Stephan Kulow composed:

...

...

Felix Miata composed:

...

...

...

the other thing that routinely annoys me about working initrds getting rebuilt every time any package that ever affects what makes up an initrd gets an update[1]. Why shouldn't working initrds be left alone, and new packages get put into them only either when a new kernel is installed, mkinitrd is called by a user, or an affirmative answer is given to a request to rebuild by an updated package installation? The way initrds get clobbered now, what point is there to enabling multiversion for kernels?

It would have saved my laptop, but then again people whos system is fixed by an update would love to have the initrd updated on updates - so where shall we stop?

Answered at least partially by the original thread fork you quoted. As yet no one has answered the second question in the fork. What is the point of enabling multiversion for kernels if initrds are always clobbered?

I guess we should leave old kernels with old initrds, but I don't think we should keep an old initrd for the new kernel.

Where would an old initrd for "the new kernel" come from? New kernels of necessity get new initrds. Did you mean to write "newest kernel"? That may be a good idea as long as the newest kernel is not also the only kernel. Another idea would be a user preference option in /etc/sysconfig/kernel similar to the multiversion option in zypp.conf, or in zypp.conf itself, stipulating whether to ever rebuild initrds or how many kernels' initrds to rebuild at updates time. It might be better to have it do all or all except one by default in GA installations, but in milestone, beta and maybe RC installations to leave all except newest kernel's initrd untouched by updating. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Tue, Jun 25, 2013 at 11:51 PM, Carlos E. R. wrote:

On Tuesday, 2013-06-25 at 21:13 +0400, Andrey Borzenkov wrote:

...

В Tue, 25 Jun 2013 13:02:23 -0400 Felix Miata пишет:

...

On 2013-06-25 18:12 (GMT+0200) Stephan Kulow composed:

...

It looks like I just broke my laptop by updating to factory, I can't get past the initrd with something that sounds like broken storage setup in systemd/mkinitrd/udev. So I advise everyone not to update before I found out it was my fault :)

This is the other thing that routinely annoys me about working initrds getting rebuilt every time any package that ever affects what makes up an initrd gets an update[1]. Why shouldn't working initrds be left alone, and new packages get put into them only either when a new kernel is installed, mkinitrd is called by a user, or an affirmative answer is given to a request to rebuild by an updated package installation? The way initrds get clobbered now, what point is there to enabling multiversion for kernels?

I dare to guess that the reason is to ensure that binaries in initrd match binaries on your system.

But it is an interesting point.

If we have multiversion, but initrd is remade for both old and current versions, a failure breaks all kernel versions. We can not be sure that the old version will keep working as a failsafe.

Indeed, but even if you leave it untouched, an old initrd with a newer root could also break in interesting ways. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 26.06.2013 11:32, Felix Miata wrote:

On 2013-06-25 21:13 (GMT+0400) Andrey Borzenkov composed:

...

Felix Miata composed:

...

...

This is the other thing that routinely annoys me about working initrds getting rebuilt every time any package that ever affects what makes up an initrd gets an update[1]. Why shouldn't working initrds be left alone, and new packages get put into them only either when a new kernel is installed, mkinitrd is called by a user, or an affirmative answer is given to a request to rebuild by an updated package installation? The way initrds get clobbered now, what point is there to enabling multiversion for kernels?

...

I dare to guess that the reason is to ensure that binaries in initrd match binaries on your system.

Your reply seems to answer only the first of the two separate questions.

Is it better to have binaries match in a clobbering initrd that won't boot the system? I think not.

And how do you know forehand if the new binaries break or fix the system? Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Wed, Jun 26, 2013 at 10:13 AM, Felix Miata wrote:

On 2013-06-26 12:11 (GMT+0200) Stephan Kulow composed:

...

...

...

...

The way initrds get clobbered now, what point is there to enabling multiversion for kernels?

...

...

Is it better to have binaries match in a clobbering initrd that won't boot the system? I think not.

...

And how do you know forehand if the new binaries break or fix the system?

Again, this is about behavior when multiversion is enabled for kernels.

What ain't broke don't need fixin.

It may need fixing. Think new glibc that needs new kernel. Now you update the kernel, but not the initrd's libc, kaboom. I'm not that knowledgeable about libc, so maybe this particular case isn't an issue. But the idea is that binaries and kernel may need to match, and if you don't update the initrd, the system could also break. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

I too suffered from this and it was hell to debug since each attempt trashed all existing initreads due to totally dumb behaviour by mkinitrd, ie rebuild all initreads not just rebuild those you say, or need it (ie vmlinuz exists but NO matching initrd). I attach a perl script to build a git tag and only its initrd. Linus is right, there are far too many idiots, in distributions and desktops eg KDE BREAKING things, and far too little TESTING and DOCUMENTING. I have just tossed Kmail 2 for its crappy, buggey behavior and am thinking about tossing KDE and SuSE for the same ... Fixate on NEW things, break old things stupidity. As a 15 year SuSE and 25 year Solaris user this will pain me but I am not up for a week in hell with each new Gold Master. MFG,brian

Am Wed, 26 Jun 2013 14:42:36 -0400, scheib Felix Miata:

On 2013-06-26 13:46 (GMT-0400) Claudio Freire composed:

...

...

...

...

...

> The way initrds get clobbered > now, what point is there to enabling multiversion for kernels?

...

...

...

...

Is it better to have binaries match in a clobbering initrd that won't boot the system? I think not.

...

...

...

And how do you know forehand if the new binaries break or fix the system?

...

...

Again, this is about behavior when multiversion is enabled for kernels.

...

...

What ain't broke don't need fixin.

...

It may need fixing. Think new glibc that needs new kernel.

Exactly: NEW KERNEL needs new this or that or vice versa!!! New kernel is not the subject of the thread. The thread asks why unconditionally clobber old kernel's initrd installed with last updates and older kernel's initrd installed with updates before that, etc, etc: IOW, why clobber initrds that worked for not new kernels, so that when something bad happens in newest kernel and/or initrd and/or mkinitrd that kernel and matching initrd that worked in the past stands a good chance of getting a successful boot so that whatever broke can be fixed without need for a rescue boot from some other media.

Again, what purpose does multiversion for kernel serve with this current practice of clobbering *all* initrds *every* time *anything* affecting initrd makeup gets changed??? What?

...

Now you update the kernel, but not the initrd's libc, kaboom.

Why is everyone responding to this thread ignoring the keywords?:

RE-build

Existing

Multiversion

Multiple

Every

...

I'm not that knowledgeable about libc, so maybe this particular case isn't an issue. But the idea is that binaries and kernel may need to match,

Maybe they do, maybe they don't. I've plenty of times saved kernel, initrd and modules at version upgrade time and found they continue to be functional and highly preferable to rescue booting after several subsequent iterations of kernel version upgrades.

...

and if you don't update the initrd, the system could also break.

As original thread starter wrote, updating broke the system, and presumably since multiversion for kernels is enabled by default, in the process broke the previously installed kernel/initrd pair(s) that worked in the past, and had they not been, would have been able to be used to correct the new failure that was also applied to destroy what had been working.

Again, what is the point of by default enabling of multiversion for kernel if not to function as a better-than-nothing proven fallback kernel/initrd to be used in the event that the newest kernel/initrd pair does not work? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation)

Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-- Greetings (mit freundlichen Grüßen), Brian. Dr. Brian O'Mahoney Email: omb@teraflex.ch Phone: +44 (0) 7711 489965 GPG Key ID: E4A3BCF8 2009-12-11, old PGP Key Id: 0xA0481D676FBC700C

On 2013-06-26 20:09 (GMT-0300) Claudio Freire composed:

...

...

Now you update the kernel, but not the initrd's libc, kaboom.

...

Why is everyone responding to this thread ignoring the keywords?:

...

RE-build

...

Existing

...

Multiversion

...

Multiple

You should re-read my post.

New kernel get new initrd (or not, not my point).

OLD kernel with OLD initrd goes kaboom, because the filesystem does have the new glibc.

Never in my recollected experience has that happened. Unlike most people, I'm an OFM user, and with few exceptions retune my Grub menu after each installation or update process that changes it or the content of /boot. In similar manner I commonly backup initrds before updating, and on arbitrary occasions boot using the initrd created when the particular older kernel selected to test was originally installed, not one more recently created, for the express purpose of verifying that both older kernel and initrd do still get the system up and usable. More than once I've done a, urpmi --auto-select, yum upgrade, zypper dup or "clean"[1] installation to acquire a major version change only to find the new kernel had a broken initrd or no initrd at all, yet was able to use a kernel, initrd and /lib/modules set from outside the package management system to get it up, diagnosed, and often, repaired. I much prefer troubleshooting and repairing an installation failure using the installed system itself whenever possible instead of rescue media and chroot. This is why I'm routinely annoyed by initrd clobber, and find multiversioning of kernel so nearly worthless. [1] Technically this process is often unclean. What I often do is manually delete most binaries and their directories, leaving backups of selected config files in place for easy recommissioning and/or reference. My selection not unusually includes also the newest kernel, initrd, and its /lib/modules set. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 06/26/2013 09:18 PM, root pecked at the keyboard and wrote:

OK, you are trying to mitigate the systems not fix the problem which is 'mkinitrd' created by SuSE... so (a) it should NOT remake all initrds except with a --force-all option, (b) it should only when needed, (c) it should have an simple cl option to specify which to make, just ONE,

about 20 lines of bash script ...

MFG, Brian

Why not just use the parameter that is available in mkinitrd: -k kernel list List of kernel images for which initrd files are created. Defaults to all kernels found in /boot. and specify thr new kernel during the install phase? -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Claudio Freire writes:

It does matter, because, you'd boot, but you wouldn't get init going after leaving the initrd. So it's a half-boot.

And how does clobbering the initrd with a non-working glibc improve that situation? Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Andrey Borzenkov writes:

On Thu, Jun 27, 2013 at 12:03 PM, Andreas Schwab wrote:

...

Claudio Freire writes:

...

It does matter, because, you'd boot, but you wouldn't get init going after leaving the initrd. So it's a half-boot.

And how does clobbering the initrd with a non-working glibc improve that situation?

You get potential problem both ways.

But without rebuilding initrds the potential problems are significantly smaller. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am 27.06.2013 01:09, schrieb Claudio Freire:

New kernel get new initrd (or not, not my point).

OLD kernel with OLD initrd goes kaboom, because the filesystem does have the new glibc.

This does not happen in practice. -- Stefan Seyfried "If your lighter runs out of fluid or flint and stops making fire, and you can't be bothered to figure out about lighter fluid or flint, that is not Zippo's fault." -- bkw -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Claudio Freire wrote:

New kernel get new initrd (or not, not my point).

OLD kernel with OLD initrd goes kaboom, because the filesystem does have the new glibc. You don't keep multiversion snapshots of your filesystems to boot from, I imagine. If you do, please send me a link or two on how you set that up.

Do ALL kernels need a separate initrd? Doesn't module versioning work at all? Or, if not so well, how about 1 initrd with multiple lib/modules dirs for multiple kernel versions? Filesystem doesn't use/need libc, it needs kernel drivers. So what's this about needing different libc's for different filesystems to boot from? Don't hate me -- but why not 1 libc -- store it on disk and use it? You could use the kernel to pull libc and support utilties off of disk on demand as the system boots? Only thing needed on initrd would be kernel modules...? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt wrote:

1. Kernel drivers need modprobe to be loaded. 2. modprobe needs libc to run.

...

Don't hate me -- but why not 1 libc -- store it on disk and use it?

3. libc needs to be present alongside modprobe (=inside same initramfs image), because your kernel does not yet have the disk driver loaded to potentially use a common on-disk libc.

---- For initrd, why does 'modprobe' need to depend on libc? (i.e. not full featured modprobe on disk, but a simple, load-only version?) Couldn't it just use system calls and only provide module loading? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt wrote:

...

For initrd, why does 'modprobe' need to depend on libc? (i.e. not full featured modprobe on disk, but a simple, load-only version?)

Couldn't it just use system calls and only provide module loading?

I want to see you maintain such a monster before painting it easy.

Actually, I'd probably use a mini libc that allows tiny static binaries. http://www.musl-libc.org/how.html http://www.etalabs.net/compare_libcs.html http://www.musl-libc.org/download.html Latest version April 14, 2013... seems to go back about 2 years... The lib is configurable and providing a smaller subset of modprobe that doesn't support removing or anything other than what is needed for startup, seems like it would be a much smaller headache than the current set of problems building a large initrd. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Cristian RodrC-guez wrote:

No, it is not just kernel modules.. in current incarnations with systemd/dracut the initrd is also used for clean shutdown, not just for system startup.

dracut wouldn't be needed for an initrd that has only kernel modules would it? What's on initrd that systemd would need that isn't on the hard disk? Er... wait a sec, you mean initrd isn't deallocated after boot? ? Do you mean to say that the large initrd's that people are complaining about stay around in memory AFTER boot just so systemd can use it to shut down? If the systemd project recommends getting rid of initd for efficiency & speed, I'm sure it wouldn't be too difficult for us to do so. FWIW... (from the do-acracy of linda): the static libc from musl is 2M 2.0M /usr/local/musl/lib/libc.a 4.0K /usr/local/musl/lib/libcrypt.a 588K /usr/local/musl/lib/libc.so* modprobe (the full prog, not cut down) in dynamic state has a size of: 60K modprobe* --- It loads modules the same as the full libc version. Right now it is still dynamically linked with the 'musl' libc -- but that's IT! It doesn't need anything else to run and load modules!... As soon as I put together a static build, we could have everything we need to boot besides the kernel and modules, taking < 5MB, less if we squeeze it.

objdump -p modprobe

modprobe: file format elf64-x86-64 .... Dynamic Section: NEEDED libc.so --- But that's it. And it looks like static linking shouldn't be a problem. (this was in hour or so's worth of work tonight --- building musl libc takes <20 seconds and it built, out-of-the-box w/no special configuration. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

dieter wrote:

On Fri, 28 Jun 2013 01:35:10 -0700 Linda Walsh wrote:

...

Cristian RodrC-guez wrote:

...

No, it is not just kernel modules.. in current incarnations with systemd/dracut the initrd is also used for clean shutdown, not just for system startup.

---

dracut wouldn't be needed for an initrd that has only kernel modules would it? What's on initrd that systemd would need that isn't on the hard disk?

Er... wait a sec, you mean initrd isn't deallocated after boot? ?

Do you mean to say that the large initrd's that people are complaining about stay around in memory AFTER boot just so systemd can use it to shut down? I fully agree that small initrds are desirable and I would also prefer if it does not contain parts which are not really needed. But on the other hand for "modern PCs" I would think it is not really an issue. Even if the initrd is kept during the uptime, in case the memory would be needed it would be paged out, I assume without noticable impact on the performance.

...

If the systemd project recommends getting rid of initd for efficiency & speed, I'm sure it wouldn't be too difficult for us to do so. In my understanding there are significantly different use cases, especially small/mobile/embedded devices vs. "PCs". For the first it is known quite precisely in advance which HW can be included and therefore it is "easier" to build a monolythic kernel - and my interpretation of the systemd configuration without initrd aims at these.

For the latter there is a huge range of possible configurations, and therefore it is "reasonable" to use the initrd.

How is a PC not small, mobile or embedded compared to a mainframe, or, at the other end, compared to a cellphone? It's all relative. However, to any individual PC owner, it can be known (for those who wish to know) precisely, in advance what HW is our computers. A distro manufacturer may not know every device out there, but that's why they created scripts to create initrd disks that are tailored to users' individual computers, right? How would configuring a kernel via a similar script be any different than the current situation where it takes 10-20 minutes to generate all of the initrd scripts on my machine when SW is updated, vs. about 3-4 minutes to generate a kernel? Why not go for the lighter-weight solution and only generate the parts that need to be different for users -- their boot kernel. The rest can be read in as soon as the kernel is up. The less we put on initrd's the better. Surprise initrd-generations are a real pain -- they take 10-20 minutes vs. about 3.6 minutes to build a new kernel. So how do users benefit by having to wait so long when they upgrade SW on their system? Supposedly the benefit of booting from a ram disk was users didn't have to wait for a kernel to be recompiled... but 3.6 minutes vs. 10-20? Um... -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Andrey Borzenkov wrote:

On Fri, Jun 28, 2013 at 1:54 PM, Linda Walsh wrote:

...

How would configuring a kernel via a similar script be any different than the current situation

By forcing full development environment on every system? By making it much harder for support to analyze kernel crash dump from customer system?

Perhaps, but not necessarily. I don't know how if linux has advanced to the features of 10-20 year old unix systems or not, but dedicated kernel module linker should be able to take pre-built modules and link them into an image that could be booted from instead of copying them into a separate file-image that the kernel uses it's own internal binary images to access indirectly. It would seem that, at **worst**, a simplistic 'non-filesystem' that marks where modules start and have been appended to a kernel would be at least as "doable" as what the kernel currently does. After all, it is reading in a contiguous set of blocks, to some place in memory, then it goes through *comparatively* complicated file-access drivers to access that memory "like a real disk". However, don't forget -- it's not a real disk you've simply gained complexity and the need to read it from a separate place on disk rather than reading the kernel and the modules in 1 large read. Those modules could be compiled at a factory if you wanted to distribute a fixed binary -- but kernel people are used to analyzing kernel crash dumps all the time and don't used 'fixed-at-a-factory' crash dump images. I've never had a vendor yet who wanted a kernel crash dump from a production kernel -- they always wanted me to reproduce the problem with soem 'debug' kernel that has the symbols turned on. In that "most primitive case", your development tools involve "cat" or "dd". Now, a step above that would be some type of real linker that would cut the need for some relocation steps at boot time maybe something like "prelink" -- again, not a complicated development tool -- and again, a set of modules to prelink with -- just like you can prelink binaries on your system with a factory-glibc, can be distributed. Again... all binary-from-factory images if that is a requirement. Beyond that you can have those users who really want to rebuild their kernel (that is why we provide "src" rpms, no? They can turn on full debugging and optimize it for their HW. It might not be a debug kernel, but it might give them better performance than a generic. All of those are benefits of going with booting from a single kernel image and not going through the overhead of emulating a file system. Unix vendors had tools that allowed such prelinking and optimization for the many customers who wanted to get the most from their hardware -- 20 years ago. It doesn't seem like linux should be "incapable" of doing the same or better. Instead we are copying an oversized floppy image into ram to boot from and going through virtual fakery to access that information like a file system -- which adds complexity and overhead to the boot process -- nto that it takes ALOT of time, but you could likely read the whole thing in as one large memory image if you really wanted to speed things up... a serial read off of contiguous blocks on my HW can go up to 1GB/s. But do it randomly it can drop below 100MB/s. So for the fastest boot possible, 1 image with your needed drivers seems like the the fastest and least failure-prone option. How many times have people complained about something not booting due to the initfs? It's almost NEVER the kernel. The failure rate is at least 400% higher, overall with the current method (not on any given system that is already working, but over all instances of people installing new initrd's and upgrades. *Almost*, the only people that have problems booting from a kernel are kernel and driver developers. It's hard for average users to install a bad kernel -- there are far more checks on such, but initrd's fail frequently. At least think about the benefits because with the current system there is a visible cost already. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt wrote:

On Friday 2013-06-28 13:33, Linda Walsh wrote:

...

Perhaps, but not necessarily. I don't know how if linux has advanced to the features of 10-20 year old unix systems or not, but dedicated kernel module linker should be able to take pre-built modules and link them into an image that could be booted from instead of copying them into a separate file-image that the kernel uses it's own internal binary images to access indirectly.

What you describe is exactly the initramfs image. It may be a separate file or it may be piggybacked onto a kernel file. It does not make such a big difference because both parts will be loaded into memory by the bootloader one way or another and thereby are available to the kernel.

It does... If they are just kernel modules -- the kernel could treat them like load sections. Instead it's is a compressed disk image that needs a special kernel driver.

As usual, you talk big about "features of Unix", but apparently missed the fact that your casual Solaris initrd (well, their concept of it) is in the general excess of 100 MB - and that is without splash and ICU.

---- Solaris is a late comer. Go back to systems in the 80s and 90s if you want to see these techniques... Now.. people think space doesn't matter, so things get big and slow. Just like libc -- 2m for static progs w/libc for a trim version that works. Vs. larger sizes for shared modules with current libc.

The bootloader reads the entirety of the initramfs part, after which it is in memory and random access speed is not an issue.

Complexity is the issue -- needing special pseudo file drivers just to treat memory as a disk -- vs. just loading modules that can then run normal services from real disks.

...

How many times have people complained about something not booting due to the initfs?

Just Linda Walsh.

You are joking, I hope (either that or blind). --- 1. Who started this thread? (clobbering initrd's) Wasn't me. 2. Who started "/boot partition is too small" (due to initd?) -- not me. Those are this week with over 100 responses between them. 3. Last week "Non-booting system after zypper dup" -- crashing during initd execution. 4. back on 5/10 -- problem w/error messages in creating mkinitrd -- something that is done far more often than making a new kernel. 5. 6/21 -- lars M: mkinitrd of 12.2 has small defect == call for testing. 6. 6/19 -- 12.3 boot woes vanishing all choices except memtst... that wouldn't happen nearly as often if you didn't have to remake init rd all the time. 7. 6/11 -- running out of space on /boot -- need to purge old kernels... initrd taking alot of space. That's 6 in the last month with more before that -- NONE of those were me. initrd causes alot of problems that you seem to be missing. Sorry, you just proved my point. All the complaints about it in the past month have been alot of other people -- and you give me all the credit. Got Bias? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Friday 2013-06-28 10:35, Linda Walsh wrote:

Er... wait a sec, you mean initrd isn't deallocated after boot? ?

The cpio image is put into RAM by the bootloader, then gets extracted by the kernel on boot into a singleton instance "rootfs" (of type ramfs), after which the cpio image is freed, but of course the rootfs (cf. /proc/mounts) persists. Once the real disk is mounted, the rootfs is cleared of all files (the root directory persists, as rootfs cannot be unmounted) by way of run-init: /* * run-init.c * * Usage: exec run-init [-c /dev/console] /real-root /sbin/init "$@" * * This program should be called as the last thing in a shell script * acting as /init in an initramfs; it does the following: * * - Delete all files in the initramfs; * - Remounts /real-root onto the root filesystem; * - Chroots; * - Opens /dev/console; * - Spawns the specified init program (with arguments.) */ So either return-to-rootfs is simply not implemented on openSUSE, or systemd erects its own ramfs and goes _there_ rather than to the original (now-empty) rootfs. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Le vendredi 28 juin 2013 à 04:42 -0700, Linda Walsh a écrit :

Jan Engelhardt wrote:

...

On Friday 2013-06-28 10:35, Linda Walsh wrote:

...

Er... wait a sec, you mean initrd isn't deallocated after boot? ?

The cpio image is put into RAM by the bootloader, then gets extracted by the kernel on boot into a singleton instance "rootfs" (of type ramfs), after which the cpio image is freed, but of course the rootfs (cf. /proc/mounts) persists.

Once the real disk is mounted, the rootfs is cleared of all files (the root directory persists, as rootfs cannot be unmounted) by way of run-init:

So either return-to-rootfs is simply not implemented on openSUSE, or systemd erects its own ramfs and goes _there_ rather than to the original (now-empty) rootfs.

---

Interesting -- yet we are told systemd requires initfs to shut down the system cleanly. Hmmm... Something doesn't add up.

It doesn't require it. For some specific storage for rootfs (handled by daemon), the unmount should be handled by initramfs. -- Frederic Crozat SUSE -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

El 25/06/13 13:02, Felix Miata escribió:

On 2013-06-25 18:12 (GMT+0200) Stephan Kulow composed:

...

It looks like I just broke my laptop by updating to factory, I can't get past the initrd with something that sounds like broken storage setup in systemd/mkinitrd/udev. So I advise everyone not to update before I found out it was my fault :)

This is the other thing that routinely annoys me about working initrds getting rebuilt every time any package that ever affects what makes up an initrd gets an update[1].

The retarded part is the package manager workflow, it is has to run mkinitrd ONCE at the end of the transaction after all previous steps ended with success. Unfortunately there is no such thing as transaction,packages are installed one-by-one :-| -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

El 27/06/13 15:18, Bruno Friedmann escribió:

Is there no way to emulate it. Otherwise, could we try to use something like pigz which could be able to reduce by factor 4 the time needed to build the gzip initrd ...

Dracut already does that, I do not think we should continue hacking mkinitrd but to replace it with dracut as soon possible. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt wrote:

On Thursday 2013-06-27 21:18, Bruno Friedmann wrote:

...

Is there no way to emulate it. Otherwise, could we try to use something like pigz which could be able to reduce by factor 4 the time needed to build the gzip initrd ...

Speaking of compression types, since initrd size has been mentioned in the parallel thread of /boot to small, the use of xz could reduce the size by some 30%.

FWIW, using lzop can give boosts in speed as it cuts down the quantity read, but xz is heavy in CPU usage, and might slow things down on many systems. Unfortunately there is no one right answer, cuz, people with faster disks don't need as much compression, while people with faster cpu's vs. disks can easily make that trade. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Jan Engelhardt wrote:

On Friday 2013-06-28 03:58, Linda Walsh wrote:

...

...

Speaking of compression types, since initrd size has been mentioned in the parallel thread of /boot to small, the use of xz could reduce the size by some 30%.

---

FWIW, using lzop can give boosts in speed as it cuts down the quantity read, but xz is heavy in CPU usage, and might slow things down on many systems.

Yes, I too had LZO already in mind because of that. Is there a parallelizing implementation like there is for pigz/pbzip2/pixz/plzip?

It's usually so fast, I don't think one was written. You'd have to go to 100's of GB, to possibly see benefit, and even then, maybe not if you didn't have the memory BW. I eventually gave up using any type of compression on backups. The lowest compression/fastest was still well under 100MB/s reading off disc -- whereas raw files alone, top out at 1GB. Talking about this issue, I was thinking about my own boot -- and would probably benefit from no-compression ... only half a second, maybe, but I can see the pause. Even w/29 kernel images on my /boot, I allocated 1GB for boot, so plenty to spare. You might want to do some real-world time tests to see if compression is worth it. Many disks can hit near 100MB/s on contiguous reads. If compression slows that to 10-20MB/s, that's not great. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am 25.06.2013 19:31, schrieb Raymond Wooninck:

On Tuesday 25 June 2013 18:12:16 Stephan Kulow wrote:

...

Hi,

It looks like I just broke my laptop by updating to factory, I can't get past the initrd with something that sounds like broken storage setup in systemd/mkinitrd/udev. So I advise everyone not to update before I found out it was my fault :)

Greetings, Stephan

Hi Stephan,

Seems that OBS is on your side :-) The job to publish of Factory has been stuck for the last 5 days, so I guess that nobody can update.

But the FTP repo is broken as it is I'm afraid. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 25.06.2013 21:56, Raymond Wooninck wrote:

On Tuesday 25 June 2013 20:23:49 Stephan Kulow wrote:

...

...

Hi Stephan,

Seems that OBS is on your side :-) The job to publish of Factory has been stuck for the last 5 days, so I guess that nobody can update.

But the FTP repo is broken as it is I'm afraid.

Greetings, Stephan

Then I guess it is indeed the 61-msft.rules in /usr/lib/udev/rules.d I guess that systemd is complaining that it can't open the filesystems (indicating that it is running/waiting for the local filesystem job) and then it is not even possible to open an emergency shell as that also the root system is no longer recognizable.

Three ways to resolve it and all three requires a LiveCD to boot from

1) remove /usr/lib/udev/rules.d/61-msft.rules and rebuild the initrd. 2) Use the physical device for the filesystems in grub and fstab (/dev/sda1, etc) 3) Use UUID's instead of UID's.

I guess that Robert Milasan did something strange with UID's in that particular 61-msft.rules

I managed to repair my laptop by downgrading udev and libudev1 to factory-snapshot. Thanks for looking into this. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 06/25/2013 12:09 PM, Cristian Rodríguez wrote:

El 25/06/13 12:12, Stephan Kulow escribió:

...

Hi,

It looks like I just broke my laptop by updating to factory, I can't get past the initrd with something that sounds like broken storage setup in systemd/mkinitrd/udev. So I advise everyone not to update before I found out it was my fault :)

Or you are hitting the broken 61-msft.rules that were added to udev recently.

Thanks Christian. Removing that file allows my system to boot. With it present, there were no ata-*-part[0-9] links. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Wednesday 2013-06-26 05:16, Cristian Rodríguez wrote:

El 25/06/13 23:12, Tony Jones escribió:

...

With it present, there were no ata-*-part[0-9] links.

[61-msft.rules] is a recent SUSE-specific addition that references a bug that we cannot read.

The more reason that we should require that all patches MUST carry a *description*. patch-fix-opensuse is not sufficient at all. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Wed, Jun 26, 2013 at 11:39 AM, Robert Milasan wrote:

On Wed, 26 Jun 2013 06:08:46 +0200 (CEST) Jan Engelhardt wrote:

...

On Wednesday 2013-06-26 05:16, Cristian Rodríguez wrote:

...

El 25/06/13 23:12, Tony Jones escribió:

...

With it present, there were no ata-*-part[0-9] links.

[61-msft.rules] is a recent SUSE-specific addition that references a bug that we cannot read.

The more reason that we should require that all patches MUST carry a *description*. patch-fix-opensuse is not sufficient at all.

The rule was added to support Hyper-V SLES guest. Seems that Hyper-V presents both Type1 and Type3 identifiers for VHDs, but scsi_id would support only one of them (most probably Type3).

The rule ensures we have both links in /dev/disk/by-id, but don't get why would this be an issue in booting the system.

Because your rule overrides ID_BUS with "scsi" while it was "ata" before. So installer (which did not have this patch) set up system to use /dev/disks/by-id/ata-* and after your patch these links are no more available. This is going to break almost every system out there on update. As I already suggested, do not override ID_BUS, rather simply create explicit links /dev/disk/by-id/scsi-* with required identity information.

It is possible that the rule needs to be added to initrd with sg_inq requires.

I will check and will fix it asap.

-- Robert Milasan

L3 Support Engineer SUSE Linux (http://www.suse.com) email: rmilasan@suse.com GPG fingerprint: B6FE F4A8 0FA3 3040 3402 6FE7 2F64 167C 1909 6D1A -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

В Wed, 26 Jun 2013 12:55:55 -0400 Cristian Rodríguez пишет:

On 06/26/2013 05:09 AM, Andrey Borzenkov wrote:

...

As I already suggested, do not override ID_BUS, rather simply create explicit links /dev/disk/by-id/scsi-* with required identity information.

I would go further and say this rules do not belong to udev at all, some other hyperv specific package should contain them.

This is orthogonal to content of these rules :) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org