[opensuse-kernel] up/dup mkinitrd inefficiency
I just did a zypper up on a 12.2 system last updated 29 Aug, with kernel-desktop locked. Package count was 142. Afterward I unlocked kernel-desktop, then did zypper dup. I monitored the up process, creating initrd backups every time new initrd creation appeared onscreen. The following is the result: -rw-r--r-- 1 root root 5207254 Jun 30 16:55 i0nitrd-3.4.2-1-desktop1 -rw-r--r-- 1 root root 5272345 Aug 29 11:56 i0nitrd-3.4.2-1-desktop2 -rw-r--r-- 1 root root 5275931 Oct 22 11:38 i0nitrd-3.4.2-1-desktop3 -rw-r--r-- 1 root root 5275985 Oct 22 11:42 i0nitrd-3.4.2-1-desktop4 -rw-r--r-- 1 root root 5275953 Oct 22 11:43 i0nitrd-3.4.2-1-desktop5 -rw-r--r-- 1 root root 5275956 Oct 22 11:50 i0nitrd-3.4.2-1-desktop6 -rw-r--r-- 1 root root 5272072 Aug 29 12:50 i0nitrd-3.4.6-1.1-desktop1 -rw-r--r-- 1 root root 5276071 Oct 22 11:38 i0nitrd-3.4.6-1.1-desktop2 -rw-r--r-- 1 root root 5276013 Oct 22 11:41 i0nitrd-3.4.6-1.1-desktop3 -rw-r--r-- 1 root root 5276025 Oct 22 11:44 i0nitrd-3.4.6-1.1-desktop4 -rw-r--r-- 1 root root 5276074 Oct 22 11:51 i0nitrd-3.4.6-1.1-desktop5 lrwxrwxrwx 1 root root 26 Oct 22 11:56 initrd -> initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5276261 Oct 22 11:56 initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5275956 Oct 22 11:50 initrd-3.4.2-1-desktop -rw-r--r-- 1 root root 5276074 Oct 22 11:51 initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 24 Aug 29 12:50 initrd-curr -> initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 22 Jun 30 16:55 initrd-prev -> initrd-3.4.2-1-desktop On slow systems this really extends the update process noticeably. Could it be streamlined so that initrds only get built a maximum of once per up/dup run? -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 10/22/2012 06:04 PM, Felix Miata wrote:
I just did a zypper up on a 12.2 system last updated 29 Aug, with kernel-desktop locked. Package count was 142. Afterward I unlocked kernel-desktop, then did zypper dup. I monitored the up process, creating initrd backups every time new initrd creation appeared onscreen. The following is the result:
-rw-r--r-- 1 root root 5207254 Jun 30 16:55 i0nitrd-3.4.2-1-desktop1 -rw-r--r-- 1 root root 5272345 Aug 29 11:56 i0nitrd-3.4.2-1-desktop2 -rw-r--r-- 1 root root 5275931 Oct 22 11:38 i0nitrd-3.4.2-1-desktop3 -rw-r--r-- 1 root root 5275985 Oct 22 11:42 i0nitrd-3.4.2-1-desktop4 -rw-r--r-- 1 root root 5275953 Oct 22 11:43 i0nitrd-3.4.2-1-desktop5 -rw-r--r-- 1 root root 5275956 Oct 22 11:50 i0nitrd-3.4.2-1-desktop6 -rw-r--r-- 1 root root 5272072 Aug 29 12:50 i0nitrd-3.4.6-1.1-desktop1 -rw-r--r-- 1 root root 5276071 Oct 22 11:38 i0nitrd-3.4.6-1.1-desktop2 -rw-r--r-- 1 root root 5276013 Oct 22 11:41 i0nitrd-3.4.6-1.1-desktop3 -rw-r--r-- 1 root root 5276025 Oct 22 11:44 i0nitrd-3.4.6-1.1-desktop4 -rw-r--r-- 1 root root 5276074 Oct 22 11:51 i0nitrd-3.4.6-1.1-desktop5 lrwxrwxrwx 1 root root 26 Oct 22 11:56 initrd -> initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5276261 Oct 22 11:56 initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5275956 Oct 22 11:50 initrd-3.4.2-1-desktop -rw-r--r-- 1 root root 5276074 Oct 22 11:51 initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 24 Aug 29 12:50 initrd-curr -> initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 22 Jun 30 16:55 initrd-prev -> initrd-3.4.2-1-desktop
On slow systems this really extends the update process noticeably. Could it be streamlined so that initrds only get built a maximum of once per up/dup run?
We might need transaction support and merging of those in rpm. But let's first figure out *why* those are updated. Could you check your /var/log/zypp logfiles to see which packages triggered those? Perhaps some of the triggers could be changed. Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 2012-10-22 21:19 (GMT+0200) Andreas Jaeger composed:
Felix Miata wrote:
I just did a zypper up on a 12.2 system last updated 29 Aug, with kernel-desktop locked. Package count was 142. Afterward I unlocked kernel-desktop, then did zypper dup. I monitored the up process, creating initrd backups every time new initrd creation appeared onscreen. The following is the result:
-rw-r--r-- 1 root root 5207254 Jun 30 16:55 i0nitrd-3.4.2-1-desktop1 -rw-r--r-- 1 root root 5272345 Aug 29 11:56 i0nitrd-3.4.2-1-desktop2 -rw-r--r-- 1 root root 5275931 Oct 22 11:38 i0nitrd-3.4.2-1-desktop3 -rw-r--r-- 1 root root 5275985 Oct 22 11:42 i0nitrd-3.4.2-1-desktop4 -rw-r--r-- 1 root root 5275953 Oct 22 11:43 i0nitrd-3.4.2-1-desktop5 -rw-r--r-- 1 root root 5275956 Oct 22 11:50 i0nitrd-3.4.2-1-desktop6 -rw-r--r-- 1 root root 5272072 Aug 29 12:50 i0nitrd-3.4.6-1.1-desktop1 -rw-r--r-- 1 root root 5276071 Oct 22 11:38 i0nitrd-3.4.6-1.1-desktop2 -rw-r--r-- 1 root root 5276013 Oct 22 11:41 i0nitrd-3.4.6-1.1-desktop3 -rw-r--r-- 1 root root 5276025 Oct 22 11:44 i0nitrd-3.4.6-1.1-desktop4 -rw-r--r-- 1 root root 5276074 Oct 22 11:51 i0nitrd-3.4.6-1.1-desktop5 lrwxrwxrwx 1 root root 26 Oct 22 11:56 initrd -> initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5276261 Oct 22 11:56 initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5275956 Oct 22 11:50 initrd-3.4.2-1-desktop -rw-r--r-- 1 root root 5276074 Oct 22 11:51 initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 24 Aug 29 12:50 initrd-curr -> initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 22 Jun 30 16:55 initrd-prev -> initrd-3.4.2-1-desktop
On slow systems this really extends the update process noticeably. Could it be streamlined so that initrds only get built a maximum of once per up/dup run?
We might need transaction support and merging of those in rpm. But let's first figure out *why* those are updated. Could you check your /var/log/zypp logfiles to see which packages triggered those? Perhaps some of the triggers could be changed.
If I'm reading http://fm.no-ip.com/Tmp/SUSE/zypphistory-s122-d5000.txt correctly: # 2012-10-22 11:36:15 mkinitrd-2.7.1-62.6.1.i586.rpm installed ok # 2012-10-22 11:38:38 udev-182-4.21.1.i586.rpm installed ok # 2012-10-22 11:41:12 mdadm-3.2.5-3.17.1.i586.rpm installed ok # 2012-10-22 11:44:05 lvm2-2.02.84-26.8.2.i586.rpm installed ok # 2012-10-22 11:51:08 multipath-tools-0.4.9-3.6.1.i586.rpm installed ok # 2012-10-22 11:57:00 kernel-desktop-3.4.11-2.16.1.i686.rpm installed ok Looks like I missed backing up the first that followed mkinitrd. Seems like if mkinitrd is in the list, all that use it should have theirs skipped, and mkinitrd's scripts, if not the whole package, deferred until last. Note that all partitions on this single HD (laptop) system are traditional types, and natives are all swap, EXT2 or EXT3. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 10/22/2012 03:18 PM, Felix Miata wrote:
On 2012-10-22 21:19 (GMT+0200) Andreas Jaeger composed:
Felix Miata wrote:
I just did a zypper up on a 12.2 system last updated 29 Aug, with kernel-desktop locked. Package count was 142. Afterward I unlocked kernel-desktop, then did zypper dup. I monitored the up process, creating initrd backups every time new initrd creation appeared onscreen. The following is the result:
-rw-r--r-- 1 root root 5207254 Jun 30 16:55 i0nitrd-3.4.2-1-desktop1 -rw-r--r-- 1 root root 5272345 Aug 29 11:56 i0nitrd-3.4.2-1-desktop2 -rw-r--r-- 1 root root 5275931 Oct 22 11:38 i0nitrd-3.4.2-1-desktop3 -rw-r--r-- 1 root root 5275985 Oct 22 11:42 i0nitrd-3.4.2-1-desktop4 -rw-r--r-- 1 root root 5275953 Oct 22 11:43 i0nitrd-3.4.2-1-desktop5 -rw-r--r-- 1 root root 5275956 Oct 22 11:50 i0nitrd-3.4.2-1-desktop6 -rw-r--r-- 1 root root 5272072 Aug 29 12:50 i0nitrd-3.4.6-1.1-desktop1 -rw-r--r-- 1 root root 5276071 Oct 22 11:38 i0nitrd-3.4.6-1.1-desktop2 -rw-r--r-- 1 root root 5276013 Oct 22 11:41 i0nitrd-3.4.6-1.1-desktop3 -rw-r--r-- 1 root root 5276025 Oct 22 11:44 i0nitrd-3.4.6-1.1-desktop4 -rw-r--r-- 1 root root 5276074 Oct 22 11:51 i0nitrd-3.4.6-1.1-desktop5 lrwxrwxrwx 1 root root 26 Oct 22 11:56 initrd -> initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5276261 Oct 22 11:56 initrd-3.4.11-2.16-desktop -rw-r--r-- 1 root root 5275956 Oct 22 11:50 initrd-3.4.2-1-desktop -rw-r--r-- 1 root root 5276074 Oct 22 11:51 initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 24 Aug 29 12:50 initrd-curr -> initrd-3.4.6-1.1-desktop lrwxrwxrwx 1 root root 22 Jun 30 16:55 initrd-prev -> initrd-3.4.2-1-desktop
On slow systems this really extends the update process noticeably. Could it be streamlined so that initrds only get built a maximum of once per up/dup run?
We might need transaction support and merging of those in rpm. But let's first figure out *why* those are updated. Could you check your /var/log/zypp logfiles to see which packages triggered those? Perhaps some of the triggers could be changed.
If I'm reading http://fm.no-ip.com/Tmp/SUSE/zypphistory-s122-d5000.txt correctly:
# 2012-10-22 11:36:15 mkinitrd-2.7.1-62.6.1.i586.rpm installed ok # 2012-10-22 11:38:38 udev-182-4.21.1.i586.rpm installed ok # 2012-10-22 11:41:12 mdadm-3.2.5-3.17.1.i586.rpm installed ok # 2012-10-22 11:44:05 lvm2-2.02.84-26.8.2.i586.rpm installed ok # 2012-10-22 11:51:08 multipath-tools-0.4.9-3.6.1.i586.rpm installed ok
# 2012-10-22 11:57:00 kernel-desktop-3.4.11-2.16.1.i686.rpm installed ok
Looks like I missed backing up the first that followed mkinitrd. Seems like if mkinitrd is in the list, all that use it should have theirs skipped, and mkinitrd's scripts, if not the whole package, deferred until last.
Note that all partitions on this single HD (laptop) system are traditional types, and natives are all swap, EXT2 or EXT3.
I usually have a number of kernels in directory /boot. At present, there are 28 such entries. Under those conditions, the mkinitrd step takes about an hour to run. I did an update with apper last night, and it took about 3 hours to install the new packages. My suggestion is that a special flag be made available, and that the flag be set by any package that needs new initrd files. When all the packages have been installed, then call mkinitrd if the flag is set. Larry -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Mon, Oct 22, 2012 at 04:40:23PM -0500, Larry Finger wrote:
I usually have a number of kernels in directory /boot. At present, there are 28 such entries. Under those conditions, the mkinitrd step takes about an hour to run. I did an update with apper last night, and it took about 3 hours to install the new packages.
My suggestion is that a special flag be made available, and that the flag be set by any package that needs new initrd files. When all the packages have been installed, then call mkinitrd if the flag is set.
I've seen this a lot as well. But why isn't anyone trying to solve the root problem here, why does mkinitrd take so long? It didn't used to do this for 12.1, what changed to cause it to suck up CPU time? thanks, greg k-h -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Greg KH wrote:
On Mon, Oct 22, 2012 at 04:40:23PM -0500, Larry Finger wrote:
My suggestion is that a special flag be made available, and that the flag be set by any package that needs new initrd files. When all the packages have been installed, then call mkinitrd if the flag is set.
I've seen this a lot as well. But why isn't anyone trying to solve the root problem here, why does mkinitrd take so long? It didn't used to do this for 12.1, what changed to cause it to suck up CPU time?
It's not a case of a 'root' problem, though, is it? What you're saying is that there are two separate, independent problems: (1) mkinitrd is being called repeatedly when it only needs to be called once. (2) mkinitrd has had a performance regression between 12.1 and 12.2 Larry's suggestion seems like a good fix for problem (1) that could be coded now. Problem (2) needs somebody to verify that there is a significant difference between 12.1 and 12.2 and then identify the cause either by code inspection or bisecting commits. Then somebody needs to see whether there is a reasonable fix. Maybe a bugzilla would start the process. So it seems like (2) has a significantly greater time to repair than (1), which would be another reason to fix (1) now. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 2012-10-23 11:01 (GMT+0100) Dave Howorth composed:
(1) mkinitrd is being called repeatedly when it only needs to be called once.
I filed https://bugzilla.novell.com/show_bug.cgi?id=786318 for this.
(2) mkinitrd has had a performance regression between 12.1 and 12.2
Note the attachment to the bug is from a very old 600MHz Piii laptop running 12.2, but it's nevertheless indicative of time spent running mkinitrd each time. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Monday 22 of October 2012 16:40EN, Larry Finger wrote:
My suggestion is that a special flag be made available, and that the flag be set by any package that needs new initrd files. When all the packages have been installed, then call mkinitrd if the flag is set.
IMHO the solution should be more general as similar problem concerns other post-install actions as well. For instance, rebuilding kpathsea cache after updating almost any TeX package became extremely annoying since texlive was separated into a lot of small packages. And I guess more examples of the same problem could be found. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed, Oct 24, 2012 at 08:06:32AM +0200, Michal Kube??ek wrote:
On Monday 22 of October 2012 16:40EN, Larry Finger wrote:
My suggestion is that a special flag be made available, and that the flag be set by any package that needs new initrd files. When all the packages have been installed, then call mkinitrd if the flag is set.
IMHO the solution should be more general as similar problem concerns other post-install actions as well. For instance, rebuilding kpathsea cache after updating almost any TeX package became extremely annoying since texlive was separated into a lot of small packages. And I guess more examples of the same problem could be found.
Anothe one is gtk-update-icon-cache. Many packages call it as post-install action, and on slow systems it takes some time to complete. Many times I have seen an upgrade that could be completed in a couple of minutes take more than ten just because half of the rpms updated are calling gtk-update-icon-cache. Giacomo -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
participants (7)
-
Andreas Jaeger -
Dave Howorth -
Felix Miata -
Giacomo Comes -
Greg KH -
Larry Finger -
Michal Kubeček