[opensuse-factory] what it takes to get factory out

older
[opensuse-factory] Factory Kernel...

Stephan Kulow

20 Jul 2009 20 Jul '09

12:48

Hi, Just wanted to share some log file checking with you. I wanted to know how the rebuild this weekend worked out, so I checked the chain upto the last built package and the last time it built (often packages get build twice due to cycles). As you can see, it takes roughly 24h from start of the rebuild cycle on friday afternoon to the middle of the GNOME stack and roughly another day from the middle of the GNOME stack to the end of Ooo. But note that the list only gives you a rough idea as it's only looking at i586, but both architectures interact with each other, e.g. there is a 5 hour gap between gnome-vfs2 and libgnome. This is not the build time of libgnome, but the fact that x86_64 was a bit slower and finished libbonobui.2.10 only at 8pm and caused i586 to build 2.10 too - while i586's version was unchanged to 2.9, so it discarded it. But on x86_64 the value of %_libexecdir changed, so libbonoboui files changed. In case you're still reading, don't worry :) Greetings, Stephan gcc Fri Jul 17 17:17:04 2009 bison Sat Jul 18 01:53:55 2009 automake Fri Jul 17 17:18:38 2009 perl-gettext Sat Jul 18 06:38:12 2009 help2man Fri Jul 17 20:58:36 2009 autoconf Fri Jul 17 17:30:01 2009 libsepol Sat Jul 18 06:28:45 2009 glibc Fri Jul 17 18:19:15 2009 pcre Fri Jul 17 19:47:56 2009 libdb-4_5 Sat Jul 18 06:58:01 2009 perl Fri Jul 17 20:11:57 2009 perl-libintl-perl Sat Jul 18 06:52:16 2009 gettext-runtime Sat Jul 18 02:04:05 2009 gmp Sat Jul 18 02:09:27 2009 gcc44 Sat Jul 18 02:23:48 2009 gpm Sat Jul 18 06:50:28 2009 ncurses Sat Jul 18 03:48:09 2009 ppl Sat Jul 18 04:11:26 2009 cloog Sat Jul 18 04:50:06 2009 bzip2 Sat Jul 18 04:50:22 2009 tcl Sat Jul 18 06:28:48 2009 binutils Sat Jul 18 04:56:17 2009 libtool Sat Jul 18 05:56:16 2009 mpfr Sat Jul 18 06:12:43 2009 zlib Sat Jul 18 06:20:37 2009 elfutils Sat Jul 18 06:30:46 2009 rpm Sat Jul 18 06:47:45 2009 keyutils Sat Jul 18 07:09:49 2009 krb5 Sat Jul 18 09:03:09 2009 cyrus-sasl Sat Jul 18 08:24:31 2009 openldap2-client Sat Jul 18 08:43:48 2009 pwdutils Sat Jul 18 09:17:13 2009 mysql Sat Jul 18 10:39:53 2009 bind Sat Jul 18 11:09:57 2009 ksh Sat Jul 18 11:40:02 2009 graphviz Sat Jul 18 12:11:49 2009 avahi Sat Jul 18 12:28:13 2009 cups Sat Jul 18 12:51:04 2009 gtk2 Sat Jul 18 13:20:48 2009 libglade2 Sat Jul 18 13:29:16 2009 python-gtk Sat Jul 18 14:04:21 2009 avahi-glib2 Sat Jul 18 14:26:06 2009 gnome-vfs2 Sat Jul 18 14:59:59 2009 libgnome Sat Jul 18 20:24:02 2009 libbonoboui Sat Jul 18 21:40:23 2009 libgnomeui Sat Jul 18 22:08:46 2009 mozilla-xulrunner191 Sat Jul 18 23:56:51 2009 java-1_6_0-openjdk Sun Jul 19 01:25:09 2009 wsdl4j Sun Jul 19 15:12:48 2009 jakarta-commons-httpclient3 Sun Jul 19 15:19:36 2009 axis Sun Jul 19 15:40:30 2009 mx4j Sun Jul 19 16:01:32 2009 log4j Sun Jul 19 16:23:49 2009 ant-antlr Sun Jul 19 14:29:37 2009 OpenOffice_org-bootstrap Sun Jul 19 15:27:26 2009 OpenOffice_org-ure Sun Jul 19 16:24:49 2009 OpenOffice_org-libs-extern Sun Jul 19 17:03:23 2009 OpenOffice_org-libs-gui Sun Jul 19 18:39:23 2009 OpenOffice_org Sun Jul 19 19:04:03 2009 OpenOffice_org-icon-themes Sun Jul 19 19:13:25 2009 OpenOffice_org-openclipart Sun Jul 19 19:25:21 2009 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Show replies by date

Richard Guenther

20 Jul 20 Jul

13:17

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...

Hi,

Just wanted to share some log file checking with you. I wanted to know how the rebuild this weekend worked out, so I checked the chain upto the last built package and the last time it built (often packages get build twice due to cycles).

As you can see, it takes roughly 24h from start of the rebuild cycle on friday afternoon to the middle of the GNOME stack and roughly another day from the middle of the GNOME stack to the end of Ooo.

But note that the list only gives you a rough idea as it's only looking at i586, but both architectures interact with each other, e.g. there is a 5 hour gap between gnome-vfs2 and libgnome. This is not the build time of libgnome, but the fact that x86_64 was a bit slower and finished libbonobui.2.10 only at 8pm and caused i586 to build 2.10 too - while i586's version was unchanged to 2.9, so it discarded it. But on x86_64 the value of %_libexecdir changed, so libbonoboui files changed. In case you're still reading, don't worry :)

I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances. -> home:rguenther:hw, changes in rpm and build. As there is plenty of time left for day 1 I'll try this blogging thing (eeeeeek!). Richard. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dominique Leuenberger

13:31

...

...
...
On 7/20/2009 at 15:17, Richard Guenther wrote:

...

I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

I hope this won't end up in dozens of 'package x has higher version but less fitting architecture' due to foo-1-1.1 in x86_64 and foo-1-1.2 in i586 this is something that was avoided now with keeping them in sync if I'm not mistaken. (zypper might play nicely with this. not sure about other PMs). Dominique -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

13:56

Am Montag 20 Juli 2009 schrieb Dominique Leuenberger:

...

...
...
...
On 7/20/2009 at 15:17, Richard Guenther wrote:

I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

I hope this won't end up in dozens of 'package x has higher version but less fitting architecture' due to

foo-1-1.1 in x86_64 and foo-1-1.2 in i586

this is something that was avoided now with keeping them in sync if I'm not mistaken. (zypper might play nicely with this. not sure about other PMs).

zypper has to play nice because we too often have build failures only on one arch. And afaik most repos are not configured to use rebuild counter sync. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

14:41

On Mon, 20 Jul 2009, Dominique Leuenberger wrote:

...

...
...
...
On 7/20/2009 at 15:17, Richard Guenther wrote:

...
I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

I hope this won't end up in dozens of 'package x has higher version but less fitting architecture' due to

foo-1-1.1 in x86_64 and foo-1-1.2 in i586

this is something that was avoided now with keeping them in sync if I'm not mistaken. (zypper might play nicely with this. not sure about other PMs).

That may of course happen, but you need to have repositories for two architectures registered, no? I do not see this as a big problem anyway - it could happen as intermediate state during a rebuild already. Richard. -- Richard Guenther Novell / SUSE Labs SUSE LINUX Products GmbH - Nuernberg - AG Nuernberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

14:44

On Mon, 20 Jul 2009, Richard Guenther wrote:

...

On Mon, 20 Jul 2009, Dominique Leuenberger wrote:

...
...
...
...
On 7/20/2009 at 15:17, Richard Guenther wrote:

...
I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

I hope this won't end up in dozens of 'package x has higher version but less fitting architecture' due to

foo-1-1.1 in x86_64 and foo-1-1.2 in i586

this is something that was avoided now with keeping them in sync if I'm not mistaken. (zypper might play nicely with this. not sure about other PMs).

That may of course happen, but you need to have repositories for two architectures registered, no? I do not see this as a big problem anyway - it could happen as intermediate state during a rebuild already.

Btw, the main motivation here is not getting rid of the build counter sync but instead allowing us to drop unchanged sub-packgages to shrink the amount of packages to download and upgrade when tracking Factory. This at least requires that the rebuild counter does not appear in package dependencies (which my changes ensure by dropping the rebuild counter from %release during dependency tracking). Richard -- Richard Guenther Novell / SUSE Labs SUSE LINUX Products GmbH - Nuernberg - AG Nuernberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dirk Müller

20:33

On Monday 20 July 2009, Richard Guenther wrote:

...

That may of course happen, but you need to have repositories for two architectures registered, no?

which is the regular case if you use x86_64? Greetings, Dirk -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

21 Jul 21 Jul

08:55

On Mon, 20 Jul 2009, Dirk Müller wrote:

...

On Monday 20 July 2009, Richard Guenther wrote:

...
That may of course happen, but you need to have repositories for two architectures registered, no?

which is the regular case if you use x86_64?

Is it? (Why should it be?) Richard. -- Richard Guenther Novell / SUSE Labs SUSE LINUX Products GmbH - Nuernberg - AG Nuernberg - HRB 16746 - GF: Markus Rex

Dominique Leuenberger

08:56

...

...
...
On 7/21/2009 at 10:55, Richard Guenther wrote: That may of course happen, but you need to have repositories for two architectures registered, no?

which is the regular case if you use x86_64?

Is it? (Why should it be?)

The official distribution repositories are bi-arch. i586 and x86_64 are in the same repo. As such, if you're running an x86_64 install, you always have two archs in your repos registered. Dominique -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dirk Müller

09:22

On Tuesday 21 July 2009, Richard Guenther wrote:

...

...
...
That may of course happen, but you need to have repositories for two architectures registered, no? which is the regular case if you use x86_64? Is it? (Why should it be?)

the opensuse update channels are biarch (and factory channel is as well). Gruss, Dirk -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Larry Stotler

17:16

On Tue, Jul 21, 2009 at 5:22 AM, Dirk Müller wrote:

...

the opensuse update channels are biarch (and factory channel is as well).

Isn't it tri-arch? PPC is in it as well IIRC...... -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Evan McClain

17:21

On Tue, Jul 21, 2009 at 01:16:54PM -0400, Larry Stotler wrote:

...

On Tue, Jul 21, 2009 at 5:22 AM, Dirk Müller wrote:

...
the opensuse update channels are biarch (and factory channel is as well).

Isn't it tri-arch? PPC is in it as well IIRC......

No... Us ppc users were kicked off to powerpc.opensuse.org :P -- Evan McClain /* No Comment */ evan.mcclain@gatech.edu -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Dave Plater

19:43

Evan McClain wrote:

...

On Tue, Jul 21, 2009 at 01:16:54PM -0400, Larry Stotler wrote:

...
On Tue, Jul 21, 2009 at 5:22 AM, Dirk Müller wrote:

...
the opensuse update channels are biarch (and factory channel is as well).

Isn't it tri-arch? PPC is in it as well IIRC......

No... Us ppc users were kicked off to powerpc.opensuse.org :P

Build service still has ppc repos and a lot is still building. Regards Dave P -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Marcus Meissner

21:47

On Tue, Jul 21, 2009 at 01:16:54PM -0400, Larry Stotler wrote:

...

On Tue, Jul 21, 2009 at 5:22 AM, Dirk Müller wrote:

...
the opensuse update channels are biarch (and factory channel is as well).

Isn't it tri-arch? PPC is in it as well IIRC......

Quad Arch, as we have some ppc64 subpackages too. But this is mostly the update repo. Ciao, Marcus -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

20 Jul 20 Jul

13:37

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...

I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

-> home:rguenther:hw, changes in rpm and build.

As there is plenty of time left for day 1 I'll try this blogging thing (eeeeeek!).

How does that get rid of the need to sync build counters? I can't see it improving anything but I can see a lot of things that would break. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

13:40

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

-> home:rguenther:hw, changes in rpm and build.

As there is plenty of time left for day 1 I'll try this blogging thing (eeeeeek!).

How does that get rid of the need to sync build counters? I can't see it improving anything but I can see a lot of things that would break.

The reason to sync build counters was that we use a random source package for the source tree. Thus, the source package refered to from binary rpms wasn't the one available in the repo. IIRC. At least I cannot see a single other reason. Richard. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

13:46

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...
Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

-> home:rguenther:hw, changes in rpm and build.

As there is plenty of time left for day 1 I'll try this blogging thing (eeeeeek!).

How does that get rid of the need to sync build counters? I can't see it improving anything but I can see a lot of things that would break.

The reason to sync build counters was that we use a random source package for the source tree. Thus, the source package refered to from binary rpms wasn't the one available in the repo. IIRC. Same for noarch packages.

At least I cannot see a single other reason.

But I can't see in your patches any logic that will make rpm -Uvh *.rpm update packages that were rebuilt Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

13:48

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
On Mon, 20 Jul 2009, Stephan Kulow wrote:

...
Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
I have just finished Milestone 1 of my hackweek project that gets rid of the need to sync build counters between the arch instances.

-> home:rguenther:hw, changes in rpm and build.

As there is plenty of time left for day 1 I'll try this blogging thing (eeeeeek!).

How does that get rid of the need to sync build counters? I can't see it improving anything but I can see a lot of things that would break.

The reason to sync build counters was that we use a random source package for the source tree. Thus, the source package refered to from binary rpms wasn't the one available in the repo. IIRC. Same for noarch packages.

Good point. I'll check what happens there and fixit if necessary (luckily build is noarch, so I'll notice as soon as the builds happen...)

...

...
At least I cannot see a single other reason.

But I can't see in your patches any logic that will make rpm -Uvh *.rpm update packages that were rebuilt

They will be updated because they have a newer release (yes, the release for binary rpms is still $release.$rebuildcount). Richard. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

13:55

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...

They will be updated because they have a newer release (yes, the release for binary rpms is still $release.$rebuildcount). OK, that's a basic misunderstanding from my side then. Then remains one reason to sync the build counter: documentation. For base packages (your area) changes might always be associated with source changes, but for packages high up the stack, we often fix them by a rebuild and not by a checkin. And "the fix is in 2.10" is just much simpler to state than "the fix is in 2.27 on i586 and 2.39 on x86_64".

Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Richard Guenther

14:01

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
They will be updated because they have a newer release (yes, the release for binary rpms is still $release.$rebuildcount). OK, that's a basic misunderstanding from my side then. Then remains one reason to sync the build counter: documentation. For base packages (your area) changes might always be associated with source changes, but for packages high up the stack, we often fix them by a rebuild and not by a checkin. And "the fix is in 2.10" is just much simpler to state than "the fix is in 2.27 on i586 and 2.39 on x86_64".

Well, if a rebuild fixes something then we should IMHO do conscious rebuilds by doing a (dummy) source upload. Like ------------------------------------------------------------------- Mon Jul 20 14:31:50 CEST 2009 - rguenther@suse.de - re-build with new gcc to fix miscompile that then even has a changelog that explains why this particular version is a good idea to update to. But now I am of course likely stepping on to many peoples toes again ... Heh, but it's hackweek. Nobody will force you to take my patches ;) Hacking RPM is so much fun! (not.) Richard. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stephan Kulow

14:15

Am Montag 20 Juli 2009 schrieb Richard Guenther:

...

On Mon, 20 Jul 2009, Stephan Kulow wrote:

...
Am Montag 20 Juli 2009 schrieb Richard Guenther:

...
They will be updated because they have a newer release (yes, the release for binary rpms is still $release.$rebuildcount).

OK, that's a basic misunderstanding from my side then. Then remains one reason to sync the build counter: documentation. For base packages (your area) changes might always be associated with source changes, but for packages high up the stack, we often fix them by a rebuild and not by a checkin. And "the fix is in 2.10" is just much simpler to state than "the fix is in 2.27 on i586 and 2.39 on x86_64".

Well, if a rebuild fixes something then we should IMHO do conscious rebuilds by doing a (dummy) source upload. Like

------------------------------------------------------------------- Mon Jul 20 14:31:50 CEST 2009 - rguenther@suse.de

- re-build with new gcc to fix miscompile

that then even has a changelog that explains why this particular version is a good idea to update to.

But now I am of course likely stepping on to many peoples toes again ...

Heh, but it's hackweek. Nobody will force you to take my patches ;)

I won't stand in the way. I actually like having the srcrpm not changing unless the sources changed - I'm just saying that we still might sync the build numbers, if anything starting with the RCs to make maintenance easier Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

5397

Age (days ago)

5398

Last active (days ago)

List overview

Download

20 comments

8 participants

participants (8)

Dave Plater
Dirk Müller
Dominique Leuenberger
Evan McClain
Larry Stotler
Marcus Meissner
Richard Guenther
Stephan Kulow