Petr Uzel wrote:

On Fri, Aug 19, 2011 at 08:29:19AM +0200, Per Jessen wrote:

...

Cristian Rodríguez wrote:

...

I would like to propose replacing (not removing) the default command line tools for compression, specifically "gzip" for "pigz" and "bzip2" for pbzip2. These tools are backward compatible and have dramatically better performance because they do compression in parallel utilizing current multiple CPU machines.

Possible paths to do this

a) Simple adding "update-alternatives" support to all the mentioned tools and let the user to decide, however we install pigz and pbzip2 by default. I do not endorse this way.

b) Building pigz with executable name "gzip" and rename gzip to gnu-gzip, and bzip2 to bzip2.old or something similar. This is the way endorse

What is the reason for keeping the old tools?

In case there is a bug in pigz/pbzip2 (you know, all sw has bugs :) ), it will be handy to have the old tools.

Petr

Very true, but if the new parallel tools have not been sufficiently tested, we should not be making them default just yet. Then it would be better to go with Cristians option 1 and leave it to the user. /Per -- Per Jessen, Zürich (22.9°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Petr Uzel wrote:

On Fri, Aug 19, 2011 at 09:32:38AM +0200, Per Jessen wrote:

...

Petr Uzel wrote:

...

On Fri, Aug 19, 2011 at 08:29:19AM +0200, Per Jessen wrote:

...

Cristian Rodríguez wrote:

...

I would like to propose replacing (not removing) the default command line tools for compression, specifically "gzip" for "pigz" and "bzip2" for pbzip2. These tools are backward compatible and have dramatically better performance because they do compression in parallel utilizing current multiple CPU machines.

Possible paths to do this

a) Simple adding "update-alternatives" support to all the mentioned tools and let the user to decide, however we install pigz and pbzip2 by default. I do not endorse this way.

b) Building pigz with executable name "gzip" and rename gzip to gnu-gzip, and bzip2 to bzip2.old or something similar. This is the way endorse

What is the reason for keeping the old tools?

In case there is a bug in pigz/pbzip2 (you know, all sw has bugs :) ), it will be handy to have the old tools.

Petr

Very true, but if the new parallel tools have not been sufficiently tested, we should not be making them default just yet. Then it would be better to go with Cristians option 1 and leave it to the user.

Well, gzip and bzip2 are older then their parallel counterparts which means they're by definition more tested (still, the "all sw has bugs" applies to them as well). However, while looking into pbzip2/pigz changelogs I couldn't find any recent mention about serious crash or dataloss, therefore I would consider these tools stable and tested enough to replace old bzip2/gzip in Factory.

Ah, yes - in Factory, no problem. -- Per Jessen, Zürich (24.6°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Marcus Meissner wrote:

On Fri, Aug 19, 2011 at 11:10:05AM +0200, Per Jessen wrote:

...

...

...

Very true, but if the new parallel tools have not been sufficiently tested, we should not be making them default just yet. Then it would be better to go with Cristians option 1 and leave it to the user.

Well, gzip and bzip2 are older then their parallel counterparts which means they're by definition more tested (still, the "all sw has bugs" applies to them as well). However, while looking into pbzip2/pigz changelogs I couldn't find any recent mention about serious crash or dataloss, therefore I would consider these tools stable and tested enough to replace old bzip2/gzip in Factory.

Ah, yes - in Factory, no problem.

As I am the one worrying about security issues...

pigz - based on zlib, so I think it likely will have no new issues.

pbzip2 - seems to be written from scratch? So might have more security issues.

Judging by the homepage (http://compression.ca/pbzip2/) pbzip2 does use libbzip2. -- Per Jessen, Zürich (24.7°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Le 19/08/2011 09:32, Per Jessen a écrit :

...

Very true, but if the new parallel tools have not been sufficiently tested, we should not be making them default just yet. Then it would be better to go with Cristians option 1 and leave it to the user.

there are always people that prefere old known bug to new unknown ones, so as long as the old tools are maintained it's better to keep them

jdd +1

On Friday 19 August 2011 10:35:30 jdd wrote: pete -- Powered by openSUSE 11.3 (x86_64) Kernel: 2.6.34.10-0.2-desktop KDE Development Platform: 4.6.00 (4.6.0) 15:15 up 21:12, 5 users, load average: 0.00, 0.02, 0.06 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stefan Seyfried wrote:

Am Fri, 19 Aug 2011 08:29:19 +0200 schrieb Per Jessen :

...

...

c) As these tools are more frecuently used with "tar", make it to prefer the parralel versions, current gnu-tar already tries different implementations when the default ones are not present, would be a matter of altering the order if needed. I take no stance on this way.

'less' should probably also prefer the p-tools.

why? What would it help?

I assume that the parallel decompression will also improve the performance when one less'es a gzipped file. -- Per Jessen, Zürich (24.5°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Stefan Seyfried wrote:

Am Fri, 19 Aug 2011 11:09:22 +0200 schrieb Per Jessen :

...

Stefan Seyfried wrote:

...

Am Fri, 19 Aug 2011 08:29:19 +0200 schrieb Per Jessen :

...

'less' should probably also prefer the p-tools.

why? What would it help?

I assume that the parallel decompression will also improve the performance when one less'es a gzipped file.

what parallel decompression are you talking about? (Always funny when you find out the people most pushing a feature don't know anything about it ;-P)

Hey, I'm not pushing anything, but I guess I assumed (wrongly) that gzip also did decompression in parallel. -- Per Jessen, Zürich (24.8°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Friday 19 August 2011, Cristian Rodríguez wrote:

On 19/08/11 05:00, Stefan Seyfried wrote:

...

pseife@susi:~> pigz --rsyncable pigz abort: invalid option: rsyncable not implemented yet: -R

That option was implemented IIRC, there must be a bug, fill a bugzilla report, thanks !

Even it would be implemented, is it sure that compression with old gzip and pigz always gives the same files? Here it doesn't: rudi@quant:/tmp> time ~/tmp/pigz-2.1.6/pigz -c hist.csv |md5sum 95311648cd2813bfd2d9a2a265b9c9de - real 0m8.839s user 0m34.222s sys 0m0.987s rudi@quant:/tmp> time gzip -c hist.csv |md5sum 736ed650389ca836c6f7aec93c071eb3 - real 0m29.950s user 0m29.797s sys 0m0.359s Don't know whether I'am doing wrong but sometimes I rely on it that gunzip bla.gz gzip bla restores the same bla.gz as before. If gzip will be replaced by pigz then I can never do that again with any of my existing .gz files. BTW you can see that user time is 15% higher with pigz so compression per core cycle is worse. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Tue, 30 Aug 2011 01:20:35 +0200 schrieb Anders Johansson :

The elapsed time (real) tells you how much time the command actually took to complete, and that dropped to less than a third

He meant to express that it is less efficient - in "number of CPU cycles spent" Which is to be expected, as threading surely introduces some overhead. The measurement is nonsense anyway :-) -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Tue, Aug 30, 2011 at 12:18:59PM +0200, Anders Johansson wrote:

On Tuesday 30 August 2011 11:22:15 Stefan Seyfried wrote:

...

Am Tue, 30 Aug 2011 01:20:35 +0200

schrieb Anders Johansson :

...

The elapsed time (real) tells you how much time the command actually took to complete, and that dropped to less than a third

He meant to express that it is less efficient - in "number of CPU cycles spent"

Which is to be expected, as threading surely introduces some overhead.

That was exactly my point. The only interesting thing is overall time, unless you're trying to minimize power usage, which might be interesting for embedded systems, but for a general desktop system, speed and performance usually take center stage

Hi, I don't agree with that! General desktop systems still run many processes, which often include zipping/unzipping, in the background and I would *not* want these processes to use all my CPU cores with high %. For foreground/interactive processes, it can be very useful, but it's for the user to determine when (just like with make -j). I would very much prefer the default to stay the same. Mischa -- Nikhef Room H155 Science Park 105 Tel. +31-20-592 5102 1098 XG Amsterdam Fax +31-20-592 5155 The Netherlands Email msalle@nikhef.nl __ .. ... _._. .... ._ ... ._ ._.. ._.. .._.. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Tuesday 30 August 2011, Claudio Freire wrote:

...

BTW you can see that user time is 15% higher with pigz so compression per core cycle is worse.

That's called cache pressure. It happens when more than one process is running. Run four gzips concurrently, you'll probably experience the same increase.

Yes, ... ... and ouf course you will see that gzip has more throughput in any case while the system is less on load: # 4 x gzip in parallel rudi@quant:/tmp> time ( for i in `seq 1 4` ; do gzip -c hist.csv & done ;wait ) > /dev/null real 0m30.500s user 1m59.311s sys 0m0.978s # 4 x pigz in parallel in parallel rudi@quant:/tmp> time ( for i in `seq 1 4` ; do ~rudi/tmp/pigz-2.1.6/pigz -c hist.csv & done ;wait ) > /dev/null real 0m35.164s user 2m18.944s sys 0m1.236s # 4 x pigz sequential -bash: syntax error near unexpected token `)' rudi@quant:/tmp> time ( for i in `seq 1 4` ; do ~rudi/tmp/pigz-2.1.6/pigz -c hist.csv ;done ;wait ) > /dev/null real 0m35.147s user 2m16.647s sys 0m3.398s cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Tuesday 30 August 2011, Cristian Rodríguez wrote:

is /tmp and in ram ? otherwise your "tests" are meaningless, because you are then measuring I/O aka. the wrong thing!

The real time/user time ratio tells you that it was not waiting for I/O. And HD cache is as fast as RAM: rudi@quant:/tmp> du -sh hist.csv 847M hist.csv rudi@quant:/tmp> time cat hist.csv > /dev/null real 0m0.254s user 0m0.006s sys 0m0.247s cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Tue, 30 Aug 2011 00:44:21 +0200 schrieb Rüdiger Meier :

Even it would be implemented, is it sure that compression with old gzip and pigz always gives the same files? Here it doesn't:

It doesn't need to. I'm pretty sure that compression with different versions of old gzip will yield different files.

Don't know whether I'am doing wrong but sometimes I rely on it that

You do wrong.

gunzip bla.gz gzip bla restores the same bla.gz as before.

You can't rely on that. -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Tue, Aug 30, 2011 at 12:29 PM, Ruediger Meier wrote:

AFAIK gzip's determinism is the the only reason why it was possible to implement --rsyncable.

Did you try if pigz is deterministic? It could be deterministic, only different than gzip. There is no reason for it not to be deterministic. There is lots of reasons for it to be different than gzip, though. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Tue, 30 Aug 2011 12:29:46 +0200 schrieb Ruediger Meier :

On Tuesday 30 August 2011, Stefan Seyfried wrote:

...

I'm pretty sure that compression with different versions of old gzip will yield different files.

If you are sure then it should be easy to give an example ...

Even I've tested all version from 1.2.4 (1993) until 1.4 (2010) on different architectures. All deterministic.

11.4 x86_64: seife@server:~/tmp> md5sum urandom 77b8857e80e450231c555122ccc4f789 urandom seife@server:~/tmp> gzip -c urandom|md5sum 27713792de0f0821bf7dfa5c668d70c2 - ARM embedded: /tmp # md5sum urandom 77b8857e80e450231c555122ccc4f789 urandom /tmp # gzip -c urandom | md5sum 2f96e249b14ec2908d04b3bd8e532203 -

AFAIK gzip's determinism is the the only reason why it was possible to implement --rsyncable.

No, not at all. -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Hi, On Tuesday 30 August 2011, Ruediger Meier wrote:

On Tuesday 30 August 2011, Stefan Seyfried wrote:

...

11.4 x86_64: seife@server:~/tmp> md5sum urandom 77b8857e80e450231c555122ccc4f789 urandom seife@server:~/tmp> gzip -c urandom|md5sum 27713792de0f0821bf7dfa5c668d70c2 -

ARM embedded: /tmp # md5sum urandom 77b8857e80e450231c555122ccc4f789 urandom /tmp # gzip -c urandom | md5sum 2f96e249b14ec2908d04b3bd8e532203 -

Could you please post gzip --version for both systems. And try again with gzip --no-name -c urandom | md5sum and maybe also just the sizes gzip --no-name -c urandom | wc -c

I'm still curious about this. Without --no-name it's very unlikely that gzip behaves deterministic. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Wednesday 31 August 2011, Stefan Seyfried wrote:

Am Wed, 31 Aug 2011 14:45:28 +0200 I found that the embedded machine uses busybox gzip. When using gzip from debian ARM and --no-name, the compressed file is identical.

But that proves my point: don't rely on undocumented behaviour. Different implementation might implement the undocumented bits differently.

Yes, I simply vote for "Please don't change the default implementation".

The only thing that's documented (besides the file format) is, that compressing and then decompressing yields the same file. Not the other way round.

We are talking about "changing the implementation _per_default_" not about the protocol or implementation itself. "The admin is probably to stupid to use pigz instead of gzip" - give him pigz per default - we know better as him - usually all people want it fast and colored. If he is not as stupid as we think then he could switch back. I don't like that argumentation. Keep the smart admin as he is and give the stupid one the possibility to get smarter. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Saturday 03 September 2011, Gerald Pfeifer wrote:

On Thu, 1 Sep 2011, Rüdiger Meier wrote:

...

"The admin is probably to stupid to use pigz instead of gzip" - give him pigz per default - we know better as him - usually all people want it fast and colored. If he is not as stupid as we think then he could switch back.

Another way of looking at this is seeing pigz just as the next version of gzip given that the authors of the latter have not adjusted it to the needs of many users in 2011.

Hm, but Mark Adler the author of pigz is also an author of gzip. Probably he hasn't got it upstream for a good reason or he is just a good developer who don't want to bother it's users. To say it again I like pigz. But renaming ... ? Why not replacing it in /usr/bin/compress - the Posix way to define a Z compressor. If user asks for gzip please give him gzip. cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Philipp Thomas wrote:

On Sun, 4 Sep 2011 00:51:09 +0200, Rüdiger Meier wrote:

...

To say it again I like pigz. But renaming ... ?

...

Why not replacing it in /usr/bin/compress - the Posix way to define a Z compressor. If user asks for gzip please give him gzip.

Then 99% of all users will continue to use gzip just because they don't know better.

Which isn't really a problem, is it? -- Per Jessen, Zürich (21.9°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Ralf Lang wrote:

Am Sonntag, 4. September 2011, 15:51:28 schrieb Per Jessen:

...

Philipp Thomas wrote:

...

On Sun, 4 Sep 2011 00:51:09 +0200, Rüdiger Meier

wrote:

...

To say it again I like pigz. But renaming ... ?

Why not replacing it in /usr/bin/compress - the Posix way to define a Z compressor. If user asks for gzip please give him gzip.

Then 99% of all users will continue to use gzip just because they don't know better.

Which isn't really a problem, is it?

If it wasn't viewed as the wrong thing, the question had not been raised.

Thank you, I am aware of that. I was trying to argue that that view is wrong. -- Per Jessen, Zürich (21.1°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Dienstag, 30. August 2011, 12:29:46 schrieb Ruediger Meier:

On Tuesday 30 August 2011, Stefan Seyfried wrote:

...

I'm pretty sure that compression with different versions of old gzip will yield different files.

If you are sure then it should be easy to give an example ...

Even I've tested all version from 1.2.4 (1993) until 1.4 (2010) on different architectures. All deterministic.

AFAIK gzip's determinism is the the only reason why it was possible to implement --rsyncable.

cu, Rudi

Deterministic in themselves or tested against all other versions of gzip? -- Ralf Lang Linux Consultant / Developer B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Thursday 01 September 2011 00:57:53 Rüdiger Meier wrote:

Even it's an undocumented feature I would not change it per default just to make it faster for people who don't manage to type pigz instead of gzip.

To me the most obvious point is that there are lots of scripts out there that call gzip. Command line invocations aren't the big deal. It's not just about "type this instead of that", it's about changing all the scripts. Maybe it can be done with update-alternatives? :) Both gzip and bzip2 are way too slow. I'm not sure how this new pigz thing measures against xz, but when I tested xz I was very impressed by both speed and compression ratios. We need improvements Ideally we should have a "meta compressor", one program with one set of parameters, and a configurable backend, selectable through configuration, preferably in YaST. That way everything is API compatible, and the admin can select what he wants in a nice and easy way as the default. Multiple executables shouldn't be necessary in an open source world, developing a new compression algorithm should just involve developing a new backend library to the standard. Everything would then be able to take advantage of a new compression without changing any code. Ah, dreams... Anders -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On 01/09/11 06:38, Ruediger Meier wrote:

On Thursday 01 September 2011, Anders Johansson wrote:

...

To me the most obvious point is that there are lots of scripts out there that call gzip. Command line invocations aren't the big deal. It's not just about "type this instead of that", it's about changing all the scripts. Both gzip and bzip2 are way too slow.

There is not that much to change, for example I just have this on all my machines: /etc/logrotate.conf compresscmd /usr/bin/xz compressoptions -3 uncompresscmd /usr/bin/unxz

Well, that IMHO is another legacy error, I think logrotate should use libarchive instead of forking X compression programs and support only one compression format, one that achieves the best compression ratio. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On 01/09/11 12:47, Ruediger Meier wrote:

So you would need to change the code and recompile logrotate every time when a better compression format comes out.

Which is once every a few years.. or once a decade.. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Cristian Rodríguez wrote:

On 01/09/11 06:38, Ruediger Meier wrote:

...

On Thursday 01 September 2011, Anders Johansson wrote:

...

To me the most obvious point is that there are lots of scripts out there that call gzip. Command line invocations aren't the big deal. It's not just about "type this instead of that", it's about changing all the scripts. Both gzip and bzip2 are way too slow.

There is not that much to change, for example I just have this on all my machines: /etc/logrotate.conf compresscmd /usr/bin/xz compressoptions -3 uncompresscmd /usr/bin/unxz

Well, that IMHO is another legacy error, I think logrotate should use libarchive instead of forking X compression programs and support only one compression format, one that achieves the best compression ratio.

Definitely -1. Without knowing what the compressed files might be used for afterwards, chosing a suitable compression algorithm is impossible. -- Per Jessen, Zürich (21.8°C) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

At Fri, 19 Aug 2011 11:24:11 +0200, Lars Müller wrote:

On Thu, Aug 18, 2011 at 07:53:36PM -0400, Cristian Rodríguez wrote:

...

I would like to propose replacing (not removing) the default command line tools for compression, specifically "gzip" for "pigz" and "bzip2" for pbzip2. These tools are backward compatible and have dramatically better performance because they do compression in parallel utilizing current multiple CPU machines.

Possible paths to do this

a) Simple adding "update-alternatives" support to all the mentioned tools and let the user to decide, however we install pigz and pbzip2 by default. I do not endorse this way.

Arguments please. update-alternatives is a mechanism we use for other alternatives we provide and is known to work. Therefore I would prefer if we reuse the same mechanism for pbzip2.

What are the drawbacks you see with this mechanism?

Complexity. Note that I'm not objecting this. I would take this option, too. In general, however, update-alternative make things complicated than static configuration. A complex thing is beautiful if it works, but gives more trouble when it doesn't work. Takashi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Fri, Aug 19, 2011 at 12:05:37PM -0400, Cristian Rodríguez wrote:

On 19/08/11 05:40, Takashi Iwai wrote:

...

...

What are the drawbacks you see with this mechanism?

Complexity.

Yes, that's my point, this update-alternatives thing introduces yet another layer of "magic" that does not always work smoothly.

openSUSE, Linux, Open Source Software _always_ doesn't work smoothly. I suggest we drop all of this stuff asap. Sorry I can't acceppt this as an arguement. Bug IDs please. That's what we can judge on. Lars -- Lars Müller [ˈlaː(r)z ˈmʏlɐ] Samba Team SUSE Linux, Maxfeldstraße 5, 90409 Nürnberg, Germany

On Mon, Aug 22, 2011 at 12:36 PM, Takashi Iwai wrote:

At Fri, 19 Aug 2011 18:46:03 +0200, Lars Müller wrote:

...

On Fri, Aug 19, 2011 at 12:05:37PM -0400, Cristian Rodríguez wrote:

...

On 19/08/11 05:40, Takashi Iwai wrote:

...

...

What are the drawbacks you see with this mechanism?

Complexity.

Yes, that's my point, this update-alternatives thing introduces yet another layer of "magic" that does not always work smoothly.

openSUSE, Linux, Open Source Software _always_ doesn't work smoothly.  I suggest we drop all of this stuff asap.

Sorry I can't acceppt this as an arguement. Bug IDs please.  That's what we can judge on.

I understand Cristian's concern because I also suffered from a bug of update-alternative a few times.  It happened more often while upgrading distro versions.

Just try to upgrade your system from 10.1 to 12.1 one by one.  At some point, you'll likely get an non-working java, for example. Then enter a bug report.  Someone will close it immediately as WONTFIX for such an old version :)

The complexity itself is a bad thing.  A golden rule is "always remember KISS".  That is, if we can avoid this extra complexity, we should avoid it.

OTOH, in the case of gzip/pigz, it's difficult to achieve the same functionality (replacing gzip dynamically per installation status) without a help of such a complex layer.  That's why I'm for this option even though I have no good feeling to update-alternative.

Takashi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

I've used pigz with kiwi. The results is impressive. 2min versus 7min on image creation -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Mon, 29 Aug 2011 10:41:28 +0200 schrieb Vincent Untz :

Le vendredi 19 août 2011, à 11:24 +0200, Lars Müller a écrit :

...

Arguments please. update-alternatives is a mechanism we use for other alternatives we provide and is known to work. Therefore I would prefer if we reuse the same mechanism for pbzip2.

+1.

Sure, update-alternatives is not very friendly at first, but once you've used it, it's fine :-) If you need help to use it, just ask!

it breaks "command not found". try "cnf javac" on a FACTORY machine and try to guess where to get it from. -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Mon, 29 Aug 2011 13:50:03 +0200 schrieb Vincent Untz :

cnf doesn't seem to work in factory for me anyway.

OOPS. Just checked and that's true. So just a coincidence that I tried it on an update-alternatives using package...

But do the packages providing javac with update-alternatives have a %ghost file for it?

...no idea, but it doesn't look like it has: seife@susi:~> rpm -ql java-1_6_0-openjdk-devel|grep javac /usr/lib64/jvm/java-1.6.0-openjdk-1.6.0/bin/javac /usr/share/man/man1/javac-java-1.6.0-openjdk.1.gz -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am 19.08.2011 18:07, schrieb Cristian Rodríguez:

On 19/08/11 07:03, Kim Leyendecker wrote:

...

...

Am 19.08.2011 01:53, schrieb Cristian Rodríguez: Agreed from my site. I´m using pbzip2 on try for now. By the way, do you have a feature request @ openFATE? There is no feature request, I guess we will have to fill one.;)

I beg you to do so. Later we all will delete the posts of the thread and forget about it... -- -o) Kim Leyendecker /\\ openSUSE Ambassador, openSUSE Wiki Team DE _\_v http://www.opensuse.org - Linux for open minds -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Montag, 29. August 2011 schrieb Ruediger Meier:

On Friday 19 August 2011, Cristian Rodríguez wrote:

...

a) Simple adding "update-alternatives" support to all the mentioned tools and let the user to decide, however we install pigz and pbzip2 by default. I do not endorse this way.

Hopefully providing them as "update-alternatives" and making them even the default are two different steps to be done in different openSuSE releases!

These tools are so essential that a possible replacement must done very carefully. For example what happens on a quad core CPU with "make -j 4" if make invokes 4 multithreaded pbzip2? A load of 16!? Is this what the user wants?

A load of 16 doesn't mean that the computer explodes, it just means that the kernel has more options to pick IO. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Mon, 29 Aug 2011, Ruediger Meier wrote:

I dont't want to have my scripts messed up per default. Please don't make such extreme behavior changings for such old and stable tools without long testing period.

That's what we have openSUSE Factory for, don't we? +1 for replacing gz by pigz by default. Gerald -- Dr. Gerald Pfeifer || SUSE || Director Product Management -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On Tuesday 30 August 2011, Claudio Freire wrote:

On Mon, Aug 29, 2011 at 1:28 PM, Ruediger Meier wrote:

...

Yes, I just mentioned that users (like me) may have parallelized their jobs already. I'am sure that invoking 4 single threaded gzips will give you always more throughput than one 4-threaded and even more than 4 4-threaded ones. And the IO and CPU load will keep the machine more usable. Also I guess the mem usage will increase with more threads.

Well, in *that* case people could use -p 1.

Why they should? This is exactly the point. -p 1 is the default for gzip since 20 years or something. Option -p doesn't even exists for gzip. It's paradox to tell people to use -p for gzip when it doesn't exists. Those scripts will only run if they have the right update-alternative choosen or what? Is this good? Sorry, I simply doen't get the point why the hell when I type gzip (gnu-zip) suddenly another kind of pigz should pop up? I am actually capaple of typing pigz when I want to use it (and surely I will use it where it makes sense.) cu, Rudi -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

Am Tue, 30 Aug 2011 10:03:52 +0200 schrieb Claudio Freire :

Well, in *that* case people could use -p 1.

but not -R :-) -- Stefan Seyfried "Dispatch war rocket Ajax to bring back his body!" -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org

On 29/08/11 07:57, Ruediger Meier wrote:

On Friday 19 August 2011, Cristian Rodríguez wrote:

...

a) Simple adding "update-alternatives" support to all the mentioned tools and let the user to decide, however we install pigz and pbzip2 by default. I do not endorse this way.

Hopefully providing them as "update-alternatives" and making them even the default are two different steps to be done in different openSuSE releases!

These tools are so essential that a possible replacement must done very carefully. For example what happens on a quad core CPU with "make -j 4" if make invokes 4 multithreaded pbzip2? A load of 16!? Is this what the user wants?

No, it is not that dumb, it will be multi threaded only when the input file is big enough to benefit from it, by default IIRC is 900K. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-factory+help@opensuse.org