On 2012-04-20 09:55:44 (+0200), Adrian Schröter wrote:

Am Donnerstag, 19. April 2012, 22:21:52 schrieb Angelos Tzotsos:

...

There was a small discussion among Application:Geo project members about this issue today. I was asked to bring this issue to this list so that we can all discuss it.

...

The problem: Currently it is forbidden to package binaries in OBS without building from source code. This causes issues when we need to build very complex Java applications or web applications (eg like Geoserver, Geonetwork, JOSM etc). In Java it is nearly impossible to build everything without bringing jar binaries in the game at some point (eg maven depends on itself to build). If we want to do this without binaries it would take a very large effort for many packages and it would be a dependency nightmare for packagers.

...

On the other hand, the build.o.o instance has open source and transparency in mind, so the policy to request sources and not binaries is fair.

While I think it is not a good idea to have exceptions, some exceptions for very few packages to be able to boot strap may be okay from my POV. On the other hand, all other languages, like C/C++/C# or even pascal and haskal can boot strap themselfs. Why should java be an exception?

Sorry, very long email ahead, but I'd like to clarify a few points to not have to argue about them again and again ^^ Java can bootstrap itself. OpenJDK bootstraps from C (the kernel of the JVM (Java Virtual Machine) is written in C). Maven can bootstrap itself too, and does so, actually. It _can_ be done, and is done for OpenJDK at least, that's not the issue. It is a question of effort vs benefit. If we take the example of Maven, which is particularily painful because it is pretty complex to build by bootstrapping and by using our "Linux" (rpm/deb) technical model for "packages" and "dependencies": Maven, as most Java based things, is designed to be portable across all operating systems that can run a JVM (of a certain version) For that reason, and because another unfortunately quite prominent operating system does not have a comparable package management model (*), it does not use a portable way of installing components of software other than just files that are downloaded and put somewhere (in the case of Maven, that's ~/.m2/repository). On a side note, that is an approach you will find with -- as far as I know -- every other build tool for Java projects which does dependency management (ivy (which is an add-on for Ant), gradle, etc...) +-+------------------------------------------------------------- | (*) To prevent off-topic discussions about Windows and MSI: +-+------------------------------------------------------------- |S| Yes, it has, to some extent, but it has always and partly |I| still is very difficult to use, and simply isn't the model |D| that has developed with its users: Windows users have always |E| and will always just download archives somewhere and use them | | from there, without much system integration -- nothing |N| *prevents* java archives to be used and handled centrally in |O| the same way as DLLs (well, sort of, for some of them ...) or |T| in the same way as it is done on Linux, but it is simply not |E| how Windows users and developers do it, for whatever reasons | | (I have a few ideas about those reasons but won't mention | | them because I don't want to be rude). | | | | In any way, let's not discuss this here, it is irrelevant. +-+------------------------------------------------------------- Hence, we are faced with two different models of managing dependencies and, in a way, the Java approach is a lot more like MacOSX' DMG, or klik on Linux, or just prehistoric zip files on Windows, which is to include most dependencies (libraries, tools) as part of an "image" of some sort (which is usually a tarball or a zip file). So, the real question is: do we want to fight an uphill battle against what 95% of Java developers are doing, or even forced to do as that's what their build environments are doing and transform every possible Java based piece of software (**) we want to have in the distro or repositories into our RPM philosophy/approach (***), or do we want to be more pragmatic about it (being aware of the tradeoffs) ? (**) which spans a very broad horizon from desktop sorts of applications (including things like Eclipse and Netbeans) to server-room stuff like Tomcat or JBoss (***) which is to remove all the Java libraries (jar files) that are shipped with a Java software distribution or retrieved using mechanisms such as Maven's, and instead have those as individual dependencies -- e.g. remove xerces from ant and have ant use a system-wide xercesImpl.jar from /usr/share/java/... Okay, now that you've answered that question for yourself thinking about Vuze or Eclipse or Netbeans, ask yourself the same question again in the shoes of a system administrator with Tomcat or JBoss.

(It is okay to have a build cycle in OBS, it should be just only very few packages and not >100 what would make the bootstrap process rather long).

Once you have a few build tools (Ant, Maven) and their dependencies (which are a lot more than that, probably around 50 to 100), you're good. But that's quite some amount of work.

The other side are add-on packages for libs and applications. I personally see no reason why we should become just a binary only hoster here and have an exception just because it is written in java.

Very clearly: convenience for users (which very explictly includes system administrators). Personally, I don't believe it makes much sense to fight the uphill battle I described above. In my not so humble opinion, it is perfectly fine to ship e.g. Eclipse or Ant or Maven with all the jar files it needs as its own set of dependencies (e.g. have Eclipse's dependencies in /usr/lib/eclipse/lib/ instead of /usr/share/java/log4j, /usr/share/java/asm, ... and a million others) as they ship from upstream rather than building every single of those dependencies on its own. Pros: + we can actually provide packages of such things without too much effort Cons: - we can't patch the sources ourselves Now, both points raise questions themselves: - do we want to provide such pieces of software as packages in the first place ? * after all, one can simply grab a tarball of Eclipse or Maven from their respective websites and install them, even without being root, aka "the windows way" * some of us hate java, mostly because they don't understand it or because of a very few GUI java applications that are slow - how likely is it that we are going to patch an upstream java library or application ourselves, rather than updating it to an upstream release that fixes the issue ? * very few in this project are fluent with java * is it an issue to push a newer version to fix rather than backporting changes/patches for those packages ? I have my personal opinion on those points above, I'll leave it up to everyone to wager theirs.

Esp. when you look at the current Google vs. Oracle fight, it makes it actually obvious for me that we need to be able to do a legal review for the java stack.

Are you sure that you understand what you are talking about ?

The downside might be that we loose quite some packages because we do not have anyone able or willing to package it anymore. Can we live without Eclipse, josm and friends?

Well, assuming you mean "without Eclipse *packages*", yes and no: * yes, because there are other ways of installing java based applications and libraries (I personally just pull eclipse from eclipse.org and put them into some directory under my home), especially for desktop applications * no if in any way we want to attract more developers (tools such as Eclipse, Netbeans, Maven, Ant are paramount to doing so, and I'm not only talking about Java developers) * no if we want to provide packages of more server oriented applications such as Tomcat, JBoss, Wildfire, Tigase, ...), which are obviously much more interesting to install in a system-wide way using our usual mechanisms (rpm, zypp).

Open question is what is more important here.

Certainly, but the other question is whether we want a holistic or a pragmatic approach. Or, rather: can we live with a more pragmatic approach ? Personally, I believe that sticking to the holistic approach ("always build from source, always. period.") just for the sake of sticking to it is stupid. If we are aware of drawbacks, it should be perfectly fine to make exceptions, if the advantages are worth those drawbacks. Which kind of brings us back to your question :) My personal opinion: hell yes, there are packages where the drawbacks are worth the benefits. But there are also lots of others where it isn't. cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf

Some tine ago I started to package Apache Geronimo. After a few days, I'd stopped, because there are around 400 jars packaged; some sources need additional sources just for packaging/compiling. It would have taken month to build that with the resources I can spend. Currently I'm building the package from the Apache binaries and do not offer the package via the obs. It's sometimes very frustrating, because even when some of the packages have already been built for suse, the versions do not match with the required versions. I'd very much love to package binary jars for some projects. But I also understand, that it doesn't fit to the build-service philosophy. Couldn't we have a special repository, where such kind of packages are allowed? Best regards, Johannes Am 20.04.12 12:58, schrieb Ludwig Nussel:

Pascal Bleser wrote:

...

[...] - how likely is it that we are going to patch an upstream java library or application ourselves, rather than updating it to an upstream release that fixes the issue ? * very few in this project are fluent with java * is it an issue to push a newer version to fix rather than backporting changes/patches for those packages ?

Well, just recently we had the case where a fix was only a few lines but the package couldn't be rebuilt from java source and upstream only offered a major version update. That sucks. It's embarrassing if a package is flagged as free software but you can't actually fix it. If at all exceptions should be granted only on a case by case basis after thorough review. Also such packages should not be put in the oss tree.

cu Ludwig

-- Johannes Weberhofer Weberhofer GmbH, Austria, Vienna -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

Sorry, I didn't mean binary jars; a better wording would be "non-source-distribution" packages, in contrary to the source packages. Eg. Geronimo uses http://www.apache.org/dyn/closer.cgi/geronimo/2.2.1/geronimo-2.2.1-source-re... as source package and http://www.apache.org/dyn/closer.cgi/geronimo/2.2.1/geronimo-tomcat6-javaee5... as compiled/packaged installation package. For my needs, I'm packaging the second one. A great thing would be, when the OBS could run the maven tool using it's download feature! The the complete packaging process could be done in OBS very easily, but external jars would be downloaded from their maven repos. Best regards, Johannes Am 20.04.12 13:25, schrieb Guido Berhoerster:

* Johannes Weberhofer [2012-04-20 13:13]:

...

Some tine ago I started to package Apache Geronimo. After a few days, I'd stopped, because there are around 400 jars packaged; some sources need additional sources just for packaging/compiling. It would have taken month to build that with the resources I can spend. Currently I'm building the package from the Apache binaries and do not offer the package via the obs. It's sometimes very frustrating, because even when some of the packages have already been built for suse, the versions do not match with the required versions. I'd very much love to package binary jars for some projects. But I also understand, that it doesn't fit to the build-service philosophy. Couldn't we have a special repository, where such kind of packages are allowed?

How are binary jars different from statically linked binaries and private library copies which are rightfully outlawed. Having a package with 400 inlined jars sounds like a nightmare, are the packager or upstream closely monitoring 400 upstream projects for security issues?

-- Johannes Weberhofer Weberhofer GmbH, Austria, Vienna -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

On 2012-04-20 15:52:39 (+0200), Ludwig Nussel wrote:

Johannes Weberhofer wrote:

...

A great thing would be, when the OBS could run the maven tool using it's download feature! The the complete packaging process could be done in OBS very easily, but external jars would be downloaded from their maven repos.

What about patching that tool to produce .spec file for those deps instead? Similar to how perl module packages are created these days.

Not quite the same: * perl builds can pull dependencies from CPAN when you build, similar to Maven, in a way, but that is _not_ what we make spec files from: * we let perl module builds break on missing dependencies and then generate spec files for those manually, one by once, using metadata from CPAN Patching Maven to produce spec files is very different from what we do with Perl modules. There are several things that we can do, of course, such as to extract dependency information from POM files. It's just XML. I did a proposal and even a naive implementation (as a Perl script) that uses that data to create RPMs of java bits using binaries. (as a proposed intermediate solution) It is difficult to do anything like that at build time though. You run mvn to package some Java library. Maven analyses the POM file to see which dependencies are needed. It pulls those dependencies (binary bytecode, not sources that it will build too, it's not gentoo's "make world" ;)) from the internet or from custom Maven repositories. So then we would need to implement some mechanism on Maven that, instead, would say "okay, stop here, we don't have this, package this first". That would then be similar to how we handle Perl modules (or Ruby gems or Python .. erm.. bits of cheese for that matter). Would be a bit awkward though, as we'd transform or even remove one of the key elements of Maven :) cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf

On 04/20/2012 01:58 PM, Ludwig Nussel wrote:

...

[...] - how likely is it that we are going to patch an upstream java library or application ourselves, rather than updating it to an upstream release that fixes the issue ? * very few in this project are fluent with java * is it an issue to push a newer version to fix rather than backporting changes/patches for those packages ? Well, just recently we had the case where a fix was only a few lines but the package couldn't be rebuilt from java source and upstream only offered a major version update. That sucks. It's embarrassing if a package is flagged as free software but you can't actually fix it. If at all exceptions should be granted only on a case by case basis after thorough review. Also such packages should not be put in

Pascal Bleser wrote: the oss tree.

cu Ludwig

How about if we create a new tree for those Java applications? I think we should not use NonFree since this would confuse users, that they use non free software. Angelos -- Angelos Tzotsos Remote Sensing Laboratory National Technical University of Athens http://users.ntua.gr/tzotsos -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-04-20 15:58, Ludwig Nussel wrote:

Angelos Tzotsos wrote:

...

On 04/20/2012 01:58 PM, Ludwig Nussel wrote:

...

How about if we create a new tree for those Java applications? I think we should not use NonFree since this would confuse users, that they use non free software.

There is no usable source, you can't fix bugs in the code yourself and you probably don't even know what license that random dump of bundled jars has. Calling it non-free is the closest match.

call it unknown... :-) - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/ iEYEARECAAYFAk+RrRIACgkQIvFNjefEBxrj8wCeMIYIAkBGpE9oR9BE7RZSaDUv aJEAn3H2uMRkrm0TbpqCYLPitcnQiacs =2SK2 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-04-20 20:40, Angelos Tzotsos wrote:

On 04/20/2012 09:38 PM, Carlos E. R. wrote: On 2012-04-20 15:58, Ludwig Nussel wrote:

...

...

...

Angelos Tzotsos wrote:

...

...

...

bundled jars has. Calling it non-free is the closest match. call it unknown... :-)

nice :)

Maybe they should not be included in the OBS, but they can be included somewhere else, as external addons for convenience, so that it is clear that they don't meet the same rules and expectations. It is not closed source, in the sense that flash is, for example, but it is not open source in the same sense as the rest of the distro. So... unknown, undetermined, whatever. - -- Cheers / Saludos, Carlos E. R. (from 11.4 x86_64 "Celadon" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/ iEYEARECAAYFAk+RrtUACgkQIvFNjefEBxrSLwCfRTOVQzIrEGkLgYz6vqOT0IE5 3akAn083x8/YX7onWkE1kHlhJBs/D5eC =fB0F -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

On 2012-04-20 15:58:21 (+0200), Ludwig Nussel wrote:

Angelos Tzotsos wrote:

...

On 04/20/2012 01:58 PM, Ludwig Nussel wrote:

...

...

[...] - how likely is it that we are going to patch an upstream java library or application ourselves, rather than updating it to an upstream release that fixes the issue ? * very few in this project are fluent with java * is it an issue to push a newer version to fix rather than backporting changes/patches for those packages ? Well, just recently we had the case where a fix was only a few lines but the package couldn't be rebuilt from java source and upstream only offered a major version update. That sucks. It's embarrassing if a package is flagged as free software but you can't actually fix it. If at all exceptions should be granted only on a case by case basis after thorough review. Also such packages should not be put in

Pascal Bleser wrote: the oss tree.

...

How about if we create a new tree for those Java applications? I think we should not use NonFree since this would confuse users, that they use non free software.

There is no usable source, you can't fix bugs in the code yourself

Of course there is usable source. You can build it with Ant, Maven, Gradle, or quite often even with just a one line javac invocation. There are different approaches on handling and facilitating this. The essential problems are: 1) to bootstrap the build environments, i.e. get Maven built (and patched in order to lookup jars in /usr/share/java/blah too) 2) package the many hundreds of often used libraries and frameworks and such (oh yes, surprise surprise, java is very widely used (that's even an understatement), and there are huge amounts of open source pieces of software in java) I made a proposal a while ago to have a "lesser evil" mechanism to have packages from binary distributions at least as a short or mid term solution until we have a chance of catching up with those dependencies. It would also be a very sorry initiative if we did that on our own. Oh there is jpackage but, I don't know, seems a bit dormant.

and you probably don't even know what license that random dump of bundled jars has.

That is indeed quite a task but from more than 10 years of day to day experience of coding with java (and lots of open source java libraries and bits), I can say that the set is large (as said, there is a huge amount of open source software in java, and a high degree of reuse of libraries), but it's not totally random. What I mean to say, I guess, is that you keep running into mostly the same jars. I'll grant you definitely one thing here: unfortunately, the idea of Maven style repositories (which are by now a de-facto standard and used by almost all dependency management tools, obviously Maven itself, but also Gradle or Ivy) is quite good and sound, and also includes metadata about the component and its dependencies (obviously) is a missed opportunity in terms of license information: most POM files (which are component descriptors with all sorts of metadata, in XML format) lack licensing information. Then again, it wouldn't really be reliable either. The same accounts for non-java stuff too though, you can run into libraries written in C that say "GPL" on freshmeat (or freshcode, as it's called now) but effectively has different copyrights in individual source code files in its archive.

Calling it non-free is the closest match.

Sorry, are you saying the Apache Software License or the GPL/LGPL is not considered "oss" any more? Surely you're not. cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf

On 2012-04-23 10:04:06 (+0200), Ludwig Nussel wrote:

Pascal Bleser wrote:

...

On 2012-04-20 15:58:21 (+0200), Ludwig Nussel wrote: [...]

...

and you probably don't even know what license that random dump of bundled jars has.

...

That is indeed quite a task but from more than 10 years of day to day experience of coding with java (and lots of open source java libraries and bits), I can say that the set is large (as said, there is a huge amount of open source software in java, and a high degree of reuse of libraries), but it's not totally random. What I mean to say, I guess, is that you keep running into mostly the same jars.

How large is that common set? Maybe having the top most used jars would help reduce the amount of extra jars an applications pcakager needs to care for.

I would say certainly at least around 50 libraries (log4j, slf4j, lots of apache commons libraries, etc...).

...

[...] The same accounts for non-java stuff too though, you can run into libraries written in C that say "GPL" on freshmeat (or freshcode, as it's called now) but effectively has different copyrights in individual source code files in its archive.

That isn't nearly as common as with java I guess and we have the automatic licence digger in Factory to detect such bastards.

No it's not more common with java than with C/C++. I personally do run into such cases sometimes when packaging C/C++ stuff.

...

...

Calling it non-free is the closest match.

...

Sorry, are you saying the Apache Software License or the GPL/LGPL is not considered "oss" any more? Surely you're not.

Well, the random samples I've seen recently had proprietary .jars bundled in software that said it's GPL. I don't know what the result of such a bundling is, but I as user feel fooled if that's called free software. So even if an application claims it's GPL but crucial components of it are proprietary you can't put that into an 'oss' repo. Maybe it's even illegal to put something like that anywhere. Only a lawyer can tell but IANAL.

If by proprietary you mean "non open source license" then that is quite possible, because a few reference implementations of non-core standards around Java have historically been offered under a free-to-use but not open source license by Sun (and now Oracle). There aren't that many of them, but some are really key stuff, such as in the JEE space, which is why you keep running into them here and there. That being said, there are always open source alternatives (e.g. the servlet or EJB APIs and implementations from Glassfish, JBoss, Apache, ...). When you run into such non open source libraries bundled with an open source application/library, then it's almost always that the upstream devs didn't know about the alternatives. Or that they were Windows users and didn't know nor care much about copyrights. At least not enough to notice that that "free to use" library from Sun/Oracle may not be shipped as something that is supposedly "open source"... well, might even be a matter of opinion, as the software itself is e.g. under ASL, but it depends on something that is not open source. That is of course unacceptable to _us_ as we build a distribution that only ships open source components and we cannot depend on something that isn't. I run into such situations occasionally and use alternative implementations of those standards (as said, usually from the Glassfish, JBoss or Apache projects). Just replace one jar file with the other, done. From a copyright compatibility point of view: I think that there is rarely any "pure GPL" software written in Java, it's almost always "GPL with exception", because the GPL was written with C in mind (see all the text about "linking" and "aggregating") but it doesn't clearly relate to how Java works. The "GPL with exception" permits using something (say, a jar file) that is under that "GPL with exception" license with another jar that is under Apache license, etc..., and also clarifies that you may write non GPL software that uses a jar file that is under that "GPL with exception" copyright. On a side note, the GPL 3 in combination with the ASL 2.0 has clarified that situation with regards to copyright compatibility (as in, they are explicitly compatible). Also, there have been numerous statements from the FSF that what the "GPL with exception" clarifies is permitted with the "pure GPL" (even 2.0) in their opinion (and I guess they should know it). cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf

On 04/20/2012 04:55 PM, Adrian Schröter wrote:

Am Freitag, 20. April 2012, 12:20:45 schrieb Pascal Bleser:

...

...

Am Donnerstag, 19. April 2012, 22:21:52 schrieb Angelos Tzotsos:

...

There was a small discussion among Application:Geo project members about this issue today. I was asked to bring this issue to this list so that we can all discuss it. The problem: Currently it is forbidden to package binaries in OBS without building from source code. This causes issues when we need to build very complex Java applications or web applications (eg like Geoserver, Geonetwork, JOSM etc). In Java it is nearly impossible to build everything without bringing jar binaries in the game at some point (eg maven depends on itself to build). If we want to do this without binaries it would take a very large effort for many packages and it would be a dependency nightmare for packagers. On the other hand, the build.o.o instance has open source and transparency in mind, so the policy to request sources and not binaries is fair. While I think it is not a good idea to have exceptions, some exceptions for very few packages to be able to boot strap may be okay from my POV. On the other hand, all other languages, like C/C++/C# or even pascal and haskal can boot strap themselfs. Why should java be an exception? Sorry, very long email ahead, but I'd like to clarify a few

On 2012-04-20 09:55:44 (+0200), Adrian Schröter wrote: points to not have to argue about them again and again ^^

Java can bootstrap itself. OpenJDK bootstraps from C (the kernel of the JVM (Java Virtual Machine) is written in C). Maven can bootstrap itself too, and does so, actually.

It _can_ be done, and is done for OpenJDK at least, that's not the issue. It is a question of effort vs benefit. I aggree.

...

If we take the example of Maven, which is particularily painful because it is pretty complex to build by bootstrapping and by using our "Linux" (rpm/deb) technical model for "packages" and "dependencies":

Maven, as most Java based things, is designed to be portable across all operating systems that can run a JVM (of a certain version) I admit, that I do not have any inside of maven. But when we can not create multiple packages, it is still better to create one big package, but generate everything from source there IMHO.

...

For that reason, and because another unfortunately quite prominent operating system does not have a comparable package management model (*), it does not use a portable way of installing components of software other than just files that are downloaded and put somewhere (in the case of Maven, that's ~/.m2/repository). On a side note, that is an approach you will find with -- as far as I know -- every other build tool for Java projects which does dependency management (ivy (which is an add-on for Ant), gradle, etc...) When it can not be done inside of rpm, maybe it is up to the users to install it with their tools (similar to gem). But it should not be part of our distro. And currently we consider everything inside of build.opensuse.org as a potential candidate for openSUSE (can be discussed as well of course).

...

+-+------------------------------------------------------------- | (*) To prevent off-topic discussions about Windows and MSI: +-+------------------------------------------------------------- |S| Yes, it has, to some extent, but it has always and partly |I| still is very difficult to use, and simply isn't the model |D| that has developed with its users: Windows users have always |E| and will always just download archives somewhere and use them | | from there, without much system integration -- nothing |N| *prevents* java archives to be used and handled centrally in |O| the same way as DLLs (well, sort of, for some of them ...) or |T| in the same way as it is done on Linux, but it is simply not |E| how Windows users and developers do it, for whatever reasons | | (I have a few ideas about those reasons but won't mention | | them because I don't want to be rude). | | | | In any way, let's not discuss this here, it is irrelevant. +-+-------------------------------------------------------------

Hence, we are faced with two different models of managing dependencies and, in a way, the Java approach is a lot more like MacOSX' DMG, or klik on Linux, or just prehistoric zip files on Windows, which is to include most dependencies (libraries, tools) as part of an "image" of some sort (which is usually a tarball or a zip file).

So, the real question is: do we want to fight an uphill battle against what 95% of Java developers are doing, or even forced to do as that's what their build environments are doing and transform every possible Java based piece of software (**) we want to have in the distro or repositories into our RPM philosophy/approach (***), or do we want to be more pragmatic about it (being aware of the tradeoffs) ?

(**) which spans a very broad horizon from desktop sorts of applications (including things like Eclipse and Netbeans) to server-room stuff like Tomcat or JBoss

(***) which is to remove all the Java libraries (jar files) that are shipped with a Java software distribution or retrieved using mechanisms such as Maven's, and instead have those as individual dependencies -- e.g. remove xerces from ant and have ant use a system-wide xercesImpl.jar from /usr/share/java/...

Okay, now that you've answered that question for yourself thinking about Vuze or Eclipse or Netbeans, ask yourself the same question again in the shoes of a system administrator with Tomcat or JBoss. Frankly, you as java developer. Are you using our rpms at all? Or do you prefer to install the latest version yourself as jar file?

In my personal POV, it is better not to ship this software then to break our promisses, we usually gave when shipping rpms. We do promise that the stuff is reviewed and we are able to fix it in a compatible way _ourself_. Anything where this is not true should not be part of our distro IMHO.

...

...

The other side are add-on packages for libs and applications. I personally see no reason why we should become just a binary only hoster here and have an exception just because it is written in java. Very clearly: convenience for users (which very explictly includes system administrators). You would win some users here and loose others, maybe more important ones on

... the other side. Eg. companies who want to preload our stuff, but we won't be able to give any guarantees for our stuff anymore.

...

...

...

Esp. when you look at the current Google vs. Oracle fight, it makes it actually obvious for me that we need to be able to do a legal review for the java stack. Are you sure that you understand what you are talking about ? I had esp. the copyright violation in mind. Oracle showed that they are willing to sue others when they copy non-free stuff from them. And exactly this information is lost when you have just jar files. Our license digger will be able to detect such copyright violations anymore (and it does find quite some issues in the factory submissions, all of them would be not deteced anymore).

...

...

The downside might be that we loose quite some packages because we do not have anyone able or willing to package it anymore. Can we live without Eclipse, josm and friends? Well, assuming you mean "without Eclipse *packages*" yep

...

, yes and no:

* yes, because there are other ways of installing java based applications and libraries (I personally just pull eclipse from eclipse.org and put them into some directory under my home), especially for desktop applications

* no if in any way we want to attract more developers (tools such as Eclipse, Netbeans, Maven, Ant are paramount to doing so, and I'm not only talking about Java developers)

* no if we want to provide packages of more server oriented applications such as Tomcat, JBoss, Wildfire, Tigase, ...), which are obviously much more interesting to install in a system-wide way using our usual mechanisms (rpm, zypp).

...

Open question is what is more important here. Certainly, but the other question is whether we want a holistic or a pragmatic approach. Or, rather: can we live with a more pragmatic approach ?

Personally, I believe that sticking to the holistic approach ("always build from source, always. period.") just for the sake of sticking to it is stupid. If we are aware of drawbacks, it should be perfectly fine to make exceptions, if the advantages are worth those drawbacks. Which kind of brings us back to your question :)

My personal opinion: hell yes, there are packages where the drawbacks are worth the benefits. But there are also lots of others where it isn't. I am still obsolute 100% behind that the openSUSE distro must be fully open source and being manageable by us (or everybody who is able to work on sources).

It might be okay that we define some other namespace in OBS where these rules are explicit not required. (Btw, the todays german court decision between YouTube& GEMA showed that courts may require us to screen submission from users before making them accessable. But that has only partly some effect here).

Some very good points there Adrian. I agree that openSUSE must remain 100% open source and if including Java binaries means less openness, then lets stick to not having some Java packages in OBS. Regards, Angelos -- Angelos Tzotsos Remote Sensing Laboratory National Technical University of Athens http://users.ntua.gr/tzotsos -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org

On 2012-04-20 15:55:35 (+0200), Adrian Schröter wrote:

Am Freitag, 20. April 2012, 12:20:45 schrieb Pascal Bleser: [...]

...

Maven, as most Java based things, is designed to be portable across all operating systems that can run a JVM (of a certain version)

I admit, that I do not have any inside of maven. But when we can not create multiple packages, it is still better to create one big package, but generate everything from source there IMHO.

The thing with Maven is that: - packaging Maven itself is very tricky precisely because of its bootstrapping: it downloads Maven modules to build itself after bootstrapping a small Maven core - Maven would require to be patched, too, to * avoid Maven downloading artifacts dynamically while building (although offline mode should be enough for that (mvn -o)) * look up libraries in our RPM based installation scheme (/usr/share/java/.../*.jar), which is quite different from Maven's (*) (*) Maven uses the following layout, which has established itself as a de-facto standard in the java world: $M2_REPO/groupId/artifactId/version/artifactId-version.jar e.g. ~/.m2/repository/net/sourceforge/cobertura/cobertura/1.9/cobertura-1.9.jar (groupId is "net.sourceforge.cobertura" here, which is why it maps to the filesystem and to HTTP URLs as "net/sourceforge/cobertura") I think it would make things a lot easier in many regards if instead of /usr/share/java/foo/foo.jar we were using the same scheme as Maven for java library RPMs, namely something like /usr/share/java/repository/net/sourceforge/cobertura/cobertura/1.9/cobertura-1.9.jar instead of /usr/share/java/cobertura/cobertura-1.9.jar That way, using Maven or Gradle or Ivy, it would be as simple as defining one of the repositories to be /usr/share/java/repository instead of having to implement our current scheme with patches and hacks in Maven.

...

For that reason, and because another unfortunately quite prominent operating system does not have a comparable package management model (*), it does not use a portable way of installing components of software other than just files that are downloaded and put somewhere (in the case of Maven, that's ~/.m2/repository). On a side note, that is an approach you will find with -- as far as I know -- every other build tool for Java projects which does dependency management (ivy (which is an add-on for Ant), gradle, etc...)

When it can not be done inside of rpm, maybe it is up to the users to install it with their tools (similar to gem). But it should not be part of our distro. And currently we consider everything inside of build.opensuse.org as a potential candidate for openSUSE (can be discussed as well of course).

Sure, that's not the issue. There are a few things that I believe would still be useful as RPMs though, specifically to have something to start/bootstrap whole chains of dependencies with. Specifically: * Maven * Ant * Gradle (those are the three most common build tools) Ant is already packaged in openSUSE since a long time (although the package is really weird and doesn't always work because it apparently strips off some parts of it or hides them into non-obvious subpackages, but I never investigated the details or a fix.) And Eclipse and Netbeans probably make a lot of sense too, as those are very widely use development environments, and not just for Java. [...]

...

So, the real question is: do we want to fight an uphill battle against what 95% of Java developers are doing, or even forced to do as that's what their build environments are doing and transform every possible Java based piece of software (**) we want to have in the distro or repositories into our RPM philosophy/approach (***), or do we want to be more pragmatic about it (being aware of the tradeoffs) ? [...] Okay, now that you've answered that question for yourself thinking about Vuze or Eclipse or Netbeans, ask yourself the same question again in the shoes of a system administrator with Tomcat or JBoss.

Frankly, you as java developer. Are you using our rpms at all? Or do you prefer to install the latest version yourself as jar file?

I don't, never. But I'm not sure I can be compared to the average developer who comes from a Windows or MacOSX background. And then there are a few desktop applications in Java that cater to users rather than developers. For sysadmins who run production systems, I can see the advantage of using RPM packages of containers such as Tomcat, Jetty, JBoss, etc... though.

In my personal POV, it is better not to ship this software then to break our promisses, we usually gave when shipping rpms. We do promise that the stuff is reviewed and we are able to fix it in a compatible way _ourself_. Anything where this is not true should not be part of our distro IMHO.

Okay, valid point, although it's not always true, there are exceptions already, such as Firefox, but the reasons are quite different (massive codebase for Firefox, and issues with build systems and dependency handling for java)

...

...

...

The other side are add-on packages for libs and applications. I personally see no reason why we should become just a binary only hoster here and have an exception just because it is written in java.

...

Very clearly: convenience for users (which very explictly includes system administrators).

You would win some users here and loose others, maybe more important ones on the other side. Eg. companies who want to preload our stuff, but we won't be able to give any guarantees for our stuff anymore.

I believe that not catering to developers is a huge mistake. Seriously, how many actual developers are there in our community? From what I can see, and that includes inside SUSE, very, very few. How much people do we have who are capable and willing to contribute to real development efforts in our project? Very, very few. I don't mean hacking some perl scripts, I mean proper software development. Larger stuff. That's certainly a different topic and debate to be had, but to me, those are linked.

...

...

...

Esp. when you look at the current Google vs. Oracle fight, it makes it actually obvious for me that we need to be able to do a legal review for the java stack.

...

Are you sure that you understand what you are talking about ?

I had esp. the copyright violation in mind. Oracle showed that they are willing to sue others when they copy non-free stuff from them. And exactly this information is lost when you have just jar files. Our license digger will be able to detect such copyright violations anymore (and it does find quite some issues in the factory submissions, all of them would be not deteced anymore).

Oracle did not show anything up to now, there are basing their claims on currently very dubious and unclear rules (especially regarding the term "API" and what _their_ interpretation of it is in court). There's an article or two on the current proceedings at groklaw, one can read it in more depth over there. And, furthermore, we are not implementing our own virtual machine based on Java, are we ? Those are completely different things. The documentation that Oracle has provided and shown during the trial up to now very clearly makes the point, again, if it had to be made at all, that there is no issue whatsoever with software that is developed using java and/or runs on java. Bar none. As with libraries written in C, it only comes down to the license of the library itself, and the compatibility to its dependencies. Exactly the same really. See the JVM as being the glibc, from a licensing point of view. And openjdk itself is GPL. Again, to make this very clear: there is _absolutely_ no issue with licensing and open source here, except those that we have with non-java components too (the license of the piece of software itself, whether it is accurate and doesn't have individual files that have different copyright headers, and the license compatibility with its dependencies, unless it's under Apache license because then we don't even have to care about license compatibility). If we package binary upstream tarballs as RPMs using their dependencies inside the package, then there is potentially an additional verification to be made on the license of those, as those jar files typically don't include information about their licenses -- that is, if we don't trust upstream to make the same verifications prior to releasing something. Except, again, if it's under Apache license because AFAICR it doesn't have any clauses on license compatibility towards dependencies. It can even depend on non open source bits, if I'm not mistaken. IANAL, but I'm pretty sure, and I could check in detail if needed, although we have more qualified people for that ^^ [...]

...

Certainly, but the other question is whether we want a holistic or a pragmatic approach. Or, rather: can we live with a more pragmatic approach ?

...

Personally, I believe that sticking to the holistic approach ("always build from source, always. period.") just for the sake of sticking to it is stupid. If we are aware of drawbacks, it should be perfectly fine to make exceptions, if the advantages are worth those drawbacks. Which kind of brings us back to your question :)

...

My personal opinion: hell yes, there are packages where the drawbacks are worth the benefits. But there are also lots of others where it isn't.

I am still obsolute 100% behind that the openSUSE distro must be fully open source and being manageable by us (or everybody who is able to work on sources).

There is no discussion to be had about the open source aspect, because we all agree on that, and because all those java libraries/dependencies/applications _are_ open source (the vast majority being under http://spdx.org/licenses/Apache-2.0 which doesn't have any copyright compatibility issues with dependencies). (Yes, it's the third time I'm saying that, but I want this to be _very_ clear :))

It might be okay that we define some other namespace in OBS where these rules are explicit not required. (Btw, the todays german court decision between YouTube & GEMA showed that courts may require us to screen submission from users before making them accessable. But that has only partly some effect here).

I'm not sure how that relates to this matter. Do you mean we would need to make complete source code reviews before accepting anything? Then move everything to another country, because that's simply not feasible, regardless of java or not. cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf

On 2012-04-23 10:08:43 (+0200), Ludwig Nussel wrote:

Pascal Bleser wrote:

...

[...] I think it would make things a lot easier in many regards if instead of /usr/share/java/foo/foo.jar we were using the same scheme as Maven for java library RPMs, namely something like

...

/usr/share/java/repository/net/sourceforge/cobertura/cobertura/1.9/cobertura-1.9.jar

...

instead of

...

/usr/share/java/cobertura/cobertura-1.9.jar

...

That way, using Maven or Gradle or Ivy, it would be as simple as defining one of the repositories to be /usr/share/java/repository instead of having to implement our current scheme with patches and hacks in Maven.

And to add to that, that's really as simple as patching a Maven package or adding a few words of XML in Maven's settings.xml e.g. <repositories> <repository> <id>system</id> <name>System wide RPM repository</name> <url>file:///usr/share/java/repository</url> <layout>default</layout> <snapshots> <enabled>false</enabled> <updatePolicy>never</updatePolicy> </snapshots> <releases> <enabled>true</enabled> <updatePolicy>never</updatePolicy> </releases> </repository> </repositories> Note that we might also be able to take another approach and keep the current scheme but implement a new <layout/> and add that to our Maven package (which doesn't exist yet, as far as I know) which would then map ${groupId}, ${artifactId} and ${version} (commonly referred to as "GAV") to our current layout, something like file:///usr/share/java/${artifactId}/${artifactId}-${version}.jar if I remember correctly (and one can notice that the ${groupId} information is not present in our current scheme *g*). But then again, the names of our java packages (I don't mean openjdk) don't necessarily reflect their de-facto official names as defined in the central Maven repository (e.g. Google's Guava library is groupId="com.google.guava", artifactId="guava" while it might have been packaged as an RPM as java-google-guava or whatever, which wouldn't match.)

That sounds like a very low hanging fruit. What's the reason for the current scheme?

I have no idea, but I would assume not knowing about the Maven repository layout. I guess it stems from jpackage, which might actually be older than Maven ^^ I really don't know, but the current layout really doesn't have any advantages at all, none. cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf