On 2011-08-15 00:19:38 (-0700), Peter Linnell
I tested this and it mostly just works TM. Not bad for a prototype :)
Yeah, I managed to package spring framework (spring-beans) and all its dependencies with it, but just locally, I didn't commit it to any repository (yet).
On 08/14/2011 04:13 PM, Pascal Bleser wrote:
I've written a successful prototype that does the following: * generates a completely functional .spec file based on the metadata of java artifacts (POM) that is available at Maven Central (http://repo1.maven.org/maven2/) * does *NOT* build from source but, instead, simply takes the .jar file from Maven Central and installs it into %{_javadir} (including the usual symlinks) * optionally also creates a subpackage with the source jar from Maven Central (very useful for IDEs such as Eclipse) * all that including dependencies, as far as they're declared in the POM file * written in Perl (hey, it's a prototype ;)), just one file, no exotic dependencies
The script is there: https://gitorious.org/pbleser/pbleser/blobs/master/pom2spec
Here's an example on how to run it: 1) pull the script and chmod it: cd ~/bin wget https://gitorious.org/pbleser/pbleser/blobs/raw/master/pom2spec chmod +x pom2spec
2) go to your OBS sandbox and create a new package: osc mkpac log4j-log4j-1_2_15
This part failed for me. But I am on a new laptop, so I needed to init my home repo locally.
That's supposed to be ran from your osc sandbox, from a checked out project directory.
3) generate the .spec file, passing groupId artifactId and version: pom2spec log4j log4j 1.2.15
(if you don't specify the version, pom2spec will give you a list of possible versions, i.e. those that are available in Maven Central)
4) well, obviously, build it ;) osc build ...
Missing step, move the spec into the project/package directory.
Yeah, I forgot to mention: 2b) cd log4j-log4j-1_2_15 ;)
===== The longer story... [...] Hence, I believe that having them as "bad" RPMs (because they're not built from source but just install the already compiled .jar files from Maven Central (hence upstream)) is better than not having them _at all_. Of course, starting from there, we should still continue to improve our tool set and experience with building Java packages from source, in order to replace at least some of those "bad" RPMs with "good" ones.
I see this as as tool and a repo to bootstrap building more java packages. We may need a separate repo just so the "bad" rpms intially.
Yes, that might be an option. But really, as long as we don't need to apply patches to the upstream sources (never seen the need for that), that situation is perfectly fine IMHO. *And* provided we don't do backport patches.
The drawback is that we cannot add our own patches to the upstream sources, for obvious reasons. But the advantage of this approach is that it is very easy to build the RPMs. And, to be honest, I don't think that, in the past or in the future, we often need to add patches to portable Java source code that haven't been included upstream already.
99% of the patching I have seen to java packages are to the build.xml files for messing with class paths or install paths. I cannot recall ever seeing the sources themselves patched in rpm packages.
Me neither. At worst, it means updating the package to a git/svn snapshot of upstream (e.g. log4j-log4j-1_2_15-1.2.15+r9876)
Sure, that doesn't work well with the typical policy for the distribution updates (backport patches instead of version bumps) but hey... better than nothing, don't you think ?
Yes and in the java world backports really do not exist afaik. Want a bug fixed ? Update..
Right. There are no SONAMEs in Java and while several projects are very careful with not breaking backwards API compatibility (there are well-defined rules for ABI compatibility on bytecode level too, not sure many projects pay attention to that though), most don't. Well. Pretty much the same as with C/C++ anyway, SONAMEs or not (at least from my experience). A consequence, I believe, is that we would be better off providing several versions, even minor, of the same libraries. For dependencies, we should just use the required versions of those dependencies as they are defined by the upstream projects in their POM file. Sure, we might (and would/will) end up with foo.bar -> org.spam-spam-1_0 = 1.0 and com.vikings -> org.spam-spam-1_1 = 1.1 (or whatever the package naming will be) But hey, that's what the upstream devs have defined as versions in the POM file. Might as well run it with that instead of the latest (which might not be compatible at all, neither on API nor ABI level). [...]
That (${ŋroupId}:${artifactId}) already is a unique and canonical name as of Maven Central, which is what 90%+ Java projects on the planet use (either through Maven, or Ant+Ivy, or Gradle, or ...).
But that's too long for Joliet. Is that still an issue ?
Works for me. I did not see an issue testing. Where I am more curious is how we map the provides auto generated by rpm vs build requires in the spec files generated from the pom file. That will need some attention it seems.
We could certainly try to add something to RPM in order to autoreq some Provides and Requires. The Provides and Requires I mentioned are generated by the pom2spec script itself. Quite some time ago, I wrote a prototype that can determine Requires: and Provides: automatically from analyzing the bytecode (in .jar or .class files). It uses jdepend [1]. Don't remember where it is, but if someone is interested, I can dig it out, must be somewhere on my harddisk. I even managed to compile it into a native binary with gcj ;) [1] http://www.clarkware.com/software/JDepend.html We'll end up with *very* long lists of dependencies though, and they wouldn't necessarily be correct -- as an example, Spring Framework is an integration layer with many technologies and libraries and for most of them, it does not require them to be present at runtime (e.g. if you don't use the JMS support, you don't need to have the JMS jars in the classpath at runtime) -- Java uses late binding at runtime, a .jar file is completely different from a .so. Having the pom2spec generate both the Requires and the Provides gives better results in my opinion: * sure, they *might* be wrong, as we don't analyze the bytecode to find *all* dependencies with 100% accuracy (jdepend can give us that) * the Requires are those that have been specified by upstream in the POM, and they ought to know which dependencies make sense, and which don't (see the spring framework example above) Given that an enormous amount of open source and non open source Java projects on the planet (and that's indeed quite a huge amount) use and depend on those POMs to be at least partially correct, and very much so in their dependency information, I don't doubt that cases where the POM is actually missing dependencies will be rare. And fixed by upstream if we file it as a bug.
Another thing is that I believe that it would make our life easier if we also included the ${version} in the %{NAME} of the package, in order to be able to install several different versions of libraries side-by-side. That way we can always reflect upstream's dependencies as defined in their POM file (e.g. org.springframework:spring-beans:2.5.6.SEC02 requiring log4j:log4j:1.2.15).
Example: Name: org.springframework.spring-beans-2_5_6_SEC02 Version: 2.5.6.SEC02
Yes, that's a pretty long package name alright.
That would also mean dropping the symlink that drops the %{version} in %{_javadir}.
How will this affect runtimes ? The rest we can live with.
It does not affect libraries at all. It only affects applications (server or desktop). The shell script wrapper that starts e.g. Netbeans, or Tomcat, or ... needs to build the CLASSPATH environment variable accordingly. For transitive dependencies, the pom2spec script also generates .classpath files (e.g. /usr/share/java/foo.org-spam-1.2.3.classpath) that can be aggregated. With what I've hacked in pom2spec, it could be something like this: ---8<----------------------------------------------------- #!/bin/sh CLASSPATH="" for d in \ org.springframework.spring-beans-2.5.6.SEC02 \ ... \ ... \ ; do CLASSPATH="$CLASSPATH:$(cat /usr/share/java/$d)" done CLASSPATH="${CLASSPATH#:}" export CLASSPATH exec java -jar \ /usr/share/java/my.awesome.application-1.2.jar ---8<----------------------------------------------------- But that's just an idea, if someone has a better one... ;) From my very long experience as a Java developer, I certainly find it much more deterministic to always run an application against the exact dependencies that upstream has defined (e.g. log4j 1.2.11 and not 1.2.15 because 1.2.15 happens to be the latest). [...]
Thanks for taking this on.. Java packaging has become a pain point and I do not want to see either openSUSE nor SLES become less usable as a java platform.
That would be extremely silly too. I'm quite aware that most SUSErs or even people in the community at large hate Java for whatever historic or religious reasons, but the fact is that there is a huge amount of open source projects out there in Java, and that's not going to go away any time soon. Especially if we want to also go into "the cloud" (blech, buzzword), that's really quite Java centric (Hadoop, Solr, Infinispan, OpenFire/Tigase, etherpad, .........). cheers -- -o) Pascal Bleser /\\ http://opensuse.org -- we haz green _\_v http://fosdem.org -- we haz conf