Packaging applications requiring nodejs libraries
Hi list,
do we have any guidelines for packaging applications that require nodejs
libraries?
The new release of rstudio includes a web based editor written in
typescript. And as usual, it has a *huge* dependency graph. Would it be
fine if I just bundle them inside rstudio, because there's no way that
I'll be able to package the full dependency graph of those node modules.
Thanks in advance,
Dan
--
Dan Čermák
Am 18.01.21 um 17:06 schrieb Dan Čermák
Hi list,
do we have any guidelines for packaging applications that require nodejs libraries?
The new release of rstudio includes a web based editor written in typescript. And as usual, it has a *huge* dependency graph. Would it be fine if I just bundle them inside rstudio, because there's no way that I'll be able to package the full dependency graph of those node modules.
Thanks in advance,
Dan
There is https://en.opensuse.org/openSUSE:Packaging_nodejs TL;DR: It is hard, if not impossible. Many python packages don't touch the bundled nodejs resources, because unbundling them is next to impossible most of the time.
Am Montag, 18. Januar 2021, 17:12:37 CET schrieb Ben Greiner:
Many python packages don't touch the bundled nodejs resources, because unbundling them is next to impossible most of the time.
Yes...I tried to package tryton-sao for some time, and it was a nightmare - even though I had support from MargueriteSu. There was a change every day. I dropped it finally. nodejs is not made for RPM packaging.... My 2c Axel
Dan Čermák wrote:
do we have any guidelines for packaging applications that require nodejs libraries?
The new release of rstudio includes a web based editor written in typescript. And as usual, it has a *huge* dependency graph. Would it be fine if I just bundle them inside rstudio, because there's no way that I'll be able to package the full dependency graph of those node modules.
For those not into Node.js.. huge really means huge in this context. For cockpit that only uses some Nodejs for minifying js and css at build time it's 1000+ tarballs. Sometimes four versions of the same module at the same time. So insane that it's basically impossible to either package individually, nor to manage manually in any way. The only chance is to bundle. I hate packages that have some magic "vendor.tar.gz" that nobody knows how it was created though. So I came up with an obs service¹ to handle the nodejs modules. It manages source lines in the spec file generated from package-lock.json and also downloads the tarballs. So the package actually refers to pristine upstream sources and the packager does not have to manually deal with them. Downside is that you end up with a package that has more than a thousand source tarballs. That is only viable when done on server side. Otherwise you wouldn't see the forest for all the trees anymore. Iow service mode "enabled" is required, ie can't be included in the distro so far. I'm trying to figure out ways to get around that atm. The obs service is deployed on build.opensuse.org already so you can play with it. Adam just wrote an npm proxy² so "npm install" can be used offline at build time. To be integrated. IOW no turnkey solution yet, all WIP. cu Ludwig [1] https://github.com/openSUSE/obs-service-node_modules [2] https://github.com/openSUSE/npm-localhost-proxy -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
Hi,
Just some info: Fedora will drop those gillionish nodejs packages entirely
in next version, and endorse the "vendoring"[0]. Debian discussed it too
recently [1]
[0] https://fedoraproject.org/wiki/Changes/NodejsLibrariesBundleByDefault
[1] https://lwn.net/SubscriberLink/842319/8adb13e08d0302bd/
shenle
On Mon, Jan 18, 2021 at 12:13 PM Ludwig Nussel
Dan Čermák wrote:
do we have any guidelines for packaging applications that require nodejs libraries?
The new release of rstudio includes a web based editor written in typescript. And as usual, it has a *huge* dependency graph. Would it be fine if I just bundle them inside rstudio, because there's no way that I'll be able to package the full dependency graph of those node modules.
For those not into Node.js.. huge really means huge in this context. For cockpit that only uses some Nodejs for minifying js and css at build time it's 1000+ tarballs. Sometimes four versions of the same module at the same time. So insane that it's basically impossible to either package individually, nor to manage manually in any way. The only chance is to bundle. I hate packages that have some magic "vendor.tar.gz" that nobody knows how it was created though. So I came up with an obs service¹ to handle the nodejs modules. It manages source lines in the spec file generated from package-lock.json and also downloads the tarballs. So the package actually refers to pristine upstream sources and the packager does not have to manually deal with them. Downside is that you end up with a package that has more than a thousand source tarballs. That is only viable when done on server side. Otherwise you wouldn't see the forest for all the trees anymore. Iow service mode "enabled" is required, ie can't be included in the distro so far. I'm trying to figure out ways to get around that atm. The obs service is deployed on build.opensuse.org already so you can play with it.
Adam just wrote an npm proxy² so "npm install" can be used offline at build time. To be integrated. IOW no turnkey solution yet, all WIP.
cu Ludwig
[1] https://github.com/openSUSE/obs-service-node_modules [2] https://github.com/openSUSE/npm-localhost-proxy
-- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On 1/19/21 4:50 AM, slb wrote:
Hi,
Just some info: Fedora will drop those gillionish nodejs packages entirely in next version, and endorse the "vendoring"[0]. Debian discussed it too recently [1]
[0] https://fedoraproject.org/wiki/Changes/NodejsLibrariesBundleByDefault https://fedoraproject.org/wiki/Changes/NodejsLibrariesBundleByDefault [1] https://lwn.net/SubscriberLink/842319/8adb13e08d0302bd/ https://lwn.net/SubscriberLink/842319/8adb13e08d0302bd/
Hi all, Indeed. I was always against doing "simple" things like npm2rpm - it's just an exercise in repackaging from one container to another. It's busy work that is actually detrimental to user experience. In case of NPM packages, the user experience is to be able to run `npm install` `npm install` requires network, but it can be run against a localhost network and hence the idea of just having a simple registry on localhost that contains all the required packages. Done. I can say the problem has been solved although implementation is WIP. There are two other problems that need resolutions to be able to actually contribute positively to the current NPM ecosystem. 1. how to provide security updates for released NPM modules? (conceptually solved) 2. how to rebuild/update packages that BR these NPM modules. I think this will require a special OBS service. Anyway, the future of NPM in openSUSE looks more like NPM than RPM. And for applications that require NPM to build their RPMs, the target process will have to be more transparent than opaque. More to come. - Adam
On Tuesday 2021-01-19 11:36, Adam Majer wrote:
There are two other problems that need resolutions to be able to actually contribute positively to the current NPM ecosystem.
1. how to provide security updates for released NPM modules? (conceptually solved)
npm kinda maneuvred itself into this situation that's equivalent to a system static archives - it would only be fair that npm got the security treatment of such archaic concepts. ;-)
2. how to rebuild/update packages that BR these NPM modules. I think this will require a special OBS service.
1. create openSUSE:Factory:TransitiveRebuildZone that has
participants (7)
-
Adam Majer -
Axel Braun -
Ben Greiner -
Dan Čermák -
Jan Engelhardt -
Ludwig Nussel -
slb