influxdb2: nodejs packaging help required
Hi, I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently. I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball. -- With best regards, Matwey V. Kornilov
On 2/23/21 11:11 PM, Matwey V. Kornilov wrote:
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb <https://github.com/influxdata/influxdb> that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
The most correct way would be to package each dependency individually following [1] which is obviously an insane, for some go packages we copy each library that we don't have a package for into its own see [2] for example, note that for a legal review we need to note the license of each lib. Unfortunately the only way that doesn't require some significant effort is to disable building the webui, you could however make it easy to renable with a define so if people really want it they can build the package themselves locally. 1. https://en.opensuse.org/openSUSE:Packaging_nodejs 2. https://build.opensuse.org/package/view_file/science:machinelearning/tensorf... -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hi Matwey, "Matwey V. Kornilov" <matwey.kornilov@gmail.com> writes:
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you. Hope this helps, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all. The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community. I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors. Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app. The result is not a problem created by "old vs new" or people "just not getting it". These are propaganda influences at best, and lies at worst. The problem is that the nodejs community lacks the foresight, engineering capability, and desire for cooperation -- and the solution is to simply drop the entire ecosystem until they redesign their dependency package management from the ground up. People with more influence than I have will need to see this and move forward with something of this nature for it to get better. Just simply drop them, and get the other distros to as well. This never should have gotten this far. -C On Wed, Feb 24, 2021 at 1:34 AM Dan Čermák <dcermak@suse.com> wrote:
Hi Matwey,
"Matwey V. Kornilov" <matwey.kornilov@gmail.com> writes:
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
Hope this helps,
Dan
-- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany
(HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
To clarify on my previous message -- I am suggesting to drop the entire nodejs ecosystem any related packages, not just related to influxdb. I am also suggesting that counterparts in other distributions also should do this. -C On Wed, Feb 24, 2021 at 2:26 AM Chris Punches <punches.chris@gmail.com> wrote:
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
The result is not a problem created by "old vs new" or people "just not getting it". These are propaganda influences at best, and lies at worst. The problem is that the nodejs community lacks the foresight, engineering capability, and desire for cooperation -- and the solution is to simply drop the entire ecosystem until they redesign their dependency package management from the ground up.
People with more influence than I have will need to see this and move forward with something of this nature for it to get better. Just simply drop them, and get the other distros to as well.
This never should have gotten this far.
-C
On Wed, Feb 24, 2021 at 1:34 AM Dan Čermák <dcermak@suse.com> wrote:
Hi Matwey,
"Matwey V. Kornilov" <matwey.kornilov@gmail.com> writes:
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
Hope this helps,
Dan
-- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany
(HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Chris Punches <punches.chris@gmail.com> writes:
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
This suggestion is not going to solve anything whatsoever. The nodejs community doesn't care about distribution packaging, because they just `npm install` their dependencies into some outdated Ubuntu or Alpine docker container (most often from their Mac or Windows machine). The best you're going to get from them is a shrug. On the other hand, actual users of openSUSE and all other Linux distributions will no longer have convenient access to applications leveraging node modules via their OS' package manager. So by dropping every node package, we're only going to make matters worse and us as a distro less relevant. I fail to see how this should improve anything. Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
I do agree that their development communities would shrug at this, but, if every distro drops the nodejs ecosystem from their package repos, the applications leveraging node modules that the users get currently through their package managers -- (most of the notables they aren't getting them from official repos because it's such a flaky set of workarounds) -- will fall into obscurity, and quickly. Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the package manager (if it ever was, let's be real). That's an environmental stressor for an actor in a system: If there is need for the utility provided, that demand will generate alternatives. If there's not, then those apps, if they want to remain relevant, will end up being rewritten in more sustainable runtimes. The nodejs developers would quickly realize the importance of these problems they're creating and come up with solutions if the runtime is to survive. If all these distros continue to grind their wheels toiling at and around this problem that needs solved upstream it takes the pressure off upstream to solve it. While I'm taking shots at things, librpm needs modernized too. The "alternative package management" problem is springing up bad solutions that are harmful to the systems they operate on. Another topic but with similar drivers. -C On Wed, Feb 24, 2021 at 2:57 AM Dan Čermák <dcermak@suse.com> wrote:
Chris Punches <punches.chris@gmail.com> writes:
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
This suggestion is not going to solve anything whatsoever. The nodejs community doesn't care about distribution packaging, because they just `npm install` their dependencies into some outdated Ubuntu or Alpine docker container (most often from their Mac or Windows machine). The best you're going to get from them is a shrug.
On the other hand, actual users of openSUSE and all other Linux distributions will no longer have convenient access to applications leveraging node modules via their OS' package manager.
So by dropping every node package, we're only going to make matters worse and us as a distro less relevant. I fail to see how this should improve anything.
Cheers,
Dan
-- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany
(HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Chris Punches <punches.chris@gmail.com> writes:
I do agree that their development communities would shrug at this, but, if every distro drops the nodejs ecosystem from their package repos, the applications leveraging node modules that the users get currently through their package managers -- (most of the notables they aren't getting them from official repos because it's such a flaky set of workarounds) -- will fall into obscurity, and quickly.
Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the package manager (if it ever was, let's be real). That's an environmental stressor for an actor in a system: If there is need for the utility provided, that demand will generate alternatives. If there's not, then those apps, if they want to remain relevant, will end up being rewritten in more sustainable runtimes.
The nodejs developers would quickly realize the importance of these problems they're creating and come up with solutions if the runtime is to survive.
Do you really think that nodejs modules are primarily consumed via the OS' package manager? Because they are not. The majority of the node ecosystem is consumed via npm. npm itself is also not installed via zypper or dnf but is pre-installed inside the container used by the dev or just grabbed directly via `curl $URL | sudo sh`. Do I like this? Not at all. Will anything improve if we remove every nodejs package? Not at all, things will just get worse for users of the distro, while the node ecosystem *will not care*. You're of course free to submit delete requests to Factory, but don't except any support with that.
If all these distros continue to grind their wheels toiling at and around this problem that needs solved upstream it takes the pressure off upstream to solve it.
What we actually need, is a way to ship node modules in a sane way and provide additional value by doing that. Adam's and Ludwig's nodejs bundler does both and is imho a step into the right direction. Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Wednesday, February 24, 2021 9:05:27 AM CET Chris Punches wrote:
Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the package manager (if it ever was, let's be real). That's an environmental stressor for an actor in a system: If there is need for the utility provided, that demand will generate alternatives. If there's not, then those apps, if they want to remain relevant, will end up being rewritten in more sustainable runtimes.
Ahh but this is the problem. openSUSE needs to include some code that use npm, like cockpit. And what is worse, a generalization of this argument will also drop Python, Ruby, Go or Rust code, or basically all that is not C/C++. I agree that nodejs ecosystem is an extreme case, but in any case there is code living there that we need in some openSUSE flavors / components. -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On 2/24/21 7:34 AM, Dan Čermák wrote:
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
This is exactly correct. To build in OBS, one needs to bundle dependencies. What we've done now is to bundle them for a localhost npm proxy that then allows you to run `npm install` inside OBS. An example of a spec file that would use this is, osc less home:adamm:branches:systemsmanagement:cockpit/cockpit-podman cockpit-podman.spec The node_modules service is run locally with a package-lock.json file as per link from Dan. - Adam
participants (6)
-
Adam Majer
-
Alberto Planas
-
Chris Punches
-
Dan Čermák
-
Matwey V. Kornilov
-
Simon Lees