influxdb2: nodejs packaging help required
Hi, I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently. I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball. -- With best regards, Matwey V. Kornilov
On 2/23/21 11:11 PM, Matwey V. Kornilov wrote:
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
The most correct way would be to package each dependency individually following [1] which is obviously an insane, for some go packages we copy each library that we don't have a package for into its own see [2] for example, note that for a legal review we need to note the license of each lib. Unfortunately the only way that doesn't require some significant effort is to disable building the webui, you could however make it easy to renable with a define so if people really want it they can build the package themselves locally. 1. https://en.opensuse.org/openSUSE:Packaging_nodejs 2. https://build.opensuse.org/package/view_file/science:machinelearning/tensorf... -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hi Matwey,
"Matwey V. Kornilov"
Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency
explosion and Ludwig kindly pointed me to
https://github.com/openSUSE/obs-service-node_modules which takes care of
the bundling for you.
Hope this helps,
Dan
--
Dan Čermák
If I can offer an outside opinion... after watching how the nodejs
ecosystem interacts with the rest of the os on these systems....after
observing the issues packagers are constantly running into with nodejs
packages when packaging for other distros....as well as opensuse -- my
observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's
junkware. It was never meant not to be outside of rare instances of
extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to
'junkware', but, it makes no attempt to fix their broken package management
model, is a vessel for security problems to otherwise secure operating
systems. They know these problems exist. They do not desire to fix them.
They view their way of package distribution as superior, without any regard
for the rest of the system -ir even the environments in their own use cases
when designing this architecture- because that community comes from a
culture that does not acknowledge that the OS exists. I have heard these
conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models
because every distro is trying hard to integrate their poorly designed
package delivery pipelines and dependency models for no other reason than
that alot of software that should not have been written in this runtime,
was written in this runtime, and we all need the latest app.
The result is not a problem created by "old vs new" or people "just not
getting it". These are propaganda influences at best, and lies at worst.
The problem is that the nodejs community lacks the foresight, engineering
capability, and desire for cooperation -- and the solution is to simply
drop the entire ecosystem until they redesign their dependency package
management from the ground up.
People with more influence than I have will need to see this and move
forward with something of this nature for it to get better. Just simply
drop them, and get the other distros to as well.
This never should have gotten this far.
-C
On Wed, Feb 24, 2021 at 1:34 AM Dan Čermák
Hi Matwey,
"Matwey V. Kornilov"
writes: Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
Hope this helps,
Dan
-- Dan Čermák
Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
To clarify on my previous message -- I am suggesting to drop the entire
nodejs ecosystem any related packages, not just related to influxdb. I am
also suggesting that counterparts in other distributions also should do
this.
-C
On Wed, Feb 24, 2021 at 2:26 AM Chris Punches
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
The result is not a problem created by "old vs new" or people "just not getting it". These are propaganda influences at best, and lies at worst. The problem is that the nodejs community lacks the foresight, engineering capability, and desire for cooperation -- and the solution is to simply drop the entire ecosystem until they redesign their dependency package management from the ground up.
People with more influence than I have will need to see this and move forward with something of this nature for it to get better. Just simply drop them, and get the other distros to as well.
This never should have gotten this far.
-C
On Wed, Feb 24, 2021 at 1:34 AM Dan Čermák
wrote: Hi Matwey,
"Matwey V. Kornilov"
writes: Hi,
I would like to package influxdb2: https://github.com/influxdata/influxdb that is the next major version for server:database/influxdb Probably, they will coexist for some time, but it doesn't matter currently.
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
Hope this helps,
Dan
-- Dan Čermák
Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Chris Punches
If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
This suggestion is not going to solve anything whatsoever. The nodejs
community doesn't care about distribution packaging, because they just
`npm install` their dependencies into some outdated Ubuntu or Alpine
docker container (most often from their Mac or Windows machine). The
best you're going to get from them is a shrug.
On the other hand, actual users of openSUSE and all other Linux
distributions will no longer have convenient access to applications
leveraging node modules via their OS' package manager.
So by dropping every node package, we're only going to make matters
worse and us as a distro less relevant. I fail to see how this should
improve anything.
Cheers,
Dan
--
Dan Čermák
I do agree that their development communities would shrug at this, but, if
every distro drops the nodejs ecosystem from their package repos, the
applications leveraging node modules that the users get currently through
their package managers -- (most of the notables they aren't getting them
from official repos because it's such a flaky set of workarounds) -- will
fall into obscurity, and quickly.
Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the
package manager (if it ever was, let's be real). That's an environmental
stressor for an actor in a system: If there is need for the utility
provided, that demand will generate alternatives. If there's not, then
those apps, if they want to remain relevant, will end up being rewritten in
more sustainable runtimes.
The nodejs developers would quickly realize the importance of these
problems they're creating and come up with solutions if the runtime is to
survive.
If all these distros continue to grind their wheels toiling at and around
this problem that needs solved upstream it takes the pressure off upstream
to solve it.
While I'm taking shots at things, librpm needs modernized too. The
"alternative package management" problem is springing up bad solutions that
are harmful to the systems they operate on. Another topic but with similar
drivers.
-C
On Wed, Feb 24, 2021 at 2:57 AM Dan Čermák
Chris Punches
writes: If I can offer an outside opinion... after watching how the nodejs ecosystem interacts with the rest of the os on these systems....after observing the issues packagers are constantly running into with nodejs packages when packaging for other distros....as well as opensuse -- my observed opinion is that no distro should package them at all.
The maintainers for npm simply do not care to integrate with the os. It's junkware. It was never meant not to be outside of rare instances of extreme hubris in the javascript community.
I know it's not good to call something so many people contribute to 'junkware', but, it makes no attempt to fix their broken package management model, is a vessel for security problems to otherwise secure operating systems. They know these problems exist. They do not desire to fix them. They view their way of package distribution as superior, without any regard for the rest of the system -ir even the environments in their own use cases when designing this architecture- because that community comes from a culture that does not acknowledge that the OS exists. I have heard these conversations take place on development shop floors.
Drop them. The nodejs maintainers get away with these arrogant models because every distro is trying hard to integrate their poorly designed package delivery pipelines and dependency models for no other reason than that alot of software that should not have been written in this runtime, was written in this runtime, and we all need the latest app.
This suggestion is not going to solve anything whatsoever. The nodejs community doesn't care about distribution packaging, because they just `npm install` their dependencies into some outdated Ubuntu or Alpine docker container (most often from their Mac or Windows machine). The best you're going to get from them is a shrug.
On the other hand, actual users of openSUSE and all other Linux distributions will no longer have convenient access to applications leveraging node modules via their OS' package manager.
So by dropping every node package, we're only going to make matters worse and us as a distro less relevant. I fail to see how this should improve anything.
Cheers,
Dan
-- Dan Čermák
Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Chris Punches
I do agree that their development communities would shrug at this, but, if every distro drops the nodejs ecosystem from their package repos, the applications leveraging node modules that the users get currently through their package managers -- (most of the notables they aren't getting them from official repos because it's such a flaky set of workarounds) -- will fall into obscurity, and quickly.
Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the package manager (if it ever was, let's be real). That's an environmental stressor for an actor in a system: If there is need for the utility provided, that demand will generate alternatives. If there's not, then those apps, if they want to remain relevant, will end up being rewritten in more sustainable runtimes.
The nodejs developers would quickly realize the importance of these problems they're creating and come up with solutions if the runtime is to survive.
Do you really think that nodejs modules are primarily consumed via the OS' package manager? Because they are not. The majority of the node ecosystem is consumed via npm. npm itself is also not installed via zypper or dnf but is pre-installed inside the container used by the dev or just grabbed directly via `curl $URL | sudo sh`. Do I like this? Not at all. Will anything improve if we remove every nodejs package? Not at all, things will just get worse for users of the distro, while the node ecosystem *will not care*. You're of course free to submit delete requests to Factory, but don't except any support with that.
If all these distros continue to grind their wheels toiling at and around this problem that needs solved upstream it takes the pressure off upstream to solve it.
What we actually need, is a way to ship node modules in a sane way and
provide additional value by doing that. Adam's and Ludwig's nodejs
bundler does both and is imho a step into the right direction.
Cheers,
Dan
--
Dan Čermák
On Wednesday, February 24, 2021 9:05:27 AM CET Chris Punches wrote:
Sure, SuperMegaApp3000 in nodejs won't be accessible to users through the package manager (if it ever was, let's be real). That's an environmental stressor for an actor in a system: If there is need for the utility provided, that demand will generate alternatives. If there's not, then those apps, if they want to remain relevant, will end up being rewritten in more sustainable runtimes.
Ahh but this is the problem. openSUSE needs to include some code that use npm, like cockpit. And what is worse, a generalization of this argument will also drop Python, Ruby, Go or Rust code, or basically all that is not C/C++. I agree that nodejs ecosystem is an extreme case, but in any case there is code living there that we need in some openSUSE flavors / components. -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On 2/24/21 7:34 AM, Dan Čermák wrote:
I've found that influxdb2 is provided with a built in WebUI interface based on nodejs(?) and it requires about 2000 packages from npm to be built. Even though I can do it manually, there is no internet connection in OBS. I have not found any info on the wifi how to overcome this issue. Probably I need to preferetch these sources from npm and put them as a tarball.
I have run into exactly the same problem with node's dependency explosion and Ludwig kindly pointed me to https://github.com/openSUSE/obs-service-node_modules which takes care of the bundling for you.
This is exactly correct. To build in OBS, one needs to bundle dependencies. What we've done now is to bundle them for a localhost npm proxy that then allows you to run `npm install` inside OBS. An example of a spec file that would use this is, osc less home:adamm:branches:systemsmanagement:cockpit/cockpit-podman cockpit-podman.spec The node_modules service is run locally with a package-lock.json file as per link from Dan. - Adam
participants (6)
-
Adam Majer
-
Alberto Planas
-
Chris Punches
-
Dan Čermák
-
Matwey V. Kornilov
-
Simon Lees