On 19 Nov 2019, at 10:43, William Brown wrote:

...

On 19 Nov 2019, at 10:39, Neal Gompa wrote:

On Mon, Nov 18, 2019 at 6:58 PM William Brown wrote:

...

Hi,

Recently I've been trying to package some utilities for SUSE and Fedora via OBS that are written in Rust. I'm running into a bit of a problem though.

Rust as a language has a very aggressive release cycle compared to what we expect on a platform like SUSE - 6 weeks (https://github.com/rust-lang/rust/blob/master/RELEASES.md). In each cycle Rust releases new features, and despite the core language being "stable" and safe (which is great!) new features tend to be used very rapidly by library authors. For example the release from Version 1.34.0 (2019-04-11) contains convert::TryFrom, which is now in use by a large number of libraries.

Rust also has an (unfortunate) requirement that you are essentially forced to use cargo which is a build and dependency management tool. Cargo is extremely opinionated and inflexible which makes it difficult to use. However as it's also a dependency management tool, this has encouraged an npm-style ecosystem of dependencies to spring up on https://crates.io/ - and it's effectively the only way to use libraries in Rust. But additionally, it's promoted a system where a library may have a large number of small dependencies too.

Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers.

Which leads me to the question of "what to do".

* I don't believe it's feasible to ask Rust to "slow down". It's just not going to happen, and they will keep adding features that people will "want" to use. * We can't expect people *not* to update their dependencies in projects as that would prevent security updates being included. So we have to accept projects that will use "latest and greatest" complier features somewhere in their dependency graph. * Which leaves distros (like us) speeding up our rust compiler cycle somehow.

For opensuse I can probably do something like adding devel:languages:rust as a respository to my project. Would this strategy be viable on SLE? What about for projects which we want to expose packages for fedora or other? Are we able to have toolchains move faster than our base system?

I'm really looking for ideas on how we can sustainably use rust projects within the governance and social constraints that exist in the space. Ideas?

Unfortunately, SUSE rust compiler packaging spec isn't the same as the RH/Fedora/Mageia one, so essentially it takes double the work to keep things up to date everywhere. If we integrated the SUSE support bits into the RH/Fedora/Mageia rust packaging spec, that could make things easier. It's not exactly the highest thing on my list to look into right now, though...

That said, Fedora and Mageia update the Rust compiler within a week of a new stable compiler release. As of right now, Fedora 29 and higher all have Rust 1.39.0. CentOS/RHEL 7 will receive Rust 1.39.0 soon too[1].

Hmmmm, I'm seeing that on Fedora 30 on OBS that this is failing due to a lack of try_from:

https://build.opensuse.org/package/live_build_log/home:firstyear:kanidm/kani...

So this indicates that our Fedora 30 in obs may not be getting these updates?

Okay, so when you do: Project -> Repositories -> Add from a distribution -> select fedora 30 That only adds fedora 30 standard: https://build.opensuse.org/repositories/Fedora:30 But not update. This is why all the f30 builds I have are failing. This seems like a bug or error to me.

...

Crate packaging and macros are completely in sync among Fedora, Mageia, and openSUSE. These are provided by the rust-packaging and rust-srpm-macros packages. This code is pretty much the same across all three distributions.

I find the process of shipping updates to SUSE distributions thoroughly annoying and unappealing, so I don't personally do anything to target anything other than Tumbleweed. The process is too slow because it's often bottlenecked on either legal-auto or the all-too-tiny number of reviewers for SRs and MRs. This is also true for Tumbleweed, but it seems to be somewhat faster there. I'd be more interested in helping ship updates to Leap if we could make this process less awful.

I think this process has to become less awful, else we'll be missing out on plenty of rust tooling that is quickly coming available for platforms.

...

[1]: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-037907db2c

-- 真実はいつも一つ！/ Always, there's only one truth!

— Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server SUSE Labs

-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 19 Nov 2019, at 11:04, Neal Gompa wrote:

On Mon, Nov 18, 2019 at 7:32 PM William Brown wrote:

...

Okay, so when you do:

Project -> Repositories -> Add from a distribution -> select fedora 30

That only adds fedora 30 standard:

https://build.opensuse.org/repositories/Fedora:30

But not update. This is why all the f30 builds I have are failing. This seems like a bug or error to me.

It is done this way to minimize automatic rebuilds by default, as new updated packages in the repositories will trigger rebuilds. If you want updates, just change "standard" to "update" and call it gravy. Most don't require this, but your Rust case probably will.

For what it's worth, openSUSE Leap targets are similarly set up.

This is really counter intuitive to anyone who is reasonably inexperienced at packaging (IE me), which is probably another barrier to getting people to do tasks like this. It could be improved both as a default (include updates because that's the expected behaviour) or via UI (offer updates as an option in the distribution menu, or make it easier to alter standard -> update in the respository list).

-- 真実はいつも一つ！/ Always, there's only one truth!

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 19 Nov 2019, at 11:19, Simon Lees wrote:

On 11/19/19 11:07 AM, William Brown wrote:

...

...

On 19 Nov 2019, at 11:04, Neal Gompa wrote:

On Mon, Nov 18, 2019 at 7:32 PM William Brown wrote:

...

Okay, so when you do:

Project -> Repositories -> Add from a distribution -> select fedora 30

That only adds fedora 30 standard:

https://build.opensuse.org/repositories/Fedora:30

But not update. This is why all the f30 builds I have are failing. This seems like a bug or error to me.

It is done this way to minimize automatic rebuilds by default, as new updated packages in the repositories will trigger rebuilds. If you want updates, just change "standard" to "update" and call it gravy. Most don't require this, but your Rust case probably will.

For what it's worth, openSUSE Leap targets are similarly set up.

This is really counter intuitive to anyone who is reasonably inexperienced at packaging (IE me), which is probably another barrier to getting people to do tasks like this. It could be improved both as a default (include updates because that's the expected behaviour) or via UI (offer updates as an option in the distribution menu, or make it easier to alter standard -> update in the respository list).

One reason that it is done like this by default is because core SLE / Leap packages shouldn't be break API/ABI for updates, so in the vast majority of cases building against standard is fine and saves huge amounts of resources on obs. You are only really hitting this issue because you are working with one of the few echosystems that as you say doesn't follow our traditional model.

Ecosystems are changing, and less and less are following the traditional model - I think we can't keep treating them as exceptions, because this will become the norm for many languages. So how can we improve this? How can we make rust a first class language for our platforms so that we can keep including relevant and modern tools in our platforms?

--

Simon Lees (Simotek) http://simotek.net

Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 19 Nov 2019, at 12:27, Simon Lees wrote:

On 11/19/19 11:22 AM, William Brown wrote:

...

...

On 19 Nov 2019, at 11:19, Simon Lees wrote:

On 11/19/19 11:07 AM, William Brown wrote:

...

...

On 19 Nov 2019, at 11:04, Neal Gompa wrote:

On Mon, Nov 18, 2019 at 7:32 PM William Brown wrote:

...

Okay, so when you do:

Project -> Repositories -> Add from a distribution -> select fedora 30

That only adds fedora 30 standard:

https://build.opensuse.org/repositories/Fedora:30

But not update. This is why all the f30 builds I have are failing. This seems like a bug or error to me.

It is done this way to minimize automatic rebuilds by default, as new updated packages in the repositories will trigger rebuilds. If you want updates, just change "standard" to "update" and call it gravy. Most don't require this, but your Rust case probably will.

For what it's worth, openSUSE Leap targets are similarly set up.

This is really counter intuitive to anyone who is reasonably inexperienced at packaging (IE me), which is probably another barrier to getting people to do tasks like this. It could be improved both as a default (include updates because that's the expected behaviour) or via UI (offer updates as an option in the distribution menu, or make it easier to alter standard -> update in the respository list).

One reason that it is done like this by default is because core SLE / Leap packages shouldn't be break API/ABI for updates, so in the vast majority of cases building against standard is fine and saves huge amounts of resources on obs. You are only really hitting this issue because you are working with one of the few echosystems that as you say doesn't follow our traditional model.

Ecosystems are changing, and less and less are following the traditional model - I think we can't keep treating them as exceptions, because this will become the norm for many languages.

So how can we improve this? How can we make rust a first class language for our platforms so that we can keep including relevant and modern tools in our platforms?

This is not just a tooling problem but also a process problem, the Haskell team have managed to script there entire process, there scripts can automatically push updates too there 2500 packages. This then revealed a different set of issues where our manual package review team and legal team didn't have enough manpower to regularly deal with the extra 2500 requests and I think for now the end solution was to cut back and just have the most useful packages in factory and the rest in a devel repo.

So it has been shown that atleast with one platform once the core is there and working, handling the frequent updates of various packages can be largely automated at least for tumbleweed as long as they are capable of doing offline builds.

For SUSE there is definitely a discussion that should be had around whether we should promise binary compatibility for the whole stack but that's a discussion for SUSE's product managers etc.

Who needs to be contacted to begin this discussion and get actions happening - I'm in a situation where I will be making rust a requirement for 389-ds soon, so this needs to have solutions.

--

Simon Lees (Simotek) http://simotek.net

Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On Mon, Nov 18, 2019 at 8:58 PM Simon Lees wrote:

On 11/19/19 11:22 AM, William Brown wrote:

...

...

On 19 Nov 2019, at 11:19, Simon Lees wrote:

On 11/19/19 11:07 AM, William Brown wrote:

...

...

On 19 Nov 2019, at 11:04, Neal Gompa wrote:

On Mon, Nov 18, 2019 at 7:32 PM William Brown wrote:

...

Okay, so when you do:

Project -> Repositories -> Add from a distribution -> select fedora 30

That only adds fedora 30 standard:

https://build.opensuse.org/repositories/Fedora:30

But not update. This is why all the f30 builds I have are failing. This seems like a bug or error to me.

It is done this way to minimize automatic rebuilds by default, as new updated packages in the repositories will trigger rebuilds. If you want updates, just change "standard" to "update" and call it gravy. Most don't require this, but your Rust case probably will.

For what it's worth, openSUSE Leap targets are similarly set up.

This is really counter intuitive to anyone who is reasonably inexperienced at packaging (IE me), which is probably another barrier to getting people to do tasks like this. It could be improved both as a default (include updates because that's the expected behaviour) or via UI (offer updates as an option in the distribution menu, or make it easier to alter standard -> update in the respository list).

One reason that it is done like this by default is because core SLE / Leap packages shouldn't be break API/ABI for updates, so in the vast majority of cases building against standard is fine and saves huge amounts of resources on obs. You are only really hitting this issue because you are working with one of the few echosystems that as you say doesn't follow our traditional model.

Ecosystems are changing, and less and less are following the traditional model - I think we can't keep treating them as exceptions, because this will become the norm for many languages.

So how can we improve this? How can we make rust a first class language for our platforms so that we can keep including relevant and modern tools in our platforms?

This is not just a tooling problem but also a process problem, the Haskell team have managed to script there entire process, there scripts can automatically push updates too there 2500 packages. This then revealed a different set of issues where our manual package review team and legal team didn't have enough manpower to regularly deal with the extra 2500 requests and I think for now the end solution was to cut back and just have the most useful packages in factory and the rest in a devel repo.

For Rust crates, we can easily have similar automation. We're still building up the set of crates, but because of the whole manpower problem with reviews, it's going to be a rough time.

So it has been shown that atleast with one platform once the core is there and working, handling the frequent updates of various packages can be largely automated at least for tumbleweed as long as they are capable of doing offline builds.

For SUSE there is definitely a discussion that should be had around whether we should promise binary compatibility for the whole stack but that's a discussion for SUSE's product managers etc.

The bottleneck is almost entirely in the review process. We can do large updates easily enough. The review process does not handle large stack updates well. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

Neal Gompa schrieb:

[...] I find the process of shipping updates to SUSE distributions thoroughly annoying and unappealing, so I don't personally do anything to target anything other than Tumbleweed. The process is too slow because it's often bottlenecked on either legal-auto or the all-too-tiny number of reviewers for SRs and MRs. This is also true for Tumbleweed, but it seems to be somewhat faster there. I'd be more interested in helping ship updates to Leap if we could make this process less awful.

If the sources are exactly the same as in Factory, no extra reviews should be needed. legal-auto would automatically pass the review if the sources were already approved in another project for example¹. 15.2 does not add extra review-team on sources that come from SLE or Factory and are therefore known to have seen a review before. If maintenance doesn't do that yet, I'd suggest to push for smarter review handling there. cu Ludwig [1] corner cases with changed matching patterns aside, I know -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 247165 (AG München) -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 11/19/19 6:27 PM, Richard Biener wrote:

On Tue, 19 Nov 2019, William Brown wrote:

...

Hi,

Recently I've been trying to package some utilities for SUSE and Fedora via OBS that are written in Rust. I'm running into a bit of a problem though.

Rust as a language has a very aggressive release cycle compared to what we expect on a platform like SUSE - 6 weeks (https://github.com/rust-lang/rust/blob/master/RELEASES.md). In each cycle Rust releases new features, and despite the core language being "stable" and safe (which is great!) new features tend to be used very rapidly by library authors. For example the release from Version 1.34.0 (2019-04-11) contains convert::TryFrom, which is now in use by a large number of libraries.

Rust also has an (unfortunate) requirement that you are essentially forced to use cargo which is a build and dependency management tool. Cargo is extremely opinionated and inflexible which makes it difficult to use. However as it's also a dependency management tool, this has encouraged an npm-style ecosystem of dependencies to spring up on https://crates.io/ - and it's effectively the only way to use libraries in Rust. But additionally, it's promoted a system where a library may have a large number of small dependencies too.

Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers.

Which leads me to the question of "what to do".

* I don't believe it's feasible to ask Rust to "slow down". It's just not going to happen, and they will keep adding features that people will "want" to use. * We can't expect people *not* to update their dependencies in projects as that would prevent security updates being included. So we have to accept projects that will use "latest and greatest" complier features somewhere in their dependency graph. * Which leaves distros (like us) speeding up our rust compiler cycle somehow.

For opensuse I can probably do something like adding devel:languages:rust as a respository to my project. Would this strategy be viable on SLE? What about for projects which we want to expose packages for fedora or other? Are we able to have toolchains move faster than our base system?

I'm really looking for ideas on how we can sustainably use rust projects within the governance and social constraints that exist in the space. Ideas?

When facing issues like this we need to think whether having "packages" for all of this is really a good fit for the purpose. I think it would be appropriate to concentrate on enabling people to bootstrap/update something like a /usr/local/rust "repository" (or $HOME/.rust?) with the latest and greatest from upstream. This means providing rust/cargo (and whatever else needed) packages plus scripting that will fetch, build and install the newest releases. Trying to fit external ecosystems with own "package management" into ours may not be the best way time is spent.

But not needing to go and download the "bootstrap binaries" from an untrusted source is reasonable.

All this probably applies to other languages as well, though it may be that there only the package management part is a problem there, not too fast evolving core tools.

Unfortunately as parts of SLE start to make use of some of these languages it becomes more complex then that. Many of the reasons we do the things we do the way we do at the moment are that it makes our compliance processes significantly easier (and in many cases possible at all). -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 11/19/19 7:15 PM, Jan Engelhardt wrote:

An alternative would be to devote the manned resources to not grow minute dependencies on large stacks. (That ship has probably sailed in terms of firefox-in-the-distro, (...)

Unless Firefox has changed since the last time I touched the codebase, I'm pretty sure that it has all the Rust crates vendored which it needs. So, Firefox alone shouldn't be a problem. Adrian

On Wed, 20 Nov 2019, William Brown wrote:

...

On 20 Nov 2019, at 05:57, John Paul Adrian Glaubitz wrote:

On 11/19/19 7:15 PM, Jan Engelhardt wrote:

...

An alternative would be to devote the manned resources to not grow minute dependencies on large stacks. (That ship has probably sailed in terms of firefox-in-the-distro, (...)

Unless Firefox has changed since the last time I touched the codebase, I'm pretty sure that it has all the Rust crates vendored which it needs. So, Firefox alone shouldn't be a problem.

I think Rust projects will always tend to vendoring, because the huge micro-dependency style system that exists means we'll never keep up trying to package these. Kanidm has ~200 crate indirect dependencies on it's own, I can't imagine how many firefox has. It's a waste of our time to try to package all 200 when we could be putting that time into other things instead. This essentially means we are always at the whims of those vendored dependencies and what they require, and that is highly likely to be the "latest" stable compiler.

I think we should aim to:

* Have rustc/cargo move with the upstream cadence, with automation as suggested into all SLE/LEAP/TW/Other * Do not attempt to package crates - only package leaves IE consuming projects * Improve our rust macros and packaging guides around this topic.

As much as we want to wring our hands and slow down rust, and do things "our way" we can't - we have to follow and adapt in this case.

I wonder whether for packages requiring rust it would be most sensible to package up all dependences into the package itself - that at least ensures a "stable" set of dependences for individual packages even if it means that a dependence might exist twice in different versions in different packages. Not sure if that also means packaging up their own rust as some package up their own clang/llvm ... Richard. -- Richard Biener SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)

On Wednesday, November 20, 2019 12:53:06 AM CET William Brown wrote:

...

On 20 Nov 2019, at 05:57, John Paul Adrian Glaubitz wrote:

...

On 11/19/19 7:15 PM, Jan Engelhardt wrote: An alternative would be to devote the manned resources to not grow minute dependencies on large stacks. (That ship has probably sailed in terms of firefox-in-the-distro, (...)

Unless Firefox has changed since the last time I touched the codebase, I'm pretty sure that it has all the Rust crates vendored which it needs. So, Firefox alone shouldn't be a problem.

I think Rust projects will always tend to vendoring, because the huge micro-dependency style system that exists means we'll never keep up trying to package these.

For tarballs that comes from upstream can be the case, but I am not sure that this ideal for openSUSE.

Kanidm has ~200 crate indirect dependencies on it's own, I can't imagine how many firefox has. It's a waste of our time to try to package all 200 when we could be putting that time into other things instead.

You already pointed a reason of why doing this effort makes sense: security updates. We have a full build system designed to track the dependencies, and build the required subset automatically. Throwing that out of the window for a new language is IMO wrong. We do traditionally vendoring in Java, but an heroic effort of one developer is changing this, reusing solutions from Fedora, RedHat or Debian, and a lot of smart work.

This essentially means we are always at the whims of those vendored dependencies and what they require, and that is highly likely to be the "latest" stable compiler.

I think we should aim to:

* Have rustc/cargo move with the upstream cadence, with automation as suggested into all SLE/LEAP/TW/Other

* Do not attempt to package crates - only package leaves IE consuming

+1, I think that this is a good starting point. Neal, what do you think of the idea from Ludwig of having a system rust compiler (more stable and conservative, used to build all the openSUSE rings), and a updated version of the compiler co-installable for developers. projects I do not agree here. The effort of designing scripts to update the crates, macros that will help, supporting multiple versions of the same crate, etc is so big compared with `cargo vendor`! Maybe we can do something similar of what go is doing: https://en.opensuse.org/openSUSE:Packaging_Go Maybe using only BuildRequires instead of both Requires and BuildRequires if this is possible, and designing a similar set of macros here. Neal, what is the plan in Fedora for this?

* Improve our rust macros and packaging guides around this topic.

Yes.

As much as we want to wring our hands and slow down rust, and do things "our way" we can't - we have to follow and adapt in this case.

So where do we start to start to achieve this?

You are making very hard questions : )) I am personally in a phase that I try to have a view of what we want, and what are the good decisions. Fedora and Debian communities are spending a lot of thinking here, so before moving I want to have more information. -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer

...

...

This essentially means we are always at the whims of those vendored dependencies and what they require, and that is highly likely to be the "latest" stable compiler.

I think we should aim to:

* Have rustc/cargo move with the upstream cadence, with automation as suggested into all SLE/LEAP/TW/Other

+1, I think that this is a good starting point.

Neal, what do you think of the idea from Ludwig of having a system rust compiler (more stable and conservative, used to build all the openSUSE rings), and a updated version of the compiler co-installable for developers.

It's not a _terrible_ idea. The only problem with it is that Firefox forces Rust to move forward anyway. That's why RHEL doesn't do this right now, either.

This isn't viable - the moment I, a developer, take my project from development -> packaging, I'm now on an older compiler that my dependencies won't compile with. They have to be the same version, and they have to both move at the rate upstream rust is moving as many dependencies really do move at that cadence too. This is even more important when you remember that rust doesn't distinguish security releases from feature ones, so an update to a dependency for security, may require newer compiler versions.

...

...

As much as we want to wring our hands and slow down rust, and do things "our way" we can't - we have to follow and adapt in this case.

So where do we start to start to achieve this?

You are making very hard questions : ))

Yes - because I want kanidm in opensuse at some point, and 389-ds is about to pick up rust as a requirement which will absolutely affect future leap versions. I want (need?) this solved. :)

...

I am personally in a phase that I try to have a view of what we want, and what are the good decisions. Fedora and Debian communities are spending a lot of thinking here, so before moving I want to have more information.

The assumption here is that Rust needs to be slowed down. That's not the case at all. You can move fast _without_ breaking things. The problem is that today there are no Rust compiler and language developers who understand this and help factor that into the future development of the stack.

I think that again, Rust has different targets (containers, static applications, continual integration and release) than an OS packaging system (shared libs, C, patched backports). It's not that they don't understand as much as they actively chose to follow a different path. Which is why we are struggling. The mental model of how we did things with C based programs doesn't apply. For years Red Hat / Fedora have tried to for python to be packaged like it's C, when it's not (which has left pip/pypi in bad places) and causes python on a system to be missing many libraries that do exist. This is why python as a community exclusively recommends virtualenv, not OS/system python. We'll probably end up in the exact same place with rust (cargo vs zypper in crate-foo). So we can either keep fighting against it, and people will just use cargo + vendoring anyway, or we can accept that the ship has sailed and try to work with it instead. For example: * https://build.opensuse.org/package/show/openSUSE:Leap:15.0/cargo * https://build.opensuse.org/package/show/home:gladiac/rav1e * https://build.opensuse.org/package/show/home:luke_nukem:rust_apps/nushell * https://build.opensuse.org/package/show/openSUSE:Leap:15.2/ripgrep

from M Matz:

...

That's a downside to QA process, not the packaging issue anymore. As far as I know we are discussing packaging right now, aren't we?

William (rightly) wants to have something realistic for the distro. Packaging (as in merely creating and submitting packages easily) is only one part, it can't be seen in isolation. If you can submit a thousand packages in 10 minutes doesn't really matter, if those thousand packages then don't land in Factory, or only six months after submission. The whole thing needs to work on all levels.

So the more I'm reading into this thread the more I think that my conclusion here is: * We should have rust/cargo's release cadence seperate from the release cycles of a distro (ie fedora modularity in a way, or tw style) * Packaging crates to rpms is not feasible or scalable, and we should not attempt it - cargo already has vendor and all the packages I've seen in obs today use vendored dependencies. * We should instead focus on packaging "edge" programs that use rust IE firefox, ripgrep. * Focus pkg QA efforts on the edge packages * We could consider integration of tools like cargo-audit to scan for security issues in packages (similar to clamav in obs already) to help ensure vendored deps are "up to date". * Improve the rpm spec docs related to rust/cargo to help make it easier for people to give us packaged programs. — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 21 Nov 2019, at 09:45, John Paul Adrian Glaubitz wrote:

Hi!

On 11/20/19 11:26 PM, William Brown wrote:

...

...

The assumption here is that Rust needs to be slowed down. That's not the case at all. You can move fast _without_ breaking things. The problem is that today there are no Rust compiler and language developers who understand this and help factor that into the future development of the stack.

I think that again, Rust has different targets (containers, static applications, continual integration and release) than an OS packaging system (shared libs, C, patched backports). It's not that they don't understand as much as they actively chose to follow a different path.

But packages like 389-ds and librsvg are anything but typical container packages which is the main problem here. People don't necessarily criticize that Rust is fast moving target at the moment. They criticize that people start using Rust for system-critical components while Rust is a moving target.

Rust won't slow down though, and has no intent too. I think we have to accept it's fast moving.

...

Which is why we are struggling. The mental model of how we did things with C based programs doesn't apply.

Using stable APIs and not updating hundreds of packages just because you need to install a newer version of Firefox or some security updates isn't the mental model of C. It's the mental model of sanity.

We simply can't simply use the Rust release model for enterprise distributions because no matter what Rust upstream will claim: New upstream versions will always bring regressions, in one way or another. Changing command line options or configuration file formats is already a regression in production use. No paying customer really wants to see an update breaking their whole server setup. It's simply not acceptable. Imagine SLE running in a critical system at a bank on an IBM zSeries mainframe responsible for millions of customers and then you start upgrade 250+ packages for a security fix and break the whole database.

We can complain all we like here, but firefox is using rust, and other packages will use it too. We have to be able to adapt here ...

...

For years Red Hat / Fedora have tried to for python to be packaged like it's C, when it's not (which has left pip/pypi in bad places) and causes python on a system to be missing many libraries that do exist.

Again, this isn't a language issue. This is upstream projects not understanding how software is used in production environments. If 90% of the developers of an upstream projects are running distributions like Fedora Rawhide, Arch Linux or Tumbleweed, they naturally don't understand why you wouldn't want all packages to be bleeding edge all the time. But this isn't something you can use in a production environment with hundreds, thousands or even millions of customers.

...

This is why python as a community exclusively recommends virtualenv, not OS/system python.

So, how exactly do you expect SUSE or RedHat being able to support a paying customer when their whole Python environment has been installed through pip?

They don't offer them support in these cases. This in mind SUSE/RH do support 'whole applications' like openshift, 389-ds or jboss, and "how those applications are composed" doesn't really factor into it, and they support them still.

Our engineers simply can't support a software environment that we have no control over. The main reason why software is being packaged in distribution besides convenience is being able to provide something that can be supported by the distribution vendor. If we just give our customers a minimal base system and just let them install all their software through cargo, pip, npm and what not, we won't be able to provide support for their environment. We might as well drop the concept of a distribution then. But that's when enterprise customers will return to proprietary systems like Windows Server or AIX.

...

We'll probably end up in the exact same place with rust (cargo vs zypper in crate-foo). So we can either keep fighting against it, and people will just use cargo + vendoring anyway, or we can accept that the ship has sailed and try to work with it instead.

As long as we want to provide a distribution that we can provide support for, we will have to stick to the "old model" and that ship isn't going to sail anywhere.

And that's fine, but we have to be able to support more than just that old model else we will be left behind.

...

So the more I'm reading into this thread the more I think that my conclusion here is:

* We should have rust/cargo's release cadence seperate from the release cycles of a distro (ie fedora modularity in a way, or tw style) * Packaging crates to rpms is not feasible or scalable, and we should not attempt it - cargo already has vendor and all the packages I've seen in obs today use vendored dependencies. * We should instead focus on packaging "edge" programs that use rust IE firefox, ripgrep. * Focus pkg QA efforts on the edge packages * We could consider integration of tools like cargo-audit to scan for security issues in packages (similar to clamav in obs already) to help ensure vendored deps are "up to date". * Improve the rpm spec docs related to rust/cargo to help make it easier for people to give us packaged programs.

It could also happen that certain software projects like librsvg or 389-ds are being forked in their last non-Rust version to avoid the bootstrap issues. It's not that projects haven't been forked in the past over such fundamental project philosophy issues (XFree86/X.Org, libav/ffmpeg, OpenOffice/LibreOffice, eglibc/glibc, ecgs/gcc etc).

I don't think the hammer method that some projects are currently using to push Rust is going to be successful in the long-term.

As the person who is adding rust to 389-ds, the maintainer of it at suse, and part of the upstream team (the rh people are also on board with this), I don't think I'll be forking "the last non-rust version" as that would be a bit self defeating ... We need productive solutions, not complaining that rust is different to "how it's always been". — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

Alright, I'm going to come out and say it. You need to pull that stick out of your rear.

This comment is extremely inappropriate regardless of the situation, and it has no place on a mailing list or in any communication to any persons. I've made my argument already in the thread and I think we are just going in a circle at the moment. I'm going to step back and think about this topic for a while before I say more. — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On Thursday, November 21, 2019 8:40:19 AM CET Richard Biener wrote: [cut a lot here]

From the start of the thread I remember there were two conflicting things - first packaging where vendoring seems to work and while it's not ideal it's probably going to be what we need to do due to being "slow". I interpret your above sentences as that this is intended to work this way.

Then there's the need of a developer using Rust (but not necessarily packaging software for a distro) who wants the latest and greatest. I've suggested we provide means to easily bootstrap such upstream Rust - and you've indicated that yes, indeed, thats supported and even the way it is supposed to work, but there's no openSUSE "package" to start this (in Factor).

What is wrong on the idea of having two compilers: the system one used for the distribution, and the one that can be updated and expected to be used by developers. Another idea is to have one single compiler, the system-wide one, and provide a rustup binary that the developer can use to install rust and cargo on the users $HOME. Dany Marcoux already submitted the package, but later was closed. I personally prefer the 2 compiler idea, tho, but this is irrelevant. [more cut]

Indeed. We're not going to make Rust "go away". Vendoring works fine enough for "leafs" like Firefox but indeed having the rust stack in core components imposes some scalability issues and people should think twice before introducing a rust dependency into, say, systemd. But obviously those who do the work decide, and as alwaks talk is cheap.

I am sorry but... vendoring is not working. OBS is not aware of what is inside, so any security update in any component in the closure can hurt us a lot. We can use vendoring meanwhile the big issue is addressed, but I would never go for this path without a deadline in mind. -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer

On 11/22/19 9:38 AM, William Brown wrote:

...

On 22 Nov 2019, at 03:04, Alberto Planas Dominguez wrote:

On Thursday, November 21, 2019 8:40:19 AM CET Richard Biener wrote:

[cut a lot here] [more cut]

...

Indeed. We're not going to make Rust "go away". Vendoring works fine enough for "leafs" like Firefox but indeed having the rust stack in core components imposes some scalability issues and people should think twice before introducing a rust dependency into, say, systemd. But obviously those who do the work decide, and as alwaks talk is cheap.

I am sorry but... vendoring is not working. OBS is not aware of what is inside, so any security update in any component in the closure can hurt us a lot.

I'd suggest we wont be able to stop people vendoring because the rate of flux in crates.io is so high, continually adding and packaging more crates is going to be a high resource cost on people. If I had to submit 200 rpm's to suse just to submit kanidm, I'd probably give up at the sheer thought of that effort on me and others.

Yes and we do things like this with many python applications because its what customers expect. The cloud team invests significant resources into this area and maintain a significant number of python applications.

Security updates become a requirement of the projects submitting to ensure their vendored dependencies are up to date. I think that we can detect this with cargo audit as a build step in the spec instead, similar to how we use clamav.

I guess the feeling I'm getting overall from this thread and peoples input is the idea of "support" and what that is, and how it works, and what goes into that.

SUSE/SLE/Distros have always been about supporting "layers", such as a kernel or tls library. They are updated dynamically and the higher layers can inherit their changes seamlessly. This encourages backporting, patches etc. They also have C compilers that have *rapidly* changed in the last two decades, which while making code "faster", has also introduced (sometimes breaking) changes when you recompile the same code. This unpredictability makes us want to stick to a known compiler for a version so that all the layers are consistent. It means you have to have these layers work for multiple applications at the same time, via ABI's that are "stable". This promotes us keeping one version of a layer part that is supported for a long time.

Rust, firefox, docker and others, have become about supporting "applications". The interface is no longer between layers, but the user-facing parts, and the application as a whole is a "static linked" whole stack that is supported as is. The composition of that static application is the responsibility of the project, who is expected to re-build and re-release often. Rust has no ABI or guarantees about it, shifting this to compile time rather than link time to ensure you consume the API correctly. A single project can have multiple versions of the same crate in it's dep stack (rand is a key example here, in large projects it's been observed to have 4 different api versions). This in mind, due to Rust's guarantees many of the memory safety issues that heavily affect C do not exist, meaning there are less stability and security releases in comparison, and that static stack is much more reliable as a whole, lending again to supporting the "application" as a whole unit rather than supporting the "layers" that make it up.

I guess maybe we should think about Rust applications the same way we think about docker applications, they have much more in common as staticly linked units than say comparing Rust to C.

This would be somewhat easier to do if people were using rust in a similar way just for containerized applications. Unfortunately this is not strictly how rust is being used, its starting to be used in core system libraries like librsvg which means that at least in some cases we have to think of it in the traditional mindset. The reason customers buy SLES is because they want a base system that doesn't change, many already think there are some parts of the base system that they think we update too often, and many get grumpy with us on the odd occasion when we have to ship a version update instead of backporting security updates. I won't argue against vendoring being a significantly easier and less effort at the same time though its not what our customers expect which is why SUSE's engineering department invests significant resources into not vendoring. Its not uncommon for SUSE Engineers to contribute patches upstream to projects adding options to the build system to use the system version of libraries not the vendored ones. As Rust usage increases this is something SUSE will have to deal with but where and how they allocate resources will be up to them, if the amount of effort required to manage the parts of the rust stack that your packages are likely to maintain is more then you have time for then that is a concern you should be raising with your manager. It wouldn't surprise if as rust usage grows SUSE needs to end up hiring a dedicated maintainer / maintainers for it, in the same way we have specific maintainers looking after the python / java / etc stacks for the whole distro. I can understand that there are probably other things you could be better spending your time on then maintaining Rust, but that doesn't mean we should just give up on dealing with it properly and finding a proper solution. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

On 25 Nov 2019, at 19:45, Richard Biener wrote:

On Fri, 22 Nov 2019, William Brown wrote:

...

...

On 22 Nov 2019, at 03:04, Alberto Planas Dominguez wrote:

On Thursday, November 21, 2019 8:40:19 AM CET Richard Biener wrote:

[cut a lot here]

...

From the start of the thread I remember there were two conflicting things - first packaging where vendoring seems to work and while it's not ideal it's probably going to be what we need to do due to being "slow". I interpret your above sentences as that this is intended to work this way.

Then there's the need of a developer using Rust (but not necessarily packaging software for a distro) who wants the latest and greatest. I've suggested we provide means to easily bootstrap such upstream Rust - and you've indicated that yes, indeed, thats supported and even the way it is supposed to work, but there's no openSUSE "package" to start this (in Factor).

What is wrong on the idea of having two compilers: the system one used for the distribution, and the one that can be updated and expected to be used by developers.

Another idea is to have one single compiler, the system-wide one, and provide a rustup binary that the developer can use to install rust and cargo on the users $HOME. Dany Marcoux already submitted the package, but later was closed.

I personally prefer the 2 compiler idea, tho, but this is irrelevant.

The 2 compiler idea is kind of a problem? I have a version on my machine that I was developing with (1.39.0), and encountered version (1.36.0) on leap which didn't have a feature I required.

Of course this happens with C++ language features evolving as well. And we _do_ have multiple(!) GCC C++ compilers around, also specifically to address this problem. So I see no problem with having multiple rust compilers around.

It's not the presence of multiple compilers thats a problem, it's the rules around them. C really hasn't changed much - new gcc versions fix bugs or work faster etc. It's not a big deal. Rust compiler versions introduce large, changing feature sets, that really affect the usage of the language. This year alone I think maybeuninit, async/await, try_from all come to mind. The issue is that what we ask developers to use (latest and greatest, all upstream deps) is in conflict when we go to package those software into suse which will use a "stable" compiler (older version, ie missing features) because those upstream dependencies will latch onto those new features very quickly. There are already upstream crates that won't compile with a 6 month old compiler - let alone 1 year or more. I think my point is that I want the path from developer to packager and maintainer to be simpler and laid out, and our ecosystem supporting that instead of throwing up barriers or hurdles. It's also worth bearing in mind that rust does not have most of the issues which makes C developers so anxious or conservative around versioning and releases. As mentioned, rust is closer as an experience to docker than C. — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On Wednesday, November 20, 2019 11:26:41 PM CET William Brown wrote:

...

...

Neal, what do you think of the idea from Ludwig of having a system rust compiler (more stable and conservative, used to build all the openSUSE rings), and a updated version of the compiler co-installable for developers.> It's not a _terrible_ idea. The only problem with it is that Firefox forces Rust to move forward anyway. That's why RHEL doesn't do this right now, either.

This isn't viable - the moment I, a developer, take my project from development -> packaging, I'm now on an older compiler that my dependencies won't compile with.

I understand. This is the reason of having two version of the compiler in Leap / SLE and one single version in TW. One version will be the system compiler, associated to the service pack release, or in the case of Leap, to the subversion number. This is the compiler used to build the distribution and will be updated in each SPx, based on some criteria that balance change and stability. Because Rust is very strong in the backward compatibility, I think that we can expect that in each SP / Leap release, the system compiler will be updated to the last version. For developers, there will be a more update version available officially, that is moving at the same peace that upstream. This is the only one that developers will use, and a `zypper up` will take care of updating it in the distribution. But we cannot expect that the system compiler will be updated at the same speed that upstream, because this requires that every 6 weeks, a routinary `zypper up` will update the compiler, but also the user applications that are using rust. This goes against the stability guarantees and reproducibility that SLE and Leap offer to the users. Imagine that I use the new compiler in my SLE to get my Firefox, and I get a different binary that the one distributed with the system.

They have to be the same version, and they have to both move at the rate upstream rust is moving as many dependencies really do move at that cadence too. This is even more important when you remember that rust doesn't distinguish security releases from feature ones, so an update to a dependency for security, may require newer compiler versions.

Yes, exactly. You will be using the compiler for developers.

...

The assumption here is that Rust needs to be slowed down. That's not the case at all. You can move fast _without_ breaking things. The problem is that today there are no Rust compiler and language developers who understand this and help factor that into the future development of the stack.

I think that again, Rust has different targets (containers, static applications, continual integration and release) than an OS packaging system (shared libs, C, patched backports). It's not that they don't understand as much as they actively chose to follow a different path.

Which is why we are struggling. The mental model of how we did things with C based programs doesn't apply. For years Red Hat / Fedora have tried to for python to be packaged like it's C, when it's not (which has left pip/pypi in bad places) and causes python on a system to be missing many libraries that do exist. This is why python as a community exclusively recommends virtualenv, not OS/system python. We'll probably end up in the exact same place with rust (cargo vs zypper in crate-foo). So we can either keep fighting against it, and people will just use cargo + vendoring anyway, or we can accept that the ship has sailed and try to work with it instead.

No, there is a difference. In Python you need to install the library before it can be used, as is an interpreted language, and in C you need to install the library as is dynamically linked to the application. The proposal of packaging the crates is not about using `zypper in rust- regex`, is about using `BuildRequires: rust-regex` during the creation of `kanidm`. No crate will be usually installed in the system, as the RPM only contains the source code. A change in `rust-regex` in OBS will trigger a rebuild of `kanidm`, as will be any update of any crate that belongs to the transitive closure of the dependencies of your package. This avoid vendoring, and avoid the security implications related with it. But during the daily development of kanidm, you are free to use cargo to install the crates in the user space. During the packaging phase of kanidm, the last version of the crate needs to be available in OBS before it can be compiled, tho. So multiversioning is something that we need to address in OBS, with the same expectations that cargo have.

...

from M Matz:

...

That's a downside to QA process, not the packaging issue anymore. As far as I know we are discussing packaging right now, aren't we?

William (rightly) wants to have something realistic for the distro. Packaging (as in merely creating and submitting packages easily) is only one part, it can't be seen in isolation. If you can submit a thousand packages in 10 minutes doesn't really matter, if those thousand packages then don't land in Factory, or only six months after submission. The whole thing needs to work on all levels.

So the more I'm reading into this thread the more I think that my conclusion here is:

* We should have rust/cargo's release cadence seperate from the release cycles of a distro (ie fedora modularity in a way, or tw style)

IMHO we do not mix the real problem of the review process, with the pace of upgrade. If we need to upgrade a lot of packages, and we have the tools to do that, lets fix the review process. I am sure that this way we help more. I am in favor of moving fast, but again, there are reasons in SLE / Leap (reproducibility build, stability guarantees) that are not related with the review process problem.

* Packaging crates to rpms is not feasible or scalable, and we should not attempt it - cargo already has vendor and all the packages I've seen in obs today use vendored dependencies.

Packaging crates is is feasible. Is not scalable because the review process, not because we cannot update all the packages every day. and use it the in the spec file as a BuildRequires as is expected (and as is implemented right now on Fedora) We can use vendoring now, until we fix those problems, but we need to aim to the better solution.

* We should instead focus on packaging "edge" programs that use rust IE firefox, ripgrep.

For TW, sure.

* Focus pkg QA efforts on the edge packages

* We could consider integration of tools like cargo-audit to scan for security issues in packages (similar to clamav in obs already) to help ensure vendored deps are "up to date".

Something like this is a good idea, like the feature that github provides, that is pointing to you that have a requirement on a library that have doing only this integration is not fixing the vendoring issue.

* Improve the rpm spec docs related to rust/cargo to help make it easier for people to give us packaged programs.

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer

On Tue, Nov 19, 2019 at 6:51 AM John Paul Adrian Glaubitz wrote:

Hi!

On 11/19/19 12:58 AM, William Brown wrote:

...

Rust as a language has a very aggressive release cycle compared to what we expect on a platform like SUSE - 6 weeks (https://github.com/rust-lang/rust/blob/master/RELEASES.md). In each cycle Rust releases new features, and despite the core language being "stable" and safe (which is great!) new features tend to be used very rapidly by library authors. For example the release from Version 1.34.0 (2019-04-11) contains convert::TryFrom, which is now in use by a large number of libraries.

Is that really that common? I have had a lot of interaction with Rust upstream through Debian and I was always told it's normally only the Rust compiler itself which uses new features in its own code which is why it always needs to be built with either its own version or the previous version, older or newer versions usually break.

Unfortunately, the cake is a lie. The Rust community does not have a culture of conservatism. To be fair, they have not yet been burned by using the newest features as soon as they become available. But this is why Fedora is on the Rust compiler update treadmill.

It sometimes happens that 3rd party Rust software projects are affected by it like you say, but I have observed that only occasionally unless the Rust compiler is really old. But maybe the situation is worse when using SLE due its long-term character.

The situation is definitely aggravated in RHEL and SLE, since it's fairly difficult to update the compiler in those distributions (because of the difficulty of the update submission and approval process, not because of a technical problem).

However, generally I think that Rust should slow down with frequent incompatible changes to the language itself as this prevents independent compiler implementations to keep up. mrustc is currently compatible with Rust 1.29.0 and from GCC upstream I have heard that the frequent changes are the main blocker for adopting a Rust frontend in GCC.

The Rust community needs to get burned a few times first. But I think we're a long ways away from that happening. The Rust compiler developers are smart guys, but they don't understand that bootstrapping Rust is not supposed to be so hard. It doesn't help that Rust upstream doesn't care much about concerns from distributions and downstream developers. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 19 Nov 2019, at 22:55, Jan Engelhardt wrote:

On Tuesday 2019-11-19 13:02, Neal Gompa wrote:

...

The Rust compiler developers are smart guys, but they don't understand that bootstrapping Rust is not supposed to be so hard.

So then, what does smart mean actually?

The issue with rust is not smart as in intellect, it's that rust is targeted at specific use cases - which is standalone static applications (firefox, and containers that are self contained) and not distros (shared lib, long lived versions and toolchains.) I think that throwing remarks that could be misinterpreted isn't really going to help us here and we should look at what we can change that's within our power instead. — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 19-11-19 16 h 44, Ludwig Nussel wrote:

William Brown schrieb:

...

[...] Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers.

From what I can tell Rust has two use cases in the stable distros. The first one is that it's needed by other system components. Regularly updated, more or less leaf packages Firefox as but unfortunately also as low as librsvg. For packages like the latter we most likely do not want Rust to change very six weeks, especially when that kind of use case spreads. So for that it would be better to upgrade the system Rust only for service packs ie once per year.

The other use case is developers that want the latest and greatest for their own purposes. For that it should be possible to create a parallel installable Rust package and regularly provide newer versions for it. That works for Leap via regular maintenance updates as well as SLE via Package Hub. I've been told that Rust is meant to be pretty good at backwards compatibility so it's probably sufficient to provide one additional Rust package that follows upstream releases, instead of a new one for all kinds of versions like eg Go does.

cu Ludwig

rustup[0] is what developers should use when developing in Rust. It's the official Rust toolchain installer. I packaged it in my home[1] and submitted a request[2] in hopes that it would get into Factory, but in the end, I didn't get much feedback beside from Neal Gompa. I decided to revoke the request. As you can see in the request, I talked about rust and rustup coexisting, but this would need a bit more work. I didn't look into this, but I'm willing to do it. It definitely seems to be possible as documented[3] upstream. [0]: https://github.com/rust-lang/rustup [1]: https://build.opensuse.org/package/show/home:dmarcoux/rustup.rs [2]: https://build.opensuse.org/request/show/684463 [3]: https://github.com/rust-lang/rustup#working-with-distribution-rust-packages -- Dany Marcoux Full Stack Web Developer - Open Build Service SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 20 Nov 2019, at 03:04, Dany Marcoux wrote:

On 19-11-19 16 h 44, Ludwig Nussel wrote:

...

William Brown schrieb:

...

[...] Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers.

From what I can tell Rust has two use cases in the stable distros. The first one is that it's needed by other system components. Regularly updated, more or less leaf packages Firefox as but unfortunately also as low as librsvg. For packages like the latter we most likely do not want Rust to change very six weeks, especially when that kind of use case spreads. So for that it would be better to upgrade the system Rust only for service packs ie once per year.

The other use case is developers that want the latest and greatest for their own purposes. For that it should be possible to create a parallel installable Rust package and regularly provide newer versions for it. That works for Leap via regular maintenance updates as well as SLE via Package Hub. I've been told that Rust is meant to be pretty good at backwards compatibility so it's probably sufficient to provide one additional Rust package that follows upstream releases, instead of a new one for all kinds of versions like eg Go does.

cu Ludwig

rustup[0] is what developers should use when developing in Rust. It's the

official Rust toolchain installer. I packaged it in my home[1] and submitted

a request[2] in hopes that it would get into Factory, but in the end, I didn't

get much feedback beside from Neal Gompa. I decided to revoke the request. As

you can see in the request, I talked about rust and rustup coexisting, but this

would need a bit more work. I didn't look into this, but I'm willing to do it.

It definitely seems to be possible as documented[3] upstream.

This does not address the issue - how do we get from development to a system-buildable and shippable project. This is key here, we need to think about that full experience from development to deployment - not just development.

[0]: https://github.com/rust-lang/rustup

[1]: https://build.opensuse.org/package/show/home:dmarcoux/rustup.rs

[2]: https://build.opensuse.org/request/show/684463

[3]: https://github.com/rust-lang/rustup#working-with-distribution-rust-packages

-- Dany Marcoux Full Stack Web Developer - Open Build Service

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer

-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 20 Nov 2019, at 18:35, Dany Marcoux wrote:

On 19-11-20 00 h 55, William Brown wrote:

...

...

On 20 Nov 2019, at 03:04, Dany Marcoux wrote:

On 19-11-19 16 h 44, Ludwig Nussel wrote:

...

William Brown schrieb:

...

[...] Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers. From what I can tell Rust has two use cases in the stable distros. The first one is that it's needed by other system components. Regularly updated, more or less leaf packages Firefox as but unfortunately also as low as librsvg. For packages like the latter we most likely do not want Rust to change very six weeks, especially when that kind of use case spreads. So for that it would be better to upgrade the system Rust only for service packs ie once per year.

The other use case is developers that want the latest and greatest for their own purposes. For that it should be possible to create a parallel installable Rust package and regularly provide newer versions for it. That works for Leap via regular maintenance updates as well as SLE via Package Hub. I've been told that Rust is meant to be pretty good at backwards compatibility so it's probably sufficient to provide one additional Rust package that follows upstream releases, instead of a new one for all kinds of versions like eg Go does.

cu Ludwig

rustup[0] is what developers should use when developing in Rust. It's the

official Rust toolchain installer. I packaged it in my home[1] and submitted

a request[2] in hopes that it would get into Factory, but in the end, I didn't

get much feedback beside from Neal Gompa. I decided to revoke the request. As

you can see in the request, I talked about rust and rustup coexisting, but this

would need a bit more work. I didn't look into this, but I'm willing to do it.

It definitely seems to be possible as documented[3] upstream. This does not address the issue - how do we get from development to a system-buildable and shippable project. This is key here, we need to think about that full experience from development to deployment - not just development.

Right now, we have nothing for the development part, unless we want to stick to

whatever rust version openSUSE provides (which is a bad developer experience).

This is what my proposal would address.

We can not seperate deverloper from the deployment - else we fragment the experience and end up back here. No, the solution for how we ship and develop must be the same.

...

...

[0]: https://github.com/rust-lang/rustup

[1]: https://build.opensuse.org/package/show/home:dmarcoux/rustup.rs

[2]: https://build.opensuse.org/request/show/684463

[3]: https://github.com/rust-lang/rustup#working-with-distribution-rust-packages

-- Dany Marcoux Full Stack Web Developer - Open Build Service

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer

-- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

— Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server SUSE Labs

-- Dany Marcoux Full Stack Web Developer - Open Build Service

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer

— Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-packaging+owner@opensuse.org

On 11/20/19 6:35 PM, Dany Marcoux wrote:

On 19-11-20 00 h 55, William Brown wrote:

...

...

On 20 Nov 2019, at 03:04, Dany Marcoux wrote:

On 19-11-19 16 h 44, Ludwig Nussel wrote:

...

William Brown schrieb:

...

[...] Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers.  From what I can tell Rust has two use cases in the stable distros. The first one is that it's needed by other system components. Regularly updated, more or less leaf packages Firefox as but unfortunately also as low as librsvg. For packages like the latter we most likely do not want Rust to change very six weeks, especially when that kind of use case spreads. So for that it would be better to upgrade the system Rust only for service packs ie once per year.

The other use case is developers that want the latest and greatest for their own purposes. For that it should be possible to create a parallel installable Rust package and regularly provide newer versions for it. That works for Leap via regular maintenance updates as well as SLE via Package Hub. I've been told that Rust is meant to be pretty good at backwards compatibility so it's probably sufficient to provide one additional Rust package that follows upstream releases, instead of a new one for all kinds of versions like eg Go does.

cu Ludwig

rustup[0] is what developers should use when developing in Rust. It's the

official Rust toolchain installer. I packaged it in my home[1] and submitted

a request[2] in hopes that it would get into Factory, but in the end, I didn't

get much feedback beside from Neal Gompa. I decided to revoke the request. As

you can see in the request, I talked about rust and rustup coexisting, but this

would need a bit more work. I didn't look into this, but I'm willing to do it.

It definitely seems to be possible as documented[3] upstream. This does not address the issue - how do we get from development to a system-buildable and shippable project. This is key here, we need to think about that full experience from development to deployment - not just development.

Right now, we have nothing for the development part, unless we want to stick to

whatever rust version openSUSE provides (which is a bad developer experience).

This is what my proposal would address.

To a large extent if you can get the developer experience right and constantly enough updating in tumbleweed it can then be provided in places like SLE via packagehub without a huge amount of extra effort. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B