openSUSE Packaging November 2019

packaging@lists.opensuse.org

19 participants
10 discussions

On how to improve Rust packaging experience for suse

by William Brown

Hi, Recently I've been trying to package some utilities for SUSE and Fedora via OBS that are written in Rust. I'm running into a bit of a problem though. Rust as a language has a very aggressive release cycle compared to what we expect on a platform like SUSE - 6 weeks (https://github.com/rust-lang/rust/blob/master/RELEASES.md). In each cycle Rust releases new features, and despite the core language being "stable" and safe (which is great!) new features tend to be used very rapidly by library authors. For example the release from Version 1.34.0 (2019-04-11) contains convert::TryFrom, which is now in use by a large number of libraries. Rust also has an (unfortunate) requirement that you are essentially forced to use cargo which is a build and dependency management tool. Cargo is extremely opinionated and inflexible which makes it difficult to use. However as it's also a dependency management tool, this has encouraged an npm-style ecosystem of dependencies to spring up on https://crates.io/ - and it's effectively the only way to use libraries in Rust. But additionally, it's promoted a system where a library may have a large number of small dependencies too. Due to the fast release cycle, developers aggressively using new features, and the npm style micro dependency system we have a recipe for problems - if you are not using the latest stable compiler, it's extremely likely that your libraries, or their dependents may not build on your project. Which is exactly the issue I have run into where rust on fedora 30 and opensuse leap 15.1 are simply too old to support the features that have been used in the last 6 months by library developers. Which leads me to the question of "what to do". * I don't believe it's feasible to ask Rust to "slow down". It's just not going to happen, and they will keep adding features that people will "want" to use. * We can't expect people *not* to update their dependencies in projects as that would prevent security updates being included. So we have to accept projects that will use "latest and greatest" complier features somewhere in their dependency graph. * Which leaves distros (like us) speeding up our rust compiler cycle somehow. For opensuse I can probably do something like adding devel:languages:rust as a respository to my project. Would this strategy be viable on SLE? What about for projects which we want to expose packages for fedora or other? Are we able to have toolchains move faster than our base system? I'm really looking for ideas on how we can sustainably use rust projects within the governance and social constraints that exist in the space. Ideas? — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

rust 1.36 build unresolvable as no cargo >=1.35 for Leap 15.2 ARM and PowerPC

by Normand

For ARM and PowerPC, rust build failed (1) as stated above. the available rpms on repos are version 1.32 (2) There was no build failure for Factory that generated version 1.38 (3) What is the "normal" way to solve such problem ? Is it to retrieve in a home project successively source of version 1.33/1.34/1.35 (where to find them ?) and try to build locally ? (1) https://build.opensuse.org/package/show/openSUSE:Leap:15.2:ARM/rust https://build.opensuse.org/package/show/openSUSE:Leap:15.2:PowerPC/rust (2) http://download.opensuse.org/ports/aarch64/distribution/leap/15.2/repo/oss/… http://download.opensuse.org/ports/ppc/distribution/leap/15.2/repo/oss/ppc6… (3) http://download.opensuse.org/ports/aarch64/tumbleweed/repo/oss/aarch64/ http://download.opensuse.org/ports/ppc/tumbleweed/repo/oss/ppc64le/ -- Michel Normand -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

Re: [opensuse-packaging] On how to improve Rust packaging experience for suse

by William Brown

> On 19 Nov 2019, at 20:43, Alberto Planas Dominguez <aplanas(a)suse.com> wrote: > > On Tuesday, November 19, 2019 12:58:35 AM CET William Brown wrote: > > That is funny, because I set myself to help (inside my modest skills) on both > projects (Rust and Kanidm) before this issue arise. As usual I am too late. Never too late, would always be happy to have your input and advice :) > > Anyways. > >> For opensuse I can probably do something like adding devel:languages:rust as >> a respository to my project. Would this strategy be viable on SLE? What >> about for projects which we want to expose packages for fedora or other? >> Are we able to have toolchains move faster than our base system? > > I have a naive TODO: > > 1) Add in _service, script and / or document how to update rustc, and help the > rustc maintaner in the update task more pro-actively > > 2) Add _service, script and document the upgrade of the crates, helping the > work that the Fedora guys are doing in openSUSE. I think that is time for > openSUSE to help here. > > 2.5) Research how the Haskel team is doing their magic, as IMHO is a good > example to follow. > > 3) Using some data-driven mechanism, decide the crates / version will live in > TW and the ones that will live only in the devel project. I am not sure how to > tackle the multiversioning here. > > I hope to automatize all the steps, and do the submit to TW incrementally, as > I am worried about the legal review. > > The compiler (and crates) in Leap and SLE cannot follow this peace, but we can > help upstream to crystallize the idea of 'reference / stable compiler > version'. Maybe we can skip 3 / 4 versions form TW to one in Leap. > > For SLE we can use the backports. The managers can interlock later the version > for SLE, based on the requirements on the software that they are shipping. > > For Kanidm (and any project that uses Rust) this means that the devel version > can compile on TW every day, but for Leap and SLE the version of the project > and the compiler cannot change, and the fixes needs to be backported. This is > aligned with how every product works: release and backport the fixes. The issue becomes fixes to vendored dependencies (of which there are ~200 I think total. I directly have ~30 deps). Fixes to kanidm are simple to patch and backport, but fixes for vendored code ... not so much. That's what will really be the issue is rust has no concept of a security update vs a feature one - and most crates treat them as the same - updates. So you will have a vendor library that will both have security *and* feature changes, which will quickly be a maintainers nightmare. > > This can get more complicated with the crates. One usual solution in OBS is to > create a subproject for an specific release of the project (lets say, for > example, Kanidm 1.0), and link the old version of the crate from > devel:languages:rust:crates to the new project. > >> I'm really looking for ideas on how we can sustainably use rust projects >> within the governance and social constraints that exist in the space. >> Ideas? > > Neal Gompa wrote: > >> That said, Fedora and Mageia update the Rust compiler within a week of >> a new stable compiler release. As of right now, Fedora 29 and higher >> all have Rust 1.39.0. CentOS/RHEL 7 will receive Rust 1.39.0 soon >> too[1]. > > The devel project was updated almost the same day in OBS. We (openSUSE) > submitted the new version, created the upstream bug report about the issue > that makes impossible the bootstrap of the compiler, and backported it to our > packages. I am not going to check Fedora nor Mageia, but if they need to > compile 1.39 with 1.38 (and later with 1.39) they need to use this patch. > > openSUSE do not have it yet on TW because we recompile all the dependencies, > and this 1.39 is having issues with FF[1]. I am not sure if other > distributions ship the FF compiled with 1.3X and the rustc from 1.39 > independently. We cannot do that: we need to fix the issues as we find them. > > Simon Lees wrote: > >> One reason that it is done like this by default is because core SLE / >> Leap packages shouldn't be break API/ABI for updates, so in the vast >> majority of cases building against standard is fine and saves huge >> amounts of resources on obs. > > That is true. But as we cannot have ABI compatibility with rust (not stable > ABI and almost all is statically linked), I wonder if this will help to speed > the update of the compiler in SLE and Leap. > > Neal Gompa wrote: > >> Rust in openSUSE is mostly community, driven within the Fedora Rust >> SIG (Yes, Fedora Rust runs the openSUSE Rust stack). The Rust compiler >> package does get semi-regular contributions from two SUSE employees, >> but the bulk of the Rust stuff is through the Fedora Rust SIG. > > : ? is Luke Jones from Fedora? > > But in any case this is 100% true in the crates side. This is almost all done > by the excellent Fedora Rust SIG. I really think that it is time for openSUSE > to help here and with the upstream scripts from Fedora that is doing this > magic. > > I want to do my part here. I think to help (because I want to help here too), we need to think of actionable ways forward. I think we need to think about what rust *should* look like in suse, and work from that vision? > > [1] https://build.opensuse.org/project/monitor/openSUSE:Factory:Staging:M? > arch_x86_64=1&defaults=0&failed=1&repo_standard=1 > -- > SUSE Software Solutions Germany GmbH > Maxfeldstr. 5 > 90409 Nuremberg > Germany > > (HRB 36809, AG Nürnberg) > Managing Director: Felix Imendörffer — Sincerely, William Brown Senior Software Engineer, 389 Directory Server SUSE Labs -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

Get rid of :xxx package in multibuild

by Hans-Peter Jansen

Hi, here's a multibuild for PyCryptodome, that should result in two packages: python-pycryptodome and python-pycryptodomex: https://build.opensuse.org/package/show/home:frispete:python/python-pycrypt… How do I get rid of the ":x" package? Thanks, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

Re: [opensuse-packaging] On how to improve Rust packaging experience for suse

by Neal Gompa

On Tue, Nov 19, 2019 at 5:15 AM Alberto Planas Dominguez <aplanas(a)suse.com> wrote: > > Neal Gompa wrote: > > > That said, Fedora and Mageia update the Rust compiler within a week of > > a new stable compiler release. As of right now, Fedora 29 and higher > > all have Rust 1.39.0. CentOS/RHEL 7 will receive Rust 1.39.0 soon > > too[1]. > > The devel project was updated almost the same day in OBS. We (openSUSE) > submitted the new version, created the upstream bug report about the issue > that makes impossible the bootstrap of the compiler, and backported it to our > packages. I am not going to check Fedora nor Mageia, but if they need to > compile 1.39 with 1.38 (and later with 1.39) they need to use this patch. > > openSUSE do not have it yet on TW because we recompile all the dependencies, > and this 1.39 is having issues with FF[1]. I am not sure if other > distributions ship the FF compiled with 1.3X and the rustc from 1.39 > independently. We cannot do that: we need to fix the issues as we find them. > In Fedora, since we ship Firefox latest stable and Rust latest stable, we usually wind up encountering the same problems, but nothing specifically is forcing us to deal with them at the same time other than a confluence of scheduling. I'm guessing because we don't have the automatic reverse dep rebuild thing happening with Firefox (Koschei is currently not testing Firefox in this manner[1]), we probably didn't notice. This is definitely something that we should improve. It's bad if Firefox can't be built with latest rustc. [1]: https://koschei.fedoraproject.org/package/firefox > Simon Lees wrote: > > > One reason that it is done like this by default is because core SLE / > > Leap packages shouldn't be break API/ABI for updates, so in the vast > > majority of cases building against standard is fine and saves huge > > amounts of resources on obs. > > That is true. But as we cannot have ABI compatibility with rust (not stable > ABI and almost all is statically linked), I wonder if this will help to speed > the update of the compiler in SLE and Leap. > > Neal Gompa wrote: > > > Rust in openSUSE is mostly community, driven within the Fedora Rust > > SIG (Yes, Fedora Rust runs the openSUSE Rust stack). The Rust compiler > > package does get semi-regular contributions from two SUSE employees, > > but the bulk of the Rust stuff is through the Fedora Rust SIG. > > : ? is Luke Jones from Fedora? > Technically, no, though he hangs out with us in the #fedora-rust IRC channel. But Luke worked with Igor and Josh to get the Rust compiler packaging a while ago. It's now quite a bit more similar to the Fedora one, but not exactly the same just yet. There have been changes lifted from the Fedora rust compiler spec in order to fix issues with building new versions from time to time. I'm hoping that once I get a larger chunk of time, I can sit down and integrate the variations for building on SUSE into the Fedora spec, like we did for Mageia, and merge the specs. That will make ongoing maintenance of the compiler _much_ easier. Igor and I have contemplated setting up an "upstream" Dist-Git that would be wired into COPR and OBS to build the Rust stack for Fedora, Mageia, and openSUSE continuously, also permitting us to trivially push releases to all three distributions quickly and easily. We're not there yet, though... > But in any case this is 100% true in the crates side. This is almost all done > by the excellent Fedora Rust SIG. I really think that it is time for openSUSE > to help here and with the upstream scripts from Fedora that is doing this > magic. > > I want to do my part here. > I'm glad you want to help! You did a great job helping us with rust2rpm in this most recent cycle, and I'm looking forward to working with you more. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

How to properly define versioned dependencies to python-pytest

by Hans-Peter Jansen

after some struggling over the weekend, I finally have my python repo mostly back in shape again. What still bugs me are versioned dependencies to pytest. Here's the current state of affairs: python-pytest (aka pytest version 5) is python3 only. This is accompanied with python-pytest4 and python-pytest3, both conflicting with python-pytest and providing python-pytest = 3.xx/4.xx. Let's ignore 3.xx for now. python-pytest provides a python3-pytest. python-pytest4 provides python{2,3}-pytest4 Tumbleweed has this prjconf in place: # python2-pytest4 should win over python2-pytest # python3-pytest should win over python3-pytest4 since pytest5 is python3 only Prefer: python2-pytest4 python3-pytest4 but I don't understand, *how* python3-pytest is winning over python3-pytest4 here? d:l:py:backports uses this prjconf: # there is python-pytest4 and python-pytest %if 0%{suse_version} < 1550 Prefer: -python-pytest4 -python2-pytest4 -python3-pytest4 %endif This results in picking python3-5.xx and python2-4.xx, if unversioned. In my youthful(lie) carelessness, I would have hoped to find some prjconf in d:l:py:pytest, that would deal with this issue, but nothing. https://en.opensuse.org/openSUSE:Build_Service_prjconf doesn't cope with such complex issues. In my project home:frispete:python, TW seems to get this right (how?), but the Leaps are suffering when using: %if 0%{?sle_version} >= 150000 && 0%{?is_opensuse} Prefer: -python-pytest4 -python2-pytest4 -python3-pytest4 %endif While this almost does, what I want, it fails for versioned pytest requests, which result in: python-coverage: conflict for providers of python2-pytest >= 4.4.0 needed by python2-pytest- xdist, (provider python2-pytest4 conflicts with python2-pytest) python-docker: conflict for providers of python2-pytest >= 3.6.0 needed by python2-pytest- timeout, (provider python2-pytest4 conflicts with python2-pytest) Although the Leaps are defined identical, 15.0 differs significantly from 15. {1,2}, but again, I don't understand, why it is differing. Could some brave soul shed some light on this, please? Thanks, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

What results in python-bytecode-wrong-magic-value for some builds?

by Hans-Peter Jansen

Hi, any idea, where these dreaded: [ 310s] python3-nose2.noarch: W: python-bytecode-wrong-magic-value /usr/lib/python3.7/site-packages/nose2/tools/__pycache__/such.cpython-37.pyc expected 3390 or 3391 or 3392 or 3393 (3.7), found 3394 (unknown) [ 310s] The "magic" ABI version embedded in this python bytecode file isn't equal to [ 310s] that of the corresponding runtime, which will force the interpreter to [ 310s] recompile the .py source every time, ignoring the saved bytecode. stem from in some builds? E.g. here: https://build.opensuse.org/package/live_build_log/home:frispete:python/pyth… Thanks, Pete -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

Re: [opensuse-factory] Re: [opensuse-packaging] Changing %_libexecdir to /usr/libexec

by Martin Wilck

On Fri, 2019-11-08 at 10:06 +0100, Bernhard M. Wiedemann wrote: > On 07/11/2019 21.45, Neal Gompa wrote: > > /usr/lib/module-init-tools could just be a symlink to > /usr/libexec/module-init-tools so that users can switch whenever they > are ready. > This should give a smooth migration path. Yes, I thought about that. I'm not sure about the usefulness though. Definitely, it's only useful if some day we really remove the symlink. We should probably set a calender entry because it will be forgotten otherwise; the grace period has to be several years, for SLE at least. /usr/lib/module-init-tools is already a legacy path (IIRC module-init- tools last existed in SLE11) which we never dared to rename. Perhaps it's time to be bold and move /usr/lib/module-init-tools to /usr/libexec/suse-module-tools :-) Martin

1 year

Changing %_libexecdir to /usr/libexec

by Michael Schroeder

Hi folks, OBS request 738494 from Neal will change %_libexecdir from /usr/lib to /usr/libexec. This was proposed in July on the opensuse-packaging list with only Thorsten and Jan replying (Jan being in favor). I don't want to merge it right now without giving you some time to discuss this change. So please, if you have good reasons why this is a bad idea please speak up! If there are no strong objections, I'll merge the request at the end of next weak. Cheers, Michael. -- Michael Schroeder mls(a)suse.de SUSE LINUX GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-packaging+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-packaging+owner(a)opensuse.org

1 year

Board Decision on groups in spec files.

by Simon Lees

Hi All, It likely comes as no surprise to anyone one that the groups in spec file issue was raised with the board (one of these was on a public list). In this specific email we are acting in our role of "helping resolve conflicts" complaints were also made directly about the behavior of certain individuals (again including on this list). The board has chosen to deal with these issues in private as is our general approach to such issues. Firstly the board generally agrees that in the following thread [1] there was a plan and general agreement to follow it for the removal of groups if certain other issues were resolved. As such we believe they acted with good intentions when starting to remove group support from various places such as Yast, rpmlint and spec cleaner. The board decided to wait until the discussion on new proposals settled down, before going forward with the issue, at the time we had the discussion we believed this to be the case. Of all the proposals discussed the board agreed that the one submitted by Ludwig [2], to move groups into the repository meta data is technically the best solution and the one most broadly supported by contributors. As such we welcome any contributions from the community towards getting this working and integrated into tooling where it makes sense such as Yast and software.opensuse.org Given that our core tooling no longer displays groups and that the board has a copy of all the existing groups that can be used as a starting point for the new system the board has decided that including groups in spec files should now be optional with the final decision resting with the maintainer as some maintainers may still use there spec files in distro's that still support groups although we believe most packages going into factory wont. We realize that this decision will not make everyone happy but hopefully it eventually leads to better group support in the future. 1. https://lists.opensuse.org/opensuse-factory/2018-05/msg00460.html 2. https://lists.opensuse.org/opensuse-factory/2019-10/msg00263.html -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B

1 year

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

openSUSE Packaging November 2019