Devel project migrations happening soon
Hi everyone, The Git project has been in development for some time. We've had a few different attempts and proposals but eventually we've settled on moving complete source management to Git. This will be packages *and* projects. A hybrid approach, while looking simpler on paper, doesn't actually provide any benefit of migrating anything. We'll just end up in a world where we have disadvantages of both OBS and Git. SLE 16 and Leap 16 will be in Git. Actually, they are already in Git. Unfortunately, because packages sources are in OBS, submission process from Factory to SLE 16 is ... cumbersome. Packages are merged into an OBS project and then imported into Git and then imported into git project that builds SLE 16.... the package history is mostly lost in this process and things look ... To be able to submit directly to these projects and retain package histories and project histories, we have to move the sources to Git This is where this devel project migration comes in. It's vital for SLE16 success. This will also be the most "messy" part of the process, where we have least tested infrastructure and least availability of finished tooling. For information of what is happening, etc. please see, https://en.opensuse.org/openSUSE:OBS_to_Git How will these devel projects work in Git? For an example, please see https://src.opensuse.org/cockpit . The _ObsPrj repository is the project and the rest are packages. The project is referenced and built in OBS at systemsmanagement:cockpit. Package submissions for cockpit to other project, like openSUSE:Factory, still go via osc sr systemsmanagement:cockpit <package> openSUSE:Factory like before. This will only change once Factory switches to Git. If you have questions, please read the linked wiki page above first then ask away. - Adam
On Dienstag, 25. Februar 2025, 19:07:22 CET Andrei Borzenkov wrote:
project meta is still not in git. It is currently OBS only, since it specifies the used git url and one git could be build multiple times. (eg. different git branches or different archs). However, there exists already also the way to specify the build using _pbuild file and run pure local builds using pbuild tool (part of build package). We plan to sync repository definition parts of OBS meta and _pbuild later on, but this is not set in stone yet. However the build config (aka prjconf) is already maintained in the project git in the _config file. So commits and merge requests to that are already supported. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 2/25/25 19:07, Andrei Borzenkov wrote:
Project config is in _config in _PrjGit The repository setup (the build part) is still defined in the OBS project but we have plans of moving this to _PrjGit too. - Adam PS. The _PrjGit doesn't have to be called exactly this, but for purposes of the devel project and future "home" projects, this will be the name.
On 26. 02. 25, 9:24, Adam Majer wrote:
On a similar note: where is package meta for scmsync _projects_? Use case: Disable building some packages on some repositories. I.e. where to put: <build> <disable repository="15.6"/> <disable repository="Fedora_40"/> </build> ? thanks, -- js suse labs
On 2/27/25 11:05, Jiri Slaby wrote:
It will be part of the project meta BuildFlags. https://src.opensuse.org/cockpit/_ObsPrj/src/branch/master/_config BuildFlags: excludebuild:cockpit-suse-theme %if "%_repository" == "openSUSE_Factory_ARM" || "%_repository" == "openSUSE_Tumbleweed" BuildFlags: excludebuild:local-npm-registry %endif Is way to disable builds now. https://openbuildservice.org/help/manuals/obs-user-guide/cha-obs-prjconfig#s...
On Donnerstag, 27. Februar 2025, 12:24:26 CET Dominique Leuenberger wrote:
In general the goal is to obsolete the parallel meta information with the project git and having everything what we need in the sources. Either package sources or additional configuration files in the project. Build flags were often a problem since they could not be transfered into other projects as things like repository names did change. Also problems with old build results lead to issues where content is not rebuildable. So our vision so far is to obsolete them and try to stick to the exclude mechanic only. This can be specified in the package source already (eg. ExcludeArch in spec files or special tags in Dockerfile or kiwi files). We have no replacement for useforbuild yet. It could be added, but do we have really a usecase for it? In the past it caused mostly trouble. It is easy to add, if there is a usecase. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Thu, 2025-02-27 at 13:09 +0100, Adrian Schröter wrote:
I mostly use "useforbuild" to 'come out of trouble' in devel projects like Base:System, where bootstrap cycles are a pita; Granted, it's a special case, and if used unwisely, results in fun/weird issues, as you correctly state. Also, disabled packages' have lead to many questions towards me why something would behave in a devel prj as it did - and often it was indeed a stale, disabled package being used. As usual: aiming the gun at your foot and shooting will hurt; aiming at the center of the proper target helps.
On Thu, Feb 27, 2025 at 01:07:29PM +0100, Dominique Leuenberger wrote:
For other projects that do bootstrapping this is also useful - eg. kernel, and probably pretty much any programming language. Unfortunately, it's now already broken because it does not work with multibuild. When a broken revision of something that is used to build itself is uploaded unless "useforbuild" flag is available it's not possible to fix by simply uploading a new, fixed version. Thanks Michal
On Thu, 27 Feb 2025, Adrian Schröter wrote:
Yes, useforbuild setting is critical for devel:gcc. Also note that while some build enable/disable is source specific in many cases (devel:gcc!) it's specific to the repository we build against! So IMO there needs to be a way to amend/override settings in the OBS build project, having this only as part of the source is not enough (in fact it's wrong). And I agree, having it as part of the project config (which also has its build and its source part!) is just wrong. Back to the drawing board. As of _service - great, now I have to walk back for gcc15 which I finally switched to a source service after years of using a custom script. It could have been communicated earlier that source services are effectively deprecated ... Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Thu, 2025-02-27 at 13:08 +0100, Richard Biener wrote:
_service itself is fine - just not server side running (they were never acceptable into Factory though, so no change there) almost all of the GNOME stack packages use a _service file (obs_scm mode="manual) and a local helper script to update to the corresponding git tag/ref https://github.com/DimStar77/openSUSE-helpers/blob/master/Tumbleweed-Dev-Scr... ) Cheers, Dominique
On Thu, 2025-02-27 at 13:13 +0100, Dominique Leuenberger wrote:
What about automated builds triggered (e.g.) from GitHub via Webhook and OBS workflow? That won't be possible any more? That's a big step backward. Not for factory, but for devel projects and other projects that just try keep up with upstream without manual intervention. Martin
On 27/02/2025 13:08, Richard Biener wrote:
And in the Java:packages, we have a load of source services. Some of them are there also because they allow not only to tarball a repository, but also to exclude non-redistributable stuff from the resulting tarball. I migrated several packages to _service files because of the explicit rejection of some files by legal review :( Cheers F.
On Donnerstag, 27. Februar 2025, 22:02:22 CET Fridrich Strba via openSUSE Factory wrote:
again, these should still work. It is just about server side service runs (not local and neither buildtime). But this is a policy thing of the distribution and it always was. So nothing is changing for a package which already lifes in any openSUSE: or SUSE: code stream. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Thu, 27 Feb 2025, Richard Biener wrote:
So after pondering a bit on how migration could work for devel:gcc with some of the OBS features (temporarily) gone I think the solution is likely to split up devel:gcc on the OBS side and have devel:gcc itself only build oS:Factory repositories (where useforbuild is generally enabled and wanted) and have repositories for SLES and Leap be built in one (or many) separate OBS projects that can then disable useforbuild project-wide and possibly pick packages from src.o.o on a package-by-package basis (or have a filtering import, for example to exclude 'gcc' from building in these OBS projects). The reason this is now all in devel:gcc is because sources are in OBS and it's how most other devel projects handled this. Thus - moving sources to src.o.o is the opportunity to re-think the layout of OBS build projects when it comes to building packages for SLES and Leap (as "backports"). Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On 27. 02. 25, 12:21, Adam Majer wrote:
Strong NACK. This is package metadata.
Do not ever try to push this mess into any of my devel projects. (I assume (_hope_) you export/import also package meta settings.) How do I BTW disable builds per arch _and_ per repo? Using the nested ifdeffery in a global prj settings is a NO go. -- js suse labs
On 2/27/25 12:45, Jiri Slaby wrote:
Do not ever try to push this mess into any of my devel projects. (I assume (_hope_) you export/import also package meta settings.)
Currently they are ignored. In nodejs project, these were used to keep most of the obsolete repo not building, but I guess it's an excuse to kill unused packages. But we will have to deal with this on a case-by-case basis.
How do I BTW disable builds per arch _and_ per repo? Using the nested ifdeffery in a global prj settings is a NO go.
I suggest not looking in the Factory prjconfig :-) But like I write, we'll have to deal with this on a case-by-case basis. This conversion is not a 100% automatic process and needs review, etc. - Adam
On 27. 02. 25, 13:41, Adam Majer wrote:
Been there, seen that (many times). So that's exactly the reason :). It's unmaintainable mess. E.g. you remove a package, but metadata remain in prj conf. Adrian appeared to suggest there will be per-pkg configs in the git -- somewhere else in this thread. That'd indeed make a lot of sense.
But like I write, we'll have to deal with this on a case-by-case basis. This conversion is not a 100% automatic process and needs review, etc.
thumbs up thanks, -- js suse labs
On Thu, Feb 27, 2025 at 12:45:20PM +0100, Jiri Slaby wrote:
This is absolutely atrocious. Previously you could specify what to build as data, now it's code with undefined syntax. Please provide a configuration file for this as was the case pre-git. Thanks Michal
On Tue Feb 25, 2025 at 5:19 PM CET, Adam Majer wrote:
For information of what is happening, etc. please see,
OK, I am a bit slow, probably. Could you please describe a step by step, what should I do, to which branch I should submit PR, with individual commands, when I want to update my SCM-enabled package? I have in https://src.opensuse.org/mcepl/pinentry-rofi/src/branch/switch-to-sourcehut/ change of the upstream project location (I am the upstream, and I moved all my projects from the self-hosted git.cepl.eu to Sourcehut). See https://src.opensuse.org/pool/pinentry-rofi/compare/devel...mcepl/pinentry-r... How can I get this change to Factory? Best, Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 The world is coming to an end! Repent and return those library books!
On 2/26/25 09:11, Matěj Cepl wrote:
Your SCM enabled package will be cloned into the project instead of the OBS import. Then you do a `osc sr <devel project> <package> openSUSE:Factory` This step will change and this will be communicated here and via submit request reject messages, so you will not miss it ;) So, to submit changes to new devel projects, 1. fork on gitea, like github 2. git clone <your repo> 3. do changes, git add, git commit, git push 4. make pull request 5. someone will accept it (or you?) 6. osc sr ... openSUSE:Factory
If the package is already in /pool, it will be forked from pool with additional pending changes on top (in case /pool is old) The idea here is not to overwrite history. - Adam
Wed, 26 Feb 2025 09:31:59 +0100 Adam Majer <amajer@suse.de>:
The idea here is not to overwrite history.
Which history? Is force-push allowed, or example? For a large number of packages a single history can be kept. There is no need to have a separate history for Tumbleweed and SLES. I think both will naturally use two different instances. Is there a way for packages to opt-in to such unified history? In other words: "git push $remote HEAD" to different remotes? Olaf
On Mittwoch, 26. Februar 2025, 10:15:47 CET Olaf Hering via openSUSE Factory wrote:
well, yes, but we will have some synced repositories which won't differ.
Is there a way for packages to opt-in to such unified history? In other words: "git push $remote HEAD" to different remotes?
The official repos won't accept history rewriting by policy, but you could fork into your own space of course. Here you can rewrite before submitting something to a devel project. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
Wed, 26 Feb 2025 11:41:33 +0100 Adrian Schröter <adrian@suse.de>:
Well, this is sort of a start from scratch. So there should be a way to start "clean" while the overall state is still maturing, until the point of no return. Take a simple pkg like "build-compare". There will be a "primary" git repo of it somewhere, in the various "primary" git instances. I have not looked what those instances are, and what state that particular pkg has (i.e. what the git hash of the main branch is). I assume the git hash differs, even if the content is identical. At some point I will have a local copy if the devel git repo, and I will push directly to it, in case an update is required. I would like to push that HEAD to the various "primary" git instances. I assume an update from TW to SLES will not be as simple anymore as ibs sr openSUSE.org:openSUSE:Factory/pkg $prj It is certainly up to each individual maintainer to decide how much time and energy they want to spent into multiple branches of their pkgs, just like they do today. Olaf
On 2/26/25 12:02, Olaf Hering wrote:
I assume an update from TW to SLES will not be as simple anymore as ibs sr openSUSE.org:openSUSE:Factory/pkg $prj
For now, it is. In the future, this will be replaced by a pull request on internal gitea instance, where you make a PR from factory branch to SLE16 branch. If you want to do this via git only tooling, then it would be git clone gitea@...pool/pkg git push origin origin/factory:refs/for/slfo_main/factory and PR is created. We'll probably have tooling to do this in one line at a later point. - Adam
On Wednesday 2025-02-26 10:15, Olaf Hering via openSUSE Factory wrote:
I read it as "The idea here is that the /migration script/ should not overwrite history of already git-ified packages." Force-push is allowed in Git currently.
<scmsync>https://foo.bar/prj/pkg#master</scmsync> <scmsync>https://foo.bar/prj/pkg#sles</scmsync>
Hi Adam, On Tue, 2025-02-25 at 17:19 +0100, Adam Majer wrote:
If you have questions, please read the linked wiki page above first then ask away.
Is there already a timeline when it will be possible to use Gitea with just the osc command? While I'm a huge proponent of git, I think it would nice to have all the necessary "git" and "tea" commands mapped to corresponding "osc" commands for easier use. Thanks, Adrian
On Wed, 26 Feb 2025, John Paul Adrian Glaubitz via openSUSE Factory wrote:
Also while the wiki page references https://en.opensuse.org/openSUSE:OBS_with_Git I find documentation extremely lacking. This page says I have to use the gitea web frontend and jump through hoops to do stuff that was easily possible with just three osc commands (branch, commit, sr). It also lacks a description of the workflow for changes done by a maintainer without a prior branch (so in osc terms, just edit, commit). I can also imagine that osc 1.9.0 isn't up-to-date enough (that's the version in SLES15 SP6), but can't find any required version information either. Please provide more extensive documentation before doing any devel project migration. Richard.
Thanks, Adrian
-- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On 2/26/25 10:41, John Paul Adrian Glaubitz wrote:
I would like to reduce this to, git clone git commit git push origin HEAD:refs/for/<target_branch>/<session> https://git-repo.info/en/2020/03/agit-flow-and-git-repo/#agit-flow-implement... This PR creation/update workflow is part of Gitea already. Of course, if you have write access to the devel project package, you can just push changes there directly too. - Adam
On Wed, 2025-02-26 at 11:17 +0100, Adam Majer wrote:
Isn't it more like "open web browser, navigate to src.opensuse.org, find the package you want to work on, find and copy the git URL, paste it into a terminal after typing git clone"?
I have only used the "tea" utility so far and it's not as elegant to use as "osc sr" as it requires copying the commit messages manually. Temporary files are also not automatically ignored.
Of course, if you have write access to the devel project package, you can just push changes there directly too.
Adrian
On 2/26/25 12:11, John Paul Adrian Glaubitz wrote:
For now, yes, because tooling is ... sparse. This will improve. Then again, I don't know how often I've actually deleted a local osc checkout? Very rarely. The packages I work on are checked out already and I just `osc up` and `osc commit` new changes. I don't really see that changing with git -- the checkouts remain. And if you want to checkout many packages or all packages from a project, you could even use the _ObsPrj to help you there. Also, just to underline this here: This is work in progress. The usability will improve as it's developed further. But we cannot wait until things are "perfect" before we start to deploy it. - Adam
On Wed, 2025-02-26 at 12:31 +0100, Adam Majer wrote:
That might be true for packages you really maintain and are dedicated. But there are many people (including myself) that are just helping out left and right. My current workflow for 'fly-by fixes' is:
while broken fix spec file osc build done osc vc / osc ci / osc sr --cleanup; and I'm done for this package. There is a lot of elegance/help by the tooling leveraged, where many things are just working without having much to worry (incl. a home:dimstar:Factory project setup that stays around, targetting exactly to my usecase of test-building against all stagings; as that's mostly what I work on) Having to find the package in gitea, forking, git cloning (and somehow making it visible to my test project, so I can build against the various stagings) seems to be many more steps than it used to be (incl. switching between browser and cli - which is bad for productivity) No question, it CAN be done - but it's far from the streamlined process osc provided so far. Cheers, Dominique
On 2/26/25 12:51, Dominique Leuenberger wrote:
Thanks for the workflow. I've added your workflow to https://github.com/openSUSE/openSUSE-git/issues/56#issuecomment-2684916060 I agree, this is possible/doable and it's something we are aiming for. - Adam
On Wed Feb 26, 2025 at 11:17 AM CET, Adam Majer wrote:
So, does this work or not? Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 In the government of this Commonwealth, the legislative department shall never exercise the executive and judicial powers, or either of them: The executive shall never exercise the legislative and judicial powers, or either of them: The judicial shall never exercise the legislative and executive powers, or either of them: to the end it may be a government of laws and not of men. -- John Adams in the Article XXXth of the Constitution of the Commonwealth of Massachusetts
On Wed Feb 26, 2025 at 2:16 PM CET, Adam Majer wrote:
Oh, this is cool: https://src.opensuse.org/pool/pinentry-rofi/pulls/4 . Thank you! (and SR has not been created again) Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Whenever Christ our life is revealed, then also you will be revealed with Him in glory. -- Colossians 3:4 (Green’s Literal Translation)
On Wed Feb 26, 2025 at 10:41 AM CET, John Paul Adrian Glaubitz via openSUSE Factory wrote:
I have some thoughts on osc branch replacement in https://git.sr.ht/~mcepl/gosc/tree/master/item/git-devproj.sh (in other branch to be rewritten in Python), but it hopelessly incomplete. Best, Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 The ratio of literacy to illiteracy is a constant, but nowadays the illiterates can read. -- Alberto Moravia
On Mittwoch, 26. Februar 2025, 10:15:07 CET Richard Biener wrote:
indeed, this is our goal to have it as simple. Daniel is working on "git-obs", which is actually just a different frontend of osc. But named this way for easier integration into git. When you install a curent osc 0.12.x you get it installed as well. There are commands to work directly with gitea. Esp the user setup can be done. Daniel wrote this page as starting point: https://opensuse-commander.readthedocs.io/en/readthedocs-test/git-obs-quicks... There is also a fork command, unfortunatly it is not yet doing the gitea _and_ obs part yet. I will look into getting an "osc branch" replacement soonish here...
Right, please take the current stable version from openSUSE:Tools. Maintenance updates will follow later, but are currently not fast enough. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Wednesday 2025-02-26 10:15, Richard Biener wrote:
Indeed, `osc branch` would have to learn to call the gitea.opensuse.org/project/package/fork REST interface (or so) instead, which no one has implemented yet to my knowledge. You aren't volunteering, are you?
That is literally just `$EDITOR; git add/ci/push` at the package level and follows from this paragraph: Whenever obs_scm_bridge detects a change in the Git repository, a new OBSSCM commit is created from the files in the referenced Git revision.
On Wed, 26 Feb 2025, Jan Engelhardt wrote:
But I'm unsure/scared about the talk about git lfs - I've never had to use this. When I update the tarball to something new or trigger a local service run to do this, a git add is wrong, right? Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Mittwoch, 26. Februar 2025, 11:47:31 CET Richard Biener wrote:
osc branch itself won't do it, because it would need to do some thing different. And the user needs to become aware of it. *But* it can tell the user how to call git-obs fork-obs-package ... in future: https://github.com/openSUSE/osc/pull/1723 -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
Hello Adam, Am Dienstag, 25. Februar 2025, 17:19:01 Mitteleuropäische Normalzeit schrieb Adam Majer: ....
I guess this is the resason why I see this coming up : docb@P1:~/buildservice> osc maintainer xxxx fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). (fortunately this message is not that fatal as it sounds, at least I can work with osc)
For information of what is happening, etc. please see,
That does not really answer my questions. As a normal packager, what do I have to do? Thanks Axel
On 26. 02. 25, 10:22, Axel Braun wrote:
That's https://github.com/openSUSE/osc/issues/1707 -- js suse labs
On 2/26/25 10:22, Axel Braun wrote:
Hello Adam,
Hello Axel,
I was going to write "nothing to do", but actually there is Go to src.opensuse.org, login (OBS/bugzilla account) to create an account, click on Settings (under your profile) and the SSH/GPG keys and add your SSH key so you can work on the repositories.https://github.com/openSUSE/openSUSE-git/issues You are also welcome to add your GPG key there as well. For now, this is not really used for anything but I would like to make it useful in the future. An Error 500 on login with workaround is tracked here - https://github.com/openSUSE/openSUSE-git/issues/77 Another thing to do is install git-lfs I will add this to the guide. - Adam
On Wed, 2025-02-26 at 11:10 +0100, Adam Majer wrote:
Since you brought up SSH set up: I tried to do that yesterday but nowhere in the src.o.o UI was I able to find the instance / servers SSH fingerprint. Is this really not exposed anywhere in the UI? Is it documented anywhere outside of src.o.o? If so, where? If not, please do document the severs SSH fingerprint(s) somewhere so we can actually verify it. -- Have a lot of fun!
On 2/26/25 14:02, tux93@opensuse.org wrote:
The keys are below. Also posted in the GitHub ticket you created and will be put on the front-page of Gitea. I've also put them in the wiki page I originally linked t, but you know, wiki :D
- Adam
Hey, On 2/25/25 5:19 PM, Adam Majer wrote:
Who is "we"? Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson
On Tue, 25 Feb 2025 17:19:01 +0100, Adam Majer wrote:
I'm willing to try it out for a new devel project I created recently (Kernel:firmware), but it's not clear how the new gitea organization is assigned. Will it be request, review & approval? And any standard naming rules? thanks, Takashi
On 2/26/25 16:20, Takashi Iwai wrote:
We are working on a prefix for user created orgs, https://github.com/openSUSE/openSUSE-git/issues/79 The idea is for user created Orgs to have prefix `OBS_` and then these will automatically have a _PrjGit repo created/updated as packages are forked/created/deleted/updated/etc. This _PrjGit can then be referenced in OBS for building. This is meant as replacement for home projects and forks from devel projects/factory/etc. The prefix would differentiate between regular user created and admin created orgs (eg. devel projects) There really isn't much nomenclature. Gitea only supports flat org structure, and you can use [_A-Za-z0-9]+ as valid org names. You can edit the wiki if you prefer something specific. For example `firmware` for the Kernel:firmware would sound reasonable? - Adam
On Wed, 26 Feb 2025 16:37:38 +0100, Adam Majer wrote:
Now it comes to the question how to create a new organization on gitea. Is there a way to request a new official organization for a devel project? Then, what'd be the next step to put package there? Should I fork from pool to the new organization? Sorry for questions after questions, but such step-by-step instructions couldn't be found on the documentation, so far. thanks, Takashi
On 2/27/25 16:29, Takashi Iwai wrote:
I will create the organization
Then, what'd be the next step to put package there? Should I fork from pool to the new organization?
I will import the packages and then let you guys know the state in advance. The biggest problem during migration is that if something is broken with tooling and I need to fix it, it's just simpler to delete the repos and re-create them than to mess around with things already there.
Sorry for questions after questions, but such step-by-step instructions couldn't be found on the documentation, so far.
No problem. For now, we have Organization creation disabled. This will change in the future. It's to prevent spam, etc. Org creation can be enabled on per-user basis, which will help in operations. I will move the kernel-firmware migration higher on my list. Thanks! - Adam
On Wed, 2025-02-26 at 16:28 +0100, Adam Majer wrote:
Does that mean I can't do multiple test projects as was currently done in home:dimstar:* for multiple purposes? e.g. I have home:dimstar:Factory for my 'regular, short-lived' Factory fixes (as mentioned early) home:dimstar:meson - a longer-lived branch where I was experimenting for RPM 4.20 / BuildSystem: syntax (with different prjconf, to get this going) home:dimstar:kill-xdm: a very, very long lived branch (which gets too few attention) working on multiple angles of the distro. Granted, many of my projects are not just for building a package - that part has always been easy - but to actually prepare rather complete stack-updates (or proposing major reworks like dropping/obsoleting - or at least no longer mandating - the xdm wrapper) Would that translate in generating different orgs which are then linked up to the different projects in OBS? OBS projects can come and go Cheers, Dominique
On 2/26/25 16:35, Dominique Leuenberger wrote:
Does that mean I can't do multiple test projects as was currently done in home:dimstar:* for multiple purposes?
You'll be able to have as many test orgs as you'll desire :-) We may want to put a limit on this somehow in the future for obvious reasons, but there will not be a 1 org limit. The need for prefix is to differentiate admin created orgs from user created orgs and to stop namespace clutter.
Would that translate in generating different orgs which are then linked up to the different projects in OBS?
In the simplest setup, yes, one org per OBS project. You can set it up differently, but it's then less simple to follow. - Adam
On 26/02/2025 16:47, Adam Majer wrote:
What problem was supposed this git migration solve? Was solving that problem so important as to mess up with workflows of hundreds of people? Workflows that were working well and people were reasonably productive with them. This might be so nice on a drawing board or flip-chart. But for the while the whole advantage I get from all this is to have git. For which price?
The need for prefix is to differentiate admin created orgs from user created orgs and to stop namespace clutter.
But, this "namespace clutter", was it only aesthetical problem, or did it actually create a real issue in terms of resources or somesuch?
Cheers Fridrich
On 27. 02. 25, 6:40, Fridrich Strba via openSUSE Factory wrote:
(I am not involved in the proposed changes.) The price is IMO not so high. Have you actually tried this all yet? With osc, I often hit its limits. For example have you ever tried to find out who and _why_ added/changed/(and the hardest part in osc: removed) a line in a .spec file (i.e. find a .spec revision and also corresponding .changes one)? git blame/show is lovely. osc blame/cat is often hopeless. Not talking about git branch, stash, amend and many other features ;). Well guys, if you propose a change, you ought to start with a _Motivation_! thanks, -- js suse labs
On 27/02/2025 08:05, Jiri Slaby wrote:
Ok, I have really no problem with "osc blame" myself. I for instance was using "osc diff -c N" to look then for the diff of the given revision N. And going down with N if the revision was a merge one (no source change). Sure, there are some nice things in this for sure. Another one is that one will be able to submit a _prjconf change as pull request. My point is that there are workflows that people are used to and became pretty productive using them. This change to git will have a real transactional cost. As any change has. And, believe me, that once it is in production, there will be shortcomings that will be worked around and the shiny nice things will be a machine holding by zip-ties and wires. As it always happens. We have so many workflows within the eco-system. And arguing that because one does rarely something, others are in the same case, is fundamentally flawed. Now, I am only a hired hand and I will have to live with whatever comes my way. But the free time contributor that now has to spend days/weeks to change habits might be tempted to go to see elsewhere. Cheers F.
On Thu Feb 27, 2025 at 8:25 AM CET, Fridrich Strba via openSUSE Factory wrote:
That’s one possibility. The other one (and I hope it will be so) is that volunteers will finally free to forget everything about this weird one-off proprietary VCS and use git (and use git-based tools and integrations) for everything as they wanted to do for ages. Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Finally, brethren, whatsoever things are true, whatsoever things are honest, whatsoever things are just, whatsoever things are pure, whatsoever things are lovely, whatsoever things are of good report; if there be any virtue, and if there be any praise, think on these things. -- Philipians 4:8 KJV
On Thu, 2025-02-27 at 08:59 +0100, Matěj Cepl wrote:
I really have a hard time believing that OBS is what kept some people back from contributing to openSUSE. Have you tried contributing to Debian? It's far more cumbersome and bureaucratic, and yet they someone manage to attract many contributors. OBS is far better than it's reputation, especially when compared to what other distributions have to offer. There is a reason why many other projects use OBS for building their packages even for non-SUSE distributions. Adrian
Il giorno gio, 27/02/2025 alle 10.05 +0100, John Paul Adrian Glaubitz via openSUSE Factory ha scritto:
The first time I contributed to openSUSE I was like "oh my god this is a mayhem", and it was my first _successful_ contribution because I tried in the past and I just couldn't do it. Luckily there were adequate manual pages but I can't say it was a pleasant experience. The fact that in other distros the situation is worse is not an excuse to me to try improving the situation, and even seasoned contributors said they would like something more git-oriented many times. To me the actual pity is that we don't have a metric of how many people expressed that kind of feedback :-D
I actually agree with you, OBS is a wonderful batteries included build server. It's just not the right place to keep the sources IMHO. Also:
I'm sorry, I really am, but people are one alias away from that. Gitea is just one git server like another.
On jue, 2025-02-27 at 10:05 +0100, John Paul Adrian Glaubitz via openSUSE Factory wrote:
As far as I know, OBS will be still a big part of the workflow. gitea will be used to handle the code, changes, etc. and OBS is used for building packages. Correct me if I'm wrong, but it will still be possible to create projects in OBS and add packages there. The main difference with the source in gitea is that it won't be as easy as "osc branch" or "osc linkpac", but I'm sure that when the new workflow is working there will be a lot of tooling appearing, like new git plugins, osc plugins or implemented as core functionality in osc. Regards.
If you were somebody looking to contribute to the cockpit project right now, how would you do that? The factory project is here: https://build.opensuse.org/package/show/openSUSE:Factory/cockpit Clicking on the "developed at" link just errors out. I wouldn't even know where to begin looking for documentation on how to contribute. The OBS doesn't even link to the gitea forge! When I started contributing to OpenSUSE I learned the entire workflow in an hour or so just by following https://en.opensuse.org/openSUSE:Build_Service_Tutorial. How exactly will adding an extra VCS platform and a bunch more steps (with seemingly no proper documentation "yet") make it easier for anyone to contribute? Or will it just mess up the workflow of hundreds of current contributors and make any future contributions more cumbersome? Are we just going to except that the OBS User Guide[1] and the Build Service Tutorial[2] are now obsolete with no proper alternative? The only reason I contribute to OpenSUSE is because I love the extremely simple osc workflow and the low barrier of entry. Git is imperfect, complicated and not tailored for this specific usecase. I know there's no stopping this migration but I am disappointed to see it happen in its current state. [1]: https://openbuildservice.org/help/manuals/obs-user-guide/ [2]: https://en.opensuse.org/openSUSE:Build_Service_Tutorial On 2025-02-27 10:05, John Paul Adrian Glaubitz via openSUSE Factory wrote:
On Donnerstag, 27. Februar 2025, 17:05:41 CET Rein Fernhout (Levitating) via openSUSE Factory wrote:
true, that is currently a limitation of the webui for this transition phase. not sure, if we can solve that soonish, however, I will look into writing the scmsync url instead into url field of the package meta in openSUSE:Factory packages which we have migrated as work around.
The goal is to replace the home grown VCS with something what people know. And where other distros have proofen that it can be used for packaging.
Or will it just mess up the workflow of hundreds of current contributors and make any future contributions more cumbersome?
The goal is to make it easier in future for users. Because they know the tooling already, they have all branches in one git repo, don't need to hunt many independ revision systems to figure out the state of a project and so on. It might be challenging during transition time and for some old timers like myself though ;)
Are we just going to except that the OBS User Guide[1] and the Build Service Tutorial[2] are now obsolete with no proper alternative?
These will be focused on building part in first place. But we will also ship an OBS Appliance including gitea, so we can also extend the user documentation to describe packageing specific gitea topics there.
I think we have to agree that there must be a reason why it is by far the most used versioning system meanwhile. Yes, we need to focus to make OBS specific workflows as easy and esp. also at least as far as before. Actually we should become faster given that many operations can happen local now .... that should be our benchmark.
I know there's no stopping this migration but I am disappointed to see it happen in its current state.
That is the reason why only the bravest devel project owners started first. We need some practical experience to move on with the next devel projects...
-- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 2025-02-27 17:22, Adrian Schröter wrote:
Yet they will still have to deal with OBS and Gitea, two platforms likely new to any contributors. It's not like osc for scm is complicated by any means either. Anyone familiar with git should find osc easier or as easy to use than git. At least I do.
Only because of convention. We have a homegrown tool, I think we should use it. If it is has shortcomings, we should invest our time resolving those instead. Specifically git doesn't work well for large binary files and requires access to the entire revision history. Two design choices that are less than ideal for packaging. I know other distributions have considered migrating _to_ OBS. Now we are moving away ourselves because of some mystical "future contributor" that apparently isn't capable of learning an extremely simple VCS is tool but is willing to learn how to deal with developing something in a git forge which than has to be synced to a build system both of which weren't build for that purpose. Bound together by some other homegrown CLI tooling and bots. I am hoping to see the tooling and documentation grow around the new workflow. Still I have serious concerns about how this will affect the amount of contributions for new and old contributors. I am no "old timer" by any means, I only started contributing to OpenSUSE about two months ago. But the reason I did start contributing was because it's so much easier to contribute to OpenSUSE than any other distribution. The only reason I use OpenSUSE is because it is so easy to contribute. Depending on how good the tooling is when the packages I maintain are finally migrated, I might decide to stop contributing to Factory entirely.
On Thu, Feb 27, 2025 at 05:48:21PM +0100, Rein Fernhout (Levitating) via openSUSE Factory wrote:
Gitea is very similar to other git forges. So long as simple contribuitions only need interaction with gitea, and not OBS this simplifies things a lot.
Not only it is complicated, it's bad. When trying to look through history of some packages something that I would see in git in uder a minute took tens of minutes to investigate, and for some changes I did not find where it came from conclusively. It's not like better tools for working with the OBS internal VCS could not be written but it has not happened until now so I am not holding my breath. Coverting to git gives us access to the tooling others are writing, we do not need to roll our own. Thanks Michal
Hi Jiri, On Thu, 2025-02-27 at 08:05 +0100, Jiri Slaby wrote:
The price is IMO not so high. Have you actually tried this all yet?
The fact that I will have to leave the terminal and manually navigate to the project page in my web browser and retrieve the git URL is a major step backwards in my opinion. Plus, I can no longer collect and assemble packages in my own home project to build and test them there. For larger package stacks, it's an indispensable feature of OBS that I can build and test a collection of packages in my home project and even create a repository from there in order to submit them in a batch. If this feature becomes unavailable because of the transition to Gitea, larger stack updates will become impossible unless there is a different way of creating such staging setups. Furthermore, there are many small nuisances that still exist such that folders and temporary files are not automatically ignored but can accidentally get committed and the fact that commit messages are not constructed from the changes file, so that I have copy them manually. I have the impression that the easy of use and flexibility of OBS is vastly underestimated here so that some people think it can be easily replaced by Gitea. Coming from the Debian world myself, I learned to appreciate the power of OBS really fast as it's much more flexible than Debian's rather simple network of buildds (build servers) which don't support building packages for custom projects in your own home. There is just one set of builders and a system to track package sources while OBS has multiple sources and multiple builders to create multiple repos. This is hugely more useful.
I have never had to use this feature, to be honest. Those are features that I need when maintaining complex source code but not when keeping track of spec files, tarballs and patches.
Not talking about git branch, stash, amend and many other features ;).
Same applies here. I have never had the urge to use these features in OBS, despite the fact that I am actively using these all the time when hacking on upstream projects like the kernel. I have to admit the more the Gitea conversion nears, the more I am convinced that Gitea is not the right tool for the job.
Well guys, if you propose a change, you ought to start with a _Motivation_!
Yes, but the change should not mean that you lose a lot of features from OBS that made the whole build system in openSUSE so superior in the first place. Adrian
On Thu Feb 27, 2025 at 10:00 AM CET, John Paul Adrian Glaubitz via openSUSE Factory wrote:
What exactly you couldn’t do with tea? Would it be possible to write a script for doing that?
You mean https://git.sr.ht/~mcepl/gosc/tree/master/item/git-devproj.sh ? (Patches more certainly welcome, on the `devproj-py` branch there is an unfinished port of the script to Python)
Yes, editing of `.gitignore` is necessary. That’s a bother.
OBS is absolutely perfect as the build/workflow/release tool. I have never had enough experience with the Debian world (I have never managed to be a full DD), but comparing it with Fedora’s Koji/Bodhi, it is just awesome. However, OBS as VCS is nothing to write home about. Sorry.
I use `osc blame` quite often, and yeah … it kind of works. But, again, the missing integration with better front-ends (e.g., vim-fugitive) makes its use suboptimal.
I have to admit the more the Gitea conversion nears, the more I am convinced that Gitea is not the right tool for the job.
Anything more specific? Any alternatives? Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 So many time, the attitude of my heart is: “God, just tell us what to do so we can do it…without you.” Independently. -- Gretchen Konnevik: On Josh Harris, Homeschooling, and Spiritual Formulas; https://wp.me/p69TD5-1Xv
On Donnerstag, 27. Februar 2025, 09:59:54 CET John Paul Adrian Glaubitz via openSUSE Factory wrote:
that would be the case indeed, but we work on tooling to be as productive on CLI as before. There is the new git-obs tool already with some functionality part of osc. It only covers part of it for now, but it will improve.
sure, you can. Even in more ways then before.
That is great to hear. In best case we will have the best of all worlds, that is at least my goal: * gitea/git as easy entry point for new users. Also a single place for all code branches in one repo. * OBS as powerful build engine, following automatically git repositories without the needs of manual action/workflow setup. * local build not only of single packages, but larger package sets using pbuild tool. That would atract hopefully people from debian or embedded world as well. bye adrian -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Thu, 2025-02-27 at 17:05 +0100, Adrian Schröter wrote:
But why not wait with the transition until the tools are ready?
Could you elaborate on that? When I fork a package in Gitea and then update it, how do I get the forked package to be present in my home project? For example, I want to update 350 Python packages that need to be in sync. How do I fork these and make them build in my home project using Gitea? From my current understand, I can only fork packages in Gitea, update them and create a pull request which will immediately create the submission to Factory. What I need is a project that is automatically built from the packages that I have forked and updated in Gitea in order to have something for staging.
The problem is that you are adding another layer of complexity so that users now have to deal with two services instead of just an integrated one. I have the impression that the main driver behind the change was to be able to get a Github- like web interface and then the existing infrastructure was just attached to it as Gitea itself cannot provide a build system. Wouldn't it have been better to switch the revision control system in OBS from Subversion to Git while keeping the workflow and tooling, so that users can still use the "osc" command for everything and use build.opensuse.org as the canonical place for maintaining and building packages? Adrian
On Donnerstag, 27. Februar 2025, 20:28:02 CET John Paul Adrian Glaubitz wrote:
right now people can still work with osc, so nobody should be blocked. Only devel projects who agreed to get converted got converted. But we need to start somewhere to see if the new mechanics work and therefore it is good that we get devel projects based on git to verify that. Keep in mind this is now in the queue since years, we need to evaluate all the concepts and therefore it is good to have real life examples.
There are different ways: 1. just edit a package meta in your home project and add a <scmsync> element there 2. There is the fork command in the queue, analog to "osc branch" you can # git-obs fork-obs-package $project $package which will create a fork on gitea and also sets up a branched build pointing to it: https://github.com/openSUSE/osc/pull/1723
What is "immediatly creating a submission to Factory" ?
What I need is a project that is automatically built from the packages that I have forked and updated in Gitea in order to have something for staging.
I don't see why this is not possible.
During conversation time maybe, but afterwards we replace the source management system. Also with osc you have different commands for source and build handling, so the complexity hopefully just get splitted into two commands. And the hope is that people know git anyway.
definitive not, we old timers are all tied to cli.... The real benefit for maintainers would be that a package would have all branches later on in one git repo. And fixes could be more easily merged/cherry-picked between branches. Esp. when we may have way more branches in future because we may rebuild packages way more often in SLFO for different products.
we don't have subversion, but this is conflicting with the given goals of security (leaving md5sum checksums on disk) and package centric approach which is required for SLFO at least. To be clear, if someone wants to work independend of our distributions, you still will be able to use the old storage system. Also, the git-obs (part of osc) is designed to be a git sub command. So we could say that we just switch the commands, and you will be able to do gitea and obs commands all with via git as well. (however, atm git-obs is only targeting gitea) -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Mittwoch, 26. Februar 2025, 16:35:51 CET Dominique Leuenberger wrote:
Sure, you still can. You still can use project links and overlay packages. Or you can fork entire factory git (once it is converted), so you can work a fixed source snapshot. Something what we supported before only in same cases.
I don't see why you need different orgs here, you can also all maintain it in your personal space in gitea if your want. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 25. 02. 25, 17:19, Adam Majer wrote:
If you have questions, please read the linked wiki page above first then ask away.
What about _service files running on OBS (git push to a repo causes OBS fetch and build, i.e. not "manualrun" ones)? How can I take a look what was downloaded and used for build? And status of running services? There are no links to packages when the whole project is scmsync (as is planned for devel projects). thanks, -- js suse labs
On 2/27/25 11:11, Jiri Slaby wrote:
This will not work on day 1, or 50. These will need to be replaced with another logic, but this one is lower priority, especially since we do not accept remote service runs in official projects (unless that policy has changed?). If you rely on this mechanic, you can keep it in a separate OBS/IBS project (not factory devel project) where it will continue to run. Then copy the artifacts for Factory submissions. - Adam
On 2/27/25 12:53, Jiri Slaby wrote:
Sources are imported. _service(s) will just no longer run. The migration process will be something like, 1. you'll get email notification about pending conversion in a day or two. 2. then, there will be some pending project and we can work together to try to address any major problems with the conversion, etc.. Project/package mantainers tend to more know about their stuff than I can hope to discover in a short amount of time. 3. then, we place the scmsync in the prjmeta and hopefully everyone involved is not hating me too much :-) Then next project x 265 .... this will take time. Fortunately Dirk is importing factory to src.opensuse.org/pool already, so you can check your packages if things look completely broken there or not. No need to list them here, just keep in mind for future. FYI: the project list comes from: https://build.opensuse.org/projects/openSUSE:Factory:Staging/packages/dashbo... Some decade and a half ago, I was told there was a migration from Autobuild to OBS. In some ways, there were probably pain points too. - Adam
On Donnerstag, 27. Februar 2025, 13:52:11 CET Adam Majer wrote:
That is not true in general. Esp the mode="buildtime" services may still be important. It is true that we have alternatives to some services now, which are more git native. You could even use download assets out of the box now. But the distro will still require fixed sources by policy. Or do you mean that you remove _service files on import? IMHO at least the buildtime services should stay, we have no alternative to them. And they don't harm any fixed source policy. bye adrian -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 2/27/25 14:04, Adrian Schröter wrote:
I mean the remote run server side source service will no longer work. Of course, the build related services are unaffected. The _source file remain regardless and is unfiltered. - Adam
On Donnerstag, 27. Februar 2025, 14:09:30 CET Adam Majer wrote:
nitpicking, it works, but we don't want it by policy;) In any case, it should not hit any devel project, since we don't allow remote active services in factory already anyway. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 25. 02. 25, 17:19, Adam Majer wrote:
If you have questions, please read the linked wiki page above first then ask away.
Another question: How can one create a (right) gitea repository from the commandline? Apparently, obs runs/likes sha256 (was there a _reason_?). But tea does not support changing sha1 to that (as of now, AFAICS). So if one creates a repo using tea (i.e. sha1), imports it to src.o.o, it all starts crashing in git in infinite recursive call of oid_object_info_extended() :D. thanks, -- js suse labs
On 2/27/25 13:17, Neal Gompa wrote:
It was a mistake to use SHA256 right now, because nobody is ready to use it yet.
It's one of the critical pieces actually. SHA1 is deprecated already, https://csrc.nist.gov/news/2022/nist-transitioning-away-from-sha-1-for-all-a... and since we would like to rely on integrity of the source repositories via signatures (in the near future), we had no real choice but to use SHA256 here. SHA256 will be default for Git 3.0 (whenever that comes) https://www.phoronix.com/news/Git-2.46-rc0-Released The interop with SHA1 repos is not really there yet, but this will improve. Considering we have no interop with anything currently on the VCS level, this shouldn't be much of a problem. - Adam
On 2/27/25 07:26, Adam Majer wrote:
..... is not really there yet, but this will improve.
This is the real problem. - We are in a 1/4 baked state with tooling - Documentation is mostly non existent - Communication is less than concise - .... At the same time many people on this list are on the hook to deliver a fully baked distribution in the not too distant future. I do not really see how this is going to work well. Later, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU Distinguished Engineer LINUX rjschwei@suse.com IRC: robjo
Am 28.02.25 um 20:38 schrieb Robert Schweikert via openSUSE Factory:
At the same time many people on this list are on the hook to deliver a fully baked distribution in the not too distant future.
Don't worry. No customer is going to use SLES 16*.0* anyway :-P They will start with 16.2 and you have time to sort it out until then. SCNR. -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman
Hey, On 2/28/25 8:38 PM, Robert Schweikert via openSUSE Factory wrote:
There are also the real problems with the development/operation of all of the involved "glue code" and gitea. There are just a bunch of low bus-factor git repos and no dedicated SUSE resources to enhance/operate or support the tools. The development of the glue code follows next to *no* engineering best practices... - no documented contribution process - no documented feature/issue tracking process - no code review - no continuous integration - next to no automated functional tests - no static code analysis - no code style guide - no code documentation - no automated dependency tracking - no release engineering And the resulting code is run somewhere without adhering to any operational/SRE best practices... - no observability for gitea or the glue code - no health monitoring or alerting - no performance monitoring or alerting - no deployment tracking - no defined incident management - no service level objectives to adhere to - no security policy or even security reviews I do not really see how this is going to work well on those fronts either. Henne -- Henne Vogelsang http://www.opensuse.org Everybody has a plan, until they get hit. - Mike Tyson
On Thu, 2025-02-27 at 12:51 +0100, Jiri Slaby wrote:
I had looked up the reverse of your question earlier because the Gitea UI defaults to SHA256 for new repos already, the answer to your question appears to be: $ mkdir my_project $ cd my_project $ git init --object-format=sha256 . -- Have a lot of fun!
On 2/27/25 13:20, tux93@opensuse.org wrote:
Jiri is asking about how to create a repository/package. This probably will best be part of the git-obs utils as we'll want to have .gitattributes (for LFS) initialized there. - Adam
On 25. 02. 25, 17:19, Adam Majer wrote:
For information of what is happening, etc. please see,
I see Kernel:stable in the list of planned to be migrated. However our kernel still uses _link. So who is going to fix this before the beginning of Apr ;): https://bugzilla.suse.com/show_bug.cgi?id=1211226 ? thanks, -- js suse labs
On Donnerstag, 27. Februar 2025, 12:56:36 CET Jiri Slaby wrote:
You are using local links only. That works via a symlink in the project git. Getting rid of them was more a desire by release managers since changes there were hard to track in the past. (actually something what would also be easier with a project git merge request). So, I am just speaking for the OBS mechanics here, local links are no issue. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 27. 02. 25, 14:10, Adrian Schröter wrote:
I realized we actually use multibuild already, but the bug remains -- it's impossible ATM to exclude the build properly with multibuild (see the bug).
Good! Speaking about local links brings me be to another kernel _link question: We use non-local links with the kernel a lot. Like an overlay: https://build.opensuse.org/package/show/home:tiwai:kernel:stable-kasan/kerne... https://build.opensuse.org/package/show/home:jirislaby:kernel-bsc1231465/ker... https://build.opensuse.org/package/show/home:jirislaby:stable-bsc1229085/ker... https://build.suse.de/package/show/home:jirislaby:stable-lockdep/kernel-sour... https://build.suse.de/package/show/home:jirislaby:stable-kasan/kernel-source... ... How do we do that with git? And what about _aggreatepac? thanks, -- js suse labs
On Freitag, 28. Februar 2025, 07:17:21 CET Jiri Slaby wrote:
It is possible via project config, basically excluding the flavors you don't wont via BuildFlags: excludebuild: An alternative would be an extension to the scmsync handling that you could extend the git sync url with eg ?onlyflavor=kernel-default&onlyflavor=kernel-source to limit the flavors. We could implement that if someone opens a feature request in the obs-scm-bridge github component :)
well, the automatic merging of source changes was a declared anti-goal for the git implementation I have to admit. I think this will need to be solved different. If the obvious solution to rebase the git is not working, we need to think about mechanics to allow source modifications either inside or outside of OBS. There are some in my mind, but I think this is too much for this thread. Let's track this elsewhere when needed.
And what about _aggreatepac?
_aggregate files are just source files processed at scheduling/build time. So, nothing changes, just commit such a file. (even though in general aggregates are often misused one should think twice if it is really the right tool. but that is independend of the SCM). -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On 28. 02. 25, 8:54, Adrian Schröter wrote:
Good, could you comment this in the bugzilla -- it is handled by Michal and I don't completely understand the problem. There are also related (long open): cannot specify which multibuild package flavors to build in package metadata https://bugzilla.suse.com/show_bug.cgi?id=1219005 cannot specify which multibuild package flavors to use for build in package metadata https://bugzilla.suse.com/show_bug.cgi?id=1219011
I use it often to take devel:gcc/gccXX to a home: project and build-test a kernel with that gcc version. Without the need to recompile the compiler proper. thanks, -- js suse labs
On Freitag, 28. Februar 2025, 08:54:49 CET Adrian Schröter wrote:
what I forgot to mention, if you don't plan to submit these changes back, these packages will of course still be working. _link files to git managed packages still work and you can still place source files as you do. You just can not convert these to git as _link files *inside* of git repos are not supported/wanted. So nothing needs to be done here most likely. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Fri, 28 Feb 2025 09:37:43 +0100, Adrian Schröter wrote:
So, as long as it's a downstream packaging that needs only some local modifications (if any) but follows the upstream changes, it'd keep working on OBS, right? That's exactly my usages. Here, the automatic following is the thing I'd like to have. On git, it'd need merge or rebase, and this will become more cumbersome than OBS link. thanks, Takashi
On Freitag, 28. Februar 2025, 11:02:13 CET Takashi Iwai wrote:
yes. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Fri, Feb 28, 2025 at 08:54:49AM +0100, Adrian Schröter wrote:
As has been discussed in the bug this does not work because the prjconf and with it these enable/disable flags are inherited by projects that build against it. That makes it impossible to re-enable something once disabled. In any case the syntax is atrocious. The enablement/disablement is done in code that has no syntax specification. Please make a proper configuration file for this. Thanks Michal
On Montag, 3. März 2025, 11:52:45 CET Michal Suchánek wrote:
The suggested onlyflavor cgi parameter would be only used in your own project/package meta data. eg. you create in your home project a package meta with ... <scmsync>https://src.opensuse.org/pool/kernel-source?onlyflavor=kernel-source&onlyflavor=kernel-default</scmsync> ... And it would only trigger these two flavors independend of any base project or package source. (again, this is not yet implemented, but we could if wanted) -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Mon, Mar 03, 2025 at 12:52:26PM +0100, Michal Suchánek wrote:
And how it works for package links? I suppose branching packages would be done in git when the project is git-based so this would be non-issue for the common case. However, it has been pointed out that some features of package linking will be still provided only by .. package linking, and in that case the build settings youls be inherited through the link, again with no way to override. Thanks Michal
On Montag, 3. März 2025, 13:01:53 CET Michal Suchánek wrote:
This implies that you create a obs package meta with this element. You can use any classic flags in the package meta there. Also any repository configuration in your local project meta. If you want to have different flavors extract by arch and/or repository name you would need to create a second package meta which could point to the same git repository.
And how it works for package links?
in general, you can link _to_ any scmsync package object. But you can not use _link's inside of git sources. However, the obvious way out here is just to point to the same git repo. We need to distinguish between official projects, like openSUSE:* and devel projects where we have git projects. But for any personal setups like mentioned here you can still manual setup classic OBS projects and use mechanics link _link even including auto merging code.
I suppose branching packages would be done in git when the project is git-based so this would be non-issue for the common case.
yes. (even though we call it fork'ing in our terminology, since branching means something different in OBS and also in git world).
as long as you don't need it in official projects, you can still use that. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Mon, Mar 03, 2025 at 01:41:53PM +0100, Adrian Schröter wrote:
So one package for each flavor/repository combination to build, with no way to override the settings in the UI? Does not sound like a good setup.
Which does not work for adding modifications in the link.
But not for setting what to build.
Nope. The features are provided by different mutually incompatible mechanisms. So while the features exist in isolation they cannot be used together. Thanks Michal
On Mon, Mar 03, 2025 at 01:56:18PM +0100, Michal Suchánek wrote:
And because build results from such multitude of packages linked to the same git sources are not linked together when submitting from such develproject the build result will not get checnked.
On Montag, 3. März 2025, 14:05:40 CET Michal Suchánek wrote:
It is an option, if you need to override per package build defaults which are otherwise defined in the project.
Does not sound like a good setup.
I agree, but I think the problem lies deeper here. Exceptions of build behaviour per package were always be a problem. It also breaks in old mechanics when you use eg. project links or trying to reproduce the result in another OBS instance.
inside of a devel project such hacks should never be needed. They never could be submitted to factory in the past. No there would be mechanics to transfer flags because they become part of the sources.
I agree, but this is actually the goal with git approach: Having one single identifier to get an entire build setup which can be used to reproduce the entire build. This is not possible with classic OBS mechanics. Unfortunatly, this means that local hacks don't become easier ... -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Mon, Mar 03, 2025 at 02:18:38PM +0100, Adrian Schröter wrote:
They are needed - for aggregating per-architecture builds against different projects - for breaking the self-reference of package that is used to build itself Neither is needed in Factory but both are needed in the develproject to continuously provide build results as the package is updated.
And hacks in prjconf suggested multiple times throughout this thread undermine this reproducibility, as is already the case with the old mechanic and multibuiltd. I want: - everything built by default - a project configuration *DATA* file for selecting what to build, per architecture/repository/flavor Thanks Michal
On Montag, 3. März 2025, 14:53:09 CET Michal Suchánek wrote:
On Mon, Mar 03, 2025 at 02:18:38PM +0100, Adrian Schröter wrote:
On Montag, 3. März 2025, 14:05:40 CET Michal Suchánek wrote:
... (trying to get back on focus)
you can do this via the build config in a way that works in git projects and also in classic OBS setups (and solving issues there as well): There are either onlybuild: or excludebuild: tags which you can use to enable or disable builds. You can use them also for flavors, for example BuildFlags: excludebuild:<package>:<flavor> like BuildFlags: excludebuild:kernels-source:kernel-default And you can have any %if statements (repository name, architecture, ..) around these. This works with git and classic OBS, with project linking, via path elements and so on. And without the need to check at least 4 different source histories and replicating them just for a single build. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Mon, Mar 03, 2025 at 03:34:56PM +0100, Adrian Schröter wrote:
It actually does not. 1) this is inherited between repositories, and then once a build is disabled it cannot be reenabled ie. with repositories 'standard' and 'QA', 'QA' built against 'standard' the ifdef for 'standard' disables a package, and this disablement is inherited to 'QA' 2) it's an abomination This is using code for configuration. Worse, code without any specification of the syntax and semantics. Please provide per-repository configuration as DATA. Thanks Michal
On Mon, 3 Mar 2025, Michal Suchánek wrote:
The solution might be to use different OBS projects for each of the above repositories? That's what I'm considering for devel:gcc. Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Dienstag, 4. März 2025, 09:14:53 CET Richard Biener wrote:
You can omit build config rules again on a higher level if needed via prefixing it with an ! For example BuildFlags: !onlybuild:kernel-source:kernel-default should enable it again, if it was disable in another repo you are referencing it.
would not help when getting configuration via pathes, but I wonder indeed it we could find a more simpler setup to this specific solution when we draw the wanted setup in a picture ... -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 09:31:18AM +0100, Adrian Schröter wrote:
Oh the atrocity intensifies. So if you want to build a package you have to negate any build flags that might have been set in a project you are building against. How do you do that? Set up the project, get the expanded prjconf, negate any flags present, and then add yours on top? Could we just drop this atrocity in favor of actual data based per-repository configuration? Thanks Michal
On Dienstag, 4. März 2025, 09:38:52 CET Michal Suchánek wrote:
Actually, it should be easier because you could just build against the source and repositories of the base project. Atm. you have multiple revision repositories, in your case at least these: 1. Project meta repo 2. Project build config repo 3. kernel-source meta repo 4. kernel-source source repo 5. kernel-default meta repo 6. kernel-default source repo 7. kernel-obs-build meta repo 8. kernel-obs-build source repo 9. kernel-obs-qa meta repo 10. kernel-obs-qa source repo And for each kernel flavor you get two more. All of them are not connected and on a replication of the build setup you need to copy all of them and even worse, in package meta case there is code which needs to know which parts of the files can be copied, which needs to adapted and which must not be copied. In case you want to learn about the configuration of a build, you have to verify all of them independend. With the git approach there would be only 1. Project source, referencing to all package sources 2. Project meta repo (and even here we will be able to store critical parts in the source) And you don't need to fiddle around per package, they would get the config from the project source definition. Your branched project just needs to define the wanted repositories and these should have a path to their pendant of your base project. And you could even build entirely local and independend of any OBS instance using pbuild once we sync the project meta repository config with the project source store in git. IMHO it improves the situation a lot if we setup it correctly. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 09:49:58AM +0100, Adrian Schröter wrote:
It utterly fails when I want to build the package independetly of the base project build settings because these are silently imported, the format of the settings is unspecified, and there is no tooling neither in the web UI nor the commandline to alter these settings. Thanks Michal
On Dienstag, 4. März 2025, 09:58:13 CET Michal Suchánek wrote:
what do you mean here exactly? The BuildFlags are specified in https://opensuse.github.io/obs-build/pbuild.html#_about_the_build_configurat... . IMHO as good documented as any package meta flag.
and there is no tooling neither in the web UI
well, yes, the goal is to have it as part of the source. And handling the above listed independend repositories via webui is not really a serious thing either.
nor the commandline to alter these settings.
sure, it is just a git commit and push. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 11:27:12AM +0100, Adrian Schröter wrote:
Yes, the flags are. The file as a whole is not. The syntax of the conditionals is not defined. The inheritance is poorly documented. That setting these buildflags requires examining buildflags of all projects built against is definitely not explicitly documented.
What parts of package to build in which repository should be separate from the package source.
Except you have to know what setting are in projects you build against to negate them. I did not know git push had this functionality. Where is that documented? Thanks Michal
On Dienstag, 4. März 2025, 11:35:27 CET Michal Suchánek wrote:
indeed, we can add that.
The inheritance is poorly documented.
that is independ an obs feature.
That setting these buildflags requires examining buildflags of all projects built against is definitely not explicitly documented.
that is in general true for any build config setting which is there in OBS.
yes, it would be in the source of the project. But not in multiple meta objects in multiple own revision trees which you need to combine (also not that good documented, I have to admit).
The build config in a git project is stored as "_config" as normal file. So in general we have again only the build config documentation here. You can also try it without git if you want by using "osc meta prjconf -e ..." -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 11:41:58AM +0100, Adrian Schröter wrote:
The syntax is whatever rpm accepts, and rpm syntax is not documented, either. So no, you cannot unless you take on documenting rpm spec and macro file syntax, and then define which is used in which parts of the prjconf.
The inheritance is poorly documented.
that is independ an obs feature.
And previously was not involved in setting what to build, and now is. That's one of the problems. It's no longer an independent OBS feature.
Indeed. Except the old build selection settings were not in prjconf, they were separate. They could be randomly copied when packages were branched but then were separate and explicit. The whole settings would be visible as a XML document in the project they apply to, and could be altered in the web UI as well as by editing that one project. The current solution of stuffing them in prjconf is a severe regression in transperency, intelligibility, and usability.
But that does not show the settings that apply to the project, only the part stored in the project, not the upstream settings. Thanks Michal
On Dienstag, 4. März 2025, 11:57:57 CET Michal Suchánek wrote:
we don't use rpm to interpret this file to schedule build jobs. We know our parser, so we can document it.
??? any package meta setting was always only an OBS feature. There is now the additional possibility to build also independ of OBS. But we build also the same inside of OBS.
you obviously never had to debug a state, not understandable by someone, where you had to check several meta histories, sync timestamps between them, guess temporary build states at that time and that via many repositories and projects.
yes, but neither the package meta is doing that. You can just guess. In any case you would need to fetch current build state. In the past and also in the future. again, I offered you a brainstorming how to get a better solution for your very specific issue. Something where we can still replicate the build setup without plenty of individual setup tasks of copied many repositories and editing many meta files. I can imaging that we could add further functionality to give hints as part of the package source for the build flags. But you won't get me to a solution which requires many many manual steps, many independend history repos and reverse engineering of the timeline of intermediate states. That is just the horror for any tooling and for anyone who needs to debug content. Also violates clearly the goal of getting a reproducible build based on a given source. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 12:28:27PM +0100, Adrian Schröter wrote:
The prjconf is inherited between projects. The build flags that determine what to build are now in projconf. That makes prjconf now releavant to what ends up getting built. And I have to wonder how these same flags now get anywhere indeoendent of OBS. That's just not possible. You absolutely have to ask OBS what the final flags aggregated from your project and all upstream project will be.
That's exactly what prjconf leads to, and configuration independent of prjconf, specific to one project, without inheritance can avoid. The less stuff in prjconf the easier debugging. Yet you want to 'improve' OBS by stuffing more into already unintelligible and unamaintainable prjconf files. Please also consider usability by people who do not spend their days by staring ar OBS code. Thanks Michal
On Dienstag, 4. März 2025, 12:43:57 CET Michal Suchánek wrote:
IMHO it helps a lot when these config settings get re-applied by default in your branched project. Also in your special case. You don't need manual to fiddle around with flag settings in each package meta, if the design is right.
I don't insist on build config, we can also think about hints in the package source. Just the package meta itself is not acceptable. Well, except you say it is anywany only your personal pet project, not planned to be merged. There you can still use it as before and just reference to the git repo per package manually, if you want.
IMHO distributing settings to multiple places is neither intelligible and maintainable. What should be the reson that 3 config settings out of a large set should be configured different? But everything else is still build config?
Please also consider usability by people who do not spend their days by staring ar OBS code.
Well, we speak here about a very special exception setup. Something were you already use a setup what needs a lot of internal OBS knowledge in error case. And even here I think we could make it more reproducible and easy to maintain then before. All this is not used by at least 95% of package sources (or is it 99.9%?) -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, 4 Mar 2025, Adrian Schröter wrote:
It sounds like the main point about this is that there's "source" besides the source of packages. Namely there's "source" for an integration project like openSUSE:Factory - it's project config. But there's also "source" (or config) for a specific build, like for devel:gcc or for the Kernel projects. But I do _not_ see any part of the current project config as essential part of a _package_ source. IMO it's desirable to separate those two aspects clearly. For example the project config parts of packages tracked in devel project "X" should then literally be intergrated into the project config of the aggregator (openSUSE:Factory for example). But what if the same packages should also go to SLE15? It's ofthen the case (well, speaking for GCC ...) that the project config details for the aggregating project (SLE15 vs. openSUSE:Factory) are different, so a single project config in the git sources of the "devel" project isn't sufficient. So we'd need a devel-X-for-SLE15 git project that aggregates the package sources but has a separate project config in git? And if I want a special bootstrap OBS project then I of course want to have a project config (override) just there. I think I understood I can do that, even with package meta, if I avoid doing a project-wide scmlink but instead do per-package scmlinks. Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Tue, Mar 04, 2025 at 01:09:49PM +0100, Adrian Schröter wrote:
How would you design this so the fiddling is not needed? The packages are built selectively in devel/test projects, and all in release project, by design.
I cannot use it anymore because git workflow does not support package links, and package meta does not support multibuild. We do use multiple hacks that determine what to build out of the krenl sources based on the project name, and I would rather reduce them than add to them.
Stuffing all settings for a lot of unrelated things into one file with unclear syntax that changes in the middle of the file is unintelligible. It might be possible to have a better structured prjconf that is actually intelligible but we don't have one. The prj meta and package meta are XML files that can be parsed with standard tools because they use standardized syntax. At most two apply to any given package. The settings describe boolean arrays that are multiplied to get the result. Compare that with prjconf that uses non-standard syntax for which there is no known parser library. It can have arbitrary depth. It is code with conditional statements, it cannot be merely parsed and extracted, it has to be interpreted. ExcludeBuild and OnlyBuild flags are not simple booleans, they have complex interaction. They cannot be used on their own, the project meta is still needed to set what architecture to build in which repository. Unfortunately there is overlap between prjconf and meta so you get arbitrary depth prjconf + 2 layers of mete. Yet to 'imrpove' you make meta that was predominantly used unusable so that the unconstrained depth prjconf must be used. The opposite would be desirable to make the configuration intelligible: less files involved, clearer syntax, bounded configuration complexity.
The current direction makes it harder to reproduce and maintain so far.
All this is not used by at least 95% of package sources (or is it 99.9%?)
Execept 100% releases use some of these few % packages that are self-rerefencing - used to build themselves. So it's a case that's needed for every single medium built. Thanks Michal
On Dienstag, 4. März 2025, 13:38:09 CET Michal Suchánek wrote:
first of all, if you insist on package meta and links, you can still use them. Pointing links to any git managed source keeps working. As long as you don't want to work on sources and merge them back you can work as before.
works as before. You can also use %if "%_project" == "..." in build config. That has the advantage it needs to be defined only once in a central place and you don't need to redefine it in each new project.
but these files have no meaning alone. And you need to re-do it for every instance. And you can not just copy them, but you need to adapt them.
yes, and the only way out is to move the settings to the source. It can be in the package or project. build config is just one example. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 04, 2025 at 02:00:23PM +0100, Adrian Schröter wrote:
No I cannot. To have submittble package it cannot use links. That effectively bans links from develprojects.
Why would I ever do that? The config is attached to a project, implicitly. If the config is inherited to a different project that builds against the project that has this config the conditional expansion is also inherited, and the setting is applied regardless of the project name. At least that's how it works for repository name conditionals.
??? what else would you need? They refer to repositpry names, package names, and architectures which have meaning within the project they are attached to. Nothing else is required.
And you need to re-do it for every instance.
I need to do that every time I upload the kernel. At least with the meta files I start from clean slate, and do not have to examine parent projects, nor take into account inheritance between repositories. The result is clear and transparent, every time.
And you can not just copy them, but you need to adapt them.
I create them from scratch. Adapting (eg. changing repository name) is not all that difficult when parsers for the file format are ubiquitous.
What source? If you are talking about git repository that stores project state, and has the prjconf: Why it cannot have multiple files for specific purposes, and has to have one big, unintelligible mess of a file? I do not care where the prjmeta is stored so long as it is clear, intelligible, focused configuration file attached to the project. Actually to the repository, it looks to me like for projects that have multiple repositories project-global settings are the exception rather than the norm, and a lot of ifderery is used to apply per-repository settings in a per-project configuration. Thanks Michal
On Tue, Mar 04, 2025 at 02:29:14PM +0100, Michal Suchánek wrote:
Obviously, because conditionals for repositories and projects work differently. While project imacro definition is inserted at the start of the aggregated prjconf, and all code is evaluated with the final project name from the start to the end repository macro is redefined to the repository from which the code comes before each piece of the code. That's so nice, clear, transparent, easy to understand and debug design 🤮
The fact that prjconf, prjmeta, and pkgmeta are in separate, independent VCS repositories, and it's not to tell possible which revision of what was used is comletely the limitation of OBS. They can be stored in one repository, and OBS can write which revision was used in appropriate logs. Nonetheless, do you have some concrete proposal how to make managing develprojects of core packages in git workflow usable? You suggested a call. Do you think that making a call would generate such ideas? Thanks Michal
On Montag, 10. März 2025, 18:19:17 CET Michal Suchánek wrote:
indeed. But there were examples mentioned where you just have test build setups, which are not supposed to get submitted. You can keep the _link files there.
At least you get entire configure with a single "git clone" and you don't need to do many independend api calls and combine the result on your own. IMHO a big step forward of usability in this regard. But I don't mind to add further mechanics eg. in upcoming project meta file (successor of experimental _subdirs file). We could allow package specific overwrites there, similar to former package meta.
true, but we won't do deep changes there anymore as they would break compability. So we have the single repo design only via the git managed projects.
Nonetheless, do you have some concrete proposal how to make managing develprojects of core packages in git workflow usable?
Atm we only speak about the kernel here with it's special setup. Richard is currently happy to go with the suggessted approach of two independend projects afaik.
You suggested a call. Do you think that making a call would generate such ideas?
it was just an offer ... -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue, Mar 11, 2025 at 11:44:39AM +0100, Adrian Schröter wrote:
You actually do. The git clone has the prjconf only for one project, not the projects it builds against. And those do get combined in OBS, and you need the combined result, not the part stored in your repository.
IMHO a big step forward of usability in this regard.
Sure, sendmail.cf all the way. Everything in one file. Looking at the syntax and what we have in spec files and prjconf I see that there is at least common source of inspiration for that syntax.
That sounds like an actual step forward in usability.
Was it tested that it actually works? After the migration to multibuild in devel:gcc a number of things broke but the problem was simply ignored for the most part. Kernel:tools as one of the few consumers of the project had to start building more packages as a result. Thanks Michal
On Tue, Mar 11, 2025 at 12:32:53PM +0100, Michal Suchánek wrote:
Also in another branch of the thread it was mentioned that Base:System uses package flags extensively. It's a develproject for core packages that are used to build themselves, after all. Thanks Michal
On Tue, Mar 04, 2025 at 09:31:18AM +0100, Adrian Schröter wrote:
On Dienstag, 4. März 2025, 09:14:53 CET Richard Biener wrote:
On Mon, 3 Mar 2025, Michal Suchánek wrote:
So let's see what a bootstrap project looks like: I have about 5 packages. Because package links are no longer supported these are all mashed in one with multibuild. Out of this package I get about 10 different build products. One of these build products is kernel-obs-build which is used for building every package. Another of these build products is kernel-obs-qa that builds empty package. This is used to test that the kernel-obs-build can boot in OBS. The setup I want is: repository standard (+ any architecture-specific repositories when different architectures need different repositpries to build against): All package flavors except kernel-obs-qa are built. kernel-obs-build is NOT used for build. That way when it fails to boot, and the kernel is fixed the kernel and this package can be rebuilt without user intervention. repository QA (+ QA_arch for any separately built architecture) builds kernel-obs-qa, using the kernel-obs-build built in standard repository. This is already broken, and various workarounds are applied to make it sort of work. In particular per-package flags are no longer supported with multibuild. What is your proposed solution for this setup? Thanks Michal
On Tue, Mar 04, 2025 at 09:14:53AM +0100, Richard Biener wrote:
So every time somebody wants to upload a test kernel into OBS two projects would be creted, not one. Actually 4 because armv7 is built against a different project. And ~10 for Factory because each arcitecture has a separate project there. And then there would be no way to look at the build result when submitting the package because the build results would be all over the place. I don't think that's a good suggestion. Thanks Michal
On 2/25/25 08:19, Adam Majer wrote:
As one of the Maintainers of X11:LXQt and really the only actual active maintainer of devel:microos:kalpa, I can say right now, take these off the list for migration, until this workflow is more stabilized and better documented. I've made the attempt a few times now, to use the git workflow (and even managed to get one successful submission through it) and I still can't really make heads or tails how it works. I'm generally in favour of this migration, to be clear, but it's obvious that this needs to cook a little longer.
Hi there, I just had an interesting use case with this: 1. ffmpeg-4 fails to build due to changelog entries out of order 2. osc branch multimedia:libs/ffmpeg-4 -> Failure: package is developed at src.opensuse.org, fork it there 3. Login at src.opensuse.org 4. fork jengelh/ffmpeg-4 5. git clone https://src.opensuse.org/manfred-h/ffmpeg-4 6. vi ffmpeg-4.changes 7. git commit -a 8. git push -> An authentication box pops up into which I enter my OBS credentials FAILURE: Authentication at src.opensuse.org failed. I then ended up to upload my changed file in the browser and created a pull request there. Where can I find documentation which explains how I can authenticate successfully at cli, i.e. "git push"? TIA, cheers. l8er manfred
On Fri Feb 28, 2025 at 2:35 PM CET, Manfred Hollstein wrote:
Do $ git clone gitea@src.opensuse.org:manfred-h/ffmpeg-4.git instead. Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Revenge is the misguided attempt to transform shame and pain into pride. -- Oceanbreeze7 in the summary of the story “Antithesis” (on FFN)
On Fri, 28 Feb 2025, 15:05:38 +0100, Manfred Hollstein wrote:
My pull request got accepted at gitea@src.opensuse.org, looking at the requests page for openSUSE:Factory, I see this: <https://build.opensuse.org/request/show/1249264> Has this been automatically generated, or has jengelh had to do that manually? I really only want to understand the whole workflow... Cheers. l8er manfred
On Friday 2025-02-28 17:10, Manfred Hollstein wrote:
The OBS-Git combo-ecosystem/sphere does not yet have "forward?" prompt/checkbox like (native) OBS does, so forwarding is a manual process. (this falls to the package maintainer, in this case, myself). As explained on https://en.opensuse.org/openSUSE:OBS_with_Git , """You would use osc sr when submitting from e.g. games/descent3 (develpkg) to openSUSE:Factory/descent3""" This has also been reiterated in A.Majer's recent opensuse-factory posting, https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/M... near """osc sr systemsmanagement:cockpit <package> openSUSE:Factory""". thanks, Jy
On Tue, 2025-02-25 at 17:19 +0100, Adam Majer wrote:
Thanks for the helpful guidance you and others have been providing on this list. As someone who jumped onto the git-obs integration train a while ago (and found the move rather challenging and destructive to my muscle- memory OBS workflow), I have a couple of questions still, if you may please consider helping me with: * How do I branch a package on the git pool and build it simultaneously for multiple openSUSE versions? For example, to check if a change I make for TW does not end up causing build failures for the package on Leap 15.6 or so on? * How do I build multiple packages in the same branch project, like I do frequently on OBS, to test whether a change I make in lib-A may not be causing some new build failures for package B? Currently, I can branch a package X from obs://devel:libraries:c_c++ into home:FOO:TestRepo and another package Y, say, from obs://science into the same home:FOO:TestRepo to test if Y plays nicely with changes in make in X. Is this still possible using packages from git pool? Thanks and best wishes. -- Atri
On Freitag, 28. Februar 2025, 17:51:13 CET Atri Bhattacharya wrote:
the repository configuration of your project (devel, home or whatever) is independend from where the project or package sources come from. So, nothing really is changing here, you can still build against any targets from the same source.
You have at least two options 1. you can still setup a classic OBS project and just link the wanted packages together. Using old style _link files. Or you just add the git repo url in the <scmsync> element of the package meta, it will automatically follow. 2. if you have multiple packages from the same git project and you plan to submit them together later, it might make sense to fork the git project and build that one in your staging project. Please note that you can also limit the package sources you want to build there using the onlybuild parameter optional, eg: http....?onlybuild=gcc&onlybuild=glibc -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Fri, 2025-02-28 at 18:14 +0100, Adrian Schröter wrote:
This was my hope too, but for example, after I clone locally <https://src.opensuse.org/badshah400/argparse>, which is a fork from <https://src.opensuse.org/clibs/argparse>, `osc build` no longer works: ``` ~> osc build WARNING: Using EXPERIMENTAL support for git scm. The functionality may change or disappear without a prior notice! Couldn't map git branch 'master' to a project ~> osc build openSUSE_Factory x86_64 WARNING: Using EXPERIMENTAL support for git scm. The functionality may change or disappear without a prior notice! Couldn't map git branch 'master' to a project ``` Where is the corresponding OBS branch project created? Or what am I doing wrong?
OK, thanks. Will try this out. Best wishes, -- Atri
On Friday 2025-02-28 19:13, Atri Bhattacharya wrote:
In the current mode, OBS projects and Git storage are pretty much decoupled; performing an action in one system does not perform the action in another. Therefore, no automatic OBS branch project. https://en.opensuse.org/openSUSE:OBS_with_Git """Git branches on their own do not lead to any implicit server-side builds being performed [...]""" If you do want an OBS branch project, you have to create one (e.g. osc meta pkg -e home:badshash400/argparse) and specify <scmsync>https://src.opensuse.org/badshah400/argparse</scmsync>.
On Fri, 2025-02-28 at 19:22 +0100, Jan Engelhardt wrote:
Thanks Jan. Actually, I just want to be able to build a package locally after forking it on gitea. So, this is something I will have to do additionally to be able to build the forked package, is it? Best wishes. -- Atri
On Samstag, 1. März 2025, 03:47:48 CET Atri Bhattacharya wrote:
When your git repository is on src.opensuse.org and used in any project or package in build.opensuse.org, the notification happens automatically. What you say is true for any other git hosting, like github.com though. One need an additional notification there.
branching is a mechanic with the classic OBS VCS, where you have a fully copy with a link. And OBS tries automatically to merge changes when scheduling a build. This does not exist with git scmsync, but you can still: 1. forking the git package using upcoming "osc fork" command instead of using "osc branch" or 2. link any git package in a classic OBS package container. In any case you can checkout with "osc co" (which will lead into a git clone by default) and build a package local (also when you don't fork it before). -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev
On Tue Feb 25, 2025 at 5:19 PM CET, Adam Majer wrote:
I haven’t expected it necessary, but it is crazy that in this day and age there are some IT people around us (I won’t name names) who have problems with elementary concepts and operations with git, like what's `git rebase` and how to deal with conflicts between branches. I am emabarrased to even mention it, but https://git-scm.com/book/en/v2 is a really good piece of writing. Please, make sure you know its content. If you don’t know that, it is not a problem of the “half broken mess of a workflow”, it is your problem. Thank you, Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 “Are you frightened?” “Yes!” “Not nearly frightened enough!” -- Aragorn and Frodo in “LotR: Fellowship of the Ring” (e.g., https://youtu.be/EJGv5lhm-ak)
On Mon, Mar 10, 2025 at 1:04 PM Matěj Cepl <mcepl@cepl.eu> wrote:
This is an unreasonable statement to make. There are many sectors where Git is just simply *not* used and other systems are. Mercurial, Perforce, SVN, and others are heavily used in industry even today. And Git is not opinionated workflow-wise, so it is absolutely okay to be critical of the Git-OBS workflow. I certainly am not happy with it, and I'm fairly well-versed in Git and even implemented my own workflow with OBS years ago for a previous employer. Do better and learn to empathize with people. -- 真実はいつも一つ！/ Always, there's only one truth!
On Mon Mar 10, 2025 at 6:08 PM CET, Neal Gompa wrote:
I have said nothing against criticism of the Git-OBS workflow, please, go ahead and criticise, it certainly needs more people thinking about it. However, that’s something else than if somebody calls it “half broken mess of a workflow” just because he doesn’t know what “This branch is 6 commits behind pool/foo:factory” means, and he is just embarassing himself. Best, Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Give your heartache to him. (1Pt 5,7; Mt 11:28-30)
On Mon, 2025-03-10 at 18:55 +0100, Matěj Cepl wrote:
To be honest, I think the best solution would have been to integrate the git backend directly into OBS and just map git commands to osc commands, so that in the end, the workflow is not going to change too much for most users. If people will want to use git, they would still be able to do so. I would call myself sufficiently well-versed in git and I think it's a fantastic tool to work with on actual code bases. But I think it's just a little overkill for source-tracking a tarball, a changes and spec file and maybe some patches. I mean, do you really need git-bisect or git-rebase for working on a simple spec file? It's not that you are working with 150.000 lines of code with hundreds of commits per week where you actually need something like git-bisect to track a bug. If you're dealing with spec files, tracking down a bug usually boils down to looking at the spec file with your own eyes and locating the problem within a few seconds. Adrian
On Monday 2025-03-10 22:44, John Paul Adrian Glaubitz via openSUSE Factory...:
I mean, do you really need git-bisect or git-rebase for working on a simple spec file?
Bisect is a method and is orthogonal to the content you are trying to analyze.
"Both. Both is good."
On 2025-03-10 10:55, Matěj Cepl wrote:
Very mature, Matej. *Maybe* if you'd explained that in the PR, instead of mocking me, I might have reacted better. And then to take something that happened on an entirely different platform, and mock me here on the -Factory ML is real classy. I've got better things to do with my time, than deal with elitist maintainers that would rather mock people trying to contribute, than try to help them learn how to do things. For reference, since Matej decided to do a nice little passive aggressive attack, without linking to the issue in question, and a clever little statement: "I haven't expected it necessary, but it is crazy that in this day and age there are some IT people around us (I won't name names) who have problems with elementary concepts and operations with git, like what's `git rebase` and how to deal with conflicts between branches." https://src.opensuse.org/pool/python-pygit2/pulls/1 It was me. I did it, and I'm not embarassed in the least, you sanctimonious blowhard. Not all of us work in git everyday. I barely touch it most days. So no, I'm not confident in my git skills. But you decided to mock, instead of to help. So yeah, I was pissed off. But I left it in the PR where it belonged. You apparently aren't able to do that.
On Tue Mar 11, 2025 at 4:33 AM CET, Shawn Dunn wrote:
I have not mocked YOU, I have never mentioned neither your name, package, or PR in question. And the point was not mocking anybody, but to suggest for people to brush on their git skills before blaming Git-OBS workflow. Also, concerning explanation, I really believed that https://src.opensuse.org/pool/python-pygit2/pulls/1#issuecomment-13309 should be enough explanation for anybody who knows what end git is.
We both with Adam tried to explain to you what’s the problem repeatedly, and you just ignored us and got offended.
It was me. I did it, and I'm not embarassed in the least, you sanctimonious blowhard. Not all of us work in git everyday. I
Talking about mature … Anyway, exactly because not everybody works with git everyday, I thought it important to suggest for people to brush on their git knowledge. There is nothing wrong with not knowing, only with unwilligness to learn. Best, Matěj -- http://matej.ceplovi.cz/blog/, @mcepl@en.osm.town GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8 Give your heartache to him. (1Pt 5,7; Mt 11:28-30)
participants (28)
-
Adam Majer -
Adrian Schröter -
alessio.biancalana＠suse.com -
Andrei Borzenkov -
Atri Bhattacharya -
Atri Bhattacharya -
Axel Braun -
daniel.garcia＠suse.com -
Dominique Leuenberger -
Fridrich Strba -
Henne Vogelsang -
Jan Engelhardt -
Jiri Slaby -
John Paul Adrian Glaubitz -
Manfred Hollstein -
Martin Wilck -
Matěj Cepl -
Michal Suchánek -
Neal Gompa -
Olaf Hering -
Rein Fernhout (Levitating) -
Richard Biener -
Robert Schweikert -
Shawn Dunn -
Shawn W Dunn
-
Stefan Seyfried -
Takashi Iwai -
tux93＠opensuse.org