All meeting minutes can be found here: https://etherpad.opensuse.org/p/ReleaseEngineering-meeting The meeting is hosted here https://meet.opensuse.org/ReleaseEngineeringMeeting ## Attendees lkocman, guillaumeg, Sarah, DimStar, rbrown, dirk ## Leap Leap 15.5 Build456.1 looks good The roadmap suggests that RC submission deadline would be tomorrow (originally planned two weeks prior to SLES Public RC deadline). But we're not in shape to meet it regarding build failures. Therefore I suggest deferring it to the same week as SLES (two weeks from now). I'll write annoucement later today. Further submissions for our ~20 build failures (half of them are already reported) Discussing growth of downloads (metrics-o-o)? So far it seems that numbers are legit based on the conversation. Leap Micro 5.4 is Ready for transition to RC Seems like we'll have issue with IPRQ (legal staffing) https://news.opensuse.org/2023/03/28/leapmicro-54-beta-hands-on/ We could do something similar as ^ Doug do we know if the article had increased amount of views compared to the usual numbers ^? Devconf mini conference @Brno on Friday went well! https://twitter.com/openSUSE/status/1642822371391295488?s=20 15.4 image respin based on QU2 - TBD Leap 16.0 - Waiting for further info from Richard (our new Distro architect), so far it seems that we can continue with what we have. I'm personally currently at no capacity (this particular week) to work on it further. Meeting with Czech OSPO (Open Source Program Office) tomorrow at 14:30 as Czech SUSE office wants to be a sponsor. Czech branch can offer monetary as well as potential legal support (the latter depends on the scope). ## openSUSE Tumbleweed openSUSE:Factory build fail stats: 133 failed 29 unresolvable (one week ago: 200 / 53) (number dropped as i586 is now only < 2k packages instead of 15k) https://tinyurl.com/ysy4nnnz * List of packages built for i586 has been set to a bare minimum based on what wine/steam has declared as build- and runtime dependencies. There are high chances some -32bit packages people rely on are missing now. Please file bug reports, assigned to dimstar@o.o, with justification to enable the additional -32bit package (i.e what functionality are you missing; not just package names missing) * python38-* modules are no longer built for Tumbleweed. The python 3.8 interpreter is still there for now, but will likely be removed not too far in the future * Staging:C: openssl 3.1 incoming; a few packages failing to build * i586 carve-out (LegacyX86 port) from my PoV this is now considered complete. I will no longer actively look after that port and expect the community to pick this up (look after buiild fails specific to i586, monitor openQA, reach out when questions arise. I'll be there to help Letter staging projects still build for i586 (build-only, no QA), to have at least a minimal level of confidence (would love to do that for other arches too, but build power there is too limited) ## Richard (MicroOS) The MicroOS Desktop Plasma/KDE is in desperate need of help else could be at risk of being dropped (again) https://microos.opensuse.org/blog/2023-04-02-state-of-microOS-Desktop-Plasma... ## Richard (ALP Architecture) Richard in his new role as SUSE's Distribution Architect is looking at ways of improving ALP's contribution story and use by openSUSE as a base for future offerings. This is particually urgent as OBS's SUSE:ALP project is effectively in progress of being 'moved' to SUSE's internal IBS in order to comply with all the certifications SUSE need for ALP. For those who need a very brief understanding of how ALP is built - this is rather different from previous SUSE/openSUSE Products SUSE:ALP:Workbench is effectively the equivalent of Factory:Rings:0, being used for bootstrapping SUSE:ALP:Source:Standard is the name for the 'track', with there being concepts for possible future 'tracks' like "Rolling/Fast" or "Premium" or such SUSE:ALP:Source:Standard:Core is the name for where 'core' packages are built, the ones that will be used by all Products SUSE:ALP:Products:<PRODUCTNAME> is the name for where products are actually built, like a SUSE ALP Bedrock (aka Server) product and a SUSE ALP Micro product Current working plan is to recreate the above in an openSUSE:ALP namespace Everything SUSE does in their IBS equivalent namespaces will be synced 1:1 to the OBS openSUSE:ALP namespace Contribution to these layers of ALP might not be implimented in time for SUSE:ALP GA, but obviously we want that in the long term - we just also want it to be done better than we did with Closed-Leap-Gap Leap openSUSE will be able to create their own Products in the openSUSE:ALP:Products namespace - this might be where/how we build a "Leap 16.0" or maybe a more appropriately named traditional distro based on ALP, assuming the contributions for it can be found. After the above is implimented obvious next steps would include investigating establishing an openSUSE:ALP:Source:Rolling track, and mapping it to Tumbleweed so we can start using those projects for developing the 'ALP+1' codebase lkocman: Current solution for Leap 15.X is https://en.opensuse.org/Portal:Jump:OBS:SRMirroring (the bi-directional sync currently does not work). lkocman is the "feedback loop" bottleneck here. rbrown: ^ exactly - and we want to do something better than this, which might mean doing nothing until we have the time to really do something a lot better than that Adrian: people who might be interested in designing contribution process to ALP., should join the public meetings of https://en.opensuse.org/openSUSE:ALP/Workgroups/Git-Packaging-Workflow ## Max not available Leap 15.5 * New snapshot 442.1 published with KDE updates * Focus on build fails in Backports * Build stats in Backports(x86_64): 3 unresolvables, 24 fails(last week: 7 unresolvables, 38 fails) ## Guillaume - Arm openQA: * openqa-aarch64 worker is very slow. Likely related to btrfs issues, investigating. Tumbleweed: * 20230330 was the 1st released snapshot which was rebuilt with gcc13. * GCC13 adds support for AArch64 LSE and LSE2 to libatomic. Disable outline atomics, and use LSE ifuncs for 1-8 byte atomics and LSE2 ifuncs for 16-byte atomics. (On Neoverse V1, 16-byte atomics are ~4x faster due to avoiding locks) * NVIDIA: tester with aarch64 server and NVIDIA card wanted - Proprietary drivers are now available for aarch64 (only G06): https://download.nvidia.com/opensuse/tumbleweed/ - New opengpu driver also available in OBS: https://build.opensuse.org/project/monitor/X11:Drivers:Video lkocman to check on who is the aarch64 + nvidia effort blocked on. I recall that there was a chosen point of contact. No update sorry. Leap: * 15.5 aarch64: No aarch64 specific issues * 15.5 armv7: no blocker ALP: * No aarch64 specific issues WSL: * Works with x86 emulator since appx installer is x86-64, but this is not really an issue since arm64 Win11 includes x86 emulator by default. => Could we publish it on Microsoft store anyway? lkocman: Team is okay, but we need to make sure that such case is covered in openQA https://progress.opensuse.org/issues/126083. Steps documented on the wiki to install the appx from download.o.o: https://en.opensuse.org/openSUSE:WSL#With_Appx_from_openSUSE_download_server ## Sarah - s390x Tumbleweed * release is rolling * kernel was not able to build because of gcc13 problems (fixed with a workaround): https://bugzilla.opensuse.org/show_bug.cgi?id=1209724 Leap: * working * many build failures and unresolvables based on new gcc and LLVM * New topic HPC packages (enablement) at the Linux Distributions Working Group @ Open Mainframe Project (OpenMP has been enabled for gcc and developed for LLVM, but forgotten to commit) * IBM has released the IBM LinuxONE RockHopper 4 for Edge Computing and Industry 4.0 (matching default server racks): https://newsroom.ibm.com/2023-04-04-IBM-Furthers-Flexibility,-Sustainability... ## Doug * Google Summer of Code * 32 Proposals Submitted; some were submitted that were not listed on 101.opensuse.org lkocman: Do we have any number for last year? Like are we growing or is it about the same? * If interested in mentoring, email ddemaio * Lists of proposal titles not listed on 101.o.o: * Container-Based Backend for openQA: Enhancing Automated Testing of Containers. * AES Technology * Editing directory services with Yast2 * Contribute more about Rust FFI for RADIUS project * Implementing an out-of-band identity verification system for Kanidm * Name change history - openSUSE Kanidm * Rancher Microservice Workload infrastructure to Medical/Industrial areas using MQTT and HTTP * Streamlining Software Development and Packaging with the openSUSE Build Service * Water leakage detection using IOT * Ranking session planned for April 25 (deadline April 27) * oSC23 * CfP closes this Sunday, April 9 * Uncertain whether Sunday will be neccessary * 85 registered, 57 submissions * Making mock schedule * Sent package to Netherlands for 2 events * Starting Leap 15..5 release annoucement * Leap Micro 5.4 beta article had 2x views as TW update article! Awesome, I'll prep the Nextcloud "spin" of the article. Devconf.us - Boston / August cfp as extended to April 10th. We do have a placeholder for talk. ## Dirk Improved the CDN setup a bit further (https working now, caching issues fixed) and in progress of collecting in put from early testers. It looks like we have more issues to resolve. Did a strategic benchmarking exercise with various zypper options and a huge part of the slowness appears to be related to the choices of zypper options. In evaluation with zypper folks. A > factor 5 improvement even for european is possible over current setup. Other locations could be better. Sarah can provide mainframe acess * also todo announce tumbleweed maintainer policy draft currently working on testing those in a private staging test project * Started ALP:RISCV:* builds in the new build setup Biggest speedup can be observed by switching zlib to zlib-ng, so looked into fixing the build failures caused by switching to zlibo-ng- compat ## Wolfgang (Package Hub), Scott Bahling Not available * Moved duplicated packages to subpackages for SLE-15-SP5 (thanks to Max for the list of packages) * Packages in subpackages for SP5 is up to the level of SP4 (still considering putting more packages there because of migration from Leap to SLE) * patterns-mate added to openSUSE:Backports:SLE-15-SP5 (thanks Max) ## Maintenance team (Marcus or Maurizio (m4u)) lkocman: we need to syncup on Leap Micro 5.4 GA. I believe SLE Micro is only blocked on IPRQ and then they can go live. Our update channel should receive all SLEM 5.4 updates from that point in time. 15.5 setup is done - Do we want to swap key for the 4096 rsa key now? Done lkocman: We're still not in RC phase, so better now than later. openSUSE:Leap15.5 would be affected for now, backports are bit more challenging because of SLES. Wolfgang: we're injecting the key. Marcus we probably want to do only Backports 15 SP5 for now. Marcus is working also on the SLES side, the update of package with the new key was released last week, it's just not activated yet, but it will be already trusted by Leap 15.5 systems. 15.4 is working 5.3 is working 5.4 testing is now passing as well Leap 16.0 - we should revisit the update/sle repo as the current setup not exactly mirror friendly. Solutions could be dropping not so popular architectures or split repositories per architecture. securebootkey for SLES was rotated, it should be autotrusted and not noticeable. This will be in QU3, QU2 is already done. 15.5 setup is done (status 21.2.2023). lkocman needs to do official 15.5 Maint setup request beta in ~2 weeks lkocman: 15.3 EOL could lead to stopping our physical Source DVD effort, as it seems we will not produce. As this was the last release which you could still get on a physical media. Lkocman: anything against decomissioning it? Configuration setup problem for Maintenance of Leap 15.4 maintenance updates / openQA Marcus regarding Leap 15.4 Image respin - package set will change, we do need to refresh the packagelist * Lubos to talk to Jan Stehlik, we can't put all on Marcuses shoulder. https://etherpad.opensuse.org/p/ReleaseEngineering-20221110-maintenance-disc... Confirmation that QA/QA-maint team will oversee the setup (issues) Lubos: I was asked to provide requirements for the QA team. Mostly for the GA/current release but also for the update. Lubos will make wiki with requirements (something like maintenance plan perhaps). Marcus will review it. * Leap Micro 5.3 maint setup done ffmpeg - (still unsolved) possible file conflict on the next update, no idea how to avoid vendor switching at the moment. Removing the patch on the openSUSE side (that might contain security fixes) or releasing update on the packman side could fix the issue. Lubos to give Marcus some working contact for the team. (Was done, Olaf Hering) ## Adrian - OBS DimStar pointed Adrian to issue handled by Marco - unresolvables due to python3.8 drop. So far it seems like a scheduler issue. Just wait for fix, local builds are not affected. ## Open Floor metrics-o-o https://metrics.opensuse.org/ https://github.com/openSUSE/openSUSE-release-tools/tree/master/metrics/acces... Bernhard provided couple of samples and it appears, that quite some traffic is taken with a single entity using 500+ Leap 15.4 containers/day. Most of these seem to live just for a few minutes. Intention is to collect more detailed information what kind of usage do we see. Also storing GEO location would be good.