When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems. -- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu ________________________________________ From: cbbayburt <cbbayburt@suse.de> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi Simon, On 2021-06-09 14:49, Simon Avery wrote:

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

Yeah, that can be made to work, but it is a whole bunch of extra "stuff" to have to maintain, and a change in process. We just associate the live repo directly to the systems with SUSE, openSUSE, and OEL/CentOS prior to 8.x. Syncs happen overnight, and in the morning, new patches show up ready to be applied to relevant systems. We don't do any formal validation of patches/baseline for patches. I go to patches/relevant patches/security, and apply the ones that don't require a reboot and/or break something without manual intervention. It has worked for years this way. Requiring something special to work around newer versions of CentOS/OEL is just extra steps. It doesn't affect us that much, since we don't have that many of them, but I want the same functionality with those as we have with SUSE and CentOS/OEL 7 (associate the managed system with the live repo, and push patches from the web ui). I believe that is the same thing Simon was also wanting. -- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu ________________________________________ From: Michael Calmer <mc@suse.de> Sent: Thursday, June 10, 2021 8:59 AM To: users@lists.uyuni-project.org Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination. In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo. After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update. Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button. But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos. Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu

________________________________________ From: cbbayburt <cbbayburt@suse.de> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards Michael Calmer -------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

Michael, Allen; Thank you both for your replies. I will raise an Issue as suggested. I have found another workaround which I'm going to be using until a better solution emerged. For the benefit of others, it is to deploy a local version of the Centos .repo file for AppStream. Even if Uyuni also provides a "broken" Appstream via its susemanager:channels.repo file, dnf is smart enough to choose the modules from the Master mirror and updates start working again. This is not ideal since it means that each machine will be downloading packages from the main Centos mirror repos, so our external bandwidth will take a hit. However, this is a compromise that we need to make, because the only other alternatives I have is to run a CLM project which will take me several hours extra a week, or to bypass Uyuni entirely and either download all packages directly or to use an alternative repo manager. On that last point, it's clear that Uyuni is not the only toolchain that has been broken by this change to modules. Pulp<https://pulpproject.org/>, which I think has a larger *EL userbase than Uyuni, has quite a collection of bug reports about it. I understand they have managed to solve the issue, and perhaps there is a method from them that can be used for Uyuni too? I am mindful that this is not a "Suse problem" and us poor Centos/Rhel/Fedora/Rocky/Alma/Amazon/Alibaba users, who have suffered so much from Redhat's business decisions over the past years, are grateful for the support that can be given. Thanks as ever, Simon -----Original Message----- From: Michael Calmer <mc@suse.de> Sent: 10 June 2021 15:00 To: users@lists.uyuni-project.org Subject: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination. In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo. After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update. Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button. But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos. Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu

________________________________________ From: cbbayburt <cbbayburt@suse.de> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards Michael Calmer -------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

Hi Hmm, there should be no broken AppStream when you connect your client directly to Uyuni repositories. We mirror the modules.yaml files and provide it 1:1 to the client. About other management server I heard that all use the same approach we use. Let the user make a decision and convert the module repos into normal one. But I did not check the code. Am Donnerstag, 10. Juni 2021, 16:25:57 CEST schrieben Sie:

Michael, Allen; Thank you both for your replies.

I will raise an Issue as suggested.

I have found another workaround which I'm going to be using until a better solution emerged. For the benefit of others, it is to deploy a local version of the Centos .repo file for AppStream.

Even if Uyuni also provides a "broken" Appstream via its susemanager:channels.repo file, dnf is smart enough to choose the modules from the Master mirror and updates start working again.

This is not ideal since it means that each machine will be downloading packages from the main Centos mirror repos, so our external bandwidth will take a hit. However, this is a compromise that we need to make, because the only other alternatives I have is to run a CLM project which will take me several hours extra a week, or to bypass Uyuni entirely and either download all packages directly or to use an alternative repo manager.

On that last point, it's clear that Uyuni is not the only toolchain that has been broken by this change to modules. Pulp<https://pulpproject.org/>, which I think has a larger *EL userbase than Uyuni, has quite a collection of bug reports about it. I understand they have managed to solve the issue, and perhaps there is a method from them that can be used for Uyuni too? I am mindful that this is not a "Suse problem" and us poor Centos/Rhel/Fedora/Rocky/Alma/Amazon/Alibaba users, who have suffered so much from Redhat's business decisions over the past years, are grateful for the support that can be given. Thanks as ever, Simon

-----Original Message----- From: Michael Calmer <mc@suse.de> Sent: 10 June 2021 15:00 To: users@lists.uyuni-project.org Subject: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination.

In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo.

After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update.

Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button.

But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos.

Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

...

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu

________________________________________ From: cbbayburt <cbbayburt@suse.de> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards

Michael Calmer

-------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- Regards Michael Calmer -------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

Hi Both, What you describe was the case up until the most recent point release of Centos this month. I was using Uyuni repos and scheduling 'dnf -y update' to perform the updates. That installed anything that was needed and Uyuni's package refresh would update its records within 24 hours of everything that changed. A perfectly acceptable workaround. However, when Centos 8.4 was released, this no longer works. I don't know why this change breaks that, but it appears to for me. Outputs below, all repos are synced to Centos mirrors, branch /8/ However, if I add a direct-from-master CentOS8-AppStream.repo file on this client, everything "just works", which suggests to me the Uyuni repos aren't behaving exactly as the masters. Thoughts? S # cat /etc/centos-release CentOS Linux release 8.3.2011 # dnf repolist repo id repo name susemanager:c8powertools CentOS 8 Powertools susemanager:centos8-uyuni-client-x86_64 Uyuni Client Tools for CentOS 8 (x86_64) susemanager:centos8-x86_64 CentOS 8 (x86_64) susemanager:centos8-x86_64-appstream CentOS 8 AppStream (x86_64) # dnf update Last metadata expiration check: 1:09:19 ago on Fri 11 Jun 2021 06:24:18 AM BST. Error: Problem 1: package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install the best update candidate for package perl-libs-4:5.26.3-416.el8.x86_64 - cannot install the best update candidate for package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 Problem 2: perl-libs-4:5.26.3-416.el8.i686 has inferior architecture - package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch requires perl(:MODULE_COMPAT_5.26.3), but none of the providers can be installed - perl-libs-4:5.26.3-419.el8.i686 has inferior architecture - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Carp-1.50-439.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch - cannot install the best update candidate for package perl-Carp-1.42-396.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 3: package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - cannot install the best update candidate for package perl-Data-Dumper-2.167-399.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering Problem 4: problem with installed package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-1.17-396.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-1.17-395.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 5: problem with installed package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-MD5-2.55-396.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) From: Pau Garcia <pau.garcia@suse.com> Sent: 10 June 2021 15:45 To: Michael Calmer <mc@suse.de>; Simon Avery <Simon.Avery@atass-sports.co.uk>; users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi That's right: if you assign the base OS channel and the modules channel from Uyuni to the client directly, dnf should see the modules and appstreams. It's just the Uyuni WebUI may not reflect the actual software installed on the system but managing locally with dnf should work. Thank you Pau Garcia Quiles SUSE Manager Product Owner & Technical Project Manager Engineering & Innovation Phone: +34 91 048 7632 SUSE Software Solutions Spain ________________________________ From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: Thursday, June 10, 2021 4:35 PM To: Simon Avery <Simon.Avery@atass-sports.co.uk<mailto:Simon.Avery@atass-sports.co.uk>>; users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> <users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org>> Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi Hmm, there should be no broken AppStream when you connect your client directly to Uyuni repositories. We mirror the modules.yaml files and provide it 1:1 to the client. About other management server I heard that all use the same approach we use. Let the user make a decision and convert the module repos into normal one. But I did not check the code. Am Donnerstag, 10. Juni 2021, 16:25:57 CEST schrieben Sie:

Michael, Allen; Thank you both for your replies.

I will raise an Issue as suggested.

I have found another workaround which I'm going to be using until a better solution emerged. For the benefit of others, it is to deploy a local version of the Centos .repo file for AppStream.

Even if Uyuni also provides a "broken" Appstream via its susemanager:channels.repo file, dnf is smart enough to choose the modules from the Master mirror and updates start working again.

This is not ideal since it means that each machine will be downloading packages from the main Centos mirror repos, so our external bandwidth will take a hit. However, this is a compromise that we need to make, because the only other alternatives I have is to run a CLM project which will take me several hours extra a week, or to bypass Uyuni entirely and either download all packages directly or to use an alternative repo manager.

On that last point, it's clear that Uyuni is not the only toolchain that has been broken by this change to modules. Pulp<https://pulpproject.org/>, which I think has a larger *EL userbase than Uyuni, has quite a collection of bug reports about it. I understand they have managed to solve the issue, and perhaps there is a method from them that can be used for Uyuni too? I am mindful that this is not a "Suse problem" and us poor Centos/Rhel/Fedora/Rocky/Alma/Amazon/Alibaba users, who have suffered so much from Redhat's business decisions over the past years, are grateful for the support that can be given. Thanks as ever, Simon

-----Original Message----- From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: 10 June 2021 15:00 To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> Subject: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination.

In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo.

After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update.

Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button.

But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos.

Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

...

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu<mailto:allen@ua.edu>

________________________________________ From: cbbayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards

Michael Calmer

-------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- Regards Michael Calmer -------------------------------------------------------------------------- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> -------------------------------------------------------------------------- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

Hi Can, Certainly. I fetch mirror modules.yaml from a mirror and run "xz -d" to unpack it to modules.yaml Wget http://mirror.centos.org/centos/8/AppStream/x86_64/os/repodata/4e8983e548945... On Uyuni, I search for modules.yaml and find two. uyuni01:/home/simon # locate modules.yaml /var/spacewalk/rhn/modules/c8powertools/7543ae5118685eb1300dcf40ceb150cb5770f6eaada7b7c7adff36ad1d3e7899-modules.yaml /var/spacewalk/rhn/modules/centos8-x86_64-appstream/2c3714db39642790c8a1922c6cae04e7b95af59b234af60f15778d5550e3a546-modules.yaml Ignoring the Powertools one, immediately we see the filename, filesize and dates are different, as well as the mirror being compressed and the local not. Local: -rw-r--r-- 1 root root 463990 Jun 11 10:35 2c3714db39642790c8a1922c6cae04e7b95af59b234af60f15778d5550e3a546-modules.yaml Mirror version -rw-r--r-- 1 root root 538459 Jun 8 17:04 4e8983e548945eec5cc11a296621f6c0657c3afd600cb0a507e779a18f8d68ea-modules.yaml So if this is the right location for this file in Uyuni, we might be onto something. I'm feeling like nuking this Uyuni repo and starting over might be a good thing to try? S -----Original Message----- From: cbbayburt <cbbayburt@suse.de> Sent: 11 June 2021 09:48 To: users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? Hi Simon, Can you check if the 'modules.yaml' from Uyuni's synced repo in the client's cache is the same as the original one from the direct CentOS repo? If not, it might indicate a bug with updating this metadata on Uyuni's side. On 2021-06-11 08:39, Simon Avery wrote:

Hi Both,

What you describe was the case up until the most recent point release of Centos this month. I was using Uyuni repos and scheduling 'dnf -y update' to perform the updates. That installed anything that was needed and Uyuni's package refresh would update its records within 24 hours of everything that changed. A perfectly acceptable workaround.

However, when Centos 8.4 was released, this no longer works. I don't know why this change breaks that, but it appears to for me.

Outputs below, all repos are synced to Centos mirrors, branch /8/

However, if I add a direct-from-master CentOS8-AppStream.repo file on this client, everything "just works", which suggests to me the Uyuni repos aren't behaving exactly as the masters.

Thoughts?

S

# cat /etc/centos-release CentOS Linux release 8.3.2011

# dnf repolist repo id repo name susemanager:c8powertools CentOS 8 Powertools susemanager:centos8-uyuni-client-x86_64 Uyuni Client Tools for CentOS 8 (x86_64) susemanager:centos8-x86_64 CentOS 8 (x86_64) susemanager:centos8-x86_64-appstream CentOS 8 AppStream (x86_64)

# dnf update Last metadata expiration check: 1:09:19 ago on Fri 11 Jun 2021 06:24:18 AM BST. Error: Problem 1: package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install the best update candidate for package perl-libs-4:5.26.3-416.el8.x86_64 - cannot install the best update candidate for package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 Problem 2: perl-libs-4:5.26.3-416.el8.i686 has inferior architecture - package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch requires perl(:MODULE_COMPAT_5.26.3), but none of the providers can be installed - perl-libs-4:5.26.3-419.el8.i686 has inferior architecture - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Carp-1.50-439.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch - cannot install the best update candidate for package perl-Carp-1.42-396.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 3: package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - cannot install the best update candidate for package perl-Data-Dumper-2.167-399.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering Problem 4: problem with installed package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-1.17-396.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-1.17-395.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 5: problem with installed package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-MD5-2.55-396.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

From: Pau Garcia <pau.garcia@suse.com> Sent: 10 June 2021 15:45 To: Michael Calmer <mc@suse.de>; Simon Avery <Simon.Avery@atass-sports.co.uk>; users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

That's right: if you assign the base OS channel and the modules channel from Uyuni to the client directly, dnf should see the modules and appstreams. It's just the Uyuni WebUI may not reflect the actual software installed on the system but managing locally with dnf should work.

Thank you

Pau Garcia Quiles SUSE Manager Product Owner & Technical Project Manager Engineering & Innovation Phone: +34 91 048 7632 SUSE Software Solutions Spain

________________________________ From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: Thursday, June 10, 2021 4:35 PM To: Simon Avery <Simon.Avery@atass-sports.co.uk<mailto:Simon.Avery@atass-sports.co.uk>

...

; users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> <users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org>> Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

Hmm, there should be no broken AppStream when you connect your client directly to Uyuni repositories. We mirror the modules.yaml files and provide it 1:1 to the client.

About other management server I heard that all use the same approach we use. Let the user make a decision and convert the module repos into normal one. But I did not check the code.

Am Donnerstag, 10. Juni 2021, 16:25:57 CEST schrieben Sie:

...

Michael, Allen; Thank you both for your replies.

I will raise an Issue as suggested.

I have found another workaround which I'm going to be using until a better solution emerged. For the benefit of others, it is to deploy a local version of the Centos .repo file for AppStream.

Even if Uyuni also provides a "broken" Appstream via its susemanager:channels.repo file, dnf is smart enough to choose the modules from the Master mirror and updates start working again.

This is not ideal since it means that each machine will be downloading packages from the main Centos mirror repos, so our external bandwidth will take a hit. However, this is a compromise that we need to make, because the only other alternatives I have is to run a CLM project which will take me several hours extra a week, or to bypass Uyuni entirely and either download all packages directly or to use an alternative repo manager.

On that last point, it's clear that Uyuni is not the only toolchain that has been broken by this change to modules. Pulp<https://pulpproject.org/>, which I think has a larger *EL userbase than Uyuni, has quite a collection of bug reports about it. I understand they have managed to solve the issue, and perhaps there is a method from them that can be used for Uyuni too? I am mindful that this is not a "Suse problem" and us poor Centos/Rhel/Fedora/Rocky/Alma/Amazon/Alibaba users, who have suffered so much from Redhat's business decisions over the past years, are grateful for the support that can be given. Thanks as ever, Simon

-----Original Message----- From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: 10 June 2021 15:00 To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> Subject: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination.

In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo.

After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update.

Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button.

But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos.

Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

...

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu<mailto:allen@ua.edu>

________________________________________ From: cbbayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards

Michael Calmer

--------------------------------------------------------------------- ----- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> --------------------------------------------------------------------- ----- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- Regards

Michael Calmer

---------------------------------------------------------------------- ---- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> ---------------------------------------------------------------------- ---- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

Interesting. Test 1: - I moved the existing modules.yaml away from /var/spacewalk/rhn/modules/centos8-x86_64-appstream/2c3714db39642790c8a1922c6cae04e7b95af59b234af60f15778d5550e3a546-modules.yaml - Re-synced the repo, it was NOT recreated. Test 2: - I deleted the repo and the software channel for Appstream in Uyuni. - I created a new, differently named repo and software channel and synced it, confirming that a whole new set of 6133 packages was downloaded. - I re-added those machines that were on the original channel and ensured that change was applied to them - "dnf update" from those machines again failed with module errors. Test 3: - After doing the above, I did an `updatedb` and `locate modules.yaml` and no new files had been created in /var/spacewalk/rhn/modules/ for the new channel - there's no new sub directory created there at all. Test 4: - The Overview.do for a machine that is subscribed to the new Channel does show the warning "At least one of the channels this system is subscribed to contains modules" on Details. Unsubscribing it from the new Appstream channel removes that, so Uyuni has recognised that this new channel has modules. So it does look a lot like Uyuni is not only not updating the appropriate modules.yaml from the repo when it syncs, it's no longer creating it in the first place when a new channel/repo is created. (Although it must have done before!) (I'll cut and paste this into the issue) Thanks S -----Original Message----- From: cbbayburt <cbbayburt@suse.de> Sent: 11 June 2021 11:41 To: users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this? On 2021-06-11 11:46, Simon Avery wrote:

Hi Can,

Certainly.

I fetch mirror modules.yaml from a mirror and run "xz -d" to unpack it to modules.yaml

Wget http://mirror.centos.org/centos/8/AppStream/x86_64/os/repodata/4e8983e 548945eec5cc11a296621f6c0657c3afd600cb0a507e779a18f8d68ea-modules.yaml .xz

On Uyuni, I search for modules.yaml and find two.

uyuni01:/home/simon # locate modules.yaml /var/spacewalk/rhn/modules/c8powertools/7543ae5118685eb1300dcf40ceb150 cb5770f6eaada7b7c7adff36ad1d3e7899-modules.yaml /var/spacewalk/rhn/modules/centos8-x86_64-appstream/2c3714db39642790c8 a1922c6cae04e7b95af59b234af60f15778d5550e3a546-modules.yaml

Ignoring the Powertools one, immediately we see the filename, filesize and dates are different, as well as the mirror being compressed and the local not.

Local: -rw-r--r-- 1 root root 463990 Jun 11 10:35 2c3714db39642790c8a1922c6cae04e7b95af59b234af60f15778d5550e3a546-modul es.yaml Mirror version -rw-r--r-- 1 root root 538459 Jun 8 17:04 4e8983e548945eec5cc11a296621f6c0657c3afd600cb0a507e779a18f8d68ea-modul es.yaml

So if this is the right location for this file in Uyuni, we might be onto something.

Just looking at the filesizes, I can say there's a big difference between them. My suspicion is that the file synced in the Uyuni never got updated since the first sync.

I'm feeling like nuking this Uyuni repo and starting over might be a good thing to try?

If my suspicion is correct, this might solve the problem only temporarily until the file gets obsolete after some updates. But this would give us some valuable information as it would confirm the problem.

S

-----Original Message----- From: cbbayburt <cbbayburt@suse.de> Sent: 11 June 2021 09:48 To: users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

Can you check if the 'modules.yaml' from Uyuni's synced repo in the client's cache is the same as the original one from the direct CentOS repo? If not, it might indicate a bug with updating this metadata on Uyuni's side.

On 2021-06-11 08:39, Simon Avery wrote:

...

Hi Both,

What you describe was the case up until the most recent point release of Centos this month. I was using Uyuni repos and scheduling 'dnf -y update' to perform the updates. That installed anything that was needed and Uyuni's package refresh would update its records within 24 hours of everything that changed. A perfectly acceptable workaround.

However, when Centos 8.4 was released, this no longer works. I don't know why this change breaks that, but it appears to for me.

Outputs below, all repos are synced to Centos mirrors, branch /8/

However, if I add a direct-from-master CentOS8-AppStream.repo file on this client, everything "just works", which suggests to me the Uyuni repos aren't behaving exactly as the masters.

Thoughts?

S

# cat /etc/centos-release CentOS Linux release 8.3.2011

# dnf repolist repo id repo name susemanager:c8powertools CentOS 8 Powertools susemanager:centos8-uyuni-client-x86_64 Uyuni Client Tools for CentOS 8 (x86_64) susemanager:centos8-x86_64 CentOS 8 (x86_64) susemanager:centos8-x86_64-appstream CentOS 8 AppStream (x86_64)

# dnf update Last metadata expiration check: 1:09:19 ago on Fri 11 Jun 2021 06:24:18 AM BST. Error: Problem 1: package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install the best update candidate for package perl-libs-4:5.26.3-416.el8.x86_64 - cannot install the best update candidate for package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 Problem 2: perl-libs-4:5.26.3-416.el8.i686 has inferior architecture - package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch requires perl(:MODULE_COMPAT_5.26.3), but none of the providers can be installed - perl-libs-4:5.26.3-419.el8.i686 has inferior architecture - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Carp-1.50-439.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Mozilla-CA-20160104-7.module_el8.3.0+416+dee7bcef.noarch - cannot install the best update candidate for package perl-Carp-1.42-396.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 3: package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Data-Dumper-2.174-440.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - cannot install the best update candidate for package perl-Data-Dumper-2.167-399.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering Problem 4: problem with installed package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-1.17-396.module_el8.4.0+646+45e06e4a.noarch requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-1.17-395.el8.noarch - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering Problem 5: problem with installed package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch - package perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+ff426aa3.noarch requires perl(Net::SSLeay) >= 1.46, but none of the providers can be installed - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+ff426aa3.x86_64 requires libperl.so.5.26()(64bit), but none of the providers can be installed - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-416.el8.x86_64 - cannot install both perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 and perl-libs-4:5.26.3-419.el8.x86_64 - cannot install both perl-libs-4:5.26.3-416.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - cannot install both perl-libs-4:5.26.3-419.el8.x86_64 and perl-libs-4:5.30.1-452.module_el8.4.0+646+45e06e4a.x86_64 - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires perl(:MODULE_COMPAT_5.30.1), but none of the providers can be installed - package perl-Digest-MD5-2.55-397.module_el8.4.0+646+45e06e4a.x86_64 requires libperl.so.5.30()(64bit), but none of the providers can be installed - cannot install the best update candidate for package perl-Digest-MD5-2.55-396.el8.x86_64 - package perl-Net-SSLeay-1.88-1.el8.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64 is filtered out by modular filtering - package perl-libs-4:5.30.1-451.module_el8.3.0+406+78614513.x86_64 is filtered out by modular filtering - package perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64 is filtered out by modular filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

From: Pau Garcia <pau.garcia@suse.com> Sent: 10 June 2021 15:45 To: Michael Calmer <mc@suse.de>; Simon Avery <Simon.Avery@atass-sports.co.uk>; users@lists.uyuni-project.org Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

That's right: if you assign the base OS channel and the modules channel from Uyuni to the client directly, dnf should see the modules and appstreams. It's just the Uyuni WebUI may not reflect the actual software installed on the system but managing locally with dnf should work.

Thank you

Pau Garcia Quiles SUSE Manager Product Owner & Technical Project Manager Engineering & Innovation Phone: +34 91 048 7632 SUSE Software Solutions Spain

________________________________ From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: Thursday, June 10, 2021 4:35 PM To: Simon Avery <Simon.Avery@atass-sports.co.uk<mailto:Simon.Avery@atass-sports.co.uk

...

; users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org>

<users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org>> Subject: Re: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

Hmm, there should be no broken AppStream when you connect your client directly to Uyuni repositories. We mirror the modules.yaml files and provide it 1:1 to the client.

About other management server I heard that all use the same approach we use. Let the user make a decision and convert the module repos into normal one. But I did not check the code.

Am Donnerstag, 10. Juni 2021, 16:25:57 CEST schrieben Sie:

...

Michael, Allen; Thank you both for your replies.

I will raise an Issue as suggested.

I have found another workaround which I'm going to be using until a better solution emerged. For the benefit of others, it is to deploy a local version of the Centos .repo file for AppStream.

Even if Uyuni also provides a "broken" Appstream via its susemanager:channels.repo file, dnf is smart enough to choose the modules from the Master mirror and updates start working again.

This is not ideal since it means that each machine will be downloading packages from the main Centos mirror repos, so our external bandwidth will take a hit. However, this is a compromise that we need to make, because the only other alternatives I have is to run a CLM project which will take me several hours extra a week, or to bypass Uyuni entirely and either download all packages directly or to use an alternative repo manager.

On that last point, it's clear that Uyuni is not the only toolchain that has been broken by this change to modules. Pulp<https://pulpproject.org/>, which I think has a larger *EL userbase than Uyuni, has quite a collection of bug reports about it. I understand they have managed to solve the issue, and perhaps there is a method from them that can be used for Uyuni too? I am mindful that this is not a "Suse problem" and us poor Centos/Rhel/Fedora/Rocky/Alma/Amazon/Alibaba users, who have suffered so much from Redhat's business decisions over the past years, are grateful for the support that can be given. Thanks as ever, Simon

-----Original Message----- From: Michael Calmer <mc@suse.de<mailto:mc@suse.de>> Sent: 10 June 2021 15:00 To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org> Subject: [EXTERNAL EMAIL] Re: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi

I doubt that every system needs an own clone. In best case you just need 1 for all. You just need to decide what streams you want to use and not use on every system a different stream combination.

In uyuni we have special UIs (and APIs) for creating Projects and Environments. If you want an easy case, you just create 1 project with 1 environment. You create a module filters and select for every module the stream you want to have in the resulting repo.

After the setup is done, you just need to press 1 button and all the cloning and other calculations are done. You get the final repositories and can update.

Next day (if you want to patch) you just go to that project and press the build button again. It will re-build the environment and you will get the latest updates and changes from "last night" into your environment. So "Day 2" operation is just to click 1 Button.

But here I could think of a first feature request: put a schedule on it to automate the rebuilding. This may not be a good solution for big companies, but would make things easier for smaller who would otherwise just connect directly to the repos.

Am Donnerstag, 10. Juni 2021, 15:45:21 CEST schrieb Allen Beddingfield:

...

When you refer to the "Content Lifecycle Management Process", I assume you mean cloning repos from a particular day, making changes, and associating those with the managed server, instead of the live once that is being synced? That is a whole other process that has to be implemented, just to work around this. Luckily, we only have a small handful of non SUSE or openSUSE systems to deal with this problem on, but having to clone and maintain repos for them is not really feasible. I can't imagine it being an option for someone with hundreds of CentOS 8 systems, who isn't doing this with current systems.

-- Allen Beddingfield Systems Engineer Office of Information Technology The University of Alabama Office 205-348-2251 allen@ua.edu<mailto:allen@ua.edu>

________________________________________ From: cbbayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Sent: Thursday, June 10, 2021 8:35 AM To: users@lists.uyuni-project.org<mailto:users@lists.uyuni-project.org

...

Subject: [EXTERNAL] Re: Centos 8 module updates - how are people dealing with this?

Hi Simon,

On 2021-06-09 14:49, Simon Avery wrote:

...

Hello,

The recent Centos 8.4 release appears to have triggered more issues for us, and made me review how we manage our Centos 8 clients. We plan to move to Rocky soon but the same issue will persist there.

Currently: Uyuni syncs Centos repos. My workaround so far has been to run "dnf -y update" on a schedule to each client instead of patching from within Uyuni as we do with Centos7. This pulls packages from the Uyuni mirror and has largely worked okay, but not any longer - lots of module related issues on all C8 clients since I updated the repos to 8.4. I'm not clear exactly why this has triggered this problem re-appearing, but it has. My understanding of this is that Uyuni doesn't update the module metadata when it populates its repositories, so the clients can't see this and fail.

We as Uyuni team recommend consuming AppStream repositories through the Content Lifecycle Management process so that you can benefit from all the features that SUMA UI provides. However, I understand that the method you described above can be preferred for smaller environments as it is more agile and direct. For that reason, we'd like to keep this as a valid option for those who prefer this approach. Basically, I'd like to see this working in the upcoming updates of any modular repository as well. So, please feel free to open an issue on GitHub, describing the actual problem you're having in detail, and we can work towards fixing it.

...

I've read about the Uyuni method of using the Content Lifecycle and have trialled this. This does work, but we don' t particularly want to be manually doing this for each Centos update or patch cycle. We're not big enough to warrant a corporate approval cycle for Centos, so updates are directly applied to our servers. (This has resulted in relatively few issues)

The only other method I can think of is to change the clients to having local .repo files and pull updates directly from the Centos mirror. This obviously negates some of the benefits Uyuni brings to package management, but would work reliably.

So I'm wondering - how are the other Centos/Alma/Oracle/Rocky 8 users of Uyuni applying updates, and how have you overcome the module problems?

Simon Avery Linux Systems Administrator

Cheers, Can -- -- Can Bulut Bayburt <cbbayburt@suse.de<mailto:cbbayburt@suse.de>> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- Regards

Michael Calmer

-------------------------------------------------------------------- - ----- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> -------------------------------------------------------------------- - ----- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- Regards

Michael Calmer

--------------------------------------------------------------------- - ---- Michael Calmer SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: Michael.Calmer@suse.com<mailto:Michael.Calmer@suse.com> --------------------------------------------------------------------- - ---- SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer (HRB 36809, AG Nürnberg)

-- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D

SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany

-- -- Can Bulut Bayburt <cbbayburt@suse.de> Software Developer, SUSE Manager, R&D SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany