Am 29.03.24 um 20:53 schrieb Frank Krüger via openSUSE Factory:

Am 29.03.24 um 18:20 schrieb Ana Guerrero Lopez via openSUSE Factory:

...

Hi,

If you're using an up-to-date Tumbleweed, please make sure to update as soon as possible your system.

The latest versions of "xz" (5.6.0 and 5.6.1) contained malicious code ( refer to CVE-2024-3094 ) and the package in Tumbleweed has been reverted back to version 5.4.

After reading this mail, please update your system and ensure you're downgrading xz to the version *5.6.1.revertto5.4. *This version despite**itsname is version 5.4. Last step is reboot your system.

Hopefully we'll have soon more detailed information about this CVE.

Have a nice weekend!

Ana from the openSUSE release team.

FYI: https://lwn.net/Articles/967180/

Regards, Frank As for openSUSE, see https://news.opensuse.org/2024/03/29/xz-backdoor/

Regards, Frank

Michal Suchánek composed on 2024-03-29 23:39 (UTC+0100):

On Fri, Mar 29, 2024 at 06:20:27PM +0100, Ana Guerrero Lopez wrote:

...

If you're using an up-to-date Tumbleweed, please make sure to update as soon as possible your system.

...

The latest versions of "xz" (5.6.0 and 5.6.1) contained malicious code ( refer to CVE-2024-3094 ) and the package in Tumbleweed has been reverted back to version 5.4.

...

After reading this mail, please update your system and ensure you're downgrading xz to the version *5.6.1.revertto5.4. *This version despite**itsname is version 5.4. Last step is reboot your system.

...

Hopefully we'll have soon more detailed information about this CVE.

Somewhat useful information seems to be:

https://www.openwall.com/lists/oss-security/2024/03/29/4 https://boehs.org/node/everything-i-know-about-the-xz-backdoor

Current installed Slowroll xz rpm comes from xz source package 5.4.6-1.2. Is any current Slowroll admin action required or to be avoided? -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata

On 2024-03-29 23:39, Michal Suchánek wrote:

Hello,

On Fri, Mar 29, 2024 at 06:20:27PM +0100, Ana Guerrero Lopez via openSUSE Factory wrote:

...

Hopefully we'll have soon more detailed information about this CVE.

Somewhat useful information seems to be:

https://www.openwall.com/lists/oss-security/2024/03/29/4 https://boehs.org/node/everything-i-know-about-the-xz-backdoor

https://lcamtuf.substack.com/p/technologist-vs-spy-the-xz-backdoor *Technologist vs spy: the xz backdoor debate* lcamtuf’s thing -- Cheers / Saludos, Carlos E. R. (from 15.5 x86_64 at Telcontar)

* On 3/31/24 19:03, yassadmi via openSUSE Factory wrote:

The youtuber Brodie Robertson made this statement : "the malicious code end up on the distros because they build from the release tarballs instead of the git repo"

I guess they are reasons to do so, but i thought hes point had to be share here.

[TLDR: using only release tarballs as a malware distribution vector was laziness; release tarballs are a useful, widely used practical means of source code distribution; not using them won't help against adversaries hiding malware in a more refined way directly in repositories] I'm not a famous YouTuber, but I'd still like to give some insights with my release manager hat on. It's true that in this instance, the code activating the malicious code parts has only been distributed in release tarballs. Other methods of fetching the source code ship the malicious code itself, but since it's just part of test data, it's never actually embedded into the binaries and pretty much discarded after a successful build. I've now read numerous times that the practice of using release tarballs is to blame for all the mess that was caused, or, rather, that not using release tarballs would always lead to less risk, but that's simply not true. First of all, let me explain why release tarballs actually are a good thing and by no means evil in themselves. Historically, before (distributed) version control systems existed, packaging was the only meaningful way of distributing source code. This packaging need not be tarballs, even plain files on a diskette can be regarded as a package. However, tarballing (or cpioing or whatever nom-du-jour archiver is being used) the source makes it easy to compress the content - and compression is very effective for plain text data, which is primarily what source code is. Even with distributed version control systems now being the norm, release tarballs are still a very *practical* means of distributing source code. For small repositories and fast network connections, just downloading the whole repository is not a practical issue, but for huge repositories (think of Firefox, Chromium, wine, Qt) it very much still is, since the repository size is magnitudes of orders higher than tarballs for a specific release version. As a user, I'd always pick a release tarball of Chromium instead of having to download tens if not hundreds of gigabytes of repository data. Now, most VCS support a feature typically called "shallow clone", in which only data relevant to a specific revision can be requested and downloaded. Since data is typically compressed before being transmitted, one could argue that fetching data for a tag using this method is practically equivalent to fetching release tarballs. However, this fails to take the distributor's side into consideration. Release tarballs are generated once and never touched again, but transferred many, many, many times after their creation. A shallow clone creates a non-negligible system load on the server side for every such request. Technically, it would be possible to implement some form of caching to circumvent this computational effort, but so far, no VCS has done so. Also, ironically, the whole idea sounds a lot like... creating release tarballs which are then distributed... Then, there's the crucial question of how to verify the integrity of an archive - whether release tarballs or source code checkouts. There are two integrity issues - the first one is whether the data itself is not damaged but what the creator intended to distribute and the other one is whether the data contained corresponds to the data has been made publicly available in the repository. The first issue is easy to check for using a combination of checksums and digital signatures, the latter one is much more complicated, though technically also realizable. However, nobody practically does that, because in virtually all cases, it's a waste of time (and, also, will often fail! More on that soon) and system resources, either on the developer's or distributor's machine(s). In the case of xz, the build system modifications have been sneaked into the release tarball by an adversary, but that was merely done for practical reason. Obfuscating this code (and splitting it up/sneaking parts in via harmless commits as "garbage changes" or the like) and committing it to the source code repository would have been possible, but would also have required a lot more sophistication than dropping a single file into the release tarballs. Unfortunately, the dark side™ will also learn from this incident and probably be smarter about this next time to avoid detection (and to make the malware more readily available through any means of fetching the source code). There is another practical reason why release tarballs are used so widely - some (if not most) build systems are configured in a high(ish)-level fashion by developers, but need another step of parsing/generation to actually get executable scripts and other data for builds to be doable. This data changes very often (if not inevitably always) during each generator run and is typically not committed to the source code repository, because it's just unnecessary noise. As a courtesy to users, maintainers run this intermediary step on their machines while generating a release tarball, so that users don't have to install a compatible build system version (which can be an actual problem if the user's system is either much newer or much older than what the developers were using during the time the release was done) and bootstrap the build system. Xz is using a build system needing this intermediary step - GNU autotools. It also offers CMake as an alternative, which also requires an intermediary step, but this can't be reasonably cached, since it's an amalgamation of autootool's auto(re)conf and ./configure run. Crucially, this means that release tarballs will practically ALWAYS differ from what a source code repository contains, at least for software relying on GNU autotools. Common files won't differ, but there will be new, autogenerated files. That new content is typically very verbose, big and nobody actually checks it manually. While one could advocate not use GNU autotools for reasons like these, in the end, users and packagers will have to do with what the developers provide in their project, short of reimplementing the whole build system for any project that is not using their "favorite" build system (and then, that would only shift one possible failure point to end users/packagers). Then, there's the general issue of developers and maintainers not signing commits and - worst of all - release tags. While the revision CAN be some form of checksum for the eventual data, there is no way to make sure that a specific revision was actually meant to be released other than using a digital signature. Until digital signatures are mandatory for release tags, fetching source code from repositories just shifts the failure/trust point to the party hosting the source code. Lastly, release tarballs are the only practical means of distributing source code to satisfy strong copyleft license terms. I want to point out that it's by no means the *only* way to comply with the terms, but it's a very convenient and practical one. AFAIK, no free software distribution uses other ways. The upstream release tarballs might be getting repacked from time to time, but generally, no distribution "just" provides source code repositories for users to fetch source code from, also because that's not easily archivable, doesn't scale well and because it's just too easy to miss creating a tag or importing a specific source version to the distribution-specific repository and inadvertently breach the GPL. Note, that SRPMs essentially are likewise just glorified/tuned release tarballs. I hope I was able to shed a different light on what has been demonized needlessly and often the past few days. Also, if you have read to this point and wasn't able to obtain new information: congratulations, reading the TLDR on top would have been good enough. Mihai

Hello Frank and all, On 2024-03-30 T 10:28 +0100 Frank Krüger via openSUSE Factory wrote:

Am 29.03.24 um 18:20 schrieb Ana Guerrero Lopez via openSUSE Factory:

...

Hi,

If you're using an up-to-date Tumbleweed, please make sure to update as soon as possible your system.

The latest versions of "xz" (5.6.0 and 5.6.1) contained malicious code ( refer to CVE-2024-3094 ) and the package in Tumbleweed has been reverted back to version 5.4.

After reading this mail, please update your system and ensure you're downgrading xz to the version *5.6.1.revertto5.4. *This version despite**itsname is version 5.4. Last step is reboot your system.

Hopefully we'll have soon more detailed information about this CVE.

Have a nice weekend!

Ana from the openSUSE release team.

Thank you. According to the discussion at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024, in particular regarding the person who might have backdoored xz and their contributions in the past, there is a suggestion to revert to version 5.3.1 for the time being. Are there similar considerations at openSUSE? Thx.

according to my information, at this point in time the revert to 5.4 is sufficient, and a revert back to 5.3.1 not needed. Please do not forget to reboot after installing the downgraded packages. So long - MgE -- Matthias G. Eckermann, Head of Products and Partners SUSE Software Solutions Germany GmbH, Frankenstr. 146, 90461 Nuernberg (HRB 36809, AG Nürnberg) GF: Ivo Totev, Andrew McDonald, Werner Knoblich

Am 02.04.24 um 19:19 schrieb Andrei Borzenkov:

On 29.03.2024 20:20, Ana Guerrero Lopez via openSUSE Factory wrote:

...

Hi,

If you're using an up-to-date Tumbleweed, please make sure to update as soon as possible your system.

The latest versions of "xz" (5.6.0 and 5.6.1) contained malicious code ( refer to CVE-2024-3094 ) and the package in Tumbleweed has been reverted back to version 5.4.

After reading this mail, please update your system and ensure you're downgrading xz to the version *5.6.1.revertto5.4. *This version despite**itsname is version 5.4. Last step is reboot your system.

While providing patch on Tumbleweed for those users who are not aware they should not use YaST Online Update or similar on Tumbleweed is certainly very commendable, the way this patch was provided leaves something to desire. Users are greeted with strange patch with the name "reboot-really-needed" with the description "Critical update for openSUSE Tumbleweed" and "Please reboot your system NOW!". The text has no reference to xz or CVE and the whole looks like malware itself.

https://forums.opensuse.org/t/reboot-really-needed-unknown-author/173671

While it is probably too late now, may be next time such emergency patch can be presented better without scaring users.

Gah, those condescending self-proclaimed "experts" non-answering to a legitimate question in that thread are annoying. On the same or worse level than that generic reboot package.

On Wed, Apr 3, 2024 at 9:59 AM Dirk Müller wrote:

Hi Andrei,

Am Di., 2. Apr. 2024 um 19:20 Uhr schrieb Andrei Borzenkov :

...

...

After reading this mail, please update your system and ensure you're downgrading xz to the version *5.6.1.revertto5.4. *This version despite**itsname is version 5.4. Last step is reboot your system. While providing patch on Tumbleweed for those users who are not aware they should not use YaST Online Update or similar on Tumbleweed is certainly very commendable, the way this patch was provided leaves something to desire. Users are greeted with strange patch with the name "reboot-really-needed" with the description "Critical update for openSUSE Tumbleweed" and "Please reboot your system NOW!". The text has no reference to xz or CVE and the whole looks like malware itself.

This is the test update, it is unrelated to the system upgrade that can be done via zypper dup. I agree we should probably not have it in the repo, but it's good to be able to test maintenance updates (but this path isn't used in TW). The description is generic because it is unrelated to the xz backdoor

Are we talking about the same thing? andrei@tumbleweed:~> zypper info -t patch reboot-really-needed Repository 'openSUSE-Tumbleweed-Non-Oss (20240325)' is out-of-date. You can run 'zypper refresh' as root to update it. Repository 'openSUSE-Tumbleweed-Oss (20240325)' is out-of-date. You can run 'zypper refresh' as root to update it. Repository 'openSUSE-Tumbleweed-Update' is out-of-date. You can run 'zypper refresh' as root to update it. Loading repository data... Reading installed packages... Information for patch reboot-really-needed: ------------------------------------------- Repository : openSUSE-Tumbleweed-Update Name : reboot-really-needed Version : 1 Arch : noarch Vendor : adrianSuSE Status : needed Category : security Severity : critical Created On : Fri 29 Mar 2024 03:01:16 PM MSK Interactive : reboot Summary : Critical update for openSUSE Tumbleweed Description : Please reboot your system NOW! Provides : patch:reboot-really-needed = 1 Conflicts : [3] liblzma5.x86_64 < 5.6.1.revertto5.4-3.2 liblzma5.noarch < 5.6.1.revertto5.4-3.2 liblzma5.i586 < 5.6.1.revertto5.4-3.2 andrei@tumbleweed:~>

On 4/3/24 1:35 PM, Dominique Leuenberger wrote:

On Wed, 2024-04-03 at 19:03 +0200, Bernhard Voelker wrote:

...

On 3/29/24 18:20, Ana Guerrero Lopez via openSUSE Factory wrote:

...

The latest versions of "xz" (5.6.0 and 5.6.1) contained malicious code ( refer to CVE-2024-3094 ) and the package in Tumbleweed has been reverted back to version 5.4.

After the big update of 5000+ packages yesterday, is there a speicific reason for this additional downgrade today?

The following 3 packages are going to be downgraded:    liblzma5            5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1    liblzma5-x86-64-v3  5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1    xz                  5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1

The overlay in the :Update channel has been removed, as the main channel has fully caught up.

The :Update is always one check-in count ahead.

Cheers, Dominique

Does this mean there is still a backdoor issue with xz 5.4-3.2 since it is being downgraded again to 5.4-2.1 ? -- Regards, Joe

On 4/3/24 2:56 PM, John Paul Adrian Glaubitz wrote:

On Wed, 2024-04-03 at 14:53 -0400, Joe Salmeri wrote:

...

Does this mean there is still a backdoor issue with xz 5.4-3.2 since it is being downgraded again to 5.4-2.1 ?

No, the backdoor was introduced in 5.6.0 and improved in 5.6.1.

See this excellent write-up by Sam James from Gentoo [1] for more.

Adrian

...

[1] https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27

Thanks Adrian, I have read that write up and many others. The update repo downgraded xz to 5.4-3.2 ( because TW repos had previously installed 5.6.1 ) and build 20240329 also reverted to 5.4-3.2. What I'm trying to understand is why is it downgrading xz again to 5.4-2.1 ? -- Regards, Joe

On 4/3/24 3:46 PM, John Paul Adrian Glaubitz wrote:

On Wed, 2024-04-03 at 15:01 -0400, Joe Salmeri wrote:

...

The update repo downgraded xz to 5.4-3.2 ( because TW repos had previously installed 5.6.1 ) and build 20240329 also reverted to 5.4-3.2.

What I'm trying to understand is why is it downgrading xz again to 5.4-2.1 ?

This was explained earlier in the thread. Version 5.4-3.2 came from the update repository while 5.4-2.1 is the version that was merged into the main repo.

After the merge, the version from the update repo was removed and hence zypper performed a downgrade.

Adrian

Yes, I saw that and I updated my systems using the update repo the other day. What I am trying to understand is why was an even earlier version of xz ( 5.4-2.1 ) merged into the main repo instead of using the same version ( 5.4-3.2 ) of xz that was put into the update repo ? What was the reason for falling back to an even older version ? It's not just a version number difference because the 5.4-2.1 version is 13% smaller in size than the 5.4-3.2 version. -rwxr-xr-x 1 root root 100K Apr 3 15:56 /.snapshots/6/snapshot/usr/bin/xz -rwxr-xr-x 1 root root 87K Mar 28 09:51 /.snapshots/7/snapshot/usr/bin/xz -- Regards, Joe

On 04. 04. 24, 9:37, Dominique Leuenberger wrote:

Not really an issue - but I corrected :Update prj meta data to split debuginfo in the future too. This shold then help eliminate further such confusions.

Ah. Will there be then http://download.opensuse.org/debug/update/tumbleweed or alike? thanks, -- js suse labs

Hi Dominique, On Thu, 2024-04-04 at 09:37 +0200, Dominique Leuenberger wrote:

On Wed, 2024-04-03 at 16:17 -0400, Joe Salmeri wrote:

...

Yes, I saw that and I updated my systems using the update repo the other day.

What I am trying to understand is why was an even earlier version of xz ( 5.4-2.1 ) merged into the main repo instead of using the same version ( 5.4-3.2 ) of xz that was put into the update repo ?

the VERSION is the same. The RPM names are in the form NVR - Name- Version-Release.

In this specicic case: Name = xz Version = 5.6.1.revertto5.4 Release = 2.1

The release in turn is split in two pieces: CICount.RebuildCount

openSUSE:Factory:Update is an overlay over openSUSE:Factory and the source from :Factory is 'branched' into :Update - which implies it is 'one commit on OBS' later than the version in :Factory (hence 2 -> 3 change). In both cases, the package has built only once, so they were offered a .1 suffix.

...

What was the reason for falling back to an even older version ?

The :Update was removed again (as all code is in the main tree) - and thus you 'fall back' to the version that was in Factory.

...

It's not just a version number difference because the 5.4-2.1 version is 13% smaller in size than the 5.4-3.2 version.

-rwxr-xr-x 1 root root 100K Apr  3 15:56 /.snapshots/6/snapshot/usr/bin/xz -rwxr-xr-x 1 root root  87K Mar 28 09:51 /.snapshots/7/snapshot/usr/bin/xz

Observant you are! Well spotted.

The difference is that openSUSE:Factory:Update does not split out - debuginfo packages - so the binaries are not stripped.

Not really an issue - but I corrected :Update prj meta data to split debuginfo in the future too. This shold then help eliminate further such confusions.

Thanks a lot for the elaborate and detailed explanation. Out of curiosity: Why does Factory actually have an Update repository? Is it to be able to provide expedited updates for Tumbleweed necessary for occasions like the xz backdoor without having to wait for a new Tumbleweed snapshot? Adrian

On Thu, 2024-04-04 at 10:10 +0200, Dominique Leuenberger wrote:

The :Update repository was a learning back in the times of a bash CVE. This happened to strike us back then at a time where Factory was in a larger overhaul and we did not get it through QA for quite a while and we needed a way to still pish out fixes for such critical CVEs at that time.

The channel is very rarely used - usually we have a 24 hour delay between snapshots. Things that blow up badly go out quicker (keep in mind: no QA on the Update channel - so we use it really REALLY seldom).

Makes a lot of sense. Thanks! Adrian

Hello, On 2024-04-03 19:03, Bernhard Voelker wrote:

After the big update of 5000+ packages yesterday, is there a speicific reason for this additional downgrade today?

The following 3 packages are going to be downgraded:   liblzma5            5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1   liblzma5-x86-64-v3  5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1   xz                  5.6.1.revertto5.4-3.2 -> 5.6.1.revertto5.4-2.1

The xz update was removed from the :Update project: $ osc log -D openSUSE:Factory:Update/xz | head -n5 ---------------------------------------------------------------------------- r2 | dimstar_suse | 2024-04-03 09:47:46 | 2a890d120d2d01e097774f63fee18178 | unknown | Merged in main tree Andreas

Op donderdag 4 april 2024 16:53:05 CEST schreef Fritz Hudnut:

This was today's Manjaro upgrade, seemingly to deal with the xz problem:

[CODE] :: Starting full system upgrade... resolving dependencies... looking for conflicting packages...

Packages (4) lib32-xz-5.6.1-2 manjaro-release-23.1.4-1 mhwd-nvidia-550.67-1 xz-5.6.1-2

Total Download Size: 0.78 MiB Total Installed Size: 2.69 MiB Net Upgrade Size: 0.03 MiB [/CODE]

Will there be more to come, or on to other things?? Took a second or so to run through. How is this related to openSUSE TW? FWIW BBcode does not work in emails.

-- Gertjan Lettink a.k.a. Knurpht openSUSE Board openSUSE Forums Team

I thought that would be "obvious" . . . the problem . . . and the response to the problem . . . in regards to efficiency, etc. It only shows that Manjaro did not yet downgrade and is still vulnerable. And

Op donderdag 4 april 2024 18:50:35 CEST schreef Fritz Hudnut: that is irrelevant here. -- Gertjan Lettink a.k.a. Knurpht openSUSE Board openSUSE Forums Team

On 04.04.2024 21:47, Ben Greiner wrote:

Am 04.04.24 um 18:58 schrieb Knurpht-openSUSE:

...

Op donderdag 4 april 2024 18:50:35 CEST schreef Fritz Hudnut:

...

I thought that would be "obvious" . . . the problem . . . and the response to the problem . . . in regards to efficiency, etc. It only shows that Manjaro did not yet downgrade and is still vulnerable.

It only shows that the Archlinux/Manjaro Maintainers are less than knowledgeable about their packages. Inspite if not building rpm or debian packages they claim to have "fixed" the backdoor while going from 5.6.1-1 to 5.6.2-2 [1].

According to the available information, backdoor was injected by code in the release tarball which was not present in the git. Arch switched from using release tarball to using git: https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/commit/881385...

The disassembly of liblzma didn't even change between those package versions.

You mean you built both versions and they were identical?

Am 4. April 2024 18:58:43 MESZ schrieb Knurpht-openSUSE :

Op donderdag 4 april 2024 18:50:35 CEST schreef Fritz Hudnut:

...

I thought that would be "obvious" . . . the problem . . . and the response to the problem . . . in regards to efficiency, etc. It only shows that Manjaro did not yet downgrade and is still vulnerable.

Absolutely wrong. Regards Eric

Fritz Hudnut wrote:

So, I appreciate the "scalpel-like" efforts of Manjaro to "keep it tidy" . . . . Whether or not it actually addresses or doesn't address it, or didn't need to address it . . . is less important to this end user . . . .

Sure, ignorance is bliss, I guess. Can we please stop going on and on about nothing on this thread? There are other openSUSE mailing lists where general discussions, like this has reduced to, belong. Thank you -- Atri