Dear openSUSE factory maintainers, I already created a comment on OBS in the Archiving:unzip package ( https://build.opensuse.org/package/show/Archiving/unzip), which I guess is the base for factory:unzip. But ... I'm writing here again because I can't even begin to imagine how many systems this bug has hit. How many installations were broken by this. How many deployment pipelines (like in our case) or manual app installations (like in our case) were broken again and again by this unzip bug. Summary for the bug: When you extract a zip archive where the files do not have explicit permissions set, around one in 10,000 files gets (reproducibly) converted into a symlink! The former content of the file then gets converted into the link target. This bug is fixed in the debian/ubuntu version of unzip so my guess is that it's fixed by one of the missing CVE patches. Usually I wouldn't be this alarmist but the basic and standard unzip program reproducibly and consistently breaking files on *all* Tumbleweed as well *all* Leap installations is pretty important I guess. kind regards, Kira Backes
Hello Kira! On 9/7/21 12:22, Kira M. Backes wrote:
But ... I'm writing here again because I can't even begin to imagine how many systems this bug has hit. How many installations were broken by this. How many deployment pipelines (like in our case) or manual app installations (like in our case) were broken again and again by this unzip bug.
Summary for the bug: When you extract a zip archive where the files do not have explicit permissions set, around one in 10,000 files gets (reproducibly) converted into a symlink! The former content of the file then gets converted into the link target. This bug is fixed in the debian/ubuntu version of unzip so my guess is that it's fixed by one of the missing CVE patches.
Do you have a reproducer for this bug, i.e. a sample ZIP archive which demonstrates the bug? Thanks, Adrian
Hello Adrian,
Do you have a reproducer for this bug, i.e. a sample ZIP archive which demonstrates the bug?
yes, you can for example download the latest zipped commit from shopware/platform: https://github.com/shopware/platform/archive/dd2bf30d0519dcd9416dc269c88edcf... When you unzip it with openSUSE's unzip in bash the file platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-redo.svg gets erroneously converted into a symlink. It was reproducible with the same file on Leap 15.3 and Tumbleweed and two different AMD CPUs, so I guess the same file should become a symlink for you. In the zip file are only two real symlinks: platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/bin/console -> shopware platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/easy-coding-standard.php -> ecs.php So any other file becoming a symlink is caused by this bug. This bug happens with nearly every zipped commit (because it has enough files) so you could for example just take an older commit: https://github.com/shopware/platform/archive/5c5f5d9b0d502dc104eee89456d1b0b... and when you unzip it files get converted to symlinks again In the last zip the file which gets converted into a symlink is: platform-5c5f5d9b0d502dc104eee89456d1b0b766a1cac2/src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-undo 2.svg kind regards, Kira
On 9/7/21 20:02, Kira M. Backes wrote:
Hello Adrian,
Do you have a reproducer for this bug, i.e. a sample ZIP archive which demonstrates the bug?
yes, you can for example download the latest zipped commit from shopware/platform:
https://github.com/shopware/platform/archive/dd2bf30d0519dcd9416dc269c88edcf... <https://github.com/shopware/platform/archive/dd2bf30d0519dcd9416dc269c88edcfd66f92add.zip>
When you unzip it with openSUSE's unzip in bash the file
platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-redo.svg
gets erroneously converted into a symlink.
It was reproducible with the same file on Leap 15.3 and Tumbleweed and two different AMD CPUs, so I guess the same file should become a symlink for you.
In the zip file are only two real symlinks:
platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/bin/console -> shopware platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/easy-coding-standard.php -> ecs.php
So any other file becoming a symlink is caused by this bug.
This bug happens with nearly every zipped commit (because it has enough files) so you could for example just take an older commit:
https://github.com/shopware/platform/archive/5c5f5d9b0d502dc104eee89456d1b0b... <https://github.com/shopware/platform/archive/5c5f5d9b0d502dc104eee89456d1b0b766a1cac2.zip>
and when you unzip it files get converted to symlinks again
In the last zip the file which gets converted into a symlink is:
platform-5c5f5d9b0d502dc104eee89456d1b0b766a1cac2/src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-undo 2.svg
Please file a bug report at bugzilla.opensuse.org with this information, that will mean it will likely reach the maintainers quicker. Thanks -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hi! On 9/7/21 12:32, Kira M. Backes wrote:
When you unzip it with openSUSE's unzip in bash the file
platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add/src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-redo.svg
gets erroneously converted into a symlink.
I can reproduce this issue. Here is my result on openSUSE Tumbleweed: glaubitz@suse-laptop:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> find . -type l ./bin/console ./src/Storefront/Resources/app/storefront/dist/assets/icon/default/editor-redo.svg ./easy-coding-standard.php glaubitz@suse-laptop:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> and here on Debian Bullseye: glaubitz@z6:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> find . -type l ./easy-coding-standard.php ./bin/console glaubitz@z6:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> Looking through the changelog of the Debian package [1], it seems we're talking about this bug [2] which is fixed by this patch [3] by Andreas Schwab. I don't see this patch in the openSUSE package [4]. Adrian
[1] http://metadata.ftp-master.debian.org/changelogs/main/u/unzip/unzip_6.0-26_c... [2] http://bugs.debian.org/717029 [3] https://sources.debian.org/src/unzip/6.0-26/debian/patches/06-initialize-the... [4] https://build.opensuse.org/package/show/Archiving/unzip
Hello! On 9/7/21 12:56, John Paul Adrian Glaubitz wrote:
Looking through the changelog of the Debian package [1], it seems we're talking about this bug [2] which is fixed by this patch [3] by Andreas Schwab. I don't see this patch in the openSUSE package [4].
I can confirm that this patch by Andreas fixes the issue for me: glaubitz@suse-laptop:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> find . -type l ./bin/console ./easy-coding-standard.php glaubitz@suse-laptop:/tmp/platform-dd2bf30d0519dcd9416dc269c88edcfd66f92add> I have uploaded the package for testing here [1]. Adrian
[1] https://build.opensuse.org/package/show/home:glaubitz:branches:Archiving/unz...
On Tue, Sep 7, 2021 at 7:22 AM Kira M. Backes <kira.backes@nrwsoft.de> wrote: Kira: I strongly suggest not to use this tool, on any distribution. Please stick to bsdtar instead. it handles most if not all compression formats. You could also write a very tiny (and memory safe) go program if you need only a decompressor like : https://github.com/artdarek/go-unzip
On Tue, Sep 7, 2021 at 7:22 AM Kira M. Backes <kira.backes@nrwsoft.de> wrote:
Dear openSUSE factory maintainers,
I already created a comment on OBS in the Archiving:unzip package (https://build.opensuse.org/package/show/Archiving/unzip), which I guess is the base for factory:unzip.
But ... I'm writing here again because I can't even begin to imagine how many systems this bug has hit. Ho
Kira: I strongly suggest not to use this tool, on any distribution. Please stick to bsdtar instead. it handles most if not all compression formats. You could also write a very tiny (and memory safe) go program if you need only a decompressor like : https://github.com/artdarek/go-unzip This is specially advisable if you are dealing with untrusted data.
On 9/7/21 16:50, Cristian Rodríguez wrote:
Kira:
I strongly suggest not to use this tool, on any distribution. Please stick to bsdtar instead. it handles most if not all compression formats.
Well, ZIP archives exist in the wild as in this example from Github, so it's not really an option to just ignore this bug. Adrian
I strongly suggest not to use this tool, on any distribution
This bug came up because composer (yes, the practically officially PHP package manager) uses unzip when it's available! You can imagine the havoc this bug caused on our dev environments when random PHP files were kind-of deleted randomly on different shopware versions...
On Tue, Sep 7, 2021 at 12:23 PM Kira M. Backes <kira.backes@nrwsoft.de> wrote:
I strongly suggest not to use this tool, on any distribution
This bug came up because composer (yes, the practically officially PHP package manager) uses unzip when it's available!
You can imagine the havoc this bug caused on our dev environments when random PHP files were kind-of deleted randomly on different shopware versions...
huh...remember to install php-zip then ..or is it still buggy ?
On Tue 7. Sep 2021 at 17:34 Cristian Rodríguez <crrodriguez@opensuse.org> wrote:
huh...remember to install php-zip then ..or is it still buggy ?
Composer officially prefers unzip when it’s installed: https://github.com/composer/composer/blob/4bcd860b6574fd71553019296965613cba... -- mit freundlichen Grüßen, Kira M. Backes -- NRWsoft Inh. Kira M. Backes Siemensstr. 7 in 42697 Solingen USt-IdNr. DE275523769 Tel: 0212 240 910 70
participants (5)
-
Cristian Rodríguez -
Cristian Rodríguez -
John Paul Adrian Glaubitz -
Kira M. Backes -
Simon Lees