Bug ID 1191929
Summary NVMe SSD slow since openSUSE-LEAP-15.3 (especially with cryptsetup / dm-crypt device)
Classification openSUSE
Product openSUSE Distribution
Version Leap 15.3
Hardware x86-64
OS openSUSE Leap 15.3
Status NEW
Severity Normal
Priority P5 - None
Component Kernel
Assignee kernel-bugs@opensuse.org
Reporter duge@pre-sense.de
QA Contact qa-bugs@suse.de
Found By ---
Blocker ---

I got a system with a NVMe SSD, whose performance with dm-crypt dropped from >
200 MB/s to below 2 MB/s from openSUSE-15.2 to openSUSE-15.3.




Test procedure:
dd oflag=direct bs="$((64 * 1024 * 1024))" count="$((16 * 1024 / 64))"
if=dev/zero of=/dev/nvme0n1
dd oflag=sync   bs="$((64 * 1024 * 1024))" count="$((16 * 1024 / 64))"
if=dev/zero of=/dev/nvme0n1
cryptsetup  --cipher "aes-cbc-essiv:sha256" create crypt_test /dev/nvme0n1
dd oflag=direct bs="$((64 * 1024 * 1024))" count="$((16 * 1024 / 64))"
if=dev/zero of=/dev/mapper/crypt_test
dd oflag=sync   bs="$((64 * 1024 * 1024))" count="$((16 * 1024 / 64))"
if=dev/zero of=/dev/mapper/crypt_test

For oflag=direct openSUSE 15.2 and 15.3 both write with over 800 MB/s to
/dev/nvme0n1 and over 300 MB/s to /dev/mapper/crypt_test.

openSUSE-15.2 will write with over 600 MB/s to /dev/nvme0n1 and over 200 MB/s
to /dev/mapper/crypt_test

openSUSE-15.3 will write with 100 MB/s to /dev/nvme0n1 and less than 2 MB/s to
/dev/mapper/crypt_test.

Finally an "mkfs.ext4 /dev/mapper/crypt_test" takes more than 20 minutes and
"iotop" shows even less I/O than 1 MB/s!

Tumbleweed-x86_64-Snapshot20211019 (Linux-5.14.11) performs like openSUSE-15.2
or even better.




localhost:~ # grep PRETTY /etc/os-release 
PRETTY_NAME="openSUSE Leap 15.3"
localhost:~ # lsblk /dev/nvme0n1
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
nvme0n1     259:0    0 238,5G  0 disk
localhost:~ # lspci -v -d 1344:5410
04:00.0 Non-Volatile memory controller: Micron Technology Inc Device 5410 (rev
01) (prog-if 02 [NVM Express])
        Subsystem: Micron Technology Inc Device 0100
        Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
        Memory at a2100000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [80] Express Endpoint, MSI 00
        Capabilities: [d0] MSI-X: Enable+ Count=32 Masked-
        Capabilities: [e0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [f8] Power Management version 3
        Capabilities: [100] Vendor Specific Information: ID=1556 Rev=1 Len=008
<?>
        Capabilities: [108] Latency Tolerance Reporting
        Capabilities: [110] L1 PM Substates
        Capabilities: [200] Advanced Error Reporting
        Capabilities: [300] #19
        Kernel driver in use: nvme
        Kernel modules: nvme
localhost:~ # smartctl -a /dev/nvme0n1 
[...]
=== START OF INFORMATION SECTION ===
Model Number:                       Micron_2200_MTFDHBA256TCK
[...]
Firmware Version:                   P1MU003
PCI Vendor/Subsystem ID:            0x1344
IEEE OUI Identifier:                0x00a075
Controller ID:                      0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          256.060.514.304 [256 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            00a075 012cd592d5
[...]


You are receiving this mail because: