openSUSE Commits

Download

commit@lists.opensuse.org

December 2020

1 participants
2154 discussions

commit lvm2.15198 for openSUSE:Leap:15.1:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package lvm2.15198 for openSUSE:Leap:15.1:Update checked in at 2020-12-01 11:34:03 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/lvm2.15198 (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "lvm2.15198" Tue Dec 1 11:34:03 2020 rev:1 rq:851970 version:2.02.180 Changes: -------- New Changes file: --- /dev/null 2020-11-18 17:46:03.679371574 +0100 +++ /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/device-mapper.changes 2020-12-01 11:34:07.907029452 +0100 @@ -0,0 +1,2361 @@ +------------------------------------------------------------------- +Mon Nov 9 14:02:17 UTC 2020 - Marcos de Souza <mpdesouza(a)suse.com> + +- update patch according to systemd version/behaviour changed (bsc#1123327) + - bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch + +------------------------------------------------------------------- +Fri Aug 21 01:52:00 UTC 2020 - heming.zhao(a)suse.com + +- lvm scan: Too many open files (bsc#1173503) + + bug-1173503_lvmetad-fix-pvs-for-many-devices.patch + +- LVM failing to activate hot spare on surprise removal (bsc#1175110) + + bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch + +- change lvm2.spec source URL + - lvm2.spec + +------------------------------------------------------------------- +Mon Aug 10 12:42:00 UTC 2020 - heming.zhao(a)suse.com + +- add missing patches + systemd lvm2-pvscan@.service StartLimitInterval misplaced (bsc#1123327) + + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + +- modify patch according to changing patch: + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-998893_make_pvscan_service_after_multipathd.patch (bsc#998893) + +- use COMMON-PATCH to manage clvm patch + + lvm2-clvm.spec + - Patch3001: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + + Patch1006: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + +------------------------------------------------------------------- +Wed Jul 22 08:08:00 UTC 2020 - heming.zhao(a)suse.com + +- On shutdown systems hangs for 90 seconds stopping LVM2 metatdata daemon (bsc#1172597) + + bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch + + bug-1172597_2-cleanup-missed-string-specifier.patch + + bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch + + bug-1172597_4-cov-check-for-socket_path-being-set.patch + + bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch + + bug-1172597_6-cov-missing-checks-of-syscalls.patch + + bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch + + bug-1172597_8-daemons-check-for-non-zero-thread_id.patch + +------------------------------------------------------------------- +Tue Feb 26 13:11:35 UTC 2020 - heming.zhao(a)suse.com + +- update patch (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +- update patch (bsc#1145231) + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +- boot takes extremely long time with 400 LUN's (bsc#1158358) + + bug-1158358_bcache-reduce-MAX_IO-to-256.patch + +- lvm2 bcache code enhancement (bsc#1164718) + + bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch + + bug-1164718_02-io-use-sync-io-if-aio-fails.patch + + bug-1164718_03-bcache-sync-io-fixes.patch + + bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch + + bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch + + bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch + + bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch + + bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch + + bug-1164718_09-config-change-scan_lvs-default-to-0.patch + + bug-1164718_10-config-add-new-setting-io_memory_size.patch + + bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch + + bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch + + bug-1164718_13-bcache-Fix-memory-leak.patch + + bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch + + bug-1164718_15-cov-release-iterator-on-error-path.patch + + bug-1164718_16-cov-check-lv_info.patch + + bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch + + bug-1164718_18-cov-validate-pagesize-is-not-negative.patch + + bug-1164718_19-cov-remove-unused-headers.patch + + bug-1164718_20-cov-check-result-of-dev_get_block_size.patch + + bug-1164718_21-gcc-clean-uninitialized-var-warning.patch + + bug-1164718_22-cov-release-iterator-on-error-path.patch + + bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch + + bug-1164718_24-pvscan-avoid-redundant-activation.patch + + bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch + + bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch + + bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch + + bug-1164718_28-lvmcache-free-resource-on-error-path.patch + +- Avoid creation of mixed-blocksize PV on LVM volume groups (LVM2) (bsc#1149408) + + bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch + + bug-1149408_02-tests-allow-mixed-block-sizes.patch + + bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch + + bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch + + bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch + + bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch + +- Use the upstream patches for LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + remove SUSE speical patch: + - bug-1150021_02-suse-special-bcache-bug-fix.patch + add patch: + + bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch + + bug-1150021_03-cov-Fix-a-leak.patch + + bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch + + bug-1150021_05-bcache-add-bcache_abort.patch + + bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch + + bug-1150021_07-bcache-add-unit-test.patch + + bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch + + bug-1150021_09-bcache-reverse-earlier-patch.patch + + bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch + + bug-1150021_11-radix-tree-Add-missing-test-case.patch + + bug-1150021_12-base-Get-Makefile-from-master.patch + + bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch + + bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch + +- Update lvm.conf file + add devices/allow_mixed_block_sizes item + add global/use_aio item + +------------------------------------------------------------------- +Thu Feb 20 10:21:15 UTC 2020 - ghe(a)suse.com + +- Fix heap memory leak in lvmetad. (bsc#1164126) + + bug-1164126_lvmetad-fix-heap-memory-leak.patch + +------------------------------------------------------------------- +Wed Feb 19 03:34:30 UTC 2020 - heming.zhao(a)suse.com + +- lvmetad uses devices/global_filter but not devices/filter after lvm2 update (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +------------------------------------------------------------------- +Mon Dec 23 07:09:00 UTC 2019 - heming.zhao(a)suse.com + +- LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + + bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch + + bug-1150021_02-suse-special-bcache-bug-fix.patch + +------------------------------------------------------------------- +Wed Nov 6 07:29:00 UTC 2019 - heming.zhao(a)suse.com + +- Fix seeing a 90 Second delay on shutdown and reboot (bsc#1155668) + + bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch + +------------------------------------------------------------------- +Wed Aug 21 10:10:30 UTC 2019 - ghe(a)suse.com + +- MD devices should be detected by LVM2 with metadata=1.0/0.9 (bsc#1145231) + + bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch + + bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch + + bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch + + bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch + + bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch + + bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +------------------------------------------------------------------- +Tue Jul 9 10:00:05 UTC 2019 - ghe(a)suse.com + +- Fix unknown feature in status message (bsc#1135984) + + bug-1135984_cache-support-no_discard_passdown.patch + +------------------------------------------------------------------- +Thu Jun 27 02:53:03 UTC 2019 - heming.zhao(a)suse.com + +- Fix using device aliases with lvmetad (bsc#1137296) + + bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch + +------------------------------------------------------------------- +Tue Apr 30 10:20:05 UTC 2019 - ghe(a)suse.com + +- Fix devices drop open error message (bsc#1122666) + + bug-1122666_devices-drop-open-error-message.patch + +------------------------------------------------------------------- +Tue Mar 19 12:02:02 UTC 2019 - Martin Liška <mliska(a)suse.cz> + +- Use %make_build in order to provide verbose output. + +------------------------------------------------------------------- +Fri Feb 1 08:20:15 UTC 2019 - ghe(a)suse.com + +- Disable the LVM lock daemon using sanlock, in order to avoid the + dependence on sanlock related packages (bsc#1121382) + +------------------------------------------------------------------- +Mon Jan 14 14:23:52 CET 2019 - kukuk(a)suse.de + +- Use %license instead of %doc [bsc#1082318] + +------------------------------------------------------------------- +Mon Nov 5 08:10:05 UTC 2018 - ghe(a)suse.com + +- Prevent writing beyond metadata area (bsc#1114113) + + bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch + ++++ 2164 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/device-mapper.changes New Changes file: --- /dev/null 2020-11-18 17:46:03.679371574 +0100 +++ /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/lvm2-clvm.changes 2020-12-01 11:34:08.091029631 +0100 @@ -0,0 +1,2361 @@ +------------------------------------------------------------------- +Mon Nov 9 14:01:51 UTC 2020 - Marcos de Souza <mpdesouza(a)suse.com> + +- update patch according to systemd version/behaviour changed (bsc#1123327) + - bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch + +------------------------------------------------------------------- +Fri Aug 21 01:52:00 UTC 2020 - heming.zhao(a)suse.com + +- lvm scan: Too many open files (bsc#1173503) + + bug-1173503_lvmetad-fix-pvs-for-many-devices.patch + +- LVM failing to activate hot spare on surprise removal (bsc#1175110) + + bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch + +- change lvm2.spec source URL + - lvm2.spec + +------------------------------------------------------------------- +Mon Aug 10 12:42:00 UTC 2020 - heming.zhao(a)suse.com + +- add missing patches + systemd lvm2-pvscan@.service StartLimitInterval misplaced (bsc#1123327) + + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + +- modify patch according to changing patch: + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-998893_make_pvscan_service_after_multipathd.patch (bsc#998893) + +- use COMMON-PATCH to manage clvm patch + + lvm2-clvm.spec + - Patch3001: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + + Patch1006: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + +------------------------------------------------------------------- +Wed Jul 22 08:08:00 UTC 2020 - heming.zhao(a)suse.com + +- On shutdown systems hangs for 90 seconds stopping LVM2 metatdata daemon (bsc#1172597) + + bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch + + bug-1172597_2-cleanup-missed-string-specifier.patch + + bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch + + bug-1172597_4-cov-check-for-socket_path-being-set.patch + + bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch + + bug-1172597_6-cov-missing-checks-of-syscalls.patch + + bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch + + bug-1172597_8-daemons-check-for-non-zero-thread_id.patch + +------------------------------------------------------------------- +Tue Feb 26 13:11:35 UTC 2020 - heming.zhao(a)suse.com + +- update patch (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +- update patch (bsc#1145231) + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +- boot takes extremely long time with 400 LUN's (bsc#1158358) + + bug-1158358_bcache-reduce-MAX_IO-to-256.patch + +- lvm2 bcache code enhancement (bsc#1164718) + + bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch + + bug-1164718_02-io-use-sync-io-if-aio-fails.patch + + bug-1164718_03-bcache-sync-io-fixes.patch + + bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch + + bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch + + bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch + + bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch + + bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch + + bug-1164718_09-config-change-scan_lvs-default-to-0.patch + + bug-1164718_10-config-add-new-setting-io_memory_size.patch + + bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch + + bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch + + bug-1164718_13-bcache-Fix-memory-leak.patch + + bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch + + bug-1164718_15-cov-release-iterator-on-error-path.patch + + bug-1164718_16-cov-check-lv_info.patch + + bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch + + bug-1164718_18-cov-validate-pagesize-is-not-negative.patch + + bug-1164718_19-cov-remove-unused-headers.patch + + bug-1164718_20-cov-check-result-of-dev_get_block_size.patch + + bug-1164718_21-gcc-clean-uninitialized-var-warning.patch + + bug-1164718_22-cov-release-iterator-on-error-path.patch + + bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch + + bug-1164718_24-pvscan-avoid-redundant-activation.patch + + bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch + + bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch + + bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch + + bug-1164718_28-lvmcache-free-resource-on-error-path.patch + +- Avoid creation of mixed-blocksize PV on LVM volume groups (LVM2) (bsc#1149408) + + bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch + + bug-1149408_02-tests-allow-mixed-block-sizes.patch + + bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch + + bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch + + bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch + + bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch + +- Use the upstream patches for LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + remove SUSE speical patch: + - bug-1150021_02-suse-special-bcache-bug-fix.patch + add patch: + + bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch + + bug-1150021_03-cov-Fix-a-leak.patch + + bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch + + bug-1150021_05-bcache-add-bcache_abort.patch + + bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch + + bug-1150021_07-bcache-add-unit-test.patch + + bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch + + bug-1150021_09-bcache-reverse-earlier-patch.patch + + bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch + + bug-1150021_11-radix-tree-Add-missing-test-case.patch + + bug-1150021_12-base-Get-Makefile-from-master.patch + + bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch + + bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch + +- Update lvm.conf file + add devices/allow_mixed_block_sizes item + add global/use_aio item + +------------------------------------------------------------------- +Thu Feb 20 10:21:15 UTC 2020 - ghe(a)suse.com + +- Fix heap memory leak in lvmetad. (bsc#1164126) + + bug-1164126_lvmetad-fix-heap-memory-leak.patch + +------------------------------------------------------------------- +Wed Feb 19 03:34:30 UTC 2020 - heming.zhao(a)suse.com + +- lvmetad uses devices/global_filter but not devices/filter after lvm2 update (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +------------------------------------------------------------------- +Mon Dec 23 07:09:00 UTC 2019 - heming.zhao(a)suse.com + +- LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + + bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch + + bug-1150021_02-suse-special-bcache-bug-fix.patch + +------------------------------------------------------------------- +Wed Nov 6 07:29:00 UTC 2019 - heming.zhao(a)suse.com + +- Fix seeing a 90 Second delay on shutdown and reboot (bsc#1155668) + + bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch + +------------------------------------------------------------------- +Wed Aug 21 10:10:30 UTC 2019 - ghe(a)suse.com + +- MD devices should be detected by LVM2 with metadata=1.0/0.9 (bsc#1145231) + + bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch + + bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch + + bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch + + bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch + + bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch + + bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +------------------------------------------------------------------- +Tue Jul 9 10:00:05 UTC 2019 - ghe(a)suse.com + +- Fix unknown feature in status message (bsc#1135984) + + bug-1135984_cache-support-no_discard_passdown.patch + +------------------------------------------------------------------- +Thu Jun 27 02:53:03 UTC 2019 - heming.zhao(a)suse.com + +- Fix using device aliases with lvmetad (bsc#1137296) + + bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch + +------------------------------------------------------------------- +Tue Apr 30 10:20:05 UTC 2019 - ghe(a)suse.com + +- Fix devices drop open error message (bsc#1122666) + + bug-1122666_devices-drop-open-error-message.patch + +------------------------------------------------------------------- +Tue Mar 19 12:02:02 UTC 2019 - Martin Liška <mliska(a)suse.cz> + +- Use %make_build in order to provide verbose output. + +------------------------------------------------------------------- +Fri Feb 1 08:20:15 UTC 2019 - ghe(a)suse.com + +- Disable the LVM lock daemon using sanlock, in order to avoid the + dependence on sanlock related packages (bsc#1121382) + +------------------------------------------------------------------- +Mon Jan 14 14:23:52 CET 2019 - kukuk(a)suse.de + +- Use %license instead of %doc [bsc#1082318] + +------------------------------------------------------------------- +Mon Nov 5 08:10:05 UTC 2018 - ghe(a)suse.com + +- Prevent writing beyond metadata area (bsc#1114113) + + bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch + ++++ 2164 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/lvm2-clvm.changes New Changes file: --- /dev/null 2020-11-18 17:46:03.679371574 +0100 +++ /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/lvm2.changes 2020-12-01 11:34:08.255029790 +0100 @@ -0,0 +1,2367 @@ +------------------------------------------------------------------- +Wed Nov 25 03:36:00 UTC 2020 - heming.zhao(a)suse.com + +- Update lvm2.spec file (bsc#1177533) + - in %preun, disable restart blk-availability.service & lvm2-monitor.service + +------------------------------------------------------------------- +Mon Nov 9 13:30:03 UTC 2020 - Marcos de Souza <mpdesouza(a)suse.com> + +- update patch according to systemd version/behaviour changed (bsc#1123327) + - bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch + +------------------------------------------------------------------- +Fri Aug 21 01:52:00 UTC 2020 - heming.zhao(a)suse.com + +- lvm scan: Too many open files (bsc#1173503) + + bug-1173503_lvmetad-fix-pvs-for-many-devices.patch + +- LVM failing to activate hot spare on surprise removal (bsc#1175110) + + bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch + +- change lvm2.spec source URL + - lvm2.spec + +------------------------------------------------------------------- +Mon Aug 10 12:42:00 UTC 2020 - heming.zhao(a)suse.com + +- add missing patches + systemd lvm2-pvscan@.service StartLimitInterval misplaced (bsc#1123327) + + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + +- modify patch according to changing patch: + bug-1123327_pvscan.service.in-Move-StartLimitInterval-to-Service.patch + + bug-998893_make_pvscan_service_after_multipathd.patch (bsc#998893) + +- use COMMON-PATCH to manage clvm patch + + lvm2-clvm.spec + - Patch3001: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + + Patch1006: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch + +------------------------------------------------------------------- +Wed Jul 22 08:08:00 UTC 2020 - heming.zhao(a)suse.com + +- On shutdown systems hangs for 90 seconds stopping LVM2 metatdata daemon (bsc#1172597) + + bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch + + bug-1172597_2-cleanup-missed-string-specifier.patch + + bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch + + bug-1172597_4-cov-check-for-socket_path-being-set.patch + + bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch + + bug-1172597_6-cov-missing-checks-of-syscalls.patch + + bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch + + bug-1172597_8-daemons-check-for-non-zero-thread_id.patch + +------------------------------------------------------------------- +Tue Feb 26 13:11:35 UTC 2020 - heming.zhao(a)suse.com + +- update patch (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +- update patch (bsc#1145231) + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +- boot takes extremely long time with 400 LUN's (bsc#1158358) + + bug-1158358_bcache-reduce-MAX_IO-to-256.patch + +- lvm2 bcache code enhancement (bsc#1164718) + + bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch + + bug-1164718_02-io-use-sync-io-if-aio-fails.patch + + bug-1164718_03-bcache-sync-io-fixes.patch + + bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch + + bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch + + bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch + + bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch + + bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch + + bug-1164718_09-config-change-scan_lvs-default-to-0.patch + + bug-1164718_10-config-add-new-setting-io_memory_size.patch + + bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch + + bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch + + bug-1164718_13-bcache-Fix-memory-leak.patch + + bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch + + bug-1164718_15-cov-release-iterator-on-error-path.patch + + bug-1164718_16-cov-check-lv_info.patch + + bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch + + bug-1164718_18-cov-validate-pagesize-is-not-negative.patch + + bug-1164718_19-cov-remove-unused-headers.patch + + bug-1164718_20-cov-check-result-of-dev_get_block_size.patch + + bug-1164718_21-gcc-clean-uninitialized-var-warning.patch + + bug-1164718_22-cov-release-iterator-on-error-path.patch + + bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch + + bug-1164718_24-pvscan-avoid-redundant-activation.patch + + bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch + + bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch + + bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch + + bug-1164718_28-lvmcache-free-resource-on-error-path.patch + +- Avoid creation of mixed-blocksize PV on LVM volume groups (LVM2) (bsc#1149408) + + bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch + + bug-1149408_02-tests-allow-mixed-block-sizes.patch + + bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch + + bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch + + bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch + + bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch + +- Use the upstream patches for LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + remove SUSE speical patch: + - bug-1150021_02-suse-special-bcache-bug-fix.patch + add patch: + + bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch + + bug-1150021_03-cov-Fix-a-leak.patch + + bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch + + bug-1150021_05-bcache-add-bcache_abort.patch + + bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch + + bug-1150021_07-bcache-add-unit-test.patch + + bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch + + bug-1150021_09-bcache-reverse-earlier-patch.patch + + bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch + + bug-1150021_11-radix-tree-Add-missing-test-case.patch + + bug-1150021_12-base-Get-Makefile-from-master.patch + + bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch + + bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch + +- Update lvm.conf file + add devices/allow_mixed_block_sizes item + add global/use_aio item + +------------------------------------------------------------------- +Thu Feb 20 10:21:15 UTC 2020 - ghe(a)suse.com + +- Fix heap memory leak in lvmetad. (bsc#1164126) + + bug-1164126_lvmetad-fix-heap-memory-leak.patch + +------------------------------------------------------------------- +Wed Feb 19 03:34:30 UTC 2020 - heming.zhao(a)suse.com + +- lvmetad uses devices/global_filter but not devices/filter after lvm2 update (bsc#1163526) + + bug-1163526_suse-special-global-filter-compat.patch + +------------------------------------------------------------------- +Mon Dec 23 07:09:00 UTC 2019 - heming.zhao(a)suse.com + +- LVM Metadata Error: Error writing device at 4096 length 512 (bsc#1150021) + + bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch + + bug-1150021_02-suse-special-bcache-bug-fix.patch + +------------------------------------------------------------------- +Wed Nov 6 07:29:00 UTC 2019 - heming.zhao(a)suse.com + +- Fix seeing a 90 Second delay on shutdown and reboot (bsc#1155668) + + bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch + +------------------------------------------------------------------- +Wed Aug 21 10:10:30 UTC 2019 - ghe(a)suse.com + +- MD devices should be detected by LVM2 with metadata=1.0/0.9 (bsc#1145231) + + bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch + + bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch + + bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch + + bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch + + bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch + + bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch + + bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch + +------------------------------------------------------------------- +Tue Jul 9 10:00:05 UTC 2019 - ghe(a)suse.com + +- Fix unknown feature in status message (bsc#1135984) + + bug-1135984_cache-support-no_discard_passdown.patch + +------------------------------------------------------------------- +Thu Jun 27 02:53:03 UTC 2019 - heming.zhao(a)suse.com + +- Fix using device aliases with lvmetad (bsc#1137296) + + bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch + +------------------------------------------------------------------- +Tue Apr 30 10:20:05 UTC 2019 - ghe(a)suse.com + +- Fix devices drop open error message (bsc#1122666) + + bug-1122666_devices-drop-open-error-message.patch + +------------------------------------------------------------------- +Tue Mar 19 12:02:02 UTC 2019 - Martin Liška <mliska(a)suse.cz> + +- Use %make_build in order to provide verbose output. + +------------------------------------------------------------------- +Fri Feb 1 08:20:15 UTC 2019 - ghe(a)suse.com + +- Disable the LVM lock daemon using sanlock, in order to avoid the + dependence on sanlock related packages (bsc#1121382) + +------------------------------------------------------------------- +Mon Jan 14 14:23:52 CET 2019 - kukuk(a)suse.de + +- Use %license instead of %doc [bsc#1082318] + ++++ 2170 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Leap:15.1:Update/.lvm2.15198.new.5913/lvm2.changes New: ---- LVM2.2.02.180.tgz LVM2.2.02.180.tgz.asc baselibs.conf bsc1080299-detect-clvm-properly.patch bug-1012973_simplify-special-case-for-md-in-69-dm-lvm-metadata.patch bug-1037309_Makefile-skip-compliling-daemons-lvmlockd-directory.patch bug-1043040_test-fix-read-ahead-issues-in-test-scripts.patch bug-1072624_test-lvmetad_dump-always-timed-out-when-using-nc.patch bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch bug-1122666_devices-drop-open-error-message.patch bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch bug-1135984_cache-support-no_discard_passdown.patch bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch bug-1149408_02-tests-allow-mixed-block-sizes.patch bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch bug-1150021_03-cov-Fix-a-leak.patch bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch bug-1150021_05-bcache-add-bcache_abort.patch bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch bug-1150021_07-bcache-add-unit-test.patch bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch bug-1150021_09-bcache-reverse-earlier-patch.patch bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch bug-1150021_11-radix-tree-Add-missing-test-case.patch bug-1150021_12-base-Get-Makefile-from-master.patch bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch bug-1158358_bcache-reduce-MAX_IO-to-256.patch bug-1163526_suse-special-global-filter-compat.patch bug-1164126_lvmetad-fix-heap-memory-leak.patch bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch bug-1164718_02-io-use-sync-io-if-aio-fails.patch bug-1164718_03-bcache-sync-io-fixes.patch bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch bug-1164718_09-config-change-scan_lvs-default-to-0.patch bug-1164718_10-config-add-new-setting-io_memory_size.patch bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch bug-1164718_13-bcache-Fix-memory-leak.patch bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch bug-1164718_15-cov-release-iterator-on-error-path.patch bug-1164718_16-cov-check-lv_info.patch bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch bug-1164718_18-cov-validate-pagesize-is-not-negative.patch bug-1164718_19-cov-remove-unused-headers.patch bug-1164718_20-cov-check-result-of-dev_get_block_size.patch bug-1164718_21-gcc-clean-uninitialized-var-warning.patch bug-1164718_22-cov-release-iterator-on-error-path.patch bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch bug-1164718_24-pvscan-avoid-redundant-activation.patch bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch bug-1164718_28-lvmcache-free-resource-on-error-path.patch bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch bug-1172597_2-cleanup-missed-string-specifier.patch bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch bug-1172597_4-cov-check-for-socket_path-being-set.patch bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch bug-1172597_6-cov-missing-checks-of-syscalls.patch bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch bug-1172597_8-daemons-check-for-non-zero-thread_id.patch bug-1173503_lvmetad-fix-pvs-for-many-devices.patch bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch bug-935623_dmeventd-fix-dso-name-wrong-compare.patch bug-950089_test-fix-lvm2-testsuite-build-error.patch bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch bug-998893_make_pvscan_service_after_multipathd.patch cmirrord_remove_date_time_from_compilation.patch device-mapper.changes device-mapper.spec fate-309425_display-dm-name-for-lv-name.patch fate-31841_fsadm-add-support-for-btrfs.patch lvm.conf lvm2-clvm.changes lvm2-clvm.spec lvm2.changes lvm2.spec pre_checkin.sh tests-specify-python3-as-the-script-interpreter.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ device-mapper.spec ++++++ # # spec file for package device-mapper # # Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed # upon. The license for this file, and modifications and additions to the # file, is the same license as for the pristine package itself (unless the # license for the pristine package is not an Open Source License, in which # case the license is the MIT License). An "Open Source License" is a # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. # Please submit bugfixes or comments via https://bugs.opensuse.org/ # %define libname libdevmapper1_03 %define libname_event libdevmapper-event1_03 ### COMMON-DEF-BEGIN ### %define lvm2_version 2.02.180 %define device_mapper_version 1.02.149 %define thin_provisioning_version 0.7.0 ### COMMON-DEF-END ### Name: device-mapper Version: %{device_mapper_version} Release: 0 Summary: Device Mapper Tools License: GPL-2.0-or-later AND LGPL-2.1-or-later Group: System/Base Url: http://www.sourceware.org/lvm2/ Source: ftp://sources.redhat.com/pub/lvm2/LVM2.%{lvm2_version}.tgz Source1: ftp://sources.redhat.com/pub/lvm2/LVM2.%{lvm2_version}.tgz.asc Source99: baselibs.conf # To detect modprobe during build BuildRequires: gcc-c++ BuildRequires: kmod-compat BuildRequires: libaio-devel BuildRequires: pkgconfig BuildRequires: suse-module-tools BuildRequires: thin-provisioning-tools >= %{thin_provisioning_version} BuildRequires: pkgconfig(libselinux) BuildRequires: pkgconfig(libsepol) BuildRequires: pkgconfig(libudev) BuildRequires: pkgconfig(systemd) Requires: thin-provisioning-tools >= %{thin_provisioning_version} Requires(post): coreutils BuildRoot: %{_tmppath}/%{name}-%{version}-build %{?systemd_requires} ### COMMON-PATCH-BEGIN ### # Upstream patches (order with creating time) Patch0001: bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch Patch0002: bug-1158358_bcache-reduce-MAX_IO-to-256.patch Patch0003: bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch Patch0004: bug-1173503_lvmetad-fix-pvs-for-many-devices.patch Patch0005: bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch Patch0006: bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch Patch0007: bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch Patch0008: bug-1164718_02-io-use-sync-io-if-aio-fails.patch Patch0009: bug-1164718_03-bcache-sync-io-fixes.patch Patch0010: bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch Patch0011: bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch Patch0012: bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch Patch0013: bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch Patch0014: bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch Patch0015: bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch Patch0016: bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch Patch0017: bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch Patch0018: bug-1164718_09-config-change-scan_lvs-default-to-0.patch Patch0019: bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch Patch0021: bug-1164718_10-config-add-new-setting-io_memory_size.patch Patch0022: bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch Patch0023: bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch Patch0024: bug-1164718_13-bcache-Fix-memory-leak.patch Patch0025: bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch Patch0026: bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch Patch0027: bug-1172597_2-cleanup-missed-string-specifier.patch Patch0028: bug-1122666_devices-drop-open-error-message.patch Patch0029: bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch Patch0030: bug-1135984_cache-support-no_discard_passdown.patch Patch0031: bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch Patch0032: bug-1164718_15-cov-release-iterator-on-error-path.patch Patch0033: bug-1164718_16-cov-check-lv_info.patch Patch0034: bug-1172597_4-cov-check-for-socket_path-being-set.patch Patch0035: bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch Patch0036: bug-1164718_18-cov-validate-pagesize-is-not-negative.patch Patch0037: bug-1164718_19-cov-remove-unused-headers.patch Patch0038: bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch Patch0039: bug-1164718_20-cov-check-result-of-dev_get_block_size.patch Patch0040: bug-1164718_21-gcc-clean-uninitialized-var-warning.patch Patch0041: bug-1164718_22-cov-release-iterator-on-error-path.patch Patch0042: bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch Patch0043: bug-1164718_24-pvscan-avoid-redundant-activation.patch Patch0044: bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch Patch0045: bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch Patch0046: bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch Patch0047: bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch Patch0048: bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch Patch0049: bug-1164718_28-lvmcache-free-resource-on-error-path.patch Patch0050: bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch Patch0051: bug-1149408_02-tests-allow-mixed-block-sizes.patch Patch0052: bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch Patch0053: bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch Patch0054: bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch Patch0055: bug-1172597_6-cov-missing-checks-of-syscalls.patch Patch0056: bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch Patch0057: bug-1172597_8-daemons-check-for-non-zero-thread_id.patch Patch0058: bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch Patch0059: bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch Patch0060: bug-1150021_03-cov-Fix-a-leak.patch Patch0061: bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch Patch0062: bug-1150021_05-bcache-add-bcache_abort.patch Patch0063: bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch Patch0064: bug-1150021_07-bcache-add-unit-test.patch Patch0065: bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch Patch0066: bug-1150021_09-bcache-reverse-earlier-patch.patch Patch0067: bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch Patch0068: bug-1150021_11-radix-tree-Add-missing-test-case.patch Patch0069: bug-1150021_12-base-Get-Makefile-from-master.patch Patch0070: bug-1164126_lvmetad-fix-heap-memory-leak.patch Patch0071: bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch Patch0072: bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch # SUSE patches: 1000+ for LVM # Never upstream Patch1001: cmirrord_remove_date_time_from_compilation.patch Patch1002: fate-309425_display-dm-name-for-lv-name.patch Patch1003: fate-31841_fsadm-add-support-for-btrfs.patch Patch1004: bug-935623_dmeventd-fix-dso-name-wrong-compare.patch Patch1005: bsc1080299-detect-clvm-properly.patch Patch1006: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch Patch1007: bug-998893_make_pvscan_service_after_multipathd.patch Patch1008: bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch Patch1009: bug-1163526_suse-special-global-filter-compat.patch Patch1010: bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch #SUSE patches 2000+ for device mapper, udev rules Patch2001: bug-1012973_simplify-special-case-for-md-in-69-dm-lvm-metadata.patch ####### # NOTE: for simplify patch job, speical patches for device mapper & # clvmd/cmirrord are still stored in COMMON-PATCH sector, which not in # device-mapper.spec & lvm2-clvm.spec ####### ### COMMON-PATCH-END ### %description Programs and man pages for configuring and using the device mapper. %prep %setup -q -n LVM2.%{lvm2_version} ### COMMON-PREP-BEGIN ### %patch0001 -p1 %patch0002 -p1 %patch0003 -p1 %patch0004 -p1 %patch0005 -p1 %patch0006 -p1 %patch0007 -p1 %patch0008 -p1 %patch0009 -p1 %patch0010 -p1 %patch0011 -p1 %patch0012 -p1 %patch0013 -p1 %patch0014 -p1 %patch0015 -p1 %patch0016 -p1 %patch0017 -p1 %patch0018 -p1 %patch0019 -p1 %patch0021 -p1 %patch0022 -p1 %patch0023 -p1 %patch0024 -p1 %patch0025 -p1 %patch0026 -p1 %patch0027 -p1 %patch0028 -p1 %patch0029 -p1 %patch0030 -p1 %patch0031 -p1 %patch0032 -p1 %patch0033 -p1 %patch0034 -p1 %patch0035 -p1 %patch0036 -p1 %patch0037 -p1 %patch0038 -p1 %patch0039 -p1 %patch0040 -p1 %patch0041 -p1 %patch0042 -p1 %patch0043 -p1 %patch0044 -p1 %patch0045 -p1 %patch0046 -p1 %patch0047 -p1 %patch0048 -p1 %patch0049 -p1 %patch0050 -p1 %patch0051 -p1 %patch0052 -p1 %patch0053 -p1 %patch0054 -p1 %patch0055 -p1 %patch0056 -p1 %patch0057 -p1 %patch0058 -p1 %patch0059 -p1 %patch0060 -p1 %patch0061 -p1 %patch0062 -p1 %patch0063 -p1 %patch0064 -p1 %patch0065 -p1 %patch0066 -p1 %patch0067 -p1 %patch0068 -p1 %patch0069 -p1 %patch0070 -p1 %patch0071 -p1 %patch0072 -p1 %patch1001 -p1 %patch1002 -p1 %patch1003 -p1 %patch1004 -p1 %patch1005 -p1 %patch1006 -p1 %patch1007 -p1 %patch1008 -p1 %patch1009 -p1 %patch1010 -p1 %patch2001 -p1 ### COMMON-PREP-END ### %build extra_opts="" ### COMMON-CONFIG-BEGIN ### export PATH=$PATH:/sbin:%{_prefix}/sbin # Why this messy fix here? someone released a wrong version... sed -ie "s/%{device_mapper_version}/1.03.01/g" VERSION_DM %configure \ --enable-dmeventd \ --enable-cmdlib \ --enable-udev_rules \ --enable-udev_sync \ --with-udev-prefix="%{_prefix}/" \ --enable-selinux \ --enable-pkgconfig \ --with-usrlibdir=%{_libdir} \ --with-usrsbindir=%{_sbindir} \ --with-default-dm-run-dir=/run \ --with-tmpfilesdir=%{_tmpfilesdir} \ --with-thin=internal \ --with-device-gid=6 \ --with-device-mode=0640 \ --with-device-uid=0 \ --with-dmeventd-path=%{_sbindir}/dmeventd \ --with-thin-check=%{_sbindir}/thin_check \ --with-thin-dump=%{_sbindir}/thin_dump \ --with-thin-repair=%{_sbindir}/thin_repair \ $extra_opts ### COMMON-CONFIG-END ### %make_build device-mapper %install make DESTDIR=%{buildroot} \ install_device-mapper \ install_systemd_units install_systemd_generators ln -s service %{buildroot}/%{_sbindir}/rcdm-event # provide 1.02 compat links for the shared libraries # this is needed for various binary packages ln -s libdevmapper.so.1.03 %{buildroot}/%{_libdir}/libdevmapper.so.1.02 ln -s libdevmapper-event.so.1.03 %{buildroot}/%{_libdir}/libdevmapper-event.so.1.02 # remove blkd, will be in lvm2 proper # without force on purpose to detect changes and fail if it happens rm %{buildroot}%{_sbindir}/blkdeactivate rm %{buildroot}%{_unitdir}/blk-availability.service rm %{buildroot}%{_unitdir}/lvm2-monitor.service rm %{buildroot}%{_mandir}/man8/blkdeactivate.8 rm %{buildroot}%{_mandir}/man8/lvm2-activation-generator.8 # compat symlinks in /sbin remove with Leap 43 mkdir -p %{buildroot}/sbin ln -s %{_sbindir}/dmsetup %{buildroot}/sbin/dmsetup %pre %service_add_pre dm-event.service dm-event.socket %post %service_add_post dm-event.service dm-event.socket %{?regenerate_initrd_post} %posttrans %{?regenerate_initrd_posttrans} %preun %service_del_preun dm-event.service dm-event.socket %postun %service_del_postun dm-event.service dm-event.socket %{?regenerate_initrd_post} %files %defattr(-,root,root) %license COPYING COPYING.LIB %doc README %doc udev/12-dm-permissions.rules /sbin/dmsetup %{_sbindir}/dmsetup %{_sbindir}/dmeventd %{_sbindir}/dmstats %{_mandir}/man8/dmstats.8%{ext_man} %{_mandir}/man8/dmsetup.8%{ext_man} %{_mandir}/man8/dmeventd.8%{ext_man} %{_udevrulesdir}/10-dm.rules %{_udevrulesdir}/13-dm-disk.rules %{_udevrulesdir}/95-dm-notify.rules %{_unitdir}/dm-event.socket %{_sbindir}/rcdm-event %{_unitdir}/dm-event.service %package -n %{libname} Summary: Library for device-mapper Group: System/Libraries Conflicts: %{name} < %{version} %description -n %{libname} Device mapper main shared library %files -n %{libname} %defattr(-,root,root) %{_libdir}/libdevmapper.so.1.03 %{_libdir}/libdevmapper.so.1.02 %post -n %{libname} if [ -f /%{_lib}/libdevmapper.so.1.03 ]; then # Special migration - the library is now in %{_libdir}, but up to the point where # zypp removes the 'old' device-mapper package, the old library 'wins' the ldloader race # resulting in binaries asking for the newer version still getting the old one. # This in turn results in funny bugs like boo#1045396 # Remove /%{_lib}/libdevmapper.so.1.02 - and the run ldconfig rm /%{_lib}/libdevmapper.so.1.03 fi /sbin/ldconfig %postun -n %{libname} -p /sbin/ldconfig %package -n %{libname_event} Summary: Event library for device-mapper Group: System/Libraries Conflicts: %{name} < %{version} %description -n %{libname_event} Device mapper event daemon shared library %files -n %{libname_event} %defattr(-,root,root) %{_libdir}/libdevmapper-event.so.1.03 %{_libdir}/libdevmapper-event.so.1.02 %post -n %{libname_event} -p /sbin/ldconfig %postun -n %{libname_event} -p /sbin/ldconfig %package devel Summary: Development package for the device mapper Group: Development/Libraries/C and C++ Requires: %{libname_event} = %{device_mapper_version} Requires: %{libname} = %{device_mapper_version} Requires: device-mapper = %{device_mapper_version} %description devel Files needed for software development using the device mapper %files devel %defattr(-,root,root) %{_libdir}/libdevmapper.so %{_libdir}/libdevmapper-event.so %{_includedir}/libdevmapper.h %{_includedir}/libdevmapper-event.h %{_libdir}/pkgconfig/devmapper.pc %{_libdir}/pkgconfig/devmapper-event.pc %changelog ++++++ lvm2-clvm.spec ++++++ # # spec file for package lvm2-clvm # # Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed # upon. The license for this file, and modifications and additions to the # file, is the same license as for the pristine package itself (unless the # license for the pristine package is not an Open Source License, in which # case the license is the MIT License). An "Open Source License" is a # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. # Please submit bugfixes or comments via https://bugs.opensuse.org/ # %define _supportsanlock 0 %define dlm_version 3.99.1 %if 0%{_supportsanlock} == 1 %define sanlock_version 3.3.0 %endif ### COMMON-DEF-BEGIN ### %define lvm2_version 2.02.180 %define device_mapper_version 1.02.149 %define thin_provisioning_version 0.7.0 ### COMMON-DEF-END ### Name: lvm2-clvm Version: %{lvm2_version} Release: 0 Summary: Clustered LVM2 License: GPL-2.0-or-later AND LGPL-2.1-or-later Group: Productivity/Clustering/HA Url: http://sources.redhat.com/cluster/clvm/ Source: ftp://sources.redhat.com/pub/lvm2/LVM2.%{lvm2_version}.tgz Source1: ftp://sources.redhat.com/pub/lvm2/LVM2.%{lvm2_version}.tgz.asc # To detect modprobe during build BuildRequires: kmod-compat BuildRequires: libaio-devel BuildRequires: libcorosync-devel BuildRequires: libdlm-devel >= %{dlm_version} BuildRequires: pkgconfig %if 0%{_supportsanlock} == 1 BuildRequires: sanlock-devel >= %{sanlock_version} %endif BuildRequires: thin-provisioning-tools >= %{thin_provisioning_version} BuildRequires: pkgconfig(blkid) BuildRequires: pkgconfig(devmapper) BuildRequires: pkgconfig(libudev) Requires: corosync Requires: device-mapper >= %{device_mapper_version} Requires: lvm2 = %{version} Requires: lvm2-cmirrord Obsoletes: cmirrord < %{version} Provides: cmirrord = %{version} ### COMMON-PATCH-BEGIN ### # Upstream patches (order with creating time) Patch0001: bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch Patch0002: bug-1158358_bcache-reduce-MAX_IO-to-256.patch Patch0003: bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch Patch0004: bug-1173503_lvmetad-fix-pvs-for-many-devices.patch Patch0005: bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch Patch0006: bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch Patch0007: bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch Patch0008: bug-1164718_02-io-use-sync-io-if-aio-fails.patch Patch0009: bug-1164718_03-bcache-sync-io-fixes.patch Patch0010: bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch Patch0011: bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch Patch0012: bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch Patch0013: bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch Patch0014: bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch Patch0015: bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch Patch0016: bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch Patch0017: bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch Patch0018: bug-1164718_09-config-change-scan_lvs-default-to-0.patch Patch0019: bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch Patch0021: bug-1164718_10-config-add-new-setting-io_memory_size.patch Patch0022: bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch Patch0023: bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch Patch0024: bug-1164718_13-bcache-Fix-memory-leak.patch Patch0025: bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch Patch0026: bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch Patch0027: bug-1172597_2-cleanup-missed-string-specifier.patch Patch0028: bug-1122666_devices-drop-open-error-message.patch Patch0029: bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch Patch0030: bug-1135984_cache-support-no_discard_passdown.patch Patch0031: bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch Patch0032: bug-1164718_15-cov-release-iterator-on-error-path.patch Patch0033: bug-1164718_16-cov-check-lv_info.patch Patch0034: bug-1172597_4-cov-check-for-socket_path-being-set.patch Patch0035: bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch Patch0036: bug-1164718_18-cov-validate-pagesize-is-not-negative.patch Patch0037: bug-1164718_19-cov-remove-unused-headers.patch Patch0038: bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch Patch0039: bug-1164718_20-cov-check-result-of-dev_get_block_size.patch Patch0040: bug-1164718_21-gcc-clean-uninitialized-var-warning.patch Patch0041: bug-1164718_22-cov-release-iterator-on-error-path.patch Patch0042: bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch Patch0043: bug-1164718_24-pvscan-avoid-redundant-activation.patch Patch0044: bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch Patch0045: bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch Patch0046: bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch Patch0047: bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch Patch0048: bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch Patch0049: bug-1164718_28-lvmcache-free-resource-on-error-path.patch Patch0050: bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch Patch0051: bug-1149408_02-tests-allow-mixed-block-sizes.patch Patch0052: bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch Patch0053: bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch Patch0054: bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch Patch0055: bug-1172597_6-cov-missing-checks-of-syscalls.patch Patch0056: bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch Patch0057: bug-1172597_8-daemons-check-for-non-zero-thread_id.patch Patch0058: bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch Patch0059: bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch Patch0060: bug-1150021_03-cov-Fix-a-leak.patch Patch0061: bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch Patch0062: bug-1150021_05-bcache-add-bcache_abort.patch Patch0063: bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch Patch0064: bug-1150021_07-bcache-add-unit-test.patch Patch0065: bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch Patch0066: bug-1150021_09-bcache-reverse-earlier-patch.patch Patch0067: bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch Patch0068: bug-1150021_11-radix-tree-Add-missing-test-case.patch Patch0069: bug-1150021_12-base-Get-Makefile-from-master.patch Patch0070: bug-1164126_lvmetad-fix-heap-memory-leak.patch Patch0071: bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch Patch0072: bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch # SUSE patches: 1000+ for LVM # Never upstream Patch1001: cmirrord_remove_date_time_from_compilation.patch Patch1002: fate-309425_display-dm-name-for-lv-name.patch Patch1003: fate-31841_fsadm-add-support-for-btrfs.patch Patch1004: bug-935623_dmeventd-fix-dso-name-wrong-compare.patch Patch1005: bsc1080299-detect-clvm-properly.patch Patch1006: bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch Patch1007: bug-998893_make_pvscan_service_after_multipathd.patch Patch1008: bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch Patch1009: bug-1163526_suse-special-global-filter-compat.patch Patch1010: bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch #SUSE patches 2000+ for device mapper, udev rules Patch2001: bug-1012973_simplify-special-case-for-md-in-69-dm-lvm-metadata.patch ####### # NOTE: for simplify patch job, speical patches for device mapper & # clvmd/cmirrord are still stored in COMMON-PATCH sector, which not in # device-mapper.spec & lvm2-clvm.spec ####### ### COMMON-PATCH-END ### %description A daemon for using LVM2 Logival Volumes in a clustered environment. %prep %setup -q -n LVM2.%{lvm2_version} ### COMMON-PREP-BEGIN ### %patch0001 -p1 %patch0002 -p1 %patch0003 -p1 %patch0004 -p1 %patch0005 -p1 %patch0006 -p1 %patch0007 -p1 %patch0008 -p1 %patch0009 -p1 %patch0010 -p1 %patch0011 -p1 %patch0012 -p1 %patch0013 -p1 %patch0014 -p1 %patch0015 -p1 %patch0016 -p1 %patch0017 -p1 %patch0018 -p1 %patch0019 -p1 %patch0021 -p1 %patch0022 -p1 %patch0023 -p1 %patch0024 -p1 %patch0025 -p1 %patch0026 -p1 %patch0027 -p1 %patch0028 -p1 %patch0029 -p1 %patch0030 -p1 %patch0031 -p1 %patch0032 -p1 %patch0033 -p1 %patch0034 -p1 %patch0035 -p1 %patch0036 -p1 %patch0037 -p1 %patch0038 -p1 %patch0039 -p1 %patch0040 -p1 %patch0041 -p1 %patch0042 -p1 %patch0043 -p1 %patch0044 -p1 %patch0045 -p1 %patch0046 -p1 %patch0047 -p1 %patch0048 -p1 %patch0049 -p1 %patch0050 -p1 %patch0051 -p1 %patch0052 -p1 %patch0053 -p1 %patch0054 -p1 %patch0055 -p1 %patch0056 -p1 %patch0057 -p1 %patch0058 -p1 %patch0059 -p1 %patch0060 -p1 %patch0061 -p1 %patch0062 -p1 %patch0063 -p1 %patch0064 -p1 %patch0065 -p1 %patch0066 -p1 %patch0067 -p1 %patch0068 -p1 %patch0069 -p1 %patch0070 -p1 %patch0071 -p1 %patch0072 -p1 %patch1001 -p1 %patch1002 -p1 %patch1003 -p1 %patch1004 -p1 %patch1005 -p1 %patch1006 -p1 %patch1007 -p1 %patch1008 -p1 %patch1009 -p1 %patch1010 -p1 %patch2001 -p1 ### COMMON-PREP-END ### %build extra_opts=" --enable-applib --enable-blkid_wiping --enable-cmdlib --enable-lvmetad --enable-lvmpolld --enable-realtime --with-default-locking-dir=/run/lock/lvm --with-default-pid-dir=/run --with-default-run-dir=/run/lvm --with-clvmd=corosync --with-cluster=internal --enable-cmirrord --enable-lvmlockd-dlm %if 0%{_supportsanlock} == 1 --enable-lvmlockd-sanlock %endif " ### COMMON-CONFIG-BEGIN ### export PATH=$PATH:/sbin:%{_prefix}/sbin # Why this messy fix here? someone released a wrong version... sed -ie "s/%{device_mapper_version}/1.03.01/g" VERSION_DM %configure \ --enable-dmeventd \ --enable-cmdlib \ --enable-udev_rules \ --enable-udev_sync \ --with-udev-prefix="%{_prefix}/" \ --enable-selinux \ --enable-pkgconfig \ --with-usrlibdir=%{_libdir} \ --with-usrsbindir=%{_sbindir} \ --with-default-dm-run-dir=/run \ --with-tmpfilesdir=%{_tmpfilesdir} \ --with-thin=internal \ --with-device-gid=6 \ --with-device-mode=0640 \ --with-device-uid=0 \ --with-dmeventd-path=%{_sbindir}/dmeventd \ --with-thin-check=%{_sbindir}/thin_check \ --with-thin-dump=%{_sbindir}/thin_dump \ --with-thin-repair=%{_sbindir}/thin_repair \ $extra_opts ### COMMON-CONFIG-END ### %make_build %install make DESTDIR=%{buildroot} \ install_cluster \ install_systemd_units install_systemd_generators make DESTDIR=%{buildroot} install -C daemons/lvmlockd make DESTDIR=%{buildroot} install -C daemons/cmirrord # lvmlockd does not have separate target install the mans by hand for now install -m0644 -D man/lvmlockd.8 %{buildroot}%{_mandir}/man8/lvmlockd.8 install -m0644 -D man/lvmlockctl.8 %{buildroot}%{_mandir}/man8/lvmlockctl.8 # rc services symlinks ln -s service %{buildroot}%{_sbindir}/rclvm2-cluster-activation ln -s service %{buildroot}%{_sbindir}/rclvm2-clvmd ln -s service %{buildroot}%{_sbindir}/rclvm2-cmirrord ln -s service %{buildroot}%{_sbindir}/rclvm2-lvmlockd ln -s service %{buildroot}%{_sbindir}/rclvm2-lvmlocking # remove files from lvm2 split due to systemd_generators picking them up rm %{buildroot}%{_unitdir}/blk-availability.service rm %{buildroot}%{_unitdir}/dm-event.service rm %{buildroot}%{_unitdir}/dm-event.socket rm %{buildroot}%{_unitdir}/lvm2-monitor.service rm %{buildroot}%{_mandir}/man8/lvm2-activation-generator.8 rm %{buildroot}%{_libexecdir}/systemd/system-generators/lvm2-activation-generator rm %{buildroot}%{_unitdir}/lvm2-lvmetad.service rm %{buildroot}%{_unitdir}/lvm2-lvmetad.socket rm %{buildroot}%{_unitdir}/lvm2-lvmpolld.service rm %{buildroot}%{_unitdir}/lvm2-lvmpolld.socket rm %{buildroot}%{_unitdir}/lvm2-pvscan@.service %files %defattr(-,root,root) %{_sbindir}/clvmd %{_sbindir}/rclvm2-cluster-activation %{_sbindir}/rclvm2-clvmd %{_unitdir}/lvm2-clvmd.service %{_unitdir}/lvm2-cluster-activation.service %{_libexecdir}/systemd/lvm2-cluster-activation %{_mandir}/man8/clvmd.8%{ext_man} %package -n lvm2-cmirrord Summary: Clustered RAID 1 support using device-mapper and corosync Group: Productivity/Clustering/HA Requires: corosync Requires: device-mapper >= %{device_mapper_version} Requires: lvm2 = %{version} Requires: lvm2-clvm %description -n lvm2-cmirrord A daemon for using LVM2 Logival Volumes in a clustered environment. %files -n lvm2-cmirrord %defattr(-,root,root) %{_sbindir}/cmirrord %{_libexecdir}/systemd/system/lvm2-cmirrord.service %{_sbindir}/rclvm2-cmirrord %{_mandir}/man8/cmirrord.8%{ext_man} %package -n lvm2-lockd Summary: LVM locking daemon Group: Productivity/Clustering/HA Recommends: libdlm >= %{dlm_version} Requires: lvm2 = %{version} %if 0%{_supportsanlock} == 1 Requires: sanlock >= %{sanlock_version} %endif %{?systemd_requires} %description -n lvm2-lockd LVM commands use lvmlockd to coordinate access to shared storage. %pre -n lvm2-lockd %service_add_pre lvm2-lvmlockd.service lvm2-lvmlocking.service %post -n lvm2-lockd %service_add_post lvm2-lvmlockd.service lvm2-lvmlocking.service %preun -n lvm2-lockd %service_del_preun lvm2-lvmlockd.service lvm2-lvmlocking.service %postun -n lvm2-lockd %service_del_postun lvm2-lvmlockd.service lvm2-lvmlocking.service %files -n lvm2-lockd %defattr(-,root,root,) %{_sbindir}/lvmlockd %{_sbindir}/lvmlockctl %{_mandir}/man8/lvmlockd.8%{ext_man} %{_mandir}/man8/lvmlockctl.8%{ext_man} %{_unitdir}/lvm2-lvmlockd.service %{_unitdir}/lvm2-lvmlocking.service %{_sbindir}/rclvm2-lvmlockd %{_sbindir}/rclvm2-lvmlocking %changelog ++++++ lvm2.spec ++++++ ++++ 656 lines (skipped) ++++++ baselibs.conf ++++++ libdevmapper1_03 conflicts "device-mapper-<targettype> < <version>" libdevmapper-event1_03 conflicts "device-mapper-<targettype> < <version>" device-mapper-devel requires "libdevmapper1_03-<targettype> = <version>" requires "libdevmapper-event1_03-<targettype> = <version>" ++++++ bsc1080299-detect-clvm-properly.patch ++++++ Index: LVM2.2.02.178/configure =================================================================== --- LVM2.2.02.178.orig/configure +++ LVM2.2.02.178/configure @@ -10888,8 +10888,6 @@ if [ `expr x"$CLVMD" : '.*corosync.*'` ! fi ################################################################################ -if test "$CLVMD" != none; then - # Check whether --with-clvmd-pidfile was given. if test "${with_clvmd_pidfile+set}" = set; then : withval=$with_clvmd_pidfile; CLVMD_PIDFILE=$withval @@ -10902,8 +10900,6 @@ cat >>confdefs.h <<_ACEOF #define CLVMD_PIDFILE "$CLVMD_PIDFILE" _ACEOF -fi - ################################################################################ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build cluster mirror log daemon" >&5 $as_echo_n "checking whether to build cluster mirror log daemon... " >&6; } Index: LVM2.2.02.178/configure.ac =================================================================== --- LVM2.2.02.178.orig/configure.ac +++ LVM2.2.02.178/configure.ac @@ -959,7 +959,6 @@ fi ################################################################################ dnl -- clvmd pidfile -if test "$CLVMD" != none; then AC_ARG_WITH(clvmd-pidfile, AC_HELP_STRING([--with-clvmd-pidfile=PATH], [clvmd pidfile [PID_DIR/clvmd.pid]]), @@ -967,7 +966,6 @@ if test "$CLVMD" != none; then CLVMD_PIDFILE="$DEFAULT_PID_DIR/clvmd.pid") AC_DEFINE_UNQUOTED(CLVMD_PIDFILE, ["$CLVMD_PIDFILE"], [Path to clvmd pidfile.]) -fi ################################################################################ dnl -- Build cluster mirror log daemon ++++++ bug-1012973_simplify-special-case-for-md-in-69-dm-lvm-metadata.patch ++++++ From 0913b597d61b9b430654d7ab06528cdfcfaf06f4 Mon Sep 17 00:00:00 2001 From: NeilBrown <neilb(a)suse.com> Date: Wed, 4 Jan 2017 14:20:53 +1100 Subject: [PATCH] Simplify special-case for md in 69-dm-lvm-metadata.rules This special casing brings little value. It appears to attempt to determine if the array is active yet or not, and to skip processing if the array has not yet been started. However, if the array hasn't been started, then "blkid" will not have been able to read a signature, so: ENV{ID_FS_TYPE}!="LVM2_member|LVM1_member", GOTO="lvm_end" will have caused all this code to be skipped. Further, this code causes incorrect behaviour in at least one case. It assumes that the first "add" event should be ignored, as it will be followed by a "change" event which indicates the array coming on line. This is consistent with how the kernel sends events, but not always consistent with how this script sees event. Specifically: if the initrd has "mdadm" support installed, but not "lvm2" support, then the initial "add" and "change" events will happen while the initrd is in charge and this file is not available. Once the root filesystem is mountd, this file will be available and "udevadm trigger --action=add" will be run. So the first and only event seen by this script for an md device will be "add", and it will incorrectly ignore it. So replace the special handling with code that simply jumps to lvm_scan on any 'add' or 'change' event. Signed-off-by: NeilBrown <neilb(a)suse.com> --- udev/69-dm-lvm-metad.rules.in | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/udev/69-dm-lvm-metad.rules.in b/udev/69-dm-lvm-metad.rules.in index bd75fc8efcd5..fcbb7f755eba 100644 --- a/udev/69-dm-lvm-metad.rules.in +++ b/udev/69-dm-lvm-metad.rules.in @@ -51,13 +51,11 @@ ENV{DM_UDEV_PRIMARY_SOURCE_FLAG}=="1", ENV{DM_ACTIVATION}=="1", GOTO="lvm_scan" GOTO="lvm_end" # MD device: +# Need to scan on both 'add' and 'change' LABEL="next" KERNEL!="md[0-9]*", GOTO="next" -IMPORT{db}="LVM_MD_PV_ACTIVATED" -ACTION=="add", ENV{LVM_MD_PV_ACTIVATED}=="1", GOTO="lvm_scan" -ACTION=="change", ENV{LVM_MD_PV_ACTIVATED}!="1", TEST=="md/array_state", ENV{LVM_MD_PV_ACTIVATED}="1", GOTO="lvm_scan" -ACTION=="add", KERNEL=="md[0-9]*p[0-9]*", GOTO="lvm_scan" -ENV{LVM_MD_PV_ACTIVATED}!="1", ENV{SYSTEMD_READY}="0" +ACTION=="add", GOTO="lvm_scan" +ACTION=="change", GOTO="lvm_scan" GOTO="lvm_end" # Loop device: -- 2.11.0 ++++++ bug-1037309_Makefile-skip-compliling-daemons-lvmlockd-directory.patch ++++++ From c0d05011007bb46232e64435e05f6f6da212655b Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Tue, 4 Jul 2017 20:11:17 +0800 Subject: [PATCH] build: skip compliling daemons/lvmlockd directory bsc#1037309 Signed-off-by: Eric Ren <zren(a)suse.com> --- configure | 3 ++- daemons/Makefile.in | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/configure b/configure index e1ae0e8..c442459 100755 --- a/configure +++ b/configure @@ -11865,7 +11865,8 @@ test -n "$LVMPOLLD" && BUILD_LVMPOLLD=$LVMPOLLD $as_echo "$BUILD_LVMPOLLD" >&6; } ################################################################################ -BUILD_LVMLOCKD=no +# BUILD_LVMLOCKD=no +BUILD_LVMLOCKD=yes { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build lvmlockdsanlock" >&5 $as_echo_n "checking whether to build lvmlockdsanlock... " >&6; } diff --git a/daemons/Makefile.in b/daemons/Makefile.in index ebbd740..58d5fef 100644 --- a/daemons/Makefile.in +++ b/daemons/Makefile.in @@ -40,9 +40,9 @@ ifeq ("@BUILD_LVMPOLLD@", "yes") SUBDIRS += lvmpolld endif -ifeq ("@BUILD_LVMLOCKD@", "yes") - SUBDIRS += lvmlockd -endif +# ifeq ("@BUILD_LVMLOCKD@", "yes") +# SUBDIRS += lvmlockd +# endif ifeq ("@BUILD_LVMDBUSD@", "yes") SUBDIRS += lvmdbusd -- 2.10.2 ++++++ bug-1043040_test-fix-read-ahead-issues-in-test-scripts.patch ++++++ From 037eb35169d4a322c3b6d75c5b3491c77c41b4a3 Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Sat, 17 Jun 2017 21:32:40 +0800 Subject: [PATCH] test: fix read ahead issues in test scripts We have a SUSE specific kernel patch, which tunes the kernel "VM_MAX_READAHEAD" from 128k to 512k. Thus, some testing scripts are affected by this change. See bsc#1043040 for more infos. Signed-off-by: Eric Ren <zren(a)suse.com> --- test/shell/lvcreate-usage.sh | 4 ++-- test/shell/read-ahead.sh | 8 +++++++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/test/shell/lvcreate-usage.sh b/test/shell/lvcreate-usage.sh index 4b5c2b6..e2e1b8f 100644 --- a/test/shell/lvcreate-usage.sh +++ b/test/shell/lvcreate-usage.sh @@ -173,10 +173,10 @@ check lv_field $vg/$lv3 lv_read_ahead "8.00k" check lv_field $vg/$lv3 lv_kernel_read_ahead "8.00k" lvcreate -L 8 -n $lv4 --readahead auto $vg check lv_field $vg/$lv4 lv_read_ahead "auto" -check lv_field $vg/$lv4 lv_kernel_read_ahead "128.00k" +check lv_field $vg/$lv4 lv_kernel_read_ahead "512.00k" lvcreate -L 8 -n $lv5 -i2 --stripesize 16k --readahead auto $vg check lv_field $vg/$lv5 lv_read_ahead "auto" -check lv_field $vg/$lv5 lv_kernel_read_ahead "128.00k" +check lv_field $vg/$lv5 lv_kernel_read_ahead "512.00k" lvcreate -L 8 -n $lv6 -i2 --stripesize 128k --readahead auto $vg check lv_field $vg/$lv6 lv_read_ahead "auto" check lv_field $vg/$lv6 lv_kernel_read_ahead "512.00k" diff --git a/test/shell/read-ahead.sh b/test/shell/read-ahead.sh index 9045de8..4a4991b 100644 --- a/test/shell/read-ahead.sh +++ b/test/shell/read-ahead.sh @@ -32,11 +32,17 @@ lvchange -r 640 $vg/$lv check lv_field $vg/$lv lv_read_ahead 640 --units s --nosuffix lvremove -ff $vg +#See bsc#1043040 +default_ra=$(blockdev --getra "$dev1") #COMM "read ahead is properly inherited from underlying PV" blockdev --setra 768 "$dev1" vgscan lvcreate -n $lv -L4m $vg "$dev1" -test "$(blockdev --getra "$DM_DEV_DIR/$vg/$lv")" -eq 768 +if [ "$default_ra" -lt 768 ]; then + test "$(blockdev --getra "$DM_DEV_DIR/$vg/$lv")" -eq 768 +else + test $(blockdev --getra "$DM_DEV_DIR/$vg/$lv") -eq "$default_ra" +fi lvremove -ff $vg # Check default, active/inactive values for read_ahead / kernel_read_ahead -- 2.10.2 ++++++ bug-1072624_test-lvmetad_dump-always-timed-out-when-using-nc.patch ++++++ From 6ff44e96eb804f9024bf3f606d207bd863f0e672 Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Wed, 13 Dec 2017 18:53:00 +0800 Subject: [PATCH] test: lvmetad_dump always timed out when using nc lvmetad_dump uses either "socat" or "nc" to communicate with lvmetad. But when using "nc" if "socat" is not available, nc will listen forever by default, causing the testcase timed out. Signed-off-by: Eric Ren <zren(a)suse.com> --- test/lib/aux.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/lib/aux.sh b/test/lib/aux.sh index 6bc7bd47e..4603c1504 100644 --- a/test/lib/aux.sh +++ b/test/lib/aux.sh @@ -243,14 +243,14 @@ lvmetad_talk() { local use=nc if type -p socat >& /dev/null; then use=socat - elif echo | not nc -U "$TESTDIR/lvmetad.socket" ; then + elif echo | not nc -w 1 -U "$TESTDIR/lvmetad.socket" ; then echo "WARNING: Neither socat nor nc -U seems to be available." 1>&2 echo "## failed to contact lvmetad." return 1 fi if test "$use" = nc ; then - nc -U "$TESTDIR/lvmetad.socket" + nc -w 1 -U "$TESTDIR/lvmetad.socket" else socat "unix-connect:$TESTDIR/lvmetad.socket" - fi | tee -a lvmetad-talk.txt -- 2.13.6 ++++++ bug-1114113_metadata-prevent-writing-beyond-metadata-area.patch ++++++ From ab27d5dc2a5c3bf23ab8fed438f1542015dc723d Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 29 Oct 2018 11:06:00 -0500 Subject: [PATCH] metadata: prevent writing beyond metadata area lvm uses a bcache block size of 128K. A bcache block at the end of the metadata area will overlap the PEs from which LVs are allocated. How much depends on alignments. When lvm reads and writes one of these bcache blocks to update VG metadata, it can also be reading and writing PEs that belong to an LV. If these overlapping PEs are being written to by the LV user (e.g. filesystem) at the same time that lvm is modifying VG metadata in the overlapping bcache block, then the user's updates to the PEs can be lost. This patch is a quick hack to prevent lvm from writing past the end of the metadata area. --- lib/device/bcache.c | 79 +++++++++++++++++++++++++++++++++++++++++-- lib/device/bcache.h | 3 ++ lib/format_text/format-text.c | 10 ++++++ lib/label/label.c | 35 ++++++++++++++++++- lib/label/label.h | 2 ++ lib/metadata/mirror.c | 4 +++ 6 files changed, 130 insertions(+), 3 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 531d83b10..62352563a 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -156,6 +156,10 @@ static void _async_destroy(struct io_engine *ioe) dm_free(e); } +static int _last_byte_fd; +static uint64_t _last_byte_offset; +static int _last_byte_sector_size; + static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, sector_t sb, sector_t se, void *data, void *context) { @@ -163,12 +167,53 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, struct iocb *cb_array[1]; struct control_block *cb; struct async_engine *e = _to_async(ioe); + sector_t offset; + sector_t nbytes; + sector_t limit_nbytes; + sector_t extra_nbytes = 0; if (((uintptr_t) data) & e->page_mask) { log_warn("misaligned data buffer"); return false; } + offset = sb << SECTOR_SHIFT; + nbytes = (se - sb) << SECTOR_SHIFT; + + /* + * If bcache block goes past where lvm wants to write, then clamp it. + */ + if ((d == DIR_WRITE) && _last_byte_offset && (fd == _last_byte_fd)) { + if (offset > _last_byte_offset) { + log_error("Limit write at %llu len %llu beyond last byte %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)_last_byte_offset); + return false; + } + + if (offset + nbytes > _last_byte_offset) { + limit_nbytes = _last_byte_offset - offset; + if (limit_nbytes % _last_byte_sector_size) + extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + + if (extra_nbytes) { + log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)(limit_nbytes + extra_nbytes)); + nbytes = limit_nbytes + extra_nbytes; + } else { + log_debug("Limit write at %llu len %llu to len %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes); + nbytes = limit_nbytes; + } + } + } + cb = _cb_alloc(e->cbs, context); if (!cb) { log_warn("couldn't allocate control block"); @@ -179,10 +224,22 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, cb->cb.aio_fildes = (int) fd; cb->cb.u.c.buf = data; - cb->cb.u.c.offset = sb << SECTOR_SHIFT; - cb->cb.u.c.nbytes = (se - sb) << SECTOR_SHIFT; + cb->cb.u.c.offset = offset; + cb->cb.u.c.nbytes = nbytes; cb->cb.aio_lio_opcode = (d == DIR_READ) ? IO_CMD_PREAD : IO_CMD_PWRITE; +#if 0 + if (d == DIR_READ) { + log_debug("io R off %llu bytes %llu", + (unsigned long long)cb->cb.u.c.offset, + (unsigned long long)cb->cb.u.c.nbytes); + } else { + log_debug("io W off %llu bytes %llu", + (unsigned long long)cb->cb.u.c.offset, + (unsigned long long)cb->cb.u.c.nbytes); + } +#endif + cb_array[0] = &cb->cb; do { r = io_submit(e->aio_context, 1, cb_array); @@ -1153,3 +1210,21 @@ bool bcache_invalidate_fd(struct bcache *cache, int fd) //---------------------------------------------------------------- +void bcache_set_last_byte(struct bcache *cache, int fd, uint64_t offset, int sector_size) +{ + _last_byte_fd = fd; + _last_byte_offset = offset; + _last_byte_sector_size = sector_size; + if (!sector_size) + _last_byte_sector_size = 512; +} + +void bcache_unset_last_byte(struct bcache *cache, int fd) +{ + if (_last_byte_fd == fd) { + _last_byte_fd = 0; + _last_byte_offset = 0; + _last_byte_sector_size = 0; + } +} + diff --git a/lib/device/bcache.h b/lib/device/bcache.h index b0aebb49d..cb902ef36 100644 --- a/lib/device/bcache.h +++ b/lib/device/bcache.h @@ -158,6 +158,9 @@ bool bcache_write_bytes(struct bcache *cache, int fd, uint64_t start, size_t len bool bcache_zero_bytes(struct bcache *cache, int fd, uint64_t start, size_t len); bool bcache_set_bytes(struct bcache *cache, int fd, uint64_t start, size_t len, uint8_t val); +void bcache_set_last_byte(struct bcache *cache, int fd, uint64_t offset, int sector_size); +void bcache_unset_last_byte(struct bcache *cache, int fd); + //---------------------------------------------------------------- #endif diff --git a/lib/format_text/format-text.c b/lib/format_text/format-text.c index 5c7b72f8e..4160ba810 100644 --- a/lib/format_text/format-text.c +++ b/lib/format_text/format-text.c @@ -400,10 +400,14 @@ static int _raw_write_mda_header(const struct format_type *fmt, MDA_HEADER_SIZE - sizeof(mdah->checksum_xl))); + dev_set_last_byte(dev, start_byte + MDA_HEADER_SIZE); + if (!dev_write_bytes(dev, start_byte, MDA_HEADER_SIZE, mdah)) { + dev_unset_last_byte(dev); log_error("Failed to write mda header to %s fd %d", dev_name(dev), dev->bcache_fd); return 0; } + dev_unset_last_byte(dev); return 1; } @@ -677,10 +681,13 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, (unsigned long long)(mdac->rlocn.size - new_wrap), (unsigned long long)new_wrap); + dev_set_last_byte(mdac->area.dev, mdac->area.start + mdah->size); + if (!dev_write_bytes(mdac->area.dev, mdac->area.start + mdac->rlocn.offset, (size_t) (mdac->rlocn.size - new_wrap), fidtc->raw_metadata_buf)) { log_error("Failed to write metadata to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); + dev_unset_last_byte(mdac->area.dev); goto out; } @@ -694,10 +701,13 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, (size_t) new_wrap, fidtc->raw_metadata_buf + mdac->rlocn.size - new_wrap)) { log_error("Failed to write metadata wrap to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); + dev_unset_last_byte(mdac->area.dev); goto out; } } + dev_unset_last_byte(mdac->area.dev); + mdac->rlocn.checksum = calc_crc(INITIAL_CRC, (uint8_t *)fidtc->raw_metadata_buf, (uint32_t) (mdac->rlocn.size - new_wrap)); diff --git a/lib/label/label.c b/lib/label/label.c index e5aa2c129..5377847b3 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -173,6 +173,7 @@ int label_write(struct device *dev, struct label *label) { char buf[LABEL_SIZE] __attribute__((aligned(8))); struct label_header *lh = (struct label_header *) buf; + uint64_t offset; int r = 1; if (!label->labeller->ops->write) { @@ -207,11 +208,17 @@ int label_write(struct device *dev, struct label *label) return 0; } - if (!dev_write_bytes(dev, label->sector << SECTOR_SHIFT, LABEL_SIZE, buf)) { + offset = label->sector << SECTOR_SHIFT; + + dev_set_last_byte(dev, offset + LABEL_SIZE); + + if (!dev_write_bytes(dev, offset, LABEL_SIZE, buf)) { log_debug_devs("Failed to write label to %s", dev_name(dev)); r = 0; } + dev_unset_last_byte(dev); + return r; } @@ -1354,9 +1361,12 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) } } + dev_set_last_byte(dev, start + len); + if (!bcache_zero_bytes(scan_bcache, dev->bcache_fd, start, len)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } @@ -1364,9 +1374,11 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) if (!bcache_flush(scan_bcache)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } + dev_unset_last_byte(dev); return true; } @@ -1400,9 +1412,12 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) } } + dev_set_last_byte(dev, start + len); + if (!bcache_set_bytes(scan_bcache, dev->bcache_fd, start, len, val)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } @@ -1410,9 +1425,27 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) if (!bcache_flush(scan_bcache)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } + + dev_unset_last_byte(dev); return true; } +void dev_set_last_byte(struct device *dev, uint64_t offset) +{ + unsigned int phys_block_size = 0; + unsigned int block_size = 0; + + dev_get_block_size(dev, &phys_block_size, &block_size); + + bcache_set_last_byte(scan_bcache, dev->bcache_fd, offset, phys_block_size); +} + +void dev_unset_last_byte(struct device *dev) +{ + bcache_unset_last_byte(scan_bcache, dev->bcache_fd); +} + diff --git a/lib/label/label.h b/lib/label/label.h index 5b83bc734..ae6a4f5f4 100644 --- a/lib/label/label.h +++ b/lib/label/label.h @@ -126,5 +126,7 @@ bool dev_read_bytes(struct device *dev, uint64_t start, size_t len, void *data); bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data); bool dev_write_zeros(struct device *dev, uint64_t start, size_t len); bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val); +void dev_set_last_byte(struct device *dev, uint64_t offset); +void dev_unset_last_byte(struct device *dev); #endif diff --git a/lib/metadata/mirror.c b/lib/metadata/mirror.c index c7d8a9e94..b1dcaa0e2 100644 --- a/lib/metadata/mirror.c +++ b/lib/metadata/mirror.c @@ -302,10 +302,14 @@ static int _write_log_header(struct cmd_context *cmd, struct logical_volume *lv) return 0; } + dev_set_last_byte(dev, sizeof(log_header)); + if (!dev_write_bytes(dev, UINT64_C(0), sizeof(log_header), &log_header)) { + dev_unset_last_byte(dev); log_error("Failed to write log header to %s.", name); return 0; } + dev_unset_last_byte(dev); label_scan_invalidate(dev); -- 2.12.3 ++++++ bug-1122666_devices-drop-open-error-message.patch ++++++ From 559cf0cd1e226baf63a98c39572264fbf5c3f6b4 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Tue, 23 Apr 2019 09:39:42 -0500 Subject: [PATCH] devices: drop open error message This open error is being printed in more common, non-error circumstances than expected. After a number of complaints make it only a debug message. --- lib/device/dev-io.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/lib/device/dev-io.c b/lib/device/dev-io.c index 2a83a9657..6996a44dc 100644 --- a/lib/device/dev-io.c +++ b/lib/device/dev-io.c @@ -572,10 +572,7 @@ int dev_open_flags(struct device *dev, int flags, int direct, int quiet) } } #endif - if (quiet) - log_sys_debug("open", name); - else - log_sys_error("open", name); + log_sys_debug("open", name); dev->flags |= DEV_OPEN_FAILURE; return 0; -- 2.21.0 ++++++ bug-1123327-pvscan-service-use-StartLimitIntervalSec.patch ++++++ From ce79b62bc2fcaddf65e090ec3fef75926cddf82c Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 28 Feb 2019 08:50:37 -0600 Subject: [PATCH] pvscan service: use StartLimitIntervalSec systemd changed the name --- scripts/lvm2_pvscan_systemd_red_hat@.service.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/scripts/lvm2_pvscan_systemd_red_hat@.service.in +++ b/scripts/lvm2_pvscan_systemd_red_hat@.service.in @@ -2,7 +2,7 @@ Description=LVM2 PV scan on device %i Documentation=man:pvscan(8) DefaultDependencies=no -StartLimitInterval=0 +StartLimitIntervalSec=0 BindsTo=dev-block-%i.device Requires=lvm2-lvmetad.socket After=lvm2-lvmetad.socket lvm2-lvmetad.service multipathd.service ++++++ bug-1135984_cache-support-no_discard_passdown.patch ++++++ From adf9bf80a32500b45b37eb24b98fa7c2c933019e Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Wed, 5 Jun 2019 14:31:34 +0200 Subject: [PATCH] cache: support no_discard_passdown Recent kernel version from kernel commit: de7180ff908b2bc0342e832dbdaa9a5f1ecaa33a started to report in cache status line new flag: no_discard_passdown Whenever lvm spots unknown status it reports: Unknown feature in status: So add reconginzing this feature flag and also report this with 'lvs -o+kernel_discards' When no_discard_passdown is found in status 'nopassdown' gets reported for this field (roughly matching what we report for thin-pools). --- WHATS_NEW | 1 + WHATS_NEW_DM | 1 + lib/metadata/lv.c | 5 +++++ libdm/libdevmapper.h | 1 + libdm/libdm-targets.c | 2 ++ 5 files changed, 10 insertions(+) diff --git a/lib/metadata/lv.c b/lib/metadata/lv.c index 2abe42db7..538b292a2 100644 --- a/lib/metadata/lv.c +++ b/lib/metadata/lv.c @@ -245,6 +245,11 @@ char *lvseg_kernel_discards_dup_with_info_and_seg_status(struct dm_pool *mem, co return 0; } s = get_pool_discards_name(d); + } else if (lvdm->seg_status.type == SEG_STATUS_CACHE) { + if (lvdm->seg_status.cache->feature_flags & + DM_CACHE_FEATURE_NO_DISCARD_PASSDOWN) { + s = "nopassdown"; + } } if (!(ret = dm_pool_strdup(mem, s))) { diff --git a/libdm/libdevmapper.h b/libdm/libdevmapper.h index 19032d775..e885f52ae 100644 --- a/libdm/libdevmapper.h +++ b/libdm/libdevmapper.h @@ -1893,6 +1893,7 @@ int dm_tree_node_add_raid_target_with_params_v2(struct dm_tree_node *node, #define DM_CACHE_FEATURE_WRITETHROUGH 0x00000002 #define DM_CACHE_FEATURE_PASSTHROUGH 0x00000004 #define DM_CACHE_FEATURE_METADATA2 0x00000008 /* cache v1.10 */ +#define DM_CACHE_FEATURE_NO_DISCARD_PASSDOWN 0x00000010 struct dm_config_node; /* diff --git a/libdm/libdm-targets.c b/libdm/libdm-targets.c index 876678943..d0a8b4332 100644 --- a/libdm/libdm-targets.c +++ b/libdm/libdm-targets.c @@ -296,6 +296,8 @@ int dm_get_status_cache(struct dm_pool *mem, const char *params, s->feature_flags |= DM_CACHE_FEATURE_PASSTHROUGH; else if (!strncmp(p, "metadata2 ", 10)) s->feature_flags |= DM_CACHE_FEATURE_METADATA2; + else if (!strncmp(p, "no_discard_passdown ", 20)) + s->feature_flags |= DM_CACHE_FEATURE_NO_DISCARD_PASSDOWN; else log_error("Unknown feature in status: %s", params); -- 2.12.3 ++++++ bug-1137296_pvremove-vgextend-fix-using-device-aliases-with-lvmetad.patch ++++++ From b13ebfa4c289a5bc6eb4f8ba26126db8e6d78296 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 26 Jun 2019 16:03:42 -0500 Subject: [PATCH] pvremove/vgextend: fix using device aliases with lvmetad These commands were looking for the requested device alias before dev_cache_scan had created the list of devs on the system, so they would fail and report the dev wasn't found. --- tools/toollib.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tools/toollib.c b/tools/toollib.c index 3221e5f1ca..1b01cccfa2 100644 --- a/tools/toollib.c +++ b/tools/toollib.c @@ -5543,6 +5543,15 @@ int pvcreate_each_device(struct cmd_context *cmd, lvmcache_label_scan(cmd); + /* + * When using lvmetad, we want to do a dev cache scan here (if not done + * already) so that the dev_cache_get just below will be able to find + * device aliases. When not using lvmetad, the label_scan just above + * has done dev_cache_scan, and this will not be run. + */ + if (!dev_cache_has_scanned()) + dev_cache_scan(); + /* * Translate arg names into struct device's. */ -- 2.21.0 ++++++ bug-1145231_apply-obtain_device_list_from_udev-to-all-libudev-us.patch ++++++ From 57cde6063f3073e73542c5f7f59d0269ad9b703b Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Tue, 5 Feb 2019 10:15:40 -0600 Subject: [PATCH] apply obtain_device_list_from_udev to all libudev usage udev_dev_is_md_component and udev_dev_is_mpath_component are not used for obtaining the device list, but they still use libudev for device info. When there are problems with udev, these functions can get stuck. So, use the existing obtain_device_list_from_udev config setting to also control whether these "is component" functions are used, which gives us a way to avoid using libudev entirely when it's causing problems. --- lib/device/dev-type.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/device/dev-type.c b/lib/device/dev-type.c index 33ebb73b2c..e909c0f067 100644 --- a/lib/device/dev-type.c +++ b/lib/device/dev-type.c @@ -1063,6 +1063,9 @@ int udev_dev_is_mpath_component(struct device *dev) const char *value; int ret = 0; + if (!obtain_device_list_from_udev()) + return 0; + if (!(udev_device = _udev_get_dev(dev))) return 0; @@ -1092,6 +1095,9 @@ int udev_dev_is_md_component(struct device *dev) const char *value; int ret = 0; + if (!obtain_device_list_from_udev()) + return 0; + if (!(udev_device = _udev_get_dev(dev))) return 0; -- 2.21.0 ++++++ bug-1145231_lvmetad-improve-scan-for-pvscan-all.patch ++++++ From c527a0cbfc391645d30407d2dc4a30275c6472f1 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 27 Aug 2018 11:15:35 -0500 Subject: [PATCH] lvmetad: improve scan for pvscan all For 'pvscan --cache' avoid using dev_iter in the loop after the label_scan by passing the necessary devs back from the label_scan for the continued pvscan. The dev_iter functions reapply the filters which will trigger more io when we don't need or want it. With many devs, incidental opens from the filters (not controlled by the label scan) can lead to too many open files. --- lib/cache/lvmetad.c | 34 ++++++++++++------------- lib/label/label.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/label/label.h | 1 + 3 files changed, 91 insertions(+), 17 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index a1ab41aab..acbb52e54 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -2322,8 +2322,8 @@ bad: int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) { - struct dev_iter *iter; - struct device *dev; + struct device_list *devl, *devl2; + struct dm_list scan_devs; daemon_reply reply; char *future_token; const char *reason; @@ -2339,6 +2339,8 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) } retry: + dm_list_init(&scan_devs); + /* * If another update is in progress, delay to allow it to finish, * rather than interrupting it with our own update. @@ -2348,7 +2350,7 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) replacing_other_update = 1; } - label_scan(cmd); + label_scan_pvscan_all(cmd, &scan_devs); lvmcache_pvscan_duplicate_check(cmd); @@ -2357,19 +2359,14 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) return 0; } - log_verbose("Scanning all devices to update lvmetad."); - - if (!(iter = dev_iter_create(cmd->lvmetad_filter, 1))) { - log_error("dev_iter creation failed"); - return 0; - } + log_verbose("Scanning metadata from %d devices to update lvmetad.", + dm_list_size(&scan_devs)); future_token = _lvmetad_token; _lvmetad_token = (char *) LVMETAD_TOKEN_UPDATE_IN_PROGRESS; if (!_token_update(&replaced_update)) { log_error("Failed to update lvmetad which had an update in progress."); - dev_iter_destroy(iter); _lvmetad_token = future_token; return 0; } @@ -2385,12 +2382,10 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) if (do_wait && !retries) { retries = 1; log_warn("WARNING: lvmetad update in progress, retrying update."); - dev_iter_destroy(iter); _lvmetad_token = future_token; goto retry; } log_warn("WARNING: lvmetad update in progress, skipping update."); - dev_iter_destroy(iter); _lvmetad_token = future_token; return 0; } @@ -2404,15 +2399,22 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) was_silent = silent_mode(); init_silent(1); - while ((dev = dev_iter_get(iter))) { + dm_list_iterate_items_safe(devl, devl2, &scan_devs) { if (sigint_caught()) { ret = 0; stack; break; } - if (!lvmetad_pvscan_single(cmd, dev, NULL, NULL)) { - ret = 0; + dm_list_del(&devl->list); + + ret = lvmetad_pvscan_single(cmd, devl->dev, NULL, NULL); + + label_scan_invalidate(devl->dev); + + dm_free(devl); + + if (!ret) { stack; break; } @@ -2420,8 +2422,6 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) init_silent(was_silent); - dev_iter_destroy(iter); - _lvmetad_token = future_token; /* diff --git a/lib/label/label.c b/lib/label/label.c index bafa54366..837033c4b 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -876,6 +876,79 @@ int label_scan(struct cmd_context *cmd) return 1; } +int label_scan_pvscan_all(struct cmd_context *cmd, struct dm_list *scan_devs) +{ + struct dm_list all_devs; + struct dev_iter *iter; + struct device_list *devl, *devl2; + struct device *dev; + + log_debug_devs("Finding devices to scan"); + + dm_list_init(&all_devs); + + /* + * Iterate through all the devices in dev-cache (block devs that appear + * under /dev that could possibly hold a PV and are not excluded by + * filters). Read each to see if it's an lvm device, and if so + * populate lvmcache with some basic info about the device and the VG + * on it. This info will be used by the vg_read() phase of the + * command. + */ + dev_cache_scan(); + + if (!(iter = dev_iter_create(cmd->lvmetad_filter, 0))) { + log_error("Scanning failed to get devices."); + return 0; + } + + while ((dev = dev_iter_get(iter))) { + if (!(devl = dm_zalloc(sizeof(*devl)))) + return 0; + devl->dev = dev; + dm_list_add(&all_devs, &devl->list); + + /* + * label_scan should not generally be called a second time, + * so this will usually not be true. + */ + if (_in_bcache(dev)) { + bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _scan_dev_close(dev); + } + }; + dev_iter_destroy(iter); + + log_debug_devs("Found %d devices to scan", dm_list_size(&all_devs)); + + if (!scan_bcache) { + if (!_setup_bcache(dm_list_size(&all_devs))) + return 0; + } + + _scan_list(cmd, cmd->lvmetad_filter, &all_devs, NULL); + + dm_list_iterate_items_safe(devl, devl2, &all_devs) { + dm_list_del(&devl->list); + + /* + * If this device is lvm's then, return it to pvscan + * to do the further pvscan. (We could have _scan_list + * just set a result in devl indicating the result, but + * instead we're just checking indirectly if _scan_list + * saved lvmcache info for the dev which also means it's + * an lvm device.) + */ + + if (lvmcache_has_dev_info(devl->dev)) + dm_list_add(scan_devs, &devl->list); + else + dm_free(devl); + } + + return 1; +} + /* * Scan and cache lvm data from the listed devices. If a device is already * scanned and cached, this replaces the previously cached lvm data for the diff --git a/lib/label/label.h b/lib/label/label.h index 5ed8bc86b..5b83bc734 100644 --- a/lib/label/label.h +++ b/lib/label/label.h @@ -116,6 +116,7 @@ void label_scan_confirm(struct device *dev); int label_scan_setup_bcache(void); int label_scan_open(struct device *dev); int label_scan_open_excl(struct device *dev); +int label_scan_pvscan_all(struct cmd_context *cmd, struct dm_list *scan_devs); /* * Wrappers around bcache equivalents. -- 2.12.3 ++++++ bug-1145231_pvscan-lvmetad-use-full-md-filter-when-md-1.0-device.patch ++++++ From a01e1fec0fe7c2fa61577c0e636e907cde7279ea Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 29 Nov 2018 14:06:20 -0600 Subject: [PATCH] pvscan lvmetad: use full md filter when md 1.0 devices are present Apply the same logic to pvscan/lvmetad that was added to the non-lvmetad label_scan in commit 3fd75d1b: scan: use full md filter when md 1.0 devices are present Before scanning, check if any of the devs on the system are md 0.90/1.0, and if so make the scan read both the start and the end of the device so that the components of those md versions can be ignored. --- tools/pvscan.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tools/pvscan.c b/tools/pvscan.c index 2915db599..3755684d2 100644 --- a/tools/pvscan.c +++ b/tools/pvscan.c @@ -18,6 +18,8 @@ #include "lvmetad.h" #include "lvmcache.h" +extern int use_full_md_check; + struct pvscan_params { int new_pvs_found; int pvs_found; @@ -302,6 +304,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) struct dm_list found_vgnames; struct device *dev; struct device_list *devl; + struct dev_iter *iter; const char *pv_name; const char *reason = NULL; int32_t major = -1; @@ -443,6 +446,22 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) /* Creates a list of dev names from /dev, sysfs, etc; does not read any. */ dev_cache_scan(); + /* See the same check in label_scan() to handle md 0.9/1.0 components. */ + if (!(iter = dev_iter_create(cmd->full_filter, 0))) { + log_error("Scanning failed to get devices."); + return 0; + } + while ((dev = dev_iter_get(iter))) { + if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) { + cmd->use_full_md_check = 1; + use_full_md_check = 1; + log_debug("Found md with end superblock %s", dev_name(dev)); + } + } + dev_iter_destroy(iter); + if (!use_full_md_check) + log_debug("No md devs with end superblock"); + dm_list_init(&single_devs); while (argc--) { -- 2.12.3 ++++++ bug-1145231_pvscan-lvmetad-use-udev-info-to-improve-md-component.patch ++++++ From a188b1e513ed5ca0f5f3702c823490f5610d4495 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 30 Nov 2018 16:32:32 -0600 Subject: [PATCH] pvscan lvmetad: use udev info to improve md component detection When no md devs are started, pvscan will only scan the start of an md component, and if it has a superblock at the end may not exclude it. udev may already have info identifying it as an md component, so use that. --- lib/device/dev-md.c | 14 ++++++++++-- lib/device/dev-type.c | 62 +++++++++++++++++++++++++++++++++++++++++---------- lib/device/dev-type.h | 1 + lib/label/label.c | 6 +++++ tools/pvscan.c | 3 ++- 5 files changed, 71 insertions(+), 15 deletions(-) diff --git a/lib/device/dev-md.c b/lib/device/dev-md.c index 185499baf..972850726 100644 --- a/lib/device/dev-md.c +++ b/lib/device/dev-md.c @@ -190,14 +190,24 @@ out: int dev_is_md(struct device *dev, uint64_t *offset_found, int full) { + int ret; /* * If non-native device status source is selected, use it * only if offset_found is not requested as this * information is not in udev db. */ - if ((dev->ext.src == DEV_EXT_NONE) || offset_found) - return _native_dev_is_md(dev, offset_found, full); + if ((dev->ext.src == DEV_EXT_NONE) || offset_found) { + ret = _native_dev_is_md(dev, offset_found, full); + + if (!full) { + if (!ret || (ret == -EAGAIN)) { + if (udev_dev_is_md_component(dev)) + return 1; + } + } + return ret; + } if (dev->ext.src == DEV_EXT_UDEV) return _udev_dev_is_md(dev); diff --git a/lib/device/dev-type.c b/lib/device/dev-type.c index af4b40760..33ebb73b2 100644 --- a/lib/device/dev-type.c +++ b/lib/device/dev-type.c @@ -1004,25 +1004,23 @@ int dev_is_rotational(struct dev_types *dt, struct device *dev) * failed already due to timeout in udev - in both cases the * udev_device_get_is_initialized returns 0. */ -#define UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT 100 -#define UDEV_DEV_IS_MPATH_COMPONENT_USLEEP 100000 +#define UDEV_DEV_IS_COMPONENT_ITERATION_COUNT 100 +#define UDEV_DEV_IS_COMPONENT_USLEEP 100000 -int udev_dev_is_mpath_component(struct device *dev) +static struct udev_device *_udev_get_dev(struct device *dev) { struct udev *udev_context = udev_get_library_context(); struct udev_device *udev_device = NULL; - const char *value; int initialized = 0; unsigned i = 0; - int ret = 0; if (!udev_context) { log_warn("WARNING: No udev context available to check if device %s is multipath component.", dev_name(dev)); - return 0; + return NULL; } while (1) { - if (i >= UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT) + if (i >= UDEV_DEV_IS_COMPONENT_ITERATION_COUNT) break; if (udev_device) @@ -1030,7 +1028,7 @@ int udev_dev_is_mpath_component(struct device *dev) if (!(udev_device = udev_device_new_from_devnum(udev_context, 'b', dev->dev))) { log_warn("WARNING: Failed to get udev device handler for device %s.", dev_name(dev)); - return 0; + return NULL; } #ifdef HAVE_LIBUDEV_UDEV_DEVICE_GET_IS_INITIALIZED @@ -1042,19 +1040,32 @@ int udev_dev_is_mpath_component(struct device *dev) #endif log_debug("Device %s not initialized in udev database (%u/%u, %u microseconds).", dev_name(dev), - i + 1, UDEV_DEV_IS_MPATH_COMPONENT_ITERATION_COUNT, - i * UDEV_DEV_IS_MPATH_COMPONENT_USLEEP); + i + 1, UDEV_DEV_IS_COMPONENT_ITERATION_COUNT, + i * UDEV_DEV_IS_COMPONENT_USLEEP); - usleep(UDEV_DEV_IS_MPATH_COMPONENT_USLEEP); + usleep(UDEV_DEV_IS_COMPONENT_USLEEP); i++; } if (!initialized) { log_warn("WARNING: Device %s not initialized in udev database even after waiting %u microseconds.", - dev_name(dev), i * UDEV_DEV_IS_MPATH_COMPONENT_USLEEP); + dev_name(dev), i * UDEV_DEV_IS_COMPONENT_USLEEP); goto out; } +out: + return udev_device; +} + +int udev_dev_is_mpath_component(struct device *dev) +{ + struct udev_device *udev_device; + const char *value; + int ret = 0; + + if (!(udev_device = _udev_get_dev(dev))) + return 0; + value = udev_device_get_property_value(udev_device, DEV_EXT_UDEV_BLKID_TYPE); if (value && !strcmp(value, DEV_EXT_UDEV_BLKID_TYPE_MPATH)) { log_debug("Device %s is multipath component based on blkid variable in udev db (%s=\"%s\").", @@ -1074,6 +1085,28 @@ out: udev_device_unref(udev_device); return ret; } + +int udev_dev_is_md_component(struct device *dev) +{ + struct udev_device *udev_device; + const char *value; + int ret = 0; + + if (!(udev_device = _udev_get_dev(dev))) + return 0; + + value = udev_device_get_property_value(udev_device, DEV_EXT_UDEV_BLKID_TYPE); + if (value && !strcmp(value, DEV_EXT_UDEV_BLKID_TYPE_SW_RAID)) { + log_debug("Device %s is md raid component based on blkid variable in udev db (%s=\"%s\").", + dev_name(dev), DEV_EXT_UDEV_BLKID_TYPE, value); + ret = 1; + goto out; + } +out: + udev_device_unref(udev_device); + return ret; +} + #else int udev_dev_is_mpath_component(struct device *dev) @@ -1081,4 +1114,9 @@ int udev_dev_is_mpath_component(struct device *dev) return 0; } +int udev_dev_is_md_component(struct device *dev) +{ + return 0; +} + #endif diff --git a/lib/device/dev-type.h b/lib/device/dev-type.h index f629a0278..264438339 100644 --- a/lib/device/dev-type.h +++ b/lib/device/dev-type.h @@ -62,6 +62,7 @@ int dev_is_swap(struct device *dev, uint64_t *signature, int full); int dev_is_luks(struct device *dev, uint64_t *signature, int full); int dasd_is_cdl_formatted(struct device *dev); int udev_dev_is_mpath_component(struct device *dev); +int udev_dev_is_md_component(struct device *dev); int dev_is_lvm1(struct device *dev, char *buf, int buflen); int dev_is_pool(struct device *dev, char *buf, int buflen); diff --git a/lib/label/label.c b/lib/label/label.c index b26ff3370..e01608d2c 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -957,6 +957,12 @@ int label_scan_pvscan_all(struct cmd_context *cmd, struct dm_list *scan_devs) bcache_invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } + + if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) { + cmd->use_full_md_check = 1; + use_full_md_check = 1; + log_debug("Found md component in sysfs with end superblock %s", dev_name(dev)); + } }; dev_iter_destroy(iter); diff --git a/tools/pvscan.c b/tools/pvscan.c index 3755684d2..877b6b2db 100644 --- a/tools/pvscan.c +++ b/tools/pvscan.c @@ -455,7 +455,8 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) { cmd->use_full_md_check = 1; use_full_md_check = 1; - log_debug("Found md with end superblock %s", dev_name(dev)); + log_debug("Found md component in sysfs with end superblock %s", dev_name(dev)); + break; } } dev_iter_destroy(iter); -- 2.12.3 ++++++ bug-1145231_scan-enable-full-md-filter-when-md-1.0-devices-are-p.patch ++++++ From e7bb50880901a4462e350ce0d272a63aa8440781 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 18 Oct 2018 11:32:32 -0500 Subject: [PATCH] scan: enable full md filter when md 1.0 devices are present The previous commit de2863739f2ea17d89d0e442379109f967b5919d scan: use full md filter when md 1.0 devices are present needs the use_full_md_check flag in the md filter, but the cmd struct is not available when the filter is run, so that commit wasn't working. Fix this by setting the flag in a global variable. (This was fixed in the master branch with commit 8eab37593 in which the cmd struct was passed to the filters, but it was an intrusive change, so this commit is using the less intrusive global variable.) --- lib/filters/filter-md.c | 33 +++++++-------------------------- lib/label/label.c | 13 ++++++++++++- 2 files changed, 19 insertions(+), 27 deletions(-) diff --git a/lib/filters/filter-md.c b/lib/filters/filter-md.c index ad5b8e4e8..e03ff5059 100644 --- a/lib/filters/filter-md.c +++ b/lib/filters/filter-md.c @@ -16,6 +16,9 @@ #include "lib.h" #include "filter.h" +/* See label.c comment about this hack. */ +extern int use_full_md_check; + #ifdef __linux__ #define MSG_SKIPPING "%s: Skipping md component device" @@ -80,7 +83,7 @@ * that will not pass. */ -static int _passes_md_filter(struct device *dev, int full) +static int _passes_md_filter(struct dev_filter *f, struct device *dev) { int ret; @@ -91,7 +94,7 @@ static int _passes_md_filter(struct device *dev, int full) if (!md_filtering()) return 1; - ret = dev_is_md(dev, NULL, full); + ret = dev_is_md(dev, NULL, use_full_md_check); if (ret == -EAGAIN) { /* let pass, call again after scan */ @@ -104,6 +107,7 @@ static int _passes_md_filter(struct device *dev, int full) return 1; if (ret == 1) { + log_debug_devs("md filter full %d excluding md component %s", use_full_md_check, dev_name(dev)); if (dev->ext.src == DEV_EXT_NONE) log_debug_devs(MSG_SKIPPING, dev_name(dev)); else @@ -121,18 +125,6 @@ static int _passes_md_filter(struct device *dev, int full) return 1; } -static int _passes_md_filter_lite(struct dev_filter *f __attribute__((unused)), - struct device *dev) -{ - return _passes_md_filter(dev, 0); -} - -static int _passes_md_filter_full(struct dev_filter *f __attribute__((unused)), - struct device *dev) -{ - return _passes_md_filter(dev, 1); -} - static void _destroy(struct dev_filter *f) { if (f->use_count) @@ -150,18 +142,7 @@ struct dev_filter *md_filter_create(struct cmd_context *cmd, struct dev_types *d return NULL; } - /* - * FIXME: for commands that want a full md check (pvcreate, vgcreate, - * vgextend), we do an extra read at the end of every device that the - * filter looks at. This isn't necessary; we only need to do the full - * md check on the PVs that these commands are trying to use. - */ - - if (cmd->use_full_md_check) - f->passes_filter = _passes_md_filter_full; - else - f->passes_filter = _passes_md_filter_lite; - + f->passes_filter = _passes_md_filter; f->destroy = _destroy; f->use_count = 0; f->private = dt; diff --git a/lib/label/label.c b/lib/label/label.c index e76ddd4b2..e5aa2c129 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -27,6 +27,7 @@ #include <unistd.h> #include <sys/time.h> +int use_full_md_check; /* FIXME Allow for larger labels? Restricted to single sector currently */ @@ -868,8 +869,18 @@ int label_scan(struct cmd_context *cmd) * devs in 'pvs', which is a pretty harmless effect from a * pretty uncommon situation. */ - if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) + if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) { cmd->use_full_md_check = 1; + + /* This is a hack because 'cmd' is not passed + into the filters so we can't check the flag + in the cmd struct. The master branch has + changed the filters in commit 8eab37593eccb + to accept cmd, but it's a complex change + that I'm trying to avoid in the stable branch. */ + + use_full_md_check = 1; + } }; dev_iter_destroy(iter); -- 2.12.3 ++++++ bug-1145231_scan-md-metadata-version-0.90-is-at-the-end-of-disk.patch ++++++ From 0e42ebd6d4012d210084a9ccf8d76f853726de3c Mon Sep 17 00:00:00 2001 From: Peter Rajnoha <prajnoha(a)redhat.com> Date: Thu, 29 Nov 2018 11:51:05 -0600 Subject: [PATCH] scan: md metadata version 0.90 is at the end of disk commit de28637 scan: use full md filter when md 1.0 devices are present missed the fact that md superblock version 0.90 also puts metadata at the end of the device, so the full md filter needs to be used when either 0.90 or 1.0 is present. --- lib/device/dev-md.c | 2 +- lib/filters/filter-md.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/device/dev-md.c b/lib/device/dev-md.c index 7196dc007..185499baf 100644 --- a/lib/device/dev-md.c +++ b/lib/device/dev-md.c @@ -422,7 +422,7 @@ int dev_is_md_with_end_superblock(struct dev_types *dt, struct device *dev) log_very_verbose("Device %s %s is %s.", dev_name(dev), attribute, version_string); - if (!strcmp(version_string, "1.0")) + if (!strcmp(version_string, "1.0") || !strcmp(version_string, "0.90")) return 1; return 0; } diff --git a/lib/filters/filter-md.c b/lib/filters/filter-md.c index e03ff5059..2011e1d5a 100644 --- a/lib/filters/filter-md.c +++ b/lib/filters/filter-md.c @@ -47,7 +47,7 @@ extern int use_full_md_check; * 3. use udev to detect components * * mode 1 will not detect and exclude components of md devices - * that use superblock version 1.0 which is at the end of the device. + * that use superblock version 0.9 or 1.0 which is at the end of the device. * * mode 2 will detect these, but mode 2 doubles the i/o done by label * scan, since there's a read at both the start and end of every device. @@ -60,11 +60,11 @@ extern int use_full_md_check; * * - the command is pvcreate/vgcreate/vgextend, which format new * devices, and if the user ran these commands on a component - * device of an md device 1.0, then it would cause problems. + * device of an md device 0.9 or 1.0, then it would cause problems. * FIXME: this would only really need to scan the end of the * devices being formatted, not all devices. * - * - it sees an md device on the system using version 1.0. + * - it sees an md device on the system using version 0.9 or 1.0. * The point of this is just to avoid displaying md components * from the 'pvs' command. * FIXME: the cost (double i/o) may not be worth the benefit -- 2.12.3 ++++++ bug-1145231_scan-use-full-md-filter-when-md-1.0-devices-are-pres.patch ++++++ From de2863739f2ea17d89d0e442379109f967b5919d Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 15 Jun 2018 11:42:10 -0500 Subject: [PATCH] scan: use full md filter when md 1.0 devices are present The md filter can operate in two native modes: - normal: reads only the start of each device - full: reads both the start and end of each device md 1.0 devices place the superblock at the end of the device, so components of this version will only be identified and excluded when lvm uses the full md filter. Previously, the full md filter was only used in commands that could write to the device. Now, the full md filter is also applied when there is an md 1.0 device present on the system. This means the 'pvs' command can avoid displaying md 1.0 components (at the cost of doubling the i/o to every device on the system.) (The md filter can operate in a third mode, using udev, but this is disabled by default because there have been problems with reliability of the info returned from udev.) --- lib/cache/lvmcache.c | 2 +- lib/device/dev-md.c | 27 ++++++++++---- lib/device/dev-type.h | 1 + lib/filters/filter-md.c | 74 +++++++++++++++++++------------------- lib/label/label.c | 14 ++++++++ test/shell/pvcreate-md-fake-hdr.sh | 3 +- 6 files changed, 75 insertions(+), 46 deletions(-) diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c index 3e681a2ba..a2ee0cd43 100644 --- a/lib/cache/lvmcache.c +++ b/lib/cache/lvmcache.c @@ -998,7 +998,7 @@ int lvmcache_dev_is_unchosen_duplicate(struct device *dev) * unused_duplicate_devs list, and restrict what we allow done with it. * * In the case of md components, we usually filter these out in filter-md, - * but in the special case of md superblocks <= 1.0 where the superblock + * but in the special case of md superblock version 1.0 where the superblock * is at the end of the device, filter-md doesn't always eliminate them * first, so we eliminate them here. * diff --git a/lib/device/dev-md.c b/lib/device/dev-md.c index f5a736fc2..7196dc007 100644 --- a/lib/device/dev-md.c +++ b/lib/device/dev-md.c @@ -142,13 +142,6 @@ static int _native_dev_is_md(struct device *dev, uint64_t *offset_found, int ful * command if it should do a full check (cmd->use_full_md_check), * and set it for commands that could possibly write to an md dev * (pvcreate/vgcreate/vgextend). - * - * For old md versions with magic numbers at the end of devices, - * the md dev components won't be filtered out here when full is 0, - * so they will be scanned, and appear as duplicate PVs in lvmcache. - * The md device itself will be chosen as the primary duplicate, - * and the components are dropped from the list of duplicates in, - * i.e. a kind of post-scan filtering. */ if (!full) { sb_offset = 0; @@ -414,6 +407,26 @@ unsigned long dev_md_stripe_width(struct dev_types *dt, struct device *dev) return stripe_width_sectors; } +int dev_is_md_with_end_superblock(struct dev_types *dt, struct device *dev) +{ + char version_string[MD_MAX_SYSFS_SIZE]; + const char *attribute = "metadata_version"; + + if (MAJOR(dev->dev) != dt->md_major) + return 0; + + if (_md_sysfs_attribute_scanf(dt, dev, attribute, + "%s", &version_string) != 1) + return -1; + + log_very_verbose("Device %s %s is %s.", + dev_name(dev), attribute, version_string); + + if (!strcmp(version_string, "1.0")) + return 1; + return 0; +} + #else int dev_is_md(struct device *dev __attribute__((unused)), diff --git a/lib/device/dev-type.h b/lib/device/dev-type.h index 843e2545b..f629a0278 100644 --- a/lib/device/dev-type.h +++ b/lib/device/dev-type.h @@ -76,6 +76,7 @@ int wipe_known_signatures(struct cmd_context *cmd, struct device *dev, const cha /* Type-specific device properties */ unsigned long dev_md_stripe_width(struct dev_types *dt, struct device *dev); +int dev_is_md_with_end_superblock(struct dev_types *dt, struct device *dev); /* Partitioning */ int major_max_partitions(struct dev_types *dt, int major); diff --git a/lib/filters/filter-md.c b/lib/filters/filter-md.c index ab97b5946..ad5b8e4e8 100644 --- a/lib/filters/filter-md.c +++ b/lib/filters/filter-md.c @@ -29,43 +29,43 @@ * * (This is assuming lvm.conf md_component_detection=1.) * - * If lvm does *not* ignore the components, then lvm will read lvm - * labels from the md dev and from the component devs, and will see - * them all as duplicates of each other. LVM duplicate resolution - * will then kick in and keep the md dev around to use and ignore - * the components. - * - * It is better to exclude the components as early as possible during - * lvm processing, ideally before lvm even looks for labels on the - * components, so that duplicate resolution can be avoided. There are - * a number of ways that md components can be excluded earlier than - * the duplicate resolution phase: - * - * - When external_device_info_source="udev", lvm discovers a device is - * an md component by asking udev during the initial filtering phase. - * However, lvm's default is to not use udev for this. The - * alternative is "native" detection in which lvm tries to detect - * md components itself. - * - * - When using native detection, lvm's md filter looks for the md - * superblock at the start of devices. It will see the md superblock - * on the components, exclude them in the md filter, and avoid - * handling them later in duplicate resolution. - * - * - When using native detection, lvm's md filter will not detect - * components when the md device has an older superblock version that - * places the superblock at the end of the device. This case will - * fall back to duplicate resolution to exclude components. - * - * A variation of the description above occurs for lvm commands that - * intend to create new PVs on devices (pvcreate, vgcreate, vgextend). - * For these commands, the native md filter also reads the end of all - * devices to check for the odd md superblocks. - * - * (The reason that external_device_info_source is not set to udev by - * default is that there have be issues with udev not being promptly - * or reliably updated about md state changes, causing the udev info - * that lvm uses to be occasionally wrong.) + * If lvm does *not* ignore the components, then lvm may read lvm + * labels from the component devs and potentially the md dev, + * which can trigger duplicate detection, and/or cause lvm to display + * md components as PVs rather than ignoring them. + * + * If scanning md componenents causes duplicates to be seen, then + * the lvm duplicate resolution will exclude the components. + * + * The lvm md filter has three modes: + * + * 1. look for md superblock at the start of the device + * 2. look for md superblock at the start and end of the device + * 3. use udev to detect components + * + * mode 1 will not detect and exclude components of md devices + * that use superblock version 1.0 which is at the end of the device. + * + * mode 2 will detect these, but mode 2 doubles the i/o done by label + * scan, since there's a read at both the start and end of every device. + * + * mode 3 is used when external_device_info_source="udev". It does + * not require any io from lvm, but this mode is not used by default + * because there have been problems getting reliable info from udev. + * + * lvm uses mode 2 when: + * + * - the command is pvcreate/vgcreate/vgextend, which format new + * devices, and if the user ran these commands on a component + * device of an md device 1.0, then it would cause problems. + * FIXME: this would only really need to scan the end of the + * devices being formatted, not all devices. + * + * - it sees an md device on the system using version 1.0. + * The point of this is just to avoid displaying md components + * from the 'pvs' command. + * FIXME: the cost (double i/o) may not be worth the benefit + * (not showing md components). */ /* diff --git a/lib/label/label.c b/lib/label/label.c index 837033c4b..e76ddd4b2 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -856,6 +856,20 @@ int label_scan(struct cmd_context *cmd) bcache_invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } + + /* + * When md devices exist that use the old superblock at the + * end of the device, then in order to detect and filter out + * the component devices of those md devs, we need to enable + * the full md filter which scans both the start and the end + * of every device. This doubles the amount of scanning i/o, + * which we want to avoid. FIXME: it may not be worth the + * cost of double i/o just to avoid displaying md component + * devs in 'pvs', which is a pretty harmless effect from a + * pretty uncommon situation. + */ + if (dev_is_md_with_end_superblock(cmd->dev_types, dev)) + cmd->use_full_md_check = 1; }; dev_iter_destroy(iter); diff --git a/test/shell/pvcreate-md-fake-hdr.sh b/test/shell/pvcreate-md-fake-hdr.sh index b89fe4377..4c9ac7cbc 100644 --- a/test/shell/pvcreate-md-fake-hdr.sh +++ b/test/shell/pvcreate-md-fake-hdr.sh @@ -89,6 +89,7 @@ sleep 1 # (when mdadm supports repair) if mdadm --action=repair "$mddev" ; then sleep 1 + pvscan -vvvv # should be showing correctly PV3 & PV4 - pvs + pvs -vvvv "$dev3" "$dev4" fi -- 2.12.3 ++++++ bug-1149408_01-vgcreate-vgextend-restrict-PVs-with-mixed-block-size.patch ++++++ From a57b92dec396ac79c57e02e54c04e205aa85f084 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 1 Aug 2019 10:06:47 -0500 Subject: [PATCH] vgcreate/vgextend: restrict PVs with mixed block sizes Avoid having PVs with different logical block sizes in the same VG. This prevents LVs from having mixed block sizes, which can produce file system errors. The new config setting devices/allow_mixed_block_sizes (default 0) can be changed to 1 to return to the unrestricted mode. (cherry picked from commit 0404539edb25e4a9d3456bb3e6b402aa2767af6b) Conflicts: tools/lvmcmdline.c tools/toollib.c --- lib/commands/toolcontext.h | 1 + lib/config/config_settings.h | 5 +++++ lib/metadata/metadata-exported.h | 1 + lib/metadata/metadata.c | 44 +++++++++++++++++++++++++++++++++++++ tools/lvmcmdline.c | 2 ++ tools/toollib.c | 47 ++++++++++++++++++++++++++++++++++++++++ tools/vgcreate.c | 2 ++ 7 files changed, 102 insertions(+) diff --git a/lib/commands/toolcontext.h b/lib/commands/toolcontext.h index 4b2a079..497f4bd 100644 --- a/lib/commands/toolcontext.h +++ b/lib/commands/toolcontext.h @@ -155,6 +155,7 @@ struct cmd_context { unsigned include_shared_vgs:1; /* report/display cmds can reveal lockd VGs */ unsigned include_active_foreign_vgs:1; /* cmd should process foreign VGs with active LVs */ unsigned vg_read_print_access_error:1; /* print access errors from vg_read */ + unsigned allow_mixed_block_sizes:1; unsigned force_access_clustered:1; unsigned lockd_gl_disable:1; unsigned lockd_vg_disable:1; diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 0e81252..7981d5d 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -470,6 +470,11 @@ cfg(devices_allow_changes_with_duplicate_pvs_CFG, "allow_changes_with_duplicate_ "Enabling this setting allows the VG to be used as usual even with\n" "uncertain devices.\n") +cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 0, vsn(2, 3, 6), NULL, 0, NULL, + "Allow PVs in the same VG with different logical block sizes.\n" + "When allowed, the user is responsible to ensure that an LV is\n" + "using PVs with matching block sizes when necessary.\n") + cfg_array(allocation_cling_tag_list_CFG, "cling_tag_list", allocation_CFG_SECTION, CFG_DEFAULT_UNDEFINED, CFG_TYPE_STRING, NULL, vsn(2, 2, 77), NULL, 0, NULL, "Advise LVM which PVs to use when searching for new space.\n" "When searching for free space to extend an LV, the 'cling' allocation\n" diff --git a/lib/metadata/metadata-exported.h b/lib/metadata/metadata-exported.h index 75caba1..9c24132 100644 --- a/lib/metadata/metadata-exported.h +++ b/lib/metadata/metadata-exported.h @@ -591,6 +591,7 @@ struct pvcreate_params { unsigned is_remove : 1; /* is removing PVs, not creating */ unsigned preserve_existing : 1; unsigned check_failed : 1; + unsigned check_consistent_block_size : 1; }; struct lvresize_params { diff --git a/lib/metadata/metadata.c b/lib/metadata/metadata.c index cb38f66..81a6029 100644 --- a/lib/metadata/metadata.c +++ b/lib/metadata/metadata.c @@ -699,12 +699,40 @@ int vg_extend_each_pv(struct volume_group *vg, struct pvcreate_params *pp) { struct pv_list *pvl; unsigned int max_phys_block_size = 0; + unsigned int physical_block_size, logical_block_size; + unsigned int prev_lbs = 0; + int inconsistent_existing_lbs = 0; log_debug_metadata("Adding PVs to VG %s.", vg->name); if (vg_bad_status_bits(vg, RESIZEABLE_VG)) return_0; + /* + * Check if existing PVs have inconsistent block sizes. + * If so, do not enforce new devices to be consistent. + */ + dm_list_iterate_items(pvl, &vg->pvs) { + logical_block_size = 0; + physical_block_size = 0; + + if (!dev_get_direct_block_sizes(pvl->pv->dev, &physical_block_size, &logical_block_size)) + continue; + + if (!logical_block_size) + continue; + + if (!prev_lbs) { + prev_lbs = logical_block_size; + continue; + } + + if (prev_lbs != logical_block_size) { + inconsistent_existing_lbs = 1; + break; + } + } + dm_list_iterate_items(pvl, &pp->pvs) { log_debug_metadata("Adding PV %s to VG %s.", pv_dev_name(pvl->pv), vg->name); @@ -715,6 +743,22 @@ int vg_extend_each_pv(struct volume_group *vg, struct pvcreate_params *pp) return 0; } + logical_block_size = 0; + physical_block_size = 0; + + if (!dev_get_direct_block_sizes(pvl->pv->dev, &physical_block_size, &logical_block_size)) + log_warn("WARNING: PV %s has unknown block size.", pv_dev_name(pvl->pv)); + + else if (prev_lbs && logical_block_size && (logical_block_size != prev_lbs)) { + if (vg->cmd->allow_mixed_block_sizes || inconsistent_existing_lbs) + log_debug("Devices have inconsistent block sizes (%u and %u)", prev_lbs, logical_block_size); + else { + log_error("Devices have inconsistent logical block sizes (%u and %u).", + prev_lbs, logical_block_size); + return 0; + } + } + if (!add_pv_to_vg(vg, pv_dev_name(pvl->pv), pvl->pv, 0)) { log_error("PV %s cannot be added to VG %s.", pv_dev_name(pvl->pv), vg->name); diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c index 6a1ab11..9c39f48 100644 --- a/tools/lvmcmdline.c +++ b/tools/lvmcmdline.c @@ -2306,6 +2306,8 @@ static int _get_current_settings(struct cmd_context *cmd) if (cmd->cname->flags & CAN_USE_ONE_SCAN) cmd->can_use_one_scan = 1; + cmd->allow_mixed_block_sizes = find_config_tree_bool(cmd, devices_allow_mixed_block_sizes_CFG, NULL); + cmd->partial_activation = 0; cmd->degraded_activation = 0; activation_mode = find_config_tree_str(cmd, activation_mode_CFG, NULL); diff --git a/tools/toollib.c b/tools/toollib.c index 1b01ccc..42179d9 100644 --- a/tools/toollib.c +++ b/tools/toollib.c @@ -5481,6 +5481,8 @@ int pvcreate_each_device(struct cmd_context *cmd, struct device_list *devl; const char *pv_name; int consistent = 0; + unsigned int physical_block_size, logical_block_size; + unsigned int prev_pbs = 0, prev_lbs = 0; int must_use_all = (cmd->cname->flags & MUST_USE_ALL_ARGS); int found; unsigned i; @@ -5559,6 +5561,51 @@ int pvcreate_each_device(struct cmd_context *cmd, pd->dev = dev_cache_get(pd->name, cmd->full_filter); /* + * Check for consistent block sizes. + */ + if (pp->check_consistent_block_size) { + dm_list_iterate_items(pd, &pp->arg_devices) { + if (!pd->dev) + continue; + + logical_block_size = 0; + physical_block_size = 0; + + if (!dev_get_direct_block_sizes(pd->dev, &physical_block_size, &logical_block_size)) { + log_warn("WARNING: Unknown block size for device %s.", dev_name(pd->dev)); + continue; + } + + if (!logical_block_size) { + log_warn("WARNING: Unknown logical_block_size for device %s.", dev_name(pd->dev)); + continue; + } + + if (!prev_lbs) { + prev_lbs = logical_block_size; + prev_pbs = physical_block_size; + continue; + } + + if (prev_lbs == logical_block_size) { + /* Require lbs to match, just warn about unmatching pbs. */ + if (!cmd->allow_mixed_block_sizes && prev_pbs && physical_block_size && + (prev_pbs != physical_block_size)) + log_warn("WARNING: Devices have inconsistent physical block sizes (%u and %u).", + prev_pbs, physical_block_size); + continue; + } + + if (!cmd->allow_mixed_block_sizes) { + log_error("Devices have inconsistent logical block sizes (%u and %u).", + prev_lbs, logical_block_size); + log_print("See lvm.conf allow_mixed_block_sizes."); + return 0; + } + } + } + + /* * Use process_each_pv to search all existing PVs and devices. * * This is a slightly different way to use process_each_pv, because the diff --git a/tools/vgcreate.c b/tools/vgcreate.c index 4356d99..7add53b 100644 --- a/tools/vgcreate.c +++ b/tools/vgcreate.c @@ -48,6 +48,8 @@ int vgcreate(struct cmd_context *cmd, int argc, char **argv) /* Don't create a new PV on top of an existing PV like pvcreate does. */ pp.preserve_existing = 1; + pp.check_consistent_block_size = 1; + if (!vgcreate_params_set_defaults(cmd, &vp_def, NULL)) return EINVALID_CMD_LINE; vp_def.vg_name = vg_name; -- 1.8.3.1 ++++++ bug-1149408_02-tests-allow-mixed-block-sizes.patch ++++++ From f758d16b30afd1848ea874b5ebd20185c5300320 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 1 Aug 2019 10:13:41 -0500 Subject: [PATCH] tests: allow-mixed-block-sizes (cherry picked from commit cd8a0133fa28a0dff2714c80f822e8c665fe6237) --- test/shell/allow-mixed-block-sizes.sh | 44 +++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 test/shell/allow-mixed-block-sizes.sh diff --git a/test/shell/allow-mixed-block-sizes.sh b/test/shell/allow-mixed-block-sizes.sh new file mode 100644 index 0000000..1803256 --- /dev/null +++ b/test/shell/allow-mixed-block-sizes.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash + +# Copyright (C) 2019 Red Hat, Inc. All rights reserved. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions +# of the GNU General Public License v.2. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +SKIP_WITH_LVMPOLLD=1 + +. lib/inittest + +dd if=/dev/zero of=loopa bs=$((1024*1024)) count=2 2> /dev/null +dd if=/dev/zero of=loopb bs=$((1024*1024)) count=2 2> /dev/null +LOOP1=$(losetup -f loopa --sector-size 4096 --show) +LOOP2=$(losetup -f loopb --show) + +echo $LOOP1 +echo $LOOP2 + +aux extend_filter "a|$LOOP1|" +aux extend_filter "a|$LOOP2|" + +not vgcreate --config 'devices {allow_mixed_block_sizes=0 scan="/dev"}' $vg $LOOP1 $LOOP2 +vgcreate --config 'devices {allow_mixed_block_sizes=1 scan="/dev"}' $vg $LOOP1 $LOOP2 +vgs --config 'devices {allow_mixed_block_sizes=1 scan="/dev"}' $vg + +wipefs -a $LOOP1 +wipefs -a $LOOP2 + +vgcreate --config 'devices {allow_mixed_block_sizes=1 scan="/dev"}' $vg $LOOP1 +vgs --config 'devices {allow_mixed_block_sizes=1 scan="/dev"}' $vg +not vgextend --config 'devices {allow_mixed_block_sizes=0 scan="/dev"}' $vg $LOOP2 +vgextend --config 'devices {allow_mixed_block_sizes=1 scan="/dev"}' $vg $LOOP2 + +losetup -d $LOOP1 +losetup -d $LOOP2 +rm loopa +rm loopb + -- 1.8.3.1 ++++++ bug-1149408_03-tests-allow-mixed-block-sizes-skip-with-older-losetu.patch ++++++ From 2327f3997bfb70d67299f3dfa45436132e0a8521 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 11 Sep 2019 13:52:51 -0500 Subject: [PATCH] tests: allow-mixed-block-sizes skip with older losetup (cherry picked from commit 69b7c00a77e09d23d57eef9d35afb1dc4663478c) --- test/shell/allow-mixed-block-sizes.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/shell/allow-mixed-block-sizes.sh b/test/shell/allow-mixed-block-sizes.sh index 1803256..2f0b3a0 100644 --- a/test/shell/allow-mixed-block-sizes.sh +++ b/test/shell/allow-mixed-block-sizes.sh @@ -14,6 +14,8 @@ SKIP_WITH_LVMPOLLD=1 . lib/inittest +losetup -h | grep sector-size || skip + dd if=/dev/zero of=loopa bs=$((1024*1024)) count=2 2> /dev/null dd if=/dev/zero of=loopb bs=$((1024*1024)) count=2 2> /dev/null LOOP1=$(losetup -f loopa --sector-size 4096 --show) -- 1.8.3.1 ++++++ bug-1149408_04-config-allow_mixed_block_sizes-set-default-to-1.patch ++++++ From f53cce1804236e69b0ec99f490f0c2d2d001cf5d Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 24 Oct 2019 10:02:34 -0500 Subject: [PATCH] config: allow_mixed_block_sizes set default to 1 to use the old behavior by default, which allows using mixed block sizes. The user will need to set it to 0 in lvm.conf to turn on the new restrictions. --- lib/config/config_settings.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 7981d5d..498ec23 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -470,7 +470,7 @@ cfg(devices_allow_changes_with_duplicate_pvs_CFG, "allow_changes_with_duplicate_ "Enabling this setting allows the VG to be used as usual even with\n" "uncertain devices.\n") -cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 0, vsn(2, 3, 6), NULL, 0, NULL, +cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 1, vsn(2, 3, 6), NULL, 0, NULL, "Allow PVs in the same VG with different logical block sizes.\n" "When allowed, the user is responsible to ensure that an LV is\n" "using PVs with matching block sizes when necessary.\n") -- 1.8.3.1 ++++++ bug-1149408_05-config-allow_mixed_block_sizes-set-version-2.02.187.patch ++++++ From 96fd312e85ce38a8fc8cb80f106f3fdc49c31a97 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 24 Oct 2019 10:19:59 -0500 Subject: [PATCH] config: allow_mixed_block_sizes set version 2.02.187 --- lib/config/config_settings.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 498ec23..429bff1 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -470,7 +470,7 @@ cfg(devices_allow_changes_with_duplicate_pvs_CFG, "allow_changes_with_duplicate_ "Enabling this setting allows the VG to be used as usual even with\n" "uncertain devices.\n") -cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 1, vsn(2, 3, 6), NULL, 0, NULL, +cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 1, vsn(2, 2, 187), NULL, 0, NULL, "Allow PVs in the same VG with different logical block sizes.\n" "When allowed, the user is responsible to ensure that an LV is\n" "using PVs with matching block sizes when necessary.\n") -- 1.8.3.1 ++++++ bug-1149408_06-suse-special-config-allow_mixed_block_sizes-set-version-2.02.180.patch ++++++ By heming.zhao(a)suse.com This is SUSE speical patch. Commit 96fd312e85ce38a8fc8cb80f106f3fdc49c31a97 sets since version 2.2.187 and default value is 1, this patch change version to 2.2.180 and default value is 0. --- lib/config/config_settings.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 498ec23..429bff1 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -470,7 +470,7 @@ cfg(devices_allow_changes_with_duplicate_pvs_CFG, "allow_changes_with_duplicate_ "Enabling this setting allows the VG to be used as usual even with\n" "uncertain devices.\n") -cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 1, vsn(2, 2, 187), NULL, 0, NULL, +cfg(devices_allow_mixed_block_sizes_CFG, "allow_mixed_block_sizes", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, 0, vsn(2, 2, 180), NULL, 0, NULL, "Allow PVs in the same VG with different logical block sizes.\n" "When allowed, the user is responsible to ensure that an LV is\n" "using PVs with matching block sizes when necessary.\n") -- 1.8.3.1 ++++++ bug-1150021_01-fix-dev_unset_last_byte-after-write-error.patch ++++++ From 245d7fcd590583166f6cdd78a81bac903fea92eb Mon Sep 17 00:00:00 2001 From: Heming Zhao <heming.zhao(a)suse.com> Date: Wed, 13 Nov 2019 11:28:22 -0600 Subject: [PATCH] fix dev_unset_last_byte after write error ev_unset_last_byte() must be called while the fd is still valid. After a write error, dev_unset_last_byte() must be called before closing the dev and resetting the fd. In the write error path, dev_unset_last_byte() was being called after label_scan_invalidate() which meant that it would not unset the last_byte values. After a write error, dev_unset_last_byte() is now called in dev_write_bytes() before label_scan_invalidate(), instead of by the caller of dev_write_bytes(). In the common case of a successful write, the sequence is still: dev_set_last_byte(); dev_write_bytes(); dev_unset_last_byte(); Signed-off-by: Zhao Heming <heming.zhao(a)suse.com> --- lib/format_text/format-text.c | 3 --- lib/label/label.c | 8 ++++---- lib/metadata/mirror.c | 1 - 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/lib/format_text/format-text.c b/lib/format_text/format-text.c index 6eea07fe46..026f93a13c 100644 --- a/lib/format_text/format-text.c +++ b/lib/format_text/format-text.c @@ -402,7 +402,6 @@ static int _raw_write_mda_header(const struct format_type *fmt, dev_set_last_byte(dev, start_byte + MDA_HEADER_SIZE); if (!dev_write_bytes(dev, start_byte, MDA_HEADER_SIZE, mdah)) { - dev_unset_last_byte(dev); log_error("Failed to write mda header to %s fd %d", dev_name(dev), dev->bcache_fd); return 0; } @@ -691,7 +690,6 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, (size_t) (mdac->rlocn.size - new_wrap), fidtc->raw_metadata_buf)) { log_error("Failed to write metadata to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); - dev_unset_last_byte(mdac->area.dev); goto out; } @@ -705,7 +703,6 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, (size_t) new_wrap, fidtc->raw_metadata_buf + mdac->rlocn.size - new_wrap)) { log_error("Failed to write metadata wrap to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); - dev_unset_last_byte(mdac->area.dev); goto out; } } diff --git a/lib/label/label.c b/lib/label/label.c index e4a106854f..ae5c1427ea 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -216,7 +216,7 @@ int label_write(struct device *dev, struct label *label) if (!dev_write_bytes(dev, offset, LABEL_SIZE, buf)) { log_debug_devs("Failed to write label to %s", dev_name(dev)); - r = 0; + return 0; } dev_unset_last_byte(dev); @@ -621,7 +621,6 @@ static int _scan_list(struct cmd_context *cmd, struct dev_filter *f, int submit_count; int scan_failed; int is_lvm_device; - int error; int ret; dm_list_init(&wait_devs); @@ -668,12 +667,11 @@ static int _scan_list(struct cmd_context *cmd, struct dev_filter *f, dm_list_iterate_items_safe(devl, devl2, &wait_devs) { bb = NULL; - error = 0; scan_failed = 0; is_lvm_device = 0; if (!bcache_get(scan_bcache, devl->dev->bcache_fd, 0, 0, &bb)) { - log_debug_devs("Scan failed to read %s error %d.", dev_name(devl->dev), error); + log_debug_devs("Scan failed to read %s.", dev_name(devl->dev)); scan_failed = 1; scan_read_errors++; scan_failed_count++; @@ -1385,6 +1383,7 @@ bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data) if (!bcache_write_bytes(scan_bcache, dev->bcache_fd, start, len, data)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } @@ -1392,6 +1391,7 @@ bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data) if (!bcache_flush(scan_bcache)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } diff --git a/lib/metadata/mirror.c b/lib/metadata/mirror.c index 4e88f25fce..068ebd2c56 100644 --- a/lib/metadata/mirror.c +++ b/lib/metadata/mirror.c @@ -305,7 +305,6 @@ static int _write_log_header(struct cmd_context *cmd, struct logical_volume *lv) dev_set_last_byte(dev, sizeof(log_header)); if (!dev_write_bytes(dev, UINT64_C(0), sizeof(log_header), &log_header)) { - dev_unset_last_byte(dev); log_error("Failed to write log header to %s.", name); return 0; } -- 2.16.4 ++++++ bug-1150021_02-radix-tree-Bring-radix-tree-up-to-date-with-the-mast.patch ++++++ ++++ 3130 lines (skipped) ++++++ bug-1150021_03-cov-Fix-a-leak.patch ++++++ From 36523a398d50381f8355e719b98c3ce584b81a23 Mon Sep 17 00:00:00 2001 From: Marian Csontos <mcsontos(a)redhat.com> Date: Tue, 27 Aug 2019 12:21:41 +0200 Subject: [PATCH] cov: Fix a leak --- lib/device/bcache.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index a250bfd..d487ca2 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -516,6 +516,7 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes, (unsigned long long)extra_nbytes, (unsigned long long)_last_byte_sector_size); + free(io); return false; } } @@ -547,8 +548,8 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, log_debug("Device write error %d offset %llu len %llu", errno, (unsigned long long)(where + pos), (unsigned long long)(len - pos)); - free(io); - return false; + free(io); + return false; } pos += rv; } -- 1.8.3.1 ++++++ bug-1150021_04-bcache-Bring-bcache-into-sync-with-master-branch.patch ++++++ ++++ 734 lines (skipped) ++++++ bug-1150021_05-bcache-add-bcache_abort.patch ++++++ From 232f779db4a4cb3c9a1cc628f011f7b28fb6f158 Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Mon, 28 Oct 2019 14:29:47 +0000 Subject: [PATCH 3/9] [bcache] add bcache_abort() This gives us a way to cope with write failures. --- lib/device/bcache.c | 33 ++++++++++++++++++++++++++ lib/device/bcache.h | 7 ++++++ test/unit/bcache_t.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 106 insertions(+), 1 deletion(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 9f67a27..72cc599 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -1297,6 +1297,39 @@ bool bcache_invalidate_fd(struct bcache *cache, int fd) //---------------------------------------------------------------- +static bool _abort_v(struct radix_tree_iterator *it, + uint8_t *kb, uint8_t *ke, union radix_value v) +{ + struct block *b = v.ptr; + + if (b->ref_count) { + log_fatal("bcache_abort: block (%d, %llu) still held", + b->fd, (unsigned long long) b->index); + return true; + } + + _unlink_block(b); + _free_block(b); + + // We can't remove the block from the radix tree yet because + // we're in the middle of an iteration. + return true; +} + +void bcache_abort_fd(struct bcache *cache, int fd) +{ + union key k; + struct radix_tree_iterator it; + + k.parts.fd = fd; + + it.visit = _abort_v; + radix_tree_iterate(cache->rtree, k.bytes, k.bytes + sizeof(k.parts.fd), &it); + radix_tree_remove_prefix(cache->rtree, k.bytes, k.bytes + sizeof(k.parts.fd)); +} + +//---------------------------------------------------------------- + void bcache_set_last_byte(struct bcache *cache, int fd, uint64_t offset, int sector_size) { _last_byte_fd = fd; diff --git a/lib/device/bcache.h b/lib/device/bcache.h index 790d0fe..f9067f7 100644 --- a/lib/device/bcache.h +++ b/lib/device/bcache.h @@ -144,6 +144,13 @@ bool bcache_invalidate(struct bcache *cache, int fd, block_address index); */ bool bcache_invalidate_fd(struct bcache *cache, int fd); +/* + * Call this function if flush, or invalidate fail and you do not + * wish to retry the writes. This will throw away any dirty data + * not written. If any blocks for fd are held, then it will call + * abort(). + */ +void bcache_abort_fd(struct bcache *cache, int fd); //---------------------------------------------------------------- // The next four functions are utilities written in terms of the above api. diff --git a/test/unit/bcache_t.c b/test/unit/bcache_t.c index 92c2d57..668d24d 100644 --- a/test/unit/bcache_t.c +++ b/test/unit/bcache_t.c @@ -793,7 +793,6 @@ static void test_invalidate_after_write_error(void *context) static void test_invalidate_held_block(void *context) { - struct fixture *f = context; struct mock_engine *me = f->me; struct bcache *cache = f->cache; @@ -810,6 +809,67 @@ static void test_invalidate_held_block(void *context) } //---------------------------------------------------------------- +// abort tests + +static void test_abort_no_blocks(void *context) +{ + struct fixture *f = context; + struct bcache *cache = f->cache; + int fd = 17; + + // We have no expectations + bcache_abort_fd(cache, fd); +} + +static void test_abort_single_block(void *context) +{ + struct fixture *f = context; + struct bcache *cache = f->cache; + struct block *b; + int fd = 17; + + T_ASSERT(bcache_get(cache, fd, 0, GF_ZERO, &b)); + bcache_put(b); + + bcache_abort_fd(cache, fd); + + // no write should be issued + T_ASSERT(bcache_flush(cache)); +} + +static void test_abort_only_specific_fd(void *context) +{ + struct fixture *f = context; + struct mock_engine *me = f->me; + struct bcache *cache = f->cache; + struct block *b; + int fd1 = 17, fd2 = 18; + + T_ASSERT(bcache_get(cache, fd1, 0, GF_ZERO, &b)); + bcache_put(b); + + T_ASSERT(bcache_get(cache, fd1, 1, GF_ZERO, &b)); + bcache_put(b); + + T_ASSERT(bcache_get(cache, fd2, 0, GF_ZERO, &b)); + bcache_put(b); + + T_ASSERT(bcache_get(cache, fd2, 1, GF_ZERO, &b)); + bcache_put(b); + + bcache_abort_fd(cache, fd2); + + // writes for fd1 should still be issued + _expect_write(me, fd1, 0); + _expect_write(me, fd1, 1); + + _expect(me, E_WAIT); + _expect(me, E_WAIT); + + T_ASSERT(bcache_flush(cache)); +} + +//---------------------------------------------------------------- // Chasing a bug reported by dct static void _cycle(struct fixture *f, unsigned nr_cache_blocks) @@ -897,6 +957,11 @@ static struct test_suite *_small_tests(void) T("invalidate-read-error", "invalidate a block that errored", test_invalidate_after_read_error); T("invalidate-write-error", "invalidate a block that errored", test_invalidate_after_write_error); T("invalidate-fails-in-held", "invalidating a held block fails", test_invalidate_held_block); + + T("abort-with-no-blocks", "you can call abort, even if there are no blocks in the cache", test_abort_no_blocks); + T("abort-single-block", "single block get silently discarded", test_abort_single_block); + T("abort-specific-fd", "abort doesn't effect other fds", test_abort_only_specific_fd); + T("concurrent-reads-after-invalidate", "prefetch should still issue concurrent reads after invalidate", test_concurrent_reads_after_invalidate); -- 1.8.3.1 ++++++ bug-1150021_06-label-Use-bcache_abort_fd-to-ensure-blocks-are-no-lo.patch ++++++ From babde3da5530e4bbda5fb474fb5ea32a55adf16f Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Mon, 28 Oct 2019 15:01:47 +0000 Subject: [PATCH 4/9] [label] Use bcache_abort_fd() to ensure blocks are no longer in the cache. The return value from bcache_invalidate_fd() was not being checked. So I've introduced a little function, _invalidate_fd() that always calls bcache_abort_fd() if the write fails. --- lib/label/label.c | 42 +++++++++++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/lib/label/label.c b/lib/label/label.c index ae5c142..70b7934 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -594,6 +594,14 @@ static void _drop_bad_aliases(struct device *dev) } } +// Like bcache_invalidate, only it throws any dirty data away if the +// write fails. +static void _invalidate_fd(struct bcache *cache, int fd) +{ + if (!bcache_invalidate_fd(cache, fd)) + bcache_abort_fd(cache, fd); +} + /* * Read or reread label/metadata from selected devs. * @@ -704,7 +712,7 @@ static int _scan_list(struct cmd_context *cmd, struct dev_filter *f, * drop it from bcache. */ if (scan_failed || !is_lvm_device) { - bcache_invalidate_fd(scan_bcache, devl->dev->bcache_fd); + _invalidate_fd(scan_bcache, devl->dev->bcache_fd); _scan_dev_close(devl->dev); } @@ -881,7 +889,7 @@ int label_scan(struct cmd_context *cmd) * so this will usually not be true. */ if (_in_bcache(dev)) { - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } @@ -1066,7 +1074,7 @@ int label_scan_devs(struct cmd_context *cmd, struct dev_filter *f, struct dm_lis dm_list_iterate_items(devl, devs) { if (_in_bcache(devl->dev)) { - bcache_invalidate_fd(scan_bcache, devl->dev->bcache_fd); + _invalidate_fd(scan_bcache, devl->dev->bcache_fd); _scan_dev_close(devl->dev); } } @@ -1085,7 +1093,7 @@ int label_scan_devs_rw(struct cmd_context *cmd, struct dev_filter *f, struct dm_ dm_list_iterate_items(devl, devs) { if (_in_bcache(devl->dev)) { - bcache_invalidate_fd(scan_bcache, devl->dev->bcache_fd); + _invalidate_fd(scan_bcache, devl->dev->bcache_fd); _scan_dev_close(devl->dev); } @@ -1107,7 +1115,7 @@ int label_scan_devs_excl(struct dm_list *devs) dm_list_iterate_items(devl, devs) { if (_in_bcache(devl->dev)) { - bcache_invalidate_fd(scan_bcache, devl->dev->bcache_fd); + _invalidate_fd(scan_bcache, devl->dev->bcache_fd); _scan_dev_close(devl->dev); } /* @@ -1127,7 +1135,7 @@ int label_scan_devs_excl(struct dm_list *devs) void label_scan_invalidate(struct device *dev) { if (_in_bcache(dev)) { - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } } @@ -1208,7 +1216,7 @@ int label_read(struct device *dev) dm_list_add(&one_dev, &devl->list); if (_in_bcache(dev)) { - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } @@ -1314,7 +1322,7 @@ int label_scan_open_excl(struct device *dev) if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_EXCL)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen excl %s", dev_name(dev)); - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); } dev->flags |= DEV_BCACHE_EXCL; @@ -1322,6 +1330,18 @@ int label_scan_open_excl(struct device *dev) return label_scan_open(dev); } +int label_scan_open_rw(struct device *dev) +{ + if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { + /* FIXME: avoid tossing out bcache blocks just to replace fd. */ + log_debug("Close and reopen rw %s", dev_name(dev)); + _invalidate_fd(scan_bcache, dev->bcache_fd); + _scan_dev_close(dev); + } + dev->flags |= DEV_BCACHE_WRITE; + return label_scan_open(dev); +} + bool dev_read_bytes(struct device *dev, uint64_t start, size_t len, void *data) { if (!scan_bcache) { @@ -1363,7 +1383,7 @@ bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data) if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); dev->flags |= DEV_BCACHE_WRITE; @@ -1411,7 +1431,7 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); dev->flags |= DEV_BCACHE_WRITE; @@ -1462,7 +1482,7 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); - bcache_invalidate_fd(scan_bcache, dev->bcache_fd); + _invalidate_fd(scan_bcache, dev->bcache_fd); _scan_dev_close(dev); dev->flags |= DEV_BCACHE_WRITE; -- 1.8.3.1 ++++++ bug-1150021_07-bcache-add-unit-test.patch ++++++ From 056eb0a8809aee3ca888aff4fe8e22bf8884a0cc Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Tue, 29 Oct 2019 10:33:31 +0000 Subject: [PATCH 5/9] [bcache] add unit test abort-forces-read --- test/unit/bcache_t.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/test/unit/bcache_t.c b/test/unit/bcache_t.c index 668d24d..2a8f931 100644 --- a/test/unit/bcache_t.c +++ b/test/unit/bcache_t.c @@ -837,6 +837,29 @@ static void test_abort_single_block(void *context) T_ASSERT(bcache_flush(cache)); } +static void test_abort_forces_reread(void *context) +{ + struct fixture *f = context; + struct mock_engine *me = f->me; + struct bcache *cache = f->cache; + struct block *b; + int fd = 17; + + _expect_read(me, fd, 0); + _expect(me, E_WAIT); + T_ASSERT(bcache_get(cache, fd, 0, GF_DIRTY, &b)); + bcache_put(b); + + bcache_abort_fd(cache, fd); + T_ASSERT(bcache_flush(cache)); + + // Check the block is re-read + _expect_read(me, fd, 0); + _expect(me, E_WAIT); + T_ASSERT(bcache_get(cache, fd, 0, 0, &b)); + bcache_put(b); +} + static void test_abort_only_specific_fd(void *context) { struct fixture *f = context; @@ -960,6 +983,7 @@ static struct test_suite *_small_tests(void) T("abort-with-no-blocks", "you can call abort, even if there are no blocks in the cache", test_abort_no_blocks); T("abort-single-block", "single block get silently discarded", test_abort_single_block); + T("abort-forces-read", "if a block has been discarded then another read is necc.", test_abort_forces_reread); T("abort-specific-fd", "abort doesn't effect other fds", test_abort_only_specific_fd); T("concurrent-reads-after-invalidate", "prefetch should still issue concurrent reads after invalidate", -- 1.8.3.1 ++++++ bug-1150021_08-bcache-pass-up-the-error-from-io_submit-rather-than.patch ++++++ From 6370c20d392fb6228273695cd9f1a2ae43ff1e77 Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Tue, 29 Oct 2019 10:39:20 +0000 Subject: [PATCH 6/9] [bcache] pass up the error from io_submit rather than using generic -EIO Author: Heming Zhao --- lib/device/bcache.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 72cc599..56e3db3 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -246,6 +246,10 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, if (r < 0) { _cb_free(e->cbs, cb); + ((struct block *) context)->error = r; + log_warn("io_submit <%c> off %llu bytes %llu return %d:%s", + (d == DIR_READ) ? 'R' : 'W', (long long unsigned)offset, + (long long unsigned) nbytes, r, strerror(-r)); return false; } @@ -784,8 +788,7 @@ static void _issue_low_level(struct block *b, enum dir d) dm_list_move(&cache->io_pending, &b->list); if (!cache->engine->issue(cache->engine, d, b->fd, sb, se, b->data, b)) { - /* FIXME: if io_submit() set an errno, return that instead of EIO? */ - _complete_io(b, -EIO); + _complete_io(b, b->error); return; } } -- 1.8.3.1 ++++++ bug-1150021_09-bcache-reverse-earlier-patch.patch ++++++ From 1e2e12f19c584c3568c255c5d89ed8aadbe61b9c Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Tue, 29 Oct 2019 15:14:07 +0000 Subject: [PATCH 7/9] [bcache] reverse earlier patch. It broke some unit tests, for v. little benefit --- lib/device/bcache.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 56e3db3..72cc599 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -246,10 +246,6 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, if (r < 0) { _cb_free(e->cbs, cb); - ((struct block *) context)->error = r; - log_warn("io_submit <%c> off %llu bytes %llu return %d:%s", - (d == DIR_READ) ? 'R' : 'W', (long long unsigned)offset, - (long long unsigned) nbytes, r, strerror(-r)); return false; } @@ -788,7 +784,8 @@ static void _issue_low_level(struct block *b, enum dir d) dm_list_move(&cache->io_pending, &b->list); if (!cache->engine->issue(cache->engine, d, b->fd, sb, se, b->data, b)) { - _complete_io(b, b->error); + /* FIXME: if io_submit() set an errno, return that instead of EIO? */ + _complete_io(b, -EIO); return; } } -- 1.8.3.1 ++++++ bug-1150021_10-bcache-bcache_invalidate_fd-only-remove-prefixes-on.patch ++++++ From 019fa6f8eec799ebc7a9b9c5b31354ff1717db12 Mon Sep 17 00:00:00 2001 From: Joe Thornber <ejt(a)redhat.com> Date: Tue, 29 Oct 2019 15:21:11 +0000 Subject: [PATCH 8/9] [bcache] bcache_invalidate_fd, only remove prefixes on success. --- lib/device/bcache.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 72cc599..b64707e 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -1291,7 +1291,10 @@ bool bcache_invalidate_fd(struct bcache *cache, int fd) it.success = true; it.it.visit = _invalidate_v; radix_tree_iterate(cache->rtree, k.bytes, k.bytes + sizeof(k.parts.fd), &it.it); - radix_tree_remove_prefix(cache->rtree, k.bytes, k.bytes + sizeof(k.parts.fd)); + + if (it.success) + radix_tree_remove_prefix(cache->rtree, k.bytes, k.bytes + sizeof(k.parts.fd)); + return it.success; } -- 1.8.3.1 ++++++ bug-1150021_11-radix-tree-Add-missing-test-case.patch ++++++ ++++ 1688 lines (skipped) ++++++ bug-1150021_12-base-Get-Makefile-from-master.patch ++++++ From 440ced192a5ae6d5fa7c191981abcdf59926e90b Mon Sep 17 00:00:00 2001 From: Marian Csontos <mcsontos(a)redhat.com> Date: Mon, 20 Jan 2020 14:36:16 +0100 Subject: [PATCH] base: Get Makefile from master --- base/Makefile | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 base/Makefile diff --git a/base/Makefile b/base/Makefile new file mode 100644 index 0000000..056ea59 --- /dev/null +++ b/base/Makefile @@ -0,0 +1,38 @@ +# Copyright (C) 2018 Red Hat, Inc. All rights reserved. +# +# This file is part of the device-mapper userspace tools. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions +# of the GNU Lesser General Public License v.2.1. +# +# You should have received a copy of the GNU Lesser General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +# Uncomment this to build the simple radix tree. You'll need to make clean too. +# Comment to build the advanced radix tree. +#base/data-struct/radix-tree.o: CFLAGS += -DSIMPLE_RADIX_TREE + +# NOTE: this Makefile only works as 'include' for toplevel Makefile +# which defined all top_* variables + +BASE_SOURCE=\ + base/data-struct/radix-tree.c + +BASE_TARGET = base/libbase.a +BASE_DEPENDS = $(BASE_SOURCE:%.c=%.d) +BASE_OBJECTS = $(BASE_SOURCE:%.c=%.o) +CLEAN_TARGETS += $(BASE_DEPENDS) $(BASE_OBJECTS) \ + $(BASE_SOURCE:%.c=%.gcda) \ + $(BASE_SOURCE:%.c=%.gcno) \ + $(BASE_TARGET) + +$(BASE_TARGET): $(BASE_OBJECTS) + @echo " [AR] $@" + $(Q) $(RM) $@ + $(Q) $(AR) rsv $@ $(BASE_OBJECTS) > /dev/null + +ifeq ("$(DEPENDS)","yes") +-include $(BASE_DEPENDS) +endif -- 1.8.3.1 ++++++ bug-1150021_13-Fix-rounding-writes-up-to-sector-size.patch ++++++ From 44c460954be5c63cf5338bd9151344fe2626565f Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 24 Jul 2019 11:32:13 -0500 Subject: [PATCH 1/2] Fix rounding writes up to sector size Do this at two levels, although one would be enough to fix the problem seen recently: - Ignore any reported sector size other than 512 of 4096. If either sector size (physical or logical) is reported as 512, then use 512. If neither are reported as 512, and one or the other is reported as 4096, then use 4096. If neither is reported as either 512 or 4096, then use 512. - When rounding up a limited write in bcache to be a multiple of the sector size, check that the resulting write size is not larger than the bcache block itself. (This shouldn't happen if the sector size is 512 or 4096.) (cherry picked from commit 7550665ba49ac7d497d5b212e14b69298ef01361) Conflicts: lib/device/dev-io.c --- lib/device/bcache.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index b64707e..77d1543 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -169,6 +169,7 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, sector_t offset; sector_t nbytes; sector_t limit_nbytes; + sector_t orig_nbytes; sector_t extra_nbytes = 0; if (((uintptr_t) data) & e->page_mask) { @@ -191,11 +192,41 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, return false; } + /* + * If the bcache block offset+len goes beyond where lvm is + * intending to write, then reduce the len being written + * (which is the bcache block size) so we don't write past + * the limit set by lvm. If after applying the limit, the + * resulting size is not a multiple of the sector size (512 + * or 4096) then extend the reduced size to be a multiple of + * the sector size (we don't want to write partial sectors.) + */ if (offset + nbytes > _last_byte_offset) { limit_nbytes = _last_byte_offset - offset; - if (limit_nbytes % _last_byte_sector_size) + + if (limit_nbytes % _last_byte_sector_size) { extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + /* + * adding extra_nbytes to the reduced nbytes (limit_nbytes) + * should make the final write size a multiple of the + * sector size. This should never result in a final size + * larger than the bcache block size (as long as the bcache + * block size is a multiple of the sector size). + */ + if (limit_nbytes + extra_nbytes > nbytes) { + log_warn("Skip extending write at %llu len %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + extra_nbytes = 0; + } + } + + orig_nbytes = nbytes; + if (extra_nbytes) { log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", (unsigned long long)offset, @@ -210,6 +241,22 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes); nbytes = limit_nbytes; } + + /* + * This shouldn't happen, the reduced+extended + * nbytes value should never be larger than the + * bcache block size. + */ + if (nbytes > orig_nbytes) { + log_error("Invalid adjusted write at %llu len %llu adjusted %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)orig_nbytes, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + return false; + } } } @@ -403,6 +450,7 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, uint64_t nbytes = len; sector_t limit_nbytes = 0; sector_t extra_nbytes = 0; + sector_t orig_nbytes = 0; if (offset > _last_byte_offset) { log_error("Limit write at %llu len %llu beyond last byte %llu", @@ -415,9 +463,30 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, if (offset + nbytes > _last_byte_offset) { limit_nbytes = _last_byte_offset - offset; - if (limit_nbytes % _last_byte_sector_size) + + if (limit_nbytes % _last_byte_sector_size) { extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + /* + * adding extra_nbytes to the reduced nbytes (limit_nbytes) + * should make the final write size a multiple of the + * sector size. This should never result in a final size + * larger than the bcache block size (as long as the bcache + * block size is a multiple of the sector size). + */ + if (limit_nbytes + extra_nbytes > nbytes) { + log_warn("Skip extending write at %llu len %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + extra_nbytes = 0; + } + } + + orig_nbytes = nbytes; + if (extra_nbytes) { log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", (unsigned long long)offset, @@ -432,6 +501,22 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes); nbytes = limit_nbytes; } + + /* + * This shouldn't happen, the reduced+extended + * nbytes value should never be larger than the + * bcache block size. + */ + if (nbytes > orig_nbytes) { + log_error("Invalid adjusted write at %llu len %llu adjusted %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)orig_nbytes, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + return false; + } } where = offset; -- 1.8.3.1 ++++++ bug-1150021_14-bcache-Fix-memory-leak-in-error-path.patch ++++++ From deaf304ee6d88cd47632a345b92b3949cd06d752 Mon Sep 17 00:00:00 2001 From: Marian Csontos <mcsontos(a)redhat.com> Date: Wed, 4 Mar 2020 13:22:10 +0100 Subject: [PATCH 2/2] bcache: Fix memory leak in error path --- lib/device/bcache.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 77d1543..a74b6b3 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -515,6 +515,7 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes, (unsigned long long)extra_nbytes, (unsigned long long)_last_byte_sector_size); + free(io); return false; } } -- 1.8.3.1 ++++++ bug-1155668_systemd-add-missing-Before-shutdown.target-to-LVM2.patch ++++++ Note by Heming Zhao: this patch combines 0a726a7e268b3 & 03d6cfdd99fb ---- From 0a726a7e268b31856615491809af73bda5d4d6f9 Mon Sep 17 00:00:00 2001 From: Peter Rajnoha <prajnoha(a)redhat.com> Date: Tue, 9 Apr 2019 12:40:55 +0200 Subject: [PATCH] systemd: add missing Before=shutdown.target to LVM2 services to fix shutdown ordering We already used Conflicts=shutdown target to stop LVM2 services on shutdown. But we still missed the ordering - the shutdown.target should be reached only after all the services are really stopped. Reported here: https://github.com/lvmteam/lvm2/issues/17 --- commit 03d6cfdd99fb1c2fd85721ab6faebbb1b6658873 Author: Peter Rajnoha <prajnoha(a)redhat.com> Date: Thu Apr 11 12:22:27 2019 +0200 systemd: put back DefaultDependencies=no for lvmpolld and lvmetad socket unit Previous commit 0a726a7e268b31856615491809af73bda5d4d6f9 removed this by mistake - we have to keep the DefaultDependencies=no - the sockets.target is after sysinit.target. --- scripts/blk_availability_systemd_red_hat.service.in | 1 + scripts/lvm2_activation_generator_systemd_red_hat.c | 2 ++ scripts/lvm2_cluster_activation_systemd_red_hat.service.in | 1 + scripts/lvm2_clvmd_systemd_red_hat.service.in | 2 +- scripts/lvm2_cmirrord_systemd_red_hat.service.in | 2 +- scripts/lvm2_lvmetad_systemd_red_hat.service.in | 1 + scripts/lvm2_lvmpolld_systemd_red_hat.service.in | 1 + scripts/lvm2_monitoring_systemd_red_hat.service.in | 2 +- 8 files changed, 8 insertions(+), 3 deletions(-) diff --git a/scripts/blk_availability_systemd_red_hat.service.in b/scripts/blk_availability_systemd_red_hat.service.in index 9462072ccc..05fe2eee5b 100644 --- a/scripts/blk_availability_systemd_red_hat.service.in +++ b/scripts/blk_availability_systemd_red_hat.service.in @@ -1,5 +1,6 @@ [Unit] Description=Availability of block devices +Before=shutdown.target After=lvm2-activation.service lvm2-lvmetad.service iscsi-shutdown.service iscsi.service iscsid.service fcoe.service DefaultDependencies=no Conflicts=shutdown.target diff --git a/scripts/lvm2_activation_generator_systemd_red_hat.c b/scripts/lvm2_activation_generator_systemd_red_hat.c index 487582f1df..f85a28fe04 100644 --- a/scripts/lvm2_activation_generator_systemd_red_hat.c +++ b/scripts/lvm2_activation_generator_systemd_red_hat.c @@ -127,6 +127,8 @@ static int generate_unit(const char *dir, int unit, int sysinit_needed) "SourcePath=/etc/lvm/lvm.conf\n" "DefaultDependencies=no\n", f); + fputs("Conflicts=shutdown.target\n", f); + if (unit == UNIT_NET) { fprintf(f, "After=%s iscsi.service fcoe.service\n" "Before=remote-fs-pre.target shutdown.target\n\n" diff --git a/scripts/lvm2_cluster_activation_systemd_red_hat.service.in b/scripts/lvm2_cluster_activation_systemd_red_hat.service.in index bf4aa1eca6..6a966b3a5d 100644 --- a/scripts/lvm2_cluster_activation_systemd_red_hat.service.in +++ b/scripts/lvm2_cluster_activation_systemd_red_hat.service.in @@ -1,6 +1,7 @@ [Unit] Description=Clustered LVM volumes activation service Requires=lvm2-clvmd.service +Before=shutdown.target After=lvm2-clvmd.service lvm2-cmirrord.service OnFailure=lvm2-clvmd.service DefaultDependencies=no diff --git a/scripts/lvm2_clvmd_systemd_red_hat.service.in b/scripts/lvm2_clvmd_systemd_red_hat.service.in index ced27744e0..2af365f700 100644 --- a/scripts/lvm2_clvmd_systemd_red_hat.service.in +++ b/scripts/lvm2_clvmd_systemd_red_hat.service.in @@ -2,7 +2,7 @@ Description=Clustered LVM daemon Documentation=man:clvmd(8) After=dlm.service corosync.service -Before=remote-fs-pre.target +Before=remote-fs-pre.target shutdown.target Requires=network.target dlm.service corosync.service RefuseManualStart=true RefuseManualStop=true diff --git a/scripts/lvm2_cmirrord_systemd_red_hat.service.in b/scripts/lvm2_cmirrord_systemd_red_hat.service.in index e482b9a99e..fc73aea4e9 100644 --- a/scripts/lvm2_cmirrord_systemd_red_hat.service.in +++ b/scripts/lvm2_cmirrord_systemd_red_hat.service.in @@ -3,7 +3,7 @@ Description=Clustered LVM mirror log daemon Documentation=man:cmirrord(8) Requires=corosync.service After=corosync.service -Before=remote-fs-pre.target +Before=remote-fs-pre.target shutdown.target DefaultDependencies=no Conflicts=shutdown.target diff --git a/scripts/lvm2_lvmetad_systemd_red_hat.service.in b/scripts/lvm2_lvmetad_systemd_red_hat.service.in index 92e6d695f1..960f32dab7 100644 --- a/scripts/lvm2_lvmetad_systemd_red_hat.service.in +++ b/scripts/lvm2_lvmetad_systemd_red_hat.service.in @@ -2,6 +2,7 @@ Description=LVM2 metadata daemon Documentation=man:lvmetad(8) Requires=lvm2-lvmetad.socket +Before=shutdown.target After=lvm2-lvmetad.socket DefaultDependencies=no Conflicts=shutdown.target diff --git a/scripts/lvm2_lvmpolld_systemd_red_hat.service.in b/scripts/lvm2_lvmpolld_systemd_red_hat.service.in index 4ad4e61990..a06cbe985f 100644 --- a/scripts/lvm2_lvmpolld_systemd_red_hat.service.in +++ b/scripts/lvm2_lvmpolld_systemd_red_hat.service.in @@ -2,6 +2,7 @@ Description=LVM2 poll daemon Documentation=man:lvmpolld(8) Requires=lvm2-lvmpolld.socket +Before=shutdown.target After=lvm2-lvmpolld.socket DefaultDependencies=no Conflicts=shutdown.target diff --git a/scripts/lvm2_monitoring_systemd_red_hat.service.in b/scripts/lvm2_monitoring_systemd_red_hat.service.in index ebc069e250..4bf87a0dd1 100644 --- a/scripts/lvm2_monitoring_systemd_red_hat.service.in +++ b/scripts/lvm2_monitoring_systemd_red_hat.service.in @@ -3,7 +3,7 @@ Description=Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progres Documentation=man:dmeventd(8) man:lvcreate(8) man:lvchange(8) man:vgchange(8) Requires=dm-event.socket lvm2-lvmetad.socket After=dm-event.socket dm-event.service lvm2-lvmetad.socket lvm2-activation.service lvm2-lvmetad.service -Before=local-fs-pre.target +Before=local-fs-pre.target shutdown.target DefaultDependencies=no Conflicts=shutdown.target -- 2.16.4 ++++++ bug-1158358_bcache-reduce-MAX_IO-to-256.patch ++++++ From a991664dec02b62681a4a3eea7fbee2fc3a88700 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 24 Aug 2018 14:46:51 -0500 Subject: [PATCH] bcache: reduce MAX_IO to 256 This is the number of concurrent async io requests that the scan layer will submit to the bcache layer. There will be an open fd for each of these, so it is best to keep this well below the default limit for max open files (1024), otherwise lvm may get EMFILE from open(2) when there are around 1024 devices to scan on the system. --- lib/device/bcache.c | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index b1f7d2a..531d83b 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -196,7 +196,15 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, return true; } -#define MAX_IO 1024 +/* + * MAX_IO is returned to the layer above via bcache_max_prefetches() which + * tells the caller how many devices to submit io for concurrently. There will + * be an open file descriptor for each of these, so keep it low enough to avoid + * reaching the default max open file limit (1024) when there are over 1024 + * devices being scanned. + */ + +#define MAX_IO 256 #define MAX_EVENT 64 static bool _async_wait(struct io_engine *ioe, io_complete_fn fn) -- 1.8.3.1 ++++++ bug-1163526_suse-special-global-filter-compat.patch ++++++ diff -Nupr a/lib/config/config_settings.h b/lib/config/config_settings.h --- a/lib/config/config_settings.h 2020-02-26 14:42:52.145985792 +0800 +++ b/lib/config/config_settings.h 2020-02-26 14:44:30.785268395 +0800 @@ -304,6 +304,15 @@ cfg_array(devices_filter_CFG, "filter", "filter = [ \"a|^/dev/hda8$|\", \"r|.*/|\" ]\n" "#\n") +cfg(devices_global_filter_compat_CFG, "global_filter_compat", devices_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, 1, vsn(2, 2, 98), NULL, 0, NULL, + "Limit the block devices that are used by LVM system components.\n" + "This config item global_filter_compat is SUSE special.\n" + "The default value is 1, which means the devices/global_filter\n" + "behaviour is same as before. When the value is 0, user should\n" + "use global_filter to control system-wide software, e.g. \n" + "udev and lvmetad\n" + "global_filter_compat are not opened by LVM.\n") + cfg_array(devices_global_filter_CFG, "global_filter", devices_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_STRING, "#Sa|.*/|", vsn(2, 2, 98), NULL, 0, NULL, "Limit the block devices that are used by LVM system components.\n" "Because devices/filter may be overridden from the command line, it is\n" diff -Nupr a/lib/label/label.c b/lib/label/label.c --- a/lib/label/label.c 2020-02-26 14:43:06.657880252 +0800 +++ b/lib/label/label.c 2020-02-26 15:09:16.290464500 +0800 @@ -977,6 +977,7 @@ int label_scan_pvscan_all(struct cmd_con struct dev_iter *iter; struct device_list *devl, *devl2; struct device *dev; + struct dev_filter *filter; log_debug_devs("Finding devices to scan"); @@ -992,7 +993,16 @@ int label_scan_pvscan_all(struct cmd_con */ dev_cache_scan(); - if (!(iter = dev_iter_create(cmd->lvmetad_filter, 0))) { + /* + * See bsc#1163526 + * stable-2.02 commit c527a0cbfc3 changed filter usage. + */ + if (find_config_tree_bool(cmd, devices_global_filter_compat_CFG, NULL)) { + filter = cmd->full_filter; + } else { + filter = cmd->lvmetad_filter; + } + if (!(iter = dev_iter_create(filter, 0))) { log_error("Scanning failed to get devices."); return 0; } @@ -1030,7 +1040,7 @@ int label_scan_pvscan_all(struct cmd_con return_0; } - _scan_list(cmd, cmd->lvmetad_filter, &all_devs, NULL); + _scan_list(cmd, filter, &all_devs, NULL); dm_list_iterate_items_safe(devl, devl2, &all_devs) { dm_list_del(&devl->list); diff -Nupr a/man/lvmetad.8_main b/man/lvmetad.8_main --- a/man/lvmetad.8_main 2020-02-26 14:42:41.450063577 +0800 +++ b/man/lvmetad.8_main 2020-02-26 15:29:11.797769716 +0800 @@ -47,6 +47,24 @@ system, or if the global_filter has chan This can be done by running pvscan --cache, or it will be done automatically by the next LVM command that's run. +The global_filter behaviour had been changed from upstream stable-2.02 +branch patch: c527a0cbfc391645d30407d2dc4a30275c6472f1. Before this patch, +command like 'pvscan --cache' didn't use devices/global_filter but devices/filter. +This patch fixed the wrong behaviour. The upstream code change is +"technically right", but for compatibility, SUSE introduce a new configure +item global_filter_compat. It default value is 1, It means lvm2 uses +devices/filter & devices/global_filter as before. When user sets this value 0, +It will make udev/lvmetad to use devices/global_filter. The code change makes +lvm2 filter behaviour seem unchange in default. In another word, the customer +doesn't need to set up anything. This patch is SUSE special, customer can use +global_filter_compat=0 to make lvm2 filter bahaviour like upstream code. + +At last, the lvm.conf in rpm package also added this config item. +When system does update, the lvm.conf will be copied to /etc/lvm/lvm.conf.rpmnew. +It can make customer to notice something different in lvm.conf.new. User +can use 'lvmconfig -l devices/global_filter_compat' to check if the lvm2 +contains this new item. + When lvmetad is not used, LVM commands revert to scanning disks for LVM metadata. ++++++ bug-1164126_lvmetad-fix-heap-memory-leak.patch ++++++ From bcf9556b8fcd16ad8997f80cc92785f295c66701 Mon Sep 17 00:00:00 2001 From: wangjufeng <wangjufeng(a)huawei.com> Date: Wed, 5 Feb 2020 13:39:39 +0800 Subject: [PATCH] lvmetad: fix heap memory leak error could be reproduced by calling pvs periodically: #!/bin/bash while : do pvs sleep 1 done use top command to watch RES memory of lvmetad. After a few minutes, its RES memory will grow for a few KB. Then stop calling pvs, while its RES will not decrease. This is because, when lvmetad make reponse for clent request, it will malloc new chunk for s->vgid_to_metadata, while actually the new chunk should be added to the reponse dm_config_tree, or it will make the chunk list of s->vgid_to_metadata keep growing. Signed-off-by: wangjufeng <wangjufeng(a)huawei.com> Reviewed-by: liuzhiqiang <liuzhiqiang26(a)huawei.com> Reviewed-by: guiyao <guiyao(a)huawei.com> --- daemons/lvmetad/lvmetad-core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemons/lvmetad/lvmetad-core.c b/daemons/lvmetad/lvmetad-core.c index 2628022e8..76aa62cc1 100644 --- a/daemons/lvmetad/lvmetad-core.c +++ b/daemons/lvmetad/lvmetad-core.c @@ -730,7 +730,7 @@ static response vg_lookup(lvmetad_state *s, request r) if (!(res.cft->root = n = dm_config_create_node(res.cft, "response"))) goto nomem_un; - if (!(n->v = dm_config_create_value(cft))) + if (!(n->v = dm_config_create_value(res.cft))) goto nomem_un; n->parent = res.cft->root; -- 2.12.3 ++++++ bug-1164718_01-vgcreate-close-exclusive-fd-after-pvcreate.patch ++++++ From a75eb8d74c367b88d5b323d5dfb2e6556f2ad680 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 1 Aug 2018 10:26:28 -0500 Subject: [PATCH] vgcreate: close exclusive fd after pvcreate When vgcreate does an automatic pvcreate, it opens the dev with O_EXCL to ensure no other subsystem is using the device. This exclusive fd remained in bcache and prevented activation parts of lvm from using the dev. This appeared with vgcreate of a sanlock VG because of the unique combination where the dev is not yet a PV, so pvcreate is needed, and the vgcreate also creates and activates an internal LV for sanlock. Fix this by closing the exclusive fd after it's used by pvcreate so that it won't interfere with other bits of lvm that may try to use the device. --- lib/label/label.c | 6 +++--- tools/toollib.c | 7 +++++++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/lib/label/label.c b/lib/label/label.c index ac37713..bafa543 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -1190,7 +1190,7 @@ bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data) return false; } - if (!(dev->flags & DEV_BCACHE_WRITE)) { + if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); bcache_invalidate_fd(scan_bcache, dev->bcache_fd); @@ -1236,7 +1236,7 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) return false; } - if (!(dev->flags & DEV_BCACHE_WRITE)) { + if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); bcache_invalidate_fd(scan_bcache, dev->bcache_fd); @@ -1282,7 +1282,7 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) return false; } - if (!(dev->flags & DEV_BCACHE_WRITE)) { + if (_in_bcache(dev) && !(dev->flags & DEV_BCACHE_WRITE)) { /* FIXME: avoid tossing out bcache blocks just to replace fd. */ log_debug("Close and reopen to write %s", dev_name(dev)); bcache_invalidate_fd(scan_bcache, dev->bcache_fd); diff --git a/tools/toollib.c b/tools/toollib.c index 413937f..6ae78bd 100644 --- a/tools/toollib.c +++ b/tools/toollib.c @@ -5872,6 +5872,13 @@ do_command: pd->name); } + /* + * Don't keep devs open excl in bcache because the excl will prevent + * using that dev elsewhere. + */ + dm_list_iterate_items(devl, &rescan_devs) + label_scan_invalidate(devl->dev); + dm_list_iterate_items(pd, &pp->arg_fail) log_debug("%s: command failed for %s.", cmd->command->name, pd->name); -- 1.8.3.1 ++++++ bug-1164718_02-io-use-sync-io-if-aio-fails.patch ++++++ From f8ce9bf3bc38df8f89021517d7967d94ace129df Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 16 Nov 2018 12:21:20 -0600 Subject: [PATCH] io: use sync io if aio fails io_setup() for aio may fail if a system has reached the aio request limit. In this case, fall back to using sync io. Also, lvm use of aio can be disabled entirely with config setting global/use_aio=0. The system limit for aio requests can be seen from /proc/sys/fs/aio-max-nr The current usage of aio requests can be seen from /proc/sys/fs/aio-nr The system limit for aio requests can be increased by setting fs.aio-max-nr using sysctl. Also add last-byte limit to the sync io code. --- lib/commands/toolcontext.c | 2 ++ lib/config/config_settings.h | 3 +++ lib/config/defaults.h | 1 + lib/device/bcache.c | 42 ++++++++++++++++++++++++++++++++++++++++++ lib/label/label.c | 17 +++++++++++++---- lib/misc/lvm-globals.c | 11 +++++++++++ lib/misc/lvm-globals.h | 2 ++ 7 files changed, 74 insertions(+), 4 deletions(-) diff --git a/lib/commands/toolcontext.c b/lib/commands/toolcontext.c index c9596e2..ecd8fd7 100644 --- a/lib/commands/toolcontext.c +++ b/lib/commands/toolcontext.c @@ -333,6 +333,8 @@ static void _init_logging(struct cmd_context *cmd) find_config_tree_bool(cmd, global_test_CFG, NULL); init_test(cmd->default_settings.test); + init_use_aio(find_config_tree_bool(cmd, global_use_aio_CFG, NULL)); + /* Settings for logging to file */ if (find_config_tree_bool(cmd, log_overwrite_CFG, NULL)) append = 0; diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index f49709c..3433d0c 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -935,6 +935,9 @@ cfg(global_lvdisplay_shows_full_device_path_CFG, "lvdisplay_shows_full_device_pa "Previously this was always shown as /dev/vgname/lvname even when that\n" "was never a valid path in the /dev filesystem.\n") +cfg(global_use_aio_CFG, "use_aio", global_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_USE_AIO, vsn(2, 2, 183), NULL, 0, NULL, + "Use async I/O when reading and writing devices.\n") + cfg(global_use_lvmetad_CFG, "use_lvmetad", global_CFG_SECTION, 0, CFG_TYPE_BOOL, DEFAULT_USE_LVMETAD, vsn(2, 2, 93), "@DEFAULT_USE_LVMETAD@", 0, NULL, "Use lvmetad to cache metadata and reduce disk scanning.\n" "When enabled (and running), lvmetad provides LVM commands with VG\n" diff --git a/lib/config/defaults.h b/lib/config/defaults.h index 7cebd84..fc2f603 100644 --- a/lib/config/defaults.h +++ b/lib/config/defaults.h @@ -59,6 +59,7 @@ #define DEFAULT_METADATA_READ_ONLY 0 #define DEFAULT_LVDISPLAY_SHOWS_FULL_DEVICE_PATH 0 #define DEFAULT_UNKNOWN_DEVICE_NAME "[unknown]" +#define DEFAULT_USE_AIO 1 #define DEFAULT_SANLOCK_LV_EXTEND_MB 256 diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 6235256..571ee7a 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -388,6 +388,48 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, return false; } + /* + * If bcache block goes past where lvm wants to write, then clamp it. + */ + if ((d == DIR_WRITE) && _last_byte_offset && (fd == _last_byte_fd)) { + uint64_t offset = where; + uint64_t nbytes = len; + sector_t limit_nbytes = 0; + sector_t extra_nbytes = 0; + + if (offset > _last_byte_offset) { + log_error("Limit write at %llu len %llu beyond last byte %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)_last_byte_offset); + return false; + } + + if (offset + nbytes > _last_byte_offset) { + limit_nbytes = _last_byte_offset - offset; + if (limit_nbytes % _last_byte_sector_size) + extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + + if (extra_nbytes) { + log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)(limit_nbytes + extra_nbytes)); + nbytes = limit_nbytes + extra_nbytes; + } else { + log_debug("Limit write at %llu len %llu to len %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes); + nbytes = limit_nbytes; + } + } + + where = offset; + len = nbytes; + } + while (len) { do { if (d == DIR_READ) diff --git a/lib/label/label.c b/lib/label/label.c index 5377847..b26ff33 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -798,7 +798,7 @@ out: static int _setup_bcache(int cache_blocks) { - struct io_engine *ioe; + struct io_engine *ioe = NULL; if (cache_blocks < MIN_BCACHE_BLOCKS) cache_blocks = MIN_BCACHE_BLOCKS; @@ -806,9 +806,18 @@ static int _setup_bcache(int cache_blocks) if (cache_blocks > MAX_BCACHE_BLOCKS) cache_blocks = MAX_BCACHE_BLOCKS; - if (!(ioe = create_async_io_engine())) { - log_error("Failed to create bcache io engine."); - return 0; + if (use_aio()) { + if (!(ioe = create_async_io_engine())) { + log_warn("Failed to set up async io, using sync io."); + init_use_aio(0); + } + } + + if (!ioe) { + if (!(ioe = create_sync_io_engine())) { + log_error("Failed to set up sync io."); + return 0; + } } if (!(scan_bcache = bcache_create(BCACHE_BLOCK_SIZE_IN_SECTORS, cache_blocks, ioe))) { diff --git a/lib/misc/lvm-globals.c b/lib/misc/lvm-globals.c index 9941489..82c5706 100644 --- a/lib/misc/lvm-globals.c +++ b/lib/misc/lvm-globals.c @@ -24,6 +24,7 @@ static int _verbose_level = VERBOSE_BASE_LEVEL; static int _silent = 0; static int _test = 0; +static int _use_aio = 0; static int _md_filtering = 0; static int _internal_filtering = 0; static int _fwraid_filtering = 0; @@ -71,6 +72,11 @@ void init_test(int level) _test = level; } +void init_use_aio(int use_aio) +{ + _use_aio = use_aio; +} + void init_md_filtering(int level) { _md_filtering = level; @@ -227,6 +233,11 @@ int test_mode(void) return _test; } +int use_aio(void) +{ + return _use_aio; +} + int md_filtering(void) { return _md_filtering; diff --git a/lib/misc/lvm-globals.h b/lib/misc/lvm-globals.h index b383891..f985cfa 100644 --- a/lib/misc/lvm-globals.h +++ b/lib/misc/lvm-globals.h @@ -25,6 +25,7 @@ enum dev_ext_e; void init_verbose(int level); void init_silent(int silent); void init_test(int level); +void init_use_aio(int use_aio); void init_md_filtering(int level); void init_internal_filtering(int level); void init_fwraid_filtering(int level); @@ -58,6 +59,7 @@ const char *get_cmd_name(void); void set_sysfs_dir_path(const char *path); int test_mode(void); +int use_aio(void); int md_filtering(void); int internal_filtering(void); int fwraid_filtering(void); -- 1.8.3.1 ++++++ bug-1164718_03-bcache-sync-io-fixes.patch ++++++ From cb5405ded86209979eef54433d0b90942b2f976a Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 16 Nov 2018 13:09:29 -0600 Subject: [PATCH] bcache: sync io fixes fix lseek error check fix read/write error checks handle zero return from read and write don't return an error for short io fix partial read/write loop --- lib/device/bcache.c | 69 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 22 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 571ee7a..7384a32 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -328,7 +328,7 @@ struct io_engine *create_async_io_engine(void) e->aio_context = 0; r = io_setup(MAX_IO, &e->aio_context); if (r < 0) { - log_warn("io_setup failed"); + log_debug("io_setup failed %d", r); dm_free(e); return NULL; } @@ -371,8 +371,11 @@ static void _sync_destroy(struct io_engine *ioe) static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, sector_t sb, sector_t se, void *data, void *context) { - int r; - uint64_t len = (se - sb) * 512, where; + int rv; + off_t off; + uint64_t where; + uint64_t pos = 0; + uint64_t len = (se - sb) * 512; struct sync_engine *e = _to_sync(ioe); struct sync_io *io = malloc(sizeof(*io)); if (!io) { @@ -381,11 +384,17 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, } where = sb * 512; - r = lseek(fd, where, SEEK_SET); - if (r < 0) { - log_warn("unable to seek to position %llu", (unsigned long long) where); - free(io); - return false; + + off = lseek(fd, where, SEEK_SET); + if (off == (off_t) -1) { + log_warn("Device seek error %d for offset %llu", errno, (unsigned long long)where); + free(io); + return false; + } + if (off != (off_t) where) { + log_warn("Device seek failed for offset %llu", (unsigned long long)where); + free(io); + return false; } /* @@ -430,28 +439,44 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, len = nbytes; } - while (len) { - do { - if (d == DIR_READ) - r = read(fd, data, len); - else - r = write(fd, data, len); + while (pos < len) { + if (d == DIR_READ) + rv = read(fd, (char *)data + pos, len - pos); + else + rv = write(fd, (char *)data + pos, len - pos); - } while ((r < 0) && ((r == EINTR) || (r == EAGAIN))); + if (rv == -1 && errno == EINTR) + continue; + if (rv == -1 && errno == EAGAIN) + continue; + + if (!rv) + break; - if (r < 0) { - log_warn("io failed %d", r); + if (rv < 0) { + if (d == DIR_READ) + log_debug("Device read error %d offset %llu len %llu", errno, + (unsigned long long)(where + pos), + (unsigned long long)(len - pos)); + else + log_debug("Device write error %d offset %llu len %llu", errno, + (unsigned long long)(where + pos), + (unsigned long long)(len - pos)); free(io); return false; - } - - len -= r; + } + pos += rv; } - if (len) { - log_warn("short io %u bytes remaining", (unsigned) len); + if (pos < len) { + if (d == DIR_READ) + log_warn("Device read short %u bytes remaining", (unsigned)(len - pos)); + else + log_warn("Device write short %u bytes remaining", (unsigned)(len - pos)); + /* free(io); return false; + */ } -- 1.8.3.1 ++++++ bug-1164718_04-lvconvert-restrict-command-matching-for-no-option-va.patch ++++++ From 9e296c9c6f3fa92fdc839b30012b51ee28d23d1a Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 23 Jul 2018 11:08:12 -0500 Subject: [PATCH] lvconvert: restrict command matching for no option variant The 'lvconvert LV' command def has caused multiple problems for command matching because it matches the required options of any lvconvert command. Any lvconvert with incorrect options ends up matching 'lvconvert LV', which then produces an error about incorrect options being used for 'lvconvert LV'. This prevents suggestions from nearest-command partial command matches. Add a special case for 'lvconvert LV' so that it won't be used as a partial match for a command that has options specified. --- lib/commands/toolcontext.h | 1 + tools/command-lines.in | 2 +- tools/lvmcmdline.c | 14 ++++++++++++++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/lib/commands/toolcontext.h b/lib/commands/toolcontext.h index bc0573671..da5d582fb 100644 --- a/lib/commands/toolcontext.h +++ b/lib/commands/toolcontext.h @@ -95,6 +95,7 @@ struct cmd_context { char **argv; struct arg_values *opt_arg_values; struct dm_list arg_value_groups; + int opt_count; /* total number of options (beginning with - or --) */ /* * Position args remaining after command name diff --git a/tools/command-lines.in b/tools/command-lines.in index 9d407d08a..b7aefa349 100644 --- a/tools/command-lines.in +++ b/tools/command-lines.in @@ -700,7 +700,7 @@ RULE: all and lv_is_converting # for compat since this was how it used to be done. lvconvert LV_mirror_raid OO: OO_LVCONVERT -ID: lvconvert_start_poll +ID: lvconvert_plain DESC: Poll LV to continue conversion (also see --startpoll) DESC: or waits till conversion/mirror syncing is finished FLAGS: SECONDARY_SYNTAX diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c index 0dd24ec05..298ead7f9 100644 --- a/tools/lvmcmdline.c +++ b/tools/lvmcmdline.c @@ -118,6 +118,7 @@ static const struct command_function _command_functions[CMD_COUNT] = { /* lvconvert utility to trigger polling on an LV. */ { lvconvert_start_poll_CMD, lvconvert_start_poll_cmd }, + { lvconvert_plain_CMD, lvconvert_start_poll_cmd }, /* lvconvert utilities for creating/maintaining thin and cache objects. */ { lvconvert_to_thinpool_CMD, lvconvert_to_pool_cmd }, @@ -1578,6 +1579,17 @@ static struct command *_find_command(struct cmd_context *cmd, const char *path, if (arg_is_set(cmd, help_ARG) || arg_is_set(cmd, help2_ARG) || arg_is_set(cmd, longhelp_ARG) || arg_is_set(cmd, version_ARG)) return &commands[i]; + /* + * The 'lvconvert LV' cmd def matches any lvconvert cmd which throws off + * nearest-command partial-match suggestions. Make it a special case so + * that it won't be used as a close match. If the command has any option + * set (other than -v), don't attempt to match it to 'lvconvert LV'. + */ + if (commands[i].command_enum == lvconvert_plain_CMD) { + if (cmd->opt_count - cmd->opt_arg_values[verbose_ARG].count) + continue; + } + match_required = 0; /* required parameters that match */ match_ro = 0; /* required opt_args that match */ match_rp = 0; /* required pos_args that match */ @@ -2096,6 +2108,8 @@ static int _process_command_line(struct cmd_context *cmd, int *argc, char ***arg if (goval == '?') return 0; + cmd->opt_count++; + /* * translate the option value used by getopt into the enum * value (e.g. foo_ARG) from the args array. -- 2.24.0 ++++++ bug-1164718_05-lvmetad-only-disable-if-repair-will-do-something.patch ++++++ From 322d4ed05e348c6d88f3cb880485e5777298c361 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 30 Nov 2018 14:20:43 -0600 Subject: [PATCH] lvmetad: only disable if repair will do something lvconvert --repair would disable lvmetad at the start of the command. This would leave lvmetad disabled even if the command did nothing. Move the step to disable lvmetad until later, just before some actual repair is done. There are now numerous cases where nothing is actually done and lvmetad is not disabled. --- lib/cache/lvmetad.c | 33 +++++++++++++++++++++++++++++---- tools/lvconvert.c | 13 +++++++++++++ tools/lvmcmdline.c | 2 -- 3 files changed, 42 insertions(+), 6 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index acbb52e54..1b48fd344 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -31,6 +31,7 @@ static daemon_handle _lvmetad = { .error = 0 }; static int _lvmetad_use = 0; static int _lvmetad_connected = 0; static int _lvmetad_daemon_pid = 0; +static int _was_connected = 0; static char *_lvmetad_token = NULL; static const char *_lvmetad_socket = NULL; @@ -114,8 +115,10 @@ static int _log_debug_inequality(const char *name, struct dm_config_node *a, str void lvmetad_disconnect(void) { - if (_lvmetad_connected) + if (_lvmetad_connected) { daemon_close(_lvmetad); + _was_connected = 1; + } _lvmetad_connected = 0; _lvmetad_use = 0; @@ -2973,14 +2976,33 @@ int lvmetad_vg_is_foreign(struct cmd_context *cmd, const char *vgname, const cha */ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) { + daemon_handle tmph = { .error = 0 }; daemon_reply reply; + int dis = 0; - if (!_lvmetad_use) - return; + /* + * If we were using lvmetad at the start of the command, but are not + * now, then _was_connected should still be set. In this case we + * want to make a temp connection just to disable it. + */ + if (!_lvmetad_use) { + if (_was_connected) { + /* Create a special temp connection just to send disable */ + tmph = lvmetad_open(_lvmetad_socket); + if (tmph.socket_fd < 0 || tmph.error) { + log_warn("Failed to connect to lvmetad to disable."); + return; + } + dis = 1; + } else { + /* We were never using lvmetad, don't start now. */ + return; + } + } log_debug_lvmetad("Sending lvmetad disabled %s", reason); - reply = daemon_send_simple(_lvmetad, "set_global_info", + reply = daemon_send_simple(tmph, "set_global_info", "token = %s", "skip", "global_disable = " FMTd64, (int64_t)1, "disable_reason = %s", reason, @@ -2994,6 +3016,9 @@ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) log_error("Failed response from lvmetad."); daemon_reply_destroy(reply); + + if (dis) + daemon_close(tmph); } void lvmetad_clear_disabled(struct cmd_context *cmd) diff --git a/tools/lvconvert.c b/tools/lvconvert.c index d7a618dc6..028781fe4 100644 --- a/tools/lvconvert.c +++ b/tools/lvconvert.c @@ -17,6 +17,7 @@ #include "polldaemon.h" #include "lv_alloc.h" #include "lvconvert_poll.h" +#include "lvmetad-client.h" #define MAX_PDATA_ARGS 10 /* Max number of accepted args for d-m-p-d tools */ @@ -1073,6 +1074,9 @@ static int _lvconvert_mirrors_repair(struct cmd_context *cmd, return 1; } + log_warn("WARNING: Disabling lvmetad cache for repair command."); + lvmetad_set_disabled(cmd, LVMETAD_DISABLE_REASON_REPAIR); + failed_mimages = _failed_mirrors_count(lv); failed_logs = _failed_logs_count(lv); @@ -2321,6 +2325,9 @@ static int _lvconvert_thin_pool_repair(struct cmd_context *cmd, goto deactivate_pmslv; } + log_warn("WARNING: Disabling lvmetad cache for repair command."); + lvmetad_set_disabled(cmd, LVMETAD_DISABLE_REASON_REPAIR); + if (!(ret = exec_cmd(cmd, (const char * const *)argv, &status, 1))) { log_error("Repair of thin metadata volume of thin pool %s failed (status:%d). " "Manual repair required!", @@ -2519,6 +2526,9 @@ static int _lvconvert_cache_repair(struct cmd_context *cmd, goto deactivate_pmslv; } + log_warn("WARNING: Disabling lvmetad cache for repair command."); + lvmetad_set_disabled(cmd, LVMETAD_DISABLE_REASON_REPAIR); + if (!(ret = exec_cmd(cmd, (const char * const *)argv, &status, 1))) { log_error("Repair of cache metadata volume of cache %s failed (status:%d). " "Manual repair required!", @@ -3538,6 +3548,9 @@ static int _lvconvert_repair_pvs_raid(struct cmd_context *cmd, struct logical_vo _lvconvert_repair_pvs_raid_ask(cmd, &do_it); if (do_it) { + log_warn("WARNING: Disabling lvmetad cache for repair command."); + lvmetad_set_disabled(cmd, LVMETAD_DISABLE_REASON_REPAIR); + if (!(failed_pvs = _failed_pv_list(lv->vg))) return_0; diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c index 298ead7f9..a9c3e4180 100644 --- a/tools/lvmcmdline.c +++ b/tools/lvmcmdline.c @@ -2914,8 +2914,6 @@ int lvm_run_command(struct cmd_context *cmd, int argc, char **argv) } if (cmd->command->command_enum == lvconvert_repair_CMD) { - log_warn("WARNING: Disabling lvmetad cache for repair command."); - lvmetad_set_disabled(cmd, LVMETAD_DISABLE_REASON_REPAIR); log_warn("WARNING: Not using lvmetad because of repair."); lvmetad_make_unused(cmd); } -- 2.24.0 ++++++ bug-1164718_06-lvmetad-fix-disabling-in-previous-commit.patch ++++++ From 9764ee0b3f21ed717e1b6248ce4c41dd552bb0ec Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 30 Nov 2018 15:49:03 -0600 Subject: [PATCH] lvmetad: fix disabling in previous commit it broke the case where a connection already exists. --- lib/cache/lvmetad.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index 1b48fd3..f068fc6 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -2978,7 +2978,7 @@ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) { daemon_handle tmph = { .error = 0 }; daemon_reply reply; - int dis = 0; + int tmp_con = 0; /* * If we were using lvmetad at the start of the command, but are not @@ -2993,7 +2993,7 @@ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) log_warn("Failed to connect to lvmetad to disable."); return; } - dis = 1; + tmp_con = 1; } else { /* We were never using lvmetad, don't start now. */ return; @@ -3002,13 +3002,23 @@ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) log_debug_lvmetad("Sending lvmetad disabled %s", reason); - reply = daemon_send_simple(tmph, "set_global_info", + if (tmp_con) + reply = daemon_send_simple(tmph, "set_global_info", "token = %s", "skip", "global_disable = " FMTd64, (int64_t)1, "disable_reason = %s", reason, "pid = " FMTd64, (int64_t)getpid(), "cmd = %s", get_cmd_name(), NULL); + else + reply = daemon_send_simple(_lvmetad, "set_global_info", + "token = %s", "skip", + "global_disable = " FMTd64, (int64_t)1, + "disable_reason = %s", reason, + "pid = " FMTd64, (int64_t)getpid(), + "cmd = %s", get_cmd_name(), + NULL); + if (reply.error) log_error("Failed to send message to lvmetad %d", reply.error); @@ -3017,7 +3027,7 @@ void lvmetad_set_disabled(struct cmd_context *cmd, const char *reason) daemon_reply_destroy(reply); - if (dis) + if (tmp_con) daemon_close(tmph); } -- 1.8.3.1 ++++++ bug-1164718_07-filter-add-config-setting-to-skip-scanning-LVs.patch ++++++ From 6a5575e9597c50a7ebc89895de154db3fa13d753 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 29 Aug 2018 13:14:18 -0500 Subject: [PATCH 1/2] filter: add config setting to skip scanning LVs devices/scan_lvs (default 1) determines whether lvm will scan LVs for layered PVs. The lvm behavior has always been to scan LVs, but it's rare for LVs to have layered PVs, and much more common for there to be many LVs that substantially slow down scanning with no benefit. This is implemented in the usable filter, and has the same effect as listing all LVs in the global_filter. --- lib/activate/activate.h | 1 + lib/activate/dev_manager.c | 5 +++++ lib/commands/toolcontext.c | 4 ++-- lib/config/config_settings.h | 3 +++ lib/config/defaults.h | 2 ++ lib/filters/filter-usable.c | 27 ++++++++++++++++++++++----- lib/filters/filter.h | 2 +- 7 files changed, 36 insertions(+), 8 deletions(-) diff --git a/lib/activate/activate.h b/lib/activate/activate.h index d2c6db7c0..524d2bfa1 100644 --- a/lib/activate/activate.h +++ b/lib/activate/activate.h @@ -234,6 +234,7 @@ struct dev_usable_check_params { unsigned int check_suspended:1; unsigned int check_error_target:1; unsigned int check_reserved:1; + unsigned int check_lv:1; }; /* diff --git a/lib/activate/dev_manager.c b/lib/activate/dev_manager.c index d281b6047..1f4ec653b 100644 --- a/lib/activate/dev_manager.c +++ b/lib/activate/dev_manager.c @@ -640,6 +640,11 @@ int device_is_usable(struct device *dev, struct dev_usable_check_params check) } } + if (check.check_lv && uuid && !strncmp(uuid, "LVM-", 4)) { + /* Skip LVs */ + goto out; + } + if (check.check_reserved && uuid && (!strncmp(uuid, CRYPT_TEMP, sizeof(CRYPT_TEMP) - 1) || !strncmp(uuid, STRATIS, sizeof(STRATIS) - 1))) { diff --git a/lib/commands/toolcontext.c b/lib/commands/toolcontext.c index ecd8fd775..3289b384c 100644 --- a/lib/commands/toolcontext.c +++ b/lib/commands/toolcontext.c @@ -1115,7 +1115,7 @@ static struct dev_filter *_init_lvmetad_filter_chain(struct cmd_context *cmd) nr_filt++; /* usable device filter. Required. */ - if (!(filters[nr_filt] = usable_filter_create(cmd->dev_types, + if (!(filters[nr_filt] = usable_filter_create(cmd, cmd->dev_types, lvmetad_used() ? FILTER_MODE_PRE_LVMETAD : FILTER_MODE_NO_LVMETAD))) { log_error("Failed to create usabled device filter"); @@ -1235,7 +1235,7 @@ int init_filters(struct cmd_context *cmd, unsigned load_persistent_cache) } nr_filt++; } - if (!(filter_components[nr_filt] = usable_filter_create(cmd->dev_types, FILTER_MODE_POST_LVMETAD))) { + if (!(filter_components[nr_filt] = usable_filter_create(cmd, cmd->dev_types, FILTER_MODE_POST_LVMETAD))) { log_verbose("Failed to create usable device filter."); goto bad; } diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 3433d0ca7..8fe7ec98c 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -345,6 +345,9 @@ cfg(devices_sysfs_scan_CFG, "sysfs_scan", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, "This is a quick way of filtering out block devices that are not\n" "present on the system. sysfs must be part of the kernel and mounted.)\n") +cfg(devices_scan_lvs_CFG, "scan_lvs", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, DEFAULT_SCAN_LVS, vsn(2, 2, 182), NULL, 0, NULL, + "Scan LVM LVs for layered PVs.\n") + cfg(devices_multipath_component_detection_CFG, "multipath_component_detection", devices_CFG_SECTION, 0, CFG_TYPE_BOOL, DEFAULT_MULTIPATH_COMPONENT_DETECTION, vsn(2, 2, 89), NULL, 0, NULL, "Ignore devices that are components of DM multipath devices.\n") diff --git a/lib/config/defaults.h b/lib/config/defaults.h index fc2f603fc..12d8e1d53 100644 --- a/lib/config/defaults.h +++ b/lib/config/defaults.h @@ -267,4 +267,6 @@ #define DEFAULT_THIN_POOL_AUTOEXTEND_THRESHOLD 100 #define DEFAULT_THIN_POOL_AUTOEXTEND_PERCENT 20 +#define DEFAULT_SCAN_LVS 1 + #endif /* _LVM_DEFAULTS_H */ diff --git a/lib/filters/filter-usable.c b/lib/filters/filter-usable.c index 2de2a0f2e..bb392a9db 100644 --- a/lib/filters/filter-usable.c +++ b/lib/filters/filter-usable.c @@ -20,6 +20,11 @@ #include "dev-ext-udev-constants.h" #endif +struct filter_data { + filter_mode_t mode; + int skip_lvs; +}; + static const char *_too_small_to_hold_pv_msg = "Too small to hold a PV"; static int _native_check_pv_min_size(struct device *dev) @@ -101,7 +106,9 @@ static int _check_pv_min_size(struct device *dev) static int _passes_usable_filter(struct dev_filter *f, struct device *dev) { - filter_mode_t mode = *((filter_mode_t *) f->private); + struct filter_data *data = f->private; + filter_mode_t mode = data->mode; + int skip_lvs = data->skip_lvs; struct dev_usable_check_params ucp = {0}; int r = 1; @@ -114,6 +121,7 @@ static int _passes_usable_filter(struct dev_filter *f, struct device *dev) ucp.check_suspended = ignore_suspended_devices(); ucp.check_error_target = 1; ucp.check_reserved = 1; + ucp.check_lv = skip_lvs; break; case FILTER_MODE_PRE_LVMETAD: ucp.check_empty = 1; @@ -121,6 +129,7 @@ static int _passes_usable_filter(struct dev_filter *f, struct device *dev) ucp.check_suspended = 0; ucp.check_error_target = 1; ucp.check_reserved = 1; + ucp.check_lv = skip_lvs; break; case FILTER_MODE_POST_LVMETAD: ucp.check_empty = 0; @@ -128,6 +137,7 @@ static int _passes_usable_filter(struct dev_filter *f, struct device *dev) ucp.check_suspended = ignore_suspended_devices(); ucp.check_error_target = 0; ucp.check_reserved = 0; + ucp.check_lv = skip_lvs; break; } @@ -161,8 +171,9 @@ static void _usable_filter_destroy(struct dev_filter *f) dm_free(f); } -struct dev_filter *usable_filter_create(struct dev_types *dt __attribute__((unused)), filter_mode_t mode) +struct dev_filter *usable_filter_create(struct cmd_context *cmd, struct dev_types *dt __attribute__((unused)), filter_mode_t mode) { + struct filter_data *data; struct dev_filter *f; if (!(f = dm_zalloc(sizeof(struct dev_filter)))) { @@ -173,14 +184,20 @@ struct dev_filter *usable_filter_create(struct dev_types *dt __attribute__((unus f->passes_filter = _passes_usable_filter; f->destroy = _usable_filter_destroy; f->use_count = 0; - if (!(f->private = dm_zalloc(sizeof(filter_mode_t)))) { + + if (!(data = dm_zalloc(sizeof(struct filter_data)))) { log_error("Usable device filter mode allocation failed"); dm_free(f); return NULL; } - *((filter_mode_t *) f->private) = mode; - log_debug_devs("Usable device filter initialised."); + data->mode = mode; + + data->skip_lvs = !find_config_tree_bool(cmd, devices_scan_lvs_CFG, NULL); + + f->private = data; + + log_debug_devs("Usable device filter initialised (scan_lvs %d).", !data->skip_lvs); return f; } diff --git a/lib/filters/filter.h b/lib/filters/filter.h index 624582738..45572eb6b 100644 --- a/lib/filters/filter.h +++ b/lib/filters/filter.h @@ -52,7 +52,7 @@ typedef enum { FILTER_MODE_PRE_LVMETAD, FILTER_MODE_POST_LVMETAD } filter_mode_t; -struct dev_filter *usable_filter_create(struct dev_types *dt, filter_mode_t mode); +struct dev_filter *usable_filter_create(struct cmd_context *cmd, struct dev_types *dt, filter_mode_t mode); int persistent_filter_load(struct dev_filter *f, struct dm_config_tree **cft_out); -- 2.24.0 ++++++ bug-1164718_08-pvscan-lvmetad-init-should-set-updating-before-scann.patch ++++++ From dcf8f3111aea1179be1083ace772f3108b06c508 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 13 Feb 2019 17:10:49 -0600 Subject: [PATCH] pvscan: lvmetad init should set updating before scanning When pvscan needs to initialize lvmetad (e.g. lvmetad has just started and is empty), it should set the lvmetad state to "updating" before it scans any devices. Otherwise, many parallel pvscans will try to initialize lvmetad, and in some cases earlier pvscans with fewer devices information may replace newer pvscans with more recent information. --- lib/cache/lvmetad.c | 34 +++++++++++++++++++++------------- tools/pvscan.c | 9 ++++----- 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index f068fc6..86a880a 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -313,6 +313,7 @@ retry: * The caller should do a disk scan to populate lvmetad. */ if (!strcmp(daemon_token, "none")) { + log_debug_lvmetad("lvmetad initialization needed."); ret = 0; goto out; } @@ -324,10 +325,16 @@ retry: * our global filter. */ if (strcmp(daemon_token, _lvmetad_token)) { + log_debug_lvmetad("lvmetad initialization needed for different filter."); ret = 0; goto out; } + if (wait_start) + log_debug_lvmetad("lvmetad initialized during wait."); + else + log_debug_lvmetad("lvmetad initialized previously."); + out: daemon_reply_destroy(reply); return ret; @@ -2353,23 +2360,11 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) replacing_other_update = 1; } - label_scan_pvscan_all(cmd, &scan_devs); - - lvmcache_pvscan_duplicate_check(cmd); - - if (lvmcache_found_duplicate_pvs()) { - log_warn("WARNING: Scan found duplicate PVs."); - return 0; - } - - log_verbose("Scanning metadata from %d devices to update lvmetad.", - dm_list_size(&scan_devs)); - future_token = _lvmetad_token; _lvmetad_token = (char *) LVMETAD_TOKEN_UPDATE_IN_PROGRESS; if (!_token_update(&replaced_update)) { - log_error("Failed to update lvmetad which had an update in progress."); + log_error("Failed to start lvmetad update."); _lvmetad_token = future_token; return 0; } @@ -2393,6 +2388,10 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) return 0; } + log_verbose("Scanning all devices to initialize lvmetad."); + + label_scan_pvscan_all(cmd, &scan_devs); + log_debug_lvmetad("Telling lvmetad to clear its cache"); reply = _lvmetad_send(cmd, "pv_clear_all", NULL); if (!_lvmetad_handle_reply(reply, "pv_clear_all", "", NULL)) @@ -2402,6 +2401,8 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) was_silent = silent_mode(); init_silent(1); + log_debug_lvmetad("Sending %d devices to lvmetad.", dm_list_size(&scan_devs)); + dm_list_iterate_items_safe(devl, devl2, &scan_devs) { if (sigint_caught()) { ret = 0; @@ -2442,6 +2443,13 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) return 0; } + /* This will disable lvmetad if label scan found duplicates. */ + lvmcache_pvscan_duplicate_check(cmd); + if (lvmcache_found_duplicate_pvs()) { + log_warn("WARNING: Scan found duplicate PVs."); + return 0; + } + /* * If lvmetad is disabled, and no duplicate PVs were seen, then re-enable lvmetad. */ diff --git a/tools/pvscan.c b/tools/pvscan.c index 5587928..daac88f 100644 --- a/tools/pvscan.c +++ b/tools/pvscan.c @@ -381,7 +381,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) all_vgs = 1; goto activate; } - + /* * FIXME: when specific devs are named, we generally don't want to scan * any other devs, but if lvmetad is not yet populated, the first @@ -390,11 +390,10 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) * never scan any devices other than those specified. */ if (!lvmetad_token_matches(cmd)) { - log_verbose("Scanning all devices to initialize lvmetad."); - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0)) { - log_warn("WARNING: Not using lvmetad because cache update failed."); - lvmetad_make_unused(cmd); + log_warn("WARNING: Not updating lvmetad because cache update failed."); + ret = ECMD_FAILED; + goto out; } if (lvmetad_used() && lvmetad_is_disabled(cmd, &reason)) { log_warn("WARNING: Not using lvmetad because %s.", reason); -- 1.8.3.1 ++++++ bug-1164718_09-config-change-scan_lvs-default-to-0.patch ++++++ From 427e8ba3e3186e291ff26f014003d149da6ec2a8 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 20 Feb 2019 14:31:03 -0600 Subject: [PATCH 2/2] config: change scan_lvs default to 0 so that lvm does not scan LVs for PVs by default. --- lib/config/defaults.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/config/defaults.h b/lib/config/defaults.h index 12d8e1d53..7a5d36569 100644 --- a/lib/config/defaults.h +++ b/lib/config/defaults.h @@ -267,6 +267,6 @@ #define DEFAULT_THIN_POOL_AUTOEXTEND_THRESHOLD 100 #define DEFAULT_THIN_POOL_AUTOEXTEND_PERCENT 20 -#define DEFAULT_SCAN_LVS 1 +#define DEFAULT_SCAN_LVS 0 #endif /* _LVM_DEFAULTS_H */ -- 2.24.0 ++++++ bug-1164718_10-config-add-new-setting-io_memory_size.patch ++++++ From 8dbfdb5b737cf916a8b95b8d19eec67a960a6392 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Fri, 1 Mar 2019 13:55:59 -0600 Subject: [PATCH] config: add new setting io_memory_size which defines the amount of memory that lvm will allocate for bcache. Increasing this setting is required if it is smaller than a single copy of VG metadata. --- lib/commands/toolcontext.c | 2 ++ lib/config/config_settings.h | 8 ++++++++ lib/config/defaults.h | 2 ++ lib/label/label.c | 40 ++++++++++++++++++------------------ lib/misc/lvm-globals.c | 10 +++++++++ lib/misc/lvm-globals.h | 3 +++ 6 files changed, 45 insertions(+), 20 deletions(-) diff --git a/lib/commands/toolcontext.c b/lib/commands/toolcontext.c index 3289b384c..25e8b878f 100644 --- a/lib/commands/toolcontext.c +++ b/lib/commands/toolcontext.c @@ -685,6 +685,8 @@ static int _process_config(struct cmd_context *cmd) if (!_init_system_id(cmd)) return_0; + init_io_memory_size(find_config_tree_int(cmd, global_io_memory_size_CFG, NULL)); + return 1; } diff --git a/lib/config/config_settings.h b/lib/config/config_settings.h index 8fe7ec98c..75321da5e 100644 --- a/lib/config/config_settings.h +++ b/lib/config/config_settings.h @@ -1129,6 +1129,14 @@ cfg(global_notify_dbus_CFG, "notify_dbus", global_CFG_SECTION, 0, CFG_TYPE_BOOL, "When enabled, an LVM command that changes PVs, changes VG metadata,\n" "or changes the activation state of an LV will send a notification.\n") +cfg(global_io_memory_size_CFG, "io_memory_size", global_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_INT, DEFAULT_IO_MEMORY_SIZE_KB, vsn(2, 2, 184), NULL, 0, NULL, + "The amount of memory in KiB that LVM allocates to perform disk io.\n" + "LVM performance may benefit from more io memory when there are many\n" + "disks or VG metadata is large. Increasing this size may be necessary\n" + "when a single copy of VG metadata is larger than the current setting.\n" + "This value should usually not be decreased from the default; setting\n" + "it too low can result in lvm failing to read VGs.\n") + cfg(activation_udev_sync_CFG, "udev_sync", activation_CFG_SECTION, 0, CFG_TYPE_BOOL, DEFAULT_UDEV_SYNC, vsn(2, 2, 51), NULL, 0, NULL, "Use udev notifications to synchronize udev and LVM.\n" "The --nodevsync option overrides this setting.\n" diff --git a/lib/config/defaults.h b/lib/config/defaults.h index 7a5d36569..3b4438a0c 100644 --- a/lib/config/defaults.h +++ b/lib/config/defaults.h @@ -269,4 +269,6 @@ #define DEFAULT_SCAN_LVS 0 +#define DEFAULT_IO_MEMORY_SIZE_KB 4096 + #endif /* _LVM_DEFAULTS_H */ diff --git a/lib/label/label.c b/lib/label/label.c index e01608d2c..b94fd8dd6 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -772,33 +772,33 @@ out: } /* - * How many blocks to set up in bcache? Is 1024 a good max? + * num_devs is the number of devices the caller is going to scan. + * When 0 the caller doesn't know, and we use the default cache size. + * When non-zero, allocate at least num_devs bcache blocks. + * num_devs doesn't really tell us how many bcache blocks we'll use + * because it includes lvm devs and non-lvm devs, and each lvm dev + * will often use a number of bcache blocks. * - * Currently, we tell bcache to set up N blocks where N - * is the number of devices that are going to be scanned. - * Reasons why this number may not be be a good choice: - * - * - there may be a lot of non-lvm devices, which - * would make this number larger than necessary - * - * - each lvm device may use more than one cache - * block if the metadata is large enough or it - * uses more than one metadata area, which - * would make this number smaller than it - * should be for the best performance. - * - * This is even more tricky to estimate when lvmetad - * is used, because it's hard to predict how many - * devs might need to be scanned when using lvmetad. - * This currently just sets up bcache with MIN blocks. + * We don't know ahead of time if we will find some VG metadata + * that is larger than the total size of the bcache, which would + * prevent us from reading/writing the VG since we do not dynamically + * increase the bcache size when we find it's too small. In these + * cases the user would need to set io_memory_size to be larger + * than the max VG metadata size (lvm does not impose any limit on + * the metadata size.) */ -#define MIN_BCACHE_BLOCKS 32 +#define MIN_BCACHE_BLOCKS 32 /* 4MB, currently matches DEFAULT_IO_MEMORY_SIZE_KB */ #define MAX_BCACHE_BLOCKS 1024 -static int _setup_bcache(int cache_blocks) +static int _setup_bcache(int num_devs) { struct io_engine *ioe = NULL; + int iomem_kb = io_memory_size(); + int block_size_kb = (BCACHE_BLOCK_SIZE_IN_SECTORS * 512) / 1024; + int cache_blocks; + + cache_blocks = iomem_kb / block_size_kb; if (cache_blocks < MIN_BCACHE_BLOCKS) cache_blocks = MIN_BCACHE_BLOCKS; diff --git a/lib/misc/lvm-globals.c b/lib/misc/lvm-globals.c index 82c57062c..3bd5cac88 100644 --- a/lib/misc/lvm-globals.c +++ b/lib/misc/lvm-globals.c @@ -54,6 +54,7 @@ static char _sysfs_dir_path[PATH_MAX] = ""; static int _dev_disable_after_error_count = DEFAULT_DISABLE_AFTER_ERROR_COUNT; static uint64_t _pv_min_size = (DEFAULT_PV_MIN_SIZE_KB * 1024L >> SECTOR_SHIFT); static const char *_unknown_device_name = DEFAULT_UNKNOWN_DEVICE_NAME; +static int _io_memory_size_kb = DEFAULT_IO_MEMORY_SIZE_KB; void init_verbose(int level) { @@ -387,3 +388,12 @@ void init_unknown_device_name(const char *name) _unknown_device_name = name; } +int io_memory_size(void) +{ + return _io_memory_size_kb; +} + +void init_io_memory_size(int val) +{ + _io_memory_size_kb = val; +} diff --git a/lib/misc/lvm-globals.h b/lib/misc/lvm-globals.h index f985cfa9e..3007cc504 100644 --- a/lib/misc/lvm-globals.h +++ b/lib/misc/lvm-globals.h @@ -53,6 +53,7 @@ void init_pv_min_size(uint64_t sectors); void init_activation_checks(int checks); void init_retry_deactivation(int retry); void init_unknown_device_name(const char *name); +void init_io_memory_size(int val); void set_cmd_name(const char *cmd_name); const char *get_cmd_name(void); @@ -86,6 +87,7 @@ uint64_t pv_min_size(void); int activation_checks(void); int retry_deactivation(void); const char *unknown_device_name(void); +int io_memory_size(void); #define DMEVENTD_MONITOR_IGNORE -1 int dmeventd_monitor_mode(void); @@ -93,4 +95,5 @@ int dmeventd_monitor_mode(void); #define NO_DEV_ERROR_COUNT_LIMIT 0 int dev_disable_after_error_count(void); + #endif -- 2.24.0 ++++++ bug-1164718_11-io-warn-when-metadata-size-approaches-io-memory-size.patch ++++++ From 863a2e693ee95b95463d60fa8b21f4c7c084292c Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 4 Mar 2019 10:57:52 -0600 Subject: [PATCH] io: warn when metadata size approaches io memory size When a single copy of metadata gets within 1MB of the current io_memory_size value, begin printing a warning that the io_memory_size should be increased. --- lib/cache/lvmcache.c | 15 +++++++++++++ lib/cache/lvmcache.h | 3 +++ lib/format_text/format-text.c | 4 ++++ lib/label/label.c | 41 +++++++++++++++++++++++++++++++++++ 4 files changed, 63 insertions(+) diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c index a2ee0cd43..ad40d4c55 100644 --- a/lib/cache/lvmcache.c +++ b/lib/cache/lvmcache.c @@ -3044,3 +3044,18 @@ int lvmcache_scan_mismatch(struct cmd_context *cmd, const char *vgname, const ch return 1; } +static uint64_t _max_metadata_size; + +void lvmcache_save_metadata_size(uint64_t val) +{ + if (!_max_metadata_size) + _max_metadata_size = val; + else if (_max_metadata_size < val) + _max_metadata_size = val; +} + +uint64_t lvmcache_max_metadata_size(void) +{ + return _max_metadata_size; +} + diff --git a/lib/cache/lvmcache.h b/lib/cache/lvmcache.h index bf976e9c6..f43678507 100644 --- a/lib/cache/lvmcache.h +++ b/lib/cache/lvmcache.h @@ -225,4 +225,7 @@ struct volume_group *lvmcache_get_saved_vg(const char *vgid, int precommitted); struct volume_group *lvmcache_get_saved_vg_latest(const char *vgid); void lvmcache_drop_saved_vgid(const char *vgid); +uint64_t lvmcache_max_metadata_size(void); +void lvmcache_save_metadata_size(uint64_t val); + #endif diff --git a/lib/format_text/format-text.c b/lib/format_text/format-text.c index 4160ba810..b9d85a4e1 100644 --- a/lib/format_text/format-text.c +++ b/lib/format_text/format-text.c @@ -1294,6 +1294,10 @@ int read_metadata_location_summary(const struct format_type *fmt, */ vgsummary->mda_checksum = rlocn->checksum; vgsummary->mda_size = rlocn->size; + + /* Keep track of largest metadata size we find. */ + lvmcache_save_metadata_size(rlocn->size); + lvmcache_lookup_mda(vgsummary); if (!text_read_metadata_summary(fmt, dev_area->dev, MDA_CONTENT_REASON(primary_mda), diff --git a/lib/label/label.c b/lib/label/label.c index b94fd8dd6..e96a7fe36 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -21,6 +21,7 @@ #include "bcache.h" #include "toolcontext.h" #include "activate.h" +#include "metadata.h" #include <sys/stat.h> #include <fcntl.h> @@ -29,6 +30,8 @@ int use_full_md_check; +static uint64_t _current_bcache_size_bytes; + /* FIXME Allow for larger labels? Restricted to single sector currently */ /* @@ -806,6 +809,8 @@ static int _setup_bcache(int num_devs) if (cache_blocks > MAX_BCACHE_BLOCKS) cache_blocks = MAX_BCACHE_BLOCKS; + _current_bcache_size_bytes = cache_blocks * BCACHE_BLOCK_SIZE_IN_SECTORS * 512; + if (use_aio()) { if (!(ioe = create_async_io_engine())) { log_warn("Failed to set up async io, using sync io."); @@ -839,6 +844,7 @@ int label_scan(struct cmd_context *cmd) struct dev_iter *iter; struct device_list *devl, *devl2; struct device *dev; + uint64_t max_metadata_size_bytes; log_debug_devs("Finding devices to scan"); @@ -909,6 +915,41 @@ int label_scan(struct cmd_context *cmd) _scan_list(cmd, cmd->full_filter, &all_devs, NULL); + /* + * Metadata could be larger than total size of bcache, and bcache + * cannot currently be resized during the command. If this is the + * case (or within reach), warn that io_memory_size needs to be + * set larger. + * + * Even if bcache out of space did not cause a failure during scan, it + * may cause a failure during the next vg_read phase or during vg_write. + * + * If there was an error during scan, we could recreate bcache here + * with a larger size and then restart label_scan. But, this does not + * address the problem of writing new metadata that excedes the bcache + * size and failing, which would often be hit first, i.e. we'll fail + * to write new metadata exceding the max size before we have a chance + * to read any metadata with that size, unless we find an existing vg + * that has been previously created with the larger size. + * + * If the largest metadata is within 1MB of the bcache size, then start + * warning. + */ + max_metadata_size_bytes = lvmcache_max_metadata_size(); + + if (max_metadata_size_bytes + (1024 * 1024) > _current_bcache_size_bytes) { + /* we want bcache to be 1MB larger than the max metadata seen */ + uint64_t want_size_kb = (max_metadata_size_bytes / 1024) + 1024; + uint64_t remainder; + if ((remainder = (want_size_kb % 1024))) + want_size_kb = want_size_kb + 1024 - remainder; + + log_warn("WARNING: metadata may not be usable with current io_memory_size %d KiB", + io_memory_size()); + log_warn("WARNING: increase lvm.conf io_memory_size to at least %llu KiB", + (unsigned long long)want_size_kb); + } + dm_list_iterate_items_safe(devl, devl2, &all_devs) { dm_list_del(&devl->list); dm_free(devl); -- 2.24.0 ++++++ bug-1164718_12-io-increase-the-default-io-memory-from-4-to-8-MiB.patch ++++++ From 590a1ebcf78b8aae2a1e5ebaba1ac24a54435690 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 4 Mar 2019 11:18:34 -0600 Subject: [PATCH] io: increase the default io memory from 4 to 8 MiB This is the default bcache size that is created at the start of the command. It needs to be large enough to hold a single copy of metadata for a given VG, or the VG cannot be read or written (since the entire VG would not fit into available memory.) Increasing the default reduces the chances of anyone needing to increase the default to use their VG. The size can be set in lvm.conf global/io_memory_size; the lower limit is 4 MiB and the upper limit is 128 MiB. --- lib/config/defaults.h | 2 +- lib/label/label.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/config/defaults.h b/lib/config/defaults.h index 3b4438a0c..9e398d759 100644 --- a/lib/config/defaults.h +++ b/lib/config/defaults.h @@ -269,6 +269,6 @@ #define DEFAULT_SCAN_LVS 0 -#define DEFAULT_IO_MEMORY_SIZE_KB 4096 +#define DEFAULT_IO_MEMORY_SIZE_KB 8192 #endif /* _LVM_DEFAULTS_H */ diff --git a/lib/label/label.c b/lib/label/label.c index e96a7fe36..4f8e1354f 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -791,7 +791,7 @@ out: * the metadata size.) */ -#define MIN_BCACHE_BLOCKS 32 /* 4MB, currently matches DEFAULT_IO_MEMORY_SIZE_KB */ +#define MIN_BCACHE_BLOCKS 32 /* 4MB */ #define MAX_BCACHE_BLOCKS 1024 static int _setup_bcache(int num_devs) -- 2.24.0 ++++++ bug-1164718_13-bcache-Fix-memory-leak.patch ++++++ From b79f1e176f013167ca9798efb55eaf048d64e042 Mon Sep 17 00:00:00 2001 From: Marian Csontos <mcsontos(a)redhat.com> Date: Thu, 4 Apr 2019 10:18:07 +0200 Subject: [PATCH] bcache: Fix memory leak --- lib/device/bcache.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 7384a3290..f64931fd7 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -411,6 +411,7 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)offset, (unsigned long long)nbytes, (unsigned long long)_last_byte_offset); + free(io); return false; } -- 2.24.0 ++++++ bug-1164718_14-lvmcache-remove-unused_duplicate_devs-list-from-cmd.patch ++++++ From c31e6b0acad339ec3cdc0c8d7e0e01f8d816ed10 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Thu, 6 Jun 2019 14:12:19 -0500 Subject: [PATCH] lvmcache: remove unused_duplicate_devs list from cmd Save the previous duplicate PVs in a global list instead of a list on the cmd struct. dmeventd reuses the cmd struct for multiple commands, and the list entries between commands were being freed (apparently), causing a segfault in dmeventd when it tried to use items in cmd->unused_duplicate_devs that had been saved there by the previous command. --- lib/cache/lvmcache.c | 16 +++++++++------- lib/commands/toolcontext.c | 2 -- lib/commands/toolcontext.h | 1 - 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c index ad40d4c..0ce5df0 100644 --- a/lib/cache/lvmcache.c +++ b/lib/cache/lvmcache.c @@ -100,6 +100,7 @@ static struct dm_hash_table *_saved_vg_hash = NULL; static DM_LIST_INIT(_vginfos); static DM_LIST_INIT(_found_duplicate_devs); static DM_LIST_INIT(_unused_duplicate_devs); +static DM_LIST_INIT(_prev_unused_duplicate_devs); static int _scanning_in_progress = 0; static int _has_scanned = 0; static int _vgs_locked = 0; @@ -118,6 +119,7 @@ int lvmcache_init(struct cmd_context *cmd) dm_list_init(&_vginfos); dm_list_init(&_found_duplicate_devs); dm_list_init(&_unused_duplicate_devs); + dm_list_init(&_prev_unused_duplicate_devs); if (!(_vgname_hash = dm_hash_create(128))) return 0; @@ -1152,14 +1154,14 @@ next: if (!prev_unchosen1 && !prev_unchosen2) { /* - * The cmd list saves the unchosen preference across + * The prev list saves the unchosen preference across * lvmcache_destroy. Sometimes a single command will * fill lvmcache, destroy it, and refill it, and we * want the same duplicate preference to be preserved * in each instance of lvmcache for a single command. */ - prev_unchosen1 = _dev_in_device_list(dev1, &cmd->unused_duplicate_devs); - prev_unchosen2 = _dev_in_device_list(dev2, &cmd->unused_duplicate_devs); + prev_unchosen1 = _dev_in_device_list(dev1, &_prev_unused_duplicate_devs); + prev_unchosen2 = _dev_in_device_list(dev2, &_prev_unused_duplicate_devs); } dev1_major = MAJOR(dev1->dev); @@ -2575,8 +2577,8 @@ void lvmcache_destroy(struct cmd_context *cmd, int retain_orphans, int reset) dm_list_init(&_vginfos); /* - * Copy the current _unused_duplicate_devs into a cmd list before - * destroying _unused_duplicate_devs. + * Move the current _unused_duplicate_devs to _prev_unused_duplicate_devs + * before destroying _unused_duplicate_devs. * * One command can init/populate/destroy lvmcache multiple times. Each * time it will encounter duplicates and choose the preferrred devs. @@ -2584,8 +2586,8 @@ void lvmcache_destroy(struct cmd_context *cmd, int retain_orphans, int reset) * the unpreferred devs here so that _choose_preferred_devs can use * this to make the same choice each time. */ - dm_list_init(&cmd->unused_duplicate_devs); - lvmcache_get_unused_duplicate_devs(cmd, &cmd->unused_duplicate_devs); + _destroy_duplicate_device_list(&_prev_unused_duplicate_devs); + dm_list_splice(&_prev_unused_duplicate_devs, &_unused_duplicate_devs); _destroy_duplicate_device_list(&_unused_duplicate_devs); _destroy_duplicate_device_list(&_found_duplicate_devs); /* should be empty anyway */ _found_duplicate_pvs = 0; diff --git a/lib/commands/toolcontext.c b/lib/commands/toolcontext.c index 25e8b87..38e382f 100644 --- a/lib/commands/toolcontext.c +++ b/lib/commands/toolcontext.c @@ -1983,8 +1983,6 @@ struct cmd_context *create_toolcontext(unsigned is_clvmd, if (!init_lvmcache_orphans(cmd)) goto_out; - dm_list_init(&cmd->unused_duplicate_devs); - if (!_init_segtypes(cmd)) goto_out; diff --git a/lib/commands/toolcontext.h b/lib/commands/toolcontext.h index 485dca9..6e262e8 100644 --- a/lib/commands/toolcontext.h +++ b/lib/commands/toolcontext.h @@ -231,7 +231,6 @@ struct cmd_context { const char *report_list_item_separator; const char *time_format; unsigned rand_seed; - struct dm_list unused_duplicate_devs; /* save preferences between lvmcache instances */ }; /* -- 1.8.3.1 ++++++ bug-1164718_15-cov-release-iterator-on-error-path.patch ++++++ From 09aafb61e31c3781530795057c87afaeb53a2bd4 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 15:37:01 +0200 Subject: [PATCH] cov: release iterator on error path --- lib/label/label.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/label/label.c b/lib/label/label.c index 4f8e1354f..9a3f8df67 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -866,8 +866,11 @@ int label_scan(struct cmd_context *cmd) } while ((dev = dev_iter_get(iter))) { - if (!(devl = dm_zalloc(sizeof(*devl)))) + if (!(devl = dm_zalloc(sizeof(*devl)))) { + log_error("Failed to allocated device list."); + dev_iter_destroy(iter); return 0; + } devl->dev = dev; dm_list_add(&all_devs, &devl->list); -- 2.24.0 ++++++ bug-1164718_16-cov-check-lv_info.patch ++++++ From 2cd6cd34394cb25d312c73b979f4e536d431c10b Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 14:52:19 +0200 Subject: [PATCH] cov: check lv_info Use lv_info results only when valid. --- lib/label/label.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lib/label/label.c b/lib/label/label.c index 6d8156e91..83b4d995c 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -1139,10 +1139,11 @@ void label_scan_invalidate_lv(struct cmd_context *cmd, struct logical_volume *lv struct device *dev; dev_t devt; - lv_info(cmd, lv, 0, &lvinfo, 0, 0); - devt = MKDEV(lvinfo.major, lvinfo.minor); - if ((dev = dev_cache_get_by_devt(devt, NULL))) - label_scan_invalidate(dev); + if (lv_info(cmd, lv, 0, &lvinfo, 0, 0)) { + devt = MKDEV(lvinfo.major, lvinfo.minor); + if ((dev = dev_cache_get_by_devt(devt, NULL))) + label_scan_invalidate(dev); + } } /* -- 2.24.0 ++++++ bug-1164718_17-cov-add-stack-tracing-for-error-paths.patch ++++++ From 66665f5e428c2b3d7f4dbcd3e88a872db0381b93 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 14:42:16 +0200 Subject: [PATCH] cov: add stack tracing for error paths Add missing stack reports on error paths. --- lib/cache/lvmcache.c | 3 ++- lib/device/bcache.c | 3 ++- lib/device/dev-cache.c | 3 ++- lib/device/dev-io.c | 6 ++++-- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c index 0ce5df0..9890325 100644 --- a/lib/cache/lvmcache.c +++ b/lib/cache/lvmcache.c @@ -2595,7 +2595,8 @@ void lvmcache_destroy(struct cmd_context *cmd, int retain_orphans, int reset) if (retain_orphans) { struct format_type *fmt; - lvmcache_init(cmd); + if (!lvmcache_init(cmd)) + stack; dm_list_iterate_items(fmt, &cmd->formats) { if (!lvmcache_add_orphan_vginfo(fmt->orphan_vg_name, fmt)) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index f64931f..dac8be6 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -1101,7 +1101,8 @@ void bcache_destroy(struct bcache *cache) if (cache->nr_locked) log_warn("some blocks are still locked"); - bcache_flush(cache); + if (!bcache_flush(cache)) + log_warn("cache flushing failed."); _wait_all(cache); _exit_free_list(cache); _hash_table_exit(cache); diff --git a/lib/device/dev-cache.c b/lib/device/dev-cache.c index 04ee41e..a429b10 100644 --- a/lib/device/dev-cache.c +++ b/lib/device/dev-cache.c @@ -1344,7 +1344,8 @@ static int _check_for_open_devices(int close_immediate) dev_name(dev), dev->open_count); num_open++; if (close_immediate) - dev_close_immediate(dev); + if (!dev_close_immediate(dev)) + stack; } } diff --git a/lib/device/dev-io.c b/lib/device/dev-io.c index 6996a44..4590232 100644 --- a/lib/device/dev-io.c +++ b/lib/device/dev-io.c @@ -521,7 +521,8 @@ int dev_open_flags(struct device *dev, int flags, int direct, int quiet) /* dev_close_immediate will decrement this */ dev->open_count++; - dev_close_immediate(dev); + if (!dev_close_immediate(dev)) + stack; // FIXME: dev with DEV_ALLOCED is released // but code is referencing it } @@ -599,7 +600,8 @@ int dev_open_flags(struct device *dev, int flags, int direct, int quiet) if (!(dev->flags & DEV_REGULAR) && ((fstat(dev->fd, &buf) < 0) || (buf.st_rdev != dev->dev))) { log_error("%s: fstat failed: Has device name changed?", name); - dev_close_immediate(dev); + if (!dev_close_immediate(dev)) + stack; return 0; } -- 1.8.3.1 ++++++ bug-1164718_18-cov-validate-pagesize-is-not-negative.patch ++++++ From 7232458b6c114cc83b471f08abc49fd1673831c4 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 15:21:46 +0200 Subject: [PATCH] cov: validate pagesize is not negative As _init_free_list() cannot accept negative numbers --- lib/device/bcache.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index dac8be6..ccf0046 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -1040,6 +1040,11 @@ struct bcache *bcache_create(sector_t block_sectors, unsigned nr_cache_blocks, unsigned max_io = engine->max_io(engine); long pgsize = sysconf(_SC_PAGESIZE); + if ((pgsize = sysconf(_SC_PAGESIZE)) < 0) { + log_warn("bcache cannot read pagesize."); + return NULL; + } + if (!nr_cache_blocks) { log_warn("bcache must have at least one cache block"); return NULL; -- 1.8.3.1 ++++++ bug-1164718_19-cov-remove-unused-headers.patch ++++++ From b62c0787debeb3a66a2e3658d1edb8a61a6f247b Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 15:19:07 +0200 Subject: [PATCH] cov: remove unused headers --- lib/device/bcache.c | 1 - lib/device/dev-io.c | 1 - lib/device/dev-type.c | 1 - lib/format_text/archiver.c | 1 - lib/format_text/format-text.c | 1 - lib/label/label.c | 1 - lib/metadata/vg.c | 1 - lib/misc/lvm-file.c | 1 - tools/lvmcmdline.c | 1 - 9 files changed, 9 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index ccf0046..261164e 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -29,7 +29,6 @@ #include <libaio.h> #include <unistd.h> #include <linux/fs.h> -#include <sys/ioctl.h> #include <sys/user.h> #define SECTOR_SHIFT 9L diff --git a/lib/device/dev-io.c b/lib/device/dev-io.c index 4590232..99ff803 100644 --- a/lib/device/dev-io.c +++ b/lib/device/dev-io.c @@ -17,7 +17,6 @@ #include "device.h" #include "metadata.h" #include "memlock.h" -#include "locking.h" #include <limits.h> #include <sys/stat.h> diff --git a/lib/device/dev-type.c b/lib/device/dev-type.c index ce4a05b..14a28dc 100644 --- a/lib/device/dev-type.c +++ b/lib/device/dev-type.c @@ -17,7 +17,6 @@ #include "xlate.h" #include "config.h" #include "metadata.h" -#include "bcache.h" #include "label.h" #include <libgen.h> diff --git a/lib/format_text/archiver.c b/lib/format_text/archiver.c index c8aeb47..467968a 100644 --- a/lib/format_text/archiver.c +++ b/lib/format_text/archiver.c @@ -18,7 +18,6 @@ #include "format-text.h" #include "lvm-string.h" #include "lvmcache.h" -#include "lvmetad.h" #include "memlock.h" #include "toolcontext.h" #include "locking.h" diff --git a/lib/format_text/format-text.c b/lib/format_text/format-text.c index a09c349..f39051c 100644 --- a/lib/format_text/format-text.c +++ b/lib/format_text/format-text.c @@ -29,7 +29,6 @@ #include "label.h" #include "lvmcache.h" #include "lvmetad.h" -#include "memlock.h" #include <unistd.h> #include <sys/param.h> diff --git a/lib/label/label.c b/lib/label/label.c index 83b4d99..d6212b1 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -26,7 +26,6 @@ #include <sys/stat.h> #include <fcntl.h> #include <unistd.h> -#include <sys/time.h> int use_full_md_check; diff --git a/lib/metadata/vg.c b/lib/metadata/vg.c index b8b1501..42801b9 100644 --- a/lib/metadata/vg.c +++ b/lib/metadata/vg.c @@ -18,7 +18,6 @@ #include "display.h" #include "activate.h" #include "toolcontext.h" -#include "lvmcache.h" #include "archiver.h" #include "lvmetad.h" diff --git a/lib/misc/lvm-file.c b/lib/misc/lvm-file.c index 36caaa7..7b789e5 100644 --- a/lib/misc/lvm-file.c +++ b/lib/misc/lvm-file.c @@ -14,7 +14,6 @@ */ #include "lib.h" -#include "lvm-file.h" #include <unistd.h> #include <sys/stat.h> diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c index a9c3e41..0840c65 100644 --- a/tools/lvmcmdline.c +++ b/tools/lvmcmdline.c @@ -19,7 +19,6 @@ #include "label.h" #include "lvm-version.h" #include "lvmlockd.h" -#include "lvmetad-client.h" #include "stub.h" #include "last-path-component.h" -- 1.8.3.1 ++++++ bug-1164718_20-cov-check-result-of-dev_get_block_size.patch ++++++ From 82e7426028a90fff891e0b3fafa15826758e18e8 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 14:56:07 +0200 Subject: [PATCH] cov: check result of dev_get_block_size --- lib/label/label.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/label/label.c b/lib/label/label.c index 9a3f8df..6d8156e 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -1498,7 +1498,10 @@ void dev_set_last_byte(struct device *dev, uint64_t offset) unsigned int phys_block_size = 0; unsigned int block_size = 0; - dev_get_block_size(dev, &phys_block_size, &block_size); + if (!dev_get_block_size(dev, &phys_block_size, &block_size)) { + stack; + return; /* FIXME: error path ? */ + } bcache_set_last_byte(scan_bcache, dev->bcache_fd, offset, phys_block_size); } -- 1.8.3.1 ++++++ bug-1164718_21-gcc-clean-uninitialized-var-warning.patch ++++++ From b0e1019add988b91fb41355566a34a3b2e182569 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 22:49:24 +0200 Subject: [PATCH] gcc: clean uninitialized var warning MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some older gcc versions shows this (FP) warning: label/label.c:360: warning: â€sectorâ€™ may be used uninitialized in this function --- lib/label/label.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/label/label.c b/lib/label/label.c index d6212b15b..a535bd4d3 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -357,7 +357,7 @@ static int _process_block(struct cmd_context *cmd, struct dev_filter *f, char label_buf[LABEL_SIZE] __attribute__((aligned(8))); struct label *label = NULL; struct labeller *labeller; - uint64_t sector; + uint64_t sector = 0; int ret = 0; int pass; -- 2.24.0 ++++++ bug-1164718_22-cov-release-iterator-on-error-path.patch ++++++ From 23478d9d2161d1233d0dde1ca40c81d47292bd87 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Fri, 28 Jun 2019 13:26:11 +0200 Subject: [PATCH] cov: release iterator on error path Another missed release on error path. --- lib/label/label.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/label/label.c b/lib/label/label.c index a535bd4d3..bc31a7c67 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -912,7 +912,7 @@ int label_scan(struct cmd_context *cmd) if (!scan_bcache) { if (!_setup_bcache(dm_list_size(&all_devs))) - return 0; + return_0; } _scan_list(cmd, cmd->full_filter, &all_devs, NULL); @@ -987,8 +987,11 @@ int label_scan_pvscan_all(struct cmd_context *cmd, struct dm_list *scan_devs) } while ((dev = dev_iter_get(iter))) { - if (!(devl = dm_zalloc(sizeof(*devl)))) + if (!(devl = dm_zalloc(sizeof(*devl)))) { + log_error("Failed to allocated device list."); + dev_iter_destroy(iter); return 0; + } devl->dev = dev; dm_list_add(&all_devs, &devl->list); @@ -1013,7 +1016,7 @@ int label_scan_pvscan_all(struct cmd_context *cmd, struct dm_list *scan_devs) if (!scan_bcache) { if (!_setup_bcache(dm_list_size(&all_devs))) - return 0; + return_0; } _scan_list(cmd, cmd->lvmetad_filter, &all_devs, NULL); -- 2.24.0 ++++++ bug-1164718_23-Fix-rounding-writes-up-to-sector-size.patch ++++++ From 7550665ba49ac7d497d5b212e14b69298ef01361 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 24 Jul 2019 11:32:13 -0500 Subject: [PATCH] Fix rounding writes up to sector size Do this at two levels, although one would be enough to fix the problem seen recently: - Ignore any reported sector size other than 512 of 4096. If either sector size (physical or logical) is reported as 512, then use 512. If neither are reported as 512, and one or the other is reported as 4096, then use 4096. If neither is reported as either 512 or 4096, then use 512. - When rounding up a limited write in bcache to be a multiple of the sector size, check that the resulting write size is not larger than the bcache block itself. (This shouldn't happen if the sector size is 512 or 4096.) --- lib/device/bcache.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++-- lib/device/dev-io.c | 52 +++++++++++++++++++++++++++++++ lib/device/device.h | 8 +++-- lib/label/label.c | 27 +++++++++++++--- 4 files changed, 168 insertions(+), 8 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 261164e..a250bfd 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -169,6 +169,7 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, sector_t offset; sector_t nbytes; sector_t limit_nbytes; + sector_t orig_nbytes; sector_t extra_nbytes = 0; if (((uintptr_t) data) & e->page_mask) { @@ -191,11 +192,41 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, return false; } + /* + * If the bcache block offset+len goes beyond where lvm is + * intending to write, then reduce the len being written + * (which is the bcache block size) so we don't write past + * the limit set by lvm. If after applying the limit, the + * resulting size is not a multiple of the sector size (512 + * or 4096) then extend the reduced size to be a multiple of + * the sector size (we don't want to write partial sectors.) + */ if (offset + nbytes > _last_byte_offset) { limit_nbytes = _last_byte_offset - offset; - if (limit_nbytes % _last_byte_sector_size) + + if (limit_nbytes % _last_byte_sector_size) { extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + /* + * adding extra_nbytes to the reduced nbytes (limit_nbytes) + * should make the final write size a multiple of the + * sector size. This should never result in a final size + * larger than the bcache block size (as long as the bcache + * block size is a multiple of the sector size). + */ + if (limit_nbytes + extra_nbytes > nbytes) { + log_warn("Skip extending write at %llu len %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + extra_nbytes = 0; + } + } + + orig_nbytes = nbytes; + if (extra_nbytes) { log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", (unsigned long long)offset, @@ -210,6 +241,22 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes); nbytes = limit_nbytes; } + + /* + * This shouldn't happen, the reduced+extended + * nbytes value should never be larger than the + * bcache block size. + */ + if (nbytes > orig_nbytes) { + log_error("Invalid adjusted write at %llu len %llu adjusted %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)orig_nbytes, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + return false; + } } } @@ -404,6 +451,7 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, uint64_t nbytes = len; sector_t limit_nbytes = 0; sector_t extra_nbytes = 0; + sector_t orig_nbytes = 0; if (offset > _last_byte_offset) { log_error("Limit write at %llu len %llu beyond last byte %llu", @@ -416,9 +464,30 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, if (offset + nbytes > _last_byte_offset) { limit_nbytes = _last_byte_offset - offset; - if (limit_nbytes % _last_byte_sector_size) + + if (limit_nbytes % _last_byte_sector_size) { extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + /* + * adding extra_nbytes to the reduced nbytes (limit_nbytes) + * should make the final write size a multiple of the + * sector size. This should never result in a final size + * larger than the bcache block size (as long as the bcache + * block size is a multiple of the sector size). + */ + if (limit_nbytes + extra_nbytes > nbytes) { + log_warn("Skip extending write at %llu len %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + extra_nbytes = 0; + } + } + + orig_nbytes = nbytes; + if (extra_nbytes) { log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", (unsigned long long)offset, @@ -433,6 +502,22 @@ static bool _sync_issue(struct io_engine *ioe, enum dir d, int fd, (unsigned long long)limit_nbytes); nbytes = limit_nbytes; } + + /* + * This shouldn't happen, the reduced+extended + * nbytes value should never be larger than the + * bcache block size. + */ + if (nbytes > orig_nbytes) { + log_error("Invalid adjusted write at %llu len %llu adjusted %llu limit %llu extra %llu sector_size %llu", + (unsigned long long)offset, + (unsigned long long)orig_nbytes, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)extra_nbytes, + (unsigned long long)_last_byte_sector_size); + return false; + } } where = offset; diff --git a/lib/device/dev-io.c b/lib/device/dev-io.c index 99ff803..fd8d349 100644 --- a/lib/device/dev-io.c +++ b/lib/device/dev-io.c @@ -135,6 +135,58 @@ static int _io(struct device_area *where, char *buffer, int should_write, dev_io return (total == (size_t) where->size); } +int dev_get_direct_block_sizes(struct device *dev, unsigned int *physical_block_size, + unsigned int *logical_block_size) +{ + int fd = dev->bcache_fd; + int do_close = 0; + unsigned int pbs = 0; + unsigned int lbs = 0; + + if (dev->physical_block_size || dev->logical_block_size) { + *physical_block_size = dev->physical_block_size; + *logical_block_size = dev->logical_block_size; + return 1; + } + + if (fd <= 0) { + if (!dev_open_readonly(dev)) + return 0; + fd = dev_fd(dev); + do_close = 1; + } + + /* + * BLKPBSZGET from kernel comment for blk_queue_physical_block_size: + * "the lowest possible sector size that the hardware can operate on + * without reverting to read-modify-write operations" + */ + if (ioctl(fd, BLKPBSZGET, &pbs)) { + stack; + pbs = 0; + } + + /* + * BLKSSZGET from kernel comment for blk_queue_logical_block_size: + * "the lowest possible block size that the storage device can address." + */ + if (ioctl(fd, BLKSSZGET, &lbs)) { + stack; + lbs = 0; + } + + dev->physical_block_size = pbs; + dev->logical_block_size = lbs; + + *physical_block_size = pbs; + *logical_block_size = lbs; + + if (do_close && !dev_close_immediate(dev)) + stack; + + return 1; +} + /*----------------------------------------------------------------- * LVM2 uses O_DIRECT when performing metadata io, which requires * block size aligned accesses. If any io is not aligned we have diff --git a/lib/device/device.h b/lib/device/device.h index bbd965a..aafc401 100644 --- a/lib/device/device.h +++ b/lib/device/device.h @@ -67,8 +67,10 @@ struct device { int open_count; int error_count; int max_error_count; - int phys_block_size; - int block_size; + int phys_block_size; /* From either BLKPBSZGET or BLKSSZGET, don't use */ + int block_size; /* From BLKBSZGET, returns bdev->bd_block_size, likely set by fs, probably don't use */ + int physical_block_size; /* From BLKPBSZGET: lowest possible sector size that the hardware can operate on without reverting to read-modify-write operations */ + int logical_block_size; /* From BLKSSZGET: lowest possible block size that the storage device can address */ int read_ahead; int bcache_fd; uint32_t flags; @@ -132,6 +134,8 @@ void dev_size_seqno_inc(void); * All io should use these routines. */ int dev_get_block_size(struct device *dev, unsigned int *phys_block_size, unsigned int *block_size); +int dev_get_direct_block_sizes(struct device *dev, unsigned int *physical_block_size, + unsigned int *logical_block_size); int dev_get_size(struct device *dev, uint64_t *size); int dev_get_read_ahead(struct device *dev, uint32_t *read_ahead); int dev_discard_blocks(struct device *dev, uint64_t offset_bytes, uint64_t size_bytes); diff --git a/lib/label/label.c b/lib/label/label.c index bc31a7c..8107e33 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -1498,15 +1498,34 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) void dev_set_last_byte(struct device *dev, uint64_t offset) { - unsigned int phys_block_size = 0; - unsigned int block_size = 0; + unsigned int physical_block_size = 0; + unsigned int logical_block_size = 0; + unsigned int bs; - if (!dev_get_block_size(dev, &phys_block_size, &block_size)) { + if (!dev_get_direct_block_sizes(dev, &physical_block_size, &logical_block_size)) { stack; return; /* FIXME: error path ? */ } - bcache_set_last_byte(scan_bcache, dev->bcache_fd, offset, phys_block_size); + if ((physical_block_size == 512) && (logical_block_size == 512)) + bs = 512; + else if ((physical_block_size == 4096) && (logical_block_size == 4096)) + bs = 4096; + else if ((physical_block_size == 512) || (logical_block_size == 512)) { + log_debug("Set last byte mixed block sizes physical %u logical %u using 512", + physical_block_size, logical_block_size); + bs = 512; + } else if ((physical_block_size == 4096) || (logical_block_size == 4096)) { + log_debug("Set last byte mixed block sizes physical %u logical %u using 4096", + physical_block_size, logical_block_size); + bs = 4096; + } else { + log_debug("Set last byte mixed block sizes physical %u logical %u using 512", + physical_block_size, logical_block_size); + bs = 512; + } + + bcache_set_last_byte(scan_bcache, dev->bcache_fd, offset, bs); } void dev_unset_last_byte(struct device *dev) -- 1.8.3.1 ++++++ bug-1164718_24-pvscan-avoid-redundant-activation.patch ++++++ From 8bcd482cc5fc58c9d5ff5305f1f1bff5d80eaa92 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 19 Aug 2019 14:22:01 -0500 Subject: [PATCH] pvscan: avoid redundant activation Use temp files in /run/lvm/vgs_online/ to keep track of when a VG has been autoactivated by pvscan. When pvscan autoactivates a VG, it creates a temp file with the VG's name. Before a subsequent pvscan tries to autoactivate the same VG, it checks if a temp file exists for the VG name, and if so it skips it. This can commonly happen when many devices appear on the system at once, which generates several concurrent pvscans. In this case the first pvscan does initialization by scanning all devices and activating any complete VGs. The other pvscans would attempt to activate the same complete VGs again. This extra work could create a bottleneck of pvscan commands. If a VG is deactivated by vgchange, the vg online file is removed. If PVs are then disconnected/reconnected, pvscan will again autoactivate the VG. Also, this patch disables the VG refresh that could be called from pvscan --cache -aay if lvmetad detects metadata inconsistencies. The role of pvscan should be limited to basic autoactivation, and any refresh scenarios are special cases that are not appropriate for automation. The warning printed by commands retrying an lvmetad connection has been reduced to once every 10 seconds. New output messages have been added to pvscan to record when pvscan is falling back to direct activation of all VGs. --- lib/cache/lvmetad.c | 11 ++++-- tools/pvscan.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++--- tools/vgchange.c | 13 +++++++ 3 files changed, 123 insertions(+), 9 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index 86a880a..d7e798d 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -248,7 +248,7 @@ int lvmetad_token_matches(struct cmd_context *cmd) const char *daemon_token; unsigned int delay_usec = 0; unsigned int wait_sec = 0; - uint64_t now = 0, wait_start = 0; + uint64_t now = 0, wait_start = 0, last_warn = 0; int ret = 1; wait_sec = (unsigned int)_lvmetad_update_timeout; @@ -294,12 +294,15 @@ retry: wait_start = now; if (now - wait_start > wait_sec) { - log_warn("WARNING: Not using lvmetad after %u sec lvmetad_update_wait_time.", wait_sec); + log_warn("pvscan[%d] WARNING: Not using lvmetad after %u sec lvmetad_update_wait_time.", getpid(), wait_sec); goto fail; } - log_warn("WARNING: lvmetad is being updated, retrying (setup) for %u more seconds.", - wait_sec - (unsigned int)(now - wait_start)); + if (now - last_warn >= 10) { + last_warn = now; + log_warn("pvscan[%d] WARNING: lvmetad is being updated, retrying (setup) for %u more seconds.", + getpid(), wait_sec - (unsigned int)(now - wait_start)); + } /* Delay a random period between 1 and 2 seconds. */ delay_usec = 1000000 + lvm_even_rand(&_lvmetad_cmd->rand_seed, 1000000); diff --git a/tools/pvscan.c b/tools/pvscan.c index daac88f..c21845c 100644 --- a/tools/pvscan.c +++ b/tools/pvscan.c @@ -18,8 +18,13 @@ #include "lvmetad.h" #include "lvmcache.h" +#include <dirent.h> +#include <sys/file.h> + extern int use_full_md_check; +void online_vg_file_remove(const char *vgname); + struct pvscan_params { int new_pvs_found; int pvs_found; @@ -142,6 +147,82 @@ static int _lvmetad_clear_dev(dev_t devno, int32_t major, int32_t minor) return 1; } +static const char *_vgs_online_dir = DEFAULT_RUN_DIR "/vgs_online"; + +static void _online_dir_setup(void) +{ + struct stat st; + int rv; + + if (!stat(DEFAULT_RUN_DIR, &st)) + goto do_vgs; + + log_debug("Creating run_dir."); + dm_prepare_selinux_context(DEFAULT_RUN_DIR, S_IFDIR); + rv = mkdir(DEFAULT_RUN_DIR, 0755); + dm_prepare_selinux_context(NULL, 0); + + if ((rv < 0) && stat(DEFAULT_RUN_DIR, &st)) + log_error("Failed to create %s %d", DEFAULT_RUN_DIR, errno); + +do_vgs: + if (!stat(_vgs_online_dir, &st)) + return; + + log_debug("Creating vgs_online_dir."); + dm_prepare_selinux_context(_vgs_online_dir, S_IFDIR); + rv = mkdir(_vgs_online_dir, 0755); + dm_prepare_selinux_context(NULL, 0); + + if ((rv < 0) && stat(_vgs_online_dir, &st)) + log_error("Failed to create %s %d", _vgs_online_dir, errno); +} + +static int _online_vg_file_create(struct cmd_context *cmd, const char *vgname) +{ + char path[PATH_MAX]; + int fd; + + if (dm_snprintf(path, sizeof(path), "%s/%s", _vgs_online_dir, vgname) < 0) { + log_error("Path %s/%s is too long.", _vgs_online_dir, vgname); + return 0; + } + + log_debug("Create vg online: %s", path); + + fd = open(path, O_CREAT | O_EXCL | O_TRUNC | O_RDWR, S_IRUSR | S_IWUSR); + if (fd < 0) { + log_debug("Failed to create %s: %d", path, errno); + return 0; + } + + /* We don't care about syncing, these files are not even persistent. */ + + if (close(fd)) + log_sys_debug("close", path); + + return 1; +} + +void online_vg_file_remove(const char *vgname) +{ + char path[PATH_MAX]; + + if (dm_snprintf(path, sizeof(path), "%s/%s", _vgs_online_dir, vgname) < 0) { + log_error("Path %s/%s is too long.", _vgs_online_dir, vgname); + return; + } + + log_debug("Unlink vg online: %s", path); + + /* + * There will only be an online file for the vg if it was + * autoactivated, so in many cases there will not be one. + */ + + unlink(path); +} + /* * pvscan --cache does not perform any lvmlockd locking, and * pvscan --cache -aay skips autoactivation in lockd VGs. @@ -190,8 +271,6 @@ static int _pvscan_autoactivate_single(struct cmd_context *cmd, const char *vg_n struct volume_group *vg, struct processing_handle *handle) { struct pvscan_aa_params *pp = (struct pvscan_aa_params *)handle->custom_handle; - unsigned int refresh_retries = REFRESH_BEFORE_AUTOACTIVATION_RETRIES; - int refresh_done = 0; if (vg_is_clustered(vg)) return ECMD_PROCESSED; @@ -204,6 +283,10 @@ static int _pvscan_autoactivate_single(struct cmd_context *cmd, const char *vg_n log_debug("pvscan autoactivating VG %s.", vg_name); +#if 0 + unsigned int refresh_retries = REFRESH_BEFORE_AUTOACTIVATION_RETRIES; + int refresh_done = 0; + /* * Refresh LVs in a VG that has "changed" from finding a PV. * The meaning of "changed" is determined in lvmetad, and is @@ -241,8 +324,14 @@ static int _pvscan_autoactivate_single(struct cmd_context *cmd, const char *vg_n if (!refresh_done) log_warn("%s: refresh before autoactivation failed.", vg->name); } +#endif - log_debug_activation("Autoactivating VG %s.", vg_name); + if (!_online_vg_file_create(cmd, vg->name)) { + log_print("pvscan[%d] VG %s skip autoactivation.", getpid(), vg->name); + return ECMD_PROCESSED; + } + + log_print("pvscan[%d] VG %s run autoactivation.", getpid(), vg->name); if (!vgchange_activate(cmd, vg, CHANGE_AAY)) { log_error("%s: autoactivation failed.", vg->name); @@ -307,6 +396,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) struct dev_iter *iter; const char *pv_name; const char *reason = NULL; + const char *dev_arg = NULL; int32_t major = -1; int32_t minor = -1; int devno_args = 0; @@ -346,7 +436,12 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) log_error("Both --major and --minor required to identify devices."); return EINVALID_CMD_LINE; } - + + if (!devno_args && argc) + dev_arg = *argv; + + _online_dir_setup(); + if (!lock_vol(cmd, VG_GLOBAL, LCK_VG_READ, NULL)) { log_error("Unable to obtain global lock."); return ECMD_FAILED; @@ -358,7 +453,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) * still activate LVs even though it's not updating the cache. */ if (do_activate && !lvmetad_used()) { - log_verbose("Activating all VGs without lvmetad."); + log_print("pvscan[%d] activating all directly (lvmetad unused) %s", getpid(), dev_arg ?: ""); all_vgs = 1; devno_args = 0; goto activate; @@ -399,6 +494,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) log_warn("WARNING: Not using lvmetad because %s.", reason); lvmetad_make_unused(cmd); } + log_print("pvscan[%d] activating all directly (lvmetad token) %s", getpid(), dev_arg ?: ""); all_vgs = 1; goto activate; } @@ -415,6 +511,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) if (lvmetad_is_disabled(cmd, &reason)) { log_warn("WARNING: Not using lvmetad because %s.", reason); lvmetad_make_unused(cmd); + log_print("pvscan[%d] activating all directly (lvmetad disabled) %s", getpid(), dev_arg ?: ""); all_vgs = 1; goto activate; } @@ -610,6 +707,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) */ if (lvmetad_used() && lvmetad_is_disabled(cmd, &reason)) { log_warn("WARNING: Not using lvmetad because %s.", reason); + log_print("pvscan[%d] activating directly (lvmetad disabled in scan) %s", getpid(), dev_arg ?: ""); lvmetad_make_unused(cmd); } diff --git a/tools/vgchange.c b/tools/vgchange.c index e15fca7..52abc11 100644 --- a/tools/vgchange.c +++ b/tools/vgchange.c @@ -20,6 +20,8 @@ struct vgchange_params { unsigned int lock_start_sanlock : 1; }; +extern void online_vg_file_remove(const char *vgname); + /* * Increments *count by the number of _new_ monitored devices. */ @@ -259,6 +261,17 @@ int vgchange_activate(struct cmd_context *cmd, struct volume_group *vg, r = 0; } + if (!do_activate && r) { + /* + * While the VG is inactive, the PVs may go offline. + * If the PVs then come back online, pvscan --cache -aay + * which should autoactivate the VG again. We need + * to remove the vg online file in order for the pvscan + * to autoactivate the VG again. + */ + online_vg_file_remove(vg->name); + } + /* Print message only if there was not found a missing VG */ log_print_unless_silent("%d logical volume(s) in volume group \"%s\" now active", lvs_in_vg_activated(vg), vg->name); -- 1.8.3.1 ++++++ bug-1164718_25-devs-check-for-no-dev-when-dropping-aliases.patch ++++++ From f50af80199f723f7b1970ee33ddf959ea79fcbef Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 16 Oct 2019 13:32:28 -0500 Subject: [PATCH] devs: check for no dev when dropping aliases When scanning fails to find a device path and looks for device aliases, check if the device itself still exists to avoid a potential segfault. --- lib/label/label.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/label/label.c b/lib/label/label.c index 8107e33f7..e4a106854 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -734,6 +734,11 @@ static int _scan_list(struct cmd_context *cmd, struct dev_filter *f, retried_open = 1; dm_list_iterate_items_safe(devl, devl2, &reopen_devs) { + if (!devl->dev) { + dm_list_del(&devl->list); + continue; + } + _drop_bad_aliases(devl->dev); if (dm_list_empty(&devl->dev->aliases)) { -- 2.24.0 ++++++ bug-1164718_26-pvscan-fix-activation-of-incomplete-VGs.patch ++++++ From 6b12930860a993624d6325aec2e9c561f4412aa9 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Tue, 3 Sep 2019 15:14:08 -0500 Subject: [PATCH] pvscan: fix activation of incomplete VGs For a long time there has been a bug in the activation done by the initial pvscan (which scans all devs to initialize the lvmetad cache.) It was attempting to activate all VGs, even those that were not complete. lvmetad tells pvscan when a VG is complete, and pvscan needs to use this information to decide which VGs to activate. When there are problems that prevent lvmetad from being used (e.g. lvmetad is disabled or not running), pvscan activation cannot use lvmetad to determine when a VG is complete, so it now checks if devices are present for all PVs in the VG before activating. (The recent commit "pvscan: avoid redundant activation" could make this bug more apparent because redundant activations can cover up the effect of activating an incomplete VG and missing some LV activations.) --- lib/cache/lvmetad.c | 15 ++++++++---- lib/cache/lvmetad.h | 2 +- tools/lvmcmdline.c | 2 +- tools/lvscan.c | 2 +- tools/pvscan.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++----- tools/vgcfgrestore.c | 2 +- tools/vgimport.c | 2 +- tools/vgimportclone.c | 2 +- tools/vgscan.c | 2 +- 9 files changed, 77 insertions(+), 17 deletions(-) diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index d7e798d..d242260 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -1702,6 +1702,13 @@ int lvmetad_pv_found(struct cmd_context *cmd, const struct id *pvid, struct devi changed = daemon_reply_int(reply, "changed", 0); } + if (vg && vg->system_id && vg->system_id[0] && + cmd->system_id && cmd->system_id[0] && + strcmp(vg->system_id, cmd->system_id)) { + log_debug_lvmetad("Ignore foreign VG %s on %s", vg->name , dev_name(dev)); + goto out; + } + /* * If lvmetad now sees all PVs in the VG, it returned the * "complete" status string. Add this VG name to the list @@ -1732,7 +1739,7 @@ int lvmetad_pv_found(struct cmd_context *cmd, const struct id *pvid, struct devi log_error("str_list_add failed"); } } - +out: daemon_reply_destroy(reply); return result; @@ -2333,7 +2340,7 @@ bad: * generally revert disk scanning and not use lvmetad. */ -int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) +int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait, struct dm_list *found_vgnames) { struct device_list *devl, *devl2; struct dm_list scan_devs; @@ -2415,7 +2422,7 @@ int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait) dm_list_del(&devl->list); - ret = lvmetad_pvscan_single(cmd, devl->dev, NULL, NULL); + ret = lvmetad_pvscan_single(cmd, devl->dev, found_vgnames, NULL); label_scan_invalidate(devl->dev); @@ -2758,7 +2765,7 @@ void lvmetad_validate_global_cache(struct cmd_context *cmd, int force) * we rescanned for the token, and the time we acquired the global * lock.) */ - if (!lvmetad_pvscan_all_devs(cmd, 1)) { + if (!lvmetad_pvscan_all_devs(cmd, 1, NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); return; diff --git a/lib/cache/lvmetad.h b/lib/cache/lvmetad.h index 73c2645..55ce16a 100644 --- a/lib/cache/lvmetad.h +++ b/lib/cache/lvmetad.h @@ -151,7 +151,7 @@ int lvmetad_pvscan_single(struct cmd_context *cmd, struct device *dev, struct dm_list *found_vgnames, struct dm_list *changed_vgnames); -int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait); +int lvmetad_pvscan_all_devs(struct cmd_context *cmd, int do_wait, struct dm_list *found_vgnames); int lvmetad_vg_clear_outdated_pvs(struct volume_group *vg); void lvmetad_validate_global_cache(struct cmd_context *cmd, int force); diff --git a/tools/lvmcmdline.c b/tools/lvmcmdline.c index 0840c65..6a1ab11 100644 --- a/tools/lvmcmdline.c +++ b/tools/lvmcmdline.c @@ -2980,7 +2980,7 @@ int lvm_run_command(struct cmd_context *cmd, int argc, char **argv) */ if (lvmetad_used() && !_cmd_no_lvmetad_autoscan(cmd)) { if (cmd->include_foreign_vgs || !lvmetad_token_matches(cmd)) { - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, cmd->include_foreign_vgs ? 1 : 0)) { + if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, cmd->include_foreign_vgs ? 1 : 0, NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); } diff --git a/tools/lvscan.c b/tools/lvscan.c index c38208a..34e9f31 100644 --- a/tools/lvscan.c +++ b/tools/lvscan.c @@ -103,7 +103,7 @@ int lvscan(struct cmd_context *cmd, int argc, char **argv) /* Needed because this command has NO_LVMETAD_AUTOSCAN. */ if (lvmetad_used() && (!lvmetad_token_matches(cmd) || lvmetad_is_disabled(cmd, &reason))) { - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0)) { + if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0, NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); } diff --git a/tools/pvscan.c b/tools/pvscan.c index c21845c..2e7a864 100644 --- a/tools/pvscan.c +++ b/tools/pvscan.c @@ -38,6 +38,7 @@ struct pvscan_params { struct pvscan_aa_params { int refresh_all; + int all_vgs; unsigned int activate_errors; struct dm_list changed_vgnames; }; @@ -223,6 +224,28 @@ void online_vg_file_remove(const char *vgname) unlink(path); } +static void _online_files_remove(const char *dirpath) +{ + char path[PATH_MAX]; + DIR *dir; + struct dirent *de; + + if (!(dir = opendir(dirpath))) + return; + + while ((de = readdir(dir))) { + if (de->d_name[0] == '.') + continue; + + memset(path, 0, sizeof(path)); + snprintf(path, sizeof(path), "%s/%s", dirpath, de->d_name); + if (unlink(path)) + log_sys_debug("unlink", path); + } + if (closedir(dir)) + log_sys_debug("closedir", dirpath); +} + /* * pvscan --cache does not perform any lvmlockd locking, and * pvscan --cache -aay skips autoactivation in lockd VGs. @@ -271,6 +294,8 @@ static int _pvscan_autoactivate_single(struct cmd_context *cmd, const char *vg_n struct volume_group *vg, struct processing_handle *handle) { struct pvscan_aa_params *pp = (struct pvscan_aa_params *)handle->custom_handle; + struct pv_list *pvl; + int incomplete = 0; if (vg_is_clustered(vg)) return ECMD_PROCESSED; @@ -281,6 +306,24 @@ static int _pvscan_autoactivate_single(struct cmd_context *cmd, const char *vg_n if (is_lockd_type(vg->lock_type)) return ECMD_PROCESSED; + /* + * This all_vgs case only happens in fallback cases when there's some + * problem preventing the use of lvmetad. When lvmetad can be properly + * used, the found_vgnames list should have the names of complete VGs + * that should be activated. + */ + if (pp->all_vgs) { + dm_list_iterate_items(pvl, &vg->pvs) { + if (!pvl->pv->dev) + incomplete++; + } + + if (incomplete) { + log_print("pvscan[%d] VG %s incomplete (need %d).", getpid(), vg->name, incomplete); + return ECMD_PROCESSED; + } + } + log_debug("pvscan autoactivating VG %s.", vg_name); #if 0 @@ -377,6 +420,7 @@ static int _pvscan_autoactivate(struct cmd_context *cmd, struct pvscan_aa_params if (all_vgs) { cmd->cname->flags |= ALL_VGS_IS_DEFAULT; pp->refresh_all = 1; + pp->all_vgs = 1; } ret = process_each_vg(cmd, 0, NULL, NULL, vgnames, 0, 0, handle, _pvscan_autoactivate_single); @@ -463,17 +507,23 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) * Scan all devices when no args are given. */ if (!argc && !devno_args) { + _online_files_remove(_vgs_online_dir); + log_verbose("Scanning all devices."); - if (!lvmetad_pvscan_all_devs(cmd, 1)) { + if (!lvmetad_pvscan_all_devs(cmd, 1, &found_vgnames)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); + all_vgs = 1; } if (lvmetad_used() && lvmetad_is_disabled(cmd, &reason)) { log_warn("WARNING: Not using lvmetad because %s.", reason); lvmetad_make_unused(cmd); + all_vgs = 1; } - all_vgs = 1; + + if (!all_vgs && do_activate) + log_print("pvscan[%d] activating all complete VGs (no args)", getpid()); goto activate; } @@ -485,7 +535,7 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) * never scan any devices other than those specified. */ if (!lvmetad_token_matches(cmd)) { - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0)) { + if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0, &found_vgnames)) { log_warn("WARNING: Not updating lvmetad because cache update failed."); ret = ECMD_FAILED; goto out; @@ -493,9 +543,12 @@ static int _pvscan_cache(struct cmd_context *cmd, int argc, char **argv) if (lvmetad_used() && lvmetad_is_disabled(cmd, &reason)) { log_warn("WARNING: Not using lvmetad because %s.", reason); lvmetad_make_unused(cmd); + all_vgs = 1; + log_print("pvscan[%d] activating all directly (lvmetad disabled from scan) %s", getpid(), dev_arg ?: ""); } - log_print("pvscan[%d] activating all directly (lvmetad token) %s", getpid(), dev_arg ?: ""); - all_vgs = 1; + + if (!all_vgs) + log_print("pvscan[%d] activating all complete VGs for init", getpid()); goto activate; } @@ -807,7 +860,7 @@ int pvscan(struct cmd_context *cmd, int argc, char **argv) /* Needed because this command has NO_LVMETAD_AUTOSCAN. */ if (lvmetad_used() && (!lvmetad_token_matches(cmd) || lvmetad_is_disabled(cmd, &reason))) { - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0)) { + if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, 0, NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); } diff --git a/tools/vgcfgrestore.c b/tools/vgcfgrestore.c index 48a2fa4..e7f9848 100644 --- a/tools/vgcfgrestore.c +++ b/tools/vgcfgrestore.c @@ -177,7 +177,7 @@ rescan: } if (!refresh_filters(cmd)) stack; - if (!lvmetad_pvscan_all_devs(cmd, 1)) { + if (!lvmetad_pvscan_all_devs(cmd, 1, NULL)) { log_warn("WARNING: Failed to scan devices."); log_warn("WARNING: Update lvmetad with pvscan --cache."); goto out; diff --git a/tools/vgimport.c b/tools/vgimport.c index ea50198..d4455ec 100644 --- a/tools/vgimport.c +++ b/tools/vgimport.c @@ -96,7 +96,7 @@ int vgimport(struct cmd_context *cmd, int argc, char **argv) * import it. */ if (lvmetad_used()) { - if (!lvmetad_pvscan_all_devs(cmd, 1)) { + if (!lvmetad_pvscan_all_devs(cmd, 1, NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); } diff --git a/tools/vgimportclone.c b/tools/vgimportclone.c index c4c5d4c..ac3766b 100644 --- a/tools/vgimportclone.c +++ b/tools/vgimportclone.c @@ -377,7 +377,7 @@ out: if (!refresh_filters(cmd)) stack; - if (!lvmetad_pvscan_all_devs(cmd, 1)) { + if (!lvmetad_pvscan_all_devs(cmd, 1, NULL)) { log_warn("WARNING: Failed to scan devices."); log_warn("WARNING: Update lvmetad with pvscan --cache."); } diff --git a/tools/vgscan.c b/tools/vgscan.c index f9fa382..a1ef264 100644 --- a/tools/vgscan.c +++ b/tools/vgscan.c @@ -101,7 +101,7 @@ int vgscan(struct cmd_context *cmd, int argc, char **argv) log_verbose("Ignoring vgscan --cache command because lvmetad is not in use."); if (lvmetad_used() && (arg_is_set(cmd, cache_long_ARG) || !lvmetad_token_matches(cmd) || lvmetad_is_disabled(cmd, &reason))) { - if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, arg_is_set(cmd, cache_long_ARG))) { + if (lvmetad_used() && !lvmetad_pvscan_all_devs(cmd, arg_is_set(cmd, cache_long_ARG), NULL)) { log_warn("WARNING: Not using lvmetad because cache update failed."); lvmetad_make_unused(cmd); } -- 1.8.3.1 ++++++ bug-1164718_27-lvmetad-fix-sync-cache-to-lvmetad.patch ++++++ From 5d6bf1efb225b964bfff398277e68345acdac1d0 Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Wed, 25 Sep 2019 14:23:14 -0500 Subject: [PATCH] lvmetad: fix sync cache to lvmetad error could be reproduced follow those steps: #!/bin/bash vgcreate vgtest /dev/sdb lvcreate -L 100M -n lv1 vgtest while : do service lvm2-lvmetad restart vgs & pvscan & lvcreate -L 100M -n lv2 vgtest & lvchange /dev/vgtest/lv1 --addtag xxxxx & wait if ! lvs|grep lv2;then echo "err create" break fi sleep 1 lvremove -y /dev/vgtest/lv2 lvchange /dev/vgtest/lv1 --deltag xxxxx done and then fail to create vgtest/lv2, actually lv2 was created, while the metadata written on disk is replaced by lvchange. It could look up lv2 by calling dmsetup table, while lvs could not. This is because, when lvmetad restarted, several lvm commands update token concurrently, when lvcreate recieve "token_mismatch", it cancle communicating with lvmetad, which leads to that lvmetad cache is not sync with the metadata on disk, then lv2 is not committed to lvmetad cache. The metadata of vgtest which lvchange query from lvmetad is out of date. After lvchange, it use the old metadata cover the new one. This patch let lvm process update token synchronously, only one command update lvmetad token at a time. lvmetad_pvscan_single send the metadata on a pv by sending "pv_found" to lvmetad, while the metadata maybe out of date after waiting for the chance to update lvmetad token. Call label_read to read metadata again. Token mismatch may lead to problems, increase log level. Signed-off-by: wangjufeng<wangjufeng(a)huawei.com> --- daemons/lvmetad/lvmetad-core.c | 21 ++++++++++++++++----- lib/cache/lvmetad.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 7 deletions(-) diff --git a/daemons/lvmetad/lvmetad-core.c b/daemons/lvmetad/lvmetad-core.c index 9ae359b..2628022 100644 --- a/daemons/lvmetad/lvmetad-core.c +++ b/daemons/lvmetad/lvmetad-core.c @@ -2669,6 +2669,7 @@ static response handler(daemon_state s, client_handle h, request r) int pid; int cache_lock = 0; int info_lock = 0; + uint64_t timegap = 0; rq = daemon_request_str(r, "request", "NONE"); token = daemon_request_str(r, "token", "NONE"); @@ -2711,12 +2712,22 @@ static response handler(daemon_state s, client_handle h, request r) state->update_cmd); } else if (prev_in_progress && this_in_progress) { + timegap = _monotonic_seconds() - state->update_begin; + if (timegap < state->update_timeout) { + pthread_mutex_unlock(&state->token_lock); + return daemon_reply_simple("token_updating", + "expected = %s", state->token, + "update_pid = " FMTd64, (int64_t)state->update_pid, + "reason = %s", "another command has populated the cache", + NULL); + } + /* Current update is cancelled and replaced by a new update */ - DEBUGLOG(state, "token_update replacing pid %d begin %llu len %d cmd %s", + WARN(state, "token_update replacing pid %d begin %llu len %d cmd %s", state->update_pid, (unsigned long long)state->update_begin, - (int)(_monotonic_seconds() - state->update_begin), + (int)(timegap), state->update_cmd); (void) dm_strncpy(prev_token, state->token, sizeof(prev_token)); @@ -2726,7 +2737,7 @@ static response handler(daemon_state s, client_handle h, request r) state->update_pid = pid; strncpy(state->update_cmd, cmd, CMD_NAME_SIZE - 1); - DEBUGLOG(state, "token_update begin %llu timeout %d pid %d cmd %s", + WARN(state, "token_update begin %llu timeout %d pid %d cmd %s", (unsigned long long)state->update_begin, state->update_timeout, state->update_pid, @@ -2737,7 +2748,7 @@ static response handler(daemon_state s, client_handle h, request r) if (state->update_pid != pid) { /* If a pid doing update was cancelled, ignore its token update at the end. */ - DEBUGLOG(state, "token_update ignored from cancelled update pid %d", pid); + WARN(state, "token_update ignored from cancelled update pid %d", pid); pthread_mutex_unlock(&state->token_lock); return daemon_reply_simple("token_mismatch", @@ -2748,7 +2759,7 @@ static response handler(daemon_state s, client_handle h, request r) NULL); } - DEBUGLOG(state, "token_update end len %d pid %d new token %s", + WARN(state, "token_update end len %d pid %d new token %s", (int)(_monotonic_seconds() - state->update_begin), state->update_pid, token); diff --git a/lib/cache/lvmetad.c b/lib/cache/lvmetad.c index d242260..61ba53e 100644 --- a/lib/cache/lvmetad.c +++ b/lib/cache/lvmetad.c @@ -565,7 +565,12 @@ static int _token_update(int *replaced_update) const char *reply_str; int update_pid; int ending_our_update; + unsigned int wait_sec = 0; + uint64_t now = 0, wait_start = 0; + wait_sec = (unsigned int)_lvmetad_update_timeout; + unsigned int delay_usec = 0; +retry: log_debug_lvmetad("Sending lvmetad token_update %s", _lvmetad_token); reply = _lvmetad_send(NULL, "token_update", NULL); @@ -581,6 +586,28 @@ static int _token_update(int *replaced_update) update_pid = (int)daemon_reply_int(reply, "update_pid", 0); reply_str = daemon_reply_str(reply, "response", ""); + if (!strcmp(reply_str, "token_updating")) { + daemon_reply_destroy(reply); + if (!(now = _monotonic_seconds())) { + log_print_unless_silent("_monotonic_seconds error"); + return 0; + } + + if (!wait_start) + wait_start = now; + + if (now - wait_start <= wait_sec) { + log_warn("lvmetad is being updated, retry for %u more seconds.", + wait_sec - (unsigned int)(now - wait_start)); + delay_usec = 1000000 + lvm_even_rand(&_lvmetad_cmd->rand_seed, 1000000); + usleep(delay_usec); + goto retry; + } + + log_print_unless_silent("Not using lvmetad after %u sec lvmetad_update_wait_time, no more try.", wait_sec); + return 0; + } + /* * A mismatch can only happen when this command attempts to set the * token to filter:<hash> at the end of its update, but the update has @@ -591,11 +618,11 @@ static int _token_update(int *replaced_update) ending_our_update = strcmp(_lvmetad_token, LVMETAD_TOKEN_UPDATE_IN_PROGRESS); - log_debug_lvmetad("Received token update mismatch expected \"%s\" our token \"%s\" update_pid %d our pid %d", + log_print_unless_silent("Received token update mismatch expected \"%s\" our token \"%s\" update_pid %d our pid %d", token_expected, _lvmetad_token, update_pid, getpid()); if (ending_our_update && (update_pid != getpid())) { - log_warn("WARNING: lvmetad was updated by another command (pid %d).", update_pid); + log_print_unless_silent("WARNING: lvmetad was updated by another command (pid %d).", update_pid); } else { /* * Shouldn't happen. -- 1.8.3.1 ++++++ bug-1164718_28-lvmcache-free-resource-on-error-path.patch ++++++ From 34bde8b6c7e517239a05334683a09f2b5075fdcc Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Fri, 8 Nov 2019 12:51:48 +0100 Subject: [PATCH] lvmcache: free resource on error path Free allocated svg on error path. Also explicitely ignore dm_strncpy() result. (We know it will end with failure here.) --- lib/cache/lvmcache.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/cache/lvmcache.c b/lib/cache/lvmcache.c index 9890325..c12ec2b 100644 --- a/lib/cache/lvmcache.c +++ b/lib/cache/lvmcache.c @@ -325,10 +325,12 @@ void lvmcache_save_vg(struct volume_group *vg, int precommitted) dm_list_init(&svg->saved_vg_to_free); - dm_strncpy(svg->vgid, (const char *)vg->id.uuid, sizeof(svg->vgid)); + /* Ignore result code, size we intentionally short-cut & pad with 0 */ + (void) dm_strncpy(svg->vgid, (const char *)vg->id.uuid, sizeof(svg->vgid)); if (!dm_hash_insert(_saved_vg_hash, svg->vgid, svg)) { log_error("lvmcache: failed to insert saved_vg %s", svg->vgid); + dm_free(svg); return; } } else { -- 1.8.3.1 ++++++ bug-1172597_1-libdaemon-use-pselect-to-avoid-condition-checking-ra.patch ++++++ From 6d4f36c2c7f782af31eca7eca63310791a4f8e93 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Wed, 10 Apr 2019 12:50:53 +0200 Subject: [PATCH] libdaemon: use pselect to avoid condition checking race To avoid tiny race on checking arrival of signal and entering select (that can latter remain stuck as signal was already delivered) switch to use pselect(). If it would needed, we can eventually add extra code for older systems without pselect(), but there are probably no such ancient systems in use. --- daemons/lvmetad/lvmetad-core.c | 2 +- daemons/lvmpolld/lvmpolld-core.c | 2 +- libdaemon/server/daemon-server.c | 38 ++++++++++++++++++++++++++++---------- libdaemon/server/daemon-server.h | 2 +- 4 files changed, 31 insertions(+), 13 deletions(-) diff --git a/daemons/lvmetad/lvmetad-core.c b/daemons/lvmetad/lvmetad-core.c index 72473d7..9ae359b 100644 --- a/daemons/lvmetad/lvmetad-core.c +++ b/daemons/lvmetad/lvmetad-core.c @@ -2953,7 +2953,7 @@ static void usage(const char *prog, FILE *file) int main(int argc, char *argv[]) { signed char opt; - struct timeval timeout; + struct timespec timeout; daemon_idle di = { .ptimeout = &timeout }; lvmetad_state ls = { .log_config = "" }; daemon_state s = { diff --git a/daemons/lvmpolld/lvmpolld-core.c b/daemons/lvmpolld/lvmpolld-core.c index fd73272..f1056c3 100644 --- a/daemons/lvmpolld/lvmpolld-core.c +++ b/daemons/lvmpolld/lvmpolld-core.c @@ -915,7 +915,7 @@ int main(int argc, char *argv[]) int option_index = 0; int client = 0, server = 0; unsigned action = ACTION_MAX; - struct timeval timeout; + struct timespec timeout; daemon_idle di = { .ptimeout = &timeout }; struct lvmpolld_state ls = { .log_config = "" }; daemon_state s = { diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index 3b19883..c61683a 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -87,7 +87,7 @@ static int _is_idle(daemon_state s) return s.idle && s.idle->is_idle && !s.threads->next; } -static struct timeval *_get_timeout(daemon_state s) +static struct timespec *_get_timeout(daemon_state s) { return s.idle ? s.idle->ptimeout : NULL; } @@ -96,7 +96,7 @@ static void _reset_timeout(daemon_state s) { if (s.idle) { s.idle->ptimeout->tv_sec = 1; - s.idle->ptimeout->tv_usec = 0; + s.idle->ptimeout->tv_nsec = 0; } } @@ -561,6 +561,8 @@ void daemon_start(daemon_state s) thread_state _threads = { .next = NULL }; unsigned timeout_count = 0; fd_set in; + sigset_t new_set, old_set; + int ret; /* * Switch to C locale to avoid reading large locale-archive file used by @@ -628,8 +630,7 @@ void daemon_start(daemon_state s) if (!s.foreground) kill(getppid(), SIGTERM); - /* - * Use daemon_main for daemon-specific init and polling, or + /* * Use daemon_main for daemon-specific init and polling, or * use daemon_init for daemon-specific init and generic lib polling. */ @@ -643,22 +644,39 @@ void daemon_start(daemon_state s) if (!s.daemon_init(&s)) failed = 1; + if (s.socket_fd >= FD_SETSIZE) + failed = 1; /* FD out of available selectable set */ + + sigfillset(&new_set); + sigprocmask(SIG_SETMASK, NULL, &old_set); + while (!failed) { _reset_timeout(s); FD_ZERO(&in); FD_SET(s.socket_fd, &in); - if (select(FD_SETSIZE, &in, NULL, NULL, _get_timeout(s)) < 0 && errno != EINTR) - perror("select error"); + + sigprocmask(SIG_SETMASK, &new_set, NULL); + if (_shutdown_requested && !s.threads->next) { + sigprocmask(SIG_SETMASK, &old_set, NULL); + INFO(&s, "shutdown requested", s.name); + break; + } + ret = pselect(s.socket_fd + 1, &in, NULL, NULL, _get_timeout(s), &old_set); + sigprocmask(SIG_SETMASK, &old_set, NULL); + + if (ret < 0) { + if (errno != EINTR && errno != EAGAIN && + (EWOULDBLOCK == EAGAIN || errno != EWOULDBLOCK)) + perror("select error"); + continue; + } + if (FD_ISSET(s.socket_fd, &in)) { timeout_count = 0; _handle_connect(s); } - _reap(s, 0); - if (_shutdown_requested && !s.threads->next) - break; - /* s.idle == NULL equals no shutdown on timeout */ if (_is_idle(s)) { DEBUGLOG(&s, "timeout occured"); diff --git a/libdaemon/server/daemon-server.h b/libdaemon/server/daemon-server.h index 2b9ceac..e77ac20 100644 --- a/libdaemon/server/daemon-server.h +++ b/libdaemon/server/daemon-server.h @@ -47,7 +47,7 @@ struct timeval; typedef struct { volatile unsigned is_idle; unsigned max_timeouts; - struct timeval *ptimeout; + struct timespec *ptimeout; } daemon_idle; struct daemon_state; -- 1.8.3.1 ++++++ bug-1172597_2-cleanup-missed-string-specifier.patch ++++++ From 228e331e270451e16126659655905a583a03a0d3 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Wed, 17 Apr 2019 11:29:35 +0200 Subject: [PATCH] cleanup: missed string specifier --- libdaemon/server/daemon-server.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index c61683a..8534b79 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -658,7 +658,7 @@ void daemon_start(daemon_state s) sigprocmask(SIG_SETMASK, &new_set, NULL); if (_shutdown_requested && !s.threads->next) { sigprocmask(SIG_SETMASK, &old_set, NULL); - INFO(&s, "shutdown requested", s.name); + INFO(&s, "%s shutdown requested", s.name); break; } ret = pselect(s.socket_fd + 1, &in, NULL, NULL, _get_timeout(s), &old_set); -- 1.8.3.1 ++++++ bug-1172597_3-libdaemon-ensure-threads-are-reaped-before-checking-.patch ++++++ From b41b112a4bf5f674acd45f12ff4efe7fa0d1d8be Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Mon, 29 Apr 2019 13:43:36 +0200 Subject: [PATCH] libdaemon: ensure threads are reaped before checking shutdown Since we are checking _shutdown_requested - we expect all threads are finished - that is currently checked only by checking ->next ptr being NULL - so this can be NULL only when _reap() function clears out all already finished threads. I'm finding this design quite problematic in its core - but as a 'trivial hotfix' - lets _reap() linked list before check for signal. There is likely a large potentical for few races - but the windows is very very small - since lvmetad has been already purged from upstream, lets go with this hotfix. --- libdaemon/server/daemon-server.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index 8534b79..79b54d0 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -655,6 +655,7 @@ void daemon_start(daemon_state s) FD_ZERO(&in); FD_SET(s.socket_fd, &in); + _reap(s, 0); sigprocmask(SIG_SETMASK, &new_set, NULL); if (_shutdown_requested && !s.threads->next) { sigprocmask(SIG_SETMASK, &old_set, NULL); -- 1.8.3.1 ++++++ bug-1172597_4-cov-check-for-socket_path-being-set.patch ++++++ From 5cf1c61152bfd5f1eef7f49509a877090f4cc26f Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Tue, 25 Jun 2019 15:11:05 +0200 Subject: [PATCH] cov: check for socket_path being set As we check for existince on entering path let's ensure it's there also on exit path. --- libdaemon/server/daemon-server.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index 79b54d0..bc58f7b 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -693,7 +693,7 @@ void daemon_start(daemon_state s) out: /* If activated by systemd, do not unlink the socket - systemd takes care of that! */ if (!_systemd_activation && s.socket_fd >= 0) - if (unlink(s.socket_path)) + if (s.socket_path && unlink(s.socket_path)) perror("unlink error"); if (s.socket_fd >= 0) -- 1.8.3.1 ++++++ bug-1172597_5-lvmetad-fix-timeout-on-shutdown.patch ++++++ From 61358d92cbf202dbb483d63a63d5adf0463bb934 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Wed, 25 Sep 2019 17:22:49 +0200 Subject: [PATCH] lvmetad: fix timeout on shutdown When lvmetad is going to be shutdown - there were 2 issue: If there was endless stream of command contacting lvmetad - the process would never finish - so now when we recieved SIGTERM - we are no longer accepting new connections and we just wait till the existing ones are finished. 2nd. issue is that actually when we are waiting for finish of all client threads - we basically want an usleep() and check if all threads are already finished - proper solution would be to singal from a thread there is some work to do for master thread - but that would be a bigger change and since lvmetad is no longer developed - keep the change minimal and just use pselect() as our '1sec.' sleeping call once we are in 'shutdown' mode. Reported-by: wangjufeng(a)huawei.com --- libdaemon/server/daemon-server.c | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index bc58f7b..62f403a 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -89,6 +89,13 @@ static int _is_idle(daemon_state s) static struct timespec *_get_timeout(daemon_state s) { + static struct timespec _tm = { 0 }; + + if (_shutdown_requested) { + _tm.tv_sec = 1; + return &_tm; + } + return s.idle ? s.idle->ptimeout : NULL; } @@ -506,7 +513,7 @@ static int _handle_connect(daemon_state s) socklen_t sl = sizeof(sockaddr); client.socket_fd = accept(s.socket_fd, (struct sockaddr *) &sockaddr, &sl); - if (client.socket_fd < 0) { + if (client.socket_fd < 0 || _shutdown_requested) { if (errno != EAGAIN || !_shutdown_requested) ERROR(&s, "Failed to accept connection: %s.", strerror(errno)); return 0; @@ -672,7 +679,7 @@ void daemon_start(daemon_state s) continue; } - if (FD_ISSET(s.socket_fd, &in)) { + if (!_shutdown_requested && FD_ISSET(s.socket_fd, &in)) { timeout_count = 0; _handle_connect(s); } -- 1.8.3.1 ++++++ bug-1172597_6-cov-missing-checks-of-syscalls.patch ++++++ From 67bdae069751e4779e738283e3d8a5873622bfc0 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Fri, 1 Nov 2019 20:25:39 +0100 Subject: [PATCH] cov: missing checks of syscalls Check for sigaction,sigprocmask,pthread_sigmask errors --- daemons/clvmd/clvmd.c | 10 ++++++---- libdaemon/server/daemon-server.c | 9 ++++++--- tools/toollib.c | 3 ++- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/daemons/clvmd/clvmd.c b/daemons/clvmd/clvmd.c index 829c5e5..e1d8a79 100644 --- a/daemons/clvmd/clvmd.c +++ b/daemons/clvmd/clvmd.c @@ -887,7 +887,8 @@ static void main_loop(int cmd_timeout) sigemptyset(&ss); sigaddset(&ss, SIGINT); sigaddset(&ss, SIGTERM); - pthread_sigmask(SIG_UNBLOCK, &ss, NULL); + if (pthread_sigmask(SIG_UNBLOCK, &ss, NULL)) + log_warn("WARNING: Failed to unblock SIGCHLD."); /* Main loop */ while (!quit) { fd_set in; @@ -1731,11 +1732,12 @@ static __attribute__ ((noreturn)) void *pre_and_post_thread(void *arg) SIGUSR2 (kills subthreads) */ sigemptyset(&ss); sigaddset(&ss, SIGUSR1); - pthread_sigmask(SIG_BLOCK, &ss, NULL); - + if (pthread_sigmask(SIG_BLOCK, &ss, NULL)) + log_warn("WARNING: Failed to block SIGUSR1."); sigdelset(&ss, SIGUSR1); sigaddset(&ss, SIGUSR2); - pthread_sigmask(SIG_UNBLOCK, &ss, NULL); + if (pthread_sigmask(SIG_UNBLOCK, &ss, NULL)) + log_warn("WARNING: Failed to unblock SIGUSR2."); /* Loop around doing PRE and POST functions until the client goes away */ while (!client->bits.localsock.finished) { diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index 62f403a..51e5866 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -663,14 +663,17 @@ void daemon_start(daemon_state s) FD_SET(s.socket_fd, &in); _reap(s, 0); - sigprocmask(SIG_SETMASK, &new_set, NULL); + if (sigprocmask(SIG_SETMASK, &new_set, NULL)) + perror("sigprocmask error"); if (_shutdown_requested && !s.threads->next) { - sigprocmask(SIG_SETMASK, &old_set, NULL); + if (sigprocmask(SIG_SETMASK, &old_set, NULL)) + perror("sigprocmask error"); INFO(&s, "%s shutdown requested", s.name); break; } ret = pselect(s.socket_fd + 1, &in, NULL, NULL, _get_timeout(s), &old_set); - sigprocmask(SIG_SETMASK, &old_set, NULL); + if (sigprocmask(SIG_SETMASK, &old_set, NULL)) + perror("sigprocmask error"); if (ret < 0) { if (errno != EINTR && errno != EAGAIN && diff --git a/tools/toollib.c b/tools/toollib.c index 42179d9..0c1c095 100644 --- a/tools/toollib.c +++ b/tools/toollib.c @@ -59,7 +59,8 @@ int become_daemon(struct cmd_context *cmd, int skip_lvm) log_verbose("Forking background process from command: %s", cmd->cmd_line); - sigaction(SIGCHLD, &act, NULL); + if (sigaction(SIGCHLD, &act, NULL)) + log_warn("WARNING: Failed to set SIGCHLD action."); if (!skip_lvm) if (!sync_local_dev_names(cmd)) { /* Flush ops and reset dm cookie */ -- 1.8.3.1 ++++++ bug-1172597_7-daemon-better-error-path-handling-for-shutdown.patch ++++++ From e9b2148dab3a1f501637b628978c93be4d2be086 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Fri, 8 Nov 2019 13:07:06 +0100 Subject: [PATCH] daemon: better error path handling for shutdown Report errors for open in better order. Ensure descriptors are not leaked. --- libdaemon/server/daemon-server.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index 51e5866..aa9ff2a 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -513,20 +513,23 @@ static int _handle_connect(daemon_state s) socklen_t sl = sizeof(sockaddr); client.socket_fd = accept(s.socket_fd, (struct sockaddr *) &sockaddr, &sl); - if (client.socket_fd < 0 || _shutdown_requested) { - if (errno != EAGAIN || !_shutdown_requested) + if (client.socket_fd < 0) { + if (errno != EAGAIN) ERROR(&s, "Failed to accept connection: %s.", strerror(errno)); - return 0; + goto bad; + } + + if (_shutdown_requested) { + ERROR(&s, "Shutdown requested."); + goto bad; } - if (fcntl(client.socket_fd, F_SETFD, FD_CLOEXEC)) + if (fcntl(client.socket_fd, F_SETFD, FD_CLOEXEC)) WARN(&s, "setting CLOEXEC on client socket fd %d failed", client.socket_fd); if (!(ts = dm_malloc(sizeof(thread_state)))) { - if (close(client.socket_fd)) - perror("close"); ERROR(&s, "Failed to allocate thread state"); - return 0; + goto bad; } ts->next = s.threads->next; @@ -538,10 +541,16 @@ static int _handle_connect(daemon_state s) if ((errno = pthread_create(&ts->client.thread_id, NULL, _client_thread, ts))) { ERROR(&s, "Failed to create client thread: %s.", strerror(errno)); - return 0; + ts->active = 0; + goto bad; } return 1; +bad: + if ((client.socket_fd >= 0) && close(client.socket_fd)) + perror("close"); + + return 0; } static void _reap(daemon_state s, int waiting) -- 1.8.3.1 ++++++ bug-1172597_8-daemons-check-for-non-zero-thread_id.patch ++++++ From bbdcdc12b2240e00f6ab8ff105e954629412b234 Mon Sep 17 00:00:00 2001 From: Zdenek Kabelac <zkabelac(a)redhat.com> Date: Fri, 8 Nov 2019 13:10:49 +0100 Subject: [PATCH] daemons: check for non-zero thread_id Do not call pthread_join if thread_id would be 0. --- libdaemon/server/daemon-server.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libdaemon/server/daemon-server.c b/libdaemon/server/daemon-server.c index aa9ff2a..9408440 100644 --- a/libdaemon/server/daemon-server.c +++ b/libdaemon/server/daemon-server.c @@ -560,7 +560,8 @@ static void _reap(daemon_state s, int waiting) while (ts) { if (waiting || !ts->active) { - if ((errno = pthread_join(ts->client.thread_id, &rv))) + if (ts->client.thread_id && + (errno = pthread_join(ts->client.thread_id, &rv))) ERROR(&s, "pthread_join failed: %s", strerror(errno)); last->next = ts->next; dm_free(ts); -- 1.8.3.1 ++++++ bug-1173503_lvmetad-fix-pvs-for-many-devices.patch ++++++ From 5502f72e41dceb85d11ec562819ad6c5db6758ae Mon Sep 17 00:00:00 2001 From: David Teigland <teigland(a)redhat.com> Date: Mon, 27 Aug 2018 11:42:25 -0500 Subject: [PATCH] lvmetad: fix pvs for many devices When using lvmetad, 'pvs' still evaluates full filters on all devices (lvmetad only provides info about PVs, but pvs needs to report info about all devices, at least sometimes.) Because some filters read the devices, pvs still reads every device, even with lvmetad (i.e. lvmetad is no help for the pvs command.) Because the device reads are not being managed by the standard label scan layer, but only happen incidentally through the filters, there is nothing to control and limit the bcache content and the open file descriptors for the devices. When there are a lot of devs on the system, the number of open fd's excedes the limit and all opens begin failing. The proper solution for this would be for pvs to really use lvmetad and not scan devs, or for pvs to do a proper label scan even when lvmetad is enabled. To avoid any major changes to the way this has worked, just work around this problem by dropping bcache and closing the fd after pvs evaluates the filter on each device. --- tools/toollib.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/tools/toollib.c b/tools/toollib.c index 6ae78bd..3221e5f 100644 --- a/tools/toollib.c +++ b/tools/toollib.c @@ -3931,7 +3931,7 @@ static int _get_arg_devices(struct cmd_context *cmd, return ret_max; } -static int _get_all_devices(struct cmd_context *cmd, struct dm_list *all_devices) +static int _get_all_devices_lvmetad(struct cmd_context *cmd, struct dm_list *all_devices) { struct dev_iter *iter; struct device *dev; @@ -3947,6 +3947,56 @@ static int _get_all_devices(struct cmd_context *cmd, struct dm_list *all_devices return ECMD_FAILED; } + /* + * The dev_iter_get applies the filter, which means it reads the device + * (since some filters read devices). The read is called from the + * filter, and done without label_scan_open, so the dev_read_bytes does + * the open. Since the open and read are not done from the label scan + * layer, there's nothing to do label_scan_invalidate and close devs + * that are not lvms. Hack around this by doing label_scan_invalidate + * here. It's dumb that we are reading all disks here when we're meant + * to be using lvmetad. process_each_pv with lvmetad should either + * just do a proper label_scan or find a way to not need to read devs + * at all. If we didn't close each dev here, all devs would remain + * open and lvm will have too many open fds. It's all because we're + * not using the label scan layer to do the scanning, but pretending a + * label scan isn't needed (because of lvmetad) and then secretly doing + * a scan anyway hidden down in the filters. + */ + + while ((dev = dev_iter_get(iter))) { + if (!(dil = dm_pool_alloc(cmd->mem, sizeof(*dil)))) { + log_error("device_id_list alloc failed."); + goto out; + } + + strncpy(dil->pvid, dev->pvid, ID_LEN); + dil->dev = dev; + dm_list_add(all_devices, &dil->list); + + label_scan_invalidate(dev); + } + + r = ECMD_PROCESSED; +out: + dev_iter_destroy(iter); + return r; +} + +static int _get_all_devices_normal(struct cmd_context *cmd, struct dm_list *all_devices) +{ + struct dev_iter *iter; + struct device *dev; + struct device_id_list *dil; + int r = ECMD_FAILED; + + log_debug("Getting list of all devices"); + + if (!(iter = dev_iter_create(cmd->full_filter, 1))) { + log_error("dev_iter creation failed."); + return ECMD_FAILED; + } + while ((dev = dev_iter_get(iter))) { if (!(dil = dm_pool_alloc(cmd->mem, sizeof(*dil)))) { log_error("device_id_list alloc failed."); @@ -3964,6 +4014,14 @@ out: return r; } +static int _get_all_devices(struct cmd_context *cmd, struct dm_list *all_devices) +{ + if (lvmetad_used()) + return _get_all_devices_lvmetad(cmd, all_devices); + else + return _get_all_devices_normal(cmd, all_devices); +} + static int _device_list_remove(struct dm_list *devices, struct device *dev) { struct device_id_list *dil; -- 1.8.3.1 ++++++ bug-1175110_dmeventd-avoid-bail-out-preventing-repair-in-raid-pl.patch ++++++ From 8d3e01ff4f94a8d36b16520a5e402dbc7539dd2c Mon Sep 17 00:00:00 2001 From: Heinz Mauelshagen <heinzm(a)redhat.com> Date: Fri, 20 Sep 2019 17:35:35 +0200 Subject: [PATCH] dmeventd: avoid bail out preventing repair in raid plugin but keep message Followup patch mentioned in previous commit 0585754593d7c010d83274c3a25dd6c3e8c8b4a8. Problem: even though dead raid component devices are detected, the raid plugin is bailing out thus preventing a repair attempt. Rational: in case of component device errors, the MD resynchronization thread runs in parallel with the thrown event being processed by the raid plugin. The plugin retrieves the raid device status but that still reflects insync regions as 0 (when it should already be total regions) because the MD thread didn't update it yet. Solution: Remove the insync regions check but keep the informal message "waiting for resynchronization" and let lvconvert carry out its pre-repair checks and optionally carry out a repair attempt. Related: https://bugzilla.redhat.com/show_bug.cgi?id=1751887 Related: https://bugzilla.redhat.com/show_bug.cgi?id=1560739 Related: https://bugzilla.redhat.com/show_bug.cgi?id=1468590 Related: https://bugzilla.redhat.com/show_bug.cgi?id=1654860 Related: https://bugzilla.redhat.com/show_bug.cgi?id=1729303 Related: https://bugzilla.redhat.com/show_bug.cgi?id=1741016 --- daemons/dmeventd/plugins/raid/dmeventd_raid.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/daemons/dmeventd/plugins/raid/dmeventd_raid.c b/daemons/dmeventd/plugins/raid/dmeventd_raid.c index 52cf43d..fda1320 100644 --- a/daemons/dmeventd/plugins/raid/dmeventd_raid.c +++ b/daemons/dmeventd/plugins/raid/dmeventd_raid.c @@ -76,14 +76,17 @@ static int _process_raid_event(struct dso_state *state, char *params, const char } if (dead) { - if (status->insync_regions < status->total_regions) { - if (!state->warned) { - state->warned = 1; - log_warn("WARNING: waiting for resynchronization to finish " - "before initiating repair on RAID device %s.", device); - } - - goto out; /* Not yet done syncing with accessible devices */ + /* + * Use the first event to run a repair ignoring any additonal ones. + * + * We presume lvconvert to do pre-repair + * checks to avoid bloat in this plugin. + */ + if (!state->warned && status->insync_regions < status->total_regions) { + state->warned = 1; + log_warn("WARNING: waiting for resynchronization to finish " + "before initiating repair on RAID device %s.", device); + /* Fall through to allow lvconvert to run. */ } if (state->failed) -- 1.8.3.1 ++++++ bug-935623_dmeventd-fix-dso-name-wrong-compare.patch ++++++ From d5246ee0d534677439cf4af780422fbb2c9068b2 Mon Sep 17 00:00:00 2001 From: Liuhua Wang <lwang(a)suse.com> Date: Thu, 9 Jul 2015 15:44:20 +0800 Subject: [PATCH] dmeventd: fix dso name wrong compare dso_name is thought as not empty and doesn't create monitor thread, because lvm sets empty dso_name with "-". Signed-off-by: Liuhua Wang <lwang(a)suse.com> References: bnc#935623 --- daemons/dmeventd/dmeventd.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git LVM2.2.02.120.orig/daemons/dmeventd/dmeventd.c LVM2.2.02.120/daemons/dmeventd/dmeventd.c index e8d1e3b..3b1952c 100644 --- LVM2.2.02.120.orig/daemons/dmeventd/dmeventd.c +++ LVM2.2.02.120/daemons/dmeventd/dmeventd.c @@ -1218,12 +1218,12 @@ static int _want_registered_device(char *dso_name, char *device_uuid, struct thread_status *thread) { /* If DSO names and device paths are equal. */ - if (dso_name && device_uuid) + if (dso_name && strcmp(dso_name, "-") && device_uuid) return !strcmp(dso_name, thread->dso_data->dso_name) && !strcmp(device_uuid, thread->device.uuid); /* If DSO names are equal. */ - if (dso_name) + if (dso_name && strcmp(dso_name, "-")) return !strcmp(dso_name, thread->dso_data->dso_name); /* If device paths are equal. */ -- 1.8.4.5 ++++++ bug-950089_test-fix-lvm2-testsuite-build-error.patch ++++++ commit 0402acbbb9f8f6066a3f7899e8cc3ae72b84ee20 Author: Zhilong Liu <zlliu(a)suse.com> Date: Wed Dec 7 02:43:10 2016 -0500 add a new test package named lvm2-testsuite(bnc#950089) + lvm2-testsuite.patch Currently this new package is not enabled by default. Please set enable_testsuite to 1 to turn it on in the spec file. Eric: This patch is to solve the following obs building error: """ E: arch-dependent-file-in-usr-share (Badness: 590) /usr/share/lvm2-testsuite/api/lvtest.t ... """ So, move the *.t binary into /usr/lib/lvm2-testsuite/. Signed-off-by: Zhilong Liu <zlliu(a)suse.com> diff --git a/test/Makefile.in b/test/Makefile.in index f152868..f0845d7 100644 --- a/test/Makefile.in +++ b/test/Makefile.in @@ -224,7 +224,7 @@ install: .tests-stamp lib/paths-installed $(INSTALL_DATA) api/*.sh $(DATADIR)/api $(INSTALL_DATA) unit/*.sh $(DATADIR)/unit $(INSTALL_DATA) lib/mke2fs.conf $(DATADIR)/lib - $(INSTALL_PROGRAM) api/*.{t,py} $(DATADIR)/api + $(INSTALL_PROGRAM) api/*.py $(DATADIR)/api/ $(INSTALL_PROGRAM) unit/unit-test $(DATADIR)/unit $(INSTALL_PROGRAM) dbus/*.py $(DATADIR)/dbus/ $(INSTALL_DATA) lib/paths-installed $(DATADIR)/lib/paths @@ -244,6 +244,7 @@ install: .tests-stamp lib/paths-installed @cd $(EXECDIR) && for i in $(LIB_LINK_NOT); do \ echo "$(LN_S) -f not $$i"; \ $(LN_S) -f not $$i; done + $(INSTALL_PROGRAM) api/*.t $(EXECDIR) $(INSTALL_PROGRAM) -D lib/runner $(bindir)/lvm2-testsuite lib/should: lib/not ++++++ bug-978055_clvmd-try-to-refresh-device-cache-on-the-first-failu.patch ++++++ From 4f0681b1a296d88ac1dbdb26e46afed3285ad1bf Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Tue, 23 May 2017 15:09:46 +0800 Subject: [PATCH 09/10] clvmd: try to refresh device cache on the first failure 1. The original problem $ sudo lvchange -ay testvg/testlv Error locking on node 1302cf30: Volume group for uuid not found: qBKu65bSxfRq7gUf91NZuH4epLza4ifDieQJFd2to2WruVi5Brn7DxxsEgi5Zodw 2. This problem can be easily replicated a. Make clvmd running in cluster environment; b. Assume you have created LV "testlv" in local VG 'testvg' on a MD device 'md0'; c. Make sure 'md0' is stopped, and not in the device cache by executing 'clvmd -R' or 'pvscan'; d. Assemble 'md0' by issuing 'mdadm --assemble --scan --name md0'; e. To activate 'testlv', you will see the 'Error locking' problem. 3. Analysis a. After step 2.d, 'pvscan --cache ...' is triggered by udev rules, notifying 'md0' is ready. But, pvscan exits very early because lvmetad is not being used, thus doesn't go through the lock manager. Therefore, clvmd isn't aware of this udev events. The device cache hasn't 'md0'. b. In step 2.e, the client, 'lvchange -ay testvg/testlv' cmd, can find 'testlv' correctly in the client metadata, because the device list is gathered by call chain: lvm_run_command()->init_filters()->persistent_filter_load()->dev_cache_scan(). Then, it asks clvmd for "Locking VG V_testvg CR", which just drops the metadata in clmvd by call chain: do_lock_vg()->lvmcache_drop_metadata(), but the device cache is *not* refreshed. c. Finally, clvmd fails to find the lvid in activation path: do_lock_lv()->do_activate_lv()->lv_info_by_lvid() Apparently, the metadata DB is not complete without a complete device cache in clvmd. However, upstream say the pvscan tool intends to be only used with lvmetad, suggesting me not hacking there. So, we'd better fix this issue within clvmd code. Sometimes, the device cache in clvmd could be out of date. "clvmd -R" is invented for this issue. However, to run "clvmd -R" manually is not convenient, because it's hard to predict when device change would happen. This patch gives another try after refreshing the device cache. In normal, it doesn't cause any side-effect. In case of the issue above, it's worth a retry. Signed-off-by: Eric Ren <zren(a)suse.com> --- daemons/clvmd/lvm-functions.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/daemons/clvmd/lvm-functions.c b/daemons/clvmd/lvm-functions.c index 2446fd1..dcd3f9b 100644 --- a/daemons/clvmd/lvm-functions.c +++ b/daemons/clvmd/lvm-functions.c @@ -509,11 +509,14 @@ const char *do_lock_query(char *resource) int do_lock_lv(unsigned char command, unsigned char lock_flags, char *resource) { int status = 0; + int do_refresh = 0; DEBUGLOG("do_lock_lv: resource '%s', cmd = %s, flags = %s, critical_section = %d\n", resource, decode_locking_cmd(command), decode_flags(lock_flags), critical_section()); - if (!cmd->initialized.config || config_files_changed(cmd)) { +again: + if (!cmd->initialized.config || config_files_changed(cmd) + || do_refresh) { /* Reinitialise various settings inc. logging, filters */ if (do_refresh_cache()) { log_error("Updated config file invalid. Aborting."); @@ -579,6 +582,12 @@ int do_lock_lv(unsigned char command, unsigned char lock_flags, char *resource) init_test(0); pthread_mutex_unlock(&lvm_lock); + /* Try again in case device cache is stale */ + if (status == EIO && !do_refresh) { + do_refresh = 1; + goto again; + } + DEBUGLOG("Command return is %d, critical_section is %d\n", status, critical_section()); return status; } -- 2.10.2 ++++++ bug-998893_make_pvscan_service_after_multipathd.patch ++++++ From 0da043466ace27eb483114c87ce81feaf00ed682 Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Thu, 6 Apr 2017 11:22:30 +0800 Subject: [PATCH] systemd service: make pvscan service after multipathd bsc#998893 Signed-off-by: Eric Ren <zren(a)suse.com> --- scripts/lvm2_pvscan_systemd_red_hat@.service.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/scripts/lvm2_pvscan_systemd_red_hat@.service.in +++ b/scripts/lvm2_pvscan_systemd_red_hat@.service.in @@ -5,7 +5,7 @@ DefaultDependencies=no StartLimitInterval=0 BindsTo=dev-block-%i.device Requires=lvm2-lvmetad.socket -After=lvm2-lvmetad.socket lvm2-lvmetad.service +After=lvm2-lvmetad.socket lvm2-lvmetad.service multipathd.service Before=shutdown.target Conflicts=shutdown.target ++++++ cmirrord_remove_date_time_from_compilation.patch ++++++ cmirrord: Do not compile in Data and Time to avoid build retriggering in obs Signed-off-by: Thomas Renninger <trenn(a)suse.de> Index: LVM2.2.02.139/daemons/cmirrord/clogd.c =================================================================== --- LVM2.2.02.139.orig/daemons/cmirrord/clogd.c +++ LVM2.2.02.139/daemons/cmirrord/clogd.c @@ -78,7 +78,6 @@ int main(int argc, char *argv[]) kill(getppid(), SIGTERM); LOG_PRINT("Starting cmirrord:"); - LOG_PRINT(" Built: "__DATE__" "__TIME__"\n"); LOG_DBG(" Compiled with debugging."); while (!exit_now) { ++++++ fate-309425_display-dm-name-for-lv-name.patch ++++++ --- lib/config/config_settings.h | 3 +++ lib/config/defaults.h | 1 + lib/display/display.c | 13 +++++++++---- 3 files changed, 13 insertions(+), 4 deletions(-) Index: LVM2.2.02.170/lib/config/config_settings.h =================================================================== --- LVM2.2.02.170.orig/lib/config/config_settings.h +++ LVM2.2.02.170/lib/config/config_settings.h @@ -1039,6 +1039,9 @@ cfg(global_fsadm_executable_CFG, "fsadm_ "The full path to the fsadm command.\n" "LVM uses this command to help with lvresize -r operations.\n") +cfg(global_display_dm_name_for_lv_name_CFG, "display_dm_name_for_lv_name", global_CFG_SECTION, CFG_DEFAULT_COMMENTED, CFG_TYPE_BOOL, DEFAULT_DISPLAY_DM_NAME_FOR_LV_NAME, vsn(2, 2, 98), NULL, 0, NULL, + "Display dm name for lv name.\n") + cfg(global_system_id_source_CFG, "system_id_source", global_CFG_SECTION, 0, CFG_TYPE_STRING, DEFAULT_SYSTEM_ID_SOURCE, vsn(2, 2, 117), NULL, 0, NULL, "The method LVM uses to set the local system ID.\n" "Volume Groups can also be given a system ID (by vgcreate, vgchange,\n" Index: LVM2.2.02.170/lib/config/defaults.h =================================================================== --- LVM2.2.02.170.orig/lib/config/defaults.h +++ LVM2.2.02.170/lib/config/defaults.h @@ -33,6 +33,7 @@ #define DEFAULT_OBTAIN_DEVICE_LIST_FROM_UDEV 1 #define DEFAULT_EXTERNAL_DEVICE_INFO_SOURCE "none" #define DEFAULT_SYSFS_SCAN 1 +#define DEFAULT_DISPLAY_DM_NAME_FOR_LV_NAME 0 #define DEFAULT_MD_COMPONENT_DETECTION 1 #define DEFAULT_FW_RAID_COMPONENT_DETECTION 0 #define DEFAULT_MD_CHUNK_ALIGNMENT 1 Index: LVM2.2.02.170/lib/display/display.c =================================================================== --- LVM2.2.02.170.orig/lib/display/display.c +++ LVM2.2.02.170/lib/display/display.c @@ -413,10 +413,15 @@ int lvdisplay_full(struct cmd_context *c lv->vg->cmd->dev_dir, lv->vg->name, lv->name); else if (lv_is_visible(lv)) { /* Thin pool does not have /dev/vg/name link */ - if (!lv_is_thin_pool(lv)) - log_print("LV Path %s%s/%s", - lv->vg->cmd->dev_dir, - lv->vg->name, lv->name); + if (!lv_is_thin_pool(lv)) { + if (find_config_tree_bool(cmd, global_display_dm_name_for_lv_name_CFG, NULL)) { + log_print("LV Path %smapper/%s-%s", lv->vg->cmd->dev_dir, + lv->vg->name, lv->name); + } else { + log_print("LV Path %s%s/%s", lv->vg->cmd->dev_dir, + lv->vg->name, lv->name); + } + } log_print("LV Name %s", lv->name); } else log_print("Internal LV Name %s", lv->name); ++++++ fate-31841_fsadm-add-support-for-btrfs.patch ++++++ From 0e1c896caaabdf0de38466951ea281f02fe8bb77 Mon Sep 17 00:00:00 2001 From: Eric Ren <zren(a)suse.com> Date: Thu, 6 Jul 2017 17:42:58 +0800 Subject: [PATCH] fsadm: add support for btrfs Check: mount the device first and then run`btrfs filesystem scrub start -B` command Reisze: find the mount point first and resize the filesystem after get the device id since there are maybe several devices underneath btrfs filesystem --- scripts/fsadm.sh | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 72 insertions(+), 3 deletions(-) diff --git a/scripts/fsadm.sh b/scripts/fsadm.sh index a09b91534..2a29d5ad5 100755 --- a/scripts/fsadm.sh +++ b/scripts/fsadm.sh @@ -22,6 +22,7 @@ # ext2/ext3/ext4: resize2fs, tune2fs # reiserfs: resize_reiserfs, reiserfstune # xfs: xfs_growfs, xfs_info +# btrfs: btrfs # # Return values: # 0 success @@ -57,6 +58,7 @@ XFS_CHECK=xfs_check # XFS_REPAIR -n is used when XFS_CHECK is not found XFS_REPAIR=xfs_repair CRYPTSETUP=cryptsetup +BTRFS=btrfs # user may override lvm location by setting LVM_BINARY LVM=${LVM_BINARY:-lvm} @@ -76,6 +78,9 @@ BLOCKCOUNT= MOUNTPOINT= MOUNTED= REMOUNT= +FINDMNT= +UUID= +BTRFS_DEVID= PROCDIR="/proc" PROCMOUNTS="$PROCDIR/mounts" PROCSELFMOUNTINFO="$PROCDIR/self/mountinfo" @@ -227,6 +232,33 @@ detect_fs() { verbose "\"$FSTYPE\" filesystem found on \"$VOLUME\"." } +check_findmnt() { + FINDMNT=$(which findmnt 2>$NULL) + test -n "$FINDMNT" +} + +detect_fs_uuid() { + UUID=$($BLKID -o value -c $NULL -s UUID "$VOLUME" 2>$NULL) + test -n "$UUID" +} + +#find the mountpoint of this device +detect_mounted_findmnt() { + local TMP + local STR_IFS=$IFS + IFS=" $(echo -n -e '\t')" + + read -r TMP<<EOF +$($FINDMNT -nuP -o TARGET,UUID 2>$NULL | $GREP "$UUID") +EOF + + TMP=${TMP##*TARGET=\"} + TMP=${TMP%%\"*} + MOUNTED=$TMP + test -n "$MOUNTED" + + IFS=$STR_IFS +} # Check that passed mounted MAJOR:MINOR is not matching $MAJOR:MINOR of resized $VOLUME validate_mounted_major_minor() { @@ -354,8 +386,12 @@ detect_mounted_with_proc_mounts() { # check if the given device is already mounted and where # FIXME: resolve swap usage and device stacking -detect_mounted() { - if test -e "$PROCSELFMOUNTINFO"; then +detect_mounted() { + if test "$FSTYPE" = "btrfs" ; then + check_findmnt || error "Need 'findmnt' utility to work with btrfs filesystem" + detect_fs_uuid || verbose "Can't get fs UUID from \"$VOLUME\" volume" + detect_mounted_findmnt + elif test -e "$PROCSELFMOUNTINFO"; then detect_mounted_with_proc_self_mountinfo elif test -e "$PROCMOUNTS"; then detect_mounted_with_proc_mounts @@ -654,6 +690,32 @@ resize_crypt() { dry "$CRYPTSETUP" resize "$1" --size $CRYPT_RESIZE_BLOCKS || error "$CRYPTSETUP failed to resize device $1" } +######################## +# Resize btrfs filesystem +# - mounted for upsize/downsize +# - cannot resize when unmounted +######################## +resize_btrfs() { + detect_mounted + MOUNTPOINT=$MOUNTED + if [ -z "$MOUNTED" ]; then + MOUNTPOINT=$TEMPDIR + temp_mount || error "Cannot mount Btrfs filesystem" + fi + + verbose "Parsing $BTRFS filesystem show \"$MOUNTPOINT\"" + for i in $(LC_ALL=C "$BTRFS" filesystem show "$MOUNTPOINT"); do + case "$i" in + *"$VOLUME"*) BTRFS_DEVID=${i##*devid};; + esac + done + BTRFS_DEVID=${BTRFS_DEVID%%size*} + BTRFS_DEVID=$(echo $BTRFS_DEVID|sed 's/^[ \t]*//g'|sed 's/[ \t]*$'//g) + decode_size $1 1 + verbose "Resizing filesystem on device \"$VOLUME\" to $NEWSIZE bytes(btrfs devid: $BTRFS_DEVID) " + dry "$BTRFS" filesystem resize "$BTRFS_DEVID":"$NEWSIZE" "$MOUNTPOINT" +} + #################### # Resize filesystem #################### @@ -676,6 +738,7 @@ resize() { "crypto_LUKS") which "$CRYPTSETUP" > /dev/null 2>&1 || error "$CRYPTSETUP utility required to resize LUKS volume" resize_luks $NEWSIZE ;; + "btrfs") resize_btrfs $NEWSIZE ;; *) error "Filesystem \"$FSTYPE\" on device \"$VOLUME\" is not supported by this tool." ;; esac || error "Resize $FSTYPE failed." test -z "$CRYPT_SHRINK" || resize_crypt "$VOLUME_ORIG" @@ -748,6 +811,12 @@ check() { "crypto_LUKS") which "$CRYPTSETUP" > /dev/null 2>&1 || error "$CRYPTSETUP utility required." check_luks ;; + "btrfs") #mount the device first and then run scrub + MOUNTPOINT=$TEMPDIR + temp_mount || error "Cannot mount btrfs filesystem" + dry "$BTRFS" scrub start -B "$VOLUME" + test "$MOUNTPOINT" = "$TEMPDIR" && MOUNTPOINT="" temp_umount + ;; *) error "Filesystem \"$FSTYPE\" on device \"$VOLUME\" is not supported by this tool." ;; esac @@ -766,7 +835,7 @@ test -n "$FSADM_RUNNING" && exit 0 for i in "$TUNE_EXT" "$RESIZE_EXT" "$TUNE_REISER" "$RESIZE_REISER" \ "$TUNE_XFS" "$RESIZE_XFS" "$MOUNT" "$UMOUNT" "$MKDIR" \ "$RMDIR" "$BLOCKDEV" "$BLKID" "$GREP" "$READLINK" \ - "$DATE" "$FSCK" "$XFS_CHECK" "$XFS_REPAIR" "$LVM" ; do + "$DATE" "$FSCK" "$XFS_CHECK" "$XFS_REPAIR" "$LVM" "$BTRFS" ; do test -n "$i" || error "Required command definitions in the script are missing!" done -- 2.13.6 ++++++ lvm.conf ++++++ ++++ 2146 lines (skipped) ++++++ pre_checkin.sh ++++++ #!/usr/bin/env bash master="lvm2.spec" sections="COMMON-DEF COMMON-PATCH COMMON-PREP COMMON-CONFIG" for slave in device-mapper.spec lvm2-clvm.spec; do { prev=1 for section in $sections; do begin="/$section-BEGIN/" end="/$section-END/" sed -n -e "${prev},${begin}p" $slave sed -n -e "${begin},${end}p" $master | head -n -1 | tail -n +2 prev=$end done sed -n -e "${prev},\$p" $slave } > $slave.tmp && mv $slave.tmp $slave done # changelogs cp lvm2.changes lvm2-clvm.changes cp lvm2.changes device-mapper.changes ++++++ tests-specify-python3-as-the-script-interpreter.patch ++++++ From 3f768d29ceb5427b6e8de4fe35e2c1001409d750 Mon Sep 17 00:00:00 2001 From: Gang He <ghe(a)suse.com> Date: Wed, 20 Jun 2018 14:04:52 +0800 Subject: [PATCH] tests: specify python3 as the script interpreter specify /usr/bin/python3 as the script interpreter in python_lvm_unit.py.in file, otherwise, there will be a building error in OBS. Signed-off-by: Gang He <ghe(a)suse.com> --- test/api/python_lvm_unit.py.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/api/python_lvm_unit.py.in b/test/api/python_lvm_unit.py.in index 78ced7e31..c6a7c9905 100755 --- a/test/api/python_lvm_unit.py.in +++ b/test/api/python_lvm_unit.py.in @@ -1,4 +1,4 @@ -#!@PYTHON@ +#!/usr/bin/python3 # Copyright (C) 2012-2013 Red Hat, Inc. All rights reserved. # -- 2.12.3

1 0

commit lvm2-clvm for openSUSE:Leap:15.1:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package lvm2-clvm for openSUSE:Leap:15.1:Update checked in at 2020-12-01 11:34:02 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/lvm2-clvm (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.lvm2-clvm.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "lvm2-clvm" Tue Dec 1 11:34:02 2020 rev:8 rq:851970 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.7qqpZC/_old 2020-12-01 11:34:03.159024850 +0100 +++ /var/tmp/diff_new_pack.7qqpZC/_new 2020-12-01 11:34:03.163024854 +0100 @@ -1 +1 @@ -<link package='lvm2-clvm.15067' cicount='copy' /> +<link package='lvm2-clvm.15198' cicount='copy' />

1 0

commit device-mapper for openSUSE:Leap:15.1:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package device-mapper for openSUSE:Leap:15.1:Update checked in at 2020-12-01 11:34:01 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/device-mapper (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.device-mapper.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "device-mapper" Tue Dec 1 11:34:01 2020 rev:8 rq:851970 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.i1IgLv/_old 2020-12-01 11:34:02.583024293 +0100 +++ /var/tmp/diff_new_pack.i1IgLv/_new 2020-12-01 11:34:02.583024293 +0100 @@ -1 +1 @@ -<link package='device-mapper.15067' cicount='copy' /> +<link package='device-mapper.15198' cicount='copy' />

1 0

commit 00Meta for openSUSE:Leap:15.2:Images
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package 00Meta for openSUSE:Leap:15.2:Images checked in at 2020-12-01 11:15:24 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2:Images/00Meta (Old) and /work/SRC/openSUSE:Leap:15.2:Images/.00Meta.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "00Meta" Tue Dec 1 11:15:24 2020 rev:650 rq: version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ version_snapshot ++++++ --- /var/tmp/diff_new_pack.6DyAoC/_old 2020-12-01 11:15:25.965946716 +0100 +++ /var/tmp/diff_new_pack.6DyAoC/_new 2020-12-01 11:15:25.965946716 +0100 @@ -1 +1 @@ -31.265 \ No newline at end of file +31.266 \ No newline at end of file

1 0

commit MozillaThunderbird for openSUSE:Leap:15.1:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package MozillaThunderbird for openSUSE:Leap:15.1:Update checked in at 2020-12-01 08:51:48 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/MozillaThunderbird (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.MozillaThunderbird.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "MozillaThunderbird" Tue Dec 1 08:51:48 2020 rev:18 rq:851461 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.xzswAA/_old 2020-12-01 08:52:08.764355828 +0100 +++ /var/tmp/diff_new_pack.xzswAA/_new 2020-12-01 08:52:08.764355828 +0100 @@ -1 +1 @@ -<link package='MozillaThunderbird.14803' cicount='copy' /> +<link package='MozillaThunderbird.15059' cicount='copy' />

1 0

commit 00Meta for openSUSE:Leap:15.2:Images
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package 00Meta for openSUSE:Leap:15.2:Images checked in at 2020-12-01 07:15:31 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2:Images/00Meta (Old) and /work/SRC/openSUSE:Leap:15.2:Images/.00Meta.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "00Meta" Tue Dec 1 07:15:31 2020 rev:649 rq: version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ version_totest ++++++ --- /var/tmp/diff_new_pack.8O6B7V/_old 2020-12-01 07:15:33.147697543 +0100 +++ /var/tmp/diff_new_pack.8O6B7V/_new 2020-12-01 07:15:33.147697543 +0100 @@ -1 +1 @@ -31.265 \ No newline at end of file +31.266 \ No newline at end of file

1 0

commit 00Meta for openSUSE:Leap:15.1:Images
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package 00Meta for openSUSE:Leap:15.1:Images checked in at 2020-12-01 06:45:24 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Images/00Meta (Old) and /work/SRC/openSUSE:Leap:15.1:Images/.00Meta.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "00Meta" Tue Dec 1 06:45:24 2020 rev:603 rq: version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ version_snapshot ++++++ --- /var/tmp/diff_new_pack.mN7klG/_old 2020-12-01 06:45:25.798621525 +0100 +++ /var/tmp/diff_new_pack.mN7klG/_new 2020-12-01 06:45:25.798621525 +0100 @@ -1 +1 @@ -8.12.192 \ No newline at end of file +8.12.194 \ No newline at end of file

1 0

commit gnome-shell-extension-desktop-icons for openSUSE:Leap:15.2:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package gnome-shell-extension-desktop-icons for openSUSE:Leap:15.2:Update checked in at 2020-12-01 06:25:53 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2:Update/gnome-shell-extension-desktop-icons (Old) and /work/SRC/openSUSE:Leap:15.2:Update/.gnome-shell-extension-desktop-icons.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "gnome-shell-extension-desktop-icons" Tue Dec 1 06:25:53 2020 rev:2 rq:851416 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.rICUA6/_old 2020-12-01 06:25:54.553692958 +0100 +++ /var/tmp/diff_new_pack.rICUA6/_new 2020-12-01 06:25:54.553692958 +0100 @@ -1 +1 @@ -<link package='gnome-shell-extension-desktop-icons.13772' cicount='copy' /> +<link package='gnome-shell-extension-desktop-icons.15174' cicount='copy' />

1 0

commit xrdp for openSUSE:Leap:15.2:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package xrdp for openSUSE:Leap:15.2:Update checked in at 2020-12-01 06:25:48 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2:Update/xrdp (Old) and /work/SRC/openSUSE:Leap:15.2:Update/.xrdp.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "xrdp" Tue Dec 1 06:25:48 2020 rev:3 rq:851402 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.QPTAIa/_old 2020-12-01 06:25:49.565688641 +0100 +++ /var/tmp/diff_new_pack.QPTAIa/_new 2020-12-01 06:25:49.565688641 +0100 @@ -1 +1 @@ -<link package='xrdp.13941' cicount='copy' /> +<link package='xrdp.15171' cicount='copy' />

1 0

commit gnutls for openSUSE:Leap:15.2:Update
by User for buildservice source handling 01 Dec '20

01 Dec '20

Hello community, here is the log from the commit of package gnutls for openSUSE:Leap:15.2:Update checked in at 2020-12-01 06:25:43 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2:Update/gnutls (Old) and /work/SRC/openSUSE:Leap:15.2:Update/.gnutls.new.5913 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "gnutls" Tue Dec 1 06:25:43 2020 rev:2 rq:851414 version:unknown Changes: -------- New Changes file: NO CHANGES FILE!!! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ _link ++++++ --- /var/tmp/diff_new_pack.XQvcG2/_old 2020-12-01 06:25:44.893684596 +0100 +++ /var/tmp/diff_new_pack.XQvcG2/_new 2020-12-01 06:25:44.897684600 +0100 @@ -1 +1 @@ -<link package='gnutls.14655' cicount='copy' /> +<link package='gnutls.15170' cicount='copy' />

1 0

Jump to page: