December 2018 - openSUSE Commits - openSUSE Mailing Lists

commit blueberry for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package blueberry for openSUSE:Factory checked in at 2018-12-03 10:10:16 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/blueberry (Old) and /work/SRC/openSUSE:Factory/.blueberry.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "blueberry" Mon Dec 3 10:10:16 2018 rev:32 rq:653149 version:1.2.4 Changes: -------- --- /work/SRC/openSUSE:Factory/blueberry/blueberry.changes 2018-09-03 10:36:13.140842443 +0200 +++ /work/SRC/openSUSE:Factory/.blueberry.new.19453/blueberry.changes 2018-12-03 10:10:31.455701591 +0100 @@ -1,0 +2,6 @@ +Fri Nov 30 17:46:42 UTC 2018 - sor.alexei(a)meowr.ru + +- Update to version 1.2.4: + * Update translations. + +------------------------------------------------------------------- Old: ---- blueberry-1.2.3.tar.gz New: ---- blueberry-1.2.4.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ blueberry.spec ++++++ --- /var/tmp/diff_new_pack.OB8UZy/_old 2018-12-03 10:10:32.007701080 +0100 +++ /var/tmp/diff_new_pack.OB8UZy/_new 2018-12-03 10:10:32.011701077 +0100 @@ -12,13 +12,13 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # %define __requires_exclude typelib\$(St)\$ Name: blueberry -Version: 1.2.3 +Version: 1.2.4 Release: 0 Summary: A configuration tool for Bluetooth License: GPL-3.0-or-later ++++++ blueberry-1.2.3.tar.gz -> blueberry-1.2.4.tar.gz ++++++ ++++ 3654 lines of diff (skipped)

1 0

commit virt-manager for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package virt-manager for openSUSE:Factory checked in at 2018-12-03 10:10:08 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/virt-manager (Old) and /work/SRC/openSUSE:Factory/.virt-manager.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "virt-manager" Mon Dec 3 10:10:08 2018 rev:186 rq:653121 version:2.0.0 Changes: -------- --- /work/SRC/openSUSE:Factory/virt-manager/virt-manager.changes 2018-11-28 11:15:42.486704910 +0100 +++ /work/SRC/openSUSE:Factory/.virt-manager.new.19453/virt-manager.changes 2018-12-03 10:10:21.535710756 +0100 @@ -1,0 +2,7 @@ +Thu Nov 29 11:50:08 MST 2018 - carnold(a)suse.com + +- bsc#1117846 - virt-manager: Checking for virtualization packages + error + virtman-load-stored-uris.patch + +------------------------------------------------------------------- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ virt-manager.spec ++++++ --- /var/tmp/diff_new_pack.yXnNox/_old 2018-12-03 10:10:22.267710079 +0100 +++ /var/tmp/diff_new_pack.yXnNox/_new 2018-12-03 10:10:22.271710076 +0100 @@ -12,7 +12,7 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # ++++++ virtman-load-stored-uris.patch ++++++ --- /var/tmp/diff_new_pack.yXnNox/_old 2018-12-03 10:10:22.431709928 +0100 +++ /var/tmp/diff_new_pack.yXnNox/_new 2018-12-03 10:10:22.431709928 +0100 @@ -5,27 +5,21 @@ =================================================================== --- virt-manager-2.0.0.orig/virtManager/connmanager.py +++ virt-manager-2.0.0/virtManager/connmanager.py -@@ -26,12 +26,23 @@ class vmmConnectionManager(vmmGObject): +@@ -26,11 +26,17 @@ class vmmConnectionManager(vmmGObject): def __init__(self): vmmGObject.__init__(self) + from .connect import vmmConnect -+ tryuri = vmmConnect.default_uri() -+ found_uri = False ++ default_uri = vmmConnect.default_uri() self._conns = {} # Load URIs from gsettings for uri in self.config.get_conn_uris(): + if uri == 'xen:///' or uri == 'qemu:///system': -+ if tryuri and tryuri != uri: -+ logging.debug("Skipping %s because it is incompatible with local host", uri) ++ if default_uri and default_uri != uri: ++ logging.debug("Skipping local connection %s because it is incompatible with the host", uri) + continue -+ found_uri = True self.add_conn(uri) -+ if tryuri and found_uri is False: -+ logging.debug("CEA:: Didn't connect with anything, use default %s", tryuri) -+ self.add_conn(tryuri) def _cleanup(self): - for conn in self._conns.values():

1 0

commit kvm_stat for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package kvm_stat for openSUSE:Factory checked in at 2018-12-03 10:09:58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/kvm_stat (Old) and /work/SRC/openSUSE:Factory/.kvm_stat.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "kvm_stat" Mon Dec 3 10:09:58 2018 rev:6 rq:653117 version:MACRO Changes: -------- --- /work/SRC/openSUSE:Factory/kvm_stat/kvm_stat.changes 2018-11-12 09:43:49.388974146 +0100 +++ /work/SRC/openSUSE:Factory/.kvm_stat.new.19453/kvm_stat.changes 2018-12-03 10:10:20.515711698 +0100 @@ -1,0 +2,74 @@ +Thu Nov 29 19:51:15 UTC 2018 - Larry Dewey <ldewey(a)suse.com> + +* Changed the spec file to build SLE15-SP1 with the s390x patches, + and added logic to the spec file to uniquely build Factory and + SLE-15 as they do not require the first 44 patches, and the code + base is not the same +- Making kvm_stat use python3 for openSUSE (bsc#1116822) + - 0047-tools-kvm_stat-switch-python-reference-to-be-explici.patch + + 0047-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat2.patch + + 0048-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat3.patch + + 0049-tools-kvm_stat-switch-python-reference-to-be-explici.patch + +------------------------------------------------------------------- +Mon Nov 26 18:41:52 UTC 2018 - Bruce Rogers <brogers(a)suse.com> + +- Make kvm_stat use python3 (bsc#1116822) + + 0044-tools-kvm_stat-Don-t-use-deprecated-file.patch + + 0045-tools-kvm_stat-fix-python3-issues.patch + + 0046-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat.patch + + 0047-tools-kvm_stat-switch-python-reference-to-be-explici.patch + +------------------------------------------------------------------- +Tue Nov 19 20:58:23 UTC 2018 - Larry Dewey <ldewey(a)suse.com> + +* Incorporating patches and comments produced by Fei Li for + SLE15-SP1 +- Backport 43 kvm_stat patches between kernel v4.16-* and current + v4.12.14 for the kvm_stat package. Actually some of the patches + are s390x specific, and some are noarch. (FATE#325017) + + 0001-tools-kvm_stat-fix-event-counts-display-for-interrup.patch + + 0002-tools-kvm_stat-fix-undue-use-of-initial-sleeptime.patch + + 0003-tools-kvm_stat-remove-unnecessary-header-redraws.patch + + 0004-tools-kvm_stat-simplify-line-print-logic.patch + + 0005-tools-kvm_stat-remove-extra-statement.patch + + 0006-tools-kvm_stat-simplify-initializers.patch + + 0007-tools-kvm_stat-move-functions-to-corresponding-class.patch + + 0008-tools-kvm_stat-show-cursor-in-selection-screens.patch + + 0009-tools-kvm_stat-display-message-indicating-lack-of-ev.patch + + 0010-tools-kvm_stat-make-heading-look-a-bit-more-like-top.patch + + 0011-tools-kvm_stat-rename-Current-column-to-CurAvg-s.patch + + 0012-tools-kvm_stat-add-new-interactive-command-h.patch + + 0013-tools-kvm_stat-add-new-interactive-command-s.patch + + 0014-tools-kvm_stat-add-new-interactive-command-o.patch + + 0015-tools-kvm_stat-display-guest-list-in-pid-guest-selec.patch + + 0016-tools-kvm_stat-fix-error-on-interactive-command-g.patch + + 0017-tools-kvm_stat-add-new-command-line-switch-i.patch + + 0018-tools-kvm_stat-add-new-interactive-command-b.patch + + 0019-tools-kvm_stat-add-f-help-to-get-the-available-event.patch + + 0020-tools-kvm_stat-Add-Python-3-support-to-kvm_stat.patch + + 0021-tools-kvm_stat-fix-command-line-option-g.patch + + 0022-tools-kvm_stat-fix-drilldown-in-events-by-guests-mod.patch + + 0023-tools-kvm_stat-fix-missing-field-update-after-filter.patch + + 0024-tools-kvm_stat-fix-extra-handling-of-help-with-field.patch + + 0025-tools-kvm_stat-add-hint-on-f-help-to-man-page.patch + + 0026-tools-kvm_stat-fix-child-trace-events-accounting.patch + + 0027-tools-kvm_stat-handle-invalid-regular-expressions.patch + + 0028-tools-kvm_stat-suppress-usage-information-on-command.patch + + 0029-tools-kvm_stat-stop-ignoring-unhandled-arguments.patch + + 0030-tools-kvm_stat-add-line-for-totals.patch + + 0031-tools-kvm_stat-sort-f-help-output.patch + + 0032-tools-kvm_stat-simplify-the-sortkey-function.patch + + 0033-tools-kvm_stat-use-a-namedtuple-for-storing-the-valu.patch + + 0034-tools-kvm_stat-use-a-more-pythonic-way-to-iterate-ov.patch + + 0035-tools-kvm_stat-avoid-is-for-equality-checks.patch + + 0036-tools-kvm_stat-fix-crash-when-filtering-out-all-non-.patch + + 0037-tools-kvm_stat-print-error-on-invalid-regex.patch + + 0038-tools-kvm_stat-fix-debugfs-handling.patch + + 0039-tools-kvm_stat-mark-private-methods-as-such.patch + + 0040-tools-kvm_stat-eliminate-extra-guest-pid-selection-d.patch + + 0041-tools-kvm_stat-separate-drilldown-and-fields-filteri.patch + + 0042-tools-kvm_stat-group-child-events-indented-after-par.patch + + 0043-tools-kvm_stat-print-Total-line-for-multiple-events-.patch + +------------------------------------------------------------------- New: ---- 0001-tools-kvm_stat-fix-event-counts-display-for-interrup.patch 0002-tools-kvm_stat-fix-undue-use-of-initial-sleeptime.patch 0003-tools-kvm_stat-remove-unnecessary-header-redraws.patch 0004-tools-kvm_stat-simplify-line-print-logic.patch 0005-tools-kvm_stat-remove-extra-statement.patch 0006-tools-kvm_stat-simplify-initializers.patch 0007-tools-kvm_stat-move-functions-to-corresponding-class.patch 0008-tools-kvm_stat-show-cursor-in-selection-screens.patch 0009-tools-kvm_stat-display-message-indicating-lack-of-ev.patch 0010-tools-kvm_stat-make-heading-look-a-bit-more-like-top.patch 0011-tools-kvm_stat-rename-Current-column-to-CurAvg-s.patch 0012-tools-kvm_stat-add-new-interactive-command-h.patch 0013-tools-kvm_stat-add-new-interactive-command-s.patch 0014-tools-kvm_stat-add-new-interactive-command-o.patch 0015-tools-kvm_stat-display-guest-list-in-pid-guest-selec.patch 0016-tools-kvm_stat-fix-error-on-interactive-command-g.patch 0017-tools-kvm_stat-add-new-command-line-switch-i.patch 0018-tools-kvm_stat-add-new-interactive-command-b.patch 0019-tools-kvm_stat-add-f-help-to-get-the-available-event.patch 0020-tools-kvm_stat-Add-Python-3-support-to-kvm_stat.patch 0021-tools-kvm_stat-fix-command-line-option-g.patch 0022-tools-kvm_stat-fix-drilldown-in-events-by-guests-mod.patch 0023-tools-kvm_stat-fix-missing-field-update-after-filter.patch 0024-tools-kvm_stat-fix-extra-handling-of-help-with-field.patch 0025-tools-kvm_stat-add-hint-on-f-help-to-man-page.patch 0026-tools-kvm_stat-fix-child-trace-events-accounting.patch 0027-tools-kvm_stat-handle-invalid-regular-expressions.patch 0028-tools-kvm_stat-suppress-usage-information-on-command.patch 0029-tools-kvm_stat-stop-ignoring-unhandled-arguments.patch 0030-tools-kvm_stat-add-line-for-totals.patch 0031-tools-kvm_stat-sort-f-help-output.patch 0032-tools-kvm_stat-simplify-the-sortkey-function.patch 0033-tools-kvm_stat-use-a-namedtuple-for-storing-the-valu.patch 0034-tools-kvm_stat-use-a-more-pythonic-way-to-iterate-ov.patch 0035-tools-kvm_stat-avoid-is-for-equality-checks.patch 0036-tools-kvm_stat-fix-crash-when-filtering-out-all-non-.patch 0037-tools-kvm_stat-print-error-on-invalid-regex.patch 0038-tools-kvm_stat-fix-debugfs-handling.patch 0039-tools-kvm_stat-mark-private-methods-as-such.patch 0040-tools-kvm_stat-eliminate-extra-guest-pid-selection-d.patch 0041-tools-kvm_stat-separate-drilldown-and-fields-filteri.patch 0042-tools-kvm_stat-group-child-events-indented-after-par.patch 0043-tools-kvm_stat-print-Total-line-for-multiple-events-.patch 0044-tools-kvm_stat-Don-t-use-deprecated-file.patch 0045-tools-kvm_stat-fix-python3-issues.patch 0046-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat.patch 0047-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat2.patch 0048-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat3.patch 0049-tools-kvm_stat-switch-python-reference-to-be-explici.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ kvm_stat.spec ++++++ --- /var/tmp/diff_new_pack.9ueQm0/_old 2018-12-03 10:10:21.039711213 +0100 +++ /var/tmp/diff_new_pack.9ueQm0/_new 2018-12-03 10:10:21.043711210 +0100 @@ -29,13 +29,63 @@ BuildRequires: kernel-source >= 4.7.0 BuildRequires: libxslt-tools -Requires: python-curses +Requires: python3-curses Recommends: kernel >= 4.7.0 Conflicts: qemu < 2.6.90 Conflicts: qemu-kvm < 2.6.90 BuildRoot: %{_tmppath}/%{name}-%{version}-build +Patch0: 0001-tools-kvm_stat-fix-event-counts-display-for-interrup.patch +Patch1: 0002-tools-kvm_stat-fix-undue-use-of-initial-sleeptime.patch +Patch2: 0003-tools-kvm_stat-remove-unnecessary-header-redraws.patch +Patch3: 0004-tools-kvm_stat-simplify-line-print-logic.patch +Patch4: 0005-tools-kvm_stat-remove-extra-statement.patch +Patch5: 0006-tools-kvm_stat-simplify-initializers.patch +Patch6: 0007-tools-kvm_stat-move-functions-to-corresponding-class.patch +Patch7: 0008-tools-kvm_stat-show-cursor-in-selection-screens.patch +Patch8: 0009-tools-kvm_stat-display-message-indicating-lack-of-ev.patch +Patch9: 0010-tools-kvm_stat-make-heading-look-a-bit-more-like-top.patch +Patch10: 0011-tools-kvm_stat-rename-Current-column-to-CurAvg-s.patch +Patch11: 0012-tools-kvm_stat-add-new-interactive-command-h.patch +Patch12: 0013-tools-kvm_stat-add-new-interactive-command-s.patch +Patch13: 0014-tools-kvm_stat-add-new-interactive-command-o.patch +Patch14: 0015-tools-kvm_stat-display-guest-list-in-pid-guest-selec.patch +Patch15: 0016-tools-kvm_stat-fix-error-on-interactive-command-g.patch +Patch16: 0017-tools-kvm_stat-add-new-command-line-switch-i.patch +Patch17: 0018-tools-kvm_stat-add-new-interactive-command-b.patch +Patch18: 0019-tools-kvm_stat-add-f-help-to-get-the-available-event.patch +Patch19: 0020-tools-kvm_stat-Add-Python-3-support-to-kvm_stat.patch +Patch20: 0021-tools-kvm_stat-fix-command-line-option-g.patch +Patch21: 0022-tools-kvm_stat-fix-drilldown-in-events-by-guests-mod.patch +Patch22: 0023-tools-kvm_stat-fix-missing-field-update-after-filter.patch +Patch23: 0024-tools-kvm_stat-fix-extra-handling-of-help-with-field.patch +Patch24: 0025-tools-kvm_stat-add-hint-on-f-help-to-man-page.patch +Patch25: 0026-tools-kvm_stat-fix-child-trace-events-accounting.patch +Patch26: 0027-tools-kvm_stat-handle-invalid-regular-expressions.patch +Patch27: 0028-tools-kvm_stat-suppress-usage-information-on-command.patch +Patch28: 0029-tools-kvm_stat-stop-ignoring-unhandled-arguments.patch +Patch29: 0030-tools-kvm_stat-add-line-for-totals.patch +Patch30: 0031-tools-kvm_stat-sort-f-help-output.patch +Patch31: 0032-tools-kvm_stat-simplify-the-sortkey-function.patch +Patch32: 0033-tools-kvm_stat-use-a-namedtuple-for-storing-the-valu.patch +Patch33: 0034-tools-kvm_stat-use-a-more-pythonic-way-to-iterate-ov.patch +Patch34: 0035-tools-kvm_stat-avoid-is-for-equality-checks.patch +Patch35: 0036-tools-kvm_stat-fix-crash-when-filtering-out-all-non-.patch +Patch36: 0037-tools-kvm_stat-print-error-on-invalid-regex.patch +Patch37: 0038-tools-kvm_stat-fix-debugfs-handling.patch +Patch38: 0039-tools-kvm_stat-mark-private-methods-as-such.patch +Patch39: 0040-tools-kvm_stat-eliminate-extra-guest-pid-selection-d.patch +Patch40: 0041-tools-kvm_stat-separate-drilldown-and-fields-filteri.patch +Patch41: 0042-tools-kvm_stat-group-child-events-indented-after-par.patch +Patch42: 0043-tools-kvm_stat-print-Total-line-for-multiple-events-.patch +Patch43: 0044-tools-kvm_stat-Don-t-use-deprecated-file.patch +Patch44: 0045-tools-kvm_stat-fix-python3-issues.patch +Patch45: 0046-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat.patch +Patch46: 0047-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat2.patch +Patch47: 0048-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat3.patch +Patch48: 0049-tools-kvm_stat-switch-python-reference-to-be-explici.patch + %description This package provides a userspace tool "kvm_stat", which displays KVM vm exit information as a means of monitoring vm behavior. The data is taken from the @@ -46,6 +96,72 @@ # copy necessary files from kernel-source (tar -C /usr/src/linux -c COPYING tools scripts) | tar -x +%define SLE15_SP1 0%{?sle_version} == 150100 && 0%{?is_opensuse} != 1 +%define FACTORY 0%{?is_opensuse} && 0%{?suse_version} > 1500 +%define SLE_LEAP_15 0%{?sle_version} <= 150000 && 0%{?suse_version} <= 1500 + +# Only apply the first few patches to SLES15-SP1 +%if 0%{?SLE15_SP1} +%patch0 -p1 +%patch1 -p1 +%patch2 -p1 +%patch3 -p1 +%patch4 -p1 +%patch5 -p1 +%patch6 -p1 +%patch7 -p1 +%patch8 -p1 +%patch9 -p1 +%patch10 -p1 +%patch11 -p1 +%patch12 -p1 +%patch13 -p1 +%patch14 -p1 +%patch15 -p1 +%patch16 -p1 +%patch17 -p1 +%patch18 -p1 +%patch19 -p1 +%patch20 -p1 +%patch21 -p1 +%patch22 -p1 +%patch23 -p1 +%patch24 -p1 +%patch25 -p1 +%patch26 -p1 +%patch27 -p1 +%patch28 -p1 +%patch29 -p1 +%patch30 -p1 +%patch31 -p1 +%patch32 -p1 +%patch33 -p1 +%patch34 -p1 +%patch35 -p1 +%patch36 -p1 +%patch37 -p1 +%patch38 -p1 +%patch39 -p1 +%patch40 -p1 +%patch41 -p1 +%patch42 -p1 +%patch43 -p1 +%patch44 -p1 +%patch45 -p1 +%endif + +# If the build is SLE15 / Leap 15 or older +%if 0%{?SLE_LEAP_15} +%patch46 -p1 +%endif + +# If the build is Factory or Tumbleweed only +%if 0%{?FACTORY} +%patch47 -p1 +%endif + +%patch48 -p1 + %build make -C tools/kvm/kvm_stat %{?_smp_mflags} ++++++ 0001-tools-kvm_stat-fix-event-counts-display-for-interrup.patch ++++++ >From 9a19f292a43c66726007d08b62c44f99981edbe8 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:26 +0200 Subject: [PATCH 01/43] tools/kvm_stat: fix event counts display for interrupted intervals When an update interval is interrupted via key press (e.g. space), the 'Current' column value is calculated using the full interval length instead of the elapsed time, which leads to lower than actual numbers. Furthermore, the value should be rounded, not truncated. This is fixed by using the actual elapsed time for the calculation. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 124c2fc9fdf5fb1d9cea4707d7e5471e317ba3bf) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 8f74ed8e7237..4fda0b4c7354 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1009,7 +1009,8 @@ class Tui(object): self.screen.addstr(row, col, '%7.1f' % (values[0] * 100 / total,)) col += 7 if values[1] is not None: - self.screen.addstr(row, col, '%8d' % (values[1] / sleeptime,)) + self.screen.addstr(row, col, '%8d' % + round(values[1] / sleeptime)) row += 1 self.screen.refresh() @@ -1130,9 +1131,11 @@ class Tui(object): """Refreshes the screen and processes user input.""" sleeptime = DELAY_INITIAL self.refresh_header() + start = 0.0 # result based on init value never appears on screen while True: - self.refresh_body(sleeptime) + self.refresh_body(time.time() - start) curses.halfdelay(int(sleeptime * 10)) + start = time.time() sleeptime = DELAY_REGULAR try: char = self.screen.getkey() -- 2.12.3 ++++++ 0002-tools-kvm_stat-fix-undue-use-of-initial-sleeptime.patch ++++++ >From e9f1b77c35e178ce8aed246ff69d604ac6aad2a2 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:27 +0200 Subject: [PATCH 02/43] tools/kvm_stat: fix undue use of initial sleeptime We should not use the initial sleeptime for any key press that does not switch to a different screen, as that introduces an unaesthetic flicker due to two updates in quick succession. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 81468d73b6eb0ed251e7c77f2cc44c0f4edb4d36) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 3 --- 1 file changed, 3 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 4fda0b4c7354..0e0541dc184b 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1142,14 +1142,12 @@ class Tui(object): if char == 'x': self.refresh_header() self.update_drilldown() - sleeptime = DELAY_INITIAL if char == 'q': break if char == 'c': self.stats.fields_filter = DEFAULT_REGEX self.refresh_header(0) self.update_pid(0) - sleeptime = DELAY_INITIAL if char == 'f': self.show_filter_selection() sleeptime = DELAY_INITIAL @@ -1162,7 +1160,6 @@ class Tui(object): if char == 'r': self.refresh_header() self.stats.reset() - sleeptime = DELAY_INITIAL except KeyboardInterrupt: break except curses.error: -- 2.12.3 ++++++ 0003-tools-kvm_stat-remove-unnecessary-header-redraws.patch ++++++ >From fa036ccee3f478345cdddb27967f801fb375b5dc Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:28 +0200 Subject: [PATCH 03/43] tools/kvm_stat: remove unnecessary header redraws Certain interactive commands will not modify any information displayed in the header, hence we can skip them. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 2da9d4aaa7348fc13374d7398c9c7027b0a9e2cb) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 -- 1 file changed, 2 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 0e0541dc184b..d600d2c304e3 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1140,7 +1140,6 @@ class Tui(object): try: char = self.screen.getkey() if char == 'x': - self.refresh_header() self.update_drilldown() if char == 'q': break @@ -1158,7 +1157,6 @@ class Tui(object): self.show_vm_selection_by_pid() sleeptime = DELAY_INITIAL if char == 'r': - self.refresh_header() self.stats.reset() except KeyboardInterrupt: break -- 2.12.3 ++++++ 0004-tools-kvm_stat-simplify-line-print-logic.patch ++++++ >From 78c117092b5a4a9d3eb8c3127774caed2569eb14 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:29 +0200 Subject: [PATCH 04/43] tools/kvm_stat: simplify line print logic Simplify line print logic for header and data lines in interactive mode as previously suggested by Radim. While at it, add a space between the first two columns to avoid the total bleeding into the event name. Furthermore, for column 'Current', differentiate between no events being reported (empty 'Current' column) vs the case where events were reported but the average was rounded down to zero ('0' in 'Current column), for the folks who appreciate the difference. Finally: Only skip events which were not reported at all yet, instead of events that don't have a value in the current interval. Considered using constants for the field widths in the format strings. However, that would make things a bit more complicated, and considering that there are only two places where output happens, I figured it isn't worth the trouble. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 5a7d11f8dc59ddb36e89dca42a2526ea25914def) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 26 +++++++------------------- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index d600d2c304e3..3d40809a3ff2 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -887,8 +887,6 @@ class Stats(object): self.values[key] = (newval, newdelta) return self.values -LABEL_WIDTH = 40 -NUMBER_WIDTH = 10 DELAY_INITIAL = 0.25 DELAY_REGULAR = 3.0 MAX_GUEST_NAME_LEN = 48 @@ -970,13 +968,8 @@ class Tui(object): if len(regex) > MAX_REGEX_LEN: regex = regex[:MAX_REGEX_LEN] + '...' self.screen.addstr(1, 17, 'regex filter: {0}'.format(regex)) - self.screen.addstr(2, 1, 'Event') - self.screen.addstr(2, 1 + LABEL_WIDTH + NUMBER_WIDTH - - len('Total'), 'Total') - self.screen.addstr(2, 1 + LABEL_WIDTH + NUMBER_WIDTH + 7 - - len('%Total'), '%Total') - self.screen.addstr(2, 1 + LABEL_WIDTH + NUMBER_WIDTH + 7 + 8 - - len('Current'), 'Current') + self.screen.addstr(2, 1, '%-40s %10s%7s %7s' % + ('Event', 'Total', '%Total', 'Current')) self.screen.addstr(4, 1, 'Collecting data...') self.screen.refresh() @@ -1001,16 +994,11 @@ class Tui(object): values = stats[key] if not values[0] and not values[1]: break - col = 1 - self.screen.addstr(row, col, key) - col += LABEL_WIDTH - self.screen.addstr(row, col, '%10d' % (values[0],)) - col += NUMBER_WIDTH - self.screen.addstr(row, col, '%7.1f' % (values[0] * 100 / total,)) - col += 7 - if values[1] is not None: - self.screen.addstr(row, col, '%8d' % - round(values[1] / sleeptime)) + if values[0] is not None: + cur = int(round(values[1] / sleeptime)) if values[1] else '' + self.screen.addstr(row, 1, '%-40s %10d%7.1f %7s' % + (key, values[0], values[0] * 100 / total, + cur)) row += 1 self.screen.refresh() -- 2.12.3 ++++++ 0005-tools-kvm_stat-remove-extra-statement.patch ++++++ >From 7de35a682eebbfae33cae3d9fd77a74f850c8915 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:31 +0200 Subject: [PATCH 05/43] tools/kvm_stat: remove extra statement Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 5e3823a49c50d70cc6b92808c262a43cf3505f3c) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 1 - 1 file changed, 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 3d40809a3ff2..11aea9fb3ce0 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1069,7 +1069,6 @@ class Tui(object): except ValueError: msg = '"' + str(pid) + '": Not a valid pid' - continue def show_vm_selection_by_guest_name(self): """Draws guest selection mask. -- 2.12.3 ++++++ 0006-tools-kvm_stat-simplify-initializers.patch ++++++ >From 3dab8963d7ea41fc632f49f4f1df87b0afa22dcb Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:32 +0200 Subject: [PATCH 06/43] tools/kvm_stat: simplify initializers Simplify a couple of initialization routines: * TracepointProvider and DebugfsProvider: Pass pid into __init__() instead of switching to the requested value in an extra call after initializing to the default first. * Pass a single options object into Stats.__init__(), delaying options evaluation accordingly, instead of evaluating options first and passing several parts of the options object to Stats.__init__() individually. * Eliminate Stats.update_provider_pid(), since this 2-line function is now used in a single place only. * Remove extra call to update_drilldown() in Tui.__init__() by getting the value of options.fields right initially when parsing options. * Simplify get_providers() logic. * Avoid duplicate fields initialization by handling it once in the providers' __init__() methods. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit c469117df05955901d2950b6130770e526b1dbf4) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 74 ++++++++++++++++++++++----------------------- 1 file changed, 36 insertions(+), 38 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 11aea9fb3ce0..888d9114a04c 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -295,6 +295,13 @@ class ArchS390(Arch): ARCH = Arch.get_arch() +def is_field_wanted(fields_filter, field): + """Indicate whether field is valid according to fields_filter.""" + if not fields_filter: + return True + return re.match(fields_filter, field) is not None + + def walkdir(path): """Returns os.walk() data for specified directory. @@ -581,11 +588,11 @@ class TracepointProvider(object): Manages the events/groups from which it acquires its data. """ - def __init__(self): + def __init__(self, pid, fields_filter): self.group_leaders = [] self.filters = get_filters() - self._fields = self.get_available_fields() - self._pid = 0 + self.update_fields(fields_filter) + self.pid = pid def get_available_fields(self): """Returns a list of available event's of format 'event name(filter @@ -613,6 +620,11 @@ class TracepointProvider(object): fields += extra return fields + def update_fields(self, fields_filter): + """Refresh fields, applying fields_filter""" + self._fields = [field for field in self.get_available_fields() + if is_field_wanted(fields_filter, field)] + def setup_traces(self): """Creates all event and group objects needed to be able to retrieve data.""" @@ -726,13 +738,12 @@ class TracepointProvider(object): class DebugfsProvider(object): """Provides data from the files that KVM creates in the kvm debugfs folder.""" - def __init__(self): - self._fields = self.get_available_fields() + def __init__(self, pid, fields_filter): + self.update_fields(fields_filter) self._baseline = {} - self._pid = 0 self.do_read = True self.paths = [] - self.reset() + self.pid = pid def get_available_fields(self): """"Returns a list of available fields. @@ -742,6 +753,11 @@ class DebugfsProvider(object): """ return walkdir(PATH_DEBUGFS_KVM)[2] + def update_fields(self, fields_filter): + """Refresh fields, applying fields_filter""" + self._fields = [field for field in self.get_available_fields() + if is_field_wanted(fields_filter, field)] + @property def fields(self): return self._fields @@ -757,9 +773,8 @@ class DebugfsProvider(object): @pid.setter def pid(self, pid): + self._pid = pid if pid != 0: - self._pid = pid - vms = walkdir(PATH_DEBUGFS_KVM)[1] if len(vms) == 0: self.do_read = False @@ -821,33 +836,19 @@ class Stats(object): provider data. """ - def __init__(self, providers, pid, fields=None): - self.providers = providers - self._pid_filter = pid - self._fields_filter = fields + def __init__(self, options): + self.providers = get_providers(options) + self._pid_filter = options.pid + self._fields_filter = options.fields self.values = {} - self.update_provider_pid() - self.update_provider_filters() def update_provider_filters(self): """Propagates fields filters to providers.""" - def wanted(key): - if not self._fields_filter: - return True - return re.match(self._fields_filter, key) is not None - # As we reset the counters when updating the fields we can # also clear the cache of old values. self.values = {} for provider in self.providers: - provider_fields = [key for key in provider.get_available_fields() - if wanted(key)] - provider.fields = provider_fields - - def update_provider_pid(self): - """Propagates pid filters to providers.""" - for provider in self.providers: - provider.pid = self._pid_filter + provider.update_fields(self._fields_filter) def reset(self): self.values = {} @@ -873,7 +874,8 @@ class Stats(object): if pid != self._pid_filter: self._pid_filter = pid self.values = {} - self.update_provider_pid() + for provider in self.providers: + provider.pid = self._pid_filter def get(self): """Returns a dict with field -> (value, delta to last value) of all @@ -899,7 +901,6 @@ class Tui(object): def __init__(self, stats): self.stats = stats self.screen = None - self.update_drilldown() def __enter__(self): """Initialises curses for later use. Based on curses.wrapper @@ -1273,7 +1274,7 @@ Press any other key to refresh statistics immediately. ) optparser.add_option('-f', '--fields', action='store', - default=None, + default=DEFAULT_REGEX, dest='fields', help='fields to display (regex)', ) @@ -1300,12 +1301,10 @@ def get_providers(options): """Returns a list of data providers depending on the passed options.""" providers = [] - if options.tracepoints: - providers.append(TracepointProvider()) if options.debugfs: - providers.append(DebugfsProvider()) - if len(providers) == 0: - providers.append(TracepointProvider()) + providers.append(DebugfsProvider(options.pid, options.fields)) + if options.tracepoints or not providers: + providers.append(TracepointProvider(options.pid, options.fields)) return providers @@ -1350,8 +1349,7 @@ def main(): sys.stderr.write('Did you use a (unsupported) tid instead of a pid?\n') sys.exit('Specified pid does not exist.') - providers = get_providers(options) - stats = Stats(providers, options.pid, fields=options.fields) + stats = Stats(options) if options.log: log(stats) -- 2.12.3 ++++++ 0007-tools-kvm_stat-move-functions-to-corresponding-class.patch ++++++ >From 9a6247e2af22ad1893998822461a7dd3109eac62 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:33 +0200 Subject: [PATCH 07/43] tools/kvm_stat: move functions to corresponding classes Quite a few of the functions are used only in a single class. Moving functions accordingly to improve the overall structure. Furthermore, introduce a base class for the providers, which might also come handy for future extensions. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 099a2dfc674e3333bd4ff5e5b106ccd788aa46d7) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 327 ++++++++++++++++++++++---------------------- 1 file changed, 165 insertions(+), 162 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 888d9114a04c..a725e02d6f88 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -295,121 +295,6 @@ class ArchS390(Arch): ARCH = Arch.get_arch() -def is_field_wanted(fields_filter, field): - """Indicate whether field is valid according to fields_filter.""" - if not fields_filter: - return True - return re.match(fields_filter, field) is not None - - -def walkdir(path): - """Returns os.walk() data for specified directory. - - As it is only a wrapper it returns the same 3-tuple of (dirpath, - dirnames, filenames). - """ - return next(os.walk(path)) - - -def parse_int_list(list_string): - """Returns an int list from a string of comma separated integers and - integer ranges.""" - integers = [] - members = list_string.split(',') - - for member in members: - if '-' not in member: - integers.append(int(member)) - else: - int_range = member.split('-') - integers.extend(range(int(int_range[0]), - int(int_range[1]) + 1)) - - return integers - - -def get_pid_from_gname(gname): - """Fuzzy function to convert guest name to QEMU process pid. - - Returns a list of potential pids, can be empty if no match found. - Throws an exception on processing errors. - - """ - pids = [] - try: - child = subprocess.Popen(['ps', '-A', '--format', 'pid,args'], - stdout=subprocess.PIPE) - except: - raise Exception - for line in child.stdout: - line = line.lstrip().split(' ', 1) - # perform a sanity check before calling the more expensive - # function to possibly extract the guest name - if ' -name ' in line[1] and gname == get_gname_from_pid(line[0]): - pids.append(int(line[0])) - child.stdout.close() - - return pids - - -def get_gname_from_pid(pid): - """Returns the guest name for a QEMU process pid. - - Extracts the guest name from the QEMU comma line by processing the '-name' - option. Will also handle names specified out of sequence. - - """ - name = '' - try: - line = open('/proc/{}/cmdline'.format(pid), 'rb').read().split('\0') - parms = line[line.index('-name') + 1].split(',') - while '' in parms: - # commas are escaped (i.e. ',,'), hence e.g. 'foo,bar' results in - # ['foo', '', 'bar'], which we revert here - idx = parms.index('') - parms[idx - 1] += ',' + parms[idx + 1] - del parms[idx:idx+2] - # the '-name' switch allows for two ways to specify the guest name, - # where the plain name overrides the name specified via 'guest=' - for arg in parms: - if '=' not in arg: - name = arg - break - if arg[:6] == 'guest=': - name = arg[6:] - except (ValueError, IOError, IndexError): - pass - - return name - - -def get_online_cpus(): - """Returns a list of cpu id integers.""" - with open('/sys/devices/system/cpu/online') as cpu_list: - cpu_string = cpu_list.readline() - return parse_int_list(cpu_string) - - -def get_filters(): - """Returns a dict of trace events, their filter ids and - the values that can be filtered. - - Trace events can be filtered for special values by setting a - filter string via an ioctl. The string normally has the format - identifier==value. For each filter a new event will be created, to - be able to distinguish the events. - - """ - filters = {} - filters['kvm_userspace_exit'] = ('reason', USERSPACE_EXIT_REASONS) - if ARCH.exit_reasons: - filters['kvm_exit'] = ('exit_reason', ARCH.exit_reasons) - return filters - -libc = ctypes.CDLL('libc.so.6', use_errno=True) -syscall = libc.syscall - - class perf_event_attr(ctypes.Structure): """Struct that holds the necessary data to set up a trace event. @@ -439,25 +324,6 @@ class perf_event_attr(ctypes.Structure): self.read_format = PERF_FORMAT_GROUP -def perf_event_open(attr, pid, cpu, group_fd, flags): - """Wrapper for the sys_perf_evt_open() syscall. - - Used to set up performance events, returns a file descriptor or -1 - on error. - - Attributes are: - - syscall number - - struct perf_event_attr * - - pid or -1 to monitor all pids - - cpu number or -1 to monitor all cpus - - The file descriptor of the group leader or -1 to create a group. - - flags - - """ - return syscall(ARCH.sc_perf_evt_open, ctypes.pointer(attr), - ctypes.c_int(pid), ctypes.c_int(cpu), - ctypes.c_int(group_fd), ctypes.c_long(flags)) - PERF_TYPE_TRACEPOINT = 2 PERF_FORMAT_GROUP = 1 << 3 @@ -502,6 +368,8 @@ class Event(object): """Represents a performance event and manages its life cycle.""" def __init__(self, name, group, trace_cpu, trace_pid, trace_point, trace_filter, trace_set='kvm'): + self.libc = ctypes.CDLL('libc.so.6', use_errno=True) + self.syscall = self.libc.syscall self.name = name self.fd = None self.setup_event(group, trace_cpu, trace_pid, trace_point, @@ -518,6 +386,25 @@ class Event(object): if self.fd: os.close(self.fd) + def perf_event_open(self, attr, pid, cpu, group_fd, flags): + """Wrapper for the sys_perf_evt_open() syscall. + + Used to set up performance events, returns a file descriptor or -1 + on error. + + Attributes are: + - syscall number + - struct perf_event_attr * + - pid or -1 to monitor all pids + - cpu number or -1 to monitor all cpus + - The file descriptor of the group leader or -1 to create a group. + - flags + + """ + return self.syscall(ARCH.sc_perf_evt_open, ctypes.pointer(attr), + ctypes.c_int(pid), ctypes.c_int(cpu), + ctypes.c_int(group_fd), ctypes.c_long(flags)) + def setup_event_attribute(self, trace_set, trace_point): """Returns an initialized ctype perf_event_attr struct.""" @@ -546,8 +433,8 @@ class Event(object): if group.events: group_leader = group.events[0].fd - fd = perf_event_open(event_attr, trace_pid, - trace_cpu, group_leader, 0) + fd = self.perf_event_open(event_attr, trace_pid, + trace_cpu, group_leader, 0) if fd == -1: err = ctypes.get_errno() raise OSError(err, os.strerror(err), @@ -582,7 +469,26 @@ class Event(object): fcntl.ioctl(self.fd, ARCH.ioctl_numbers['RESET'], 0) -class TracepointProvider(object): +class Provider(object): + """Encapsulates functionalities used by all providers.""" + @staticmethod + def is_field_wanted(fields_filter, field): + """Indicate whether field is valid according to fields_filter.""" + if not fields_filter: + return True + return re.match(fields_filter, field) is not None + + @staticmethod + def walkdir(path): + """Returns os.walk() data for specified directory. + + As it is only a wrapper it returns the same 3-tuple of (dirpath, + dirnames, filenames). + """ + return next(os.walk(path)) + + +class TracepointProvider(Provider): """Data provider for the stats class. Manages the events/groups from which it acquires its data. @@ -590,10 +496,27 @@ class TracepointProvider(object): """ def __init__(self, pid, fields_filter): self.group_leaders = [] - self.filters = get_filters() + self.filters = self.get_filters() self.update_fields(fields_filter) self.pid = pid + @staticmethod + def get_filters(): + """Returns a dict of trace events, their filter ids and + the values that can be filtered. + + Trace events can be filtered for special values by setting a + filter string via an ioctl. The string normally has the format + identifier==value. For each filter a new event will be created, to + be able to distinguish the events. + + """ + filters = {} + filters['kvm_userspace_exit'] = ('reason', USERSPACE_EXIT_REASONS) + if ARCH.exit_reasons: + filters['kvm_exit'] = ('exit_reason', ARCH.exit_reasons) + return filters + def get_available_fields(self): """Returns a list of available event's of format 'event name(filter name)'. @@ -610,7 +533,7 @@ class TracepointProvider(object): """ path = os.path.join(PATH_DEBUGFS_TRACING, 'events', 'kvm') - fields = walkdir(path)[1] + fields = self.walkdir(path)[1] extra = [] for field in fields: if field in self.filters: @@ -623,7 +546,30 @@ class TracepointProvider(object): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" self._fields = [field for field in self.get_available_fields() - if is_field_wanted(fields_filter, field)] + if self.is_field_wanted(fields_filter, field)] + + @staticmethod + def get_online_cpus(): + """Returns a list of cpu id integers.""" + def parse_int_list(list_string): + """Returns an int list from a string of comma separated integers and + integer ranges.""" + integers = [] + members = list_string.split(',') + + for member in members: + if '-' not in member: + integers.append(int(member)) + else: + int_range = member.split('-') + integers.extend(range(int(int_range[0]), + int(int_range[1]) + 1)) + + return integers + + with open('/sys/devices/system/cpu/online') as cpu_list: + cpu_string = cpu_list.readline() + return parse_int_list(cpu_string) def setup_traces(self): """Creates all event and group objects needed to be able to retrieve @@ -633,9 +579,9 @@ class TracepointProvider(object): # Fetch list of all threads of the monitored pid, as qemu # starts a thread for each vcpu. path = os.path.join('/proc', str(self._pid), 'task') - groupids = walkdir(path)[1] + groupids = self.walkdir(path)[1] else: - groupids = get_online_cpus() + groupids = self.get_online_cpus() # The constant is needed as a buffer for python libs, std # streams and other files that the script opens. @@ -735,7 +681,7 @@ class TracepointProvider(object): event.reset() -class DebugfsProvider(object): +class DebugfsProvider(Provider): """Provides data from the files that KVM creates in the kvm debugfs folder.""" def __init__(self, pid, fields_filter): @@ -751,12 +697,12 @@ class DebugfsProvider(object): The fields are all available KVM debugfs files """ - return walkdir(PATH_DEBUGFS_KVM)[2] + return self.walkdir(PATH_DEBUGFS_KVM)[2] def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" self._fields = [field for field in self.get_available_fields() - if is_field_wanted(fields_filter, field)] + if self.is_field_wanted(fields_filter, field)] @property def fields(self): @@ -775,7 +721,7 @@ class DebugfsProvider(object): def pid(self, pid): self._pid = pid if pid != 0: - vms = walkdir(PATH_DEBUGFS_KVM)[1] + vms = self.walkdir(PATH_DEBUGFS_KVM)[1] if len(vms) == 0: self.do_read = False @@ -837,11 +783,23 @@ class Stats(object): """ def __init__(self, options): - self.providers = get_providers(options) + self.providers = self.get_providers(options) self._pid_filter = options.pid self._fields_filter = options.fields self.values = {} + @staticmethod + def get_providers(options): + """Returns a list of data providers depending on the passed options.""" + providers = [] + + if options.debugfs: + providers.append(DebugfsProvider(options.pid, options.fields)) + if options.tracepoints or not providers: + providers.append(TracepointProvider(options.pid, options.fields)) + + return providers + def update_provider_filters(self): """Propagates fields filters to providers.""" # As we reset the counters when updating the fields we can @@ -936,6 +894,63 @@ class Tui(object): curses.nocbreak() curses.endwin() + @staticmethod + def get_pid_from_gname(gname): + """Fuzzy function to convert guest name to QEMU process pid. + + Returns a list of potential pids, can be empty if no match found. + Throws an exception on processing errors. + + """ + pids = [] + try: + child = subprocess.Popen(['ps', '-A', '--format', 'pid,args'], + stdout=subprocess.PIPE) + except: + raise Exception + for line in child.stdout: + line = line.lstrip().split(' ', 1) + # perform a sanity check before calling the more expensive + # function to possibly extract the guest name + if (' -name ' in line[1] and + gname == self.get_gname_from_pid(line[0])): + pids.append(int(line[0])) + child.stdout.close() + + return pids + + @staticmethod + def get_gname_from_pid(pid): + """Returns the guest name for a QEMU process pid. + + Extracts the guest name from the QEMU comma line by processing the + '-name' option. Will also handle names specified out of sequence. + + """ + name = '' + try: + line = open('/proc/{}/cmdline' + .format(pid), 'rb').read().split('\0') + parms = line[line.index('-name') + 1].split(',') + while '' in parms: + # commas are escaped (i.e. ',,'), hence e.g. 'foo,bar' results + # in # ['foo', '', 'bar'], which we revert here + idx = parms.index('') + parms[idx - 1] += ',' + parms[idx + 1] + del parms[idx:idx+2] + # the '-name' switch allows for two ways to specify the guest name, + # where the plain name overrides the name specified via 'guest=' + for arg in parms: + if '=' not in arg: + name = arg + break + if arg[:6] == 'guest=': + name = arg[6:] + except (ValueError, IOError, IndexError): + pass + + return name + def update_drilldown(self): """Sets or removes a filter that only allows fields without braces.""" if not self.stats.fields_filter: @@ -953,7 +968,7 @@ class Tui(object): if pid is None: pid = self.stats.pid_filter self.screen.erase() - gname = get_gname_from_pid(pid) + gname = self.get_gname_from_pid(pid) if gname: gname = ('({})'.format(gname[:MAX_GUEST_NAME_LEN] + '...' if len(gname) > MAX_GUEST_NAME_LEN @@ -1099,7 +1114,7 @@ class Tui(object): else: pids = [] try: - pids = get_pid_from_gname(gname) + pids = self.get_pid_from_gname(gname) except: msg = '"' + gname + '": Internal error while searching, ' \ 'use pid filter instead' @@ -1232,7 +1247,7 @@ Press any other key to refresh statistics immediately. def cb_guest_to_pid(option, opt, val, parser): try: - pids = get_pid_from_gname(val) + pids = Tui.get_pid_from_gname(val) except: raise optparse.OptionValueError('Error while searching for guest ' '"{}", use "-p" to specify a pid ' @@ -1297,18 +1312,6 @@ Press any other key to refresh statistics immediately. return options -def get_providers(options): - """Returns a list of data providers depending on the passed options.""" - providers = [] - - if options.debugfs: - providers.append(DebugfsProvider(options.pid, options.fields)) - if options.tracepoints or not providers: - providers.append(TracepointProvider(options.pid, options.fields)) - - return providers - - def check_access(options): """Exits if the current user can't access all needed directories.""" if not os.path.exists('/sys/kernel/debug'): -- 2.12.3 ++++++ 0008-tools-kvm_stat-show-cursor-in-selection-screens.patch ++++++ >From a53f6e2cf574e240e400e147f10f92e6076eae4d Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:34 +0200 Subject: [PATCH 08/43] tools/kvm_stat: show cursor in selection screens Show the cursor in the interactive screens to specify pid, filter or guest name as an orientation for the user. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 664751dfc302363b8e9686b492e837bc4f15830e) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index a725e02d6f88..b9fa1ae12346 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1151,13 +1151,19 @@ class Tui(object): self.refresh_header(0) self.update_pid(0) if char == 'f': + curses.curs_set(1) self.show_filter_selection() + curses.curs_set(0) sleeptime = DELAY_INITIAL if char == 'g': + curses.curs_set(1) self.show_vm_selection_by_guest_name() + curses.curs_set(0) sleeptime = DELAY_INITIAL if char == 'p': + curses.curs_set(1) self.show_vm_selection_by_pid() + curses.curs_set(0) sleeptime = DELAY_INITIAL if char == 'r': self.stats.reset() -- 2.12.3 ++++++ 0009-tools-kvm_stat-display-message-indicating-lack-of-ev.patch ++++++ >From 7b0c9240120288190680cb553fef5a6d38a6e9ee Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:35 +0200 Subject: [PATCH 09/43] tools/kvm_stat: display message indicating lack of events Give users some indication on the reason why no data is displayed on the screen yet. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 5725393764a342b6a5420fdd10184984ca08b5f6) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index b9fa1ae12346..112fe57561a5 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1016,6 +1016,8 @@ class Tui(object): (key, values[0], values[0] * 100 / total, cur)) row += 1 + if row == 3: + self.screen.addstr(4, 1, 'No matching events reported yet') self.screen.refresh() def show_filter_selection(self): -- 2.12.3 ++++++ 0010-tools-kvm_stat-make-heading-look-a-bit-more-like-top.patch ++++++ >From df04dff6ef8405a577d685cba5bbf7b597068293 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:36 +0200 Subject: [PATCH 10/43] tools/kvm_stat: make heading look a bit more like 'top' Print header in standout font just like the 'top' command does. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit f6d753102a2469ae4ce08ef3e34d170ec583fb50) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 112fe57561a5..cd4ae362a355 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -985,7 +985,8 @@ class Tui(object): regex = regex[:MAX_REGEX_LEN] + '...' self.screen.addstr(1, 17, 'regex filter: {0}'.format(regex)) self.screen.addstr(2, 1, '%-40s %10s%7s %7s' % - ('Event', 'Total', '%Total', 'Current')) + ('Event', 'Total', '%Total', 'Current'), + curses.A_STANDOUT) self.screen.addstr(4, 1, 'Collecting data...') self.screen.refresh() -- 2.12.3 ++++++ 0011-tools-kvm_stat-rename-Current-column-to-CurAvg-s.patch ++++++ >From 1aa6a57451fde248a460649db1d4598456229374 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:37 +0200 Subject: [PATCH 11/43] tools/kvm_stat: rename 'Current' column to 'CurAvg/s' 'Current' can be misleading as it doesn't tell whether this is the amount of events in the last interval or the current average per second. Note that this necessitates widening the respective column by one more character. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 38e89c37a1e05e6e16af582b980534abda29a4d9) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index cd4ae362a355..028db219b88e 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -984,8 +984,8 @@ class Tui(object): if len(regex) > MAX_REGEX_LEN: regex = regex[:MAX_REGEX_LEN] + '...' self.screen.addstr(1, 17, 'regex filter: {0}'.format(regex)) - self.screen.addstr(2, 1, '%-40s %10s%7s %7s' % - ('Event', 'Total', '%Total', 'Current'), + self.screen.addstr(2, 1, '%-40s %10s%7s %8s' % + ('Event', 'Total', '%Total', 'CurAvg/s'), curses.A_STANDOUT) self.screen.addstr(4, 1, 'Collecting data...') self.screen.refresh() @@ -1013,7 +1013,7 @@ class Tui(object): break if values[0] is not None: cur = int(round(values[1] / sleeptime)) if values[1] else '' - self.screen.addstr(row, 1, '%-40s %10d%7.1f %7s' % + self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key, values[0], values[0] * 100 / total, cur)) row += 1 -- 2.12.3 ++++++ 0012-tools-kvm_stat-add-new-interactive-command-h.patch ++++++ >From c0c59219b3b5d817aabf1b8d9c280a97e3a0ab73 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:38 +0200 Subject: [PATCH 12/43] tools/kvm_stat: add new interactive command 'h' Display interactive commands reference on 'h'. While at it, sort interactive commands alphabetically in various places. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 1fdea7b2893045e5258a13937c3d78c425fd7aa0) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 37 ++++++++++++++++++++++++++++++++----- tools/kvm/kvm_stat/kvm_stat.txt | 2 ++ 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 028db219b88e..ff86585b12d3 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1021,6 +1021,30 @@ class Tui(object): self.screen.addstr(4, 1, 'No matching events reported yet') self.screen.refresh() + def show_help_interactive(self): + """Display help with list of interactive commands""" + msg = (' c clear filter', + ' f filter by regular expression', + ' g filter by guest name', + ' h display interactive commands reference', + ' p filter by PID', + ' q quit', + ' r reset stats', + ' x toggle reporting of stats for individual child trace' + ' events', + 'Any other key refreshes statistics immediately') + curses.cbreak() + self.screen.erase() + self.screen.addstr(0, 0, "Interactive commands reference", + curses.A_BOLD) + self.screen.addstr(2, 0, "Press any key to exit", curses.A_STANDOUT) + row = 4 + for line in msg: + self.screen.addstr(row, 0, line) + row += 1 + self.screen.getkey() + self.refresh_header() + def show_filter_selection(self): """Draws filter selection mask. @@ -1145,10 +1169,6 @@ class Tui(object): sleeptime = DELAY_REGULAR try: char = self.screen.getkey() - if char == 'x': - self.update_drilldown() - if char == 'q': - break if char == 'c': self.stats.fields_filter = DEFAULT_REGEX self.refresh_header(0) @@ -1163,13 +1183,19 @@ class Tui(object): self.show_vm_selection_by_guest_name() curses.curs_set(0) sleeptime = DELAY_INITIAL + if char == 'h': + self.show_help_interactive() if char == 'p': curses.curs_set(1) self.show_vm_selection_by_pid() curses.curs_set(0) sleeptime = DELAY_INITIAL + if char == 'q': + break if char == 'r': self.stats.reset() + if char == 'x': + self.update_drilldown() except KeyboardInterrupt: break except curses.error: @@ -1240,10 +1266,11 @@ Interactive Commands: c clear filter f filter by regular expression g filter by guest name + h display interactive commands reference p filter by PID q quit - x toggle reporting of stats for individual child trace events r reset stats + x toggle reporting of stats for individual child trace events Press any other key to refresh statistics immediately. """ diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index 109431bdc63c..2bad6f22183b 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -35,6 +35,8 @@ INTERACTIVE COMMANDS *g*:: filter by guest name +*h*:: display interactive commands reference + *p*:: filter by PID *q*:: quit -- 2.12.3 ++++++ 0013-tools-kvm_stat-add-new-interactive-command-s.patch ++++++ >From f0df6fe5e8876e55dda4762e273365cd200d27a5 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:39 +0200 Subject: [PATCH 13/43] tools/kvm_stat: add new interactive command 's' Add new command 's' to modify the update interval. Limited to a maximum of 25.5 sec and a minimum of 0.1 sec, since curses cannot handle longer and shorter delays respectively. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 64eefad2cdbf2d7c76e24d0b67e19efdbe1c97a9) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 55 +++++++++++++++++++++++++++++++++++------ tools/kvm/kvm_stat/kvm_stat.txt | 2 ++ 2 files changed, 49 insertions(+), 8 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index ff86585b12d3..8595906c3c60 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -847,8 +847,7 @@ class Stats(object): self.values[key] = (newval, newdelta) return self.values -DELAY_INITIAL = 0.25 -DELAY_REGULAR = 3.0 +DELAY_DEFAULT = 3.0 MAX_GUEST_NAME_LEN = 48 MAX_REGEX_LEN = 44 DEFAULT_REGEX = r'^[^\(]*$' @@ -859,6 +858,8 @@ class Tui(object): def __init__(self, stats): self.stats = stats self.screen = None + self._delay_initial = 0.25 + self._delay_regular = DELAY_DEFAULT def __enter__(self): """Initialises curses for later use. Based on curses.wrapper @@ -1030,6 +1031,7 @@ class Tui(object): ' p filter by PID', ' q quit', ' r reset stats', + ' s set update interval', ' x toggle reporting of stats for individual child trace' ' events', 'Any other key refreshes statistics immediately') @@ -1109,10 +1111,41 @@ class Tui(object): self.refresh_header(pid) self.update_pid(pid) break - except ValueError: msg = '"' + str(pid) + '": Not a valid pid' + def show_set_update_interval(self): + """Draws update interval selection mask.""" + msg = '' + while True: + self.screen.erase() + self.screen.addstr(0, 0, 'Set update interval (defaults to %fs).' % + DELAY_DEFAULT, curses.A_BOLD) + self.screen.addstr(4, 0, msg) + self.screen.addstr(2, 0, 'Change delay from %.1fs to ' % + self._delay_regular) + curses.echo() + val = self.screen.getstr() + curses.noecho() + + try: + if len(val) > 0: + delay = float(val) + if delay < 0.1: + msg = '"' + str(val) + '": Value must be >=0.1' + continue + if delay > 25.5: + msg = '"' + str(val) + '": Value must be <=25.5' + continue + else: + delay = DELAY_DEFAULT + self._delay_regular = delay + break + + except ValueError: + msg = '"' + str(val) + '": Invalid value' + self.refresh_header() + def show_vm_selection_by_guest_name(self): """Draws guest selection mask. @@ -1159,14 +1192,14 @@ class Tui(object): def show_stats(self): """Refreshes the screen and processes user input.""" - sleeptime = DELAY_INITIAL + sleeptime = self._delay_initial self.refresh_header() start = 0.0 # result based on init value never appears on screen while True: self.refresh_body(time.time() - start) curses.halfdelay(int(sleeptime * 10)) start = time.time() - sleeptime = DELAY_REGULAR + sleeptime = self._delay_regular try: char = self.screen.getkey() if char == 'c': @@ -1177,23 +1210,28 @@ class Tui(object): curses.curs_set(1) self.show_filter_selection() curses.curs_set(0) - sleeptime = DELAY_INITIAL + sleeptime = self._delay_initial if char == 'g': curses.curs_set(1) self.show_vm_selection_by_guest_name() curses.curs_set(0) - sleeptime = DELAY_INITIAL + sleeptime = self._delay_initial if char == 'h': self.show_help_interactive() if char == 'p': curses.curs_set(1) self.show_vm_selection_by_pid() curses.curs_set(0) - sleeptime = DELAY_INITIAL + sleeptime = self._delay_initial if char == 'q': break if char == 'r': self.stats.reset() + if char == 's': + curses.curs_set(1) + self.show_set_update_interval() + curses.curs_set(0) + sleeptime = self._delay_initial if char == 'x': self.update_drilldown() except KeyboardInterrupt: @@ -1270,6 +1308,7 @@ Interactive Commands: p filter by PID q quit r reset stats + s set update interval x toggle reporting of stats for individual child trace events Press any other key to refresh statistics immediately. """ diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index 2bad6f22183b..cc019b09e0f5 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -43,6 +43,8 @@ INTERACTIVE COMMANDS *r*:: reset stats +*s*:: set update interval + *x*:: toggle reporting of stats for child trace events Press any other key to refresh statistics immediately. -- 2.12.3 ++++++ 0014-tools-kvm_stat-add-new-interactive-command-o.patch ++++++ >From 09c9bd860de66ff7a09b29de3e13f8cb1ca17924 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:41 +0200 Subject: [PATCH 14/43] tools/kvm_stat: add new interactive command 'o' Add new interactive command 'o' to toggle sorting by 'CurAvg/s' (default) and 'Total' columns. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 6667ae8f395099257afca0963838d2dc50a18da7) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 17 ++++++++++++++++- tools/kvm/kvm_stat/kvm_stat.txt | 2 ++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 8595906c3c60..0f74a0e1307e 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -851,6 +851,7 @@ DELAY_DEFAULT = 3.0 MAX_GUEST_NAME_LEN = 48 MAX_REGEX_LEN = 44 DEFAULT_REGEX = r'^[^\(]*$' +SORT_DEFAULT = 0 class Tui(object): @@ -860,6 +861,7 @@ class Tui(object): self.screen = None self._delay_initial = 0.25 self._delay_regular = DELAY_DEFAULT + self._sorting = SORT_DEFAULT def __enter__(self): """Initialises curses for later use. Based on curses.wrapper @@ -997,14 +999,23 @@ class Tui(object): self.screen.clrtobot() stats = self.stats.get() - def sortkey(x): + def sortCurAvg(x): + # sort by current events if available if stats[x][1]: return (-stats[x][1], -stats[x][0]) else: return (0, -stats[x][0]) + + def sortTotal(x): + # sort by totals + return (0, -stats[x][0]) total = 0. for val in stats.values(): total += val[0] + if self._sorting == SORT_DEFAULT: + sortkey = sortCurAvg + else: + sortkey = sortTotal for key in sorted(stats.keys(), key=sortkey): if row >= self.screen.getmaxyx()[0]: @@ -1028,6 +1039,7 @@ class Tui(object): ' f filter by regular expression', ' g filter by guest name', ' h display interactive commands reference', + ' o toggle sorting order (Total vs CurAvg/s)', ' p filter by PID', ' q quit', ' r reset stats', @@ -1218,6 +1230,8 @@ class Tui(object): sleeptime = self._delay_initial if char == 'h': self.show_help_interactive() + if char == 'o': + self._sorting = not self._sorting if char == 'p': curses.curs_set(1) self.show_vm_selection_by_pid() @@ -1305,6 +1319,7 @@ Interactive Commands: f filter by regular expression g filter by guest name h display interactive commands reference + o toggle sorting order (Total vs CurAvg/s) p filter by PID q quit r reset stats diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index cc019b09e0f5..e24ac464d341 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -37,6 +37,8 @@ INTERACTIVE COMMANDS *h*:: display interactive commands reference +*o*:: toggle sorting order (Total vs CurAvg/s) + *p*:: filter by PID *q*:: quit -- 2.12.3 ++++++ 0015-tools-kvm_stat-display-guest-list-in-pid-guest-selec.patch ++++++ >From 8be25f3d6989b4a74115431589f475e3e65e576b Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Wed, 7 Jun 2017 21:08:43 +0200 Subject: [PATCH 15/43] tools/kvm_stat: display guest list in pid/guest selection screens Display a (possibly inaccurate) list of all running guests. Note that we leave a bit of extra room above the list for potential error messages. Furthermore, we deliberately do not reject pids or guest names that are not in our list, as we cannot rule out that our fuzzy approach might be in error somehow. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 865279c53ca9d88718d974bb014b2c6ce259ac75) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 49 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 0f74a0e1307e..b219bfd9781c 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -897,15 +897,9 @@ class Tui(object): curses.nocbreak() curses.endwin() - @staticmethod - def get_pid_from_gname(gname): - """Fuzzy function to convert guest name to QEMU process pid. - - Returns a list of potential pids, can be empty if no match found. - Throws an exception on processing errors. - - """ - pids = [] + def get_all_gnames(self): + """Returns a list of (pid, gname) tuples of all running guests""" + res = [] try: child = subprocess.Popen(['ps', '-A', '--format', 'pid,args'], stdout=subprocess.PIPE) @@ -915,11 +909,40 @@ class Tui(object): line = line.lstrip().split(' ', 1) # perform a sanity check before calling the more expensive # function to possibly extract the guest name - if (' -name ' in line[1] and - gname == self.get_gname_from_pid(line[0])): - pids.append(int(line[0])) + if ' -name ' in line[1]: + res.append((line[0], self.get_gname_from_pid(line[0]))) child.stdout.close() + return res + + def print_all_gnames(self, row): + """Print a list of all running guests along with their pids.""" + self.screen.addstr(row, 2, '%8s %-60s' % + ('Pid', 'Guest Name (fuzzy list, might be ' + 'inaccurate!)'), + curses.A_UNDERLINE) + row += 1 + try: + for line in self.get_all_gnames(): + self.screen.addstr(row, 2, '%8s %-60s' % (line[0], line[1])) + row += 1 + if row >= self.screen.getmaxyx()[0]: + break + except Exception: + self.screen.addstr(row + 1, 2, 'Not available') + + def get_pid_from_gname(self, gname): + """Fuzzy function to convert guest name to QEMU process pid. + + Returns a list of potential pids, can be empty if no match found. + Throws an exception on processing errors. + + """ + pids = [] + for line in self.get_all_gnames(): + if gname == line[1]: + pids.append(int(line[0])) + return pids @staticmethod @@ -1105,6 +1128,7 @@ class Tui(object): 'This might limit the shown data to the trace ' 'statistics.') self.screen.addstr(5, 0, msg) + self.print_all_gnames(7) curses.echo() self.screen.addstr(3, 0, "Pid [0 or pid]: ") @@ -1174,6 +1198,7 @@ class Tui(object): 'This might limit the shown data to the trace ' 'statistics.') self.screen.addstr(5, 0, msg) + self.print_all_gnames() curses.echo() self.screen.addstr(3, 0, "Guest [ENTER or guest]: ") gname = self.screen.getstr() -- 2.12.3 ++++++ 0016-tools-kvm_stat-fix-error-on-interactive-command-g.patch ++++++ >From 41570aeb99a581dbe69afc3dda742dc4273190bb Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Sun, 25 Jun 2017 21:34:14 +0200 Subject: [PATCH 16/43] tools/kvm_stat: fix error on interactive command 'g' Fix an instance where print_all_gnames() is called without the mandatory argument, resulting in a stack trace. To reproduce, simply press 'g' in interactive mode. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 61f381bb7e1a8e9250aa32b3963a7a5c4b92cbf5) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index b219bfd9781c..d6bc0b1ff15b 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1198,7 +1198,7 @@ class Tui(object): 'This might limit the shown data to the trace ' 'statistics.') self.screen.addstr(5, 0, msg) - self.print_all_gnames() + self.print_all_gnames(7) curses.echo() self.screen.addstr(3, 0, "Guest [ENTER or guest]: ") gname = self.screen.getstr() -- 2.12.3 ++++++ 0017-tools-kvm_stat-add-new-command-line-switch-i.patch ++++++ >From df37b4bc21c01ce3a4a7081449710f8faa903e96 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Sun, 25 Jun 2017 21:34:15 +0200 Subject: [PATCH 17/43] tools/kvm_stat: add new command line switch '-i' It might be handy to display the full history of event stats to compare the current event distribution against any available historic data. Since we have that available for debugfs, we offer a respective command line option to display what's available. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit ab7ef193fab628fc5da6fd4f4672ffd0d1bb53df) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 34 ++++++++++++++++++++++++++++++---- tools/kvm/kvm_stat/kvm_stat.txt | 4 ++++ 2 files changed, 34 insertions(+), 4 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index d6bc0b1ff15b..76a05fc631a2 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -684,12 +684,14 @@ class TracepointProvider(Provider): class DebugfsProvider(Provider): """Provides data from the files that KVM creates in the kvm debugfs folder.""" - def __init__(self, pid, fields_filter): + def __init__(self, pid, fields_filter, include_past): self.update_fields(fields_filter) self._baseline = {} self.do_read = True self.paths = [] self.pid = pid + if include_past: + self.restore() def get_available_fields(self): """"Returns a list of available fields. @@ -733,7 +735,14 @@ class DebugfsProvider(Provider): self.reset() def read(self, reset=0): - """Returns a dict with format:'file name / field -> current value'.""" + """Returns a dict with format:'file name / field -> current value'. + + Parameter 'reset': + 0 plain read + 1 reset field counts to 0 + 2 restore the original field counts + + """ results = {} # If no debugfs filtering support is available, then don't read. @@ -750,8 +759,10 @@ class DebugfsProvider(Provider): for field in self._fields: value = self.read_field(field, path) key = path + field - if reset: + if reset == 1: self._baseline[key] = value + if reset == 2: + self._baseline[key] = 0 if self._baseline.get(key, -1) == -1: self._baseline[key] = value results[field] = (results.get(field, 0) + value - @@ -774,6 +785,11 @@ class DebugfsProvider(Provider): self._baseline = {} self.read(1) + def restore(self): + """Reset field counters""" + self._baseline = {} + self.read(2) + class Stats(object): """Manages the data providers and the data they provide. @@ -794,7 +810,8 @@ class Stats(object): providers = [] if options.debugfs: - providers.append(DebugfsProvider(options.pid, options.fields)) + providers.append(DebugfsProvider(options.pid, options.fields, + options.dbgfs_include_past)) if options.tracepoints or not providers: providers.append(TracepointProvider(options.pid, options.fields)) @@ -1273,6 +1290,8 @@ class Tui(object): sleeptime = self._delay_initial if char == 'x': self.update_drilldown() + # prevents display of current values on next refresh + self.stats.get() except KeyboardInterrupt: break except curses.error: @@ -1384,6 +1403,13 @@ Press any other key to refresh statistics immediately. dest='once', help='run in batch mode for one second', ) + optparser.add_option('-i', '--debugfs-include-past', + action='store_true', + default=False, + dest='dbgfs_include_past', + help='include all available data on past events for ' + 'debugfs', + ) optparser.add_option('-l', '--log', action='store_true', default=False, diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index e24ac464d341..851372d263cc 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -70,6 +70,10 @@ OPTIONS --debugfs:: retrieve statistics from debugfs +-i:: +--debugfs-include-past:: + include all available data on past events for debugfs + -p<pid>:: --pid=<pid>:: limit statistics to one virtual machine (pid) -- 2.12.3 ++++++ 0018-tools-kvm_stat-add-new-interactive-command-b.patch ++++++ >From 46a5dd617ac4619849e340e6fed1719677666144 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Date: Sun, 25 Jun 2017 21:34:16 +0200 Subject: [PATCH 18/43] tools/kvm_stat: add new interactive command 'b' Toggle display total number of events by guest (debugfs only). When switching to display of events by guest, field filters remain active. I.e. the number of events per guest reported considers only events matching the filters. Likewise with pid/guest filtering. Note that when switching to display of events by guest, DebugfsProvider remains to collect data for events as it did before, but the read() method summarizes the values by pid. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 5c1954d25d1b9e857be2a4c77437312075875589) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 87 ++++++++++++++++++++++++++++++++++++----- tools/kvm/kvm_stat/kvm_stat.txt | 2 + 2 files changed, 79 insertions(+), 10 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 76a05fc631a2..b8f0634baf28 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -665,7 +665,7 @@ class TracepointProvider(Provider): self.setup_traces() self.fields = self._fields - def read(self): + def read(self, by_guest=0): """Returns 'event name: current value' for all enabled events.""" ret = defaultdict(int) for group in self.group_leaders: @@ -734,7 +734,7 @@ class DebugfsProvider(Provider): self.do_read = True self.reset() - def read(self, reset=0): + def read(self, reset=0, by_guest=0): """Returns a dict with format:'file name / field -> current value'. Parameter 'reset': @@ -765,8 +765,16 @@ class DebugfsProvider(Provider): self._baseline[key] = 0 if self._baseline.get(key, -1) == -1: self._baseline[key] = value - results[field] = (results.get(field, 0) + value - - self._baseline.get(key, 0)) + increment = (results.get(field, 0) + value - + self._baseline.get(key, 0)) + if by_guest: + pid = key.split('-')[0] + if pid in results: + results[pid] += increment + else: + results[pid] = increment + else: + results[field] = increment return results @@ -852,18 +860,44 @@ class Stats(object): for provider in self.providers: provider.pid = self._pid_filter - def get(self): + def get(self, by_guest=0): """Returns a dict with field -> (value, delta to last value) of all provider data.""" for provider in self.providers: - new = provider.read() - for key in provider.fields: + new = provider.read(by_guest=by_guest) + for key in new if by_guest else provider.fields: oldval = self.values.get(key, (0, 0))[0] newval = new.get(key, 0) newdelta = newval - oldval self.values[key] = (newval, newdelta) return self.values + def toggle_display_guests(self, to_pid): + """Toggle between collection of stats by individual event and by + guest pid + + Events reported by DebugfsProvider change when switching to/from + reading by guest values. Hence we have to remove the excess event + names from self.values. + + """ + if any(isinstance(ins, TracepointProvider) for ins in self.providers): + return 1 + if to_pid: + for provider in self.providers: + if isinstance(provider, DebugfsProvider): + for key in provider.fields: + if key in self.values.keys(): + del self.values[key] + else: + oldvals = self.values.copy() + for key in oldvals: + if key.isdigit(): + del self.values[key] + # Update oldval (see get()) + self.get(to_pid) + return 0 + DELAY_DEFAULT = 3.0 MAX_GUEST_NAME_LEN = 48 MAX_REGEX_LEN = 44 @@ -879,6 +913,7 @@ class Tui(object): self._delay_initial = 0.25 self._delay_regular = DELAY_DEFAULT self._sorting = SORT_DEFAULT + self._display_guests = 0 def __enter__(self): """Initialises curses for later use. Based on curses.wrapper @@ -1027,8 +1062,12 @@ class Tui(object): if len(regex) > MAX_REGEX_LEN: regex = regex[:MAX_REGEX_LEN] + '...' self.screen.addstr(1, 17, 'regex filter: {0}'.format(regex)) + if self._display_guests: + col_name = 'Guest Name' + else: + col_name = 'Event' self.screen.addstr(2, 1, '%-40s %10s%7s %8s' % - ('Event', 'Total', '%Total', 'CurAvg/s'), + (col_name, 'Total', '%Total', 'CurAvg/s'), curses.A_STANDOUT) self.screen.addstr(4, 1, 'Collecting data...') self.screen.refresh() @@ -1037,7 +1076,7 @@ class Tui(object): row = 3 self.screen.move(row, 0) self.screen.clrtobot() - stats = self.stats.get() + stats = self.stats.get(self._display_guests) def sortCurAvg(x): # sort by current events if available @@ -1065,6 +1104,8 @@ class Tui(object): break if values[0] is not None: cur = int(round(values[1] / sleeptime)) if values[1] else '' + if self._display_guests: + key = self.get_gname_from_pid(key) self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key, values[0], values[0] * 100 / total, cur)) @@ -1073,9 +1114,26 @@ class Tui(object): self.screen.addstr(4, 1, 'No matching events reported yet') self.screen.refresh() + def show_msg(self, text): + """Display message centered text and exit on key press""" + hint = 'Press any key to continue' + curses.cbreak() + self.screen.erase() + (x, term_width) = self.screen.getmaxyx() + row = 2 + for line in text: + start = (term_width - len(line)) / 2 + self.screen.addstr(row, start, line) + row += 1 + self.screen.addstr(row + 1, (term_width - len(hint)) / 2, hint, + curses.A_STANDOUT) + self.screen.getkey() + def show_help_interactive(self): """Display help with list of interactive commands""" - msg = (' c clear filter', + msg = (' b toggle events by guests (debugfs only, honors' + ' filters)', + ' c clear filter', ' f filter by regular expression', ' g filter by guest name', ' h display interactive commands reference', @@ -1256,6 +1314,14 @@ class Tui(object): sleeptime = self._delay_regular try: char = self.screen.getkey() + if char == 'b': + self._display_guests = not self._display_guests + if self.stats.toggle_display_guests(self._display_guests): + self.show_msg(['Command not available with tracepoints' + ' enabled', 'Restart with debugfs only ' + '(see option \'-d\') and try again!']) + self._display_guests = not self._display_guests + self.refresh_header() if char == 'c': self.stats.fields_filter = DEFAULT_REGEX self.refresh_header(0) @@ -1359,6 +1425,7 @@ Requirements: the large number of files that are possibly opened. Interactive Commands: + b toggle events by guests (debugfs only, honors filters) c clear filter f filter by regular expression g filter by guest name diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index 851372d263cc..e5cf836be8a1 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -29,6 +29,8 @@ meaning of events. INTERACTIVE COMMANDS -------------------- [horizontal] +*b*:: toggle events by guests (debugfs only, honors filters) + *c*:: clear filter *f*:: filter by regular expression -- 2.12.3 ++++++ 0019-tools-kvm_stat-add-f-help-to-get-the-available-event.patch ++++++ >From 93fca877853b89ccf7243c09b9e1bc271e5a3c8d Mon Sep 17 00:00:00 2001 From: Lin Ma <lma(a)suse.com> Date: Tue, 25 Jul 2017 19:05:54 +0800 Subject: [PATCH 19/43] tools/kvm_stat: add '-f help' to get the available event list Signed-off-by: Lin Ma <lma(a)suse.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 67fbcd62f54d4503e3dc63b68af1c6757b74e050) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index b8f0634baf28..218b7396f0fe 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -474,7 +474,7 @@ class Provider(object): @staticmethod def is_field_wanted(fields_filter, field): """Indicate whether field is valid according to fields_filter.""" - if not fields_filter: + if not fields_filter or fields_filter == "help": return True return re.match(fields_filter, field) is not None @@ -1499,7 +1499,8 @@ Press any other key to refresh statistics immediately. action='store', default=DEFAULT_REGEX, dest='fields', - help='fields to display (regex)', + help='''fields to display (regex) + "-f help" for a list of available events''', ) optparser.add_option('-p', '--pid', action='store', @@ -1562,6 +1563,17 @@ def main(): stats = Stats(options) + if options.fields == "help": + event_list = "\n" + s = stats.get() + for key in s.keys(): + if key.find('(') != -1: + key = key[0:key.find('(')] + if event_list.find('\n' + key + '\n') == -1: + event_list += key + '\n' + sys.stdout.write(event_list) + return "" + if options.log: log(stats) elif not options.once: -- 2.12.3 ++++++ 0020-tools-kvm_stat-Add-Python-3-support-to-kvm_stat.patch ++++++ >From 27ffcf036bda1ca51997197ec0c8977293cfb4c9 Mon Sep 17 00:00:00 2001 From: Jeremy Cline <jeremy(a)jcline.org> Date: Wed, 4 Oct 2017 03:08:11 +0000 Subject: [PATCH 20/43] tools/kvm_stat: Add Python 3 support to kvm_stat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Make kvm_stat support Python 3 by changing the use of "print" to a function rather than a statement, switching from "iteritems" and "iterkeys" (removed in Python 3) to "items" and "keys" respectively, and decoding bytes to strings when dealing with text. With this change, kvm_stat is usable with Python 2.6 and greater. Signed-off-by: Jeremy Cline <jeremy(a)jcline.org> Signed-off-by: Radim Krčmář <rkrcmar(a)redhat.com> (cherry picked from commit 9cc5fbbb8ca2178d94f2eeeb2ce675293a3f8ae2) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 218b7396f0fe..179fe5c44701 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -19,9 +19,11 @@ Three different ways of output formatting are available: The data is sampled from the KVM's debugfs entries and its perf events. """ +from __future__ import print_function import curses import sys +import locale import os import time import optparse @@ -225,6 +227,8 @@ IOCTL_NUMBERS = { 'RESET': 0x00002403, } +ENCODING = locale.getpreferredencoding(False) + class Arch(object): """Encapsulates global architecture specific data. @@ -669,7 +673,7 @@ class TracepointProvider(Provider): """Returns 'event name: current value' for all enabled events.""" ret = defaultdict(int) for group in self.group_leaders: - for name, val in group.read().iteritems(): + for name, val in group.read().items(): if name in self._fields: ret[name] += val return ret @@ -958,7 +962,7 @@ class Tui(object): except: raise Exception for line in child.stdout: - line = line.lstrip().split(' ', 1) + line = line.decode(ENCODING).lstrip().split(' ', 1) # perform a sanity check before calling the more expensive # function to possibly extract the guest name if ' -name ' in line[1]: @@ -1008,7 +1012,7 @@ class Tui(object): name = '' try: line = open('/proc/{}/cmdline' - .format(pid), 'rb').read().split('\0') + .format(pid), 'r').read().split('\0') parms = line[line.index('-name') + 1].split(',') while '' in parms: # commas are escaped (i.e. ',,'), hence e.g. 'foo,bar' results @@ -1173,7 +1177,7 @@ class Tui(object): .format(self.stats.fields_filter)) self.screen.addstr(3, 0, "New regex: ") curses.echo() - regex = self.screen.getstr() + regex = self.screen.getstr().decode(ENCODING) curses.noecho() if len(regex) == 0: self.stats.fields_filter = DEFAULT_REGEX @@ -1207,7 +1211,7 @@ class Tui(object): curses.echo() self.screen.addstr(3, 0, "Pid [0 or pid]: ") - pid = self.screen.getstr() + pid = self.screen.getstr().decode(ENCODING) curses.noecho() try: @@ -1236,7 +1240,7 @@ class Tui(object): self.screen.addstr(2, 0, 'Change delay from %.1fs to ' % self._delay_regular) curses.echo() - val = self.screen.getstr() + val = self.screen.getstr().decode(ENCODING) curses.noecho() try: @@ -1276,7 +1280,7 @@ class Tui(object): self.print_all_gnames(7) curses.echo() self.screen.addstr(3, 0, "Guest [ENTER or guest]: ") - gname = self.screen.getstr() + gname = self.screen.getstr().decode(ENCODING) curses.noecho() if not gname: @@ -1372,25 +1376,25 @@ def batch(stats): s = stats.get() for key in sorted(s.keys()): values = s[key] - print '%-42s%10d%10d' % (key, values[0], values[1]) + print('%-42s%10d%10d' % (key, values[0], values[1])) except KeyboardInterrupt: pass def log(stats): """Prints statistics as reiterating key block, multiple value blocks.""" - keys = sorted(stats.get().iterkeys()) + keys = sorted(stats.get().keys()) def banner(): for k in keys: - print '%s' % k, - print + print(k, end=' ') + print() def statline(): s = stats.get() for k in keys: - print ' %9d' % s[k][1], - print + print(' %9d' % s[k][1], end=' ') + print() line = 0 banner_repeat = 20 while True: -- 2.12.3 ++++++ 0021-tools-kvm_stat-fix-command-line-option-g.patch ++++++ >From 5f82066299cb3b9e41af3556337da30a8717de56 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:19 +0100 Subject: [PATCH 21/43] tools/kvm_stat: fix command line option '-g' Specifying a guest via '-g foo' always results in an error: $ kvm_stat -g foo Usage: kvm_stat [options] kvm_stat: error: Error while searching for guest "foo", use "-p" to specify a pid instead Reason is that Tui.get_pid_from_gname() is not static, as it is supposed to be. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Tested-by: Christian Borntraeger <borntraeger(a)de.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 19e8e54f4309eaa438237aa1973fe40c331903d4) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 179fe5c44701..9241f949af63 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -953,7 +953,8 @@ class Tui(object): curses.nocbreak() curses.endwin() - def get_all_gnames(self): + @staticmethod + def get_all_gnames(): """Returns a list of (pid, gname) tuples of all running guests""" res = [] try: @@ -966,7 +967,7 @@ class Tui(object): # perform a sanity check before calling the more expensive # function to possibly extract the guest name if ' -name ' in line[1]: - res.append((line[0], self.get_gname_from_pid(line[0]))) + res.append((line[0], Tui.get_gname_from_pid(line[0]))) child.stdout.close() return res @@ -987,7 +988,8 @@ class Tui(object): except Exception: self.screen.addstr(row + 1, 2, 'Not available') - def get_pid_from_gname(self, gname): + @staticmethod + def get_pid_from_gname(gname): """Fuzzy function to convert guest name to QEMU process pid. Returns a list of potential pids, can be empty if no match found. @@ -995,7 +997,7 @@ class Tui(object): """ pids = [] - for line in self.get_all_gnames(): + for line in Tui.get_all_gnames(): if gname == line[1]: pids.append(int(line[0])) -- 2.12.3 ++++++ 0022-tools-kvm_stat-fix-drilldown-in-events-by-guests-mod.patch ++++++ >From 51eebdf61a40469611b250143ba8a58a3cb86252 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:20 +0100 Subject: [PATCH 22/43] tools/kvm_stat: fix drilldown in events-by-guests mode When displaying debugfs events listed by guests, an attempt to switch to reporting of stats for individual child trace events results in garbled output. Reason is that when toggling drilldown, the update of the stats doesn't honor when events are displayed by guests, as indicated by Tui._display_guests. To reproduce, run 'kvm_stat -d' and press 'b' followed by 'x'. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit faa06650418bf28d07426fcfdc5213782fb131f6) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 9241f949af63..f0beda705072 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1363,7 +1363,7 @@ class Tui(object): if char == 'x': self.update_drilldown() # prevents display of current values on next refresh - self.stats.get() + self.stats.get(self._display_guests) except KeyboardInterrupt: break except curses.error: -- 2.12.3 ++++++ 0023-tools-kvm_stat-fix-missing-field-update-after-filter.patch ++++++ >From 7349fc2723704fe22e81efcdc5aaaa5731a5b22f Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:21 +0100 Subject: [PATCH 23/43] tools/kvm_stat: fix missing field update after filter change When updating the fields filter, tracepoint events of fields previously not visible were not enabled, as TracepointProvider.update_fields() updated the member variable directly instead of using the setter, which triggers the event enable/disable. To reproduce, run 'kvm_stat -f kvm_exit', press 'c' to remove the filter, and notice that no add'l fields that do not match the regex 'kvm_exit' will appear. This issue was introduced by commit c469117df059 ("tools/kvm_stat: simplify initializers"). Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 67c162b0892ac481e47bef06d9c6231ee993843a) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index f0beda705072..1494113df827 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -549,8 +549,8 @@ class TracepointProvider(Provider): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" - self._fields = [field for field in self.get_available_fields() - if self.is_field_wanted(fields_filter, field)] + self.fields = [field for field in self.get_available_fields() + if self.is_field_wanted(fields_filter, field)] @staticmethod def get_online_cpus(): -- 2.12.3 ++++++ 0024-tools-kvm_stat-fix-extra-handling-of-help-with-field.patch ++++++ >From 17b5b4949b8225ec76780838360f520f3f6abf72 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:22 +0100 Subject: [PATCH 24/43] tools/kvm_stat: fix extra handling of 'help' with fields filter Commit 67fbcd62f54d ("tools/kvm_stat: add '-f help' to get the available event list") added support for '-f help'. However, the extra handling of 'help' will also take effect when 'help' is specified as a regex in interactive mode via 'f'. This results in display of all events while only those matching this regex should be shown. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit b74faa930deb2e37ed5caa0abfc687c8c532e946) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 1494113df827..639f9a40a9e4 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -478,7 +478,7 @@ class Provider(object): @staticmethod def is_field_wanted(fields_filter, field): """Indicate whether field is valid according to fields_filter.""" - if not fields_filter or fields_filter == "help": + if not fields_filter: return True return re.match(fields_filter, field) is not None @@ -1570,6 +1570,7 @@ def main(): stats = Stats(options) if options.fields == "help": + stats.fields_filter = None event_list = "\n" s = stats.get() for key in s.keys(): -- 2.12.3 ++++++ 0025-tools-kvm_stat-add-hint-on-f-help-to-man-page.patch ++++++ >From 4e19e813f328b0ca4031b7f694b4fa21eb8be890 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:24 +0100 Subject: [PATCH 25/43] tools/kvm_stat: add hint on '-f help' to man page The man page update for this new functionality was omitted. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit f3d11b0e8619bbb053d3e13f2271819fb01c1e2a) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index e5cf836be8a1..e9e8e6b8a1be 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -86,7 +86,7 @@ OPTIONS -f<fields>:: --fields=<fields>:: - fields to display (regex) + fields to display (regex), "-f help" for a list of available events -h:: --help:: -- 2.12.3 ++++++ 0026-tools-kvm_stat-fix-child-trace-events-accounting.patch ++++++ >From 3c5eabd99472e3c753e7660e9bde38be61c376bc Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:23 +0100 Subject: [PATCH 26/43] tools/kvm_stat: fix child trace events accounting Child trace events were included in calculation of the overall total, which is used for calculation of the percentages of the '%Total' column. However, the parent trace envents' stats summarize the child trace events, hence we'd incorrectly account for them twice, leading to slightly wrong stats. With this fix, we use the correct total. Consequently, the sum of the child trace events' '%Total' column values is identical to the respective value of the respective parent event. However, this also means that the sum of the '%Total' column values will aggregate to more than 100 percent. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit fff8c9eb48aa58259071b5df0e6d4c1c0bc1ba51) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 6 +++--- tools/kvm/kvm_stat/kvm_stat.txt | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 639f9a40a9e4..f2711b0a04f5 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1095,14 +1095,14 @@ class Tui(object): # sort by totals return (0, -stats[x][0]) total = 0. - for val in stats.values(): - total += val[0] + for key in stats.keys(): + if key.find('(') is -1: + total += stats[key][0] if self._sorting == SORT_DEFAULT: sortkey = sortCurAvg else: sortkey = sortTotal for key in sorted(stats.keys(), key=sortkey): - if row >= self.screen.getmaxyx()[0]: break values = stats[key] diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index e9e8e6b8a1be..b5b3810c9e94 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -50,6 +50,8 @@ INTERACTIVE COMMANDS *s*:: set update interval *x*:: toggle reporting of stats for child trace events + :: *Note*: The stats for the parents summarize the respective child trace + events Press any other key to refresh statistics immediately. -- 2.12.3 ++++++ 0027-tools-kvm_stat-handle-invalid-regular-expressions.patch ++++++ >From 34f151004c143d186530fde6e71271891f773700 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:25 +0100 Subject: [PATCH 27/43] tools/kvm_stat: handle invalid regular expressions Passing an invalid regular expression on the command line results in a traceback. Note that interactive specification of invalid regular expressions is not affected To reproduce, run "kvm_stat -f '*'". Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 08e20a6300e106d5feb89c9e47ea479533fec46f) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index f2711b0a04f5..c34a1c511e68 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1524,6 +1524,13 @@ Press any other key to refresh statistics immediately. callback=cb_guest_to_pid, ) (options, _) = optparser.parse_args(sys.argv) + try: + # verify that we were passed a valid regex up front + re.compile(options.fields) + except re.error: + sys.exit('Error: "' + options.fields + '" is not a valid regular ' + 'expression') + return options -- 2.12.3 ++++++ 0028-tools-kvm_stat-suppress-usage-information-on-command.patch ++++++ >From 370dd4a6b7e76bdb1bd815732a78c85305bc297e Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:26 +0100 Subject: [PATCH 28/43] tools/kvm_stat: suppress usage information on command line errors Errors while parsing the '-g' command line argument result in display of usage information prior to the error message. This is a bit confusing, as the command line is syntactically correct. To reproduce, run 'kvm_stat -g' and specify a non-existing or inactive guest. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 822cfe3e4813c8f52199362b0e689fba9459ddc9) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index c34a1c511e68..dcfeb57d54fe 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1456,16 +1456,13 @@ Press any other key to refresh statistics immediately. try: pids = Tui.get_pid_from_gname(val) except: - raise optparse.OptionValueError('Error while searching for guest ' - '"{}", use "-p" to specify a pid ' - 'instead'.format(val)) + sys.exit('Error while searching for guest "{}". Use "-p" to ' + 'specify a pid instead?'.format(val)) if len(pids) == 0: - raise optparse.OptionValueError('No guest by the name "{}" ' - 'found'.format(val)) + sys.exit('Error: No guest by the name "{}" found'.format(val)) if len(pids) > 1: - raise optparse.OptionValueError('Multiple processes found (pids: ' - '{}) - use "-p" to specify a pid ' - 'instead'.format(" ".join(pids))) + sys.exit('Error: Multiple processes found (pids: {}). Use "-p" ' + 'to specify the desired pid'.format(" ".join(pids))) parser.values.pid = pids[0] optparser = optparse.OptionParser(description=description_text, -- 2.12.3 ++++++ 0029-tools-kvm_stat-stop-ignoring-unhandled-arguments.patch ++++++ >From 7c7d471aafaede78a90868683760daf1846c606b Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:27 +0100 Subject: [PATCH 29/43] tools/kvm_stat: stop ignoring unhandled arguments Unhandled arguments, which could easily include typos, are simply ignored. We should be strict to avoid undetected typos. To reproduce start kvm_stat with an extra argument, e.g. 'kvm_stat -d bnuh5ol' and note that this will actually work. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 73fab6ffbd83795e38974bb438e7afce0242c61a) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index dcfeb57d54fe..f613e2beef7b 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1520,7 +1520,9 @@ Press any other key to refresh statistics immediately. help='restrict statistics to guest by name', callback=cb_guest_to_pid, ) - (options, _) = optparser.parse_args(sys.argv) + options, unkn = optparser.parse_args(sys.argv) + if len(unkn) != 1: + sys.exit('Error: Extra argument(s): ' + ' '.join(unkn[1:])) try: # verify that we were passed a valid regex up front re.compile(options.fields) -- 2.12.3 ++++++ 0030-tools-kvm_stat-add-line-for-totals.patch ++++++ >From 4d87728c7b703309d8c4789c0cbf61d63bb1fde4 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 11 Dec 2017 12:25:29 +0100 Subject: [PATCH 30/43] tools/kvm_stat: add line for totals Add a line for the total number of events and current average at the bottom of the body. Note that both values exclude child trace events. I.e. if drilldown is activated via interactive command 'x', only the totals are accounted, or we'd be counting these twice (see previous commit "tools/kvm_stat: fix child trace events accounting"). Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit cf656c76614c6ec5b016233cac29738881c83c08) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index f613e2beef7b..9d55afbd549e 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1102,8 +1102,9 @@ class Tui(object): sortkey = sortCurAvg else: sortkey = sortTotal + tavg = 0 for key in sorted(stats.keys(), key=sortkey): - if row >= self.screen.getmaxyx()[0]: + if row >= self.screen.getmaxyx()[0] - 1: break values = stats[key] if not values[0] and not values[1]: @@ -1115,9 +1116,15 @@ class Tui(object): self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key, values[0], values[0] * 100 / total, cur)) + if cur is not '' and key.find('(') is -1: + tavg += cur row += 1 if row == 3: self.screen.addstr(4, 1, 'No matching events reported yet') + else: + self.screen.addstr(row, 1, '%-40s %10d %8s' % + ('Total', total, tavg if tavg else ''), + curses.A_BOLD) self.screen.refresh() def show_msg(self, text): -- 2.12.3 ++++++ 0031-tools-kvm_stat-sort-f-help-output.patch ++++++ >From ab90bd3ab0e013a9261f6787ec6aa902a3edb236 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 21 Dec 2017 13:03:27 +0100 Subject: [PATCH 31/43] tools/kvm_stat: sort '-f help' output Sort the fields returned by specifying '-f help' on the command line. While at it, simplify the code a bit, indent the output and eliminate an extra blank line at the beginning. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit aa12f594f97efe50223611dbd13ecca4e8dafee6) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 9d55afbd549e..f38c97aa131d 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1582,17 +1582,13 @@ def main(): stats = Stats(options) - if options.fields == "help": + if options.fields == 'help': stats.fields_filter = None - event_list = "\n" - s = stats.get() - for key in s.keys(): - if key.find('(') != -1: - key = key[0:key.find('(')] - if event_list.find('\n' + key + '\n') == -1: - event_list += key + '\n' - sys.stdout.write(event_list) - return "" + event_list = [] + for key in stats.get().keys(): + event_list.append(key.split('(', 1)[0]) + sys.stdout.write(' ' + '\n '.join(sorted(set(event_list))) + '\n') + sys.exit(0) if options.log: log(stats) -- 2.12.3 ++++++ 0032-tools-kvm_stat-simplify-the-sortkey-function.patch ++++++ >From ba093f25fe5e029af6a111624a9b3d73b9d69c0c Mon Sep 17 00:00:00 2001 From: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Date: Tue, 9 Jan 2018 13:27:01 +0100 Subject: [PATCH 32/43] tools/kvm_stat: simplify the sortkey function The 'sortkey' function references a value in its enclosing scope (closure). This is not common practice for a sort key function so let's replace it. Additionally, the function 'sorted' has already a parameter for reversing the result therefore the inversion of the values is unneeded. The check for stats[x][1] is also superfluous as it's ensured that this value is initialized with 0. Signed-off-by: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Tested-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit faa312a543283c717342cd332b5b9247bd305dce) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index f38c97aa131d..1f78d3259a86 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1083,30 +1083,23 @@ class Tui(object): self.screen.move(row, 0) self.screen.clrtobot() stats = self.stats.get(self._display_guests) - - def sortCurAvg(x): - # sort by current events if available - if stats[x][1]: - return (-stats[x][1], -stats[x][0]) - else: - return (0, -stats[x][0]) - - def sortTotal(x): - # sort by totals - return (0, -stats[x][0]) total = 0. for key in stats.keys(): if key.find('(') is -1: total += stats[key][0] if self._sorting == SORT_DEFAULT: - sortkey = sortCurAvg + def sortkey((_k, v)): + # sort by (delta value, overall value) + return (v[1], v[0]) else: - sortkey = sortTotal + def sortkey((_k, v)): + # sort by overall value + return v[0] + tavg = 0 - for key in sorted(stats.keys(), key=sortkey): + for key, values in sorted(stats.items(), key=sortkey, reverse=True): if row >= self.screen.getmaxyx()[0] - 1: break - values = stats[key] if not values[0] and not values[1]: break if values[0] is not None: -- 2.12.3 ++++++ 0033-tools-kvm_stat-use-a-namedtuple-for-storing-the-valu.patch ++++++ >From bca7af24e4bc9bbfc085d95a8fcad6bb1593ae95 Mon Sep 17 00:00:00 2001 From: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Date: Tue, 9 Jan 2018 13:27:02 +0100 Subject: [PATCH 33/43] tools/kvm_stat: use a namedtuple for storing the values Use a namedtuple for storing the values as it allows to access the fields of a tuple via names. This makes the overall code much easier to read and to understand. Access by index is still possible as before. Signed-off-by: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Tested-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 006f1548ac13d67d21865416a0f4e8062df1a85f) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 1f78d3259a86..ee1297226145 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -33,7 +33,7 @@ import resource import struct import re import subprocess -from collections import defaultdict +from collections import defaultdict, namedtuple VMX_EXIT_REASONS = { 'EXCEPTION_NMI': 0, @@ -803,6 +803,9 @@ class DebugfsProvider(Provider): self.read(2) +EventStat = namedtuple('EventStat', ['value', 'delta']) + + class Stats(object): """Manages the data providers and the data they provide. @@ -870,10 +873,10 @@ class Stats(object): for provider in self.providers: new = provider.read(by_guest=by_guest) for key in new if by_guest else provider.fields: - oldval = self.values.get(key, (0, 0))[0] + oldval = self.values.get(key, EventStat(0, 0)).value newval = new.get(key, 0) newdelta = newval - oldval - self.values[key] = (newval, newdelta) + self.values[key] = EventStat(newval, newdelta) return self.values def toggle_display_guests(self, to_pid): @@ -1086,28 +1089,28 @@ class Tui(object): total = 0. for key in stats.keys(): if key.find('(') is -1: - total += stats[key][0] + total += stats[key].value if self._sorting == SORT_DEFAULT: def sortkey((_k, v)): # sort by (delta value, overall value) - return (v[1], v[0]) + return (v.delta, v.value) else: def sortkey((_k, v)): # sort by overall value - return v[0] + return v.value tavg = 0 for key, values in sorted(stats.items(), key=sortkey, reverse=True): if row >= self.screen.getmaxyx()[0] - 1: break - if not values[0] and not values[1]: + if not values.value and not values.delta: break - if values[0] is not None: - cur = int(round(values[1] / sleeptime)) if values[1] else '' + if values.value is not None: + cur = int(round(values.delta / sleeptime)) if values.delta else '' if self._display_guests: key = self.get_gname_from_pid(key) self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % - (key, values[0], values[0] * 100 / total, + (key, values.value, values.value * 100 / total, cur)) if cur is not '' and key.find('(') is -1: tavg += cur @@ -1378,7 +1381,7 @@ def batch(stats): s = stats.get() for key in sorted(s.keys()): values = s[key] - print('%-42s%10d%10d' % (key, values[0], values[1])) + print('%-42s%10d%10d' % (key, values.value, values.delta)) except KeyboardInterrupt: pass @@ -1395,7 +1398,7 @@ def log(stats): def statline(): s = stats.get() for k in keys: - print(' %9d' % s[k][1], end=' ') + print(' %9d' % s[k].delta, end=' ') print() line = 0 banner_repeat = 20 -- 2.12.3 ++++++ 0034-tools-kvm_stat-use-a-more-pythonic-way-to-iterate-ov.patch ++++++ >From 0e014103612e740588db517e705c77d64d456e84 Mon Sep 17 00:00:00 2001 From: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Date: Tue, 9 Jan 2018 13:27:03 +0100 Subject: [PATCH 34/43] tools/kvm_stat: use a more pythonic way to iterate over dictionaries If it's clear that the values of a dictionary will be used then use the '.items()' method. Signed-off-by: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Tested-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> [Include fix for logging mode by Stefan Raspl] Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 0eb578009a1d530a11846d7c4733a5db04730884) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index ee1297226145..a82015f08d75 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1087,9 +1087,10 @@ class Tui(object): self.screen.clrtobot() stats = self.stats.get(self._display_guests) total = 0. - for key in stats.keys(): + for key, values in stats.items(): if key.find('(') is -1: - total += stats[key].value + total += values.value + if self._sorting == SORT_DEFAULT: def sortkey((_k, v)): # sort by (delta value, overall value) @@ -1379,8 +1380,7 @@ def batch(stats): s = stats.get() time.sleep(1) s = stats.get() - for key in sorted(s.keys()): - values = s[key] + for key, values in sorted(s.items()): print('%-42s%10d%10d' % (key, values.value, values.delta)) except KeyboardInterrupt: pass @@ -1391,14 +1391,14 @@ def log(stats): keys = sorted(stats.get().keys()) def banner(): - for k in keys: - print(k, end=' ') + for key in keys: + print(key, end=' ') print() def statline(): s = stats.get() - for k in keys: - print(' %9d' % s[k].delta, end=' ') + for key in keys: + print(' %9d' % s[key].delta, end=' ') print() line = 0 banner_repeat = 20 -- 2.12.3 ++++++ 0035-tools-kvm_stat-avoid-is-for-equality-checks.patch ++++++ >From d2779855de45d1ac8b5152cad6f2d47fc9292317 Mon Sep 17 00:00:00 2001 From: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Date: Tue, 9 Jan 2018 13:27:04 +0100 Subject: [PATCH 35/43] tools/kvm_stat: avoid 'is' for equality checks Use '==' for equality checks and 'is' when comparing identities. An example where '==' and 'is' behave differently: >>> a = 4242 >>> a == 4242 True >>> a is 4242 False Signed-off-by: Marc Hartmayer <mhartmay(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 369d5a85bb782ecf63c5bae9686c7e6104eea991) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index a82015f08d75..6d956824dcb6 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1088,7 +1088,7 @@ class Tui(object): stats = self.stats.get(self._display_guests) total = 0. for key, values in stats.items(): - if key.find('(') is -1: + if key.find('(') == -1: total += values.value if self._sorting == SORT_DEFAULT: @@ -1113,7 +1113,7 @@ class Tui(object): self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key, values.value, values.value * 100 / total, cur)) - if cur is not '' and key.find('(') is -1: + if cur != '' and key.find('(') == -1: tavg += cur row += 1 if row == 3: -- 2.12.3 ++++++ 0036-tools-kvm_stat-fix-crash-when-filtering-out-all-non-.patch ++++++ >From 0731f3872508a8c4a7df9f77eeebd5046bffcbf8 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 5 Feb 2018 13:59:57 +0100 Subject: [PATCH 36/43] tools/kvm_stat: fix crash when filtering out all non-child trace events When we apply a filter that will only leave child trace events, we receive a ZeroDivisionError when calculating the percentages. In that case, provide percentages based on child events only. To reproduce, run 'kvm_stat -f .*[\(].*'. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 3df33a0f34a3883b6696bff8cc8fcda3c7444a62) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 6d956824dcb6..7038766c99f1 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1087,9 +1087,15 @@ class Tui(object): self.screen.clrtobot() stats = self.stats.get(self._display_guests) total = 0. + ctotal = 0. for key, values in stats.items(): if key.find('(') == -1: total += values.value + else: + ctotal += values.value + if total == 0.: + # we don't have any fields, or all non-child events are filtered + total = ctotal if self._sorting == SORT_DEFAULT: def sortkey((_k, v)): -- 2.12.3 ++++++ 0037-tools-kvm_stat-print-error-on-invalid-regex.patch ++++++ >From 429350ff8a1868b03d5c351e29ecbaa277f95454 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Mon, 5 Feb 2018 13:59:58 +0100 Subject: [PATCH 37/43] tools/kvm_stat: print error on invalid regex Entering an invalid regular expression did not produce any indication of an error so far. To reproduce, press 'f' and enter 'foo(' (with an unescaped bracket). Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 1cd8bfb1ed9962be6d80d5020508922aa93653ac) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 7038766c99f1..cb7f485c2e70 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1179,6 +1179,7 @@ class Tui(object): Asks for a valid regex and sets the fields filter accordingly. """ + msg = '' while True: self.screen.erase() self.screen.addstr(0, 0, @@ -1187,6 +1188,7 @@ class Tui(object): self.screen.addstr(2, 0, "Current regex: {0}" .format(self.stats.fields_filter)) + self.screen.addstr(5, 0, msg) self.screen.addstr(3, 0, "New regex: ") curses.echo() regex = self.screen.getstr().decode(ENCODING) @@ -1201,6 +1203,7 @@ class Tui(object): self.refresh_header() return except re.error: + msg = '"' + regex + '": Not a valid regular expression' continue def show_vm_selection_by_pid(self): -- 2.12.3 ++++++ 0038-tools-kvm_stat-fix-debugfs-handling.patch ++++++ >From d03e510a1217edce2fdc690c88ba988007f67789 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:24 +0100 Subject: [PATCH 38/43] tools/kvm_stat: fix debugfs handling Te checks for debugfs assumed that debugfs is always mounted at /sys/kernel/debug - which is likely, but not guaranteed. This is addressed by checking /proc/mounts for the actual location. Furthermore, when debugfs was mounted, but the kvm module not loaded, a misleading error pointing towards debugfs not present was given. To reproduce, (a) run kvm_stat with debugfs mounted at a place different from /sys/kernel/debug (b) run kvm_stat with debugfs mounted but kvm module not loaded Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 1fd6a708c8438403dee17eb411cf81ffba13cf43) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 40 ++++++++++++++++++++++++++-------------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index cb7f485c2e70..2b97ad8045b4 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -331,9 +331,6 @@ class perf_event_attr(ctypes.Structure): PERF_TYPE_TRACEPOINT = 2 PERF_FORMAT_GROUP = 1 << 3 -PATH_DEBUGFS_TRACING = '/sys/kernel/debug/tracing' -PATH_DEBUGFS_KVM = '/sys/kernel/debug/kvm' - class Group(object): """Represents a perf event group.""" @@ -1547,17 +1544,6 @@ Press any other key to refresh statistics immediately. def check_access(options): """Exits if the current user can't access all needed directories.""" - if not os.path.exists('/sys/kernel/debug'): - sys.stderr.write('Please enable CONFIG_DEBUG_FS in your kernel.') - sys.exit(1) - - if not os.path.exists(PATH_DEBUGFS_KVM): - sys.stderr.write("Please make sure, that debugfs is mounted and " - "readable by the current user:\n" - "('mount -t debugfs debugfs /sys/kernel/debug')\n" - "Also ensure, that the kvm modules are loaded.\n") - sys.exit(1) - if not os.path.exists(PATH_DEBUGFS_TRACING) and (options.tracepoints or not options.debugfs): sys.stderr.write("Please enable CONFIG_TRACING in your kernel " @@ -1575,7 +1561,33 @@ def check_access(options): return options +def assign_globals(): + global PATH_DEBUGFS_KVM + global PATH_DEBUGFS_TRACING + + debugfs = '' + for line in file('/proc/mounts'): + if line.split(' ')[0] == 'debugfs': + debugfs = line.split(' ')[1] + break + if debugfs == '': + sys.stderr.write("Please make sure that CONFIG_DEBUG_FS is enabled in " + "your kernel, mounted and\nreadable by the current " + "user:\n" + "('mount -t debugfs debugfs /sys/kernel/debug')\n") + sys.exit(1) + + PATH_DEBUGFS_KVM = os.path.join(debugfs, 'kvm') + PATH_DEBUGFS_TRACING = os.path.join(debugfs, 'tracing') + + if not os.path.exists(PATH_DEBUGFS_KVM): + sys.stderr.write("Please make sure that CONFIG_KVM is enabled in " + "your kernel and that the modules are loaded.\n") + sys.exit(1) + + def main(): + assign_globals() options = get_options() options = check_access(options) -- 2.12.3 ++++++ 0039-tools-kvm_stat-mark-private-methods-as-such.patch ++++++ >From df198e3bb88053964f3b051325c769d65760bfeb Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:26 +0100 Subject: [PATCH 39/43] tools/kvm_stat: mark private methods as such Helps quite a bit reading the code when it's obvious when a method is intended for internal use only. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit c0e8c21eae616ed8703c1b4b01046a1578ee875c) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 132 ++++++++++++++++++++++---------------------- 1 file changed, 66 insertions(+), 66 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 2b97ad8045b4..e08fe887764f 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -373,8 +373,8 @@ class Event(object): self.syscall = self.libc.syscall self.name = name self.fd = None - self.setup_event(group, trace_cpu, trace_pid, trace_point, - trace_filter, trace_set) + self._setup_event(group, trace_cpu, trace_pid, trace_point, + trace_filter, trace_set) def __del__(self): """Closes the event's file descriptor. @@ -387,7 +387,7 @@ class Event(object): if self.fd: os.close(self.fd) - def perf_event_open(self, attr, pid, cpu, group_fd, flags): + def _perf_event_open(self, attr, pid, cpu, group_fd, flags): """Wrapper for the sys_perf_evt_open() syscall. Used to set up performance events, returns a file descriptor or -1 @@ -406,7 +406,7 @@ class Event(object): ctypes.c_int(pid), ctypes.c_int(cpu), ctypes.c_int(group_fd), ctypes.c_long(flags)) - def setup_event_attribute(self, trace_set, trace_point): + def _setup_event_attribute(self, trace_set, trace_point): """Returns an initialized ctype perf_event_attr struct.""" id_path = os.path.join(PATH_DEBUGFS_TRACING, 'events', trace_set, @@ -416,8 +416,8 @@ class Event(object): event_attr.config = int(open(id_path).read()) return event_attr - def setup_event(self, group, trace_cpu, trace_pid, trace_point, - trace_filter, trace_set): + def _setup_event(self, group, trace_cpu, trace_pid, trace_point, + trace_filter, trace_set): """Sets up the perf event in Linux. Issues the syscall to register the event in the kernel and @@ -425,7 +425,7 @@ class Event(object): """ - event_attr = self.setup_event_attribute(trace_set, trace_point) + event_attr = self._setup_event_attribute(trace_set, trace_point) # First event will be group leader. group_leader = -1 @@ -434,8 +434,8 @@ class Event(object): if group.events: group_leader = group.events[0].fd - fd = self.perf_event_open(event_attr, trace_pid, - trace_cpu, group_leader, 0) + fd = self._perf_event_open(event_attr, trace_pid, + trace_cpu, group_leader, 0) if fd == -1: err = ctypes.get_errno() raise OSError(err, os.strerror(err), @@ -497,12 +497,12 @@ class TracepointProvider(Provider): """ def __init__(self, pid, fields_filter): self.group_leaders = [] - self.filters = self.get_filters() + self.filters = self._get_filters() self.update_fields(fields_filter) self.pid = pid @staticmethod - def get_filters(): + def _get_filters(): """Returns a dict of trace events, their filter ids and the values that can be filtered. @@ -518,7 +518,7 @@ class TracepointProvider(Provider): filters['kvm_exit'] = ('exit_reason', ARCH.exit_reasons) return filters - def get_available_fields(self): + def _get_available_fields(self): """Returns a list of available event's of format 'event name(filter name)'. @@ -546,11 +546,11 @@ class TracepointProvider(Provider): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" - self.fields = [field for field in self.get_available_fields() + self.fields = [field for field in self._get_available_fields() if self.is_field_wanted(fields_filter, field)] @staticmethod - def get_online_cpus(): + def _get_online_cpus(): """Returns a list of cpu id integers.""" def parse_int_list(list_string): """Returns an int list from a string of comma separated integers and @@ -572,17 +572,17 @@ class TracepointProvider(Provider): cpu_string = cpu_list.readline() return parse_int_list(cpu_string) - def setup_traces(self): + def _setup_traces(self): """Creates all event and group objects needed to be able to retrieve data.""" - fields = self.get_available_fields() + fields = self._get_available_fields() if self._pid > 0: # Fetch list of all threads of the monitored pid, as qemu # starts a thread for each vcpu. path = os.path.join('/proc', str(self._pid), 'task') groupids = self.walkdir(path)[1] else: - groupids = self.get_online_cpus() + groupids = self._get_online_cpus() # The constant is needed as a buffer for python libs, std # streams and other files that the script opens. @@ -663,7 +663,7 @@ class TracepointProvider(Provider): # The garbage collector will get rid of all Event/Group # objects and open files after removing the references. self.group_leaders = [] - self.setup_traces() + self._setup_traces() self.fields = self._fields def read(self, by_guest=0): @@ -692,9 +692,9 @@ class DebugfsProvider(Provider): self.paths = [] self.pid = pid if include_past: - self.restore() + self._restore() - def get_available_fields(self): + def _get_available_fields(self): """"Returns a list of available fields. The fields are all available KVM debugfs files @@ -704,7 +704,7 @@ class DebugfsProvider(Provider): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" - self._fields = [field for field in self.get_available_fields() + self._fields = [field for field in self._get_available_fields() if self.is_field_wanted(fields_filter, field)] @property @@ -758,7 +758,7 @@ class DebugfsProvider(Provider): paths.append(dir) for path in paths: for field in self._fields: - value = self.read_field(field, path) + value = self._read_field(field, path) key = path + field if reset == 1: self._baseline[key] = value @@ -779,7 +779,7 @@ class DebugfsProvider(Provider): return results - def read_field(self, field, path): + def _read_field(self, field, path): """Returns the value of a single field from a specific VM.""" try: return int(open(os.path.join(PATH_DEBUGFS_KVM, @@ -794,7 +794,7 @@ class DebugfsProvider(Provider): self._baseline = {} self.read(1) - def restore(self): + def _restore(self): """Reset field counters""" self._baseline = {} self.read(2) @@ -811,13 +811,12 @@ class Stats(object): """ def __init__(self, options): - self.providers = self.get_providers(options) + self.providers = self._get_providers(options) self._pid_filter = options.pid self._fields_filter = options.fields self.values = {} - @staticmethod - def get_providers(options): + def _get_providers(self, options): """Returns a list of data providers depending on the passed options.""" providers = [] @@ -829,7 +828,7 @@ class Stats(object): return providers - def update_provider_filters(self): + def _update_provider_filters(self): """Propagates fields filters to providers.""" # As we reset the counters when updating the fields we can # also clear the cache of old values. @@ -850,7 +849,7 @@ class Stats(object): def fields_filter(self, fields_filter): if fields_filter != self._fields_filter: self._fields_filter = fields_filter - self.update_provider_filters() + self._update_provider_filters() @property def pid_filter(self): @@ -972,7 +971,7 @@ class Tui(object): return res - def print_all_gnames(self, row): + def _print_all_gnames(self, row): """Print a list of all running guests along with their pids.""" self.screen.addstr(row, 2, '%8s %-60s' % ('Pid', 'Guest Name (fuzzy list, might be ' @@ -1035,7 +1034,7 @@ class Tui(object): return name - def update_drilldown(self): + def _update_drilldown(self): """Sets or removes a filter that only allows fields without braces.""" if not self.stats.fields_filter: self.stats.fields_filter = DEFAULT_REGEX @@ -1043,11 +1042,11 @@ class Tui(object): elif self.stats.fields_filter == DEFAULT_REGEX: self.stats.fields_filter = None - def update_pid(self, pid): + def _update_pid(self, pid): """Propagates pid selection to stats object.""" self.stats.pid_filter = pid - def refresh_header(self, pid=None): + def _refresh_header(self, pid=None): """Refreshes the header.""" if pid is None: pid = self.stats.pid_filter @@ -1078,7 +1077,7 @@ class Tui(object): self.screen.addstr(4, 1, 'Collecting data...') self.screen.refresh() - def refresh_body(self, sleeptime): + def _refresh_body(self, sleeptime): row = 3 self.screen.move(row, 0) self.screen.clrtobot() @@ -1127,7 +1126,7 @@ class Tui(object): curses.A_BOLD) self.screen.refresh() - def show_msg(self, text): + def _show_msg(self, text): """Display message centered text and exit on key press""" hint = 'Press any key to continue' curses.cbreak() @@ -1142,7 +1141,7 @@ class Tui(object): curses.A_STANDOUT) self.screen.getkey() - def show_help_interactive(self): + def _show_help_interactive(self): """Display help with list of interactive commands""" msg = (' b toggle events by guests (debugfs only, honors' ' filters)', @@ -1168,9 +1167,9 @@ class Tui(object): self.screen.addstr(row, 0, line) row += 1 self.screen.getkey() - self.refresh_header() + self._refresh_header() - def show_filter_selection(self): + def _show_filter_selection(self): """Draws filter selection mask. Asks for a valid regex and sets the fields filter accordingly. @@ -1192,18 +1191,18 @@ class Tui(object): curses.noecho() if len(regex) == 0: self.stats.fields_filter = DEFAULT_REGEX - self.refresh_header() + self._refresh_header() return try: re.compile(regex) self.stats.fields_filter = regex - self.refresh_header() + self._refresh_header() return except re.error: msg = '"' + regex + '": Not a valid regular expression' continue - def show_vm_selection_by_pid(self): + def _show_vm_selection_by_pid(self): """Draws PID selection mask. Asks for a pid until a valid pid or 0 has been entered. @@ -1219,7 +1218,7 @@ class Tui(object): 'This might limit the shown data to the trace ' 'statistics.') self.screen.addstr(5, 0, msg) - self.print_all_gnames(7) + self._print_all_gnames(7) curses.echo() self.screen.addstr(3, 0, "Pid [0 or pid]: ") @@ -1235,13 +1234,13 @@ class Tui(object): continue else: pid = 0 - self.refresh_header(pid) - self.update_pid(pid) + self._refresh_header(pid) + self._update_pid(pid) break except ValueError: msg = '"' + str(pid) + '": Not a valid pid' - def show_set_update_interval(self): + def _show_set_update_interval(self): """Draws update interval selection mask.""" msg = '' while True: @@ -1271,9 +1270,9 @@ class Tui(object): except ValueError: msg = '"' + str(val) + '": Invalid value' - self.refresh_header() + self._refresh_header() - def show_vm_selection_by_guest_name(self): + def _show_vm_selection_by_guest_name(self): """Draws guest selection mask. Asks for a guest name until a valid guest name or '' is entered. @@ -1289,15 +1288,15 @@ class Tui(object): 'This might limit the shown data to the trace ' 'statistics.') self.screen.addstr(5, 0, msg) - self.print_all_gnames(7) + self._print_all_gnames(7) curses.echo() self.screen.addstr(3, 0, "Guest [ENTER or guest]: ") gname = self.screen.getstr().decode(ENCODING) curses.noecho() if not gname: - self.refresh_header(0) - self.update_pid(0) + self._refresh_header(0) + self._update_pid(0) break else: pids = [] @@ -1314,17 +1313,17 @@ class Tui(object): msg = '"' + gname + '": Multiple matches found, use pid ' \ 'filter instead' continue - self.refresh_header(pids[0]) - self.update_pid(pids[0]) + self._refresh_header(pids[0]) + self._update_pid(pids[0]) break def show_stats(self): """Refreshes the screen and processes user input.""" sleeptime = self._delay_initial - self.refresh_header() + self._refresh_header() start = 0.0 # result based on init value never appears on screen while True: - self.refresh_body(time.time() - start) + self._refresh_body(time.time() - start) curses.halfdelay(int(sleeptime * 10)) start = time.time() sleeptime = self._delay_regular @@ -1333,32 +1332,33 @@ class Tui(object): if char == 'b': self._display_guests = not self._display_guests if self.stats.toggle_display_guests(self._display_guests): - self.show_msg(['Command not available with tracepoints' - ' enabled', 'Restart with debugfs only ' - '(see option \'-d\') and try again!']) + self._show_msg(['Command not available with ' + 'tracepoints enabled', 'Restart with ' + 'debugfs only (see option \'-d\') and ' + 'try again!']) self._display_guests = not self._display_guests - self.refresh_header() + self._refresh_header() if char == 'c': self.stats.fields_filter = DEFAULT_REGEX - self.refresh_header(0) - self.update_pid(0) + self._refresh_header(0) + self._update_pid(0) if char == 'f': curses.curs_set(1) - self.show_filter_selection() + self._show_filter_selection() curses.curs_set(0) sleeptime = self._delay_initial if char == 'g': curses.curs_set(1) - self.show_vm_selection_by_guest_name() + self._show_vm_selection_by_guest_name() curses.curs_set(0) sleeptime = self._delay_initial if char == 'h': - self.show_help_interactive() + self._show_help_interactive() if char == 'o': self._sorting = not self._sorting if char == 'p': curses.curs_set(1) - self.show_vm_selection_by_pid() + self._show_vm_selection_by_pid() curses.curs_set(0) sleeptime = self._delay_initial if char == 'q': @@ -1367,11 +1367,11 @@ class Tui(object): self.stats.reset() if char == 's': curses.curs_set(1) - self.show_set_update_interval() + self._show_set_update_interval() curses.curs_set(0) sleeptime = self._delay_initial if char == 'x': - self.update_drilldown() + self._update_drilldown() # prevents display of current values on next refresh self.stats.get(self._display_guests) except KeyboardInterrupt: -- 2.12.3 ++++++ 0040-tools-kvm_stat-eliminate-extra-guest-pid-selection-d.patch ++++++ >From 51ad2c1eb33a8fcf857b067c65cb03ed79c1a8bf Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:27 +0100 Subject: [PATCH 40/43] tools/kvm_stat: eliminate extra guest/pid selection dialog We can do with a single dialog that takes both, pids and guest names. Note that we keep both interactive commands, 'p' and 'g' for now, to avoid confusion among users used to a specific key. While at it, we improve on some minor glitches regarding curses usage, e.g. cursor still visible when not supposed to be. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 516f1190a1e0cce12128a6446e6438745c8de62a) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 110 ++++++++++++++-------------------------- tools/kvm/kvm_stat/kvm_stat.txt | 4 +- 2 files changed, 39 insertions(+), 75 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index e08fe887764f..d0add1dc4730 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1044,6 +1044,8 @@ class Tui(object): def _update_pid(self, pid): """Propagates pid selection to stats object.""" + self.screen.addstr(4, 1, 'Updating pid filter...') + self.screen.refresh() self.stats.pid_filter = pid def _refresh_header(self, pid=None): @@ -1147,10 +1149,10 @@ class Tui(object): ' filters)', ' c clear filter', ' f filter by regular expression', - ' g filter by guest name', + ' g filter by guest name/PID', ' h display interactive commands reference', ' o toggle sorting order (Total vs CurAvg/s)', - ' p filter by PID', + ' p filter by guest name/PID', ' q quit', ' r reset stats', ' s set update interval', @@ -1202,44 +1204,6 @@ class Tui(object): msg = '"' + regex + '": Not a valid regular expression' continue - def _show_vm_selection_by_pid(self): - """Draws PID selection mask. - - Asks for a pid until a valid pid or 0 has been entered. - - """ - msg = '' - while True: - self.screen.erase() - self.screen.addstr(0, 0, - 'Show statistics for specific pid.', - curses.A_BOLD) - self.screen.addstr(1, 0, - 'This might limit the shown data to the trace ' - 'statistics.') - self.screen.addstr(5, 0, msg) - self._print_all_gnames(7) - - curses.echo() - self.screen.addstr(3, 0, "Pid [0 or pid]: ") - pid = self.screen.getstr().decode(ENCODING) - curses.noecho() - - try: - if len(pid) > 0: - pid = int(pid) - if pid != 0 and not os.path.isdir(os.path.join('/proc/', - str(pid))): - msg = '"' + str(pid) + '": Not a running process' - continue - else: - pid = 0 - self._refresh_header(pid) - self._update_pid(pid) - break - except ValueError: - msg = '"' + str(pid) + '": Not a valid pid' - def _show_set_update_interval(self): """Draws update interval selection mask.""" msg = '' @@ -1272,17 +1236,17 @@ class Tui(object): msg = '"' + str(val) + '": Invalid value' self._refresh_header() - def _show_vm_selection_by_guest_name(self): + def _show_vm_selection_by_guest(self): """Draws guest selection mask. - Asks for a guest name until a valid guest name or '' is entered. + Asks for a guest name or pid until a valid guest name or '' is entered. """ msg = '' while True: self.screen.erase() self.screen.addstr(0, 0, - 'Show statistics for specific guest.', + 'Show statistics for specific guest or pid.', curses.A_BOLD) self.screen.addstr(1, 0, 'This might limit the shown data to the trace ' @@ -1290,32 +1254,39 @@ class Tui(object): self.screen.addstr(5, 0, msg) self._print_all_gnames(7) curses.echo() - self.screen.addstr(3, 0, "Guest [ENTER or guest]: ") - gname = self.screen.getstr().decode(ENCODING) + curses.curs_set(1) + self.screen.addstr(3, 0, "Guest or pid [ENTER exits]: ") + guest = self.screen.getstr().decode(ENCODING) curses.noecho() - if not gname: - self._refresh_header(0) - self._update_pid(0) + pid = 0 + if not guest or guest == '0': break - else: - pids = [] - try: - pids = self.get_pid_from_gname(gname) - except: - msg = '"' + gname + '": Internal error while searching, ' \ - 'use pid filter instead' - continue - if len(pids) == 0: - msg = '"' + gname + '": Not an active guest' + if guest.isdigit(): + if not os.path.isdir(os.path.join('/proc/', guest)): + msg = '"' + guest + '": Not a running process' continue - if len(pids) > 1: - msg = '"' + gname + '": Multiple matches found, use pid ' \ - 'filter instead' - continue - self._refresh_header(pids[0]) - self._update_pid(pids[0]) + pid = int(guest) break + pids = [] + try: + pids = self.get_pid_from_gname(guest) + except: + msg = '"' + guest + '": Internal error while searching, ' \ + 'use pid filter instead' + continue + if len(pids) == 0: + msg = '"' + guest + '": Not an active guest' + continue + if len(pids) > 1: + msg = '"' + guest + '": Multiple matches found, use pid ' \ + 'filter instead' + continue + pid = pids[0] + break + curses.curs_set(0) + self._refresh_header(pid) + self._update_pid(pid) def show_stats(self): """Refreshes the screen and processes user input.""" @@ -1347,20 +1318,13 @@ class Tui(object): self._show_filter_selection() curses.curs_set(0) sleeptime = self._delay_initial - if char == 'g': - curses.curs_set(1) - self._show_vm_selection_by_guest_name() - curses.curs_set(0) + if char == 'g' or char == 'p': + self._show_vm_selection_by_guest() sleeptime = self._delay_initial if char == 'h': self._show_help_interactive() if char == 'o': self._sorting = not self._sorting - if char == 'p': - curses.curs_set(1) - self._show_vm_selection_by_pid() - curses.curs_set(0) - sleeptime = self._delay_initial if char == 'q': break if char == 'r': diff --git a/tools/kvm/kvm_stat/kvm_stat.txt b/tools/kvm/kvm_stat/kvm_stat.txt index b5b3810c9e94..0811d860fe75 100644 --- a/tools/kvm/kvm_stat/kvm_stat.txt +++ b/tools/kvm/kvm_stat/kvm_stat.txt @@ -35,13 +35,13 @@ INTERACTIVE COMMANDS *f*:: filter by regular expression -*g*:: filter by guest name +*g*:: filter by guest name/PID *h*:: display interactive commands reference *o*:: toggle sorting order (Total vs CurAvg/s) -*p*:: filter by PID +*p*:: filter by guest name/PID *q*:: quit -- 2.12.3 ++++++ 0041-tools-kvm_stat-separate-drilldown-and-fields-filteri.patch ++++++ >From 2bc0796f97b852855fa5a58de695d76e08ef9b90 Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:28 +0100 Subject: [PATCH 41/43] tools/kvm_stat: separate drilldown and fields filtering Drilldown (i.e. toggle display of child trace events) was implemented by overriding the fields filter. This resulted in inconsistencies: E.g. when drilldown was not active, adding a filter that also matches child trace events would not only filter fields according to the filter, but also add in the child trace events matching the filter. E.g. on x86, setting 'kvm_userspace_exit' as the fields filter after startup would result in display of kvm_userspace_exit(DCR), although that wasn't previously present - not exactly what one would expect from a filter. This patch addresses the issue by keeping drilldown and fields filter separate. While at it, we also fix a PEP8 issue by adding a blank line at one place (since we're in the area...). We implement this by adding a framework that also allows to define a taxonomy among the debugfs events to identify child trace events. I.e. drilldown using 'x' can now also work with debugfs. A respective parent- child relationship is only known for S390 at the moment, but could be added adjusting other platforms' ARCH.dbg_is_child() methods accordingly. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 18e8f4100ef14f924514fbd91eb67bd5fa5396b7) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 143 +++++++++++++++++++++++++++++++------------- 1 file changed, 100 insertions(+), 43 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index d0add1dc4730..cc9e6885b3e8 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -228,6 +228,7 @@ IOCTL_NUMBERS = { } ENCODING = locale.getpreferredencoding(False) +TRACE_FILTER = re.compile(r'^[^\(]*$') class Arch(object): @@ -260,6 +261,11 @@ class Arch(object): return ArchX86(SVM_EXIT_REASONS) return + def tracepoint_is_child(self, field): + if (TRACE_FILTER.match(field)): + return None + return field.split('(', 1)[0] + class ArchX86(Arch): def __init__(self, exit_reasons): @@ -267,6 +273,10 @@ class ArchX86(Arch): self.ioctl_numbers = IOCTL_NUMBERS self.exit_reasons = exit_reasons + def debugfs_is_child(self, field): + """ Returns name of parent if 'field' is a child, None otherwise """ + return None + class ArchPPC(Arch): def __init__(self): @@ -282,6 +292,10 @@ class ArchPPC(Arch): self.ioctl_numbers['SET_FILTER'] = 0x80002406 | char_ptr_size << 16 self.exit_reasons = {} + def debugfs_is_child(self, field): + """ Returns name of parent if 'field' is a child, None otherwise """ + return None + class ArchA64(Arch): def __init__(self): @@ -289,6 +303,10 @@ class ArchA64(Arch): self.ioctl_numbers = IOCTL_NUMBERS self.exit_reasons = AARCH64_EXIT_REASONS + def debugfs_is_child(self, field): + """ Returns name of parent if 'field' is a child, None otherwise """ + return None + class ArchS390(Arch): def __init__(self): @@ -296,6 +314,12 @@ class ArchS390(Arch): self.ioctl_numbers = IOCTL_NUMBERS self.exit_reasons = None + def debugfs_is_child(self, field): + """ Returns name of parent if 'field' is a child, None otherwise """ + if field.startswith('instruction_'): + return 'exit_instruction' + + ARCH = Arch.get_arch() @@ -472,6 +496,10 @@ class Event(object): class Provider(object): """Encapsulates functionalities used by all providers.""" + def __init__(self, pid): + self.child_events = False + self.pid = pid + @staticmethod def is_field_wanted(fields_filter, field): """Indicate whether field is valid according to fields_filter.""" @@ -499,7 +527,7 @@ class TracepointProvider(Provider): self.group_leaders = [] self.filters = self._get_filters() self.update_fields(fields_filter) - self.pid = pid + super(TracepointProvider, self).__init__(pid) @staticmethod def _get_filters(): @@ -519,7 +547,7 @@ class TracepointProvider(Provider): return filters def _get_available_fields(self): - """Returns a list of available event's of format 'event name(filter + """Returns a list of available events of format 'event name(filter name)'. All available events have directories under @@ -547,7 +575,8 @@ class TracepointProvider(Provider): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" self.fields = [field for field in self._get_available_fields() - if self.is_field_wanted(fields_filter, field)] + if self.is_field_wanted(fields_filter, field) or + ARCH.tracepoint_is_child(field)] @staticmethod def _get_online_cpus(): @@ -671,8 +700,12 @@ class TracepointProvider(Provider): ret = defaultdict(int) for group in self.group_leaders: for name, val in group.read().items(): - if name in self._fields: - ret[name] += val + if name not in self._fields: + continue + parent = ARCH.tracepoint_is_child(name) + if parent: + name += ' ' + parent + ret[name] += val return ret def reset(self): @@ -690,7 +723,7 @@ class DebugfsProvider(Provider): self._baseline = {} self.do_read = True self.paths = [] - self.pid = pid + super(DebugfsProvider, self).__init__(pid) if include_past: self._restore() @@ -705,7 +738,8 @@ class DebugfsProvider(Provider): def update_fields(self, fields_filter): """Refresh fields, applying fields_filter""" self._fields = [field for field in self._get_available_fields() - if self.is_field_wanted(fields_filter, field)] + if self.is_field_wanted(fields_filter, field) or + ARCH.debugfs_is_child(field)] @property def fields(self): @@ -766,14 +800,15 @@ class DebugfsProvider(Provider): self._baseline[key] = 0 if self._baseline.get(key, -1) == -1: self._baseline[key] = value - increment = (results.get(field, 0) + value - - self._baseline.get(key, 0)) - if by_guest: - pid = key.split('-')[0] - if pid in results: - results[pid] += increment - else: - results[pid] = increment + parent = ARCH.debugfs_is_child(field) + if parent: + field = field + ' ' + parent + else: + if by_guest: + field = key.split('-')[0] # set 'field' to 'pid' + increment = value - self._baseline.get(key, 0) + if field in results: + results[field] += increment else: results[field] = increment @@ -815,6 +850,7 @@ class Stats(object): self._pid_filter = options.pid self._fields_filter = options.fields self.values = {} + self._child_events = False def _get_providers(self, options): """Returns a list of data providers depending on the passed options.""" @@ -863,12 +899,29 @@ class Stats(object): for provider in self.providers: provider.pid = self._pid_filter + @property + def child_events(self): + return self._child_events + + @child_events.setter + def child_events(self, val): + self._child_events = val + for provider in self.providers: + provider.child_events = val + def get(self, by_guest=0): """Returns a dict with field -> (value, delta to last value) of all - provider data.""" + provider data. + Key formats: + * plain: 'key' is event name + * child-parent: 'key' is in format '<child> <parent>' + * pid: 'key' is the pid of the guest, and the record contains the + aggregated event data + These formats are generated by the providers, and handled in class TUI. + """ for provider in self.providers: new = provider.read(by_guest=by_guest) - for key in new if by_guest else provider.fields: + for key in new: oldval = self.values.get(key, EventStat(0, 0)).value newval = new.get(key, 0) newdelta = newval - oldval @@ -901,10 +954,10 @@ class Stats(object): self.get(to_pid) return 0 + DELAY_DEFAULT = 3.0 MAX_GUEST_NAME_LEN = 48 MAX_REGEX_LEN = 44 -DEFAULT_REGEX = r'^[^\(]*$' SORT_DEFAULT = 0 @@ -1034,14 +1087,6 @@ class Tui(object): return name - def _update_drilldown(self): - """Sets or removes a filter that only allows fields without braces.""" - if not self.stats.fields_filter: - self.stats.fields_filter = DEFAULT_REGEX - - elif self.stats.fields_filter == DEFAULT_REGEX: - self.stats.fields_filter = None - def _update_pid(self, pid): """Propagates pid selection to stats object.""" self.screen.addstr(4, 1, 'Updating pid filter...') @@ -1063,8 +1108,7 @@ class Tui(object): .format(pid, gname), curses.A_BOLD) else: self.screen.addstr(0, 0, 'kvm statistics - summary', curses.A_BOLD) - if self.stats.fields_filter and self.stats.fields_filter \ - != DEFAULT_REGEX: + if self.stats.fields_filter: regex = self.stats.fields_filter if len(regex) > MAX_REGEX_LEN: regex = regex[:MAX_REGEX_LEN] + '...' @@ -1080,6 +1124,9 @@ class Tui(object): self.screen.refresh() def _refresh_body(self, sleeptime): + def is_child_field(field): + return field.find('(') != -1 + row = 3 self.screen.move(row, 0) self.screen.clrtobot() @@ -1087,7 +1134,11 @@ class Tui(object): total = 0. ctotal = 0. for key, values in stats.items(): - if key.find('(') == -1: + if self._display_guests: + if self.get_gname_from_pid(key): + total += values.value + continue + if not key.find(' ') != -1: total += values.value else: ctotal += values.value @@ -1104,19 +1155,26 @@ class Tui(object): # sort by overall value return v.value + sorted_items = sorted(stats.items(), key=sortkey, reverse=True) + + # print events tavg = 0 - for key, values in sorted(stats.items(), key=sortkey, reverse=True): + for key, values in sorted_items: if row >= self.screen.getmaxyx()[0] - 1: break - if not values.value and not values.delta: - break + if values == (0, 0): + continue + if not self.stats.child_events and key.find(' ') != -1: + continue if values.value is not None: cur = int(round(values.delta / sleeptime)) if values.delta else '' if self._display_guests: key = self.get_gname_from_pid(key) - self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % - (key, values.value, values.value * 100 / total, - cur)) + if not key: + continue + self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key + .split(' ')[0], values.value, + values.value * 100 / total, cur)) if cur != '' and key.find('(') == -1: tavg += cur row += 1 @@ -1192,7 +1250,7 @@ class Tui(object): regex = self.screen.getstr().decode(ENCODING) curses.noecho() if len(regex) == 0: - self.stats.fields_filter = DEFAULT_REGEX + self.stats.fields_filter = '' self._refresh_header() return try: @@ -1310,7 +1368,7 @@ class Tui(object): self._display_guests = not self._display_guests self._refresh_header() if char == 'c': - self.stats.fields_filter = DEFAULT_REGEX + self.stats.fields_filter = '' self._refresh_header(0) self._update_pid(0) if char == 'f': @@ -1335,9 +1393,7 @@ class Tui(object): curses.curs_set(0) sleeptime = self._delay_initial if char == 'x': - self._update_drilldown() - # prevents display of current values on next refresh - self.stats.get(self._display_guests) + self.stats.child_events = not self.stats.child_events except KeyboardInterrupt: break except curses.error: @@ -1351,7 +1407,8 @@ def batch(stats): time.sleep(1) s = stats.get() for key, values in sorted(s.items()): - print('%-42s%10d%10d' % (key, values.value, values.delta)) + print('%-42s%10d%10d' % (key.split(' ')[0], values.value, + values.delta)) except KeyboardInterrupt: pass @@ -1362,7 +1419,7 @@ def log(stats): def banner(): for key in keys: - print(key, end=' ') + print(key.split(' ')[0], end=' ') print() def statline(): @@ -1473,7 +1530,7 @@ Press any other key to refresh statistics immediately. ) optparser.add_option('-f', '--fields', action='store', - default=DEFAULT_REGEX, + default='', dest='fields', help='''fields to display (regex) "-f help" for a list of available events''', -- 2.12.3 ++++++ 0042-tools-kvm_stat-group-child-events-indented-after-par.patch ++++++ >From 5d3fb91fd764b50efab0fcfe9cf2da7d2f14cf2b Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:29 +0100 Subject: [PATCH 42/43] tools/kvm_stat: group child events indented after parent We keep the current logic that sorts all events (parent and child), but re-shuffle the events afterwards, grouping the children after the respective parent. Note that the percentage column for child events gives the percentage of the parent's total. Since we rework the logic anyway, we modify the total average calculation to use the raw numbers instead of the (rounded) averages. Note that this can result in differing numbers (between total average and the sum of the individual averages) due to rounding errors. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit df72ecfc790fa01de1c41f836ff51d12f9c40318) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 89 ++++++++++++++++++++++++++++++--------------- 1 file changed, 59 insertions(+), 30 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index cc9e6885b3e8..97574f632ccd 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1127,6 +1127,45 @@ class Tui(object): def is_child_field(field): return field.find('(') != -1 + def insert_child(sorted_items, child, values, parent): + num = len(sorted_items) + for i in range(0, num): + # only add child if parent is present + if parent.startswith(sorted_items[i][0]): + sorted_items.insert(i + 1, (' ' + child, values)) + + def get_sorted_events(self, stats): + """ separate parent and child events """ + if self._sorting == SORT_DEFAULT: + def sortkey((_k, v)): + # sort by (delta value, overall value) + return (v.delta, v.value) + else: + def sortkey((_k, v)): + # sort by overall value + return v.value + + childs = [] + sorted_items = [] + # we can't rule out child events to appear prior to parents even + # when sorted - separate out all children first, and add in later + for key, values in sorted(stats.items(), key=sortkey, + reverse=True): + if values == (0, 0): + continue + if key.find(' ') != -1: + if not self.stats.child_events: + continue + childs.insert(0, (key, values)) + else: + sorted_items.append((key, values)) + if self.stats.child_events: + for key, values in childs: + (child, parent) = key.split(' ') + insert_child(sorted_items, child, values, parent) + + return sorted_items + row = 3 self.screen.move(row, 0) self.screen.clrtobot() @@ -1146,44 +1185,34 @@ class Tui(object): # we don't have any fields, or all non-child events are filtered total = ctotal - if self._sorting == SORT_DEFAULT: - def sortkey((_k, v)): - # sort by (delta value, overall value) - return (v.delta, v.value) - else: - def sortkey((_k, v)): - # sort by overall value - return v.value - - sorted_items = sorted(stats.items(), key=sortkey, reverse=True) - # print events tavg = 0 - for key, values in sorted_items: - if row >= self.screen.getmaxyx()[0] - 1: + tcur = 0 + for key, values in get_sorted_events(self, stats): + if row >= self.screen.getmaxyx()[0] - 1 or values == (0, 0): break - if values == (0, 0): - continue - if not self.stats.child_events and key.find(' ') != -1: - continue - if values.value is not None: - cur = int(round(values.delta / sleeptime)) if values.delta else '' - if self._display_guests: - key = self.get_gname_from_pid(key) - if not key: - continue - self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key - .split(' ')[0], values.value, - values.value * 100 / total, cur)) - if cur != '' and key.find('(') == -1: - tavg += cur + if self._display_guests: + key = self.get_gname_from_pid(key) + if not key: + continue + cur = int(round(values.delta / sleeptime)) if values.delta else '' + if key[0] != ' ': + if values.delta: + tcur += values.delta + ptotal = values.value + ltotal = total + else: + ltotal = ptotal + self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key, + values.value, + values.value * 100 / float(ltotal), cur)) row += 1 if row == 3: self.screen.addstr(4, 1, 'No matching events reported yet') else: + tavg = int(round(tcur / sleeptime)) if tcur > 0 else '' self.screen.addstr(row, 1, '%-40s %10d %8s' % - ('Total', total, tavg if tavg else ''), - curses.A_BOLD) + ('Total', total, tavg), curses.A_BOLD) self.screen.refresh() def _show_msg(self, text): -- 2.12.3 ++++++ 0043-tools-kvm_stat-print-Total-line-for-multiple-events-.patch ++++++ >From 7c8eaab8cfdf7f433f216d91be84b9639ca3366c Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Thu, 22 Feb 2018 12:16:30 +0100 Subject: [PATCH 43/43] tools/kvm_stat: print 'Total' line for multiple events only The 'Total' line looks a bit weird when we have a single event only. This can happen e.g. due to filters. Therefore suppress when there's only a single event in the output. Signed-off-by: Stefan Raspl <raspl(a)linux.vnet.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> (cherry picked from commit 6789af030a462708f937137e05eb12ea009fb348) [FL: FATE#325017] Signed-off-by: Fei Li <fli(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 97574f632ccd..313ebb4bec6b 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1209,7 +1209,7 @@ class Tui(object): row += 1 if row == 3: self.screen.addstr(4, 1, 'No matching events reported yet') - else: + if row > 4: tavg = int(round(tcur / sleeptime)) if tcur > 0 else '' self.screen.addstr(row, 1, '%-40s %10d %8s' % ('Total', total, tavg), curses.A_BOLD) -- 2.12.3 ++++++ 0044-tools-kvm_stat-Don-t-use-deprecated-file.patch ++++++ >From 3af37d11f6e171ae471c128f35a35faae68cc992 Mon Sep 17 00:00:00 2001 From: Cole Robinson <crobinso(a)redhat.com> Date: Fri, 23 Mar 2018 18:07:17 -0400 Subject: [PATCH] tools/kvm_stat: Don't use deprecated file() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit $ python3 tools/kvm/kvm_stat/kvm_stat Traceback (most recent call last): File "tools/kvm/kvm_stat/kvm_stat", line 1668, in <module> main() File "tools/kvm/kvm_stat/kvm_stat", line 1639, in main assign_globals() File "tools/kvm/kvm_stat/kvm_stat", line 1618, in assign_globals for line in file('/proc/mounts'): NameError: name 'file' is not defined open() is the python3 way, and works on python2.6+ Signed-off-by: Cole Robinson <crobinso(a)redhat.com> Reviewed-and-tested-by: Stefan Raspl <stefan.raspl(a)linux.vnet.ibm.com> Signed-off-by: Radim Krčmář <rkrcmar(a)redhat.com> (cherry picked from commit 0866c31bf32fbd05dd25b6ed4976fa9134773227) [BR: BSC#1116822] Signed-off-by: Bruce Rogers <brogers(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 2a95389d2864..04737c1a382d 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1618,7 +1618,7 @@ def assign_globals(): global PATH_DEBUGFS_TRACING debugfs = '' - for line in file('/proc/mounts'): + for line in open('/proc/mounts'): if line.split(' ')[0] == 'debugfs': debugfs = line.split(' ')[1] break -- 2.19.1 ++++++ 0045-tools-kvm_stat-fix-python3-issues.patch ++++++ >From 11c9adb22aa804348b48807ecba10d40878858be Mon Sep 17 00:00:00 2001 From: Stefan Raspl <stefan.raspl(a)de.ibm.com> Date: Fri, 24 Aug 2018 14:03:55 +0200 Subject: [PATCH] tools/kvm_stat: fix python3 issues MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Python3 returns a float for a regular division - switch to a division operator that returns an integer. Furthermore, filters return a generator object instead of the actual list - wrap result in yet another list, which makes it still work in both, Python2 and 3. Signed-off-by: Stefan Raspl <raspl(a)linux.ibm.com> Signed-off-by: Radim Krčmář <rkrcmar(a)redhat.com> (cherry picked from commit 58f33cfe73076b6497bada4f7b5bda961ed68083) [BR: BSC#1116822] Signed-off-by: Bruce Rogers <brogers(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 313ebb4bec6b..789fa3ad5508 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -762,7 +762,7 @@ class DebugfsProvider(Provider): if len(vms) == 0: self.do_read = False - self.paths = filter(lambda x: "{}-".format(pid) in x, vms) + self.paths = list(filter(lambda x: "{}-".format(pid) in x, vms)) else: self.paths = [] -- 2.19.1 ++++++ 0046-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat.patch ++++++ >From b6fcf014ddfc2f02a9310f8e4d2fd28890566ba5 Mon Sep 17 00:00:00 2001 From: Bruce Rogers <brogers(a)suse.com> Date: Wed, 21 Nov 2018 12:10:44 -0700 Subject: [PATCH] tools/kvm_stat: apply python 2to3 fixes to kvm_stat [BR: BSC#1116822] Signed-off-by: Bruce Rogers <brogers(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 789fa3ad5508..ed6518bf2ab5 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -19,7 +19,7 @@ Three different ways of output formatting are available: The data is sampled from the KVM's debugfs entries and its perf events. """ -from __future__ import print_function + import curses import sys @@ -384,9 +384,9 @@ class Group(object): """ length = 8 * (1 + len(self.events)) read_format = 'xxxxxxxx' + 'Q' * len(self.events) - return dict(zip([event.name for event in self.events], + return dict(list(zip([event.name for event in self.events], struct.unpack(read_format, - os.read(self.events[0].fd, length)))) + os.read(self.events[0].fd, length))))) class Event(object): @@ -592,8 +592,8 @@ class TracepointProvider(Provider): integers.append(int(member)) else: int_range = member.split('-') - integers.extend(range(int(int_range[0]), - int(int_range[1]) + 1)) + integers.extend(list(range(int(int_range[0]), + int(int_range[1]) + 1))) return integers @@ -699,7 +699,7 @@ class TracepointProvider(Provider): """Returns 'event name: current value' for all enabled events.""" ret = defaultdict(int) for group in self.group_leaders: - for name, val in group.read().items(): + for name, val in list(group.read().items()): if name not in self._fields: continue parent = ARCH.tracepoint_is_child(name) @@ -762,7 +762,7 @@ class DebugfsProvider(Provider): if len(vms) == 0: self.do_read = False - self.paths = list(filter(lambda x: "{}-".format(pid) in x, vms)) + self.paths = list([x for x in vms if "{}-".format(pid) in x]) else: self.paths = [] @@ -943,7 +943,7 @@ class Stats(object): for provider in self.providers: if isinstance(provider, DebugfsProvider): for key in provider.fields: - if key in self.values.keys(): + if key in list(self.values.keys()): del self.values[key] else: oldvals = self.values.copy() @@ -1137,19 +1137,21 @@ class Tui(object): def get_sorted_events(self, stats): """ separate parent and child events """ if self._sorting == SORT_DEFAULT: - def sortkey((_k, v)): + def sortkey(xxx_todo_changeme): # sort by (delta value, overall value) + (_k, v) = xxx_todo_changeme return (v.delta, v.value) else: - def sortkey((_k, v)): + def sortkey(xxx_todo_changeme1): # sort by overall value + (_k, v) = xxx_todo_changeme1 return v.value childs = [] sorted_items = [] # we can't rule out child events to appear prior to parents even # when sorted - separate out all children first, and add in later - for key, values in sorted(stats.items(), key=sortkey, + for key, values in sorted(list(stats.items()), key=sortkey, reverse=True): if values == (0, 0): continue @@ -1172,7 +1174,7 @@ class Tui(object): stats = self.stats.get(self._display_guests) total = 0. ctotal = 0. - for key, values in stats.items(): + for key, values in list(stats.items()): if self._display_guests: if self.get_gname_from_pid(key): total += values.value @@ -1652,7 +1654,7 @@ def main(): if options.fields == 'help': stats.fields_filter = None event_list = [] - for key in stats.get().keys(): + for key in list(stats.get().keys()): event_list.append(key.split('(', 1)[0]) sys.stdout.write(' ' + '\n '.join(sorted(set(event_list))) + '\n') sys.exit(0) -- 2.19.1 ++++++ 0047-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat2.patch ++++++ >From d422b653bbd30152ce332eeb88154d3236b3ce5f Mon Sep 17 00:00:00 2001 From: Larry Dewey <ldewey(a)suse.com> Date: Thu, 29 Nov 2018 09:11:29 -0700 Subject: [PATCH] tools/kvm_stat: apply python 2to3 fixes to kvm_stat2 [LD: BSC#1116822] Signed-off-by: Larry Dewey <ldewey(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 8f74ed8e7237..1209c6931c81 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -315,8 +315,8 @@ def parse_int_list(list_string): integers.append(int(member)) else: int_range = member.split('-') - integers.extend(range(int(int_range[0]), - int(int_range[1]) + 1)) + integers.extend(list(range(int(int_range[0]), + int(int_range[1]) + 1))) return integers @@ -486,9 +486,9 @@ class Group(object): """ length = 8 * (1 + len(self.events)) read_format = 'xxxxxxxx' + 'Q' * len(self.events) - return dict(zip([event.name for event in self.events], + return dict(list(zip([event.name for event in self.events], struct.unpack(read_format, - os.read(self.events[0].fd, length)))) + os.read(self.events[0].fd, length))))) class Event(object): @@ -711,7 +711,7 @@ class TracepointProvider(object): """Returns 'event name: current value' for all enabled events.""" ret = defaultdict(int) for group in self.group_leaders: - for name, val in group.read().iteritems(): + for name, val in group.read().items(): if name in self._fields: ret[name] += val return ret @@ -764,7 +764,7 @@ class DebugfsProvider(object): if len(vms) == 0: self.do_read = False - self.paths = filter(lambda x: "{}-".format(pid) in x, vms) + self.paths = [x for x in vms if "{}-".format(pid) in x] else: self.paths = [] @@ -992,9 +992,9 @@ class Tui(object): else: return (0, -stats[x][0]) total = 0. - for val in stats.values(): + for val in list(stats.values()): total += val[0] - for key in sorted(stats.keys(), key=sortkey): + for key in sorted(list(stats.keys()), key=sortkey): if row >= self.screen.getmaxyx()[0]: break @@ -1174,25 +1174,25 @@ def batch(stats): s = stats.get() for key in sorted(s.keys()): values = s[key] - print '%-42s%10d%10d' % (key, values[0], values[1]) + print('%-42s%10d%10d' % (key, values[0], values[1])) except KeyboardInterrupt: pass def log(stats): """Prints statistics as reiterating key block, multiple value blocks.""" - keys = sorted(stats.get().iterkeys()) + keys = sorted(stats.get().keys()) def banner(): for k in keys: - print '%s' % k, - print + print('%s' % k, end=' ') + print() def statline(): s = stats.get() for k in keys: - print ' %9d' % s[k][1], - print + print(' %9d' % s[k][1], end=' ') + print() line = 0 banner_repeat = 20 while True: -- 2.19.1 ++++++ 0048-tools-kvm_stat-apply-python-2to3-fixes-to-kvm_stat3.patch ++++++ >From b6fcf014ddfc2f02a9310f8e4d2fd28890566ba5 Mon Sep 17 00:00:00 2001 From: Bruce Rogers <brogers(a)suse.com> Date: Wed, 21 Nov 2018 12:10:44 -0700 Subject: [PATCH] tools/kvm_stat: apply python 2to3 fixes to kvm_stat3 [LD: BSC#1116822] Signed-off-by: Larry Dewey <ldewey(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 789fa3ad5508..ed6518bf2ab5 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -19,7 +19,7 @@ Three different ways of output formatting are available: The data is sampled from the KVM's debugfs entries and its perf events. """ -from __future__ import print_function + import curses import sys @@ -384,9 +384,9 @@ class Group(object): """ length = 8 * (1 + len(self.events)) read_format = 'xxxxxxxx' + 'Q' * len(self.events) - return dict(zip([event.name for event in self.events], + return dict(list(zip([event.name for event in self.events], struct.unpack(read_format, - os.read(self.events[0].fd, length)))) + os.read(self.events[0].fd, length))))) class Event(object): @@ -592,8 +592,8 @@ class TracepointProvider(Provider): integers.append(int(member)) else: int_range = member.split('-') - integers.extend(range(int(int_range[0]), - int(int_range[1]) + 1)) + integers.extend(list(range(int(int_range[0]), + int(int_range[1]) + 1))) return integers @@ -696,7 +696,7 @@ class TracepointProvider(Provider): """Returns 'event name: current value' for all enabled events.""" ret = defaultdict(int) for group in self.group_leaders: - for name, val in group.read().items(): + for name, val in list(group.read().items()): if name not in self._fields: continue parent = ARCH.tracepoint_is_child(name) @@ -759,7 +759,7 @@ class DebugfsProvider(Provider): if len(vms) == 0: self.do_read = False - self.paths = list(filter(lambda x: "{}-".format(pid) in x, vms)) + self.paths = list([x for x in vms if "{}-".format(pid) in x]) else: self.paths = [] @@ -947,7 +947,7 @@ class Stats(object): for provider in self.providers: if isinstance(provider, DebugfsProvider): for key in provider.fields: - if key in self.values.keys(): + if key in list(self.values.keys()): del self.values[key] else: oldvals = self.values.copy() @@ -1153,7 +1153,7 @@ class Tui(object): sorted_items = [] # we can't rule out child events to appear prior to parents even # when sorted - separate out all children first, and add in later - for key, values in sorted(stats.items(), key=sortkey, + for key, values in sorted(list(stats.items()), key=sortkey, reverse=True): if values == (0, 0): continue @@ -1189,7 +1189,7 @@ class Tui(object): stats = self.stats.get(self._display_guests) total = 0. ctotal = 0. - for key, values in stats.items(): + for key, values in list(stats.items()): if self._display_guests: if self.get_gname_from_pid(key): total += values.value @@ -1687,7 +1687,7 @@ def main(): if options.fields == 'help': stats.fields_filter = None event_list = [] - for key in stats.get().keys(): + for key in list(stats.get().keys()): event_list.append(key.split('(', 1)[0]) sys.stdout.write(' ' + '\n '.join(sorted(set(event_list))) + '\n') sys.exit(0) -- 2.19.1 ++++++ 0049-tools-kvm_stat-switch-python-reference-to-be-explici.patch ++++++ >From 143ebdcad946ea4ff8f897bca5604a0f0f5ce0bf Mon Sep 17 00:00:00 2001 From: Bruce Rogers <brogers(a)suse.com> Date: Wed, 21 Nov 2018 11:31:05 -0700 Subject: [PATCH] tools/kvm_stat: switch python reference to be explicitly python3 [BR: BSC#1116822] Signed-off-by: Bruce Rogers <brogers(a)suse.com> --- tools/kvm/kvm_stat/kvm_stat | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index ed6518bf2ab5..2a95389d2864 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/python3 # # top-like utility for displaying kvm statistics # -- 2.19.1

1 0

commit icecream for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package icecream for openSUSE:Factory checked in at 2018-12-03 10:09:52 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/icecream (Old) and /work/SRC/openSUSE:Factory/.icecream.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "icecream" Mon Dec 3 10:09:52 2018 rev:51 rq:653101 version:1.2 Changes: -------- --- /work/SRC/openSUSE:Factory/icecream/icecream.changes 2018-07-23 18:00:19.424981553 +0200 +++ /work/SRC/openSUSE:Factory/.icecream.new.19453/icecream.changes 2018-12-03 10:09:52.455737607 +0100 @@ -1,0 +2,20 @@ +Fri Nov 30 15:19:43 UTC 2018 - Stephan Kulow <coolo(a)suse.com> + +- update to 1.2 + - Add more compiler flags to the list that mean build locally + * -pedantic (preprocessing only) + * -pedantic-errors (preprocessing only) + * -fsyntax-only + - don't force local compile on -include-pch + - Make load calculation better + - Limit amount of data sent at one time for slow networks/remotes + - Many updates to the tests + - Better logs of some error conditions + - Build locally if it is likely that there will not be more compiles + - Support adding gcc and clang to the same environment + - Better handling of icerun + - Cygwin now works as a client + - Don't expose Host endianness to network + - General code cleanup + +------------------------------------------------------------------- Old: ---- 1.1.tar.gz New: ---- 1.2.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ icecream.spec ++++++ --- /var/tmp/diff_new_pack.DqLJLO/_old 2018-12-03 10:09:53.011737094 +0100 +++ /var/tmp/diff_new_pack.DqLJLO/_new 2018-12-03 10:09:53.011737094 +0100 @@ -12,7 +12,7 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # @@ -21,7 +21,7 @@ %define _fillupdir %{_localstatedir}/adm/fillup-templates %endif Name: icecream -Version: 1.1 +Version: 1.2 Release: 0 Summary: For Distributed Compile in the Network License: GPL-2.0-or-later AND LGPL-2.1-or-later @@ -153,6 +153,7 @@ %license COPYING %config %{_sysconfdir}/logrotate.d/icecream %{_bindir}/icecc-create-env +%{_bindir}/icecc-test-env %{_bindir}/icecc %{_bindir}/icerun %{_sbindir}/icecc-scheduler ++++++ 1.1.tar.gz -> 1.2.tar.gz ++++++ ++++ 9795 lines of diff (skipped)

1 0

commit python-ruamel.yaml for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package python-ruamel.yaml for openSUSE:Factory checked in at 2018-12-03 10:09:46 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-ruamel.yaml (Old) and /work/SRC/openSUSE:Factory/.python-ruamel.yaml.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "python-ruamel.yaml" Mon Dec 3 10:09:46 2018 rev:10 rq:653087 version:0.15.78 Changes: -------- --- /work/SRC/openSUSE:Factory/python-ruamel.yaml/python-ruamel.yaml.changes 2018-10-02 19:48:58.161734608 +0200 +++ /work/SRC/openSUSE:Factory/.python-ruamel.yaml.new.19453/python-ruamel.yaml.changes 2018-12-03 10:09:51.675738327 +0100 @@ -1,0 +2,15 @@ +Fri Nov 30 07:43:33 UTC 2018 - Adrian Schröter <adrian(a)suse.de> + +- update to 0.15.78 + * setup issue for 3.8 (reported by Sidney Kuyateh) + * setting yaml.sort_base_mapping_type_on_output = False, will prevent explicit sorting by keys in the base representer of mappings. Roundtrip already did not do this. Usage only makes real sense for Python 3.6+ (feature request by Sebastian Gerber). + * implement Python version check in YAML metadata in _test/test_z_data.py + * fix issue with empty mapping and sequence loaded as flow-style (mapping reported by Min RK, sequence by Maged Ahmed) + * fix issue with single '?' scalar (reported by Terrance) + * fix issue with duplicate merge keys (prompted by answering a StackOverflow question by math) + * fix dropping of comment on rt before sequence item that is sequence item (reported by Thorsten Kampe) + * fix irregular output on pre-comment in sequence within sequence (reported by Thorsten Kampe) + * allow non-compact (i.e. next line) dumping sequence/mapping within sequence. + * fix regression on explicit 1.1 loading with the C based scanner/parser (reported by Tomas Vavra) + +------------------------------------------------------------------- Old: ---- ruamel.yaml-0.15.71.tar.gz New: ---- ruamel.yaml-0.15.78.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-ruamel.yaml.spec ++++++ --- /var/tmp/diff_new_pack.d6xGVq/_old 2018-12-03 10:09:52.191737850 +0100 +++ /var/tmp/diff_new_pack.d6xGVq/_new 2018-12-03 10:09:52.191737850 +0100 @@ -18,7 +18,7 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-ruamel.yaml -Version: 0.15.71 +Version: 0.15.78 Release: 0 Summary: Python YAML parser License: MIT ++++++ ruamel.yaml-0.15.71.tar.gz -> ruamel.yaml-0.15.78.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/CHANGES new/ruamel.yaml-0.15.78/CHANGES --- old/ruamel.yaml-0.15.71/CHANGES 2018-09-26 22:55:47.000000000 +0200 +++ new/ruamel.yaml-0.15.78/CHANGES 2018-11-15 17:41:48.000000000 +0100 @@ -1,3 +1,52 @@ +[0, 15, 78]: 2018-11-15 + - fix setup issue for 3.8 (reported by `Sidney Kuyateh + <https://bitbucket.org/autinerd/>`__) + +[0, 15, 77]: 2018-11-09 + - setting `yaml.sort_base_mapping_type_on_output = False`, will prevent + explicit sorting by keys in the base representer of mappings. Roundtrip + already did not do this. Usage only makes real sense for Python 3.6+ + (feature request by `Sebastian Gerber <https://bitbucket.org/spacemanspiff2007/>`__). + - implement Python version check in YAML metadata in ``_test/test_z_data.py`` + +[0, 15, 76]: 2018-11-01 + - fix issue with empty mapping and sequence loaded as flow-style + (mapping reported by `Min RK <https://bitbucket.org/minrk/>`__, sequence + by `Maged Ahmed <https://bitbucket.org/maged2/>`__) + +[0, 15, 75]: 2018-10-27 + - fix issue with single '?' scalar (reported by `Terrance + <https://bitbucket.org/OllieTerrance/>`__) + - fix issue with duplicate merge keys (prompted by `answering + <https://stackoverflow.com/a/52852106/1307905>`__ a + `StackOverflow question <https://stackoverflow.com/q/52851168/1307905>`__ + by `math <https://stackoverflow.com/users/1355634/math>`__) + +[0, 15, 75]: 2018-10-27 + - fix issue with single '?' scalar (reported by `Terrance + <https://bitbucket.org/OllieTerrance/>`__) + - fix issue with duplicate merge keys (prompted by `answering + <https://stackoverflow.com/a/52852106/1307905>`__ a + `StackOverflow question <https://stackoverflow.com/q/52851168/1307905>`__ + by `math <https://stackoverflow.com/users/1355634/math>`__) + +[0, 15, 74]: 2018-10-17 + - fix dropping of comment on rt before sequence item that is sequence item + (reported by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + +[0, 15, 73]: 2018-10-16 + - fix irregular output on pre-comment in sequence within sequence (reported + by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + - allow non-compact (i.e. next line) dumping sequence/mapping within sequence. + +[0, 15, 72]: 2018-10-06 + - fix regression on explicit 1.1 loading with the C based scanner/parser + (reported by `Tomas Vavra <https://bitbucket.org/xtomik/>`__) + +[0, 15, 72]: 2018-10-06 + - fix regression on explicit 1.1 loading with the C based scanner/parser + (reported by `Tomas Vavra <https://bitbucket.org/xtomik/>`__) + [0, 15, 71]: 2018-09-26 - fix regression where handcrafted CommentedMaps could not be initiated (reported by `Dan Helfman <https://bitbucket.org/dhelfman/>`__) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/PKG-INFO new/ruamel.yaml-0.15.78/PKG-INFO --- old/ruamel.yaml-0.15.71/PKG-INFO 2018-09-26 23:05:18.000000000 +0200 +++ new/ruamel.yaml-0.15.78/PKG-INFO 2018-11-15 17:57:44.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 1.1 Name: ruamel.yaml -Version: 0.15.71 +Version: 0.15.78 Summary: ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order Home-page: https://bitbucket.org/ruamel/yaml Author: Anthon van der Neut @@ -12,8 +12,8 @@ ``ruamel.yaml`` is a YAML 1.2 loader/dumper package for Python. - :version: 0.15.71 - :updated: 2018-09-26 + :version: 0.15.78 + :updated: 2018-11-15 :documentation: http://yaml.readthedocs.io :repository: https://bitbucket.org/ruamel/ :pypi: https://pypi.org/project/ruamel.yaml/ @@ -27,7 +27,7 @@ but new functionality is likely only to be available via the new API. If your package uses ``ruamel.yaml`` and is not listed on PyPI, drop - me an email, preferably with some infomormation on how you use the + me an email, preferably with some information on how you use the package (or a link to bitbucket/github) and I'll keep you informed when the status of the API is stable enough to make the transition. @@ -62,6 +62,43 @@ .. should insert NEXT: at the beginning of line for next key (with empty line) + 0.15.78 (2018-11-15): + - fix setup issue for 3.8 (reported by `Sidney Kuyateh + <https://bitbucket.org/autinerd/>`__) + + 0.15.77 (2018-11-09): + - setting `yaml.sort_base_mapping_type_on_output = False`, will prevent + explicit sorting by keys in the base representer of mappings. Roundtrip + already did not do this. Usage only makes real sense for Python 3.6+ + (feature request by `Sebastian Gerber <https://bitbucket.org/spacemanspiff2007/>`__). + - implement Python version check in YAML metadata in ``_test/test_z_data.py`` + + 0.15.76 (2018-11-01): + - fix issue with empty mapping and sequence loaded as flow-style + (mapping reported by `Min RK <https://bitbucket.org/minrk/>`__, sequence + by `Maged Ahmed <https://bitbucket.org/maged2/>`__) + + 0.15.75 (2018-10-27): + - fix issue with single '?' scalar (reported by `Terrance + <https://bitbucket.org/OllieTerrance/>`__) + - fix issue with duplicate merge keys (prompted by `answering + <https://stackoverflow.com/a/52852106/1307905>`__ a + `StackOverflow question <https://stackoverflow.com/q/52851168/1307905>`__ + by `math <https://stackoverflow.com/users/1355634/math>`__) + + 0.15.74 (2018-10-17): + - fix dropping of comment on rt before sequence item that is sequence item + (reported by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + + 0.15.73 (2018-10-16): + - fix irregular output on pre-comment in sequence within sequence (reported + by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + - allow non-compact (i.e. next line) dumping sequence/mapping within sequence. + + 0.15.72 (2018-10-06): + - fix regression on explicit 1.1 loading with the C based scanner/parser + (reported by `Tomas Vavra <https://bitbucket.org/xtomik/>`__) + 0.15.71 (2018-09-26): - some of the tests now live in YAML files in the `yaml.data <https://bitbucket.org/ruamel/yaml.data>`__ repository. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/README.rst new/ruamel.yaml-0.15.78/README.rst --- old/ruamel.yaml-0.15.71/README.rst 2018-09-26 23:04:41.000000000 +0200 +++ new/ruamel.yaml-0.15.78/README.rst 2018-11-15 17:41:48.000000000 +0100 @@ -4,8 +4,8 @@ ``ruamel.yaml`` is a YAML 1.2 loader/dumper package for Python. -:version: 0.15.71 -:updated: 2018-09-26 +:version: 0.15.78 +:updated: 2018-11-15 :documentation: http://yaml.readthedocs.io :repository: https://bitbucket.org/ruamel/ :pypi: https://pypi.org/project/ruamel.yaml/ @@ -19,7 +19,7 @@ but new functionality is likely only to be available via the new API. If your package uses ``ruamel.yaml`` and is not listed on PyPI, drop -me an email, preferably with some infomormation on how you use the +me an email, preferably with some information on how you use the package (or a link to bitbucket/github) and I'll keep you informed when the status of the API is stable enough to make the transition. @@ -54,6 +54,43 @@ .. should insert NEXT: at the beginning of line for next key (with empty line) +0.15.78 (2018-11-15): + - fix setup issue for 3.8 (reported by `Sidney Kuyateh + <https://bitbucket.org/autinerd/>`__) + +0.15.77 (2018-11-09): + - setting `yaml.sort_base_mapping_type_on_output = False`, will prevent + explicit sorting by keys in the base representer of mappings. Roundtrip + already did not do this. Usage only makes real sense for Python 3.6+ + (feature request by `Sebastian Gerber <https://bitbucket.org/spacemanspiff2007/>`__). + - implement Python version check in YAML metadata in ``_test/test_z_data.py`` + +0.15.76 (2018-11-01): + - fix issue with empty mapping and sequence loaded as flow-style + (mapping reported by `Min RK <https://bitbucket.org/minrk/>`__, sequence + by `Maged Ahmed <https://bitbucket.org/maged2/>`__) + +0.15.75 (2018-10-27): + - fix issue with single '?' scalar (reported by `Terrance + <https://bitbucket.org/OllieTerrance/>`__) + - fix issue with duplicate merge keys (prompted by `answering + <https://stackoverflow.com/a/52852106/1307905>`__ a + `StackOverflow question <https://stackoverflow.com/q/52851168/1307905>`__ + by `math <https://stackoverflow.com/users/1355634/math>`__) + +0.15.74 (2018-10-17): + - fix dropping of comment on rt before sequence item that is sequence item + (reported by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + +0.15.73 (2018-10-16): + - fix irregular output on pre-comment in sequence within sequence (reported + by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + - allow non-compact (i.e. next line) dumping sequence/mapping within sequence. + +0.15.72 (2018-10-06): + - fix regression on explicit 1.1 loading with the C based scanner/parser + (reported by `Tomas Vavra <https://bitbucket.org/xtomik/>`__) + 0.15.71 (2018-09-26): - some of the tests now live in YAML files in the `yaml.data <https://bitbucket.org/ruamel/yaml.data>`__ repository. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/__init__.py new/ruamel.yaml-0.15.78/__init__.py --- old/ruamel.yaml-0.15.71/__init__.py 2018-09-26 22:55:47.000000000 +0200 +++ new/ruamel.yaml-0.15.78/__init__.py 2018-11-15 17:41:48.000000000 +0100 @@ -7,8 +7,8 @@ _package_data = dict( full_package_name='ruamel.yaml', - version_info=(0, 15, 71), - __version__='0.15.71', + version_info=(0, 15, 78), + __version__='0.15.78', author='Anthon van der Neut', author_email='a.van.der.neut(a)ruamel.eu', description='ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order', # NOQA diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/anchor.py new/ruamel.yaml-0.15.78/anchor.py --- old/ruamel.yaml-0.15.71/anchor.py 1970-01-01 01:00:00.000000000 +0100 +++ new/ruamel.yaml-0.15.78/anchor.py 2018-11-15 15:46:06.000000000 +0100 @@ -0,0 +1,13 @@ + + +anchor_attrib = '_yaml_anchor' + + +class Anchor(object): + __slots__ = 'value', 'always_dump' + attrib = anchor_attrib + + def __init__(self): + # type: () -> None + self.value = None + self.always_dump = False diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/comments.py new/ruamel.yaml-0.15.78/comments.py --- old/ruamel.yaml-0.15.71/comments.py 2018-09-26 08:56:18.000000000 +0200 +++ new/ruamel.yaml-0.15.78/comments.py 2018-11-15 15:46:06.000000000 +0100 @@ -14,6 +14,7 @@ from ruamel.yaml.compat import ordereddict, PY2, string_types, MutableSliceableSequence from ruamel.yaml.scalarstring import ScalarString +from ruamel.yaml.anchor import Anchor if PY2: from collections import MutableSet, Sized, Set, MutableMapping, Mapping @@ -24,15 +25,14 @@ from typing import Any, Dict, Optional, List, Union, Optional, Iterator # NOQA # fmt: off -__all__ = ["CommentedSeq", "CommentedKeySeq", - "CommentedMap", "CommentedOrderedMap", - "CommentedSet", 'comment_attrib', 'merge_attrib'] +__all__ = ['CommentedSeq', 'CommentedKeySeq', + 'CommentedMap', 'CommentedOrderedMap', + 'CommentedSet', 'comment_attrib', 'merge_attrib'] # fmt: on comment_attrib = '_yaml_comment' format_attrib = '_yaml_format' line_col_attrib = '_yaml_line_col' -anchor_attrib = '_yaml_anchor' merge_attrib = '_yaml_merge' tag_attrib = '_yaml_tag' @@ -163,16 +163,6 @@ self.data[key] = data -class Anchor(object): - __slots__ = 'value', 'always_dump' - attrib = anchor_attrib - - def __init__(self): - # type: () -> None - self.value = None - self.always_dump = False - - class Tag(object): """store tag information for roundtripping""" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/compat.py new/ruamel.yaml-0.15.78/compat.py --- old/ruamel.yaml-0.15.71/compat.py 2018-09-21 20:01:48.000000000 +0200 +++ new/ruamel.yaml-0.15.78/compat.py 2018-11-15 15:46:06.000000000 +0100 @@ -186,18 +186,22 @@ class Nprint(object): - def __init__(self): - # type: () -> None + def __init__(self, file_name=None): + # type: (Any) -> None self._max_print = None # type: Any self._count = None # type: Any + self._file_name = file_name def __call__(self, *args, **kw): # type: (Any, Any) -> None if not bool(_debug): return + out = sys.stdout if self._file_name is None else open(self._file_name, 'a') dbgprint = print # to fool checking for print statements by dv utility - dbgprint(*args, **kw) - sys.stdout.flush() + kw1 = kw.copy() + kw1['file'] = out + dbgprint(*args, **kw1) + out.flush() if self._max_print is not None: if self._count is None: self._count = self._max_print @@ -205,8 +209,10 @@ if self._count == 0: dbgprint('forced exit\n') traceback.print_stack() - sys.stdout.flush() + out.flush() sys.exit(0) + if self._file_name: + out.close() def set_max_print(self, i): # type: (int) -> None @@ -215,7 +221,7 @@ nprint = Nprint() - +nprintf = Nprint('/var/tmp/ruamel.yaml.log') # char checkers following production rules diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/composer.py new/ruamel.yaml-0.15.78/composer.py --- old/ruamel.yaml-0.15.71/composer.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/composer.py 2018-11-15 15:46:06.000000000 +0100 @@ -5,7 +5,7 @@ import warnings from ruamel.yaml.error import MarkedYAMLError, ReusedAnchorWarning -from ruamel.yaml.compat import utf8, nprint +from ruamel.yaml.compat import utf8, nprint, nprintf # NOQA from ruamel.yaml.events import ( StreamStartEvent, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/constructor.py new/ruamel.yaml-0.15.78/constructor.py --- old/ruamel.yaml-0.15.71/constructor.py 2018-09-26 22:43:46.000000000 +0200 +++ new/ruamel.yaml-0.15.78/constructor.py 2018-11-15 15:46:06.000000000 +0100 @@ -16,8 +16,8 @@ from ruamel.yaml.nodes import * # NOQA from ruamel.yaml.nodes import (SequenceNode, MappingNode, ScalarNode) from ruamel.yaml.compat import (utf8, builtins_module, to_str, PY2, PY3, # NOQA - ordereddict, text_type, nprint, version_tnf, Hashable, - MutableSequence, MutableMapping) + ordereddict, text_type, nprint, nprintf, version_tnf, + Hashable, MutableSequence, MutableMapping) from ruamel.yaml.comments import * # NOQA from ruamel.yaml.comments import (CommentedMap, CommentedOrderedMap, CommentedSet, CommentedKeySeq, CommentedSeq, TaggedScalar, @@ -353,6 +353,25 @@ while index < len(node.value): key_node, value_node = node.value[index] if key_node.tag == u'tag:yaml.org,2002:merge': + if merge: # double << key + args = [ + 'while constructing a mapping', + node.start_mark, + 'found duplicate key "{}"'.format(key_node.value), + key_node.start_mark, + """ + To suppress this check see: + http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys + """, + """\ + Duplicate keys will become an error in future releases, and are errors + by default when using the new API. + """, + ] + if self.allow_duplicate_keys is None: + warnings.warn(DuplicateKeyFutureWarning(*args)) + else: + raise DuplicateKeyError(*args) del node.value[index] if isinstance(value_node, MappingNode): self.flatten_mapping(value_node) @@ -820,7 +839,7 @@ __import__(module_name) # object_name = '.'.join(object_name) break - except ImportError as exc: + except ImportError: continue else: module_name = builtins_module @@ -1270,11 +1289,30 @@ return value # merge = [] - merge_map_list = [] + merge_map_list = [] # type: List[Any] index = 0 while index < len(node.value): key_node, value_node = node.value[index] if key_node.tag == u'tag:yaml.org,2002:merge': + if merge_map_list: # double << key + args = [ + 'while constructing a mapping', + node.start_mark, + 'found duplicate key "{}"'.format(key_node.value), + key_node.start_mark, + """ + To suppress this check see: + http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys + """, + """\ + Duplicate keys will become an error in future releases, and are errors + by default when using the new API. + """, + ] + if self.allow_duplicate_keys is None: + warnings.warn(DuplicateKeyFutureWarning(*args)) + else: + raise DuplicateKeyError(*args) del node.value[index] if isinstance(value_node, MappingNode): merge_map_list.append((index, constructed(value_node))) @@ -1456,25 +1494,28 @@ # type: (Any) -> Any data = CommentedSeq() data._yaml_set_line_col(node.start_mark.line, node.start_mark.column) - if node.flow_style is True: - data.fa.set_flow_style() - elif node.flow_style is False: - data.fa.set_block_style() if node.comment: data._yaml_add_comment(node.comment) yield data data.extend(self.construct_rt_sequence(node, data)) + self.set_collection_style(data, node) def construct_yaml_map(self, node): # type: (Any) -> Any data = CommentedMap() data._yaml_set_line_col(node.start_mark.line, node.start_mark.column) + yield data + self.construct_mapping(node, data) + self.set_collection_style(data, node) + + def set_collection_style(self, data, node): + # type: (Any, Any) -> None + if len(data) == 0: + return if node.flow_style is True: data.fa.set_flow_style() elif node.flow_style is False: data.fa.set_block_style() - yield data - self.construct_mapping(node, data) def construct_yaml_object(self, node, cls): # type: (Any, Any) -> Any diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/emitter.py new/ruamel.yaml-0.15.78/emitter.py --- old/ruamel.yaml-0.15.71/emitter.py 2018-09-26 22:01:33.000000000 +0200 +++ new/ruamel.yaml-0.15.78/emitter.py 2018-11-15 15:46:06.000000000 +0100 @@ -156,6 +156,9 @@ self.column = 0 self.whitespace = True self.indention = True + self.compact_seq_seq = True # dash after dash + self.compact_seq_map = True # key after dash + # self.compact_ms = False # dash after key, only when excplicit key with ? self.no_newline = None # type: Optional[bool] # set if directly after `- ` # Whether the document requires an explicit document indicator @@ -404,14 +407,17 @@ if isinstance(self.event, ScalarEvent): self.expect_scalar() elif isinstance(self.event, SequenceStartEvent): + # nprintf('@', self.indention, self.no_newline, self.column) + i2, n2 = self.indention, self.no_newline # NOQA if self.event.comment: if self.event.flow_style is False and self.event.comment: if self.write_post_comment(self.event): self.indention = False self.no_newline = True if self.write_pre_comment(self.event): - self.indention = False - self.no_newline = True + pass + self.indention = i2 + self.no_newline = not self.indention if ( self.flow_level or self.canonical @@ -593,7 +599,12 @@ def expect_block_sequence(self): # type: () -> None - indentless = self.mapping_context and not self.indention + if self.mapping_context: + indentless = not self.indention + else: + indentless = False + if not self.compact_seq_seq and self.column != 0: + self.write_line_break() self.increase_indent(flow=False, sequence=True, indentless=indentless) self.state = self.expect_first_block_sequence_item @@ -626,6 +637,8 @@ def expect_block_mapping(self): # type: () -> None + if not self.mapping_context and not (self.compact_seq_map or self.column == 0): + self.write_line_break() self.increase_indent(flow=False, sequence=False) self.state = self.expect_first_block_mapping_key @@ -1011,6 +1024,8 @@ if ch in u'?:': # ToDo if self.serializer.use_version == (1, 1): flow_indicators = True + elif len(scalar) == 1: # single character + flow_indicators = True if followed_by_whitespace: block_indicators = True if ch == u'-' and followed_by_whitespace: @@ -1574,11 +1589,11 @@ breaks = ch in u'\n\x85\u2028\u2029' end += 1 - def write_comment(self, comment): - # type: (Any) -> None + def write_comment(self, comment, pre=False): + # type: (Any, bool) -> None value = comment.value - # nprint('{:02d} {:02d} {!r}'.format(self.column, comment.start_mark.column, value)) - if value[-1] == '\n': + # nprintf('{:02d} {:02d} {!r}'.format(self.column, comment.start_mark.column, value)) + if not pre and value[-1] == '\n': value = value[:-1] try: # get original column position @@ -1607,7 +1622,8 @@ self.stream.write(value) except TypeError: raise - self.write_line_break() + if not pre: + self.write_line_break() def write_pre_comment(self, event): # type: (Any) -> bool @@ -1621,7 +1637,7 @@ continue if self.column != 0: self.write_line_break() - self.write_comment(comment) + self.write_comment(comment, pre=True) if isinstance(event, start_events): comment.pre_done = True except TypeError: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/main.py new/ruamel.yaml-0.15.78/main.py --- old/ruamel.yaml-0.15.71/main.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/main.py 2018-11-15 15:46:06.000000000 +0100 @@ -146,6 +146,9 @@ self.map_indent = None self.sequence_indent = None self.sequence_dash_offset = 0 + self.compact_seq_seq = None + self.compact_seq_map = None + self.sort_base_mapping_type_on_output = None # default: sort self.top_level_colon_align = None self.prefix_colon = None @@ -252,6 +255,10 @@ if self.sequence_dash_offset is not None: _emitter.sequence_dash_offset = self.sequence_dash_offset # _emitter.block_seq_indent = self.sequence_dash_offset + if self.compact_seq_seq is not None: + _emitter.compact_seq_seq = self.compact_seq_seq + if self.compact_seq_map is not None: + _emitter.compact_seq_map = self.compact_seq_map else: if getattr(self, '_stream', None) is None: # wait for the stream @@ -283,15 +290,14 @@ # type: () -> Any attr = '_' + sys._getframe().f_code.co_name if not hasattr(self, attr): - setattr( - self, - attr, - self.Representer( - default_style=self.default_style, - default_flow_style=self.default_flow_style, - dumper=self, - ), + repres = self.Representer( + default_style=self.default_style, + default_flow_style=self.default_flow_style, + dumper=self, ) + if self.sort_base_mapping_type_on_output is not None: + repres.sort_base_mapping_type_on_output = self.sort_base_mapping_type_on_output + setattr(self, attr, repres) return getattr(self, attr) # separate output resolver? @@ -394,17 +400,17 @@ # if you just initialise the CParser, to much of resolver.py # is actually used rslvr = self.Resolver - if rslvr is ruamel.yaml.resolver.VersionedResolver: - rslvr = ruamel.yaml.resolver.Resolver + # if rslvr is ruamel.yaml.resolver.VersionedResolver: + # rslvr = ruamel.yaml.resolver.Resolver class XLoader(self.Parser, self.Constructor, rslvr): # type: ignore - def __init__(selfx, stream, version=None, preserve_quotes=None): + def __init__(selfx, stream, version=self.version, preserve_quotes=None): # type: (StreamTextType, Optional[VersionType], Optional[bool]) -> None # NOQA CParser.__init__(selfx, stream) selfx._parser = selfx._composer = selfx self.Constructor.__init__(selfx, loader=selfx) selfx.allow_duplicate_keys = self.allow_duplicate_keys - rslvr.__init__(selfx, loadumper=selfx) + rslvr.__init__(selfx, version=version, loadumper=selfx) self._stream = stream loader = XLoader(stream) @@ -709,6 +715,11 @@ # type: (Any) -> None self.sequence_dash_offset = val + def compact(self, seq_seq=None, seq_map=None): + # type: (Any, Any) -> None + self.compact_seq_seq = seq_seq + self.compact_seq_map = seq_map + class YAMLContextManager(object): def __init__(self, yaml, transform=None): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/nodes.py new/ruamel.yaml-0.15.78/nodes.py --- old/ruamel.yaml-0.15.71/nodes.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/nodes.py 2018-11-15 15:46:06.000000000 +0100 @@ -12,14 +12,14 @@ class Node(object): __slots__ = 'tag', 'value', 'start_mark', 'end_mark', 'comment', 'anchor' - def __init__(self, tag, value, start_mark, end_mark, comment=None): - # type: (Any, Any, Any, Any, Any) -> None + def __init__(self, tag, value, start_mark, end_mark, comment=None, anchor=None): + # type: (Any, Any, Any, Any, Any, Any) -> None self.tag = tag self.value = value self.start_mark = start_mark self.end_mark = end_mark self.comment = comment - self.anchor = None + self.anchor = anchor def __repr__(self): # type: () -> str @@ -78,14 +78,15 @@ __slots__ = ('style',) id = 'scalar' - def __init__(self, tag, value, start_mark=None, end_mark=None, style=None, comment=None): - # type: (Any, Any, Any, Any, Any, Any) -> None - Node.__init__(self, tag, value, start_mark, end_mark, comment=comment) + def __init__(self, tag, value, start_mark=None, end_mark=None, style=None, comment=None, + anchor=None): + # type: (Any, Any, Any, Any, Any, Any, Any) -> None + Node.__init__(self, tag, value, start_mark, end_mark, comment=comment, anchor=anchor) self.style = style class CollectionNode(Node): - __slots__ = 'flow_style', 'anchor' + __slots__ = ('flow_style', ) def __init__( self, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/parser.py new/ruamel.yaml-0.15.78/parser.py --- old/ruamel.yaml-0.15.71/parser.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/parser.py 2018-11-15 15:46:06.000000000 +0100 @@ -80,7 +80,7 @@ from ruamel.yaml.tokens import * # NOQA from ruamel.yaml.events import * # NOQA from ruamel.yaml.scanner import Scanner, RoundTripScanner, ScannerError # NOQA -from ruamel.yaml.compat import utf8, nprint # NOQA +from ruamel.yaml.compat import utf8, nprint, nprintf # NOQA if False: # MYPY from typing import Any, Dict, Optional, List # NOQA diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/representer.py new/ruamel.yaml-0.15.78/representer.py --- old/ruamel.yaml-0.15.71/representer.py 2018-09-26 22:25:27.000000000 +0200 +++ new/ruamel.yaml-0.15.78/representer.py 2018-11-15 15:46:06.000000000 +0100 @@ -6,6 +6,7 @@ from ruamel.yaml.error import * # NOQA from ruamel.yaml.nodes import * # NOQA from ruamel.yaml.compat import text_type, binary_type, to_unicode, PY2, PY3, ordereddict +from ruamel.yaml.compat import nprint, nprintf # NOQA from ruamel.yaml.scalarstring import ( LiteralScalarString, FoldedScalarString, @@ -64,6 +65,7 @@ self.represented_objects = {} # type: Dict[Any, Any] self.object_keeper = [] # type: List[Any] self.alias_key = None # type: Optional[int] + self.sort_base_mapping_type_on_output = True @property def serializer(self): @@ -208,10 +210,11 @@ best_style = True if hasattr(mapping, 'items'): mapping = list(mapping.items()) - try: - mapping = sorted(mapping) - except TypeError: - pass + if self.sort_base_mapping_type_on_output: + try: + mapping = sorted(mapping) + except TypeError: + pass for item_key, item_value in mapping: node_key = self.represent_key(item_key) node_value = self.represent_data(item_value) @@ -889,17 +892,33 @@ item_comments = {} for idx, item in enumerate(sequence): node_item = self.represent_data(item) - node_item.comment = item_comments.get(idx) + self.merge_comments(node_item, item_comments.get(idx)) if not (isinstance(node_item, ScalarNode) and not node_item.style): best_style = False value.append(node_item) if flow_style is None: - if self.default_flow_style is not None: + if len(sequence) != 0 and self.default_flow_style is not None: node.flow_style = self.default_flow_style else: node.flow_style = best_style return node + def merge_comments(self, node, comments): + # type: (Any, Any) -> Any + if comments is None: + assert hasattr(node, 'comment') + return node + if getattr(node, 'comment', None) is not None: + for idx, val in enumerate(comments): + if idx >= len(node.comment): + continue + nc = node.comment[idx] + if nc is not None: + assert val is None or val == nc + comments[idx] = nc + node.comment = comments + return node + def represent_key(self, data): # type: (Any) -> Any if isinstance(data, CommentedKeySeq): @@ -944,11 +963,13 @@ except AttributeError: item_comments = {} merge_list = [m[1] for m in getattr(mapping, merge_attrib, [])] + item_count = 0 if bool(merge_list): items = mapping.non_merged_items() else: items = mapping.items() for item_key, item_value in items: + item_count += 1 node_key = self.represent_key(item_key) node_value = self.represent_data(item_value) item_comment = item_comments.get(item_key) @@ -967,7 +988,7 @@ best_style = False value.append((node_key, node_value)) if flow_style is None: - if self.default_flow_style is not None: + if ((item_count != 0) or bool(merge_list)) and self.default_flow_style is not None: node.flow_style = self.default_flow_style else: node.flow_style = best_style diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/resolver.py new/ruamel.yaml-0.15.78/resolver.py --- old/ruamel.yaml-0.15.71/resolver.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/resolver.py 2018-11-15 15:46:06.000000000 +0100 @@ -316,8 +316,10 @@ and Yes/No/On/Off booleans. """ - def __init__(self, version=None, loader=None): - # type: (Optional[VersionType], Any) -> None + def __init__(self, version=None, loader=None, loadumper=None): + # type: (Optional[VersionType], Any, Any) -> None + if loader is None and loadumper is not None: + loader = loadumper BaseResolver.__init__(self, loader) self._loader_version = self.get_loader_version(version) self._version_implicit_resolver = {} # type: Dict[Any, Any] @@ -383,10 +385,13 @@ try: version = self.parser.yaml_version except AttributeError: - if hasattr(self.loadumper, 'typ'): - version = self.loadumper.version - else: - version = self.loadumper._serializer.use_version # dumping + try: + if hasattr(self.loadumper, 'typ'): + version = self.loadumper.version + else: + version = self.loadumper._serializer.use_version # dumping + except AttributeError: + version = None if version is None: version = self._loader_version if version is None: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/ruamel.yaml.egg-info/PKG-INFO new/ruamel.yaml-0.15.78/ruamel.yaml.egg-info/PKG-INFO --- old/ruamel.yaml-0.15.71/ruamel.yaml.egg-info/PKG-INFO 2018-09-26 23:05:18.000000000 +0200 +++ new/ruamel.yaml-0.15.78/ruamel.yaml.egg-info/PKG-INFO 2018-11-15 17:57:41.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 1.1 Name: ruamel.yaml -Version: 0.15.71 +Version: 0.15.78 Summary: ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order Home-page: https://bitbucket.org/ruamel/yaml Author: Anthon van der Neut @@ -12,8 +12,8 @@ ``ruamel.yaml`` is a YAML 1.2 loader/dumper package for Python. - :version: 0.15.71 - :updated: 2018-09-26 + :version: 0.15.78 + :updated: 2018-11-15 :documentation: http://yaml.readthedocs.io :repository: https://bitbucket.org/ruamel/ :pypi: https://pypi.org/project/ruamel.yaml/ @@ -27,7 +27,7 @@ but new functionality is likely only to be available via the new API. If your package uses ``ruamel.yaml`` and is not listed on PyPI, drop - me an email, preferably with some infomormation on how you use the + me an email, preferably with some information on how you use the package (or a link to bitbucket/github) and I'll keep you informed when the status of the API is stable enough to make the transition. @@ -62,6 +62,43 @@ .. should insert NEXT: at the beginning of line for next key (with empty line) + 0.15.78 (2018-11-15): + - fix setup issue for 3.8 (reported by `Sidney Kuyateh + <https://bitbucket.org/autinerd/>`__) + + 0.15.77 (2018-11-09): + - setting `yaml.sort_base_mapping_type_on_output = False`, will prevent + explicit sorting by keys in the base representer of mappings. Roundtrip + already did not do this. Usage only makes real sense for Python 3.6+ + (feature request by `Sebastian Gerber <https://bitbucket.org/spacemanspiff2007/>`__). + - implement Python version check in YAML metadata in ``_test/test_z_data.py`` + + 0.15.76 (2018-11-01): + - fix issue with empty mapping and sequence loaded as flow-style + (mapping reported by `Min RK <https://bitbucket.org/minrk/>`__, sequence + by `Maged Ahmed <https://bitbucket.org/maged2/>`__) + + 0.15.75 (2018-10-27): + - fix issue with single '?' scalar (reported by `Terrance + <https://bitbucket.org/OllieTerrance/>`__) + - fix issue with duplicate merge keys (prompted by `answering + <https://stackoverflow.com/a/52852106/1307905>`__ a + `StackOverflow question <https://stackoverflow.com/q/52851168/1307905>`__ + by `math <https://stackoverflow.com/users/1355634/math>`__) + + 0.15.74 (2018-10-17): + - fix dropping of comment on rt before sequence item that is sequence item + (reported by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + + 0.15.73 (2018-10-16): + - fix irregular output on pre-comment in sequence within sequence (reported + by `Thorsten Kampe <https://bitbucket.org/thorstenkampe/>`__) + - allow non-compact (i.e. next line) dumping sequence/mapping within sequence. + + 0.15.72 (2018-10-06): + - fix regression on explicit 1.1 loading with the C based scanner/parser + (reported by `Tomas Vavra <https://bitbucket.org/xtomik/>`__) + 0.15.71 (2018-09-26): - some of the tests now live in YAML files in the `yaml.data <https://bitbucket.org/ruamel/yaml.data>`__ repository. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/ruamel.yaml.egg-info/SOURCES.txt new/ruamel.yaml-0.15.78/ruamel.yaml.egg-info/SOURCES.txt --- old/ruamel.yaml-0.15.71/ruamel.yaml.egg-info/SOURCES.txt 2018-09-26 23:05:18.000000000 +0200 +++ new/ruamel.yaml-0.15.78/ruamel.yaml.egg-info/SOURCES.txt 2018-11-15 17:57:41.000000000 +0100 @@ -5,6 +5,7 @@ setup.py ./LICENSE ./__init__.py +./anchor.py ./comments.py ./compat.py ./composer.py diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/scalarstring.py new/ruamel.yaml-0.15.78/scalarstring.py --- old/ruamel.yaml-0.15.71/scalarstring.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/scalarstring.py 2018-11-15 15:46:06.000000000 +0100 @@ -3,6 +3,7 @@ from __future__ import print_function, absolute_import, division, unicode_literals from ruamel.yaml.compat import text_type +from ruamel.yaml.anchor import Anchor if False: # MYPY from typing import Text, Any, Dict, List # NOQA @@ -20,7 +21,7 @@ class ScalarString(text_type): - __slots__ = () + __slots__ = (Anchor.attrib) def __new__(cls, *args, **kw): # type: (Any, Any) -> Any @@ -30,6 +31,26 @@ # type: (Any, Any, int) -> Any return type(self)((text_type.replace(self, old, new, maxreplace))) + @property + def anchor(self): + # type: () -> Any + if not hasattr(self, Anchor.attrib): + setattr(self, Anchor.attrib, Anchor()) + return getattr(self, Anchor.attrib) + + def yaml_anchor(self): + # type: () -> Any + if not hasattr(self, Anchor.attrib): + return None + if not self.anchor.always_dump: + return None + return self.anchor + + def yaml_set_anchor(self, value, always_dump=False): + # type: (Any, bool) -> None + self.anchor.value = value + self.anchor.always_dump = always_dump + class LiteralScalarString(ScalarString): __slots__ = 'comment' # the comment after the | on the first line diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/serializer.py new/ruamel.yaml-0.15.78/serializer.py --- old/ruamel.yaml-0.15.71/serializer.py 2018-09-21 11:00:21.000000000 +0200 +++ new/ruamel.yaml-0.15.78/serializer.py 2018-11-15 15:46:06.000000000 +0100 @@ -3,7 +3,7 @@ from __future__ import absolute_import from ruamel.yaml.error import YAMLError -from ruamel.yaml.compat import nprint, DBG_NODE, dbg, string_types +from ruamel.yaml.compat import nprint, DBG_NODE, dbg, string_types, nprintf # NOQA from ruamel.yaml.util import RegExp from ruamel.yaml.events import ( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ruamel.yaml-0.15.71/setup.py new/ruamel.yaml-0.15.78/setup.py --- old/ruamel.yaml-0.15.71/setup.py 2018-09-26 09:06:04.000000000 +0200 +++ new/ruamel.yaml-0.15.78/setup.py 2018-11-15 17:41:33.000000000 +0100 @@ -44,6 +44,10 @@ class NameConstant: pass +if sys.version_info >= (3, 8): + + from ast import Str, Num, Bytes, NameConstant # NOQA + if sys.version_info < (3,): open_kw = dict() @@ -830,7 +834,7 @@ sources=[self.pn(x) for x in target['src']], libraries=[self.pn(x) for x in target.get('lib')], ) - # debug('test in target', 'test' in target, target) + # debug('test1 in target', 'test' in target, target) if 'test' not in target: # no test, just hope it works self._ext_modules.append(ext) continue @@ -871,8 +875,8 @@ print('compile error:', file_name) continue except LinkError: - debug('libyaml link error', file_name) - print('libyaml link error', file_name) + debug('link error', file_name) + print('link error', file_name) continue self._ext_modules.append(ext) except Exception as e: # NOQA @@ -893,10 +897,10 @@ https://bitbucket.org/pypa/wheel/issues/47 """ if 'bdist_wheel' not in sys.argv: - return + return False file_name = 'setup.cfg' if os.path.exists(file_name): # add it if not in there? - return + return False with open(file_name, 'w') as fp: if os.path.exists('LICENSE'): fp.write('[metadata]\nlicense-file = LICENSE\n') @@ -964,6 +968,8 @@ for k in sorted(kw): v = kw[k] print(' "{0}": "{1}",'.format(k, v)) + # if '--record' in sys.argv: + # return if dump_kw in sys.argv: sys.argv.remove(dump_kw) try: @@ -971,7 +977,26 @@ kw['long_description'] = fp.read() except Exception: pass + if nsp.wheel(kw, setup): + if nsp.nested and 'bdist_wheel' in sys.argv: + try: + d = sys.argv[sys.argv.index('-d') + 1] + except ValueError: + dist_base = os.environ.get('PYDISTBASE') + if dist_base: + d = os.path.join(dist_base, nsp.full_package_name) + else: + d = 'dist' + for x in os.listdir(d): + dashed_vs = '-' + version_str + '-' + if x.endswith('.whl') and dashed_vs in x: + # remove .pth file from the wheel + full_name = os.path.join(d, x) + print('patching .pth from', full_name) + with InMemoryZipFile(full_name) as imz: + imz.delete_from_zip_file(nsp.full_package_name + '.*.pth') + break return for x in ['-c', 'egg_info', '--egg-base', 'pip-egg-info']: if x not in sys.argv: @@ -1002,6 +1027,7 @@ if x.endswith('.whl'): # remove .pth file from the wheel full_name = os.path.join(d, x) + print('patching .pth from', full_name) with InMemoryZipFile(full_name) as imz: imz.delete_from_zip_file(nsp.full_package_name + '.*.pth') break

1 0

commit translation-update for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package translation-update for openSUSE:Factory checked in at 2018-12-03 10:09:39 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/translation-update (Old) and /work/SRC/openSUSE:Factory/.translation-update.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "translation-update" Mon Dec 3 10:09:39 2018 rev:24 rq:653083 version:15.1 Changes: -------- --- /work/SRC/openSUSE:Factory/translation-update/translation-update.changes 2018-07-10 16:17:37.189332781 +0200 +++ /work/SRC/openSUSE:Factory/.translation-update.new.19453/translation-update.changes 2018-12-03 10:09:44.115745304 +0100 @@ -1,0 +2,11 @@ +Thu Nov 29 18:08:18 CET 2018 - sbrabec(a)suse.com + +- Update to Factory specific version 20181128 that contains only + abandoned packages in static.tlst (bsc#1100398). +- Obsolete all language packages that are no longer distributed + (boo#952855#c41) and add support for it in supplementary scripts +- translation-update-spec-generate-lang-list.sh: Fix Obsoletes + generator. +- Drop many unneeded subpackages. + +------------------------------------------------------------------- Old: ---- translation-update-from-translation-update-upstream-20180426.tar.bz2 New: ---- translation-update-from-translation-update-upstream-20181128.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ translation-update.spec ++++++ ++++ 1566 lines (skipped) ++++ between /work/SRC/openSUSE:Factory/translation-update/translation-update.spec ++++ and /work/SRC/openSUSE:Factory/.translation-update.new.19453/translation-update.spec ++++++ translation-update-from-translation-update-upstream-20180426.tar.bz2 -> translation-update-from-translation-update-upstream-20181128.tar.bz2 ++++++ /work/SRC/openSUSE:Factory/translation-update/translation-update-from-translation-update-upstream-20180426.tar.bz2 /work/SRC/openSUSE:Factory/.translation-update.new.19453/translation-update-from-translation-update-upstream-20181128.tar.bz2 differ: char 11, line 1 ++++++ translation-update-spec-generate-lang-list.sh ++++++ --- /var/tmp/diff_new_pack.jvx1BM/_old 2018-12-03 10:09:45.235744270 +0100 +++ /var/tmp/diff_new_pack.jvx1BM/_new 2018-12-03 10:09:45.239744267 +0100 @@ -34,6 +34,7 @@ LNG_NAME=${LNG_NAME% (http*} LNG_NAME=${LNG_NAME% Team} LNG_NAME=${LNG_NAME% Translation} + LNG_NAME=${LNG_NAME% Translation Project} LNG_NAME=${LNG_NAME#GNOME } LNG_NAME=${LNG_NAME#Gnome } LNG_NAME=${LNG_NAME% GNOME} @@ -62,10 +63,10 @@ # Errorneous cases. case "$LNG_NAME" in # nb - # Do not accept "Novell Language" generated by Novell translation tools. + # Do not accept company or tool name. # Do not accept underbar. It means that translator filled LANG variable name. # Do not accept e-mails. - "Kjartan Maraas" | "Novell Language" | linux* | *_* | *@* | */* ) + "Kjartan Maraas" | "Novell Language" | "AgreeYa Solutions" | linux* | *_* | *@* | */* ) continue ;; esac @@ -117,7 +118,6 @@ EOF done >translation-update.spec.files.tmp -bash ./translation-update-spec-reset-lang-list.sh sed -i ' /^%prep$/{ r translation-update.spec.preamble.tmp

1 0

commit webkit2gtk3 for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package webkit2gtk3 for openSUSE:Factory checked in at 2018-12-03 10:09:25 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/webkit2gtk3 (Old) and /work/SRC/openSUSE:Factory/.webkit2gtk3.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "webkit2gtk3" Mon Dec 3 10:09:25 2018 rev:67 rq:653432 version:2.22.4 Changes: -------- --- /work/SRC/openSUSE:Factory/webkit2gtk3/webkit2gtk3.changes 2018-11-18 23:24:47.526018458 +0100 +++ /work/SRC/openSUSE:Factory/.webkit2gtk3.new.19453/webkit2gtk3.changes 2018-12-03 10:09:27.747760403 +0100 @@ -1,0 +2,16 @@ +Wed Nov 28 10:47:03 UTC 2018 - schwab(a)suse.de + +- Reduce memory constraints for riscv64 + +------------------------------------------------------------------- +Thu Nov 22 12:02:43 UTC 2018 - bjorn.lie(a)gmail.com + +- Update to version 2.22.4: + + Expose ENABLE_MEDIA_SOURCE as a public build option. + + Fix a crash when using Cairo versions between 1.15 and 1.16.0 + + Fix the build with -DLOG_DISABLED=0. + + Fix the build with ENABLE_VIDEO=OFF and ENABLE_WEB_AUDIO=OFF. + + Fix debug builds of JavaScriptCore. + + Fix several crashes and rendering issues. + +------------------------------------------------------------------- Old: ---- webkitgtk-2.22.3.tar.xz webkitgtk-2.22.3.tar.xz.asc New: ---- webkitgtk-2.22.4.tar.xz webkitgtk-2.22.4.tar.xz.asc ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ webkit2gtk3.spec ++++++ --- /var/tmp/diff_new_pack.WVNyz2/_old 2018-12-03 10:09:39.307749740 +0100 +++ /var/tmp/diff_new_pack.WVNyz2/_new 2018-12-03 10:09:39.311749736 +0100 @@ -42,7 +42,7 @@ %bcond_with python3 %endif Name: webkit2gtk3 -Version: 2.22.3 +Version: 2.22.4 Release: 0 Summary: Library for rendering web content, GTK+ Port License: LGPL-2.0-or-later AND BSD-3-Clause @@ -139,8 +139,8 @@ %package -n libwebkit2gtk%{_wk2sover} Summary: Library for rendering web content, GTK+ Port -# Require the injected bundles. The bundles are dlopen()ed Group: System/Libraries +# Require the injected bundles. The bundles are dlopen()ed Requires: webkit2gtk-4_0-injected-bundles Recommends: %{_pkgname_no_slpp}-lang Provides: %{_pkgname_no_slpp} = %{version} ++++++ _constraints ++++++ --- /var/tmp/diff_new_pack.WVNyz2/_old 2018-12-03 10:09:39.347749703 +0100 +++ /var/tmp/diff_new_pack.WVNyz2/_new 2018-12-03 10:09:39.347749703 +0100 @@ -18,4 +18,14 @@ </physicalmemory> </hardware> </overwrite> + <overwrite> + <conditions> + <arch>riscv64</arch> + </conditions> + <hardware> + <physicalmemory> + <size unit="M">7250</size> + </physicalmemory> + </hardware> + </overwrite> </constraints> ++++++ webkitgtk-2.22.3.tar.xz -> webkitgtk-2.22.4.tar.xz ++++++ /work/SRC/openSUSE:Factory/webkit2gtk3/webkitgtk-2.22.3.tar.xz /work/SRC/openSUSE:Factory/.webkit2gtk3.new.19453/webkitgtk-2.22.4.tar.xz differ: char 26, line 1

1 0

commit checkmedia for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package checkmedia for openSUSE:Factory checked in at 2018-12-03 10:09:20 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/checkmedia (Old) and /work/SRC/openSUSE:Factory/.checkmedia.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "checkmedia" Mon Dec 3 10:09:20 2018 rev:31 rq:652463 version:4.1 Changes: -------- --- /work/SRC/openSUSE:Factory/checkmedia/checkmedia.changes 2018-07-02 23:29:35.833536131 +0200 +++ /work/SRC/openSUSE:Factory/.checkmedia.new.19453/checkmedia.changes 2018-12-03 10:09:22.499765243 +0100 @@ -1,0 +2,9 @@ +Wed Nov 28 15:35:46 UTC 2018 - snwint(a)suse.de + +- merge gh#openSUSE/checkmedia#8 +- fix digest calculation in tagmedia (bsc#1117499) +- added testcases +- adjust function descriptions +- 4.1 + +-------------------------------------------------------------------- Old: ---- checkmedia-4.0.tar.xz New: ---- checkmedia-4.1.tar.xz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ checkmedia.spec ++++++ --- /var/tmp/diff_new_pack.7aUw9T/_old 2018-12-03 10:09:22.971764807 +0100 +++ /var/tmp/diff_new_pack.7aUw9T/_new 2018-12-03 10:09:22.975764804 +0100 @@ -22,7 +22,7 @@ Summary: Check SUSE installation media License: GPL-3.0-or-later Group: System/Management -Version: 4.0 +Version: 4.1 Release: 0 Url: https://github.com/openSUSE/checkmedia Source: %{name}-%{version}.tar.xz ++++++ checkmedia-4.0.tar.xz -> checkmedia-4.1.tar.xz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/VERSION new/checkmedia-4.1/VERSION --- old/checkmedia-4.0/VERSION 2018-06-29 14:17:02.000000000 +0200 +++ new/checkmedia-4.1/VERSION 2018-11-28 16:35:46.000000000 +0100 @@ -1 +1 @@ -4.0 +4.1 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/changelog new/checkmedia-4.1/changelog --- old/checkmedia-4.0/changelog 2018-06-29 14:17:02.000000000 +0200 +++ new/checkmedia-4.1/changelog 2018-11-28 16:35:46.000000000 +0100 @@ -1,3 +1,9 @@ +2018-11-28: 4.1 + - merge gh#openSUSE/checkmedia#8 + - fix digest calculation in tagmedia (bsc#1117499) + - added testcases + - adjust function descriptions + 2018-06-29: 4.0 - merge gh#openSUSE/checkmedia#6 - change tagmedia to also store checksum over partition diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tagmedia new/checkmedia-4.1/tagmedia --- old/checkmedia-4.0/tagmedia 2018-06-29 14:17:02.000000000 +0200 +++ new/checkmedia-4.1/tagmedia 2018-11-28 16:35:46.000000000 +0100 @@ -80,6 +80,7 @@ my $opt_clean = 0; my @opt_add_tag; my @opt_remove_tag; +my $opt_verbose; GetOptions( 'show' => \$opt_show, @@ -90,6 +91,7 @@ 'add-tag=s' => \@opt_add_tag, 'remove-tag=s' => \@opt_remove_tag, 'clean' => \$opt_clean, + 'verbose|v' => \$opt_verbose, 'help' => sub { usage 0 }, ); @@ -269,6 +271,8 @@ } } + print "iso blocks = $image->{iso_blocks}, partition blocks = $image->{part_blocks} @ $image->{part_start}\n" if $opt_verbose; + die "$image->{name}: unsupported image format\n" unless $image->{iso_blocks} || $image->{part_blocks}; } @@ -466,31 +470,45 @@ # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -# add_to_digest(image, blocks, to_iso, to_part) +# process_digest(digest, start, blocks, buf_start, buf_blocks, buf) +# +# Update digest. # -# Update digest reading blocks from $image->{fh}. +# The total digest needs to be calculated from start to start + blocks. The +# current buffer in buf holds file data from buf_start to buf_start + buf_blocks. # -# blocks: number of blocks to read -# to_iso: update $digest_iso -# to_part: update $digest_part +# digest: digest ref +# start: region start block +# blocks: region blocks +# buf_start: buffer start block +# buf_blocks: buffer blocks +# buf: buffer ref # # Note: a block is 0.5 kB. # -sub add_to_digest +sub process_digest { - my ($image, $blocks, $to_iso, $to_part) = @_; + my ($digest, $start, $blocks, $buf_start, $buf_blocks, $buf) = @_; - my $buf; + my $end = $start + $blocks; + my $buf_end = $buf_start + $buf_blocks; + + return if $start >= $buf_end || $end <= $buf_start; - while($blocks > 0) { - # try to read up to 1 MB at a time - my $len = $blocks > (1 << 11) ? 1 << 11 : $blocks; - my $read_len = sysread $image->{fh}, $buf, $len << 9; - die "$image->{name}: read error\n" if $read_len != $len << 9; - $digest_iso->add($buf) if $to_iso && $image->{iso_blocks}; - $digest_part->add($buf) if $to_part && $image->{part_blocks}; - $blocks -= $len; + if($start > $buf_start) { + my $skip = $start - $buf_start; + $buf_blocks -= $skip; + $buf_start = $start; + $buf = substr($buf, $skip << 9); } + + if($buf_end > $end) { + $buf_blocks -= $buf_end - $end; + $buf_end = $end; + $buf = substr($buf, 0, $buf_blocks << 9); + } + + $digest->add($buf); } @@ -509,52 +527,38 @@ # # $image->{blob_blocks} have already been read and are cached in $image->{blob}. # -# Both regions partly overlap and region 2 can overlap $image->{blob}. -# -# This function makes some effort to read the file in only one go. -# # Note: padding has been subtracted from $image->{iso_blocks}. # sub calculate_digest { my ($image) = @_; - $digest_iso->add($image->{blob}) if $image->{iso_blocks}; + my $iso_start = 0; + my $iso_blocks = $image->{iso_blocks}; + my $part_start = $image->{part_start}; + my $part_blocks = $image->{part_blocks}; - my $next_stop = $image->{iso_blocks}; - my $idx = $image->{blob_blocks}; + my $full_blocks = $part_start + $part_blocks; + $full_blocks = $iso_blocks if $iso_blocks > $full_blocks; - my $iso_rem; - my $part_rem; + my $pos = $image->{blob_blocks}; + my $step_blocks = 2048; # 1 MiB chunks - if($image->{part_start}) { - if($image->{part_start} < $image->{blob_blocks}) { - $digest_part->add( - substr($image->{blob}, $image->{part_start} << 9, ($image->{blob_blocks} - $image->{part_start}) << 9) - ); - $idx = $image->{blob_blocks}; - } - else { - add_to_digest $image, $image->{part_start} - $image->{blob_blocks}, 1, 0; - $idx = $image->{part_start}; - } + process_digest $digest_iso, $iso_start, $iso_blocks, 0, $pos, $image->{blob}; + process_digest $digest_part, $part_start, $part_blocks, 0, $pos, $image->{blob}; - $next_stop = $image->{part_start} + $image->{part_blocks}; - $next_stop = $image->{iso_blocks} if $image->{iso_blocks} && $image->{iso_blocks} < $next_stop; + while($pos < $full_blocks) { + my $buf; + my $to_read = $full_blocks - $pos; + $to_read = $step_blocks if $step_blocks < $to_read; - $part_rem = $image->{part_start} + $image->{part_blocks} - $next_stop; - } - - add_to_digest $image, $next_stop - $idx, 1, 1; + my $read_len = sysread $image->{fh}, $buf, $to_read << 9; + die "$image->{name}: read error: $to_read blocks @ $pos\n" if $read_len != $to_read << 9; - $iso_rem = $image->{iso_blocks} - $next_stop if $image->{iso_blocks}; + process_digest $digest_iso, $iso_start, $iso_blocks, $pos, $to_read, $buf; + process_digest $digest_part, $part_start, $part_blocks, $pos, $to_read, $buf; - if($part_rem) { - die "oops, assumption failed\n" if $iso_rem; - add_to_digest $image, $part_rem, 0, 1; - } - elsif($iso_rem) { - add_to_digest $image, $iso_rem, 1, 0; + $pos += $to_read; } if($image->{iso_blocks}) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/testmediacheck new/checkmedia-4.1/testmediacheck --- old/checkmedia-4.0/testmediacheck 2018-06-29 14:17:02.000000000 +0200 +++ new/checkmedia-4.1/testmediacheck 2018-11-28 16:35:46.000000000 +0100 @@ -144,6 +144,36 @@ part_blocks => 998, }, + { + name => "iso_too_small_ends_before_partition_start", + digest => "sha256", + full_blocks => 1000, + iso_blocks => 600, + pad_blocks => 100, + part_start => 700, + part_blocks => 300, + }, + + { + name => "iso_too_small_ends_at_partition_start", + digest => "sha256", + full_blocks => 1000, + iso_blocks => 700, + pad_blocks => 100, + part_start => 700, + part_blocks => 300, + }, + + { + name => "iso_too_small_ends_after_partition_start", + digest => "sha256", + full_blocks => 1000, + iso_blocks => 800, + pad_blocks => 100, + part_start => 700, + part_blocks => 300, + }, + ]; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_after_partition_start.sha256.check.ref new/checkmedia-4.1/tests/iso_too_small_ends_after_partition_start.sha256.check.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_after_partition_start.sha256.check.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_after_partition_start.sha256.check.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,15 @@ + tags: key = "pad", value = "25" + tags: key = "sha256sum", value = "624430d33fff980caf6e9801e119719419932f7399e9a73169d60ca61268ba6e" + tags: key = "partition", value = "700,300,a4e0d25439ddfc8f8e353fc94f02688bf86ea6987872d034e884dd3125b47ef6" + app: iso_too_small_ends_after_partition_start + iso size: 400 kB + pad: 50 kB + partition: start 350 kB, size 150 kB + full size: 500 kB + iso ref: 624430d33fff980caf6e9801e119719419932f7399e9a73169d60ca61268ba6e + part ref: a4e0d25439ddfc8f8e353fc94f02688bf86ea6987872d034e884dd3125b47ef6 + checking: 0% 12% 25% 38% 51% 64% 76% 89%100% + result: iso sha256 ok, partition sha256 ok + iso sha256: 624430d33fff980caf6e9801e119719419932f7399e9a73169d60ca61268ba6e +part sha256: a4e0d25439ddfc8f8e353fc94f02688bf86ea6987872d034e884dd3125b47ef6 + sha256: 0305e1d8b51dd38285f36990a615f4d7229a2862a0d901e4107900ddf5212919 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_after_partition_start.sha256.tag.ref new/checkmedia-4.1/tests/iso_too_small_ends_after_partition_start.sha256.tag.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_after_partition_start.sha256.tag.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_after_partition_start.sha256.tag.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,3 @@ +pad = 25 +sha256sum = 624430d33fff980caf6e9801e119719419932f7399e9a73169d60ca61268ba6e +partition = 700,300,a4e0d25439ddfc8f8e353fc94f02688bf86ea6987872d034e884dd3125b47ef6 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_at_partition_start.sha256.check.ref new/checkmedia-4.1/tests/iso_too_small_ends_at_partition_start.sha256.check.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_at_partition_start.sha256.check.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_at_partition_start.sha256.check.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,15 @@ + tags: key = "pad", value = "25" + tags: key = "sha256sum", value = "b19501e0668c87a857877c84d1514e3c73393fd97f371903d1555e21b4582359" + tags: key = "partition", value = "700,300,5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8" + app: iso_too_small_ends_at_partition_start + iso size: 350 kB + pad: 50 kB + partition: start 350 kB, size 150 kB + full size: 500 kB + iso ref: b19501e0668c87a857877c84d1514e3c73393fd97f371903d1555e21b4582359 + part ref: 5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8 + checking: 0% 12% 25% 38% 51% 64% 76% 89%100% + result: iso sha256 ok, partition sha256 ok + iso sha256: b19501e0668c87a857877c84d1514e3c73393fd97f371903d1555e21b4582359 +part sha256: 5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8 + sha256: d5bbdb09b68c6be36b030a1cf9551fde826881d337290b37de67f3a5877c1896 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_at_partition_start.sha256.tag.ref new/checkmedia-4.1/tests/iso_too_small_ends_at_partition_start.sha256.tag.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_at_partition_start.sha256.tag.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_at_partition_start.sha256.tag.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,3 @@ +pad = 25 +sha256sum = b19501e0668c87a857877c84d1514e3c73393fd97f371903d1555e21b4582359 +partition = 700,300,5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_before_partition_start.sha256.check.ref new/checkmedia-4.1/tests/iso_too_small_ends_before_partition_start.sha256.check.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_before_partition_start.sha256.check.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_before_partition_start.sha256.check.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,15 @@ + tags: key = "pad", value = "25" + tags: key = "sha256sum", value = "670a5364e2dbd0353bfdcc4fd285b345366e55eb66708b1f3d728f9a31c30b6f" + tags: key = "partition", value = "700,300,5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8" + app: iso_too_small_ends_before_partition_start + iso size: 300 kB + pad: 50 kB + partition: start 350 kB, size 150 kB + full size: 500 kB + iso ref: 670a5364e2dbd0353bfdcc4fd285b345366e55eb66708b1f3d728f9a31c30b6f + part ref: 5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8 + checking: 0% 12% 25% 38% 51% 64% 76% 89%100% + result: iso sha256 ok, partition sha256 ok + iso sha256: 670a5364e2dbd0353bfdcc4fd285b345366e55eb66708b1f3d728f9a31c30b6f +part sha256: 5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8 + sha256: 9c84b2c850265959879166902195d88853ffc4ffe71559833ecde1be64805b3d diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/checkmedia-4.0/tests/iso_too_small_ends_before_partition_start.sha256.tag.ref new/checkmedia-4.1/tests/iso_too_small_ends_before_partition_start.sha256.tag.ref --- old/checkmedia-4.0/tests/iso_too_small_ends_before_partition_start.sha256.tag.ref 1970-01-01 01:00:00.000000000 +0100 +++ new/checkmedia-4.1/tests/iso_too_small_ends_before_partition_start.sha256.tag.ref 2018-11-28 16:35:46.000000000 +0100 @@ -0,0 +1,3 @@ +pad = 25 +sha256sum = 670a5364e2dbd0353bfdcc4fd285b345366e55eb66708b1f3d728f9a31c30b6f +partition = 700,300,5e54749a2cd7cf135d8469df48f3047124527e21c138049980941c3153e5e0d8

1 0

commit gnutls for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package gnutls for openSUSE:Factory checked in at 2018-12-03 10:09:09 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/gnutls (Old) and /work/SRC/openSUSE:Factory/.gnutls.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "gnutls" Mon Dec 3 10:09:09 2018 rev:115 rq:652451 version:3.6.4 Changes: -------- --- /work/SRC/openSUSE:Factory/gnutls/gnutls.changes 2018-10-25 08:11:24.528224690 +0200 +++ /work/SRC/openSUSE:Factory/.gnutls.new.19453/gnutls.changes 2018-12-03 10:09:16.167771081 +0100 @@ -1,0 +2,9 @@ +Tue Nov 27 13:46:27 UTC 2018 - jbrielmaier(a)suse.de + +- search for guile-2.2 during configure, part of boo#1117121 + add patches: + * gnutls-enbale-guile-2.2.patch: search for guile-2.2 + refresh patches: + * disable-psk-file-test.patch: disable psk-file in Makefile.am + +------------------------------------------------------------------- New: ---- gnutls-enbale-guile-2.2.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ gnutls.spec ++++++ --- /var/tmp/diff_new_pack.yEgkor/_old 2018-12-03 10:09:17.119770203 +0100 +++ /var/tmp/diff_new_pack.yEgkor/_new 2018-12-03 10:09:17.127770196 +0100 @@ -42,6 +42,8 @@ Patch1: gnutls-3.5.11-skip-trust-store-tests.patch Patch2: gnutls-3.6.0-disable-flaky-dtls_resume-test.patch Patch3: disable-psk-file-test.patch +# Search for guile-2.2, which is supported since 3.5.5 +Patch4: gnutls-enbale-guile-2.2.patch BuildRequires: autogen BuildRequires: automake BuildRequires: datefudge @@ -161,6 +163,7 @@ %setup -q %patch1 -p1 %patch3 -p1 +%patch4 -p1 # dtls-resume test fails on PPC %ifarch ppc64 ppc64le ppc %patch2 -p1 ++++++ disable-psk-file-test.patch ++++++ --- /var/tmp/diff_new_pack.yEgkor/_old 2018-12-03 10:09:17.155770170 +0100 +++ /var/tmp/diff_new_pack.yEgkor/_new 2018-12-03 10:09:17.155770170 +0100 @@ -105,3 +105,15 @@ -rm -f ./$(DEPDIR)/pskself.Po -rm -f ./$(DEPDIR)/pubkey-import-export.Po -rm -f ./$(DEPDIR)/random-art.Po +diff --git a/tests/Makefile.am b/tests/Makefile.am +--- a/tests/Makefile.am 2018-11-21 16:31:27.871806950 +0100 ++++ b/tests/Makefile.am 2018-11-21 16:31:47.952191845 +0100 +@@ -167,7 +167,7 @@ + tls13-cert-key-exchange x509-cert-callback-ocsp gnutls_ocsp_resp_list_import2 \ + server-sign-md5-rep privkey-keygen mini-tls-nonblock no-signal pkcs7-gen dtls-etm \ + x509sign-verify-rsa x509sign-verify-ecdsa x509sign-verify-gost \ +- mini-alignment oids atfork prf psk-file priority-init2 \ ++ mini-alignment oids atfork prf priority-init2 \ + status-request status-request-ok status-request-missing sign-verify-ext \ + fallback-scsv pkcs8-key-decode urls dtls-rehandshake-cert \ + key-usage-rsa key-usage-ecdhe-rsa mini-session-verify-function auto-verify \ ++++++ gnutls-enbale-guile-2.2.patch ++++++ --- gnutls-3.6.4/aclocal.m4.orig 2018-10-16 17:52:16.972960988 +0200 +++ gnutls-3.6.4/aclocal.m4 2018-10-16 17:52:32.797099492 +0200 @@ -162,7 +162,7 @@ # AC_DEFUN([GUILE_PKG], [PKG_PROG_PKG_CONFIG - _guile_versions_to_search="m4_default([$1], [2.0 1.8])" + _guile_versions_to_search="m4_default([$1], [2.2 2.0 1.8])" if test -n "$GUILE_EFFECTIVE_VERSION"; then _guile_tmp="" for v in $_guile_versions_to_search; do --- gnutls-3.6.4/configure.orig 2018-10-16 18:00:13.661141247 +0200 +++ gnutls-3.6.4/configure 2018-10-16 18:00:29.857283556 +0200 @@ -62704,7 +62704,7 @@ PKG_CONFIG="" fi fi - _guile_versions_to_search="2.0 1.8" + _guile_versions_to_search="2.2 2.0 1.8" if test -n "$GUILE_EFFECTIVE_VERSION"; then _guile_tmp="" for v in $_guile_versions_to_search; do

1 0

commit zypper for openSUSE:Factory
by root 03 Dec '18

03 Dec '18

Hello community, here is the log from the commit of package zypper for openSUSE:Factory checked in at 2018-12-03 10:09:04 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/zypper (Old) and /work/SRC/openSUSE:Factory/.zypper.new.19453 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "zypper" Mon Dec 3 10:09:04 2018 rev:290 rq:652441 version:1.14.17 Changes: -------- --- /work/SRC/openSUSE:Factory/zypper/zypper.changes 2018-11-26 10:16:32.753996901 +0100 +++ /work/SRC/openSUSE:Factory/.zypper.new.19453/zypper.changes 2018-12-03 10:09:06.915779609 +0100 @@ -1,0 +2,8 @@ +Wed Nov 28 14:47:16 CET 2018 - ma(a)suse.de + +- Optionally run "zypper search-packages" after "search" (FATE#325599) +- zypper.conf: Add [search]runSearchPackages config variable. +- BuildRequires: libzypp-devel >= 17.10.0 +- version 1.14.17 + +------------------------------------------------------------------- Old: ---- zypper-1.14.16.tar.bz2 New: ---- zypper-1.14.17.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ zypper.spec ++++++ --- /var/tmp/diff_new_pack.y76CmB/_old 2018-12-03 10:09:07.715778872 +0100 +++ /var/tmp/diff_new_pack.y76CmB/_new 2018-12-03 10:09:07.719778868 +0100 @@ -27,7 +27,7 @@ BuildRequires: gcc-c++ >= 4.7 BuildRequires: gettext-devel >= 0.15 BuildRequires: libxml2-devel -BuildRequires: libzypp-devel >= 17.9.0 +BuildRequires: libzypp-devel >= 17.10.0 BuildRequires: readline-devel >= 5.1 Requires: procps %if 0%{?suse_version} @@ -47,7 +47,7 @@ Summary: Command line software manager using libzypp License: GPL-2.0-or-later Group: System/Packages -Version: 1.14.16 +Version: 1.14.17 Release: 0 Source: %{name}-%{version}.tar.bz2 Source1: %{name}-rpmlintrc ++++++ zypper-1.14.16.tar.bz2 -> zypper-1.14.17.tar.bz2 ++++++ /work/SRC/openSUSE:Factory/zypper/zypper-1.14.16.tar.bz2 /work/SRC/openSUSE:Factory/.zypper.new.19453/zypper-1.14.17.tar.bz2 differ: char 11, line 1

1 0