- openSUSE Virtual

Re: [opensuse-virtual] is there a clever way to shrink qcow2 images
by Rob Verduijn 20 Aug '13

20 Aug '13

Hi, Using separate images for partitions makes the script a tiny bit less complex. I've already been scripting the exercise, the use off multiple partition images makes it only a tiny bit more complex. qemu-nbd simply adds another device for each additional partition. #!/binbash modprobe nbd max_part=16 #number off partitions qemu-nbd -c --nocache --aio=native /path/to/image.qcow2 mount /dev/nbd0 /mnt # I didn't partiotion the image, just formated the image as ext4 dd if=/dev/zero of=/mnt/bigfile bs=1024k; sync; rm /mnt/bigfile; rm bigfile; sync; rm /mnt/bigfile; sync sleep 2 umount /dev/nbd0 # umount the device qemu-nbd /dev/nbd0 # clean up nbd devices or they will bite you qemu-img convert -c -O qcow2 /path/to/image.qcow2 /path/to/shrunk.qcow2 mv /path/to/image.qcow2 /path/to/image.qcow2.bak mv /path/to/shrunk.qcow2 /path/to/image.qcow2 The performance of qemu-nbd is rather poor, using writeback cache is not really an option since you always need to wait for the zeroes to be actually written to the hd. Also writeback is very hazardous if you use a script to umount and disconnect the nbd device, image corruption is very likely to happen since sync doesn't apply to nbd0 devices and blockdev --flushbufs /dev/nbd0 isn't foolproof either when scripting. (ctrl-c the script at the wrong time and you are in for a recovery) So the script isn't that difficult, neither is the use of multiple partitions in an image. The biggest drawback is, downtime, filling the image with zeroes takes even longer offline than online. Ok you could have a cronjob in the vm that does that nightly, but I can imagine issues when the hd of a vm is at filled to the max at regular intervals (once a month ??), even at expected times and only for a short moment. Also that would mean the shrinking cronjob is required on the host (I want the shrinking done as soon as possible after the zeroing) This has to be timed properly with the zero cronjob of the guest, this becomes rather complex with every additional guest. Regards Rob 2013/8/19 Tony Su <tonysu(a)su-networking.com>: > Not specific to qcow > In the past if I wanted to partition files I'd deploy multiple disks instead > of partitions. I don't knowi if there a significant overhead diff but I > found performance did not suffer. Once on separate disks, should be trivial > to script the procedure. > > You can execute your conversion on any storage, just as fast as possible. > So, for instance can even be cheap temporary attached storage. > > And AFAIK has to be done offline although I suppose a fancy live migraation > could be implemented so you're not offline. > Tony > > On Aug 19, 2013 11:50 AM, "Rob Verduijn" <rob.verduijn(a)gmail.com> wrote: >> >> Hello all, >> >> I'm looking for a clever way to shrink qcow2 images. >> >> what I do now is : >> >> 1 in the vm delete the files I don't need (tempfiles, spoolfiles, >> junkfiles, etc, etc) >> 2 fill the empty space with zeros >> dd if=/dev/zero of=bigfile bs=1024k ; sync ; rm bigfile ; sync >> 3 shut down the vm >> 4 convert the qcow2 to a fresh new qcow2 >> qemu-img convert -c -O qcow2 orig.qcow2 shrunk.qcow2 >> mv orig.qcow2 orig.qcow2.bak >> mv shrunk.qcow2 orig.qcow2 >> 5 test the vm >> 6 delete the backup if it works >> >> Now this is fine if you got plenty space and just want to tidy up a >> bit before it turns bad. >> >> But what if there is no space, and adding extra partitions is not that >> easy. >> (imagine an offsite, far off location that is a real pain to get into >> due to serious security hassle) >> >> Or if you are me and are seriously annoyed by the cumbersome exercise >> above and have a gut feeling that there should be a more elegant way >> (preferably one without downtime) >> >> The above exercise requires to be done on all the partitions of the >> qcow2 image if there is more than one, adding to my irritation of the >> exercise. >> >> Does anybody have any ideas on a more elegant solution ? >> >> cheers >> Rob >> -- >> To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org >> To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org >> > -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

1 0

[opensuse-virtual] Xen43 guest lockups under high-load (BUG: soft lockup - CPU#2 stuck) after recent Xen43 updates
by ar16 20 Aug '13

20 Aug '13

I run Opensuse 12.3 in a number of Xen VPS guests (@ Linode); Been running stable for a long time. After recent Xen client updates over the last couple of days, to rpm -qa | grep -i xen | sort ipset-kmp-xen-6.19_k3.7.10_1.1-32.1.x86_64 kernel-xen-3.7.10-1.16.1.x86_64 xen-libs-4.3.0_09-266.1.x86_64 xen-tools-domU-4.3.0_09-266.1.x86_64 xtables-addons-kmp-xen-2.3_k3.7.10_1.1-83.1.x86_64 under high-load (e.g. a src build) the client is fairly frequently, but randomly locking up. THe only notice I have is that my SSH session to the VPS simply disconnects ... no errors in the shell, or in system message. @ the attached admin Serial console I see only the sudden onest of, ... [ 484.176059] BUG: soft lockup - CPU#2 stuck for 22s! [sh:21413] [ 512.176064] BUG: soft lockup - CPU#2 stuck for 23s! [sh:21413] [ 548.176060] BUG: soft lockup - CPU#2 stuck for 22s! [sh:21413] [ 576.176074] BUG: soft lockup - CPU#2 stuck for 22s! [sh:21413] [ 604.176055] BUG: soft lockup - CPU#2 stuck for 22s! [sh:21413] [ 632.176060] BUG: soft lockup - CPU#2 stuck for 22s! [sh:21413] ... and then it just loops/repeats continuously, requiring a hard-restart of the VPS to get out of the loop, and recover. Guest boot config is unchanged, and quite simple cat /boot/grub/menu.lst default 0 timeout 1 title PVGRUB root (hd0) kernel /boot/vmlinuz-xen root=/dev/xvdc noresume xencons=hvc0 kbdtype=us text nofb selinux=0 apparmor=0 edd=off noshell showopts splash=verbose initrd /boot/initrd-xen Appears to be a frequent xen-related issue https://www.google.com/search?q=BUG+"soft+lockup"+xen with not much conclusive resolution. but since this just started with recent updates, I wanted to explore what's changed that might affect this, rather than random attempts from search. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

2 1

[opensuse-virtual] "EVTCHNOP failure" @ OS 12.3 + Xen 4.3 boot for some Guests; launch manually OK
by ar16 16 Aug '13

16 Aug '13

I've recently upgraded pkgs an Opensuse 12.3 + XEN 4.3 (xen-4.3.0_08-264.1.x86_64) server. It's got two guests set for auto-launch @ boot. It's been working fine for awhile. The guest cfg's have not changed in ages. Just the upgrade has occured recently. Now, reproducibly after boot, one Guest is missing, xl list Name ID Mem VCPUs State Time(s) (null) 0 985 4 r----- 44.2 <====== Why "(null)" here? systemd? test2 2 1024 4 -b---- 13.3 but launches manually xl create /etc/xen/auto/test1.cfg xl list Name ID Mem VCPUs State Time(s) (null) 0 985 4 r----- 48.2 test2 2 1024 4 -b---- 13.8 test1 3 768 2 -b---- 2.3 and functions normally. checking xen boot logs, I find these, new failures xl dmesg ... [ OK ] Started Haveged Entropy Gathering Daemon. [ OK ] Started Restore Sound Card State. [ OK ] Started LSB: irqbalance daemon providing irq balancing on MP-machines. [ OK ] Started LSB: Framebuffer setup. [ 35.614880] Event-channel device installed. [ 35.677851] usbcore: registered new interface driver usbback [ 35.711552] pciback: backend is vpci [ 35.748410] blktap_device_init: blktap device major 252 Welcome to openSUSE 12.3 "Dartmouth" - Kernel 3.7.10-1.16-xen (xvc0). testsvr login: (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 (XEN) [2013-08-15 01:42:22] event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22 [ OK ] Started LSB: Configure network interfaces and set up routing. [ OK ] Reached target Network. Starting Command Scheduler... ... Those "event_channel.c:233:d0 EVTCHNOP failure: domain 2, error -22" errors are new to me. Looks like a XEN issue. I've seen recent work on event channel deadlock on the dev list, but my xen's logs don't mention anything evtchn-related since ~ Nov 2011. Before deciding to file a bug, I need to do some local troubleshooting. Any suggestions as to what this might be about, or specific additional info that might be enlightening? AR -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

5 41

[opensuse-virtual] What the status of lxc-opensuse ?
by Per Jessen 11 Aug '13

11 Aug '13

I'm just getting started with lxc, and tried to use the lxc-opensuse script: [big snip - download stuff etc] Scanning scripts ... Resolve dependencies ... Install symlinks in /lib/mkinitrd/setup ... Install symlinks in /lib/mkinitrd/boot ... Kernel image: /boot/vmlinuz-3.7.10-1.16-debug Initrd image: /boot/initrd-3.7.10-1.16-debug Kernel Modules: <not available> Could not find map -k/boot/System.map, please specify a correct file with -M. There was an error generating the initrd (9) [snip] Resolve dependencies ... Install symlinks in /lib/mkinitrd/setup ... Install symlinks in /lib/mkinitrd/boot ... Kernel image: /boot/vmlinuz-3.7.10-1.16-debug Initrd image: /boot/initrd-3.7.10-1.16-debug device node not found root device (/dev/cciss/c0d0p2) not found There was an error generating the initrd (1) warning: %posttrans(mkinitrd-2.7.2-3.2.1.x86_64) scriptlet failed, exit status 1 removing nis flags from /var/cache/lxc/opensuse/partial-x86_64/etc/nsswitch.conf... now finalizing build dir... warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? warning: Failed to read auxiliary vector, /proc not mounted? Download complete. Copy /var/cache/lxc/opensuse/rootfs-x86_64 to /srv/lxc/myfirstone/rootfs ... Copying rootfs to /srv/lxc/myfirstone/rootfs ... Please change root-password ! BAD PASSWORD: it is too short BAD PASSWORD: is too simple grep: /srv/lxc/myfirstone/config: No such file or directory -- Per Jessen, Zürich (18.0°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

2 4

Re: [opensuse-virtual] Issues with virt-manager 0.9.5
by Michael Hill 01 Aug '13

01 Aug '13

Thanks, Tony. On Tue, Jul 16, 2013 at 6:34 PM, Tony Su <tonysu(a)su-networking.com> wrote: > - What version openSUSE are you running, and what Desktop? 13.1/Factory with GNOME Factory desktop (mostly 3.9.4). > - Is there a reason why you choose to run virt-manager 0.9.5? It's just the version I get with Factory, I guess. > - Exactly what do you mean by "import?" - Are you trying to create a > new Guest using an existing virtual disk, or something else? I'm importing the GNOME OSTree qcow2 image from here: http://build.gnome.org/ostree/buildmaster/images/z/current/ ...as described here: https://wiki.gnome.org/GnomeOSTree > - If you're trying to create a new VM, are you doing it by clicking > the "Create a new virtual machine" button top left and directly under > the "File" menu? Yes. > - Are you trying to change the disk format or just use a disk "as is?" As is, using "Import existing disk image" and "Customize configuration before install." Clicking on "Disk 1" in the customization dialogue gives the error. > Also, you should know that if you're trying to create a new VM, there > are <two> different programs, one can be launched from vm manager and > has a "pretty chrome" appearance. The other can be launched a couple > ways, one is from the Desktop, eg 'virt-manager --no-fork -c qemu:///session' to access QEMU Usermode. I imported the daily image a number of times until about a month ago (when I messed up my laptop). It can then be launched from virt-manager or GNOME Boxes. > App Launch button > System > Virtualization > Create Virtual Machines > for Xen and KVM I'll take a look at this, thanks. Mike -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

1 1

Re: [opensuse-virtual] Issues with virt-manager 0.9.5
by Michael Hill 17 Jul '13

17 Jul '13

On Wed, Jul 17, 2013 at 10:00 AM, Tony Su <tonysu(a)su-networking.com> wrote: > Factory is Beta code. Yes, I've been running it for a couple of years now, with the occasional crushing setback. I work on upstream GNOME docs, so it's good to be able to use beta versions. > If you run into problems like what you describe, you should submit a > bug to bugzilla (http://bugzilla.novell.com) I'll try to gather more information and maybe file a bug. I have 0.10.0 running well on Fedora. Thanks, Mike -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

1 0

[opensuse-virtual] Issues with virt-manager 0.9.5
by Michael Hill 16 Jul '13

16 Jul '13

I'm trying to import the GNOME OSTree current qcow2 image using virt-manager 0.9.5 on GNOME Factory. I get these messages on launch... mike@himon:~> virt-manager --no-fork -c qemu:///session /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+CellRendererSparkline::data-array after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+CellRendererSparkline::reversed after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+Sparkline::data-array after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+Sparkline::num-sets after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+Sparkline::reversed after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+Sparkline::rgb after class was initialised type_register(cls, namespace.get('__gtype_name__')) /usr/lib64/python2.7/site-packages/gobject/__init__.py:115: Warning: Attempt to add property virtManager+graphwidgets+Sparkline::filled after class was initialised type_register(cls, namespace.get('__gtype_name__')) During the import operation, when I try to specify the disk format in the interface, I get this instead of the usual list of disk formats... Error message: Error refreshing hardware page: unsupported operand type(s) for /: 'NoneType' and 'int' Details: Error refreshing hardware page: unsupported operand type(s) for /: 'NoneType' and 'int' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/details.py", line 1307, in hw_selected self.refresh_disk_page() File "/usr/share/virt-manager/virtManager/details.py", line 2921, in refresh_disk_page iotune_read_bytes_sec = disk.iotune_read_bytes_sec / 1024 TypeError: unsupported operand type(s) for /: 'NoneType' and 'int' Any suggestions where to look for a solution? I've posted this to opensuse-gnome as well. Thanks, Mike -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

1 0

Fwd: [opensuse-virtual] Best Practices / Details for Serial Console debug-level logging when using Xen + systemd ?
by Tony Su 05 Jul '13

05 Jul '13

---------- Forwarded message ---------- 1. Wondering if you have considered the method described at Freedesktop.org(one of my primary sources of info on systemd) http://freedesktop.org/wiki/Software/systemd/Debugging/ 2. Is this for a special purpose? If not and if you're just setting up a regular maintenance system, have you considered extracting and aggregating logs remotely using IP instead of serial using something like Logstash? Tony -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

3 4

[opensuse-virtual] Best Practices / Details for Serial Console debug-level logging when using Xen + systemd ?
by ar16 02 Jul '13

02 Jul '13

I boot Opensuse 12.3 + Xen 4.3 using Grub1. Reading http://en.opensuse.org/SDB:Systemd#Getting_debug_from_systemd There's a note to ln -s /usr/lib/systemd/system/serial-getty@.service /etc/systemd/system/getty.target.wants/serial-getty(a)ttyS0.service On my system, I have just: find /usr/lib/systemd/system/ -name *serial-getty* /usr/lib/systemd/system/serial-getty@.service i.e., I do NOT have the symlink for startup in place. I currently have serial console(s) enabled using the config below. It seems to work, but it's evolved to this over time, tweaked as kernel- and xen-versions have changed. I suspect it needs some clean up. My goal is to have grub, early boot, Xen and detailed, debug-level, complete boot logged to both a serial console & video out. What, if anything, SHOULD be changed in the config below for correctness? Specifically, what's recommended for getting the 'right info' form boot process for informative reporting on bugs? cat /boot/grub/menu.lst ... serial --unit=0 --speed=57600 --word=8 --parity=no --stop=1 terminal --timeout=6 serial console ... title Xen root (hd0,0) kernel /xen.gz vga=gfx-1280x1024x16 ... systemd.log_level=debug systemd.log_target=syslog-or-kmsg loglvl=debug guest_loglvl=debug com1=57600,8n1,pci console=vga,com1 console_timestamps ... apic_verbosity=verbose iommu=verbose ... module /vmlinuz-xen log_buf_len=4M console=tty0 console=xvc0,57600n8 xencons=tty earlyprintk=xen vga=0x31a ... showopts noquiet module /initrd-xen ... -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

2 3

[opensuse-virtual] howto change virtual machine creation defaults
by Rob Verduijn 18 Jun '13

18 Jun '13

Hello, I want to change the defaults when a new vm is created. ie now the default hd is a raw image that is fully allocated when you create a vm. I would like to change it to qcow2 sparse image. Does anybody know how to do this or where to find documentation for this ? Regards Rob -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe(a)opensuse.org To contact the owner, e-mail: opensuse-virtual+owner(a)opensuse.org

3 4