On Wed, Mar 27, Robert Munteanu wrote:
On Tue, 2019-03-26 at 19:25 +0100, Robert Munteanu wrote:
But it's surprising to me since I'm running this off a NVME SSD, so etcd should not be starved for IO. Is this maybe hinting at my QEMU setup not being performant enough in terms of disk IO?
Following up - I switched the master to use a raw disk file
drive file=master.raw,if=virtio \
and indeed responsiveness is much better and pods mare mostly stable.
Two questions:
1. The total disk reads still totally increasing and peaked at about 1GBps ( now at about 800MBps ). Is that value expected or should I go looking for something wrong?
I would expect disk writes, not disk reads. And your values are much, much higher then the ones on my test clusters. There must be something else going on.
2. Should the wiki contain a note regarding the disk image format? I would propose something like
While this helps a little bit in your case, I don't think that this is a generic correct statement. And to me it looks like, while this lowers your problem, it's not the root cause. There are many knobs in a virtualisation stack. E.g. the filesystem setup where you store the images has a much higher impact than raw vs. qcow2. I'm using libvirt/kvm on SLES15 with xfs or btrfs with NoCOW and qgroups disabled. If you use e.g. btrfs with COW and qgroups enabled, all other optimizations will not help, your performance will always be worse. But this is something for virtualisation experts. What you could try to do is to install the master node with YaST from the openSUSE Kubic iso image and don't use a pre-build qcow2 image. Our YaST installer uses a slightly different setup then kiwi, maybe this could also make a difference. But I think the problem is somewhere else in your virtualisation stack, I don't see this high load and disk I/O on any of my three kubic clusters, and one is even running on my notebook. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Mary Higgins, Sri Rasiah, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org