Hi Adrian, the problem magically vanished 2 hours after sending my mail. Before sending, I updated the server from 2.8.2 to 2.8.3, and rebooted. Hence 2 hours later, it finally processed the scheduled jobs. To give you an impression, how your baby behaves, when it's sheltered by strangers, I added a few related remarks below. They're mostly usability issues. Let me know, if I should open some github issues for them. On Freitag, 22. September 2017 08:08:43 you wrote:
On Freitag, 22. September 2017, 08:00:38 CEST wrote Hans-Peter Jansen:
Hi,
after a few weeks of proper operation, we experience a strange OBS behaviour now: a kernel-default package stays in scheduled stage since about one day.
A reboot of the OBS host didn't helped. The logs doesn't show something obvious.
It's a single OBS 2.8.3 host with a few workers, plenty of RAM and cores, with uplinks to OBS and PMBS. The base package kernel-source links to openSUSE.org:Kernel:stable, and built correctly as well as the locally linked kernel-syms. Just the also locally linked kernel-default package misbehaves in the described way.
Any ideas for possible reasons of this behaviour and ways to cure them?
most likely some constraint for the worker is not matching your worker:
osc cat Kernel:stable kernel-source _constraints
$ oscl cat LISA:Kernel kernel-default _constraints Server returned an error: HTTP Error 404: _constraints no such file _constraints: no such file $ oscl checkconstraints No local _constraints file. Using just the project constraints Repository Arch Worker ---------- ---- ------ penSUSE_Leap_42.3 x86_64 6 openSUSE_Leap_42.2 x86_64 6 openSUSE_Leap_42.1 x86_64 6 openSUSE_13.2 x86_64 6 Due to my primitive setup, constraints are not working at all (OBS_VM_TYPE="auto"), therefor I remove them in an complicated process, since there's no UI support for that at the moment. $ oscl co kernel-source -u $ vi _link <link project="openSUSE.org:Kernel:stable" package="kernel-source"> <patches> <delete name="_constraints"/> </patches> </link> $ oscl ci -m "remove constraints" I would love to get constraints working, but it needs OBS_VM_TYPE="kvm" at least, I guess, but the comments in /etc/sysconfig/obs-server are somewhat misleading. Anybody want to share a proper /etc/sysconfig/obs-server with me and the steps, how and where to generate the initrd for kvm workers. Related to this, I've changed some values (reduced the number of workers from 6 to 4, enabled OBS_VM_USE_TMPFS) after deployment. Obviously, it doesn't adjust itself automatically (which I understand). I guess, I need to remove the OBS-worker partitions manually. What are decent sizes for them? Automatic setup led to a somewhat unfavourable layout: yast: Device │ Size│F│Enc│Type │FS Type│Label│Mount Point │Metadata│PE Size│Str│ /dev/OBS │ 2.00 TiB│ │ │LVM2 OBS│ │ │ │LVM2 │4 MiB │ │ /dev/OBS/cache │ 25.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker │ │ │1 │ /dev/OBS/server │512.00 GiB│ │ │LV │Ext4 │ │/srv/obs │ │ │1 │ /dev/OBS/worker_root_1│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_1│ │ │1 │ /dev/OBS/worker_root_2│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_2│ │ │1 │ /dev/OBS/worker_root_3│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_3│ │ │1 │ /dev/OBS/worker_root_4│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_4│ │ │1 │ /dev/OBS/worker_root_5│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_5│ │ │1 │ /dev/OBS/worker_root_6│166.00 GiB│ │ │LV │Ext4 │ │/var/cache/obs/worker/root_6│ │ │1 │ /dev/OBS/worker_swap_1│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ /dev/OBS/worker_swap_2│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ /dev/OBS/worker_swap_3│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ /dev/OBS/worker_swap_4│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ /dev/OBS/worker_swap_5│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ /dev/OBS/worker_swap_6│512.00 MiB│ │ │LV │Swap │ │swap │ │ │1 │ df -h: /dev/mapper/OBS-server 504G 36G 468G 7% /srv/obs /dev/mapper/OBS-cache 25G 1,1G 23G 5% /var/cache/obs/worker /dev/mapper/OBS-worker_root_1 164G 4,2G 151G 3% /var/cache/obs/worker/root_1 /dev/mapper/OBS-worker_root_2 164G 680M 155G 1% /var/cache/obs/worker/root_2 /dev/mapper/OBS-worker_root_3 164G 4,4G 151G 3% /var/cache/obs/worker/root_3 /dev/mapper/OBS-worker_root_4 164G 2,3G 153G 2% /var/cache/obs/worker/root_4 /dev/mapper/OBS-worker_root_5 164G 4,5G 151G 3% /var/cache/obs/worker/root_5 /dev/mapper/OBS-worker_root_6 164G 4,2G 151G 3% /var/cache/obs/worker/root_6 tmpfs 13G 0 13G 0% /run/user/0 none 63G 0 63G 0% /var/cache/obs/worker/root_5/dev/shm none 63G 0 63G 0% /var/cache/obs/worker/root_3/dev/shm none 63G 0 63G 0% /var/cache/obs/worker/root_6/dev/shm none 63G 0 63G 0% /var/cache/obs/worker/root_1/dev/shm 164G for a worker is a little exaggerated, isn't it? Do workers really need swap? The server is running as vSphere VM with 12 cores, 128 GB RAM and a 2 TB disk dedicated to OBS. I feel really bad to ask all these silly questions. If I manage to get kvm workers incl. constraints working, I will offer a PR for /etc/sysconfig/obs-server, that hopefully avoids similar questions for any successors. Sorry, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org