crash loop on fresh master after installing weave
Hi, I have installed a fresh "Kubic Admin Node" in a VM using the latest openSUSE-Kubic-DVD-x86_64-Snapshot20220320-Media.iso image. Continuing with the instructions at https://en.opensuse.org/Kubic:kubeadm I run "kubeadm init", and it seems to work fine, so I continue with setting up the network plugin master:~ # mkdir .kube master:~ # cp /etc/kubernetes/admin.conf .kube/config master:~ # kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane,master 25m v1.23.4 master:~ # kubectl apply -f /usr/share/k8s-yaml/weave/weave.yaml serviceaccount/weave-net created clusterrole.rbac.authorization.k8s.io/weave-net created clusterrolebinding.rbac.authorization.k8s.io/weave-net created role.rbac.authorization.k8s.io/weave-net created rolebinding.rbac.authorization.k8s.io/weave-net created daemonset.apps/weave-net created master:~ # After this, it starts crash-looping, seemingly trying to start a pod named kube-system_coredns. In the following system log, the kubic admin VM starts up, and goes through the steps listed above, after which the crash loop starts: https://drive.google.com/file/d/1xzLrRfi6_7zpHrEOsI1fwLrioUiiCRdL/view?usp=s... Does anyone have an idea what's going on here? Regards, Olav
On Tue, 22 Mar 2022 at 15:54, Olav Reinert <seroton10@gmail.com> wrote:
Hi,
I have installed a fresh "Kubic Admin Node" in a VM using the latest openSUSE-Kubic-DVD-x86_64-Snapshot20220320-Media.iso image.
Continuing with the instructions at
https://en.opensuse.org/Kubic:kubeadm
I run "kubeadm init", and it seems to work fine, so I continue with setting up the network plugin
master:~ # mkdir .kube master:~ # cp /etc/kubernetes/admin.conf .kube/config master:~ # kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane,master 25m v1.23.4 master:~ # kubectl apply -f /usr/share/k8s-yaml/weave/weave.yaml serviceaccount/weave-net created clusterrole.rbac.authorization.k8s.io/weave-net created clusterrolebinding.rbac.authorization.k8s.io/weave-net created role.rbac.authorization.k8s.io/weave-net created rolebinding.rbac.authorization.k8s.io/weave-net created daemonset.apps/weave-net created master:~ #
After this, it starts crash-looping, seemingly trying to start a pod named kube-system_coredns.
In the following system log, the kubic admin VM starts up, and goes through the steps listed above, after which the crash loop starts:
https://drive.google.com/file/d/1xzLrRfi6_7zpHrEOsI1fwLrioUiiCRdL/view?usp=s...
Does anyone have an idea what's going on here?
The attached log shows no sign of anything related to coredns (can't find any mention of coredns at all) Can you produce a lot that shows what you're seeing in regards to the crash loop so I can look into it?
On Tue, 2022-03-22 at 16:01 +0100, Richard Brown wrote:
On Tue, 22 Mar 2022 at 15:54, Olav Reinert <seroton10@gmail.com> wrote:
Hi,
I have installed a fresh "Kubic Admin Node" in a VM using the latest openSUSE-Kubic-DVD-x86_64-Snapshot20220320-Media.iso image.
Continuing with the instructions at
https://en.opensuse.org/Kubic:kubeadm
I run "kubeadm init", and it seems to work fine, so I continue with setting up the network plugin
master:~ # mkdir .kube master:~ # cp /etc/kubernetes/admin.conf .kube/config master:~ # kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane,master 25m v1.23.4 master:~ # kubectl apply -f /usr/share/k8s-yaml/weave/weave.yaml serviceaccount/weave-net created clusterrole.rbac.authorization.k8s.io/weave-net created clusterrolebinding.rbac.authorization.k8s.io/weave-net created role.rbac.authorization.k8s.io/weave-net created rolebinding.rbac.authorization.k8s.io/weave-net created daemonset.apps/weave-net created master:~ #
After this, it starts crash-looping, seemingly trying to start a pod named kube-system_coredns.
In the following system log, the kubic admin VM starts up, and goes through the steps listed above, after which the crash loop starts:
https://drive.google.com/file/d/1xzLrRfi6_7zpHrEOsI1fwLrioUiiCRdL/view?usp=s...
Does anyone have an idea what's going on here?
The attached log shows no sign of anything related to coredns (can't find any mention of coredns at all)
That's weird - did the download work at all for you? There should be 7.5 MB of log output there. And there definitely is mention of coredns in it - for example: Mar 22 14:12:26 master kubelet[1257]: I0322 14:12:26.688687 1257 kuberuntime_container.go:714] "Killing container with a grace period" pod="kube-system/coredns-fc8b57f45-7gc99" podUID=0b785387-2185-45ae- a8a5-936f65dc0d4e containerName="coredns" containerID="cri- o://a37448039f3b5ce6750fc10dae19b4da7c2f672c4012ecabfe154154e3ef7cf9" gracePeriod=30
Can you produce a lot that shows what you're seeing in regards to the crash loop so I can look into it?
Sure - I've attached the log file to this email instead. Regards, Olav
On Tue, 2022-03-22 at 16:01 +0100, Richard Brown wrote:
On Tue, 22 Mar 2022 at 15:54, Olav Reinert <seroton10@gmail.com> wrote:
https://drive.google.com/file/d/1xzLrRfi6_7zpHrEOsI1fwLrioUiiCRdL/view?usp=s...
Does anyone have an idea what's going on here?
The attached log shows no sign of anything related to coredns (can't find any mention of coredns at all)
What do you mean - that you can't download the file? There is 7.5 MB of log data in it, and "coredns" occurs in thousands of lines.
Can you produce a lot that shows what you're seeing in regards to the crash loop so I can look into it?
The log I linked to is exactly that. Please try to download it again. If it fails, I'll figure out some other way to publish it. (paste.opensuse.org seems like an obvious choice, but doesn't seem to be able to cope with files this big, unfortunately.)
On Tue, 2022-03-22 at 16:01 +0100, Richard Brown wrote:
On Tue, 22 Mar 2022 at 15:54, Olav Reinert <seroton10@gmail.com> wrote:
Hi,
I have installed a fresh "Kubic Admin Node" in a VM using the latest openSUSE-Kubic-DVD-x86_64-Snapshot20220320-Media.iso image.
Continuing with the instructions at
https://en.opensuse.org/Kubic:kubeadm
I run "kubeadm init", and it seems to work fine, so I continue with setting up the network plugin
master:~ # mkdir .kube master:~ # cp /etc/kubernetes/admin.conf .kube/config master:~ # kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane,master 25m v1.23.4 master:~ # kubectl apply -f /usr/share/k8s-yaml/weave/weave.yaml serviceaccount/weave-net created clusterrole.rbac.authorization.k8s.io/weave-net created clusterrolebinding.rbac.authorization.k8s.io/weave-net created role.rbac.authorization.k8s.io/weave-net created rolebinding.rbac.authorization.k8s.io/weave-net created daemonset.apps/weave-net created master:~ #
After this, it starts crash-looping, seemingly trying to start a pod named kube-system_coredns.
In the following system log, the kubic admin VM starts up, and goes through the steps listed above, after which the crash loop starts:
https://drive.google.com/file/d/1xzLrRfi6_7zpHrEOsI1fwLrioUiiCRdL/view?usp=s...
Does anyone have an idea what's going on here?
The attached log shows no sign of anything related to coredns (can't find any mention of coredns at all)
Can you produce a lot that shows what you're seeing in regards to the crash loop so I can look into it?
$ kubectl -n kube-system logs weave-net-krxcx -c weave-init modprobe: can't load module nfnetlink (kernel/net/netfilter/nfnetlink.ko.zst): invalid module format Ignore the error if "xt_set" is built-in in the kernel So this is a duplicate of the issue reported by Attila Pinter, and idenfitied by Robert Munteau, in a separate email thread. Starting over from an older Kubic image solved it for me (for now).
participants (2)
-
Olav Reinert -
Richard Brown