[opensuse-buildservice] Thoughts on deploy Open Build Service in kubernetes cluster
Hey guys, Recently, we start to evaluate Open Build Service for our package build and publish system, considering the portability, scalability and ecosystem, our most of other services are deployed on kubernetes cluster, therefore, we are just wondering whether it's a possible or a good choice to deploy OBS in kubernetes. We would like start this thread for the purpose of discussion on these below: 1. Does there anyone have tried to deploy OBS in k8s cluster, Is there any suggestion/warning on that direction? 2. If possible, what's the best practice on how to deploying OBS, and what's the scale we can get? 3. If not, what's the disadvantages or issues we would have within that deployment? Thanks for your sharing! TommyLike -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
Hey guys,
Recently, we start to evaluate Open Build Service for our package build and publish system, considering the portability, scalability and ecosystem, our most of other services are deployed on kubernetes cluster, therefore, we are just wondering whether it's a possible or a good choice to deploy OBS in kubernetes. We would like start this thread for the purpose of discussion on these below: 1. Does there anyone have tried to deploy OBS in k8s cluster, Is there any suggestion/warning on that direction? 2. If possible, what's the best practice on how to deploying OBS, and what's the scale we can get? 3. If not, what's the disadvantages or issues we would have within that deployment?
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Donnerstag, 26. September 2019, 14:56:13 CEST schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
good choice to deploy OBS in kubernetes.
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
KVM nesting overhead? what KVM nesting? Kubernetes is docker containers, not virtual machines... Cheers MH -- Mathias Homann Mathias.Homann@openSUSE:.org irc: [Lemmy] @ freenode, ircnet obs: lemmy04 gpg key fingerprint: 8029 2240 F4DD 7776 E7D2 C042 6B8E 029E 13F2 C102 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Fri, Sep 27, 2019 at 3:49 AM Mathias Homann <Mathias.Homann@opensuse.org> wrote:
Am Donnerstag, 26. September 2019, 14:56:13 CEST schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
good choice to deploy OBS in kubernetes.
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
KVM nesting overhead? what KVM nesting? Kubernetes is docker containers, not virtual machines...
Cheers MH
Based on my understanding, there could be two possibilities. 1. When the kubernetes cluster especially the work nodes are deployed on KVM based instances, especially on public cloud. 2. When the container runtime is based on VM solution, for instance Kata container gvisor or firecracker. sometimes it's a requirement due to the security reason. 3. Or both. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Donnerstag, 26. September 2019, 21:49:11 CEST Mathias Homann wrote:
Am Donnerstag, 26. September 2019, 14:56:13 CEST schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
good choice to deploy OBS in kubernetes.
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
KVM nesting overhead? what KVM nesting? Kubernetes is docker containers, not virtual machines...
you still need a kvm inside for security reasons and to guarantee the right kernel. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am 27.09.19 um 08:11 schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 21:49:11 CEST Mathias Homann wrote:
Am Donnerstag, 26. September 2019, 14:56:13 CEST schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
good choice to deploy OBS in kubernetes.
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
KVM nesting overhead? what KVM nesting? Kubernetes is docker containers, not virtual machines...
you still need a kvm inside for security reasons and to guarantee the right kernel.
But running vms inside of containers is not nesting kvm - it only gets nested if kubernetes is running in vms, which is not necessarly the case. Greetings, Stephan -- Lighten up, just enjoy life, smile more, laugh more, and don't get so worked up about things. Kenneth Branagh -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Freitag, 27. September 2019, 08:23:43 CEST Stephan Kulow wrote:
Am 27.09.19 um 08:11 schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 21:49:11 CEST Mathias Homann wrote:
Am Donnerstag, 26. September 2019, 14:56:13 CEST schrieb Adrian Schröter:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
good choice to deploy OBS in kubernetes.
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
KVM nesting overhead? what KVM nesting? Kubernetes is docker containers, not virtual machines...
you still need a kvm inside for security reasons and to guarantee the right kernel.
But running vms inside of containers is not nesting kvm - it only gets nested if kubernetes is running in vms, which is not necessarly the case.
kubernetes nodes is usually not setup on plain hardware, but running already in some kvm. At least in the scenarios I had contact with. -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am 27.09.19 um 08:39 schrieb Adrian Schröter:
kubernetes nodes is usually not setup on plain hardware, but running already in some kvm. At least in the scenarios I had contact with. https://containerjournal.com/topics/container-ecosystems/the-rise-of-bare-me...
Greetings, Stephan -- Lighten up, just enjoy life, smile more, laugh more, and don't get so worked up about things. Kenneth Branagh -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thu, Sep 26, 2019 at 8:57 PM Adrian Schröter <adrian@suse.de> wrote:
On Donnerstag, 26. September 2019, 14:02:24 CEST TommyLike Hu wrote:
Hey guys,
Recently, we start to evaluate Open Build Service for our package build and publish system, considering the portability, scalability and ecosystem, our most of other services are deployed on kubernetes cluster, therefore, we are just wondering whether it's a possible or a good choice to deploy OBS in kubernetes. We would like start this thread for the purpose of discussion on these below: 1. Does there anyone have tried to deploy OBS in k8s cluster, Is there any suggestion/warning on that direction? 2. If possible, what's the best practice on how to deploying OBS, and what's the scale we can get? 3. If not, what's the disadvantages or issues we would have within that deployment?
running an instance is easy. Getting it integrated, solving the KVM nesting overhead (about 50% slowdown, but you may ignore it, if acceptable) and integrating into the dispatch queue is a complete different topic.
Could you please provide more detail on the `integrating into the dispatch queue is a complete different topic.` does this mean that due to the kubernetes cluster network model, the dispatch logic would incompatible? Thanks TommyLike -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (4)
-
Adrian Schröter -
Mathias Homann -
Stephan Kulow -
TommyLike Hu