[opensuse-kubic] manifestes for kubernetes workload?
Hi, most of the time, pods/services/deployments for kubernetes have their source on github.com, the containers somewhere in a registry and you can deploy them using a yaml manifest on github.com, e.g. kubectl apply -f https://raw.githubusercontent.com/paulbouwer/hello-kubernetes/master/yaml/he... Now we have already packaged some stuff, build a RPM from it in our buildservice, build a container from it in our buildservice, publish the container in our registry. How do we want to provide the manifest? We cannot point to OBS or an RPM. So my idea was, to build a sub-package, which contains the manifest with the adjusted image url pointing to our registry, install them by default on kubic and the user can use that to deploy the containers. I only would like to specify this, so that not everybody is using something else. 1. I think it should be easy to identify the RPM, so e.g. something like hello-kubernetes-<???>.rpm Which suffix could we use for this? 2. What would be a good location in the filesystem? /usr/share/defaults/manifests? One directory for everything? Every RPM having it's own directory? Any ideas or opinions? Any better ideas? Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
On Wed, 20 Mar 2019 at 14:03, Thorsten Kukuk <kukuk@suse.de> wrote:
Hi,
most of the time, pods/services/deployments for kubernetes have their source on github.com, the containers somewhere in a registry and you can deploy them using a yaml manifest on github.com, e.g. kubectl apply -f https://raw.githubusercontent.com/paulbouwer/hello-kubernetes/master/yaml/he...
Now we have already packaged some stuff, build a RPM from it in our buildservice, build a container from it in our buildservice, publish the container in our registry. How do we want to provide the manifest? We cannot point to OBS or an RPM. So my idea was, to build a sub-package, which contains the manifest with the adjusted image url pointing to our registry, install them by default on kubic and the user can use that to deploy the containers.
I only would like to specify this, so that not everybody is using something else.
1. I think it should be easy to identify the RPM, so e.g. something like hello-kubernetes-<???>.rpm Which suffix could we use for this?
I would like us to use "$foo-k8s-yaml" as the pattern for this packaging naming
2. What would be a good location in the filesystem? /usr/share/defaults/manifests? One directory for everything? Every RPM having it's own directory?
I'd like us to use "/usr/share/k8s-yaml/$foo", with 1 directory for each RPM, but with the expectation that some packages will include multiple yaml examples
Any ideas or opinions?
The -k8s-yaml packages must be installable on systems that don't have kubernetes installed, so they can be used for templating, integration with CI pipelines, or other use cases Of course, some other packages might require -k8s-yaml packages in order for us to be able to execute them automatically when we want to do stuff like automated deployment of services to a kubic cluster - Rich -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
(Re-sending with the correct sender address) Am 22.03.19 um 16:45 schrieb Richard Brown:
On Wed, 20 Mar 2019 at 14:03, Thorsten Kukuk <kukuk@suse.de> wrote:
Hi,
most of the time, pods/services/deployments for kubernetes have their source on github.com, the containers somewhere in a registry and you can deploy them using a yaml manifest on github.com, e.g. kubectl apply -f https://raw.githubusercontent.com/paulbouwer/hello-kubernetes/master/yaml/he...
Now we have already packaged some stuff, build a RPM from it in our buildservice, build a container from it in our buildservice, publish the container in our registry. How do we want to provide the manifest? We cannot point to OBS or an RPM. So my idea was, to build a sub-package, which contains the manifest with the adjusted image url pointing to our registry, install them by default on kubic and the user can use that to deploy the containers.
I only would like to specify this, so that not everybody is using something else.
1. I think it should be easy to identify the RPM, so e.g. something like hello-kubernetes-<???>.rpm Which suffix could we use for this?
I would like us to use "$foo-k8s-yaml" as the pattern for this packaging naming
2. What would be a good location in the filesystem? /usr/share/defaults/manifests? One directory for everything? Every RPM having it's own directory?
I'd like us to use "/usr/share/k8s-yaml/$foo", with 1 directory for each RPM, but with the expectation that some packages will include multiple yaml examples
Yep. There will be packages with more than one yaml file:
https://github.com/rook/rook/tree/master/cluster/examples/kubernetes/ceph
Any ideas or opinions?
The -k8s-yaml packages must be installable on systems that don't have kubernetes installed, so they can be used for templating, integration with CI pipelines, or other use cases
Of course, some other packages might require -k8s-yaml packages in order for us to be able to execute them automatically when we want to do stuff like automated deployment of services to a kubic cluster
- Rich
For Rook, users are supposed to modify the yaml files and then apply them to the cluster. As long as we don't encourage users to modify the yaml files in place, everything should be fine? -- SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany GF: Felix Imendörffer, Mary Higgins, Sri Rasiah, HRB 21284 (AG Nürnberg) -- SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany GF: Felix Imendörffer, Mary Higgins, Sri Rasiah, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
On Fri, Mar 22, Sebastian Wagner wrote:
For Rook, users are supposed to modify the yaml files and then apply them to the cluster. As long as we don't encourage users to modify the yaml files in place, everything should be fine?
Users should never have the need to modify files below /usr, beside that the next update would overwrite their changes. So yes, shipping files there, which the admin then copies to his home, adjust them and execute them is fine. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Mary Higgins, Sri Rasiah, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
On Wed, Mar 20, 2019 at 02:03:57PM +0100, Thorsten Kukuk wrote:
Hi,
most of the time, pods/services/deployments for kubernetes have their source on github.com, the containers somewhere in a registry and you can deploy them using a yaml manifest on github.com, e.g. kubectl apply -f https://raw.githubusercontent.com/paulbouwer/hello-kubernetes/master/yaml/he...
Now we have already packaged some stuff, build a RPM from it in our buildservice, build a container from it in our buildservice, publish the container in our registry. How do we want to provide the manifest? We cannot point to OBS or an RPM. So my idea was, to build a sub-package, which contains the manifest with the adjusted image url pointing to our registry, install them by default on kubic and the user can use that to deploy the containers.
I have a related question. Where we should store the manifests before they get packaged? Some github repo? Only OBS? In my opinion it would be nice to have some repo like github.com/kubic/container-manifests with manifests to deploy some containers defined in github.com/kubic/container-images. And I would love to add a Cilium manifest there. Cheers, Michal -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
Hi, On Fri, Mar 22, Michal Rostecki wrote:
I have a related question.
Where we should store the manifests before they get packaged? Some github repo? Only OBS?
This depends, as we have different use-cases. For kured e.g., the kured binary comes with a yaml file to deploy it. We adjust that to point to our registry to use our adjusted images. So it is clear, the original, unmodifed version is in the upstream github project, the adjusted file is only in OBS.
In my opinion it would be nice to have some repo like github.com/kubic/container-manifests with manifests to deploy some containers defined in github.com/kubic/container-images. And I would love to add a Cilium manifest there.
We have different options and I think it depends on the maintainer. For a single file, I don't create a github repo, I maintain that in OBS. But we could create a github repo and collect different usefull manifests, and put them all in the same RPM. We have all options. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Mary Higgins, Sri Rasiah, HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
On Fri, Mar 22, 2019 at 05:37:51PM +0100, Thorsten Kukuk wrote:
We have different options and I think it depends on the maintainer. For a single file, I don't create a github repo, I maintain that in OBS. But we could create a github repo and collect different usefull manifests, and put them all in the same RPM. We have all options.
Yeah, I was thinking about having one repo for manifests. Especially for components which are not strictly parts of Kubernetes and are not deployed by kubeadm, but are going to be part of infra for many users - like CNI plugins (i.e. Cilium, multus), ingress controllers (i.e. traefik), Prometheus, Grafana, istio etc. We already have one blog post about traefik[0], but it has the deployment manifest pasted in the content and there is no link which can be simply used... And I'm thinking about writing a similar blog post about Cilium[1], so having the manifest on github repo and RPM packages would be nice. Cheers, Michal [0] https://kubic.opensuse.org/blog/2019-01-24-traefik/ [1] As soon as all my newest changes to Envoy and boringssl packaging will be released in Tumbleweed snapshot, I'm going to submit a container image to Kubic repo, then I would like to write that blog post. :) -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
On Sat, 23 Mar 2019 at 11:15, Michal Rostecki <mrostecki@suse.de> wrote:
We already have one blog post about traefik[0], but it has the deployment manifest pasted in the content and there is no link which can be simply used... And I'm thinking about writing a similar blog post about Cilium[1], so having the manifest on github repo and RPM packages would be nice.
+1 - The Traefik example is good enough in the absence of a better solution, but now we seem to have a consensus about a better solution ;)
[1] As soon as all my newest changes to Envoy and boringssl packaging will be released in Tumbleweed snapshot, I'm going to submit a container image to Kubic repo, then I would like to write that blog post. :)
When you're ready, just go ahead and submit it All of the kubic.opensuse.org site is autogenerated from https://github.com/kubic-project/kubic-o-o The blog posts are written in markdown and live in https://github.com/kubic-project/kubic-o-o/tree/master/blog/_posts It'll be automatically rendered nicely and deployed as soon as any change in the git repo. -- To unsubscribe, e-mail: opensuse-kubic+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kubic+owner@opensuse.org
participants (4)
-
Michal Rostecki -
Richard Brown -
Sebastian Wagner -
Thorsten Kukuk