Using Kubernetes with Juju
Kubernetes (“K8s”) provides a flexible architecture for managing containerised applications at scale. See the Kubernetes documentation for more information.
The objective of this page is to give an overview of how an existing Kubernetes cluster can be integrated with Juju and what the general workflow is once that’s done. Links will be provided at the end to theoretical and practical material. Finally, although this page is not about showing how to install Kubernetes itself, we do give pointers on how to do so.
Juju Kubernetes-specific workflow
Essentially, Juju is able to treat the added cluster as it does any other of its known clouds (i.e. create models and deploy charms). There are some differences to working with such a cloud and they are called out in this section.
The k8s-specific Juju commands are
scale-application. All other concepts and commands are applied in the traditional manner.
add-k8s command makes the usual combination of
add-k8s command can be used to add the Kubernetes cluster and its credentials to the client’s local cache (option
--local) or to add these things directly to a running controller (the default behaviour). There are also options
--gke for streamlining the addition of AKS and GKE clusters.
v.2.5.0, only local behaviour is possible (there is no
--local option). The cluster configuration file will also first need to be copied to
User credentials can still be added by way of the
autoload-credentials commands. Also, at any time, the k8s CLI can be used to add a new user to the cluster.
add-k8s command can repeatedly set up different clusters as long as the contents of the configuration file has been changed accordingly. The KUBECONFIG environment variable is useful here as it will be honoured by Juju when finding the file to load.
remove-k8s command is used to remove a Kubernetes cluster from Juju’s list of known clouds.
scale-application command is used to scale a Kubernetes cluster. The
remove-unit commands do not apply to a Kubernetes model.
Charms need to be written specifically for Kubernetes workloads. See tutorial Understanding Kubernetes charms.
Running Kubernetes workloads
First off, a Kubernetes cluster will be required. Essentially, you will use it as you would any other cloud that Juju interacts with: the cluster becomes the backing cloud.
The following steps describe the general approach for using Kubernetes with Juju:
- Obtain a Kubernetes cluster
- Add the cluster
- Add a model
- Create persistent static storage (if necessary)
- Create storage pools
- Deploy a Kubernetes charm
Obtain a Kubernetes cluster
There are many ways to obtain a Kubernetes cluster. Here is a list of suggestions:
- Use the ‘kubernetes-core’ bundle, which gives a minimal two-machine cluster available in the Charm Store.
- Use the ‘canonical-kubernetes’ bundle. This is the Charmed Distribution of Kubernetes (CDK), which is a more sophisticated version of ‘kubernetes-core’. See tutorial Installing Kubernetes with CDK and using auto-configured storage.
- Use the conjure-up installer.
- Use MicroK8s. This gives you a local, fully compliant Kubernetes deployment with dynamic persistent volume support.
- When Kubernetes is deployed via charms, special integrator charms made for specific cloud vendors can greatly assist (e.g. storage). Search the Charm Store for ‘integrator’.
- Use a public Kubernetes cloud vendor such as Amazon EKS, Azure AKS, and Google GKE. To get started with GKE, see appendix Installing a GKE cluster.
Kubernetes bundles do not work well on a LXD cloud at this time. Refer to Deploying on LXD for details.
Add the cluster
Juju needs information about the cluster in order to add it. This information is part of the main Kubernetes configuration file which can be found on the Kubernetes master node. This is the same config file that
kubectl uses. Copy this configuration file to your local machine and save it as
add-k8s command will parse that file and add the cluster as a cloud.
v.2.6.1 clusters based on AKS and GKE can be added without the cluster configuration file. These types have special helper modes that enable their respective, independently installed, CLI toolkits and are accessible via options
add-k8s command can apply to the client’s local cache of clouds or to an existing controller. This works similar to the
add-cloud command; see the Adding clouds section.
conjure-up installer adds the cluster automatically.
Add a model
After having added the cluster to the controller, you can create k8s models on that controller using
juju add-model <k8s-cloud-name>. This will create a Kubernetes namespace in the cluster which is named after the model name. This namespace will host all of the pods and other resources of that model. The model also has a default storage pool called “kubernetes”.
Create persistent static storage
Create persistent static volumes for operator storage if your chosen cloud’s storage is not supported natively by Kubernetes. You will need to do the same for charm storage if your charm has storage requirements. This is done with the Kubernetes tool
Create storage pools
Create storage pools for operator storage and, if needed, charm storage. This is done in the usual way, with the
Deploy a Kubernetes charm
A Kubernetes-specific charm is deployed in standard fashion, with the
deploy command. If the charm has storage requirements you will need to specify them, as you do with a normal charm.
The below table lists configuration keys supported by Kubernetes charms that are set at deploy time. The corresponding Kubernetes meaning can be obtained from the Kubernetes documentation for Services and Ingress.
juju deploy mariadb-k8s --config kubernetes-service-loadbalancer-ip=10.1.1.1
There are two other keys that are not Kubernetes-specific:
Keys ‘juju-external-hostname’ and ‘juju-application-path’ control how the application is exposed externally using a Kubernetes Ingress Resource in conjunction with the configured ingress controller (default: nginx).
Storage theory and practical guides
The Persistent storage and Kubernetes page provides the theory on how Juju works with Kubernetes storage.
The following practical guides are available:
conjure-upinstaller can be used to install Kubernetes. See the following resources for guidance:
- Tutorial Setting up static Kubernetes storage shows how to set up statically provisioned persistent volumes with Juju by way of the ‘kubernetes-core’ charm.
- Tutorial Multi-cloud controller with GKE and auto-configured storage provides steps on how to add an existing GKE-based Kubernetes cluster to an existing controller. It also covers storage auto-configuration.
- Tutorial Installing Kubernetes with CDK and using auto-configured storage shows how to install Kubernetes with the CDK bundle and illustrates storage class and storage pool auto-configuration.
- Tutorial Using Juju with MicroK8s provides steps for getting started with Juju and MicroK8s.
- Tutorial Using Juju on Charmed Kubernetes on VMWare provides the steps for getting started with Juju on a Kubernetes cluster on VMWare.
- Tutorial Using the aws-integrator charm demonstrates deploying Kubernetes with Juju on AWS with an integrator charm.