Greetings, Kubernetes enthusiasts! In this blog post, we are going to dive deep into a fascinating and crucial aspect of Kubernetes orchestration: inter-cluster networking. In particular, we'll be exploring how to connect two Kubernetes clusters to enable the orchestration of applications that can communicate with each other.
By the end of this blog post, you'll be familiar with the method of using Cilium's ClusterMesh functionality to establish communication between two clusters.
For our illustration, we'll have a front-end server and a backend database installed on two different clusters. We will ensure these applications can freely communicate with each other.
But before we dive in, let's get the housekeeping out of the way. We must clarify some prerequisites that your clusters need to meet to ensure successful communication between them.
To effectively connect two Kubernetes clusters, a few conditions must be satisfied:
Network Accessibility: Both clusters must be network accessible to each other. This accessibility could be achieved in various ways. They might be in the same virtual network or VPC, or they may be in different networks with appropriate routing and firewall rules set up to ensure they can communicate.
Unique Cluster CIDRs: Each Kubernetes cluster uses a Cluster CIDR, which is a block of IP addresses assigned to pods within the cluster. These CIDRs must not overlap between your clusters, as it would lead to IP conflicts and communication issues.
Kubernetes Version Compatibility: Ensure both clusters are running a version of Kubernetes that is compatible with Cilium. At the time of writing, Cilium supports Kubernetes versions 1.16 and later. I do recommend however that you use a version 1.24 at least.
With these prerequisites in place, we can dive into the process.
The first step in our process is to have two Kubernetes clusters at our disposal. Each cluster will host one part of our application – the frontend and the backend.
I'm assuming you already have two clusters set up. If not, there are numerous resources available online that walk you through setting up your Kubernetes clusters. If you need further guidance Cilium themselves have a excellent guide for preparing two Azure AKS clusters for use with Cluster Mesh - https://docs.cilium.io/en/v1.13/network/clustermesh/aks-clustermesh-prep/
Ensure you have kubeconfig files for both clusters. These files contain the necessary details to connect to and authenticate with your cluster. We'll call them cluster1-config and cluster2-config for this walkthrough.
If you need further guidance on setting up clusters Cilium themselves have an example for creating clusters suitable for use with Cilium cluster mesh.
After your Kubernetes clusters are set up, the next step is to install Cilium. The installation process is straightforward thanks to Helm, a package manager for Kubernetes. Here's how to do it...
First you will need to grab the Cilium CLI, we will use this to install and configure the Cilium;
I'm installing on an Intel Mac, instructions may differ depending on your OS and architecture.
Installing Cilium is fairly straightforward. Let's walk through the installation process.
First, switch the current context to Cluster1:
2. Use the cilium CLI to install cilium.
3. Repeat these steps for Cluster2. Remember to change the cluster.name parameter to cluster2 and cluster.id parameter to 2.
Having installed Cilium on both our clusters, we're now ready to configure ClusterMesh. This functionality of Cilium will facilitate the communication between our two Kubernetes clusters.
It may take a few minutes for your clusters to connect, to see the status you can re-run the commands in step 3.
Now that we have our ClusterMesh ready, we can deploy our example application. We will be deploying a front-end server on cluster1 and a backend database on cluster2.
Here's a simple example Kubernetes manifest for a front-end server:
And for the backend database:
For the front-end application, deployment and service apply these to cluster1
For the Backend database deployment and service apply thease to cluster2
Once all deployment and services are up you will be able to pass traffic from the front end service deployed on cluster1 to the backend deployment on cluster2.
But how? If you note in the manifest for the backend service we set some annotations:
This annotation is parsed by Cilium service discovery. Any service with the same name and namespace is made available across all clusters in the cilium cluster mesh.
In our example the frontend app connects to the backend service ClusterIP of the service in cluster1 and because this Service is Cilium global service traffic is passed to backend pods in cluster2 by the Cilium CNI Cluster Mesh.
Congratulations! You've successfully set up two Kubernetes clusters, installed and configured Cilium and its ClusterMesh functionality, and deployed a front-end and backend application that can communicate across the clusters.
While the task might seem challenging at first, understanding the nuances of inter-cluster communication is vital for managing larger-scale, more complex Kubernetes environments. Understanding how to establish this kind of communication allows for more flexibility and scalability in managing workloads.
What if I told you you could achieve the same outcome without all the prerequisites and requirement to install an advanced CNI such as Cilium?
Ori Global Cloud automates application deployment lifecycle including application centric inter-cluster networking for you in one deployment operation. Stay tuned as we will be covering this in our next networking focused blog post.
Can't wait, then head over to https://ori.co/sandbox-request to try for yourself.