9.3. Network Policies

Task 9.3.1: Allowing Specific Communication Between Clusters

The following policy illustrates how to allow particular pods to communicate between two clusters.

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "allow-cross-cluster"
spec:
  description: "Allow x-wing in cluster1 to only contact rebel-base in cluster1"
  endpointSelector:
    matchLabels:
      name: x-wing
      io.cilium.k8s.policy.cluster: cluster1
  egress:
  - toEndpoints:
    - matchLabels:
        "k8s:io.kubernetes.pod.namespace": kube-system
        "k8s:k8s-app": kube-dns
    toPorts:
    - ports:
        - port: "53"
          protocol: ANY
      rules:
        dns:
          - matchPattern: "*"
  - toEndpoints:
    - matchLabels:
        name: rebel-base
        io.cilium.k8s.policy.cluster: cluster1

Kubernetes security policies are not automatically distributed across clusters, it is your responsibility to apply CiliumNetworkPolicy or NetworkPolicy in all clusters.

Create a file cnp-cm.yaml with the above content and apply the CiliumNetworkPolicy to both clusters:

kubectl --context cluster1 apply -f cnp-cm.yaml
kubectl --context cluster2 apply -f cnp-cm.yaml

Let us run our curl for loop again

XWINGPOD=$(kubectl --context cluster1 get pod -l name=x-wing -o jsonpath="{.items[0].metadata.name}")
for i in {1..10}; do
  kubectl --context cluster1 exec -it $XWINGPOD -- curl -m 1 rebel-base
done

and as an result you see:

curl: (28) Connection timed out after 1001 milliseconds
command terminated with exit code 28
curl: (28) Connection timed out after 1000 milliseconds
command terminated with exit code 28
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
curl: (28) Connection timed out after 1000 milliseconds
command terminated with exit code 28
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
curl: (28) Connection timed out after 1000 milliseconds
command terminated with exit code 28
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
curl: (28) Connection timed out after 1000 milliseconds
command terminated with exit code 28

All connections to cluster2 are dropped while the ones to cluster1 are still working.

Task 9.3.2: Cleanup

We will disconnect our cluster mesh again and delete the second cluster:

cilium clustermesh disconnect --context cluster1 --destination-context cluster2
minikube delete --profile cluster2
minikube profile cluster1