Multicluster
      Back to home
      1. Help Center
      2. Debugging
      3. Multicluster

      Bootstrapping secondary clusters fails in k10multicluster setup due to certificates 

      This articles helps in troubleshooting and resolving secondary cluster bootstrapping issues due to certificates

      Problem Description 

      Bootstrapping secondary clusters fails in k10multicluster setup even after skipping SSL verification for secondary cluster’s ingress. 

      Error message:  

      Failed to get k10-config config map (namespace: kasten-io) 

      -> Get "https://api.k8s-prod.xxx.xxxx.com:6443/api/v1/namespaces/kasten-io/configmaps/k10-config": x509: certificate signed by unknown authority

      Above error is seen while querying kube-apiserver, not while K10 is connecting to secondary cluster’s ingress.

      Analysis: 

      While bootstrapping the secondary cluster with the below command, SSL verification is disabled for the secondary cluster’s ingress using the flag --secondary-cluster-ingress-tls-insecure=true 

      ./k10multicluster bootstrap \
        --primary-context=admin \
        --primary-name=k8s-prod \
        --primary-kubeconfig=kubeconfig.prod \
        --secondary-context=admin \
        --secondary-kubeconfig=kubeconfig.dev \
        --secondary-name=k8s-dev \
        --secondary-cluster-ingress=https://k10.apps.k8s-dev.xxx.xxxx.com/k10/ \
        --secondary-cluster-ingress-tls-insecure=true

      K10multicluster tool use the kubeconfig and context provided in the above command to query the secondary cluster’s kube-apiserver.  

      Kubeconfig file should have the signed CA certificate for the kube-apiserver 

      apiVersion: v1 

      clusters: 

      - cluster: 

          certificate-authority-data: 

      If it is not present, then it should have insecure-skip-tls-verify: true in it. 

      apiVersion: v1 

      clusters: 

      - cluster: 

          server: https://api.k8s-prod.xxx.xxxx.com:6443 

          insecure-skip-tls-verify: true 

      Steps to identify the problem 

      If bootstrap still fails with the error shown above even with the certificate-authority-data populated, following command can be used to decode the CA certificate from the kubeconfig.   

      kubectl --kubeconfig=</path/to/kubeconfig> --context=<context> config view --flatten -o jsonpath='{.clusters[0].cluster.certificate-authority-data}'|base64 -d|openssl x509 -text -noout 

      The above command will error Unable to load certificate if there is a problem with the certificate.

      Resolution 

      The solution is to fetch the correct kube-apiserver’s root CA certificate from the master nodes and prepare the kubeconfig file with it.

      Once a clean kubeconfig is created, it can be used to bootstrap the secondary without any issues.