Installing Root CA Certificates to the K10 Prometheus’ Trust store

This article outlines the steps involved in adding the ConfigMap that has the CA certificates as a mount in the Prometheus pod using helm values. 

K10 provides users option to supply a root CA certificate (and intermediate CAs, if applicable) through a ConfigMap, enabling K10 to trust and verify ingress/apiserver certificates not signed by a well-known certificate authority. 

Following documentation can be referred for the process of creating a ConfigMap having the root CA certificates and using it within K10's Helm configuration. 

https://docs.kasten.io/latest/install/advanced.html#install-root-ca-in-k10-s-namespace  

Based on the process mentioned above, K10 mounts the ConfigMap as a volume in most K10 pods that require the CA for verification, using a mountPath of /etc/ssl/certs/custom-ca-bundle.pem. Consequently, K10 will trust all certificates signed by the root CA found at /etc/ssl/certs/custom-ca-bundle.pem. 

When employing K10 multicluster, there might be an issue where certificates were not being mounted in the Prometheus deployment used in conjunction with K10. This omission does not affect standalone K10 instances. 

During the bootstrapping process of a secondary K10 instance to the primary, for a K10 multicluster setup, a federation scrapeConfig is added to the primary Prometheus instance to gather metrics from the secondary K10 Prometheus. 

With this federation in place, the primary K10 Prometheus does not recognise the secondary K10 target since the certificate gets signed by an unknown authority. 

As the multicluster-manager dashboard(till K10 version 6.0.12) relies on metrics from secondary K10 instances to populate specific data, this might not show the desired metrics. 

This article outlines the steps involved in adding the ConfigMap as a mount in the Prometheus pod using Helm values. 

Prometheus upstream chart that K10 allows a helm value that enables mounting a configMap to the specified mountPath. This method will be used to install the root CA in Prometheus deployment. 

Below is the helm upgrade command that can be used to do the same. 

   helm get values k10 --output yaml --namespace=kasten-io > k10_val.yaml && \
   helm upgrade k10 kasten/k10 --namespace=kasten-io -f k10_val.yaml \
   --set “prometheus.server.extraConfigmapMounts[0].name”="custom-ca-bundle-store" \
   --set “prometheus.server.extraConfigmapMounts[0].mountPath”="/etc/ssl/certs/custom-ca-bundle.pem" \
   --set “prometheus.server.extraConfigmapMounts[0].subPath”="custom-ca-bundle.pem" \
   --set “prometheus.server.extraConfigmapMounts[0].configMap”="custom-ca-bundle-store" \
   --set “prometheus.server.extraConfigmapMounts[0].readOnly”="true" \
   --set cacertconfigmap.name="custom-ca-bundle-store" \
    --version=<CURRENT_K10_VERSION> 

The above helm upgrade command assumes that configMap named custom-ca-bundle-store is already created in Kasten namespace including the CA certificates in it. 

Once the configMap is mounted in Prometheus pod the target discovery from primary K10 Prometheus would be successful without any issues.