This article provides couple of examples of performance tuning settings for K10
K10 uses helm parameters, which can be updated for K10 performance tuning. Default settings should suite most customers' needs. However, in complex environments K10 administrators might want to tune the execution control parameters to improve K10 performance and solve various execution issues related to system resource utilization. This guide will cover most frequent cases seen in complex environments.
Note: while tuning the system, it is important to rely on the metrics K10 exposes. For many performance related metrics, K10 offers graphs visible in the K10 bundled Grafana instance. Make sure that Grafana and Prometheus are enabled (they are enabled by default).
This article covers two different performance problems that can be seen with K10Problem - 1 : K10 takes too much time to pick up an action
Not enough workers to pick up actions new actions due to existing K10 worker load.
Related Helm settings
It is possible to tune services.executor.workerCount to increase number of workers per executor instance
It is also possible to tune executorReplicas to increase the number of executor instances
Both approaches have their pros and cons. Increasing the number of workers per executor instance will not increase the resource consumption in the cluster, but at the same time may hit the limitations of the executor pod. Increasing the number of executor instances will on the opposite, consume more resources in the cluster, but will not hit the limit inside the executor pod.
Tuning impact analysis
To understand the impact of the changes, it is recommended to check in-built K10 Grafana Dashboard. Using Executor Worker Load graph, it is possible to observe the total number of workers (worker count multiplied by the number of executor instances) over the time and the worker load (how many workers are currently in use).