How to ‘Transform’ multiple resources with Regex in K10

Kasten makes use of the ‘Transforms’ feature to migrate applications between different clusters, between different storage backends and many other use cases. Transforms enable modifications to Kubernetes resources on restore.

The ability to move an application across clusters is an extremely powerful feature that enables a variety of use cases including Disaster Recovery (DR), Test/Dev with realistic data sets, and performance testing in isolated environments.

Transforms are loosely modeled on JSON patch and support regex to modify resources with a pattern. 

Our K10 documentation provides a simple example of Transformation. This article provides an example to showcase the use of regex along with transforms for modifying ingress/route hostnames.

Transforming the routes/ingresses hostnames during restores

‘Transformation replace’, operations can be used to replace the hostname in the routes/ingresses. However, if multiple ingress/route hostname follows a particular pattern, regex can be used to match the text and replace them.

Here is an example of such a scenario.

There are 4 ingresses with the hostnames as shown below

~/jai-aws-test#  kubectl get ingress -n mysql 
NAME         CLASS    HOSTS                             ADDRESS   PORTS     AGE
apigateway   <none>   prd-apigateway.apps.example.com             80, 443   4h58m
backend      <none>   cn-backend.apps.example.com                 80        4h57m
frontend     <none>   cn-frontend.apps.example.com                80        4h56m
pgdb         <none>   prd-pgdb.apps.example.com                   80        4h56m

While restoring these ingresses to another cluster, there is a requirement to change the `prd-*`,`cn-*` from the ingress hostnames to `dr-*`. The restored ingresses/routes are expected to have the below hostnames.

dr-apigateway.apps.example.com
dr-pgdb.apps.example.com
dr-backend.apps.example.com
dr-frontend.apps.example.com

Multiple transformation operations can be used to achieve this, which isn’t efficient! K10 has inbuilt support for matching patterns using regex for transformation.

Following regex expression and the JSONpath expression can be used to achieve the desired transformation outcome.

JSONPath for ingresses: /spec/rules/0/host

JSONPath for routes: /spec/host

 

Regex: (prd|cn)(.*)(\.apps\.example\.com)

Value: dr${2}${3}

 

The above expression matches pattern as shown in the image below

(Note: https://regex101.com/ can be used to test and try out regular expressions)

Adding Transforms operation from the UI using the above values

While performing restore or creating an import policy with restore, in UI, Apply transforms to restored resources under Optional Restore Settings can be used to apply the above values.

Select Add New Transform to start creating a transformation.

In the ‘New Transform’ form, fill in the name for the transformation resource in consideration and add an operation intended with the transform.

Operations that are supported include Test, Add, Remove, Copy, Move, Replace without and with Regex pattern matching.

In this example Replace with Regex(Replace element at path with JSON value. Match the target text with regex and substitute regex capturing groups with value) was selected

This operation can be tested from the UI by comparing the original and transformed resource.

Adding Transforms operation from the Policy/RestoreAction API using the above values

Transform operation can also be accomplished by editing the Policy CR using ‘kubectl edit’ to add transform field, shown below, under restoreParameters while creating an import+restore policy.

This can also be added into a restoreAction (https://docs.kasten.io/latest/api/actions.html#restoreaction). 

transforms:
- subject:
    resource: ingresses
  name: replace-ingress-host
  json:
  - op: replace
    path: /spec/rules/0/host
    value: dr${2}${3}
    regex: (prd|cn)(.*)(\.apps\.example\.com)

Trigger the import policy and verify the ingress hostnames after the restore.

~/jai-aws-test#  k get ingress -n mysql-clone
NAME         CLASS    HOSTS                            ADDRESS   PORTS     AGE
apigateway   <none>   dr-apigateway.apps.example.com             80, 443   20m
backend      <none>   dr-backend.apps.example.com                80        20m
frontend     <none>   dr-frontend.apps.example.com               80        20m
pgdb         <none>   dr-pgdb.apps.example.com                   80        20m