K10 Instant Recovery uses the Veeam Backup and Replication vPower NFS datastore to provide vSphere First Class Disks for storage. This feature enables "instant" recovery of workloads with Kubernetes.
The vPower NFS datastore is a fully functional datastore in the vSphere cluster, however it is not intended for long term usage and should only be used for volumes that have been restored from VBR(Veeam Backup and Replication) before they have been migrated or removed. The vSphere CSI driver will automatically select a datastore when creating new storage and it's important to prevent new storage from being provisioned in the vPower datastore because it is difficult to move the newly created volumes. Before using Instant Recovery, storage classes need to be adjusted so that volumes will not be created in the vPower NFS datastore.
There are two ways to control where the vSphere CSI driver will create volumes:
Named datastores in the storage class
or
Use a SPBM policy that will exclude the vPower NFS Datastore
Named datastore storage class
If there are one or two vSphere datastores that Kubernetes volumes will be allocated in, create storage classes that name those datastores. This is the simplest way to limit where volumes will be created.
Storage classes are immutable so new storage class needs to be created. Existing storage class can be modified and used.
This is a sample storage class
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: default
parameters:
datastoreurl: ds:///vmfs/volumes/vsan:5292cceaadc3cad8-ef4e5a808d92e8f7/
provisioner: csi.vsphere.vmware.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
The key change is the addition of the datastoreurl parameter. URL of a datastore can be found by navigating to the datastore within the vCenter UI as shown in the following screenshot.
SPBM Policy
If it's a complex environment, creating a SPBM policy that does not apply to the vPower NFS Datastore might be a better option.
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: default
parameters:
storagepolicyname: "vSAN Default Storage Policy" #Optional Parameter
provisioner: csi.vsphere.vmware.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
Applying a new Storage Class
Once the changes to the storage class are crafted, they will need to be applied. Depending on configuration, there are different paths as shown below
Regular Kubernetes on vSphere installations
On Kubernetes clusters on vSphere that use the regular CSI driver (not "vSphere with Tanzu" Guest Cluster) update storage classes using kubectl.
Storage classes cannot be changed, storage calls need to be downloaded, delete the original storage class and then recreate the storage class.
Download the storage class using kubectl
kubectl get sc <storage class name> -o yaml > <file name>
Edit the storage class file with your favorite editor and add the datastoreurl parameter or storagepolicyname as shown above
Delete the original storage class
kubectl delete sc <storage class name>
Recreate the storage class
kubectl create -f <file name>
Tanzu with vSphere installations
K10 Instant Recovery is not supported for Tanzu with vSphere installations at this time.
Recovering from disks created in the vPower NFS Datastore
If disks are created by accident in the vPower NFS datastore, they will need to be removed from the datastore. The easiest solution is to simply delete the volumes in question. It is possible to backup and restore applications that have been placed on the vPower NFS Datastore using K10. Before deleting the application namespace, export the backup and remove the snapshots.
Another option is to use vSphere Storage vMotion to move the virtual disks. In order to do this, Kubernetes PVs need to be mapped to their paths in vSphere. Then, determine the VMs that the volumes are attached to. Then use Storage vMotion to move the VMs to the correct datastore.