VxRail: VxVerify – kube_diff: Kubectl Client data mismatch found

During another deployment and HealthCheck by VxVerify on my favorite HCI appliance, VxRail, I ran into the warning above.

#========================#=======#========#====================================================================#==============#
|  Hostname / Category   |Status  Dell_KB |  Warnings or Failures, unless tests Passed                         ; Product S.N. |
#========================#=======#========#====================================================================#==============#
| VxRM                   | Warning 212809 | kube_diff: Kubectl client data mismatch found     

If you look at Dell’s KB 212809 article, you can only perform various test command to see if it is a issue that only support could fix, these include:

kubectl get nodes -o wide or kubectl get pods

# Where, in turn, the output must be as follows:

error: You must be logged in to the server (Unauthorized)

# This is not seen when loading the kubeconfig from the rke2 YAML file: /etc/rancher/rke2/rke2.yaml

With the following command, we can see how long various rke2 services have been running and pay attention to AGE:

kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml -n helium get nodes

# One possible output can be:

NAME      STATUS   ROLES                        AGE     VERSION
vxrail    Ready    control-plane,etcd,master    412d    v1.21.4+rke2r2

The general source of the error is that the rke2 service has been running for over 1 year the $HOME/.kube/config is expired.

IMPORTANT!!! Before you begin the following steps, take a snapshot or backup of the VxRail Manager!!!

As I described above, the KB article recommends contacting Dell Support. However, I have a preliminary solution that can be used to save time on support.

Backup the current configuration:

mv $HOME/.kube/config $HOME/.kube/config.bkp

# Generate the configuration file again:

cat /etc/rancher/rke2/rke2.yaml > $HOME/.kube/config

# Update the configuration file with the correct namespace:
kubectl config set-context --current --namespace=helium

Generate the configuration file again:

cat /etc/rancher/rke2/rke2.yaml > $HOME/.kube/config

# Update the configuration file with the correct namespace:
kubectl config set-context --current --namespace=helium

Update the configuration file with the correct namespace:

kubectl config set-context --current --namespace=helium

After that, you can run VxVerify again and the warning should go away. If it doesn’t, restarting might help.

If that doesn’t work either, revert the snapshot or restore the VxRail Manager to the version that has not yet been adapted and contact Dell Technologies Support. That’s it for this blog post. If you have any questions, please use the comment section below. 🙂