Verify the Harvester Version

Oct 13, 2025 by wbrum.org 108 views

BUG: Fail to Trigger v1.4.0 to v1.4.1 Upgrade on Witness Cluster Due to Managed Chart Harvester Not Being Ready

In this article, we will discuss a bug that prevents the upgrade of a witness cluster from v1.4.0 to v1.4.1 due to the managed chart harvester not being ready. This issue is crucial to address as it can cause significant downtime and impact the overall performance of the cluster.

The environment in which this bug occurred is as follows:

Harvester ISO version: 1.4.0
Underlying Infrastructure: 2 Baremetal Dell Precision with Xeon Gold CPUs and 192+GB RAM
1 VM for witness: This is the witness cluster that is experiencing issues with the upgrade.

The bug is characterized by the following error message:

admission webhook "validator.harvesterhci.io" denied the request: managed chart harvester is not ready, please wait for it to be ready

This error message indicates that the managed chart harvester is not ready to proceed with the upgrade, and the admission webhook is denying the request.

Unfortunately, the support bundle is still to be determined (TBD).

The following output from kubectl get bundle -A and kubectl get bundledeployment -A provides additional context to the issue:

% kubectl get bundle -A
NAMESPACE     NAME                                          BUNDLEDEPLOYMENTS-READY   STATUS
fleet-local   fleet-agent-local                             1/1                       
fleet-local   local-managed-system-agent                    0/1                       Modified(1) [Cluster fleet-local/local]; plan.upgrade.cattle.io cattle-system/system-agent-upgrader-windows modified {"spec":{"upgrade":{"securityContext":{"windowsOptions":{"hostProcess":true,"runAsUserName":"NT AUTHORITY\\SYSTEM"}}}}}
fleet-local   mcc-harvester                                 0/1                       Modified(1) [Cluster fleet-local/local]; deployment.apps harvester-system/harvester-network-webhook modified {"spec":{"template":{"spec":{"containers":[{"command":["harvester-network-webhook"],"env":[{"name":"NAMESPACE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}}],"image":"rancher/harvester-network-webhook:v0.5.5","imagePullPolicy":"IfNotPresent","name":"harvester-network-webhook","resources":{"limits":{"cpu":"200m","memory":"256Mi"},"requests":{"cpu":"10m","memory":"64Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File"}]}}}}
fleet-local   mcc-harvester-crd                             1/1                       
fleet-local   mcc-local-managed-system-upgrade-controller   1/1                       
fleet-local   mcc-rancher-logging-crd                       1/1                       
fleet-local   mcc-rancher-monitoring-crd                    1/1                       

% kubectl get bundledeployment -A

NAMESPACE                                NAME                                          DEPLOYED   MONITORED   STATUS
cluster-fleet-local-local-1a3d67d0a899   fleet-agent-local                             True       True        
cluster-fleet-local-local-1a3d67d0a899   local-managed-system-agent                    True       True        plan.upgrade.cattle.io cattle-system/system-agent-upgrader-windows modified {"spec":{"upgrade":{"securityContext":{"windowsOptions":{"hostProcess":true,"runAsUserName":"NT AUTHORITY\\SYSTEM"}}}}}
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester                                 True       True        deployment.apps harvester-system/harvester-network-webhook modified {"spec":{"template":{"spec":{"containers":[{"command":["harvester-network-webhook"],"env":[{"name":"NAMESPACE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}}],"image":"rancher/harvester-network-webhook:v0.5.5","imagePullPolicy":"IfNotPresent","name":"harvester-network-webhook","resources":{"limits":{"cpu":"200m","memory":"256Mi"},"requests":{"cpu":"10m","memory":"64Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File"}]}}}}
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester-crd                             True       True        
cluster-fleet-local-local-1a3d67d0a899   mcc-local-managed-system-upgrade-controller   True       True        
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-logging-crd                       True       True        
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-monitoring-crd                    True       True

To troubleshoot this issue, we can follow these steps:

Verify the Harvester Version: Ensure that the Harvester version is indeed 1.4.0 and that the upgrade is being attempted from this version.
Check the Managed Chart Harvester Status: Verify the status of the managed chart harvester using the kubectl get bundle -A command. If the harvester is not ready, wait for it to be ready before proceeding with the upgrade.
Review the Admission Webhook Configuration: Review the admission webhook configuration to ensure that it is correctly configured and not blocking the upgrade request.
Check the Cluster Logs: Check the cluster logs for any errors or warnings related to the upgrade process.
Attempt the Upgrade Again: Once the managed chart harvester is ready and the admission webhook configuration is correct, attempt the upgrade again.

In conclusion, the bug that prevents the upgrade of a witness cluster from v1.4.0 to v1.4.1 due to the managed chart harvester not being ready is a critical issue that requires immediate attention. By following the troubleshooting steps outlined above, we can identify and resolve the issue, ensuring that the upgrade process is successful and the cluster is upgraded to the latest version.
Q&A: Troubleshooting the Managed Chart Harvester Not Being Ready Issue

In our previous article, we discussed a bug that prevents the upgrade of a witness cluster from v1.4.0 to v1.4.1 due to the managed chart harvester not being ready. In this article, we will provide a Q&A section to help you troubleshoot and resolve this issue.

A: The managed chart harvester is a component of the Harvester system that is responsible for managing and deploying charts (applications) on the cluster. It is essential for the proper functioning of the cluster and is required for the upgrade process.

A: There could be several reasons why the managed chart harvester is not being ready, including:

Insufficient resources: The harvester may not have sufficient resources (CPU, memory, etc.) to function properly.
Configuration issues: The harvester configuration may be incorrect or incomplete, preventing it from starting or functioning correctly.
Dependency issues: The harvester may be dependent on other components or services that are not available or functioning correctly.

A: You can verify the status of the managed chart harvester using the kubectl get bundle -A command. This command will display the status of the harvester, including whether it is ready or not.

A: Some common errors that occur when the managed chart harvester is not ready include:

"managed chart harvester is not ready, please wait for it to be ready": This error message indicates that the harvester is not ready to proceed with the upgrade.
"admission webhook 'validator.harvesterhci.io' denied the request": This error message indicates that the admission webhook is denying the request due to the harvester not being ready.

A: To troubleshoot the managed chart harvester not being ready issue, you can follow these steps:

Verify the harvester version: Ensure that the harvester version is correct and that the upgrade is being attempted from this version.
Check the harvester status: Verify the status of the harvester using the kubectl get bundle -A command.
Review the harvester configuration: Review the harvester configuration to ensure that it is correct and complete.
Check for dependency issues: Check for any dependency issues that may be preventing the harvester from functioning correctly.
Attempt the upgrade again: Once the harvester is ready and the configuration is correct, attempt the upgrade again.

A: Some best practices for preventing the managed chart harvester not being ready issue include:

Regularly verify the harvester status: Regularly verify the status of the harvester to ensure that it is ready and functioning correctly.
Monitor the harvester logs: Monitor the harvester logs for any errors or warnings that may indicate issues with the harvester.
Review the harvester configuration: Regularly review the harvester configuration to ensure that it is correct and complete.
Check for dependency issues: Regularly check for any dependency issues that may be preventing the harvester from functioning correctly.

In conclusion, the managed chart harvester not being ready issue is a critical issue that requires immediate attention. By following the troubleshooting steps and best practices outlined in this article, you can identify and resolve the issue, ensuring that the upgrade process is successful and the cluster is upgraded to the latest version.