vCenter 6.7 Update 3, Unable to add Host
UPDATED February 11, 2020. The blog has been update, with new information and the recommended solution.
I have had some customers with a problem that they can’ add ESXi hosts to a vCenter after upgrading to 6.7 Update 3/3a, and adding new ESXI 6.7 Update 3 hosts.
When trying to add the host to the vCenter they get this error:
A general system error occurred: Unable to push CA certificates and CRLs to host <hostname/IP>
The problem is mentioned in the release notes, “You might be unable to add a self-signed certificate to the ESXi trust store and fail to add an ESXi host to the vCenter Server system” , but this is not an good description.
There are 2 solutions, one thats in the release notes and another is the workaround I have already published.
NOTE: This is fixed in later versions, so if you see the problem, please try to check NTP or time on the ESXi host certificate is not issued in the future.
This is a workaround: you can change an advanced setting on the vCenter; vCenter -> Configure -> Settings -> Advanced Settings:
vpxd.certmgmt.mode = thumbprint
This may also affect other operation on the ESXi hosts, I have not checked, but I think that it also means that you can not push new certifices to hosts, already added, and maybe also other things.
Note: This solution can create a new problem later see the blog.
The solution: in the VMware release notes is to change an advanced setting: “The fix adds the advanced option Config.HostAgent.ssl.keyStore.allowSelfSigned
. If you already face the issue, set this option to TRUE
to add a self-signed server certificate to the ESXi trust store”. They do forget to mention that you need to restart the management agents “services.sh restart” true the server console og SSH, or reboot the hosts.
To do this a little easier, i have made a script to do this, from PowerCLI, and also using plink.exe. This script is just a sample, that you can modify, for your use case.
$cmd = "services.sh restart" $hostname = "esx01.domain.local" $esx_Password = read-host "Input ESXi root password: " $vmhost = connect-viserver -server $hostname -user root -password $esx_password set-VMHostAdvancedConfiguration -Name "Config.HostAgent.ssl.keyStore.allowSelfSigned" -value true $sshService = Get-VmHostService | Where { $_.Key -eq “TSM-SSH”} Start-VMHostService -HostService $sshService -Confirm:$false cmd /c "echo y | C:\temp\Plink -ssh -pw $($esx_Password) root@$hostname $($cmd)" disconnect-viserver $vmhost -Confirm:$false $vmhost = connect-viserver -server $hostname -user root -password $esx_password $sshService = Get-VmHostService | Where { $_.Key -eq “TSM-SSH”} Stop-VMHostService -HostService $sshService -Confirm:$false disconnect-viserver $vmhost -Confirm:$false
Hope this will help you.
The best solution would be, not to use self issued certificates.
Note: Check if time is correct on the ESXi hosts, this could give similar problems.
I had the same issue two weeks ago with a (upgraded) vCenter using the vmca and a completely new host.
With a new installed vCenter the same host could be added without problems.
Have you changed the certificates on any of them?
I don’t know if the customer made any changes to the vmca, but that would be unlikely. The vcsa has been upgraded from 6.5 U1 to 6.7 U3 (while not migrating events/performance data).
Then I wanted to add three completely freshly installed hosts. All of them failed with that error so I tried a lot with the first one. For example a complete re-generation of the self signed host certificate and also the new extended option Config.HostAgent.ssl.keyStore.allowSelfSigned=true. Without success.
Also tried regeneration of the vmca and replacement of all certificates without success.
I thought this would be a pretty primitive bug and quickly set up a lab with a new vCenter and a host. Adding here worked fine.
At first I was wondering if it could be related to the HPE custom image, but then it worked fine with a new vCenter in production (with default host and vCenter settings).
Did you find out anything more regarding this? Experiencing it myself now.
I have not had the time to dive into the problem. I have found this blog, that might be onto the what the problem is.
https://www.funkycloudmedina.com/2019/11/unable-to-push-ca-certificates-and-crls-to-host/
I have checked, and this is not the solution, in the case i have seen.
I have updated the blog, with new information/solution.
Guys before doing all those configurations make sure that your host is running the right date and time. (use NTP if you can)
I’ve fixed this issue just by running NTP… after hours of troubleshooting….
Spot on!!
Had the same issue and after setting the correct time manually or using NTP I was able to add the hosts without problems.
Hello Amaury
Thank you for your hint. After configuring ntp on the host I cloud join the new esxi server without any problems.
Regards Lorenz
Just want to say thanks for the part about Config.HostAgent.ssl.keyStore.allowSelfSigned. i follow the vmware article and still doesn’t work, Run this in ssh “services.sh restart” and it works!