Recently I had to upgrade my vCenters (5 vCenters in linked mode) from 8.1.0.400 to 8.0U2e to mitigate the Critical security issue from Oct 2024.
I followed the recommended procedure for upgrading a group of vCenters in link mode, power them all off, take offline snapshots of all of them, power them all on again and start the upgrade process one by one.
Although I was able to upgrade 4 of them the last one failed at 97% with the message “Exception occurred in postInstallHook for autodeploy: Contract. Please check the logs…”
The Resume vCenter update did nothing and the Rollback to the previous version was not an option as it was marked out, so I had to revert all the vCenters back using the offline snapshot.
The most “painful” part of all this procedure was the support log creation (several hours).
Checking the logs I was able to identify the following entries:
Failure in AutoDeploy Contract Hook
o 2024-11-27T22:50:01.127Z autodeploy:Contract ERROR autodeploy.autodeploy_patch Failed to patch autodeploy: no such table: config_pairs
o Traceback (most recent call last):
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/payload/components-script/autodeploy/autodeploy_patch.py", line 293, in contract
o _cleanupTlsConfig(conn)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/payload/components-script/autodeploy/autodeploy_patch.py", line 179, in _cleanupTlsConfig
o dbConnection.execute(
o sqlite3.OperationalError: no such table: config_pairs
o 2024-11-27T22:50:01.128Z autodeploy:Contract ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'autodeploy:Contract' failed.
o Traceback (most recent call last):
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
o executionResult = systemExtension(args)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__
o result = self.extension(*args)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func
o return func(*args)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/payload/components-script/autodeploy/autodeploy_patch.py", line 293, in contract
o _cleanupTlsConfig(conn)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/payload/components-script/autodeploy/autodeploy_patch.py", line 179, in _cleanupTlsConfig
o dbConnection.execute(
o sqlite3.OperationalError: no such table: config_pairs
o 2024-11-27T22:50:01.148Z ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
o Traceback (most recent call last):
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 212, in patch
o executeComponentsHook(Hook.Contract, ctx, userData, statusAggregator.reportingQueue)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/utils/phase_utils.py", line 98, in executeComponentsHook
o executeComponentHook(hook, ctx, c, userData, reportingQueue)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 97, in executeComponentHook
o result = executeHook(c.patchScript, hook, args,
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 53, in executeHook
o result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
o File "/storage/seat/software-update2_ce09l8/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor_process.py", line 119, in executeHook
o raise ex
o patch_errors.ComponentError
o 2024-11-27T22:50:02.151Z WARNING root stopping status aggregation...
2024-11-27T22:50:02.152Z ERROR __main__ Patch vCSA failed
It looks like there was an empty db file at the old location of AutoDeploy DB (/var/lib/rbd/db) and the problem seems to be that the code at upgrade will move this file on top of the “real” AutoDeploy DB located at the new DB location (/etc/vmware-rbd/db/db) simply because the file at the old location exists
The way that I was able to fix it was renaming the file in /var/lib/rbd/db to db.old and then running the upgrade procedure again
rename the db file with the mv command # mv db db.old
By renaming/removing the DB file from the old location it allowed the upgrade to complete without replacing the AutoDeploy DB
Time to take new offline snapshots and run the upgrade again but this time I started with the problematic vCenter.
Success!!! The upgrade completed with no issues.
KISS
Leave a Reply