VCF 5.2.2: Troubleshooting SDDC Manager When the UI Fails to Load

A failed SDDC Manager UI can be misleading. The browser may simply show an Identity Internal Server Error, while the actual problem is deeper in the communication between SDDC Manager and vCenter.

This article walks through a real troubleshooting scenario in VMware Cloud Foundation (VCF) 5.2.2, where SDDC Manager failed to load and logs showed an InvalidLogin error while connecting to vCenter.

The Symptom

The initial symptom was:

Identity Internal Server Error

The SDDC Manager UI was not loading correctly. The first step was to check the Operations Manager log:

/var/log/vmware/vcf/operationsmanager/operationsmanager.log

The log showed that Operations Manager was unable to connect to vCenter:

Failed to connect to https://vc-l-01a.corp.local:443/sdk

java.util.concurrent.ExecutionException: (vim.fault.InvalidLogin)

This immediately pointed the investigation toward vCenter authentication and the SDDC Manager service-account relationship.

Step 1: Check Password Health

The next step was to use the SDDC Supportability and Serviceability utility to check the health of the domain components.

/opt/vmware/sddc-support/sos --password-health --domain-name ALL

The command performs a password-health check across the domain components.

The output immediately highlighted two important findings.

Finding 1: vCenter could not be queried

The vCenter entry showed:

vCenter : vc-l-01a.abc.xyz
Failed to get details

Finding 2: Workspace ONE Access had an expired account

The output also showed an expired sshuser account for Workspace ONE Access:

Workspace ONE Access : vidm-01a.corp.local

sshuser
Expiry Date : Jul 22, 2025
Expires in Days : -415
State : RED

However, the ESXi service accounts and several other components reported a healthy state.

The important clue remained the vCenter authentication failure, which correlated directly with the InvalidLogin error in the Operations Manager log.

Step 2: Reset the SDDC Manager–vCenter Connection

The next step was to follow the scripted workaround documented in Broadcom KB - 399817

The workaround was used to reset the vCenter connection and synchronize the relevant service account between SDDC Manager and vCenter.

The script was executed with:

python /tmp/sddc_svc_account_sync.py

The script confirmed that the environment was running:

SDDC Manager: 5.2.2.0

It then requested the administrator@vsphere.local password and successfully generated the required SDDC and vCenter tokens.

Step 3: Synchronize the vCenter Service Account

The script identified the vCenter:

Available vCenter Servers:
  [1] vc-l-01a.abc.xyz

After selecting the vCenter, it reported that the vCenter was not in an active state in the SDDC inventory:

vc-l-01a.abc.xyz is not in an active state in SDDC inventory
Setting status to ACTIVE

The script then reset the SDDC Manager service account password on vCenter:

Resetting service account password for
svc-sddc-l-01a-vc-l-01a@abc.xyz

Successfully reset ...'s password!

It subsequently updated the password in SDDC Manager:

Successfully updated svc-sddc-l-01a-vc-l-01a@vsphere.local's
password in SDDC Manager!

Although the script reported a 400 while triggering password remediation, the subsequent vCenter synchronization completed successfully.

The key validation message was:

VCENTER password sync completed successfully.

Step 4: Restart SDDC Manager Services

After completing the synchronization, all SDDC Manager services were restarted.

The restart included:

• Common services

• Domain Manager

• LCM

• Operations Manager

• SDDC Manager UI application

The restart was initiated using the SDDC Manager service script. At this point, the SDDC Manager UI was loading again. However, another issue was visible.

Step 5: Investigate the Large Number of Tasks

Although the UI was accessible, the Tasks section contained a large number of fetching tasks.

The next step was therefore to follow the task-cleanup procedure documented in the Broadcom KB - 429753.

The cleanup script reported:

Found 0 stale personality upload tasks.
Found 0 stale upgrade tasks.
Found 1 manifest record(s).
Total tasks found: 2397

The task history contained 2,397 tasks, of which 2,346 were older than 365 days.

Before deleting the old tasks, the script explicitly requested confirmation that a valid SDDC Manager snapshot or backup was available.

After confirmation, the old tasks were removed:

Starting deletion of 2346 Task Cleanup task(s)...

Task Cleanup: 2346/2346 (100.0%)
Deleted: 2346
Status: OK

Deleted 2346 old tasks.

Step 6: Restart the Services Again

Following the task cleanup, the SDDC Manager services were restarted again:

/opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

The restart again covered the common services, Domain Manager, LCM, Operations Manager and the SDDC Manager UI application.

Troubleshooting Flow

The overall troubleshooting sequence can be summarized as:

SDDC Manager UI
      |
      v
Identity Internal Server Error
      |
      v
Check operationsmanager.log
      |
      v
vCenter InvalidLogin
      |
      v
Run password-health check
      |
      v
vCenter authentication/service-account issue
      |
      v
Run SDDC service-account synchronization workaround
      |
      v
Reset & synchronize vCenter service account
      |
      v
Restart SDDC Manager services
      |
      v
SDDC Manager UI loads
      |
      v
Check Tasks
      |
      v
Large number of historical/stale tasks
      |
      v
Clean up old tasks
      |
      v
Restart SDDC Manager services
      |
      v
Validate SDDC Manager

Key Takeaways

This incident highlights a few useful troubleshooting principles for VCF environments.

1. Start with the logs

An Identity Internal Server Error does not necessarily mean that Identity itself is the root cause.

In this case, the Operations Manager log exposed the more useful error:

vim.fault.InvalidLogin

That shifted the investigation toward the SDDC Manager–vCenter authentication path.

2. Use SoS to establish component health

   
   

The sos --password-health --domain-name ALL check provided a quick view of the authentication state across the VCF domain.

It also helped identify which components could be queried successfully and which could not.

3. Service-account synchronization can restore the vCenter connection

The scripted workaround successfully reset the relevant vCenter service account and synchronized the updated credentials back into SDDC Manager.

The most important validation was:

VCENTER password sync completed successfully.

4. Restoring UI access may not be the end of the problem.

After the vCenter synchronization, the UI became accessible, but the Tasks page still showed a large number of tasks.

This demonstrates why post-recovery validation is important. UI accessibility alone should not be treated as complete recovery.

5. Always validate before task cleanup

The cleanup procedure explicitly requested confirmation that a valid SDDC Manager snapshot or backup was available before deleting old tasks. In this case, 2,346 tasks older than 365 days were removed successfully.

Conclusion

When SDDC Manager fails to load with an Identity Internal Server Error, avoid immediately assuming that the Identity component itself is broken.

A better troubleshooting approach is to work from the underlying service dependencies:

UI error → Operations Manager logs → vCenter connectivity → authentication → service-account synchronization → service restart → task validation.

In this VCF 5.2.2 case, the investigation revealed a vCenter InvalidLogin condition. The documented service-account synchronization workaround restored the vCenter connection, after which SDDC Manager services were restarted. A subsequent review of the Tasks section revealed a large backlog of historical tasks, which was cleaned up before performing another service restart.

The important lesson is simple:

Follow the dependency chain, validate each recovery step, and don't stop troubleshooting just because the UI starts loading again.