Removing VCF Log Management 9.1 to Free Up 2 TB
I run a 24/7 Dell XR4000 Prod’ Cluster that does not have a lot of storage to spare. With VMware vSphere Foundation (VVF) 9.1, Log Management (formerly VCF Operations for Logs) became an integrated service that runs on the VCF Management Services platform, and in my environment it was taking up almost 2 TB. For a cluster of my size that is simply too much, and I do not need centralized log management there. So the decision was easy: Log Management has to go.
What I expected to be a quick click in the UI turned into a small adventure. There is no uninstall button, I needed a cleanup script from Broadcom, the script needs the admin@vsp.local password, and I had no idea what that password was. This post walks through everything I did, including both password resets, so you do not have to piece it together from several knowledge base articles like I did.
Log Management 9.1 can be removed with Broadcom’s cleanup_component.py script. The script logs in as admin@vsp.local. If you do not know that password, you can reset it on a control plane node of the VCF Services Runtime. For that you need SSH access as vmware-system-user, whose password can also be reset. The script runs from any machine with Python 3 – in VVF there is no SDDC Manager and you do not need one.
The Environment
- VMware vSphere Foundation (VVF) 9.1.1, managed through VCF Operations
- Log Management component version 9.1.1.0.25679624, deployed natively on VCF Management Services
- Fleet Lifecycle
- VCF Services Runtime
- An Ubuntu VM to run the script from
Why there is no simple uninstall
In VCF/VVF 9.1 several components can be deployed after the initial setup under Build → Lifecycle → VCF Management → Components → Add Component. When I looked for the opposite function, there was none. At the time of writing, I could not find an uninstall workflow in that UI.
What Broadcom provides instead is a Python script, cleanup_component.py, attached to a knowledge base article (KB 441333). William Lam also describes it in a blog post about uninstalling optional Day-N components. A few things to know before you use it:
- The KB lists VCF 9.1 and VVF 9.1 as the environment. It does not apply to 9.0 deployments, where components are deleted directly in Fleet Management.
- For a natively deployed Log Management 9.1 (running on VCF Management Services) the script removes the solution completely from the fleet. That was my case, and it uses the vsp-component commands.
- For a Log Management that was imported from 9.0 (OVA appliance), the script only removes the entry from the fleet lifecycle. The old appliance keeps running and has to be deleted separately. That variant uses ova-component.
Warning: Removing the component is irreversible. Broadcom’s KB explicitly asks you to make sure you have a backup of any logs or data you still need before running the cleanup. If you change your mind later, you can deploy Log Management again via Add Component, but the old data will be gone.
The Catch: two passwords I did not remember
The script authenticates against the VCF Services Runtime as admin@vsp.local. Every VCF Services Runtime has two accounts:
- vmware-system-user is the local OS account that you use to SSH into a Services Runtime node.
- admin@vsp.local is the API account used to talk to the components running on the platform. This is the one the cleanup script needs.
Both passwords are set during the deployment of the VCF Management Services. The password you enter in the deployment wizard is used as the initial value for both accounts, and it is only visible in the Deployment UI. I did not have it anymore.
The two resets depend on each other like this:
- For the cleanup script I only need the admin@vsp.local password.
- To reset admin@vsp.local, I need a shell on a control plane node of the VCF Services Runtime, which means SSH access as vmware-system-user.
- So: if you do not know vmware-system-user either, do Option 1 first. If you do know it, skip straight to Option 2.
Option 1: Resetting the vmware-system-user password
This follows Broadcom KB 444453. The idea is to boot a control plane node into single-user mode, create a temporary user, and use it to write a new password into a Kubernetes secret. That secret then drives the password update on all VMs of the cluster.
Heads-up: The change is cluster-wide. As soon as the Kubernetes secret is patched, all virtual machines in the cluster update the password for vmware-system-user. I also recommend taking a VM snapshot of the control plane node before you reboot it into single-user mode. That snapshot is my own precaution and is not part of the KB.
Step 1: Find a control plane node
In VCF Operations go to Build → Lifecycle → VCF Management → Components, open the VCF services runtime, scroll to the Nodes section and note the name and IP address of a Control Plane node. If the control plane node is not visible there, the KB suggests looking at the memory usage of the VMs in vCenter. The control plane node is the one with the lowest memory usage.
Step 2: Boot into single-user mode
In the vSphere Client open a console for that VM and choose Power → Restart Guest OS. When the Photon bootloader screen appears, press e, move the cursor to the end of the kernel command line, add the following, and press F10:
rw init=/bin/bash
Step 3: Create a temporary user with sudo rights
The password for the temporary user needs 15 characters, with no more than three sequential characters of the same class.
useradd temp
passwd temp
echo "temp ALL=(ALL) ALL" >> /etc/sudoers.d/90-cloud-init-users
reboot -f
Step 4: Write the new password into the Kubernetes secret
After the reboot I logged in via SSH as temp, switched to root, and ran the following commands:
sudo su -
CLUSTER=$(kubectl get configmap global-config -n vmsp-platform -o jsonpath='{.data.cluster\.name}')
read -sp "Enter password: " NEW_PASSWORD
export NEW_PASSWORD
HASH=$(printf '%s' "${NEW_PASSWORD}" | vmsp passwd --password-stdin | tr -d '\n' | base64 -w0)
kubectl patch secret "${CLUSTER}-ssh-password-secret" -n vmsp-platform --type=merge -p "{\"data\":{\"sshPassword\":\"${HASH}\"}}"
If the password meets the policy, the vmsp passwd command prints a short validation message similar to this:
2026/10/21 18:17:22 /opt/vsp/password-policy.json not exist
2026/10/21 18:17:22 validating password against local pwquality.conf
Step 5: Verify and clean up
Now check that you can log in as vmware-system-user via SSH or the VM console with the new password. Then remove the temporary user:
sudo userdel temp
Option 2: Resetting the admin@vsp.local password
This follows Broadcom KB 444456. It is a much shorter procedure: you replace the password hash in the vsp-admin-secret Kubernetes secret. The KB notes that this is meant for the case where the password cannot be managed through the Fleet Management UI or API.
Step 1: Connect to a control plane node
I connected via SSH as vmware-system-user to the control plane node. My first attempt at the backup command failed, because kubectl did not find any Kubernetes configuration for that user:
kubectl get secret vsp-admin-secret -n vmsp-platform -o yaml > vsp-admin-secret.yaml.back
E1021 18:11:50.753386 13017 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"http://localhost:8080/api?timeout=32s\": dial tcp 127.0.0.1:8080: connect: connection refused"
...
The connection to the server localhost:8080 was refused - did you specify the right host or port?
The fix is to become root and point kubectl to the admin kubeconfig. The KB uses exactly this in its validation section:
kubectl get secret vsp-admin-secret -n vmsp-platform -o yaml > vsp-admin-secret.yaml.back
E1021 18:11:50.753386 13017 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"http://localhost:8080/api?timeout=32s\": dial tcp 127.0.0.1:8080: connect: connection refused"
...
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Watch out: Because of the failed first attempt, the redirect created an empty vsp-admin-secret.yaml.back in my home directory. Always check that your backup file really contains something before you continue. A valid backup starts with apiVersion: v1 and contains the password field.
Step 2: Back up the secret
kubectl get secret vsp-admin-secret -n vmsp-platform -o yaml > vsp-admin-secret.yaml.back
head -5 vsp-admin-secret.yaml.back
Step 3: Choose a new password
The new admin@vsp.local password has to meet these requirements:
- 15 to 64 characters
- At least one lowercase letter, one uppercase letter, one number and one special character from @ # $ ^ * !
- No dictionary word, no palindrome, no more than four monotonic character sequences and no three identical consecutive characters
Step 4: Set the new password
export NEW_PASSWORD='<your-new-password>'
hash=$(printf '%s' "${NEW_PASSWORD}" | vmsp passwd --password-stdin | tr -d "\n" | base64 -w0)
2026/10/21 18:17:22 /opt/vsp/password-policy.json not exist
2026/10/21 18:17:22 validating password against local pwquality.conf
kubectl patch secret vsp-admin-secret -n vmsp-platform -p "{\"data\":{\"password\":\"${hash}\"}}"
secret/vsp-admin-secret patched
Step 5: Check that the password works
The KB describes a small check that requests a bearer token with the new password. First I looked up the FQDN of the identity route:
kubectl get httproute vmsp-identity-server-route -n vmsp-platform
NAME HOSTNAMES AGE
vmsp-identity-server-route ["vvfsvc01.irgnet.wtf","vvffleet01.irgnet.wtf"] 89d
Then I requested a token with the Services Runtime FQDN. Here is a shortened version of the KB script that does not print the token:
export BASE_URL="vvfsvc01.irgnet.wtf"
export VSP_ADMIN_PASSWORD='<your-new-password>'
TOKEN=$(curl -sk --fail -XPOST "https://${BASE_URL}/api/v1/identity/token" \
--header 'content-type: application/x-www-form-urlencoded' \
--data 'grant_type=password' \
--data 'username=admin@vsp.local' \
--data "password=${VSP_ADMIN_PASSWORD}" | jq -r '.access_token')
[[ -n "$TOKEN" && "$TOKEN" != "null" ]] && echo "Auth token obtained." || echo "ERROR: Failed to obtain Bearer token"
Getting a token back means the password is set and the cleanup script will be able to authenticate.
Running the Cleanup Script
The KB says the script can run on any system with Python 3 and network access to the Fleet and VCF Management Services components, and recommends the SDDC Manager in VCF environments. VVF has no SDDC Manager, and I did not need one. Looking through the script before running it, I saw that the SDDC Manager check only applies to the vsp-cluster command, which is used for VCF Automation. For vsp-component it is not involved, so I ran it from an Ubuntu VM.
Step 1: List the component
The list command is read-only and shows which components can be removed. It is also a good test that authentication and networking work:
python3 cleanup_component.py list vsp-component --fleet-fqdn vvffleet01.irgnet.wtf --vcf-services-runtime-fqdn vvfsvc01.irgnet.wtf
Enter password for user 'admin@vsp.local' on 'vvfsvc01.irgnet.wtf':
Authenticating with VSP platform (vvfsvc01.irgnet.wtf)...
Authentication successful.
Fetching VSP components from Fleet...
Available VSP Components:
-------------------------------------------------------------------------------------------------------
ID | Type | Deployment | Version | FQDN | VCF Services Runtime FQDN
-------------------------------------------------------------------------------------------------------
<component-id> | Log management | VSP | 9.1.1.0.25679624 | vvflogs01.irgnet.wtf | vvfsvc01.irgnet.wtf
-------------------------------------------------------------------------------------------------------
There was exactly one entry, Log management, with the deployment type VSP. That confirmed it is the native 9.1 variant. I copied its ID for the next step.
Step 2: Delete the component
python3 cleanup_component.py delete vsp-component --component-id <component-id> --fleet-fqdn vvffleet01.irgnet.wtf --vcf-services-runtime-fqdn vvfsvc01.irgnet.wtf
Enter password for user 'admin@vsp.local' on 'vvfsvc01.irgnet.wtf':
Authenticating with VSP platform (vvfsvc01.irgnet.wtf)...
Authentication successful.
About to delete component '<component-id>' of type 'Log management' with fqdn vvflogs01.irgnet.wtf from the system?
Proceed with deletion? (y/n): y
Before anything is deleted, the script shows the type and FQDN of the component and asks for confirmation. This is the moment to double-check that it really says Log management and the FQDN you expect.
What happends next
The script first deletes the component from the Fleet. In parallel I could watch the same task in VCF Operations under Build → Lifecycle → VCF Management → Tasks. It is started by admin@vsp.local and shows four subtasks: unregistering the component from VCF Operations, deleting it from the SDDC lifecycle service, deleting the record and credentials from the SDDC lifecycle inventory, and deleting it from the Fleet lifecycle inventory.
Deleting component '<component-id>' from Fleet...
Delete task initiated (ID: <task-id>)
Waiting for task <task-id> to complete (timeout: 7200 seconds)...
Task status: RUNNING | Elapsed time: 0s
Task status: RUNNING | Elapsed time: 30s
Task status: RUNNING | Elapsed time: 60s
Task status: RUNNING | Elapsed time: 90s
Task status: RUNNING | Elapsed time: 120s
Task status: RUNNING | Elapsed time: 150s
Task status: RUNNING | Elapsed time: 180s
Task status: RUNNING | Elapsed time: 210s
Task status: RUNNING | Elapsed time: 240s
Task status: SUCCEEDED | Elapsed time: 270s
Successfully deleted component '<component-id>' from Fleet.
Waiting 120 seconds before checking if component <component-id> has been successfully deleted from Fleet...
The Fleet part took 270 seconds, and VCF Operations showed the task as completed about four minutes after it started:
Then the script waits two minutes and checks whether the component is really gone from the Fleet. The 404 message at this point is exactly what you want to see. After that it deletes the component from the VCF Services Runtime (VSP) itself, which took another six minutes:
Component '<component-id>' does not exist in Fleet (HTTP 404).
Deleting component '<component-id>' from VSP...
Delete task initiated (ID: <task-id>)
Waiting for task <task-id> to complete (timeout: 7200 seconds)...
Task status: PENDING | Elapsed time: 0s
Task status: RUNNING | Elapsed time: 30s
Task status: RUNNING | Elapsed time: 60s
Task status: RUNNING | Elapsed time: 90s
Task status: RUNNING | Elapsed time: 120s
Task status: RUNNING | Elapsed time: 150s
Task status: RUNNING | Elapsed time: 180s
Task status: RUNNING | Elapsed time: 210s
Task status: RUNNING | Elapsed time: 240s
Task status: RUNNING | Elapsed time: 270s
Task status: RUNNING | Elapsed time: 300s
Task status: RUNNING | Elapsed time: 330s
Task status: SUCCEEDED | Elapsed time: 360s
Successfully deleted component '<component-id>' from VSP.
The Result
Back in VCF Operations, the component list under Build → Lifecycle → VCF Management → Components now only shows VCF Operations, VCF services runtime, Fleet lifecycle, Software depot, SDDC lifecycle and Telemetry. Log management is gone, and it is offered again in the Add Component menu, which is also how you would deploy it again if you ever need it:
And that’s it from this Blog Post, if you have any question please use the comment section or consider to contact Broadcom Support! 🙂
Important: The VCF Log Management Logo is just a fantasy logo made by myself and is not official from Broadcom.