Log Management in VCF 9.1: Sizing, Partitions, Archiving and Importing Old Logs
In VCF 9.1, Aria Operations for Logs becomes Log management, and it is no longer an appliance you deploy from an OVA. It runs as a containerized service on the VCF management services, and you deploy, configure and use it entirely from VCF Operations. The search is the same idea you know from Log Insight, but sizing, retention and archiving work differently.
This post covers what you need to run it well:
How Log management is built in 9.1
Choosing a deployment model and sizing it
Deploying it, or upgrading from 9.0
Setting up NFS or S3 external storage
Log partitions, retention and archiving
Importing archived logs, including old Aria Operations for Logs archives
Moving off Aria Operations for Logs 8.x
How Log management works in 9.1
It is a service, not a VM. Log management runs as replicas on the VCF services runtime in the first VCF instance. There is no OVA to download and manual OVA deployment is not supported.
It is a Day-N task. It is not part of the initial VCF deployment. You add it from VCF Operations once the management services are running.
One per VCF Operations. Each VCF Operations instance maps to a single Log management instance.
VCF Operations is the console. You search logs under Operate → Logs and configure everything under Operate → Administration → Configurations.
Logs arrive two ways. Sources send logs directly or through a cloud proxy. A cloud proxy is deployed by default and buffers about 20 GB of logs if the network drops.
Management services log themselves. The VCF management services components send their logs automatically. vCenter and NSX Manager need log collection turned on for each adapter in VCF Operations.

Pick a deployment model
Model | What it is | Use it for |
Simple | A single small replica in the first VCF instance. vSphere HA restarts it after a host failure, with an outage of up to about 5 minutes. | Test and development only. |
High Availability | Several replicas behind an integrated load balancer. | Production. This is Broadcom's recommended model. |
Multi-Instance | Separate Log management instances in other VCF instances, not integrated with each other. | Very large fleets, or sites that can't send all their log traffic to one place. |
Sizing Log management
Size for three things: events per second (EPS), the number of connections, and storage. For a first estimate, Broadcom suggests:
EPS ≈ number of ESX hosts in the fleet × 250
Firewall logging and heavy filtering rules can push that number much higher, so check what your sources actually send.
Size profile | Per replica | EPS per replica | Fleet size (ESX hosts) | Replicas |
Small | 8 vCPU, 16 GB | 20,000 | up to 1,500 | Starts at 1 |
Medium | 16 vCPU, 32 GB | 40,000 | up to 3,000 | At least 3 |
Large | 32 vCPU, 64 GB | 60,000 | up to 4,560 | At least 3 |
Storage: each replica starts with 500 GB of disk and can grow to 9 TB.
Spare capacity: add one or two replicas beyond what the EPS needs (N+1 or N+2).
No way back: you can scale up (a bigger profile), scale out (more replicas) and add storage from the UI, but you can't scale down. Scale up during a maintenance window.
Plan the jump early: Broadcom suggests moving to Medium after about 5 to 6 Small replicas, and to Large after 6 to 8 Medium replicas.
Retention has a ceiling: each replica has a fixed number of shards (700, 800 or 900 for Small, Medium and Large), and every day of retention uses shards. Very long retention therefore belongs in the archive, not in the live log store.
Room for the runtime: the replicas run on the VCF services runtime, so make sure it has enough worker-node capacity and IP addresses.
Worked example
A fleet with 120 ESX hosts gives roughly 120 × 250 = 30,000 EPS.
Two Small replicas cover 40,000 EPS. Adding one spare for N+1 gives three Small replicas.
If you expect the fleet to keep growing, start with three Medium replicas (120,000 EPS) so you don't have to change profile later.

What about the Aria Operations for Logs sizer?
The familiar sizing estimator and the calculator spreadsheet in Broadcom KB 344251 size Aria Operations for Logs 8.x appliance clusters: Small, Medium and Large nodes rated at 2,000, 5,000 and 15,000 EPS. They are still handy for one thing: estimating what your current 8.x cluster ingests before you migrate. Size the 9.1 deployment with the replica profiles above, not with the 8.x node sizes.
Deploy Log management
Before you start
VCF Operations and the VCF management services are running on 9.1.
A new, unique FQDN for Log management, in lowercase, with forward and reverse DNS records.
An IP address on the same network as the VCF management services, outside the IP range assigned to the services runtime. Don't use the subnet's broadcast address.
Log management can't run on a custom NSX overlay network.
New deployment
Log in to VCF Operations as a user with the Administrator role.
Go to Build → Lifecycle → VCF Management.
On the Components tab, click Add component and choose Log management.
Enter the deployment details, including the FQDN, the size profile and the number of replicas.
Click Next, review the Summary and click Finish.
If you are coming from Aria Operations for Logs 8.18.x or VCF Operations for Logs 9.0, the wizard can import settings from the old instance: log filters, masking, forwarder rules and existing log collection settings. Afterwards, a PDF report lists what moved and what needs manual work.
Upgrading from VCF Operations for Logs 9.0
Go to Build → Lifecycle → VCF Management and open the Upgrade tab.
Next to Log management, click Configure. Enter the new Log management FQDN, the Size and the Number of replicas, then click Finish.
Next to VCF Operations for logs, click Upgrade and follow progress in Upgrade details.
When it finishes, download the Upgrade report from the deployment task on the Tasks tab.
The upgrade deploys a new 9.1 instance and transfers the configuration. From 9.0, agents and log sources are repointed automatically. Imported log forwarders arrive inactive, so turn them on yourself.
Where to find it afterwards: use Operate → Logs in VCF Operations. The Log management FQDN also serves a standalone UI, but it is only an emergency fallback for reading logs when VCF Operations is down.
Set up external storage
Archiving, exporting and importing logs all need an external storage location, so set one up first. Log management supports NFS and S3, including S3-compatible storage. Broadcom recommends S3 for archives. If an NFS location is unreachable when the Log management service starts, the service fails to start.
Prepare the storage
NFS: create the export first (vSAN File Service or your NAS). The root user must create the target directory, with permissions of at least 447 (777 also works). On the NFS server, run chmod -R 447 /path/to/archive as root.
S3: the bucket must already exist. The endpoint needs a certificate from a trusted CA (self-signed is not supported) and must support chunked transfer encoding. The credentials need s3:ListBucket, s3:HeadBucket, s3:PutObject and s3:DeleteObject.
Firewalls: logs can leave from any node, so allow every IP in the VCF management services node pool to reach the NFS server or S3 endpoint.
Capacity: check that there is enough free space, and keep checking it.
Add the location
In VCF Operations, go to Operate → Administration → Configurations and click the External Storage card.
Click Add New Storage and choose NFS or S3. You can add up to five locations.
For NFS, enter a Host Name and the Host Address in the form nfs://<server>:/<export>. For S3, enter the name, access key, secret key, bucket name, endpoint and region. Leave Canned acl on private, and turn on Path Style Access only if your S3-compatible storage needs it.
Click Validate, then SAVE.
Click APPLY and Continue.
Plan for a short outage: applying the change restarts the service, so Log management is unavailable for a few minutes. Logs that aren't buffered by a cloud proxy with data persistence can be lost during the restart.
Partitions, retention and archiving
Log partitions let you keep different logs for different lengths of time. Each partition has its own retention period, filters and, optionally, an archive location.
Built-in partitions: Audit records and Default partition. You can't change their filters.
Custom partitions: you can add up to 9.
Routing: partitions are checked from top to bottom. Audit records always comes first and Default partition always last. A log goes to the first partition whose filter matches, so the order matters.
Retention: set in days, with a minimum of 1. A value of 0 keeps logs until storage runs short.
The 80% rule: whatever retention you set, once the log store reaches 80%, the oldest logs are purged first, even if they are still inside their retention period.

Create a partition
Go to Operate → Administration → Configurations, click the Log Processing card and open the Log Partitions tab.
Click ADD, or use the ⋮ menu on an existing row to edit it.
Enter a Name and a Retention Period in days, and select Enable Configuration.
To archive, select Enable Archive, choose the Source (an external storage location) and enter a Storage Path relative to the root of that location, without a leading slash.
Add one or more filters, for example a hostname, an app name or a key-value pair.
Click PREVIEW to check what matches, then CREATE.
Example: keep NSX Distributed Firewall logs for 7 days in the live store, archived to S3, while vCenter and ESX logs stay in the Default partition for 30 days. Put the firewall partition first in the order so its filter is checked before broader filters.
Check the preview: the partition preview doesn't filter exactly the way the search on the Logs page does.
How archiving behaves
Timing: logs aren't held until they expire. They are archived when the index rotates, which happens at least every 24 hours, or sooner under heavy ingestion. The archive is therefore up to about a day behind.
No housekeeping: Log management doesn't prune or rotate archive files. Delete old archives with your own tooling or a storage lifecycle policy.
No free space, no archive: copying to the archive continues only while the storage has free space.
Format: archives use the same .blob format as Aria Operations for Logs and vRealize Log Insight, so older archives can be imported into 9.1.
Import archived logs
Logs that have left the live store can't be searched until you import them back.
Go to Operate → Administration → Configurations → Log Processing, open the Log Partitions tab and scroll to Imported Log Archives.
Click INITIATE NEW LOG IMPORT.
Enter a unique Name.
Choose the Operation Type: import for archives written by Log management, or legacy import for archives from Aria Operations for Logs.
Choose the Source location and enter the Storage Path, matching the path the logs were archived to.
Set the Collection Period start and end dates (MM/DD/YYYY). Only logs inside this window are imported.
Click IMPORT LOGS.

Follow the job under Operate → Administration → Control Panel → Log Import/Export Tasks. When it completes, open Operate → Logs, choose the archived logs in the import partition drop-down and set the time range you imported.
Legacy import is NFS only.
One at a time: only one imported archive can be active. To search another, purge the current one and import again.
Keep the window small: an import that pushes storage past 80% fails, so import a few days at a time.
One partition per query: a search runs against one partition at a time.
Moving off Aria Operations for Logs 8.x
No in-place upgrade. Deploy Log management as a new instance and import the 8.18.x settings by entering the old instance's FQDN and admin credentials in the wizard.
Old logs, live: for 90 days after installation, the Logs page can still query the legacy instance, if you connected it during the workflow. Finish dataset work before the connection expires.
Old logs, copied: Operate → Administration → Control Panel → Logs Data Transfer copies up to 90 days of logs. It needs identical SSH credentials on every source node, port 22 open from the Log management nodes and enough free disk space. Start with about 10 days.
Older still: point a legacy import at the 8.x NFS archive.
Agents and sources: from 8.18, reconfigure agents and log sources yourself, and repoint forwarders to the new FQDN.
Custom content: dashboards, queries and alerts don't transfer. Package them as a content pack and convert it to a management pack for VCF Operations.
Last step: only then shut down and remove the old appliances.
Day-2 checklist
Recalculate EPS when you add hosts, and add replicas before you run short.
Watch log store usage. The 80% purge removes logs silently.
Watch archive capacity and prune old archives on a schedule.
Confirm that vCenter and NSX log collection is turned on for each adapter.
Use a cloud proxy for remote sites so logs are buffered during WAN outages.
Test an archive import once, before you need it in an incident.



Comments