Enterprise Security Asset field limit warning banner and data truncation during asset merge

Applicability

Description

In Splunk Enterprise Security (ES) 8.x, an alert banner appears on the Asset and Identity Management page:

Identity: <number> assets are currently exceeding the field limits. This can result in data truncation. Sources affected: <lookup_source_name>, merge

This notification indicates that records within an individual asset source or the consolidated asset table contain multivalued fields that exceed the configured multivalue threshold.

When attempting to increase the Multivalue limit in Splunk Web under Configure > Data Enrichment > Asset and Identity Management > Asset fields, the user interface (UI) enforces an input validation constraint:
Must be between 1 and 25

Additionally, two common points of confusion arise from the notification:

1. The message is prefixed with Identity:, even when the alert pertains exclusively to asset records.
2. The notification lists both the custom source lookup name (for example, a Key-Value (KV) Store collection) and merge simultaneously.

Investigation

Step 1: Identify High-Density Multivalued Asset Fields
Run the following search Processing Language (SPL) query against the affected lookup table or Key-Value (KV) Store collection to locate records with high multivalued counts across key identifier fields:

| inputlookup <lookup_or_kvstore_name>
| eval count_ip=coalesce(mvcount(ip), 0), count_mac=coalesce(mvcount(mac), 0), count_dns=coalesce(mvcount(dns), 0), count_nt_host=coalesce(mvcount(nt_host), 0)
| eval max_count=max(count_ip, count_mac, count_dns, count_nt_host)
| sort - max_count
| table _key key ip count_ip mac count_mac dns count_dns nt_host count_nt_host max_count
| head 20

Review the resulting table to determine which specific fields (such as mac, nt_host, ip, or dns) exceed the limit of 25.

Step 2: Understand Warning Message Components
Why the Identity: prefix is displayed: All Asset and Identity Management functions in Splunk Enterprise Security are managed by the Supporting Add-on for Identity Management (SA-IdentityManagement). System health checks and UI banners generated by this add-on use the Identity: component prefix for all notifications regardless of whether the evaluation occurred on assets or identities.

* Why both the source lookup and merge are listed: The Asset and Identity engine evaluates data in two stages:

     * Source Ingestion: Records in the individual source lookup are evaluated against the limit during initial extraction. If a record contains more than 25 values, truncation occurs at the source level.

     * Entity Consolidation (merge): When individual sources are combined into the primary asset tables (asset_lookup_by_str and asset_lookup_by_cidr), the merge engine enforces the limit a second time, triggering truncation on the combined record.

* Cross-Server Warning Displays on Shared Indexer Tiers: In environments with multiple standalone Search Heads sharing an indexer cluster, background health checks query index=_internal across the shared indexers. If one Search Head generates truncation logs, other Search Heads may surface the warning banner in Splunk Web unless log searches are scoped to local hostnames.

Resolution

Option 1: Normalize Upstream Asset Data Pipeline (Recommended Best Practice)
In environments where multiple hostnames or MAC addresses share an IP address (such as Dynamic Host Configuration Protocol (DHCP) pools, Virtual Private Network (VPN) endpoints, or multi-homed network devices), upstream extraction searches or scripts may inadvertently collapse distinct host records together using commands such as mvcombine or stats values().

Refining the search or script that populates the source lookup to preserve distinct 1:1 or 1:few relationships between hostnames, MAC addresses, and IP addresses keeps records well under the default threshold of 25 and resolves the warning banner naturally.

Option 2: Increase Field Limits in inputs.conf (Workaround for UI Validation Limit)
If business requirements necessitate higher multivalued limits for specific fields: 

1. Access the Search Head via Command Line Interface (CLI) or prepare the configuration within the deployment management repository for Search Head Clusters (SHC).

2. Create or edit the local configuration file:
$SPLUNK_HOME/etc/apps/SA-IdentityManagement/local/inputs.conf

3. Locate the [identity_manager] stanza and the field_spec setting.

4. Inside the field_spec JSON string, locate the target field object under the "asset" array (for example, "name": "mac" or "name": "nt_host").

Update the "mv_limit" parameter from 25 to a higher integer (such as 100):

{"name": "mac", "type": "str", "key": true, "mv": true, "mv_limit": 100}

6. Save the file.

7. Trigger a configuration refresh in Splunk Web by navigating to:
https://<searchhead_hostname>:8000/en-US/debug/refresh
Click Refresh.

8. Allow the next scheduled Asset and Identity merge search to complete, or manually trigger the merge. The warning banner will clear once the merge finishes.

Important Note: Do not unmark the Key checkbox in Asset fields to bypass the limit. Unchecking Key disables entity correlation for that field across Enterprise Security detections and notable events.

Related Links

* Memory Considerations: Setting mv_limit significantly higher (for example, greater than 100) across multiple fields can increase memory consumption during the scheduled identity_manager search execution. If search timeouts or memory warnings occur, review limits.conf lookup and subsearch memory configurations.

* UI Navigation Reference for ES 8.x: Asset and Identity Management is accessed inside the Enterprise Security app via Configure > Data Enrichment > Asset and Identity Management or directly via relative URL /app/SplunkEnterpriseSecuritySuite/ess_asset_identity_management.