Splunk实例运行状况显示桶组件的RED
适用范围
Splunk企业描述
在过去一小时内，创建的小存储段(100)的百分比非常高，并且超过此索引器上index=<index name>的红色阈值(50)，还可能超过更多索引。“运行状况”(Health)检查选项卡中的错误消息：
Buckets Root Cause(s): The percentage of small of buckets created (100) over the last hour is very high and exceeded the red thresholds (50) for index=mbam, and possibly more indexes, on this indexer最近50条相关消息：
07-02-2019 02:16:05.989 -0700 INFO HotBucketRoller - finished moving hot to warm bid=_internal~4984~99D2CFA5-1005-45DA-8C68-0E8A5FB0C95C idx=_internal from=hot_v1_4984 to=db_1562087584_1562043339_4984 size=1048645632 caller=size_exceeded _maxHotBucketSize=1048576000 (1000MB), bucketSize=1048641536 (1000MB) 07-02-2019 00:22:57.931 -0700 INFO HotBucketRoller - finished moving hot to warm bid=_introspection~2046~99D2CFA5-1005-45DA-8C68-0E8A5FB0C95C idx=_introspection from=hot_v1_2046 to=db_1562027109_1562015374_2046 size=569110528 caller=lru maxHotBuckets=3, count=4 hot buckets,evicting_count=1 LRU hots 07-02-2019 00:00:22.067 -0700 INFO HotBucketRoller - finished moving hot to warm bid=cloudflare~324~99D2CFA5-1005-45DA-8C68-0E8A5FB0C95C idx=cloudflare from=hot_v1_324 to=db_1561885206_1561791602_324 size=394207232 caller=lru maxHotBuckets=3, count=4 hot buckets,evicting_count=1 LRU hots 07-01-2019 23:03:37.981 -0700 INFO HotBucketRoller - finished moving hot to warm bid=_internal~4983~99D2CFA5-1005-45DA-8C68-0E8A5FB0C95C idx=_internal from=hot_v1_4983 to=db_1562076172_1562032683_4983 size=1054523392 caller=size_exceeded _maxHotBucketSize=1048576000 (1000MB), bucketSize=1054404608 (1005MB) 07-01-2019 22:43:06.030 -0700 INFO HotBucketRoller - finished moving hot to warm bid=clearpass~362~99D2CFA5-1005-45DA-8C68-0E8A5FB0C95C idx=clearpass from=hot_v1_362 to=db_1561787045_1561538116_362 size=393191424 caller=lru maxHotBuckets=3, count=4 hot buckets,evicting_count=1 LRU hots
调查
上面的消息是说在过去24小时内创建的存储桶的100%小于maxDataSize的10%。这不会引起任何性能问题。要解决此警告，您可以将阈值从50增加到150，如下所示。默认配置位于$SPLUNK_HOME/etc/system/default/health.conf位置
[feature:buckets] display_name = Buckets indicator:percent_small_buckets_created_last_24h:description = "This indicator tracks the percentage of small buckets created over the last 24 hours. small bucket is defined as less than 10 % of the ‘maxDataSize’ setting in indexes.conf." indicator:percent_small_buckets_created_last_24h:red = 50 indicator:percent_small_buckets_created_last_24h:yellow = 30
此外，当为特定小时/天的数据编制索引而创建的热桶数量超过或达到system/health.conf中定义的阈值限制时，也会显示上述运行状况警告消息。
创建过多热桶的原因
Splunk使用存储段作为索引目录来索引数据。现在，当一个事件进入Splunk进行索引时，将为该事件创建新的热桶，或者根据事件的时间戳和对索引的约束，将该事件索引到现有的一个热桶。
请参阅indexes.conf的maxHotBuckets、maxHotSpanSecs和quarantinePastSecs参数
因此，如果事件的时间戳比现有存储段可接受的时间戳旧/奇怪，则Splunk将创建一个新的热存储段以索引该事件。因此，所创建的热桶数量可以达到或超过阈值。
props.conf的DATETIME_CONFIG参数未显式设置时，Splunk将尝试在事件本身中查找时间戳。
因此，从事件中提取时间戳时，如果Splunk找到与有效时间戳格式匹配的值，它将解析与事件时间相同的值，并仅使用该时间戳对其进行索引，这可能导致Splunk创建新的热桶，因为时间戳提取不正确。
请考虑在props.conf中添加时间戳属性，以使Splunk正确提取时间戳。配置时间戳识别
此外，我在splunkd.log中找到了以下有关时间戳的消息，并且根据日志格式应用正确的设置来修复它们。
06-02-2019 22:37:36.976 -0700 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (30) characters of event.
Defaulting to timestamp of previous event (Mon Jun 3 02:37:35 2019). Context: source=/data/syslogng/asa/richmond-vpn-inside/20190603.log|host=richmond-vpn-inside
|cisco:asa|48952806-21-2019 09:48:48.467 -0700 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (44) characters of event.
Defaulting to timestamp of previous event (Fri Jun 21 06:48:45 2019). Context: source=/data/syslogng/pan/vcadcl-fw-mgmt/20190621.log|host=vcadcl-fw-mgmt|pan_log|86736606-20-2019 11:29:10.742 -0700 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event.
Defaulting to timestamp of previous event (Thu Jun 20 11:29:10 2019). Context: source=/scratch/data/syslogng/vlsj-securelog/sj-anliu1/20190620.log|host=sj-anliu1
|linux_secure|526909106-14-2019 12:46:38.301 -0700 WARN DateParserVerbose - Accepted time format has changed ((?i)\s([01]\d)([0123]\d\d)\s), possibly indicating a problem
in extracting timestamps. Context: source=C:\Windows\debug\netlogon.log|host=DC2SJGLOBAL|MSAD:NT6:Netlogon|4001606-02-2019 16:17:18.106 -0700 WARN DateParserVerbose - Accepted time format has changed ((?i)(?<!\w|\d[:\.\-])(?i)(?<![\d\w])(jan|\x{3127}\x{6708}|feb|\x{4E8C}\x{6708}
|mar|\x{4E09}\x{6708}|apr|\x{56DB}\x{6708}|may|\x{4E94}\x{6708}|jun|\x{516D}\x{6708}|jul|\x{4E03}\x{6708}|aug|\x{516B}\x{6708}|sep|\x{4E5D}\x{6708}|oct|\x{5341}\x{6708}
|nov|\x{5341}\x{3127}\x{6708}|dec|\x{5341}\x{4E8C}\x{6708})[a-z,\.;]*([/\- ]) {0,2}(?i)(0?[1-9]|[12]\d|3[01])(?!:) {0,2}(?:\d\d:\d\d:\d\d(?:[\.\,]\d+)? {0,2}(?i)((?:(?:
UT|UTC|GMT(?![+-])|CET|CEST|CETDST|MET|MEST|METDST|MEZ|MESZ|EET|EEST|EETDST|WET|WEST|WETDST|MSK|MSD|IST|JST|KST|HKT|AST|ADT|EST|EDT|CST|CDT|MST|MDT|PST|PDT|CAST|CADT|EAST
|EADT|WAST|WADT|Z)|(?:GMT)?[+-]\d\d?:?(?:\d\d)?)(?!\w))?)?((?:\2|,) {0,2}(?i)(20\d\d|19\d\d|[901]\d(?!\d)))?(?!/|\w|\.\d)), possibly indicating a problem
in extracting timestamps. Context: source=C:\Windows\System32\DHCP\DhcpSrvLog-Mon.log|host=srv07-seoul|DhcpSrvLog|286986
分辨率
1.导航至$SPLUNK_HOME/etc/system/local/health.conf并追加以下内容[feature:buckets] indicator:percent_small_buckets_created_last_24h:red = 150
2.重新启动Splunk以使更改生效。
如果是时间戳或DATETIME_CONFIG问题，请写下适当的正则表达式，以正确识别事件的日期和时间，并纠正问题。
相关链接
Splunk文档配置时间戳识别
Splunk答案
运行状况：创建的小桶百分比
修订历史记录
| 版本 | 发布日期 | 备注 |
|---|---|---|
1.0 |
28-Sep-2026
|
初始版本 |
反馈