Information About Flexible NetFlow—Prevent Export Storms
Flexible NetFlow—Prevent Export Storms Overview
The Flexible NetFlow—Prevent Export Storms feature prevents export storms at a NetFlow Collecting (NFC) device, especially when multiple Flexible NetFlow (FNF) entities are configured to export FNF records to the same NFC at the same synchronized wallclock time. Export storms occur due to the creation of the synchronized cache type. Export spreading reduces the severity of export storms and mitigates their impact.
Synchronized cache with spreading requires adding the interval timestamp field for the synchronized cache. When no spreading is configured, it is recommended to add the interval as a key, but the configuration is not rejected to maintain backward compatibility. If no export spread is specified, the default behavior is immediate export. The spread time must be smaller than half of the interval. Therefore, it will be set to half the interval time or to the configured spread interval, whichever is lower (but not lower than 1 second).
- The simple implementation is a constant rate based on the cache-size/spread-interval.
- An improved implementation is based on the current-previous-interval-cache-size/spread-interval. This provides better results when the cache is not full.
The NetFlow/IPFIX header timestamp is set to the time when the record leaves the device (and not when the record leaves the NetFlow cache). The timestamp fields in the record itself capture the timestamp of the packets and are accounted for in the NetFlow cache. A new, implementation-driven concept of a “small interval” is now implicitly introduced and understood to be directly in contrast with the concept of a “large interval”. The “large interval” can be thought of as simply the sync interval as configured by the CLI. This is the interval at the beginning of which the entire export process is to be initiated. It corresponds to the "synchronized interval" that is driven and defined by the CLI. At the beginning of a “large interval”, we must take the number of records in the cache and divide that number by the number of seconds available in which to export these records, thus yielding the calculated or derived quantity of “records per second”.
For example, if there are 100,000 records in the cache and 100 seconds in which to export these records, we would calculate and store the value 1000 records/second. Because this quantity is expressed in seconds, it follows that we will need to count the records exported in small intervals that are one second in duration.
This then, implicitly, defines the notion of a “small interval”, which is, to be succinct and equal to one second. Combining this idea of small and large intervals with the need for a state or context, it quickly becomes evident that a timer thread must be able to discern if it is beginning a “small interval” or a “large interval”.