The NetFlow Collection Engine is the unsung hero of network observability. It tames the firehose of metadata from routers and switches, turning raw export datagrams into structured, searchable, enrichable records. Whether you run a small corporate network with nfdump on a VM or a global ISP using distributed pmacct clusters feeding a data lake, the principles remain the same: handle UDP loss gracefully, store efficiently, enrich meaningfully, and always respect sampling.
Flow analysis relies heavily on timestamps. If the router's clock (exporter) and the collector's clock are not synced via NTP, flow reconstruction becomes inaccurate, leading to broken TCP session graphs. netflow collection engine
| Protocol | Typical Export | Key Characteristics | |----------|----------------|----------------------| | | UDP | Fixed format, IPv4 only. Still widely used on legacy hardware. Lacks templates. | | NetFlow v9 | UDP | Template-based, supports IPv6, MPLS, and custom fields. Foundation for IPFIX. | | IPFIX | UDP/TCP/SCTP | IETF standard (RFC 7011-7015). Essentially NetFlow v9 with enterprise-specific extensions and reliable transport options. | | sFlow | UDP | Packet sampling (not flow-based). A single datagram can contain multiple flow samples and counter samples. Different architecture. | | J-Flow / NetStream | UDP | Juniper and Huawei variants, typically v5 or v9 compatible. | The NetFlow Collection Engine is the unsung hero