ElastiFlow: Monitoring network traffic with Netflow and ELK Stack and OpenWRT (Deprecated)


Setup NetFlow on OpenWRT

NetFlow is a network protocol system created by Cisco that collects active IP network traffic as it flows in or out of an interface. In my case all traffic runs from router with OpenWRT. To activate netflow monitoring we need to install softflowd package.
softflowd - is a software implementation of a flow-based network traffic monitor. softflowd reads network traffic and gathers information about active traffic flows.

1opkg update
2opkg install softflowd
3/etc/init.d/softflowd start # or stop, or restart

Config file: /etc/config/softflowd

 1root@OpenWrt:~# cat /etc/config/softflowd
 2config softflowd
 3        option enabled        '1'
 4        option interface      'br-lan'
 5        option pcap_file      ''
 6        option timeout        'maxlife=60'
 7        option max_flows      '8192'
 8        option host_port      ''
 9        option pid_file       '/var/run/softflowd.pid'
10        option control_socket '/var/run/softflowd.ctl'
11        option export_version '5'
12        option hoplimit       ''
13        option tracking_level 'full'
14        option track_ipv6     '0'
15        option sampling_rate  '10'

Useful commands:
softflowctl statistics
softflowctl dump-flows

 1root@OpenWrt:~# softflowctl statistics
 2softflowd[4563]: Accumulated statistics since 2021-02-07T19:39:59 UTC:
 3Number of active flows: 27
 4Packets processed: 16176338
 5Fragments: 2352200
 6Ignored packets: 55416 (55410 non-IP, 6 too short)
 7Flows expired: 59561 (0 forced)
 8Flows exported: 114321 in 6271 packets (0 failures)
 9Packets received by libpcap: 16231910
10Packets dropped by libpcap: 0
11Packets dropped by interface: 0
13Expired flow statistics:  minimum       average       maximum
14  Flow bytes:                  40        233515     740181706
15  Flow packets:                 1           272        728069
16  Duration:                  0.00s        17.90s       111.24s
18Expired flow reasons:
19       tcp =     13888   tcp.rst =      6337   tcp.fin =     12323
20       udp =     18408      icmp =       577   general =        20
21   maxlife =      8008
22over 2 GiB =         0
23  maxflows =         0
24   flushed =         0
26Per-protocol statistics:     Octets      Packets   Avg Life    Max Life
27           icmp (1):         712567         2244      17.02s      64.16s
28           igmp (2):           3656           89      10.80s      60.74s
29            tcp (6):     9019595755      9503208      23.06s     111.24s
30           udp (17):     4888056323      6670438       8.06s      90.58s
1root@OpenWrt:~# softflowctl dump-flows
2softflowd[4563]: Dumping flow data:
3ACTIVE seq:59825 []:46672 <> []:443 proto:6 octets>:2929 packets>:16 octets<:6911 packets<:16 start:2021-02-09T02:54:37.540 finish:2021-02-09T02:54:42.650 tcp>:1b tcp<:1b flowlabel>:00000000 flowlabel<:00000000 
4EXPIRY EVENT for flow 59825 now
6ACTIVE seq:59826 []:443 <> []:54510 proto:6 octets>:195 packets>:3 octets<:414 packets<:6 start:2021-02-09T02:54:38.569 finish:2021-02-09T02:55:32.298 tcp>:19 tcp<:19 flowlabel>:00000000 flowlabel<:00000000 
7EXPIRY EVENT for flow 59826 now

ELK Configuration

The ELK stack is an acronym used to describe a stack that comprises of three popular open-source projects: Elasticsearch, Logstash, and Kibana. Often referred to as Elasticsearch, the ELK stack gives you the ability to aggregate logs from all your systems and applications, analyze these logs, and create visualizations for application and infrastructure monitoring, faster troubleshooting, security analytics, and more.

Instalation: Here my installation example(on ubuntu 20.04 server), but I recommend to check official documentation because it changes pretty quickly.

1sudo apt install apt-transport-https
2wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
3echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
4sudo apt update
5sudo apt install elasticsearch logstash kibana
6sudo systemctl enable elasticsearch.service logstash.service kibana.service  
7sudo systemctl start elasticsearch.service kibana.service

Cause this installation for educational purposes, all configuration files are left by default.

ElastiFlow Instalation

ElastiFlow™ - provides network flow data collection and visualization using the Elastic Stack. It supports Netflow v5/v9, sFlow and IPFIX flow types. In simple words is a set of configuration files for logstash and preconfigured kibana dashboard.
Full config settings and tuning ElastiFlow is out of scope this article, here an example to make it work. Please read official Installation guide for better understanding.

 1#install the sFlow codec for Logstash
 2sudo /usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow
 3sudo /usr/share/logstash/bin/logstash-plugin list --installed
 4sudo /usr/share/logstash/bin/logstash-plugin update logstash-codec-netflow
 7# download ElastiFlow files and copy them in system
 8wget https://github.com/robcowart/elastiflow/archive/v4.0.1.tar.gz
 9tar xf v4.0.1.tar.gz
10sudo chown root:root -R elastiflow-4.0.1/
11sudo rsync -avh elastiflow-4.0.1/logstash.service.d /etc/systemd/system/
12sudo rsync -avh elastiflow-4.0.1/logstash/elastiflow /etc/logstash/
14# increasing the priority of the Logstash service
15sudo sed -i 's/Nice=19/Nice=0/g' /etc/systemd/system/logstash.service
16# specify Logstash configuration path
17sudo sed -i 's/main/elastiflow/g' /etc/logstash/pipelines.yml
18sudo sed -i 's:logstash:logstash/elastiflow:g' /etc/logstash/pipelines.yml
20#Logstash startup script to take the variables from configs
21sudo /usr/share/logstash/bin/system-install
22# apply changes for logstash servece
23sudo systemctl daemon-reload
24# start
25sudo systemctl start logstash.service
26# starting process can be viewed in
29#In kibana go to "Management/Stack Management/Saved Objects" and import

ElastiFlow screenshots

ElastiFlow screenshot