Shadowsocks bandwidth monitoring with netdata

The [cci]community/shadowsocks[/cci] package on Arch itself doesn't offer much support for bandwidth monitoring, so we must build our own. My Shadowsocks setup includes a few ports for different client users. Bandwidth data for each separate port that Shadowsocks uses allows me to monitor usage by each client.

The bandwidth data is obtained using iptables. This is accessed and parsed regularly by a Python script, whose output is read by a netdata module.

Shadowsocks configuration

For a setup where a single port is used by Shadowsocks, [cci]/etc/shadowsocks/config.json[/cci] should be:
[cc lang="javascript"]
{
"server":"0.0.0.0",
"server_port":20000,
"password":"mypassword",
"local_address": "127.0.0.1",
"local_port":1080,
"timeout":300,
"method":"aes-256-cfb",
"fast_open": false,
"workers": 1,
"prefer_ipv6": false
}
[/cc]
Replace [cci]20000[/cci] by the port number you intend for Shadowsocks.

For a setup where multiple ports are used by Shadowsocks, [cci]/etc/shadowsocks/config.json[/cci] should be:
[cc lang="javascript"]
{
"server":"0.0.0.0",
"port_password": {
"20000": "mypassword0",
"20001": "mypassword1",
"20002": "mypassword2"
},
"local_address": "127.0.0.1",
"local_port":1080,
"timeout":300,
"method":"aes-256-cfb",
"fast_open": false,
"workers": 1,
"prefer_ipv6": false
}
[/cc]
Enable and start the Shadowsocks systemd service as root user:
[cc lang="bash"]$ systemctl enable --now shadowsocks-server@config[/cc]

iptables rules

To tell iptables to monitor the number of packets and bytes sent and received by specific ports, add the following rules to iptables as root user:
[cc lang="bash"]
$ iptables -A INPUT -p tcp -m tcp --dport $ iptables -A OUTPUT -p tcp -m tcp --sport [/cc]
Run these commands for each [cci][/cci] used by Shadowsocks.

For these iptables rules to persist across reboots, create [cci]/etc/iptables/iptables.rules[/cci] with the content:
[cc]
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp -m tcp --dport 20000
-A OUTPUT -p tcp -m tcp --sport 20000
-A INPUT -p tcp -m tcp --dport 20001
-A OUTPUT -p tcp -m tcp --sport 20001
-A INPUT -p tcp -m tcp --dport 20002
-A OUTPUT -p tcp -m tcp --sport 20002
COMMIT
[/cc]
Each Shadowsocks port has its INPUT and OUTPUT rules. Enable the iptables systemd service:
[cc lang="bash"]$ systemctl enable iptables[/cc]
This service is started on boot and adds the rules given in [cci]/etc/iptables/iptables.rules[/cci].

Python script for periodic monitoring

Create [cci]/usr/bin/ssmonitor.py[/cci], which periodically checks the packets and bytes counters in iptables and writes the current bandwidth to a temporary file:
[cc lang="python"]
#!/usr/bin/python

import signal
import time
import subprocess
import csv

# Exit gracefully on SIGINT and SIGTERM signals
class GracefulKiller:
kill_now = False
def __init__(self):
signal.signal(signal.SIGINT, self.kill_gracefully)
signal.signal(signal.SIGTERM, self.kill_gracefully)

def kill_gracefully(self,signum, frame):
self.kill_now = True

# Parse iptables stdout into input and output dictionaries {'port': (pkts, bytes)}
def parse_iptables():
timestamp = time.time()
raw_in = subprocess.run(['iptables', '-nvxw', '-L', 'INPUT'], stdout=subprocess.PIPE).stdout.decode('utf-8')
raw_out = subprocess.run(['iptables', '-nvxw', '-L', 'OUTPUT'], stdout=subprocess.PIPE).stdout.decode('utf-8')
lines_in = raw_in.split('\n')
lines_out = raw_out.split('\n')
table_in = {}
table_out = {}
for l in lines_in:
cells = l.split()
if len(cells) == 10:
port = cells[9][4:]
pkts = int(cells[0])
bytes = int(cells[1])
table_in[port] = (pkts, bytes)
for l in lines_out:
cells = l.split()
if len(cells) == 10:
port = cells[9][4:]
pkts = int(cells[0])
bytes = int(cells[1])
table_out[port] = (pkts, bytes)
if len(table_in) != len(table_out):
raise ValueError('iptables INPUT and OUTPUT incompatible.')
return timestamp, table_in, table_out

# Returns input/output rates as dictionary {'port': (pkts_in, pkts_out, bytes_in, bytes_out)}
def calculate_rates(timestamp, table_in, table_out, timestamp_old, table_in_old, table_out_old):
rates = {}
for port in table_in:
dpkts_in = table_in[port][0] - table_in_old[port][0]
dpkts_out = table_out[port][0] - table_out_old[port][0]
dbytes_in = table_in[port][1] - table_in_old[port][1]
dbytes_out = table_out[port][1] - table_out_old[port][1]
dt = timestamp - timestamp_old
rates[port] = (dpkts_in / dt, dpkts_out / dt, dbytes_in / dt, dbytes_out / dt)
return rates

# Converts rates dictionary to csv string
def to_text(rates):
to_write = ""
for port in rates:
rates_str = [port] + ["{:.3f}".format(r) for r in rates[port]]
to_write += ",".join(rates_str) + "\n"
return to_write

# Handle graceful killing
if __name__ == '__main__':
killer = GracefulKiller()
timestamp_old, table_in_old, table_out_old = parse_iptables()
while True:
time.sleep(2)

timestamp, table_in, table_out = parse_iptables()
rates = calculate_rates(timestamp, table_in, table_out, timestamp_old, table_in_old, table_out_old)
timestamp_old = timestamp
table_in_old = table_in
table_out_old = table_out
to_write = to_text(rates)
f = open('/tmp/ssmonitor.tmp', 'w')
f.write(to_write)
f.close()

if killer.kill_now:
break

exit(0)
[/cc]
Make it executable:
[cc lang="bash"]$ chmod +x /usr/bin/ssmonitor.py[/cc]
You can test if the script works by running as root user:
[cc lang="bash"]$ /usr/bin/ssmonitor.py[/cc]
and checking
[cc lang="bash"]$ cat /tmp/ssmonitor.tmp[/cc]
to see if the file has been populated by current bandwidth data.

To start the script on boot, create a systemd service [cci]/usr/lib/systemd/system/ssmonitor.service[/cci]:
[cc]
[Unit]
Description=Monitors Shadowsocks bandwidth through iptables

[Service]
Type=simple
ExecStart=/usr/bin/ssmonitor.py

[Install]
WantedBy=multi-user.target
[/cc]
Enable and start it:
[cc lang="bash"]$ systemctl enable --now ssmonitor[/cc]

Netdata module

Now, we write a new netdata module to display the current Shadowsocks bandwidth. Create [cci]/usr/lib/netdata/python.d/shadowsocks.chart.py[/cci] with the content:
[cc lang="python"]
from bases.FrameworkServices.SimpleService import SimpleService
import csv

def debug(text):
f = open('/tmp/pydebug.tmp', 'w')
f.write(text)
f.close()

ORDER = ['20000_bytes', '20001_bytes', '20002_bytes', '20000_pkts', '20001_pkts', '20002_pkts']
CHARTS = {
'20000_bytes': {
'options': ['20000 bytes', '20000 bytes', 'kB/s', '20000 bytes', 'shadowsocks.20000_bytes', 'area'],
'lines': [
['20000_bytes_in', None, 'absolute', 1, 1000],
['20000_bytes_out', None, 'absolute', 1, 1000],
]},
'20001_bytes': {
'options': ['20001 bytes', '20001 bytes', 'kB/s', '20001 bytes', 'shadowsocks.20001_bytes', 'area'],
'lines': [
['20001_bytes_in', None, 'absolute', 1, 1000],
['20001_bytes_out', None, 'absolute', 1, 1000],
]},
'20002_bytes': {
'options': ['20002 bytes', '20002 bytes', 'kB/s', '20002 bytes', 'shadowsocks.20002_bytes', 'area'],
'lines': [
['20002_bytes_in', None, 'absolute', 1, 1000],
['20002_bytes_out', None, 'absolute', 1, 1000],
]},
'20000_pkts': {
'options': ['20000 packets', '20000 packets', 'packets/s', '20000 packets', 'shadowsocks.20000_packets', 'area'],
'lines': [
['20000_pkts_in', None, 'absolute', 1, 1],
['20000_pkts_out', None, 'absolute', 1, 1],
]},
'20001_pkts': {
'options': ['20001 packets', '20001 packets', 'packets/s', '20001 packets', 'shadowsocks.20001_packets', 'area'],
'lines': [
['20001_pkts_in', None, 'absolute', 1, 1],
['20001_pkts_out', None, 'absolute', 1, 1],
]},
'20002_pkts': {
'options': ['20002 packets', '20002 packets', 'packets/s', '20002 packets', 'shadowsocks.20002_packets', 'area'],
'lines': [
['20002_pkts_in', None, 'absolute', 1, 1],
['20002_pkts_out', None, 'absolute', 1, 1],
]},
}

class Service(SimpleService):
def __init__(self, configuration=None, name=None):
SimpleService.__init__(self, configuration=configuration, name=name)
self.order = ORDER
self.definitions = CHARTS

def check(self):
data = self._get_data()
if not data:
return False
return True

def _get_raw_data(self):
with open('/tmp/ssmonitor.tmp', newline='') as file:
return list(csv.reader(file))

def _get_data(self):
data = {}
try:
table = self._get_raw_data()
except:
return None
for row in table:
port = row[0]
data[port + '_pkts_in'] = float(row[1])
data[port + '_pkts_out'] = -float(row[2])
data[port + '_bytes_in'] = float(row[3])
data[port + '_bytes_out'] = -float(row[4])
return data
[/cc]
Adjust [cci]ORDER[/cci] and [cci]CHARTS[/cci] to match your Shadowsocks port setup.

Enable the module by adding the line
[cc]shadowsocks: yes[/cc]
to [cci]/etc/netdata/python.d.conf[/cci].

Finally, restart netdata to see real-time charts of Shadowsocks bandwidth:
[cc lang="bash"]$ systemctl restart netdata[/cc]

Leave a Reply

Your email address will not be published. Required fields are marked *