EdgeRouter usage monitor using vnstat


Summary

Data Usage Summary

How much data are you using per month?
Many ISPs have improved Internet performance but have imposed strict data caps on customers. Comcast in California charges most residential customers costly overage fees when they exceed 1TB of data usage within a month. And, while Comcast claims most customers do not reach this limit, it is a lot easier then Comcast would like you to understand. To receive “truly” unlimited data, Comcast charges $50 extra per month, or requires a top tier Internet plan or Business Internet account.

In our view, this is why ISPs such as Comcast or AT&T should never be given rights to an area by a City Government. Google Fiber attempted to install high speed fiber in San Jose at a much lower cost per month and without data caps. Comcast and AT&T were able to put up so many road blocks that Google put the project on hold.

Citizens missed out on free and low cost high speed internet. Now, Comcast and AT&T are free to raise rates. Impose data caps with high overage fees. And, provide less service because competition does not exist in San Jose or many other cities in California.

We can thank the San Jose City Council, the CPUC and FCC for creating this uncompetitive climate in our City, County and State – Google Fiber deployment in San Jose on HOLD.

To avoid additional charges from Comcast, or other ISPs, let’s use vnstat on our EdgeRouter PoE (ERPOE) to monitor and report our data usage. One of the many benefits of the EdgeRouter is its use of Open Source. EdgeOS is actually a striped down Debian Linux distribution. By adding a few libraries, additional function such as vnstat can be added to EdgeOS.

A few of the reports available from vnstat – Monthly, Daily, Hourly

vnstat -m
vnstat -d
vnstat -h

How to add Debian libraries to EdgeOS
To add vnstat to our Edge Router PoE, additional Debian Linux software libraries are needed. The process is simple on Linux. We tell EdgeOS where to find the Open Source libraries and ask Linux to update and install vnstat. EdgeOS 1.8 and 1.9 use Debian Wheezy repositories, while EdgeOS 2.x is expected to use Jessie repositories. To instruct EdgeOS to access the correct Open Source repos, we use Linux set system package commands in the CLI (the following was tested on EdgeOS 1.91):


configure
set system package repository wheezy components 'main contrib non-free'
set system package repository wheezy distribution wheezy
set system package repository wheezy url http://http.us.debian.org/debian
commit; save; exit

Update Debian Library Cache:


sudo apt-get update

Installing vnstat on EdgeOS
After installing the Debian libraries, to install vnstat use the command:

sudo apt-get install -y vnstat vnstati

Additional information on adding Debian libraries to EdgeOS can be found at
(How to add debian repo).

Collecting data usage information with vnstat
If our internet connection (wan) was connected to eth0; the following command would collect internet data usage:

sudo vnstat -i eth0

Check status

/etc/init.d/vnstat status

Start vnstat

/etc/init.d/vnstat start

Stop vnstat

/etc/init.d/vnstat stop

If the router is rebooted, vnstat will need to be restarted or a startup script created https://community.ubnt.com/t5/EdgeMAX-Frequently-Asked/EdgeMAX-How-do-I-run-a-start-up-script-on-boot-up/ta-p/424033

Storing vnstat data
By default, vnstat writes data to /var/lib/vnstat, this directory uses flash storage on the Edge Router. If you do not want additional R/W wear on flash storage, this data can be written to RAM.
To do this, modify DatabaseDir in /etc/vnstat.conf from /var/lib to /var/log
Of course, placing the database on RAM causes it to be deleted on router reboot but does not cause any additional flash wear.

Creating a web interface
EdgeOS contains a webserver and php. By adding a php interface to vnstat, usage data can be access via a browser.

    • Download and copy the php web interface code to “/config/[name]/vnstat” on EdgeOS
    • The /config directory survives EdgeOS firmware upgrades
    • Modify config.php in the /config/[name]/vnstat folder, add the following to config.php based on language and timezone:
$locale = 'en_US.UTF-8';
$language = 'en';
date_default_timezone_set('America/Los_Angeles');
    • Create a symbolic link in the web folder to vnstat directory:
sudo ln -s /config/yourname/vnstat /var/www/htdocs/stat
    • Configure the webserver (lighttpd) to allow data from the vnstat folder to be accessed by modifying /etc/lighttpd.conf and adjusting “url.rewrite-one”:
url.rewrite-once = (
"^/stat/(.*)" => "/stat/$1",
"^(/(lib|media|ws|tests)/.*)" => "$0",
"^/([^\?]+)(\?(.*))?$" => "/index.php/$1?$3"
)

Restarting EdgeOS webserver

sudo kill -SIGTERM $(cat /var/run/lighttpd.pid)
sudo /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf

Access vnstat web interface
http://[router]/stat/index.php

Monthly

Data Usage Monthly

 

Daily

Data Usage Daily

 

Hourly

Data Usage Hourly

 

Saving the vnstat database between EdgeOS software updates
To save the vnstat database between firmware upgrades:
/config/scripts/post-config.d/install_my_packages.sh

packages='locales screen rsync iftop iptraf mtr-tiny bmon procinfo lftp vnstat'
if [ -e $doneit ]; then
exit 0;
fi
#copy old vnstat data
cp -a /root.dev/w.o/var/lib/vnstat /var/lib/

apt-get update
apt-get install -y $packages
if [ $? == 0 ]; then
echo package install successful 
  touch $doneit
  else
  echo package install failed 
  fi

#restart vnstat
service vnstat restart

Reducing vnstat database size
By default, vnstat create databases for each interface (eth0, eth1, etc) in /var/log/vnstat
To make the database as small as possible,  The problem is that if you reboot the router, vnstat starts up and thinks this is the first time it runs so it recreates them all.  I modified the startup script to start vnstatd, wait a second, close vnstatd, delete the interface files I don’t want and then finally restart vnstatd.  /etc/init.d/vnstat:

start)
if [ ! -d "/var/log/vnstat" ]; then
mkdir /var/log/vnstat
fi
# Start the service for the first time which creates interface databases
start-stop-daemon --start --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON --$DAEMON_ARGS
sleep 1
# Stop the service so unnecessary interface database can be removed
start-stop-daemon --stop --quiet --oknodo --retry=TERM /15/KILL/5 --pidfile $PIDFILE --name $NAME
sleep 1
# Remove unnecessary interface databases
rm /var/log/vnstat/loo*
rm /var/log/vnstat/np*
# Restart service
log_daemon_msg "Starting $DESC" "$NAME"
start-stop-daemon --start --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON --$DAEMON_ARGS
log_end_msg $?
;;

Leave a comment