Mesh Network Introduction

Welcome to Network mesh emulator blog

Introduction

The purpose of this network emulator is to test evaluate and debug mesh network protocols how the B.A.T.M.A.N.  ADV protocol.

This network emulator is a front end for virtualbox openwrt machines connected trough a vde-switch and wirefilter emulating a wireless link. The front end is writed in python with pygtk gui. The python program monitor the openwrt machines with snmp protocol, showing in the main screen transmited packets for each interface, and originators interfaces for batmam-adv protocol. The vde-switches have tap interfaces connected with the host machine, this permit monitor the packet traffic with the Wire-shark program, the eth0 interface of the openwrt machines are connected to the host via the  vboxnet interface  of the host, in this way is possible to access to the openwrt console for management purpose.

The mesh is integrate with nodes and wireless connection. Each node diagram is show in the folowing figure.

Architecture

The architecture is basically compound of two elements Nodes and Links (Wirefilter) as shown in  the figure 1.

Figure 1

The nodos are shown in figure 2 this are more complex and have the following elements:

• OpenWrt, kamikaze trunk version for x86 with minimal modifications (see below)
• VirtualBox (unmodified) the version must suport vde-switch.
• Vde_switch must run two instance for nodo to support 2.4 GHz and 5.0 GHz networks. The vde-switch have a patch colorfull see below.

Figure 2

OpenWrt

A standard OpenWrt can be downloaded and configured for X86 . Once that virtual machine is runining some packages must be download (ip, snmpd, kmod-batman-adv, batctl).
For download the openwrt and configure in virtualbox the following script can be used:
#!/bin/sh
wget -c https://downloads.openwrt.org/barrier_breaker/14.07/x86/generic/openwrt-x86-generic-combined-ext4.img.gz
gunzip openwrt-x86-generic-combined-ext4.img.gz
VBoxManage convertdd openwrt-x86-generic-combined-ext4.img prueba.vdi
VM=prueba
VBoxManage createvm --name $VM --ostype "Linux" --register
VBoxManage storagectl $VM --name "SATA Controller" --add sata --controller IntelAHCI --portcount 1
VBoxManage storageattach $VM --storagectl "SATA Controller" --port 0 \
 --device 0 --type hdd --medium $VM.vdi
VBoxManage modifyvm $VM --memory 32 --vram 1
VBoxManage modifyvm $VM --nic1 bridged --bridgeadapter1 eth0 
VBoxManage startvm $VM

When the virtual machine start there are to setup the br-lan interface with the apropiate ip.
For example:
vm$ ifconfig br-lan 192.168.1.50 255.255.255.0
vm$ passwd root

For an automatic configuration of the network interfaces devices setup on boot must be use the following script and save it as  './files/etc/rc.local'  in your local OpenWrt build directory:

#!/bin/sh
# kill default openwrt network config
RED=$(ifconfig eth1 | sed '1,1!d' | sed 's/.*HWaddr //'| sed 's/.\{11\}://'| sed 's/.\{5\}$//')
NUM=$(ifconfig eth1 | sed '1,1!d' | sed 's/.*HWaddr //'| sed 's/.*://'| sed 's/[\n\ ].*//')
ifconfig br-lan down
brctl delbr br-lan
ip link delete eth0
ip addr add 192.168.100.$NUM/24 dev eth0
ip link set dev eth1 mtu 1500 up
ip link set dev eth2 mtu 1500 up
batctl -m bat0 interface add eth1
batctl -m bat0 interface add eth2
ip addr add 192.168.$RED.$NUM/24 dev bat0
ip link set dev bat0 address 90:$NUM:$NUM:$NUM:$NUM:$NUM
ip link set dev bat0 up
batctl -m bat0 originators

/etc/init.d/snmpd enable      
/etc/init.d/snmpd restart

The new version of OpenWrt not support mtu > 1500 the Batman Adv. protocol require a mtu 1532 

To setup the VM OpenWrt  we must copy the "snmp.tar" to the root directory of VM

host$ scp ./snmp.tar root@192.168.1.50:/root/ #in host
vm$cd root
vm$ tar -xf snmp.tar  
vm$ ./config.sh 192.168.1.1

The script ./config.sh install the snmp and snmpd script.
In virtualBox is difficult to pas the IP address for the interfaces this is accomplished setting the mac address in VirtualBox and int the "rc.local" script read the mac address and  set the ip in the interfaces.
You can download the fully configurate machine OpenWrt version 12.09 or OpenWrt v 14.07

In order to have many virtual machines running we must minimize RAM memory in each VM the OpenWrt version 12.09 need at least 32 MB  and the OpenWrt v 14.07 need at least 24 MB of RAM.

SNMPD

After the ``SNMP'' is installed in the OpenWrt machine, we proceed to
setup the MIB for support de batctl command of BATMAN Adv..
To add custom records to the BATMAN Adv. MIB a series of shell scripts are
run and return their output to stdout, which is captured by SNMP

When the snmpd is installed in the openwrt machine is nedded to setup the MIB for batman-adv.

To add custom records to support de batctl command of  the batman-adv MIB there are many ways  to extend the snmp fucionality:

• exec [MIBOID] NAME PROG ARGS
• sh [MIBOID] NAME PROG ARGS
• execfix NAME PROG ARGS
• extend [MIBOID] NAME PROG ARGS
• extendfix NAME PROG ARGS
• pass [-p priority] MIBOID PROG
• pass_persist [-p priority] MIBOID PROG

We choose "pass" method because this method permit set and get commands and is supported by OpenWrt
a series of shell script was conducted  running and returning to stdout what it takes SNMP.
PROG will pass control of the subtree rooted at MIBOID to the specified PROG command. GET and GETNEXT requests for OIDs within this tree will trigger this command, called as:

PROG -g OID

PROG -n OID

These script has four input parameter.

1. -g or -s
2. OID
3. type 
4. var

And return tree parameters

1. OID
2. type
3. Var

To request originators table  the next script was made:

Script name _batctl_o.sh_ (for originator list)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
 BAT=$(batctl o | sed -n 's/^\(..:..:..:..:..:..\).*/\1/p')
    echo $2
    echo "string"
    echo $BAT
fi

This script only need de command get , -s switch do nothing and return.

Script name batctl_n.sh  (for next hop list)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
  BAT=$(batctl o | sed -n 's/^\(..:..:..:..:..:..\).* \(..:..:..:..:..:..\).*/\1 \2/p')
      echo $2
      echo "string"
      echo $BAT
fi

This script only need de command get , -s switch do nothing and return too.

Script name batctl_tg.sh (for the global translation table)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
  BAT=$(batctl tg | sed -n 's/.*\(..:..:..:..:..:..\).* \(..:..:..:..:..:..\).*/\1 \2/p')
      echo $2
      echo "string"
      echo $BAT
fi

Script name batctl_tr.sh (for the trace route to level 2 )

This script need to pass the destination MAC address for the trace command this is made with a set command, the result of the trace command is saved in a file called "trace" when the get command is issue the command read this file and return de route to destination MAC. 

#!/bin/sh
if test "$1" = '-s' ;
        then
                  
  BAT=$(batctl tr $4 | sed -n 's/.*\(..:..:..:..:..:..\)[ ,\*]\+\([0     -9][0-9]*\.[0-9][0-9]*\).*/\2 \1/p')
                echo $BAT > trace
                exit
elif test "$1" = '-g' ;
        then
               BAT=$(cat trace)
               echo $2
               echo "string"
               echo $BAT
fi




Then add entries in the configuration file SNMPD /etc/snmp/snmpd.conf using the command uci in a terminal.

uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.1
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_o.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.2
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_n.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.3
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_tg.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.4
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_tr.sh
uci commit snmpd

/etc/init.d/snmpd restart

To the files /etc/snmp/snmpd.conf  append the following lines.

.......

pass  .1.3.6.1.4.1.32.1.1 /root/snmp/batctl_o.sh
pass  .1.3.6.1.4.1.32.1.2 /root/snmp/batctl_n.sh
pass  .1.3.6.1.4.1.32.1.3 /root/snmp/batctl_tg.sh
pass  .1.3.6.1.4.1.32.1.4 /root/snmp/batctl_tr.sh

From the host the snmp can be tested with the followings commands.

$ snmpget -v 1 -c public 192.168.100.11 iso.3.6.1.4.1.32.1.1.101.1
iso.3.6.1.4.1.32.1.1.101.1 = STRING: "80:03:00:00:07:41 80:03:00:00:07:31 80:02:00:00:07:31 80:02:00:00:07:21 80:03:00:00:07:21"

batctl o nexhop

$ snmpget -v 1 -c public 192.168.100.11 iso.3.6.1.4.1.32.1.2.101.1
iso.3.6.1.4.1.32.1.2.101.1 = STRING: "80:02:00:00:07:21 80:02:00:00:07:21 80:03:00:00:07:21 80:02:00:00:07:21 80:03:00:00:07:21"

In the Git Hub (https://github.com/dbritos/Network-mesh-emulator/blob/master/openwrtpass.ova) repository there are a fully configurated virtual machine.
Download openwrtpass.ova in VirtualBox go to File menu -> Import Appliance

Ip assignations in openwrt.

For assign the ip address to the VM first is assigned the MAC address to the VM. Each VM have three interfaces nic1, nic2 and nic3 this interfaces in the openwrt appear as eth0, eth1 and eth2.

• nic1 (eth0)  mac  80:01:00:00:07 + nodenumber(nn).
• nic2 (eth1)  mac  80:02:00:00:07 + nodenumber(nn). the number 2 for 2.4GHz)
• nic3 (eth2)  mac  80:05:00:00:07 + nodenumber(nn). the number 5 for 5.0GHz)

For configure the VM with this mac address the following commands are used:

VBoxManage modifyvm openwrtnn --nic1 generic --nicgenericdrv1 VDE --nicproperty1 network=/tmp/c24GHznn[2] --macaddress1 8001000007nn

VBoxManage modifyvm openwrtnn --nic2 generic --nicgenericdrv2 VDE --nicproperty2 network=/tmp/c24GHznn[2] --macaddress2 8001000007nn

VBoxManage modifyvm openwrtnn --nic3 generic --nicgenericdrv3 VDE --nicproperty3 network=/tmp/c24GHznn[2] --macaddress3 8001000007nn

Where: - nn Node number

 The script in the openwrt in /etc/rc.local read the mac address of the interface eth1 and configure the ip of the interfaces:

• eth0 192.168.100.nn 
• bat0 192.168.7.nn 

and the mac address of bat0 = 90:nn:nn:nn:nn:nn

Whit this convention of IP and MAC address is easy to follow the packets trough the nodes.
With the eth0 interface is possible to access to the nodes via ssh to the ip address 192.168.100.nn. The host have the vboxnet0 interface with the ip address 192.168.100.1. Each vde-switch have a tap interface through the wireshark can sniff the packets that transverse the vde-switch.

VirtualBox 

 The VirtualBox version must by 4.3 or higher. To verify VDE-Switch support in the network windows select in Attached to: "Generic Driver" in the Name: box verify that exist VDE.

The configuration of the virtual machines openwrt is made trough console with the VBoxManage command.

vde_switch 

The main advantage of vde_switch over uml_switch is that any clients can be attached to this virtual switch: VirtualBox, UML, tap interfaces, virtual interconnections, and not just UML instances. 

 If the vde_switches were just connected with wirefilter "patch cables" without modification, we would end up creating a broadcast domain and switch loops which we don't want: The goal is to allow the packets to travel only from one host to it's neighbor, not farther. To accomplish this, the vde_switch needs to be modified to have "coloured" ports. 

The idea is: - each port has a "colour" (an integer number) - packets are only passed from ports to others with DIFFERENT colours. 

Packets are dropped on outgoing ports if it has the SAME colour (same number) as the incoming port. 

 In this concept, the host port can have colour 1 the TAP port colour 2, while the interconnection ports have colour 0. In this way, packets can only travel from the host to (all of) the interconnection ports, or from one interconnection port to the host port. However packets can not travel between the interconnection ports, thus only allowing "one hop" connections and avoiding switch loops and shared broadcast domains. The concept is illustrated in figure 2. 

The patch against vde2-2.3.2 (current latest stable version) to add this colour patch can be find here: [vde2-2.3.2_colour.patch](http://www.open-mesh.org/attachments/download/152/vde2-2.3.2_colour.patch).

The vde-switch parched can be download from [here](https://github.com/dbritos/Network-mesh-emulator/blob/master/vde2-2.3.2-patch.tar). Attention this version can be old! 

wirefilter 

The wirefilter program is a tool where its posiblen simulate various link defects and limits as example:

• packet loss
• burst loss
• delay
• duplicates
• bandwidth
• Interface speed
• Chanel capacity
• noise (damage to packets)
•  mtu 

However as the links are only set up bidirectional, interferences can unfortunately not be simulated with this system. 

 For advanced testing it might be necessary to apply the aforementioned link defects to some packets only whereas other packets are able to traverse the emulated environment unharmed. 

Once you applied the 'ethertype' patch you can specify an ethertype which wirefilter will simply forward. To apply a packet loss of 50% to all packets except batman-adv packets, run:

wirefilter --ether 0x4305 -l 50

This patch also allows to filter batman-adv packet types. To apply a packet loss of 50% to all packets except batman-adv ICMP packets, run:

wirefilter --ether 0x4305:02 -l 50

You can specify up to 10 packet types (separated by colon). The patch against vde2-2.3.1 (current latest stable version) can be found here:

vde2-2.3.1-wirefilter-ethertype.patch

Bibliography

http://linux.die.net/man/5/snmpd.conf

http://www.bufferbloat.net/projects/cerowrt/wiki/Automated_Configuration_of_CeroWrt?version=1

http://www.net-snmp.org/docs/man/snmpd.examples.html