Mesh Networks Emulator Manual

Introducction

This is an mesh network emulator, with a protocol B.A.T.M.A.N Adv to test, evaluate and debug mesh network .

Key features:

• Its possible to build topologies to 81 nodes.

• Each node can be a different virtual machine.

• Compatibility with Wireshark.

• Its possible change the links properties while the emulation is running.

• Easy access to each virtual machine.

Prerequisites

• Linux (developed on Ubuntu 14.04)

• Python 2.7

• VirtualBox.

• OpenWrt.

• vde-switch.

• vde_switch colour patch.

• python library pygtk, gobject, cairo,math, pickle os, netsnmp, time and vboxapi.

Installing the program

The program is written in Python 2.7 once downloaded in the work directory the requirements to run are:

1. Install all of the necessary Python modules listed above. Many of them are available via pip and/or apt-get.

1. Install Virtualbox.

1. Add a Host-only Network (vboxnet0) in Virtualbox -> Preferences -> Network

1. Configure OpenWrt virtual machine.

1. Install vde_switch with the patch.

1. Run the program $>python simmeshv13.py

O you can run the install.sh

If you do not want to configure your openwrt virtual machine it can be download from the Git Hub repository this is a fully configurated virtual machine.

Download openwrtv12.ova in VirtualBox program go to File menu -> Import Appliance.

Runing the program

When the program is running present the first screen, asking for the password it is needed to setup the interfaces in the host machine.

If any work was saved with the option Menu File Save the topology saved is load automatically it is saved in “data.ms”.

In the superior bar there are a menu, File, Edit Run/Stop Tools in the following figure is show the menu File.

• New clear the work-space to begin a new topology. 
• Open present a File dialog to choose a topology file. 
• Save Store the current topology in the “data.ms” file. (First must stop the emulation). 
• Save as permit choose the file to save the topology.(First must stop the emulation). 
• Quit close all and exit the program. 

The Edit menu shown in the following figure:

have the following entry: 

• Delete Mesh erase current mesh topology 
• Delete Nodo erase the Nodes without connections. 
• Delete Link erase links that not connect any nodes. 
• Edit Wire Show the window “EDIT WIRE” this permit select the properties of the connection. Is not necessary to press the OK button to make the selection is possible to let open the window and change the selection before to put the wire in the topology screen. 
• Select VM Show the window “SELECT VIRTUAL MACHINE” this show all the virtual machines of VirtualBox and one can select the appropriate VM. Is not necessary to press the OK button to make the selection of the VM, is possible to let open the window and change the selection before to put the Node in the topology screen.

The following figure show the windows “SELECT VIRTUAL MACHINE” with openWrt selected.

The Run/Stop menu have two option : 

• Run this launch the emulation creating de VM, the vde-switch, the wirefilter and creates the interfaces TAP and vboxnet in the host, the emulator pass from edit mode to emulation mode. 
• Stop the emulation stop the VM, the vde-switch, the wirefilter and remove the host interfaces, the emulator pas from emulation mode to edit mode. 

The Tools menu and about menu is not yet implemented.

Edit mode

Edit mode is used to build the topology of the mesh network. With one click of the left button the nodes are create, the virtual machine of the node is setup selecting with the VM Select menu, the VM name prior to create the node. To delete nodes press the right button over the node and the node is deleted.

To create a link is necessary at least two nodes, maintaining pressed the left button and sliding the mouse between the two nodes, press in the origin node and release in the end node. To delete a link the same action to create a node is performed with the right button. If in wire properties 2.4GHz is selected, only this link is delete, if is necessary to delete the 5.0GHz link, this must be selected in wire properties.

When one node or one link is create this become the current node and the information on the screen is for the current node. To select other current node or current link the same action to create or delete is performed with the middle button of the mouse.

The figure show the layout of the screen with information of virtual machine name of the current node, the highlight node is the current node, the highlighted link is the current link in the second and third line is shown the wire properties with pink color for 2.4 GHz and light blue color for 5.0 GHz.

Run Mode

Run mode is when through the Run/Stop menu Run is selected the program launch the virtual machines the vde-switch the TAP interfaces of the host and the wirefilters. once the virtual machines haved boot the screen present the followin information:

In the first line the Rx/Tx packets of the current Node, over the circle of each node is presented the Rx/Tx packets of the bat0 interface, and in the right of the screen the originators and next hop list of the current Node. 

Change link properties.

Is possible to change the link properties in Run mode for do this follow the next step:

• Delete the link
• Change wire properties.
• Create a new link.

Batman-adv search a new path and if this link is appropriate, it use this link.

Check Link Properties

With de middle mouse button is posible to check de link properties. Pressing the middle button in the beginning node and releasing at the end node.

Wireshark integration.

The integration with the wireshark network analyzer is very simple run Wireshark and the program present the list of interfaces as shown in the figure:

Choose the tap interface for the node and the band (2.4GHz or 5.0 GHz) that you want to inspect and the wireshark sniff in these interface.
The notation of the tap interface “tapc24GHz11” is the following:

• tap designate a tap interface
• c24GHz designate the band of the interface 2.4GHz.
• 11 Designate the Node number 11 in this case.

Perfomance

To test the perfomance it's build a mesh network of 45 nodes as show the figure,  one can view five networks of nine nodes each one with links in 2.4 GHz band (line pink) and connected the five network with links of 5.0 GHz Band. (line light blue)

In the right of the screen there are the originators  and next hop  list for the node number 66.
In the next figure is shown the node 59 making a ping to node number 71, the times are good.

In the last figure is shown the cpu and memory usage for 36 nodes running together.

Mesh Network Introduction

Welcome to Network mesh emulator blog

Introduction

The purpose of this network emulator is to test evaluate and debug mesh network protocols how the B.A.T.M.A.N.  ADV protocol.

This network emulator is a front end for virtualbox openwrt machines connected trough a vde-switch and wirefilter emulating a wireless link. The front end is writed in python with pygtk gui. The python program monitor the openwrt machines with snmp protocol, showing in the main screen transmited packets for each interface, and originators interfaces for batmam-adv protocol. The vde-switches have tap interfaces connected with the host machine, this permit monitor the packet traffic with the Wire-shark program, the eth0 interface of the openwrt machines are connected to the host via the  vboxnet interface  of the host, in this way is possible to access to the openwrt console for management purpose.

The mesh is integrate with nodes and wireless connection. Each node diagram is show in the folowing figure.

Architecture

The architecture is basically compound of two elements Nodes and Links (Wirefilter) as shown in  the figure 1.

Figure 1

The nodos are shown in figure 2 this are more complex and have the following elements:

• OpenWrt, kamikaze trunk version for x86 with minimal modifications (see below)
• VirtualBox (unmodified) the version must suport vde-switch.
• Vde_switch must run two instance for nodo to support 2.4 GHz and 5.0 GHz networks. The vde-switch have a patch colorfull see below.

Figure 2

OpenWrt

A standard OpenWrt can be downloaded and configured for X86 . Once that virtual machine is runining some packages must be download (ip, snmpd, kmod-batman-adv, batctl).
For download the openwrt and configure in virtualbox the following script can be used:
#!/bin/sh
wget -c https://downloads.openwrt.org/barrier_breaker/14.07/x86/generic/openwrt-x86-generic-combined-ext4.img.gz
gunzip openwrt-x86-generic-combined-ext4.img.gz
VBoxManage convertdd openwrt-x86-generic-combined-ext4.img prueba.vdi
VM=prueba
VBoxManage createvm --name $VM --ostype "Linux" --register
VBoxManage storagectl $VM --name "SATA Controller" --add sata --controller IntelAHCI --portcount 1
VBoxManage storageattach $VM --storagectl "SATA Controller" --port 0 \
 --device 0 --type hdd --medium $VM.vdi
VBoxManage modifyvm $VM --memory 32 --vram 1
VBoxManage modifyvm $VM --nic1 bridged --bridgeadapter1 eth0 
VBoxManage startvm $VM

When the virtual machine start there are to setup the br-lan interface with the apropiate ip.
For example:
vm$ ifconfig br-lan 192.168.1.50 255.255.255.0
vm$ passwd root

For an automatic configuration of the network interfaces devices setup on boot must be use the following script and save it as  './files/etc/rc.local'  in your local OpenWrt build directory:

#!/bin/sh
# kill default openwrt network config
RED=$(ifconfig eth1 | sed '1,1!d' | sed 's/.*HWaddr //'| sed 's/.\{11\}://'| sed 's/.\{5\}$//')
NUM=$(ifconfig eth1 | sed '1,1!d' | sed 's/.*HWaddr //'| sed 's/.*://'| sed 's/[\n\ ].*//')
ifconfig br-lan down
brctl delbr br-lan
ip link delete eth0
ip addr add 192.168.100.$NUM/24 dev eth0
ip link set dev eth1 mtu 1500 up
ip link set dev eth2 mtu 1500 up
batctl -m bat0 interface add eth1
batctl -m bat0 interface add eth2
ip addr add 192.168.$RED.$NUM/24 dev bat0
ip link set dev bat0 address 90:$NUM:$NUM:$NUM:$NUM:$NUM
ip link set dev bat0 up
batctl -m bat0 originators

/etc/init.d/snmpd enable      
/etc/init.d/snmpd restart

The new version of OpenWrt not support mtu > 1500 the Batman Adv. protocol require a mtu 1532 

To setup the VM OpenWrt  we must copy the "snmp.tar" to the root directory of VM

host$ scp ./snmp.tar root@192.168.1.50:/root/ #in host
vm$cd root
vm$ tar -xf snmp.tar  
vm$ ./config.sh 192.168.1.1

The script ./config.sh install the snmp and snmpd script.
In virtualBox is difficult to pas the IP address for the interfaces this is accomplished setting the mac address in VirtualBox and int the "rc.local" script read the mac address and  set the ip in the interfaces.
You can download the fully configurate machine OpenWrt version 12.09 or OpenWrt v 14.07

In order to have many virtual machines running we must minimize RAM memory in each VM the OpenWrt version 12.09 need at least 32 MB  and the OpenWrt v 14.07 need at least 24 MB of RAM.

SNMPD

After the ``SNMP'' is installed in the OpenWrt machine, we proceed to
setup the MIB for support de batctl command of BATMAN Adv..
To add custom records to the BATMAN Adv. MIB a series of shell scripts are
run and return their output to stdout, which is captured by SNMP

When the snmpd is installed in the openwrt machine is nedded to setup the MIB for batman-adv.

To add custom records to support de batctl command of  the batman-adv MIB there are many ways  to extend the snmp fucionality:

• exec [MIBOID] NAME PROG ARGS
• sh [MIBOID] NAME PROG ARGS
• execfix NAME PROG ARGS
• extend [MIBOID] NAME PROG ARGS
• extendfix NAME PROG ARGS
• pass [-p priority] MIBOID PROG
• pass_persist [-p priority] MIBOID PROG

We choose "pass" method because this method permit set and get commands and is supported by OpenWrt
a series of shell script was conducted  running and returning to stdout what it takes SNMP.
PROG will pass control of the subtree rooted at MIBOID to the specified PROG command. GET and GETNEXT requests for OIDs within this tree will trigger this command, called as:

PROG -g OID

PROG -n OID

These script has four input parameter.

1. -g or -s
2. OID
3. type 
4. var

And return tree parameters

1. OID
2. type
3. Var

To request originators table  the next script was made:

Script name _batctl_o.sh_ (for originator list)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
 BAT=$(batctl o | sed -n 's/^\(..:..:..:..:..:..\).*/\1/p')
    echo $2
    echo "string"
    echo $BAT
fi

This script only need de command get , -s switch do nothing and return.

Script name batctl_n.sh  (for next hop list)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
  BAT=$(batctl o | sed -n 's/^\(..:..:..:..:..:..\).* \(..:..:..:..:..:..\).*/\1 \2/p')
      echo $2
      echo "string"
      echo $BAT
fi

This script only need de command get , -s switch do nothing and return too.

Script name batctl_tg.sh (for the global translation table)

#!/bin/sh
if test "$1" = '-s' ;
then
exit
elif test "$1" = '-g' ;
then
  BAT=$(batctl tg | sed -n 's/.*\(..:..:..:..:..:..\).* \(..:..:..:..:..:..\).*/\1 \2/p')
      echo $2
      echo "string"
      echo $BAT
fi

Script name batctl_tr.sh (for the trace route to level 2 )

This script need to pass the destination MAC address for the trace command this is made with a set command, the result of the trace command is saved in a file called "trace" when the get command is issue the command read this file and return de route to destination MAC. 

#!/bin/sh
if test "$1" = '-s' ;
        then
                  
  BAT=$(batctl tr $4 | sed -n 's/.*\(..:..:..:..:..:..\)[ ,\*]\+\([0     -9][0-9]*\.[0-9][0-9]*\).*/\2 \1/p')
                echo $BAT > trace
                exit
elif test "$1" = '-g' ;
        then
               BAT=$(cat trace)
               echo $2
               echo "string"
               echo $BAT
fi




Then add entries in the configuration file SNMPD /etc/snmp/snmpd.conf using the command uci in a terminal.

uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.1
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_o.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.2
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_n.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.3
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_tg.sh
uci commit snmpd
uci add snmpd pass
uci set snmpd.@pass[-1].miboid=.1.3.6.1.4.1.32.1.4
uci set snmpd.@pass[-1].prog=/root/snmp/batctl_tr.sh
uci commit snmpd

/etc/init.d/snmpd restart

To the files /etc/snmp/snmpd.conf  append the following lines.

.......

pass  .1.3.6.1.4.1.32.1.1 /root/snmp/batctl_o.sh
pass  .1.3.6.1.4.1.32.1.2 /root/snmp/batctl_n.sh
pass  .1.3.6.1.4.1.32.1.3 /root/snmp/batctl_tg.sh
pass  .1.3.6.1.4.1.32.1.4 /root/snmp/batctl_tr.sh

From the host the snmp can be tested with the followings commands.

$ snmpget -v 1 -c public 192.168.100.11 iso.3.6.1.4.1.32.1.1.101.1
iso.3.6.1.4.1.32.1.1.101.1 = STRING: "80:03:00:00:07:41 80:03:00:00:07:31 80:02:00:00:07:31 80:02:00:00:07:21 80:03:00:00:07:21"

batctl o nexhop

$ snmpget -v 1 -c public 192.168.100.11 iso.3.6.1.4.1.32.1.2.101.1
iso.3.6.1.4.1.32.1.2.101.1 = STRING: "80:02:00:00:07:21 80:02:00:00:07:21 80:03:00:00:07:21 80:02:00:00:07:21 80:03:00:00:07:21"

In the Git Hub (https://github.com/dbritos/Network-mesh-emulator/blob/master/openwrtpass.ova) repository there are a fully configurated virtual machine.
Download openwrtpass.ova in VirtualBox go to File menu -> Import Appliance

Ip assignations in openwrt.

For assign the ip address to the VM first is assigned the MAC address to the VM. Each VM have three interfaces nic1, nic2 and nic3 this interfaces in the openwrt appear as eth0, eth1 and eth2.

• nic1 (eth0)  mac  80:01:00:00:07 + nodenumber(nn).
• nic2 (eth1)  mac  80:02:00:00:07 + nodenumber(nn). the number 2 for 2.4GHz)
• nic3 (eth2)  mac  80:05:00:00:07 + nodenumber(nn). the number 5 for 5.0GHz)

For configure the VM with this mac address the following commands are used:

VBoxManage modifyvm openwrtnn --nic1 generic --nicgenericdrv1 VDE --nicproperty1 network=/tmp/c24GHznn[2] --macaddress1 8001000007nn

VBoxManage modifyvm openwrtnn --nic2 generic --nicgenericdrv2 VDE --nicproperty2 network=/tmp/c24GHznn[2] --macaddress2 8001000007nn

VBoxManage modifyvm openwrtnn --nic3 generic --nicgenericdrv3 VDE --nicproperty3 network=/tmp/c24GHznn[2] --macaddress3 8001000007nn

Where: - nn Node number

 The script in the openwrt in /etc/rc.local read the mac address of the interface eth1 and configure the ip of the interfaces:

• eth0 192.168.100.nn 
• bat0 192.168.7.nn 

and the mac address of bat0 = 90:nn:nn:nn:nn:nn

Whit this convention of IP and MAC address is easy to follow the packets trough the nodes.
With the eth0 interface is possible to access to the nodes via ssh to the ip address 192.168.100.nn. The host have the vboxnet0 interface with the ip address 192.168.100.1. Each vde-switch have a tap interface through the wireshark can sniff the packets that transverse the vde-switch.

VirtualBox 

 The VirtualBox version must by 4.3 or higher. To verify VDE-Switch support in the network windows select in Attached to: "Generic Driver" in the Name: box verify that exist VDE.

The configuration of the virtual machines openwrt is made trough console with the VBoxManage command.

vde_switch 

The main advantage of vde_switch over uml_switch is that any clients can be attached to this virtual switch: VirtualBox, UML, tap interfaces, virtual interconnections, and not just UML instances. 

 If the vde_switches were just connected with wirefilter "patch cables" without modification, we would end up creating a broadcast domain and switch loops which we don't want: The goal is to allow the packets to travel only from one host to it's neighbor, not farther. To accomplish this, the vde_switch needs to be modified to have "coloured" ports. 

The idea is: - each port has a "colour" (an integer number) - packets are only passed from ports to others with DIFFERENT colours. 

Packets are dropped on outgoing ports if it has the SAME colour (same number) as the incoming port. 

 In this concept, the host port can have colour 1 the TAP port colour 2, while the interconnection ports have colour 0. In this way, packets can only travel from the host to (all of) the interconnection ports, or from one interconnection port to the host port. However packets can not travel between the interconnection ports, thus only allowing "one hop" connections and avoiding switch loops and shared broadcast domains. The concept is illustrated in figure 2. 

The patch against vde2-2.3.2 (current latest stable version) to add this colour patch can be find here: [vde2-2.3.2_colour.patch](http://www.open-mesh.org/attachments/download/152/vde2-2.3.2_colour.patch).

The vde-switch parched can be download from [here](https://github.com/dbritos/Network-mesh-emulator/blob/master/vde2-2.3.2-patch.tar). Attention this version can be old! 

wirefilter 

The wirefilter program is a tool where its posiblen simulate various link defects and limits as example:

• packet loss
• burst loss
• delay
• duplicates
• bandwidth
• Interface speed
• Chanel capacity
• noise (damage to packets)
•  mtu 

However as the links are only set up bidirectional, interferences can unfortunately not be simulated with this system. 

 For advanced testing it might be necessary to apply the aforementioned link defects to some packets only whereas other packets are able to traverse the emulated environment unharmed. 

Once you applied the 'ethertype' patch you can specify an ethertype which wirefilter will simply forward. To apply a packet loss of 50% to all packets except batman-adv packets, run:

wirefilter --ether 0x4305 -l 50

This patch also allows to filter batman-adv packet types. To apply a packet loss of 50% to all packets except batman-adv ICMP packets, run:

wirefilter --ether 0x4305:02 -l 50

You can specify up to 10 packet types (separated by colon). The patch against vde2-2.3.1 (current latest stable version) can be found here:

vde2-2.3.1-wirefilter-ethertype.patch

Bibliography

http://linux.die.net/man/5/snmpd.conf

http://www.bufferbloat.net/projects/cerowrt/wiki/Automated_Configuration_of_CeroWrt?version=1

http://www.net-snmp.org/docs/man/snmpd.examples.html