DES-Cript

DES-Cript is a domain specific language (DSL) to define and describe networking experiments which can be executed on a testbed. DES-Cript aims at simplifying the execution of experiments by using a comprehensible and straight-forward design scheme and therefore assuring the repeatability of experiments.

Design Goals

As a fundamental design goal, an experiment description should be easy to read and understand for the users. Therefore, we decided to develop DES-Cript based on XML, inheriting its easy-to-read property. This allows a clear and hierarchical structure, rendering the description a comprehensible foundation of an experiment. The structure of the description is closely mapped to the workflow of a network experiment as depicted in detail below. All consecutive steps in the experiment workflow are represented as different sections in the DES-Cript file, which are basically the configuration, execution, and the evaluation.

DES-Cript introduces an abstraction layer by separating the experiment description from its actual execution. Since tasks like node access and correct timing of commands are handled by the execution software (DES-Exp), the DES-Cript file contains only the crucial matter of an experiment. Basically, a DES-Cript file contains commands that will be executed on nodes at a specific time or under specific conditions. How the commands are executed, for example via an SSH connection established over Ethernet or using the wireless mesh network is irrelevant to the user. This decreases the complexity of the description and puts the focus completely on the experiment matter.

DES-Cript should also meet the requirement of repeatability of experiments. An experiment description specifies the number of iterations it is executed, which enables to compare the results of multiple test runs of the same experiment. It is also possible to reuse and reschedule an experiment. This can be useful to execute the experiment with different parameters. The DES-Testbed for example is set-up on our campus and Access Points of different networks operate on the same frequency band. Therefore, it might be useful, to execute the experiment at night times in order to minimize interferences. However, it is also interesting to compare the results to execution times in which a higher level of interference is expected.

The communication of experiments among researches can be simplified by exchanging the corresponding DES-Cript files instead of a verbal description as is usual given. This way, experiments can be carried out on different platforms and the results can be compared and checked. Also, possible flaws in the experiment design can be discovered and corrections and improvement discussed and incorporated into the original description.

Structure

The basic structure of a DES-Cript, as shown below, is closely related to the already described workflow of an experiment. The description is divided into three major sections. First, a general section contains all meta-data of an experiment. Next follows the init section, in which the testbed configuration prior to the experiment execution is specified. Finally, the actions section lists all commands that will be executed in the experiment run.

 <experiment>
    <general>
        <name>...</name>
        <start_time>...</start_time>
        <owner>...</owner>
        <description>...</description>
        <iterations>...</iterations>
        ...
        <groups>
            <group name="groupname" role="Server">
                <members>...</members>
            </group>
        </groups>
        ...
    </general>
    
    <init>
        <action id="0">
            ...
        </action>
        ...
    </init>
    
    <actions>
        <action_block id="1">
            <action id="1">
                <group>...</group>
                <command>...</command>
                <start_time>...</start_time>
                <duration>...</duration>
                <evaluation_script>...</evaluation_script>
            </action>
            ...
        </action_block>
        ...
    </actions>
</experiment>

General

The general section contains settings which are applied to the experiment environment and general information about the particular experiment. Next to the name and the creator of the experiment, a description can be provided, which explains the design of the experiment and the matter it wants to investigate. The preferred starting time is set, which will be used by DES-Exp to schedule the experiment. The monitoring interval specifies the frequency in which the state of the testbed is monitored throughout an experiment. The number of iterations specifies how many times the execution of the experiment is repeated in one experiment execution. A pause value sets the delay between consecutive iterations. This is useful, if for example temperature or humidity measurements are carried out every minute for a long duration using the sensor network of the testbed.

Additionally, testbed nodes are arranged into groups with an assigned role. The three existing roles, server, client, and servent, allow to experiment with traditional client-server scenarios as well as with peer-to-peer applications. Commands are assigned to groups, in contrast to one command for each node. To avoid unreasonable scenarios, some restrictions are enforced to the grouping mechanism, so that in one ActionBlock, two different groups with a different role and a subset of mutual nodes, e.g., being a server and client at the same time, can not be used.

Preparation

The init section comprises the configuration of the testbed prior to the experiment execution. It includes the configuration of the nodes, such as loading a specific kernel image and modules. Next, the settings of the network adapters can be changed, for example the channel and the data rate using standard tools such as iwconfig.
Also routing daemons and server instances can be started, which will be used throughout the experiment.
In case of a proactive routing algorithm, this can be helpful since the routing topology is already established at the start of the experiment. Programs, which need to run in server mode throughout the entire experiment such as iperf, can already be started so that the clients can connect to them and start traffic flows.

Execution

The action section consists of all commands called in an experiment, it is therefore the heart of every DES-Cript file. This section consists of a series of ActionBlocks, each containing at least one action. These ActionBlocks are executed sequentially.
ActionBlocks support three different execution modes for their comprising actions: sequential, parallel, and fixed time offsets. In sequential mode, actions are executed one after the other until all actions of the ActionBlock are processed. This is useful, if actions depend on each other and therefore the order of execution is to be prevailed. Secondly, in the parallel execution mode, all actions of the ActionBlock are executed in parallel. Scenarios with parallel traffic flows in order to discover bottlenecks of a particular topology can be investigated with this mode. The third mode lets the user specify the point of time of execution for each command.
For instance, an iperf server can be started at the very beginning of an ActionBlock and the corresponding iperf client can be scheduled to start 5 seconds later.
For all actions a maximum duration can be set. If the action has not returned after this time, the process will be killed.
This is especially useful for server applications which do not terminate after a specific time.

Evaluation

After the execution of all ActionBlocks, the generated output files of the commands are collected by the testbed server. Then, the evaluation phase is carried out, thus freeing the nodes of additional computation during the experiment. For each action, an evaluation script can be specified to process the output of command and insert the formatted results into a database. The results can then be used with programs like Matlab, GNUPlot, or Maple to visualize the results. The evaluation scripts are written in Python and we provide a suite of scripts for common tools in network experiments. Additionally, a template exists which enables the user to develop a custom evaluation script for further programs. This template contains the database connection and example code, rendering the development of custom evaluation scripts a straight-forward and rapid process.

Example - Neighborhood Discovery

Broadcasting ping messages can be used to determine the connectivity of a wireless network. The results, which consist of the ping reply messages, reveal all neighbors for a particular node. This basic experiment discovers all direct links between nodes and the result can therefore be used as an indicator for the network density.

<experiment>
<general>
        <name>NeighborDiscovery</name>
        <description>Uses broadcast pings on all nodes to perform a neighbor discovery.</description>
        <start_time>2009-04-23 10:00:00</start_time>
        <iterations>1</iterations>
        <sample_interval>60</sample_interval>
        <lock_testbed>true</lock_testbed>
        <groups>
            <group>
                <name>All</name>
                <role>Server</role>
                <members>
                    <node id="t9-149"></node>
                    ...
                </members>
            </group>
        </groups>
    </general>
    <actions>
        <action_block id="1" execution_mode="2">
            <action id="1">
                <command>ping -w20 -i3 -b 192.168.18.127 -I wlan0</command>
                <group>All</group>
                <start_time>0</start_time>
                <duration>30</duration>
                <evaluation_script>3</evaluation_script>
                <evaluation_parameter>chan1</evaluation_parameter>
            </action>
        </action_block>
        <action_block id="2" execution_mode="2">
            <action id="1">
                <command>ping -w20 -i3 -b 192.168.18.255 -I wlan1</command>
                ...
            </action>   
        </action_block>
        ...
    </actions>
</experiment>

First, all testbed nodes are arranged into one group, and for each wireless network adapter a broadcast ping is issued as specified in the respective ActionBlock. The output of the ping command is parsed by an evaluation script, which inserts the processed results into the database.

Each row in the database result table represents a link between two testbed nodes and contains information of the Round-Trip Time (RTT) and packet loss rate taken from the ping command output. This data can be used to estimate the link quality between two nodes. Further analysis can be carried out on the result set in the database, such as calculating the longest or average path length in the network.