Wireless Network Experiment Methodology

Introduction

Wireless networks are often hard to replicate because the RF environment can be highly dynamic and it is all but impossible to ensure that the RF environment across the experiments is identical. It is also difficult to directly compare the results across multiple testbed facilities because the node topology, physical constraints, and RF environment are almost always different on different facilities. This inability to reproduce the results and compare findings across the experiments is a serious impediment, which we should overcome to bring scientific rigor to wireless experimentation. In this project, we explore experimental techniques to remove the environmental variations during protocol comparisons, design a metric that captures the state of the network during a wireless experiment, and compile a benchmark suite for network protocols to standardize the protocol performance evaluations.

Concurrent Experiments

Researchers typically evaluate and compare protocols on the testbeds by running them one at a time. This methodology ignores the variation in link qualities and wireless environment across these experiments. These variations can introduce significant noise in the results. Evaluating two protocols concurrently, however, suffers from inter-protocol interactions. These interactions can perturb performance even under very light load, especially timing and timing sensitive protocols. We argue that the benefits of running protocols concurrently greatly outweigh the disadvantages. Although the wireless environment is still uncontrolled, concurrent evaluations make comparisons fair and more statistically sound. Through experiments on two testbeds, we make the case for evaluating and comparing low data-rate sensor network protocols by running them concurrently.

Capturing the Link and Network State During an Experiment

Text description of the experiment setup, which is the norm, is usually not enough to reproduce the results of a wireless network experiment. Such descriptions do not capture the properties of the testbed that dictate the feasible network topology. Neither do they capture the wireless dynamics present during the experiment. We design a metric called Expected Network Delivery that succinctly captures the state of the links and the network as they pertain to the performance of a wirelss routing protocol. Such a quantitative metric that describes the experiment environment allows us to compare the results from wireless network experiment across time and across the testbeds.

Benchmark for Wireless Network Protocols

Publications