Romain Jacob (ETHZ) : Research replicability in networking research

Improving the standards of replicability in networking has recently gained traction within the community. As an important piece of the puzzle, we have developed a systematic methodology that streamlines the design and analysis of performance evaluations, and we have implemented this methodology into a framework called TriScale.

In this talk, I will introduce the main concepts of the methodology and even let you try out with TriScale. By the end to the session, you will be able to:

• Understand the difference between replicability and reproducibility, and why these notions matter;
• Understand why performance evaluation experiments must be replicable to be meaningful;
• Understand the basics of statistics required to assess replicability;
• Answer questions such as "How many time should I repeat my experiment?" rationally;
• Use the TriScale framework to help you design your next experiments, analyze your data, and report your results in a (more) replicable fashion.

The methods and principles underlying TriScale are applicable to different performance evaluation contexts, including simulations, emulations, and experiments on actual networks. Throughout the talk, we draw on networked embedded systems use cases to illustrate the main concepts, but the principles are broadly applicable to networking systems, and beyond.