DASHbed
A testbed Framework for Large Scale EmpiricalEvaluation of Real-Time DASH in Wireless Scenarios
We kindly ask that should you mention DASHbed, or use our code, in your publication, that you would reference the following paper:
D. Raca, Y. Sani, C.J. Sreenan and J.J. Quinlan, "DASHbed: a testbed Framework for Large Scale Empirical Evaluation of Real-Time DASH in Wireless Scenarios". In Proceedings of the ACM Multimedia Systems Conference (MMSys 2019), Amherst, MA, USA. June 18-21 2019. [CORA] [ACM]
Darijo Raca, Dr. Yusuf Sani and Prof. Cormac Sreenan presenting DASHbed at MMSys 2019 in Amherst, MA, USA.
Abstract: Recent years have witnessed an explosion of multimedia traffic carried over the Internet. Video-on-demand and live streaming services are the most dominant services. To ensure growth, many streaming providers have invested considerable time and effort to keep pace with ever-increasing users demand for better quality and stall abolition. HTTP adaptive streaming (HAS) are at the core of every major streaming provider. Recent years have seen sustained development in HAS algorithms. However, to compare their proposed solution, researchers need to create a framework and numerous state-of-the-art algorithms. Often, these frameworks lack flexibility and scalability, covering only a limited set of scenarios. To fill this gap, in this work we propose DASHbed, a highly customizable real-time framework for testing HAS algorithms in a wireless environment. Due to its low memory requirement, DASHbed offers a means of running large-scale experiments with a hundred competing players.
Finally, we supplement the proposed framework with a dataset consisting of results for five HAS algorithms tested in various evaluated scenarios. The dataset showcases the abilities of DASHbed and presents the adaptation metrics per segment in the generated content (such as switches, buffer-level, P.1203 values, delivery rate, stall duration, etc.), which can be used as a baseline when researchers compare it to the output of their proposed algorithm.
DASHbed: a testbed Framework for Large Scale Empirical Evaluation of Real-Time DASH in Wireless Scenarios
We provide instructions for setting testbed. We use Vagrant for managing virtual machines.
Link: Instructions Download
Note: Alternatively, you can set up DASHbed using physical machines. You follow the instructions in the link above, and download the server and client code below:
Link: DASHbed (source code) Download
Link: Modified dashc client video player (source code) Download
DATASET: Complementary dataset generated with DASHbed
Link: DASHbed Dataset Download
EXAMPLES
As an example, we randomly extract one bw trace and show the following scenarios:
- one algortihm per bw trace, 300 seconds, one run, two segments durations (2,4 seconds)
- five competing clients with different streaming algortihms, 300 seconds, one run, two segment durations (2,4 seconds)
- four competing clients with same streaming algortihm, 300 seconds, one run, 8-second segment duration
2-, 4- and 8-second logs can be viewed/downloaded from the following link: Download
Figure 1 shows the P.1203 QoE for all five algortihms in a single case scenario (4 seconds segment duration)
Figure 2 and 3 show P.1203 QoE for all five algortihms in a competing case scenario for 2 and 4 seconds segment durations, respectively
Figure 4 shows P.1203 QoE for four competing clients with same algortihm (logistic) (8 seconds segment duration)
This work has emanated from research conducted with the financial support of Science Foundation Ireland (SFI) under Grant Number 13/IA/1892.
