P.1203/VQM: Qualitative Metrics for HTTP Adaptive Streaming
P.1203/VQM: Qualitative Metrics for HTTP Adaptive Streaming - dataset and software
The following outlines the various files and scripts to use with our P.1203 and VQM datasets and software.
Details on the P.1203 JSON files with reference to our original DASH datasets:
- P.1203 Dataset - The following tables provide links to the compressed P.1203 JSON input files.
- Files are compressed per P.1203 mode, with reference to our original DASH dataset.
- Full High Definition (Full HD - 1920x1080) is available in both copyright and non-copyright content.
- The copyright content, 20 clips, is typically used for trace-based simulation, while the non-copyright content, 3 clips, providing metrics for decodable streamed content.
- Ultra High Definition (UHD - 3840x2160) content, 3 clips, is only available for P.1203 up to the Full HD resolution and is non-copyright content.
- Each zip file contains a P.1203 JSON file per segment, for each clip and across five different segment durations (2-sec, 4-sec, 6-sec, 8-sec, and 10-sec).
- Content duration is typically between 10 minutes and 16 minutes in duration.
FULL HD Copyright Content:
| P.1203 | P.1203 | P.1203 | P.1203 |
|---|---|---|---|
| mode 0 | mode 1 | mode 2 | mode 3 |
FULL HD Non-Copyright Content:
| P.1203 | P.1203 | P.1203 | P.1203 |
|---|---|---|---|
| mode 0 | mode 1 | mode 2 | mode 3 |
UHD Copyright Content:
| P.1203 | P.1203 | P.1203 | P.1203 |
|---|---|---|---|
| mode 0 | mode 1 | mode 2 | mode 3 |
The branched version of dashc can be found here. Additional options in this version of dashc include:
- [-extended], display aditional columns with information on QoE metrics - for P.1203 this will equate to 'true'
- [-preferred-codec], (h264, h265, vp09, av01) stream using the specified codec - for P.1203 this will equate to 'h264'
- [-max-rep-height] the maximum resolution height to stream at (defauls as 2160) - for P.1203 this will equate to '1080'
The VQM dataset is generated using the Command Line Video Quality Metric \textit{(CVQM)} tool available here, which is written in MATLAB. CVQM was ported to Ubuntu OS and Python script are developed to automate the calculation of VQM values for individual segments of our Full HD video dataset. The Python script, available here, while the generated dataset is available here. As the only limitation in VQM is a sufficiently large segment duration, no less than 4 seconds, we were also able to generate content for the H.265 (HEVC) element of the Full HD dataset.
A VirtualBox VM of Ubuntu 18.04, containing all the open-source programs needed to stream our UHD and Full HD Datasets using the extended version of our dashc player is available here. The extended version of dashc will generate additional columns including a P.1203 video QoE metric per row (per segment request).
Sample output of dashc is shown below complete with summary details:
WARNING: WidthxHeigth (3840x2160) of clip @ rate 41.66Mbps - is greater than defined max_rep height 1080
WARNING: WidthxHeigth (3840x2160) of clip @ rate 25.99Mbps - is greater than defined max_rep height 1080
WARNING: WidthxHeigth (3840x2160) of clip @ rate 15.56Mbps - is greater than defined max_rep height 1080
Seg_# Arr_time Del_Time Stall_Dur Rep_Level Del_Rate Act_Rate Byte_Size Buff_Level RTT Codec Width Height FPS Seg_Dur Start P.1203 QoE
1 30 30 0.000000 239 551 8 2132 2.000 0.061644 h264 320 180 24 2.000 0.000 1.936882
2 1248 555 0.000000 378 1970 547 136969 4.000 0.061612 h264 384 216 24 2.000 2.000 1.936882
3 5423 3512 0.000000 378 410 720 180164 2.488 0.061771 h264 384 216 24 2.000 4.000 1.939419
4 7163 1078 0.000000 239 802 432 108123 3.409 1.118332 h264 320 180 24 2.000 6.000 1.938224
...
...
147 187992 429 0.000000 239 1287 276 69090 209.526 0.061644 h264 320 180 24 2.000 292.000 1.951534
148 188963 308 0.000000 239 1858 286 71640 211.218 0.061690 h264 320 180 24 2.000 294.000 1.951534
149 189712 87 0.000000 239 4477 194 48706 213.131 0.061546 h264 320 180 24 2.000 296.000 1.951534
150 190492 118 0.000000 239 2497 148 37007 215.012 0.061612 h264 320 180 24 2.000 298.000 1.951534
Downloaded 150 segments (2-second h264) totalling 9.31 MB
P.1203 Quality of Experience: 1.9515
Round Trip Time (RTT) min/avg/max: 0.061/0.103/1.486
Delivery rate min/avg/max: 199.453/1460.620/6683.705 KB/s
Number of stalls/Total stall time: 0/0.000
Representation level min/avg/max: 239.515/241.369/378.577 KB/s
Resolution min/mode/max: 320x180/320x180/384x216
Enjoy, Jason (j.quinlan@cs.ucc.ie)
This work has emanated from research conducted with the financial support of Science Foundation Ireland (SFI) under Grant Number 13/IA/1892.
