What Goes Around, Comes Around
What Goes Around, Comes Around
A Third Party Benchmark Study of mmWave Device + Chipset Performance
02/21/2023 | 55 pages
SRG just released a benchmark study that we did on behalf of MediaTek. The study provides results from testing several 5G mmWave smartphones, representing chipsets from the three leading suppliers. The study focused on each smartphone’s mmWave beam tracking ability and the subsequent impact on performance with a highly-controlled test methodology.
Highlights of the Report include the following:
Our Thanks. We did this study with the support of Accuver Americas (XCAL5) and Spirent Communications (Umetrix Data). SRG is fully responsible for the data collection and all analysis and commentary provided in this report.
Our Methodology. Testing took place over a two-day period in downtown Minneapolis on the Verizon 5G mmWave network. We tested at several stationary locations with varying distances to the serving cell, including near/non-LOS conditions. We placed the reference phone – a Motorola edge (2022) smartphone with the MediaTek 5G modem – and one other smartphone under test on an electric rotating platter to evaluate how they performed while slowly turning with a full buffer data transfer taking place. We repeated the test for each of the smartphones in our study.
Our Analysis. When possible, we used XCAL5 to evaluate 5G performance at the physical layer and how the performance was impacted by the rotations. When necessary, we used Umetrix Data to analyze throughput at the application layer for those smartphones where we lacked access to physical layer parameters. In these cases, we leveraged chipset data from the Motorola edge (2022) smartphone to infer the physical layer performance of the other smartphone (e.g., fair sharing of network resources).
The Results. The report provides results for all tests, including a much richer analysis of physical layer performance attributes whenever possible. There is also a sensitivity study which shows performance without the rotations. The results demonstrate the criticality of beam tracking mechanisms and show that differences do exist across leading handsets and 5G modems.