5G: THE GREATEST SHOW ON EARTH!

Vol 16: Apples to Apples

02/17/2021 | 58 pages
Price: $1,395.00

SRG just completed its sixteenth 5G benchmark study, this time with a focus on evaluating overall network performance and its impact on the user experience, based on ETSI TR 103 559. Testing took place in the Dallas metro area, but the analysis of the data and the performance KPIs that really influence the user experience are applicable on a global basis.

Highlights of the Report include the following:

Our Thanks.  We did this study in collaboration with Rohde & Schwarz. R&S has implemented ETSI TR 103 559 into its Network Performance Score (NPS) framework, thereby providing us with a mechanism to thoroughly analyze how three operator networks in North America perform when evaluated against the ETSI recommendations. SRG is solely responsible for the analysis of the data and the commentary in the report.

Background.  ETSI has published technical recommendations on best practices for benchmarking mobile networks. These recommendations are based on global operator inputs which define how to collect and score the data, as well as the performance indicators that really influence the customer experience.

Our Goal.  Although testing occurred in the US, the results and analysis in this report are global in nature. Max speeds are nice for marketing purposes, but they have little bearing in determining the user experience with typical applications and use cases. We prove this point and show why an industry-approved and fully-disclosed approach makes sense for benchmarking purposes. In our test results, the operator with the “slowest network” finished first and the operator with the greatest 5G NR network coverage finished last.

Testing Scope.  The data collection process involved 115 hours of testing, 5,071 kilometers of driving, 1,300+ voice calls per network, and 60,000+ data-related tests (HTTP browsing, video, social media, HTTP file transfers and capacity tests, as well as interactivity tests).

Deep Analysis.  In addition to providing the overall scores and the individual scores for each test (all scoring and weighting in compliance with the TR) we delve into why the results varied by operator. We also look at 5G NR/LTE coverage and performance differences between radio access technologies (by operator), including an analysis of underlying RF parameters. Finally, we analyze IP/session information to discuss other critical factors which influenced performance (video MOS, webpage load times, latency, packet loss, etc.).

The Results are in.  For reasons explained in the report, AT&T came out on top overall (voice and data), followed by Verizon and T-Mobile. Voice results showed very little differentiation between operators, but there were big differences in data performance.