OnePlus 5 caught cheating with benchmarks yet again; this time with review units

Editor's Note: At the time of writing, OnePlus India was yet to respond to our request for a comment. This article will be updated to reflect OnePlus India's comment when we receive it. It's another year, and it's another OnePlus and it's yet another OnePlus hack. Reviewers who received the OnePlus 5 before the global launch (which took place on YouTube via a pre-recorded live broadcast) have begun to put out their reviews out today. While most of the reviews don't seem to be positive (the camera does not stand up to the new price tag) there are many who have praised the performance of the device, keeping the 8 GB RAM in mind. Oddly, that list does not include the folks at XDA, who have now claimed that the OnePlus 5, similar to the OnePlus 3T before it, has been caught cheating with benchmarks to produce unusually high scores for reviewers.Indeed, that hype train has come to a screeching halt. According to the XDA Forums, the authority online for all things associated with the word 'tweaking' and 'Android', pointed out that OnePlus may have delivered every single review unit out there with a CPU cheating mechanism built-in, one that would lead to misleading results or abnormally smooth performance in the hands of reviewers.XDA pointed out that  most reviewers have been fiddling with these tampered devices for more than a week now, and that few of those reviewers will have the resources to prove the tampering. As a result, early reviews are very likely to be biased, and this is entirely OnePlus' fault.Every CPU has a bottom end and a top end when it comes to speeds. In the case of the Snapdragon that clock speed would be 2.45 GHz at the top end using the high powered cores. While an average flagship chipset idles at about 0.98 GHz at the bottom end for the lower powered cores (Little cluster). In the case of the OnePlus 5, XDA claims that the chipset inside the device has been tweaked to idle at 1.9 GHz (1,900 Mhz). While this will allow it to see abnormally high scores on benchmarks, it should also result in lesser battery life and the phone heating up quickly.[caption id="attachment_384149" align="aligncenter" width="640"]

Image: XDA[/caption]"While no customers have a device in their hands (it just launched after all), we have learned about OnePlus’ new benchmark cheating mechanism through our review unit, which we received about ten days ago before the day the embargo breaks and reviewers are allowed to report on the device. Unfortunately, it is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results, as OnePlus provided reviewers a device that cheats on benchmarks."In the lengthy report, describing its findings, XDA said, "This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify. As a result, every OnePlus 5 review citing benchmark scores as an accolade of the phone’s success is misleading both writers and readers, and performance analyses based on synthetic benchmarks are invalidated. What is worse is that, this time around, the cheating mechanism is blatant and aimed at maximizing performance, unlike last time which did not increase scores by much on average, but did reduce variance and thermal throttling, as we found."XDA's results showed abnormally high benchmark scores compared to flagships with similar chipsets and unfortunately, even with a Qualcomm reference device (that comes from the chip maker) that the team fortunately spent some time with.Indeed, this puts OnePlus in a tight spot. When contacted, OnePlus told XDA, "People use benchmark apps in order to ascertain the performance of their device, and we want users to see the true performance of the OnePlus 5. Therefore, we have allowed benchmark apps to run in a state similar to daily usage, including the running of resource intensive apps and games. Additionally, when launching apps the OnePlus 5 runs at a similar state in order to increase the speed in which apps open. We are not overclocking the device, rather we are displaying the performance potential of the OnePlus 5."Clearly, that's not the answer anyone was looking for. Still, it's easy to say that OnePlus is not the first manufacturer to cheat when it comes to benchmarks. In a piece put out earlier this year, XDA shamed a number of manufacturers including Samsung, HTC, Sony and OnePlus for cheating on benchmarks. The problem here is that this is the second time OnePlus has been caught red-handed.XDA did mention that OnePlus may have, in all probability, changed the software on devices sold to consumers, this should show a gap in the numbers between what reviewers received (and propagated) and what its customers will really get.