DxOMark gets with the times, updates mobile camera testing protocols to account for bokeh and zoom

Popular imaging benchmark website DxOMark has updated its Mobile test protocol. The company announced the changes in the testing protocol in a series of posts on its website. The company pointed out that it launched the original testing protocol back in 2012 and smartphone cameras have drastically changed since then.[caption id="attachment_4041659" align="alignleft" width="380"] Image Credit: DxOMark[/caption]Among of the first devices that the company tested included the Apple iPhone 4 along with Samsung Galaxy S2, S3m, HTC 8X and the Nokia 808 Pureview. These days, the smartphones that the benchmarking site has to test include the HTC U11, LG G6, OnePlus 5 and other 2017 flagship devices. Just to give you a perspective about the change, the iPhone 4 packed a 5 MP camera sensor on its back and a lens with an f/2.8 aperture. Today, the HTC U11 packs a 12 MP camera sensor on its back with an f/1.75 aperture lens, OIS, and phase detection autofocus (PDAF). The iPhone 7 Plus packs in a dual camera on its back, one featuring a tele lens and the other a normal one. Bokeh and zoom are aspects that we now factor into our smartphone purchases.Some of the key changes that smartphone cameras have gone through these last few years include advanced CMOS image sensors with powerful processors, multi-frame processing, improved low-light performance, PDAF and laser-assisted autofocus. This is not all, mobile manufacturers have also started implementing dual-camera setups which enable optical-zoom and a shallow depth-of-field bokeh effect to mimic images shot by DSLRs with their massive sensors and lenses with wide apertures. The implementation of the dual camera setup is also not consistent and varies between different smartphone makers.[caption id="attachment_4041677" align="alignnone" width="825"] Image credit: DxOMark[/caption]DxOMark has expanded its testing to include new tests and expand its existing ‘outdoor and lab test scenes.’ With this new protocol, the benchmarking process now involves capturing and then analysing more than 1,500 images and two hours of video captured and recorded on every test device. The new tests include a new zoom test where it will test the zoom level at different focal lengths, a bokeh test that is based on outdoor and lab testing scenarios, etc. The new low-light test includes shooting images in a setting with brightness of just 1 lux. The benchmarking site has also added a new ‘motion-based’ test to check and accurately evaluate camera performance and the abilities of the processing technology behind the camera in the real world.It notes that most its tests are taken in default (auto) mode and the website only changes the setting to enter the portrait, flash and zoom modes. The company pointed out that this protocol revamp is not limited to photographs and expands to video tests also. The new video tests also check how smoothly and quickly the camera adapts to changing lighting conditions while recording. These new tests mean the introduction of new sub scores, which would mean changes to the scoring system. DxOMark is currently retesting devices to update its rankings based on the new protocol.