Testing Methodology
A glance at the chips on test reveals a mixed batch: we've got single and quad-core parts running at speeds ranging from 1.4GHz up to 2GHz. A straightforward benchmark would reveal which chip is the fastest - and we'll be doing that as well. However, this would merely test the implementation, and we're looking to go significantly deeper.
We'll be running a battery of benchmarks across the devices and then correcting for the different specifications. This will help to reveal the overall efficiency of the design, rather than just the particular implementation in question.
Each processor will be put through three tests: a synthetic benchmark run, a real-world benchmark run, and a graphics benchmark run.
The correction will attempt to map each processor's benchmark scores onto a theoretical implementation running at 1GHz. The multi-threaded synthetic benchmark scores will additionally be corrected for a single-core chip, whilst the real-world results will retain their multi-core abilities - as, whatever manufacturers may claim, the majority of real-world uses are largely confined to a single processing thread.
To keep things fair, each handset on test is subjected to the same conditions: benchmarks are performed while the device is connected to USB for charging, and with a SIM card inserted connecting them to Vodafone's 3G network. This provides the same background level of activity as an average handset in use.
So how did they get on?
Synthetic Results
The synthetic benchmark results assume a theoretical implementation of each chip with one core running at 1GHz. The combined figures from four popular Android benchmarks are surprising as Intel's Atom chip comes out as the top dog.
Considering the Z2460 is Intel's first real crack at the mobile-friendly system-on-chip market, that's a stunning result. It also indicates that it isn't just the high 2GHz clock speed which keeps the Razr i ticking over nicely, but a fundamentally solid design.
Most surprising was the performance of Qualcomm's Snapdragon S4 Pro chip. Designed to compete with high-end processors like the Samsung Exynos, its performance running at a theoretical 1GHz was sub-par and it came last. Even Nvidia's Tegra 3, which is known more for its graphics performance than its compute performance, outdid Qualcomm's flagship processor.
Samsung's Exynos, meanwhile, placed a solid second. The Korean manufacturer has a reputation for high-performance handsets, but its next-generation design will have to have boast some significant improvements under the hood.
The unadjusted results, which do not correct for clock-speed or the number of cores available, tell a different story, of course. Here, the high clock speed of Intel's Atom can't make up for the fact that there is just a single-core so it comes bottom of the pile. The Exynos was the best performing of the ARM chips.
