Benchmarking in your lap

It’s been a little over 2½ years since I last looked at the performance of Apple’s mobile devices. A lot has changed with the software and hardware since then, let’s take a look at how the new iPad compares to the devices we’re more familiar with.

Native performance

Test iPad/3.2 iPhone 3GS/3.0 Faster by
100,000 iterations 0.000035 secs. 0.000137 secs. 3.91x
10,000 divisions 0.000010 0.000018 1.8x
10,000 sin(x) calls 0.000012 0.000018 1.5x
10,000 string allocations 0.004321 0.007915 1.83x
10,000 function calls 0.000338 0.000600 1.78x

This first benchmark compares the native performance of the iPad against the iPhone 3GS. The version of the iPhone OS is the one originally released with both devices (3.2 and 3.0, respectively.) The application used to test the devices was a release (optimized) build for ARM v7.

On average, the iPad is about twice as fast as the iPhone 3GS when executing native (Cocoa Touch) applications. Great news for developers, because it gives us much more flexibility when creating our apps.

JavaScript performance

Test iPad/3.2 iPhone 3GS/3.0 Faster by
100,000 iterations 0.011 secs. 0.017 secs. 1.55x
10,000 divisions 0.006 0.012 2.00x
10,000 sin(x) calls 0.009 0.024 1.85x
10,000 string allocations 0.007 0.017 2.43x
10,000 function calls 0.005 0.006 1.20x

Oddly, the same 2x improvement is not seen for the same benchmarks when executing JavaScript code in web pages. Again, the iPad with 3.2 is compared against the iPhone 3GS with 3.0. Things have definitely improved, but there was a wide variation in results when performing the tests. I suspect that just-in-time compilation or other similar types of caching are affecting the results.

The raw numbers really don’t matter anyway: the browser experience on the iPad is exceptional (and will only get better.)

Native performance: Original iPhone vs. iPad

Test iPad/3.2 iPhone/2.0 Faster by
100,000 iterations 0.000035 secs. 0.015 secs. 428x
10,000 divisions 0.000010 0.004 400x
10,000 sin(x) calls 0.000012 0.105 8,750x
10,000 string allocations 0.004321 0.085 20x
10,000 function calls 0.000338 0.004 12x

The most remarkable change is when you compare the original iPhone to the iPad. Using the numbers from my original tests and the results above reveals an improvement of several orders of magnitude in just over 2½ years. I believe the technical term for this is “Holy crap!”

Note: I don’t remember if the original tests were optimized builds, or if it was even possible to get gcc to do them with a jailbreak toolchain. Even if they weren’t optimized like the current tests, the performance increases are still stunning.

All-in-all, a remarkable achievement by Apple’s engineers, especially when you consider that the battery life of these devices has gone up, rather than down.

iPad hardware

When using sysctl() to query the amount of memory available to the iPhone OS, the following values are reported:

Kind Bytes Megabytes
Physical (HW_PHYSMEM) 258,998,272 247 MB
User (HW_USERMEM) 210,284,544 200.5 MB

Again, it’s likely that the device is using 256 MB memory chips with a portion of the total being used for other purposes, such as video memory. Also, developers have noted that all of the 200 MB of user memory is not available for use by apps. The OS and background apps make use of a large portion of that memory.

And last, and unfortunately least, we have the CPU speed reported by sysctl() using HW_CPU_FREQ. It’s zero: the A4 chip is truly magical. I guess that means we have to trust the spec sheet :-)

So hats off to the engineers in Cupertino: the iPad is an inspiring achievement no matter how you benchmark it!

P.S. Check out Twitterrific for iPad to see how we’re putting this processing power to use with our first app for this new platform.

P.S.S. If you’d like to perform these tests on your own, I invite you to download PantsTest.zip.