Confidence

We witnessed something amazing yesterday: a WWDC keynote that will be remembered for a long time. And not for the reason you might expect.

As developers, it’s easy to focus on the fantastic software that was announced: a UI refresh that’s getting a thumbs up from designers and developers alike, great new user-facing features in iOS and OS X, and literally thousands of new APIs that let developers do new and amazing things with their apps. We even got a new programming language!

**BLINK**

But that all pales in comparison to the undercurrent for all these changes: Apple has a newfound confidence in itself. It’s at the top of its game, and it knows it.

This is personified by the man who ran the show: Craig Federighi. It was only four years ago that we first saw him on stage at an Apple event. His shaking hand is still painful to watch, especially if you’ve felt that same fear while giving a presentation on stage. Yesterday, we saw a different man, one that owned the stage and the products being presented.

With this confidence, we’re starting to see some important cultural change in the company.

  • Legal agreements that lets developers talk about technologies without breaking confidentiality.
  • Opening up proprietary technologies like iCloud: providing more transparent access, without hiding things through opaque APIs.
  • Improvements to the App Store that give developers better ways to manage and sell their products.

This confidence manifests itself in many ways. When was the last time you heard an Apple executive tout the best product line he’s seen in 25 years?

In short, with confidence comes a new kind of openness. As developers, we’ve always struggled with a company that doesn’t want to give anything away. Yesterday, that started to change.

Upheaval

In just under 12 days, Apple is going to introduce a new look and feel for Mac OS X. This is going to be much more disruptive than it was for iOS 7.

On iOS, an app takes up the entire screen and becomes an extension of the device. It’s also something you’re “in and out” of on a regular basis. It’s not the end of the world if one or more of your apps still looks like iOS 6 because it’s not competing with anything else and the pain is short-lived. Bank of America’s app is still on iOS 6, but I use it infrequently and usually just to deposit checks.

But on the Mac, an app usually shares the display with other apps. And they’re something that you use continuously. If a customer’s Email, Web, Calendar, Contacts and other built-in apps are using a fresh, new look, they’re going to feel put out when they switch to your app and get a legacy appearance.

You will stick out like a sore thumb.

We’ve already started the process on our own apps and are gearing up to help our clients adapt. If you haven’t carved out a large chunk of time this summer to do the same for your own app, do it now.

Counting Beans with AppViz 3

In my last post about AppViz, I mentioned that I spent about six months creating an internal accounting tool called BeanCounter. Why the hell did I do that?

Because I don’t want to run my business on financial guesses based on iTunes Connect sales reports.

The Need For Financial Accuracy

For many app developers and iBook publishers, income from iTunes Connect is a significant part of the business’ financial statement. In many cases, the App Store is a developer’s only distribution channel, so it’s responsible for 100% of gross revenue.

When you have a financial statement, you want it to be exact. If your local tax collector comes calling and you show them guestimates, you’re in for a rude awakening. You’ve got a paper trail that’s fishy and that’s going to invite closer scrutiny. It’s the reason every piece of accounting software ever written has a reconciliation feature.

Publishers pay author royalties and many app developers split revenue with a designer. When it comes time to share the monthly Apple ACH deposit, financial guesses are also a problem. Not only will you be giving your partners the wrong amount of money, you’ll eventually be reporting it incorrectly on a 1099-MISC. Your estimated payments can be off by thousands of dollars:

Sales Reports Are Not Accurate

The root of the problem is that all services except AppViz 3 use fluctuating values like fiscal calendar ending dates, payment clearing dates, and currency exchange rates. If you do, you’re making financial guesses.

Don’t take my word for it, here’s what App Annie has to say:

Why do my sales reports in App Annie not exactly match my financial reports in iTunes Connect?
App Annie fetches the sales data from the iTunes Connect “Sales and Trends” section. There are a few reasons why the sales data doesn’t match Apple’s “Financial Reports” and the amount of money you receive from Apple.
Fiscal calendar. Apple uses fiscal months rather than calendar months for payouts. To see Apple’s fiscal calendar, go to “Payments and Financial Reports” and click the “Fiscal Calendar” link. For example, fiscal March 2011 starts on Feb. 27 and ends on Mar. 26.
Clearing of payments. What matters in the case of financial reports is what payments were made during a fiscal month, not how many downloads were made. Since most people use credit cards to buy apps, it can take up to 2-3 days for a payment to go through. Also, some payments never go through!

Currency conversions. Apple’s “Financial Reports” use Apple’s own currency rates, whereas App Annie always uses today’s exchange rate.

Remember that the last two points above have a certain degree of unpredictability, which makes the figures we get from Apple’s “Sales and Trends” slightly different from the figures in Apple’s “Financial Reports”.

That last item is particularly bad: currency fluctuations will give you wildly inaccurate accounting. For example, let’s say you sold €10,000 of product and used the rate on October 28th, 2013 (1.38) to convert that amount to $13,800. Then, Apple deposited $13,400 into your account on November 7th using that day’s 1.34 rate. Hope you didn’t pay your partners using that financial guestimate of $13,800!

The bottom line is this: you cannot rely on estimates in Sales Reports when doing financial reconciliation or partner payouts. Only Financial Reports can be used for this purpose.

Financial Calculations Are Hard

The problem is compounded when you have multiple products with different partners. Figuring out how to divvy up the money turns out to be a hard problem and it’s why everyone else has punted on doing it right. It’s also a competitive advantage for us, so I’m not going to go into any details about how we get the numbers to come out right.

I will, however, give you a peek into the first hurdle you’ll encounter: you can’t use floating point values for financial calculations. If you write code like this, you’ll be in for a world of hurt as errors accumulate:

double subtotal = balanceTotal + 
    (salesTotal + adjustments);
if (subtotal != 0.0) { // is 0.00000001 a zero value?
  rate = deposit / subtotal;
}

Cocoa has an awesome subclass of NSNumber called NSDecimalNumber. Classes like NSDecimalNumber are one of the reasons that “NeXTSTEP had a long history in the financial programming community.”

When using NSDecimalNumber your code will get more verbose than the snippet above. But more importantly, it will now be exact:

NSDecimalNumber *subtotal = [balanceTotal
    decimalNumberByAdding:[salesTotal 
    decimalNumberByAdding:adjustments]];
if ([subtotal compare:[NSDecimalNumber zero]]
    != NSOrderedSame) {
  rate = [deposit decimalNumberByDividingBy:subtotal];
}

It also forces you to think about problems like rounding: do you want to use NSRoundBankers when your code calls -decimalNumberByRoundingAccordingToBehavior:?

Conclusion

I hope you now see why I thought it was wise to spend a lot of time coming up with a system that ensures financial accuracy. I’m also thrilled that this system is now a part of AppViz 3, the app that’s been keeping track of our products on iTunes Connect since the App Store first opened in June 2008.

It also makes me happy that other developers are seeing the same benefits for accurate financial reconciliation. Helping other developers is what makes me tick.

Take a moment to think about how these issues affect your own business and then take a look at AppViz 3. I think your balance sheet, your partners and the tax man will be happy you did.

AppViz, WTF?

AppViz 3 is a major rewrite of a product loved by many developers. Most parts of the app are completely new, but we know that two new features will be particularly contentious: cloud storage and a subscription model.

We’re developers selling a tool used by other developers. We know we can’t bullshit you with marketing buzzwords and a bunch of hand-waving. This post will explain exactly why we’ve introduced these features and let you draw your own conclusions. At a minimum, it will provide insight into the challenges and how we approached their solutions. If we’re lucky, you’ll agree with our pragmatism and continue to support our efforts.

tl;dr We’re not trying to screw you.

Learning from our past…

A product like AppViz should be easy to build, right? It just downloads a bunch of numbers in tab-delimited format, crunches the data and then reports it in graphs and tables. If only that were true.

We all know that the pace of iOS development has been staggering. In just five years, we’ve gone from the first apps on iOS 2 to the radical redesign of iOS 7. The iPhone itself has gone from being available in just the US to selling in a total of 100 countries. We’ve seen a completely new iOS device called the iPad and spectacular hardware improvements across the board. Mac developers also got to join in on the fun at the end of 2010. Publishers started selling iBooks like we sell apps that same year.

All of this, and it’s pretty incredible that many developers are just now celebrating their wood anniversary.

And through it all, Apple is using a time-honored development process: they’re making it up as they go along. Throughout those five years there have been a huge number of changes to the data coming out of iTunes Connect:

  • Changes to the formats and columns in existing reports
  • Changes to the data values in the reports
  • New kinds of reports, like financial and earnings reports
  • New kinds of data, like new regions and sales types
  • New offerings like iAds, Newstand and iBooks

A lot of this data doesn’t come in a nicely formatted file; the only way to get it is by scraping a web page. If you’ve ever done this, you know how fragile it can be: a web developer that makes a simple change to a <div> can ruin your finely crafted parser.

We also all know that Apple never gives away details of its future plans. When things like new regions are added, you’re lucky to get a couple of days notice. And even when you do have some advance notice, like knowing that Mac apps will be added to the reports, you still don’t have any details until they show up on Apple’s server.

At which point, you have a huge problem: every customer needs a new version and they need it now. It doesn’t matter if it’s a holiday or you’re on vacation. You’re under the gun to write some new parsing code, get it tested, and deployed as soon as possible.

If you’re still thinking about how easy it would be to write your own app to track reports, let me share a little of my own experience. Much of the work in the new financial reporting and reconciliation module is based on an internal tool called BeanCounter. I thought this tool would be fairly easy to write: all I had to do was replicate the stuff I had in our Excel spreadsheets.

After six months, I had pretty much covered all the edge cases and weird report formatting issues, but still had to manually download the reports. In fact, this final hurdle of scraping web pages is what eventually led to the partnership between the Iconfactory and IdeaSwarm. So yeah, even developers with over 35 years of professional experience fall for the “this should be easy” naiveté.

Cloud Storage

I’ll be honest. I wasn’t wild about the idea of storing iTunes Connect credentials and our data on a remote server. We live in an online world where security breaches are just another piece of daily news. I knew it would be a lot of work to keep my data safe.

Unfortunately, there’s no API for iTunes Connect (even though we’ve been asking for years.) We’d love the chance to use something like OAuth instead of raw credentials. Unfortunately, that rapid pace of change I mentioned above pretty much precludes a stable API to access iTunes. As a pragmatic developer, you have to go with what you have, not what you want.

The reality for us and many other developers is that our product team keeps growing. Having sales and other financial data locked away on my laptop became more and more of a problem. The information coming from iTunes Connect helps run our business and I needed to share it with employees and partners. If you’re managing iTunes Connect information for some or all of your paying clients, you’ll have a similar problem.

Likewise, there are some kinds of data management that I don’t want to do. For example, adding events in the product details or reading reviews. Those tasks are better handled by a product manager and a person doing support. Again, the focus has shifted from the data itself to the people who manage it.

If you’re going to share data, you need to think long and hard about who’s going to have access and how it’s stored. In our case, there are two kinds of information that need to be protected: your iTunes Connect credentials and the report data that’s collected using those credentials.

We’re using 256-bit AES encryption for the iTunes Connect credentials. If you’re storing passwords with 1Password, you’re using the same encryption. We’re not going to divulge where the keys to decrypt this data are located, but we will say they’re not stored in source code or any other location on the server’s disk. They’re very hard for an attacker to access.

As far as our report data is concerned, we store it on Amazon S3 using Server Side Encryption (SSE). Access to this data is secured using SSL over a low latency connection to an Amazon server which uses AES-256 encryption. Amazon uses this same service for their own business-critical operations.

Passing data to the AppViz server before handing it off to the application on your Mac also has some big benefits for the user experience. Remember how frustrating it was when AppViz 2 couldn’t download your data because of some change that Apple had made on their site? With this new architecture, any changes to the data parser can be made directly on the server and minimize the downtime for all customers. There are monitors in place that let us know when reports aren’t importing correctly.

Finally, storing data in the cloud also allows us to offer new and exciting products. Personally, I’m dying to see things like:

  • The contents of the Dashboard module in an email as soon as the reports are ready to download.
  • Reports available on the web so that people don’t have to install an app to just read the basic information.
  • An iOS app that lets me keep an eye on the business while I’m on the go.

(Note: these are just ideas, not a promise that anything is going to get implemented!)

Private Storage

Still, with all of that said about cloud storage, there are still a lot of developers who are truly independent: one person with one set of reports. And these individuals love to be in complete control of their privacy and data. We want to keep these customers happy, too.

To do this, we’re planning on adding “private storage” to AppViz by the end of the year. This mode will offer the same resiliency to Apple’s changes since report collection and parsing remain on the server. The difference is that no reports or credentials will be stored after the data is exchanged. Now that our goal of implementing a robust download pipeline is accomplished, this secondary mechanism can be started.

When using private storage, iTunes Connect credentials will be passed to the AppViz server to initiate the connections at Apple. Data from those connections will then be collected and passed off to the application running on your Mac. After a successful transfer, the report data will be deleted from the server. No record of your credentials or data will be left behind.

Hopefully, you’ll agree that this is the best compromise between a constantly shifting data source and your own privacy.

Subscriptions

As a developer, you know that you spend a lot of time up front building a product and then amortize those costs over the years that it’s for sale. That initial hump can kill you, but long-term earnings make you do it over and over again :-)

Unfortunately, AppViz is not one of those products. It has huge ongoing costs based purely on maintenance. To give you an idea of the scale, there have been 32 releases for AppViz 2 over the past two years. That’s over one release per month and includes a lot of non-trivial work:

  • 67 new App Store countries
  • Financial regions going from 7 to 25
  • iAd support (with many releases to track changes on the web pages)
  • Report download changes (3 separate releases for changes in April 2012 alone)
  • New Newstand categories
  • New iBooks support
  • Rankings download changes
  • Rate limiting for rankings and reviews downloads
  • Adapting to changes on financial pages
  • New categories on iOS and Mac App Stores
  • Increased rankings from top 200 to 300

In the same time period, there was one release to support Mountain Lion, Gatekeeper and Retina displays.

The increase in the size of the development team is the best metric to show how maintenance has become such a huge cost. Initially there was one developer working on AppViz full-time: now there are three.

So, why don’t we just charge for more of the upgrades? AppViz doesn’t fit the “new features, paid upgrade” model. How would you feel if an upgrade fee was required to download and parse reports with a new region? Or if support was added for a new product category like Apple TV that you’re not using? My answer is that I’d feel like I was being held hostage: the app is broken until I upgrade or I’m forced to pay for something I don’t use.

For AppViz to be a viable venture in the future, the cost of the app must reflect the costs of this ongoing maintenance. In our view, subscriptions are the best fit.

Summary

There you have it, a couple thousand words to explain what used to be a huge “WTF?” Hopefully, this comprehensive essay shows that we’ve thought about the problem and have come up with a solution that’s both pragmatic and viable. If you still have questions or concerns, the folks at IdeaSwarm would love to hear from you.

It’s the Production Line, Stupid

“Apple Absolutely Has To Make An iPhone With A Bigger Screen”

(Warning: Business Insider link)

While that may be true, take a look at what happened at the end of last year: 47.8 Million iPhones Sold.

That’s a shitload of iPhones. But how many is a shitload?

47.8M iPhones
/ 90 days = 530K per day
/ 24 hours = 22K per hour
/ 60 minutes = 369 per hour
/ 60 seconds = 6 per second

That means that every second of every day you need six cases, six headphone jacks, six LCD screens, six batteries, six CPUs, six 10µf capacitors, six 10kΩ surface mount resistors, and thousands of other components.

Look at where all the leaks about new products come from: it’s always the supply chain that produces all these components. This should give you an idea of how many partners are involved.

Basically, Apple creates a pipeline that produces devices at an amazing rate. And it’s a finely tuned machine with a lot of inertia. You don’t just walk in and say, “Hey, let’s change the screen!” Doing so would throw that machine out of balance: a new screen means that some of the components (like the headphone jack) don’t change, while others would (the case, for example.)

A finely tuned production machine takes a lot of time and money to set up. Take a look at Apple’s capital expenditures. A mere $10B in 2012.

These capital expenditures show that Apple is planning ahead. At least 2-3 years, maybe more. You don’t just walk into Foxconn and ask for 50M iPhones. You need to predict the future.

Sure, it would be great to have a larger screen iPhone from what is known about current market conditions. But was a larger screen a sure thing 2-3 years ago? While you were falling in love with the new Retina display on your iPhone 4, I bet you weren’t thinking “Man, this display has got to be bigger!”

The cost to setup that production line is also a part of the final cost of the device. More frequent changes to the production line means that each device costs more or earns less profit for Apple.

As Apple continues to sell ever more devices, I see two things that could happen:

  1. A broader range of products (meaning more, smaller production lines to spread the demand)
  2. Increase the size of the production lines along with amount of time they stay online (meaning new devices are introduced every three years instead of two)

Given the highly competitive nature of the mobile space, I suspect we’ll see the first option come into play in the coming years. In the meantime, screens tick, while speed tocks: