Lame, Until it Isn’t

Where there’s smoke, there’s fire. And as we approach WWDC 2022, there’s a lot of smoke around AR and VR. In some ways, this is going to be a huge inflection point, in other ways, it’s probably going to be a letdown.

Remember when the iPod was announced? Some folks called it lame because it didn’t meet their expectations.

The same thing will be true of anything Apple wants us to put on our face. It’s going to less impressive technically than any of the currently shipping products. And that’s good, because you don’t make fundamental changes by tweaking existing technologies. A Nomad audio player was a tweak. An Oculus headset is a tweak.

Everything we’ve seen to date with VR has been an attempt to bring information to a 3D world. Headsets are just a means to project that 3D environment so our eyes can see it.

I think Apple’s approach with AR will be completely different: they will bring 3D to an information world.

We all have the greatest source of information humankind has ever known in our pocket or purse. Much of that information relates to the world around us: weather, transportation, shopping, dining, etc. Relating that data to our physical space will be a powerful tool.

All the AR examples we’ve seen on Apple’s devices hint at this direction. They take the information on our phone and place it at derived 3D coordinates. Where people get tripped up in these demos is where the results are shown: on a standard screen.

I don’t think that’s Apple’s final goal, because any current screen technology will block your view of the real world. It’s also why I think 3D headsets will remain a niche technology: people have innate need to see what’s going on around them.

Our current  screens also use a lot of power. And that means batteries. And that means weight. Not what I want on my face, for sure.

Apple knows this and that’s why I think a new display system is the thing they’re taking time to get right. We may or may not see this new display at WWDC. I can remember a time when all we had for an iPad was a simulator.

The changes caused by a new display will be incremental. There will certainly be technical limitations in the product that are imposed by size and weight: Apple will improve on those things as components allow.

Other changes will happen because no one, including Apple, really knows how this display will be used by normal folks (we, I should note, are not normal folks). The first Apple Watch tried to do a lot of things: iteration got rid of things no one used, and improved the things everyone wanted.

It’s likely that a first iteration will also be a “satellite device” where the iPhone does the heavy lifting. Much like the original iPod relied on a Mac. The realityOS could be nothing more than widgets for a new display on your face.

That will feel lame until you realize something else: after two decades, the basic form factor and functionality of that first iPod is now an essential part of our lives and we call it an iPhone. Don’t underestimate Apple’s ability to iterate.

Written on June 3rd, 2022 for Design, Development, Observation, Opinion

Consistency Sin

This past summer we narrowly avoided a major user interface regression on Apple devices. The story ended well, but I think it’s important to look back on the situation and ask a simple question:

Why did this happen in the first place?

My answer is something I call “consistency sin”. Understanding the cause lets us avoid similar situations in the future.

Your first reaction to this nomenclature may be, “Isn’t consistency a good thing in user interfaces?”

Absolutely! Colors, fonts, and other assets should be similar within an app. Combined they help give the user a sense of place and act as a guide through an interface. And in many, cases these similarities should be maintained across platforms. There’s no sin there.

But you can get into trouble when this consistency starts to affect the user experience.

Design is not how it looks, it’s how it works.

Steve Jobs said a lot of smart things, but I use this advice most often.

The roots of consistency sin take hold when folks disregard the inherent differences between platforms. A greater importance is placed on making sure things match visually: how a person uses that design takes a back seat.

Platform controls and interface elements can differ at a fundamental level. The mouse is optimized for indirect interaction while a screen is optimized for direct interaction.

For the most part, developers are shielded from these details through the use of standard components that conform to the Human Interface Guidelines.

Enter Safari

Higher level interactions are driven by the type and quantity of information the user is working with.

To use Safari as an example, I can have hundreds of tabs open while I work on my Mac; on iOS it’s usually less than a dozen. Safari on iOS is also a full screen experience, while multiple windows and interactions with other applications are common on macOS.

Safari’s new tab design on iOS works great for me: swiping between tabs of fullscreen content is a better interaction for a limited number of pages. The grid of pages as a fallback for selection also works well for managing what I want to keep around.

The consistency sin in Safari was to come up with a good design for iOS and assume that it would also work well on iPadOS and macOS. It practice, these new tabs were difficult to use in a different work environment.

Luckily the folks working on Safari did the smartest thing possible: they listened to feedback and fixed the issues before shipping. That’s an important thing for a product that every Apple customer uses every day on every device.

It’s one thing to make a mistake, it’s a wholly different thing to deny that anything’s wrong. So let’s take a look at another example.

Notifications, Too

Notifications also suffer from consistency sin.

I look at my iPhone Lock Screen dozens of times each day, and sometimes just to just view a reminder or some other short notification. It’s quick, simple, and minimizes distractions.

On my Mac I see the Lock Screen only once or twice per day for just a few seconds as I enter a password. That means notifications occur while I’m actively working.

Again, consistency sin looks for a single solution that ignores my needs. On macOS I don’t want a minimal solution that is suitable for a mobile device. I want options that let me quickly dismiss or defer an item that’s interrupting my work. (And I certainly don’t want to hunt around in the window for a hidden control that lets me access a function.)

Placement is also an issue: on iOS controls tend toward the bottom of the screen (for reachability). The opposite is true on macOS where they tend  toward the top of the screen so they’re closer to the menu bar and window controls. Consistency sin says that notifications should always be at the top of the screen.

With iOS, there’s a nice visual and functional separation between app interactions in the lower half of the screen and notifications in the upper half. On the Mac, notifications are just another thing fighting for real estate at the top of the screen.

We’ve Only Just Begun

The bad news is that we’re likely to have more consistency sin in our future, thanks to Electron and other cross platform frameworks.

While development teams try to attain feature parity, experience parity will suffer. Everyone who’s used an iOS app and immediately thought “this is a web page” will know what that means.

There is a long history of user interface frameworks that make work easier for a product team. Every time, these solutions end up being a least common denominator that makes it more difficult for customers. Don’t be surprised when they complain: as they did for Safari, and as they do for Notifications.

If you’re a designer or developer, it’s your job to push back on the notion of consistency when it begins to affect a user’s experience. Remember design is how it works, and work is not the same on every device. 

Written on January 11th, 2022 for Design, Development

Introducing AV Remote

In my ongoing quest to release FREE apps that are useful in my daily life, I’m proud to announce the release of AV Remote. It’s a very simple remote for Denon and Marantz AV receivers.

The official apps for controlling these receivers work fine, but have way too many features for my needs. I don’t need to control zones, select inputs, or adjust equalizer presets. I just want to control the power and volume.

While using early versions of the app, I realized that I have different “listening levels”. As a result, AV Remote makes it easy to select a quiet volume for background music or something much louder for watching a movie:

AV Remote works with recent models of both Denon and Marantz receivers. If your device supports HEOS, then the app can discover the receiver on your network and begin controlling it. There’s no setup other than allowing access to your local network with Bonjour.

And if you’re looking for some good background music to put on your Apple TV, I have a solution for that too!

Written on August 30th, 2021 for Miscellaneous

jsc: My New Best Friend

A friend of mine recently pointed me at a well hidden command line tool. In the JavaScript framework used by Safari and other parts of Apple’s products, there is a tool called jsc. It’s a command line interface for JavaScript that uses the same code as the rest of the system.

You can find the binary at /System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Helpers/jsc. That path is unwieldy, so I have an alias set up that lets my just type jsc in the Terminal.

So what can you do with jsc? Pretty much anything you can do with JavaScript in a browser with the caveat that there aren’t document and window instances.

If you run jsc -h, you’ll see a lot of options for testing and profiling JavaScript. It’s clear that the WebKit team uses this internally for running tests. But we can also use it for trying out ideas and running simple utilities.

A picture is worth a thousand words, so let me show you how it can be used to solve a simple problem. I recently needed to convert some strings in our Turkish localization of Frenzic to uppercase: the lowercase “i” was getting converted to the dotless version.

JavaScript’s toLocaleUpperCase() function is the perfect way to do this, so I pulled jsc out of my tool bag and got to work. The first challenge was getting input.

Luckily, there is a readline() function that takes keyboard input and returns a value. Unluckily, that input isn’t in the encoding you’d expect it to be: characters are returned in ISO-8859-1 (Latin-1), not UTF-8. Remember, there’s no document instance so the default encoding is used.

To workaround this limitation, you can percent escape the characters to UTF-16 and then decode them back into UTF-8 with this technique:

var text = decodeURIComponent(escape(readline()));

(If any WebKit engineers are reading this, it would be nice to have a command line option like --encoding=utf-8.)

Generating output is a bit different than a browser, too. You’ll be using print() instead of console.log(). To convert the text input and display it, I used this:

print(text.toLocaleUpperCase('tr-TR'));

There are a few more built-in functions that may prove useful, but so far, I’ve only needed to read and write text. It’s undocumented, but jsc also takes standard input and can be used as a shebang:

$ echo "print(1+2);" | jsc
3

Since this is likely code I’ll have to use again, I created a Turkish.js file:

while (true) {
    print('Turkish text?');
    var text = decodeURIComponent(escape(readline()));
    print(text.toLocaleUpperCase('tr-TR'));
    print('-------------');
}

I can now run this any time with jsc Turkish.js. And you also get to see how having JavaScript in a command line can be handy. Enjoy!

Written on August 25th, 2021 for Development

Introducing Waterscope

It’s common wisdom that you should release a software product when it is minimally viable: get an early version out in the world as soon as it can perform a useful task for a customer.

When that product is for someone who is a developer that’s been coding since the dawn of time, the equation gets flipped on its head. Waterscope is a Maximally Viable Product™ and the customer is me.

The app got its start when Swift 1.0 was announced back in 2014: I wanted to build something with the new language. About that same time, I had also started learning about tides and how they are predicted. It’s a complex problem that has been vexing scientists since the three-body problem was first proposed by Newton with the publication of the Principia in 1687. Like determining the time and place for a lunar eclipse, we rely on derived approximations.

For learning a new language, tide prediction provided a lot of interesting work: data collection, complex calculations, graphical presentation, and animatable data. It also let me know when it was a good time for a dog walk.

As an ocean swimmer, I also wanted my weather app to provide information about water conditions. It turns out the scientists at the National Oceanic and Atmospheric Administration (NOAA) have that all figured out. As do the meteorologists at the National Weather Service (NWS) with their API for weather observations and forecasts. There are even high resolution images from environmental satellites launched by NASA. The United States government provides a treasure trove of data; the challenge with Waterscope was to organize and present it in a consistent manner.

Screenshots of Waterscope showing the home screen graphs, a map view with sea and land weather conditions, and a satellite view of California.

Which leads to a secondary goal for this app: to make it completely by myself. I work with some incredibly talented designers, but I wanted Waterscope to be uniquely my own. All the design, for better or worse, was created by my own hand. (The only exception is the use of SF Symbols when showing weather conditions.)

So not only was I learning Swift, I was also learning Sketch and, of course, how much time it takes to get something to feel right. Like coding, it’s not as easy as we sometimes make it look. Don’t take your designers for granted :-)

Along the way, there were some interesting hurdles. Some visual, some simple, and some complex. Many of the things I wanted to do required learning about astronomy and orbital mechanics. My sketches from Linea will give you an idea of the breadth of the challenges. (And being the day after the solstice, that first sketch is particularly relevant.)

Sketches for a user interface design that shows seasons using the illumination of the Earth, the math for linear interpolation, and a spherical projection of a point on a satellite image.
Sketches for a user interface that shows seasons by illuminating the Earth, the math for linear interpolation, and a spherical projection for a location on a satellite image.

Here I am, six years later. I’ve learned a lot, but as with my first app in Objective-C, the most important thing about this exercise was how not to use Swift. It will probably take me another 18 years to come to terms with this new language, and feel like I’ve mastered it, but a journey can’t start without the first steps. Another insight is that a programming language is just a means to an end: the hard part is not the code, it’s understanding what needs to be done.

I’m releasing Waterscope today because there are certainly other folks who will benefit from my personal weather app. There may even be some educational value in seeing how I approached a data-rich user interface (hint: Edward Tufte’s books taught me). Information can be dynamic and beautiful.

At the same time, if you’re outside the U.S., it’s unlikely to be a satisfying experience: most of the data sources and their presentation are oriented towards North America. An example: in the southern hemisphere your view of the sun and moon’s orbit is in a counterclockwise direction as you look north. Waterscope displays a clockwise orbit.

But the good news is that Waterscope, like the data it uses, is FREE to download and use. Enjoy!

Written on December 22nd, 2020 for Miscellaneous
Newer
Older