Lame, Until it Isn’t

Where there’s smoke, there’s fire. And as we approach WWDC 2022, there’s a lot of smoke around AR and VR. In some ways, this is going to be a huge inflection point, in other ways, it’s probably going to be a letdown.

Remember when the iPod was announced? Some folks called it lame because it didn’t meet their expectations.

The same thing will be true of anything Apple wants us to put on our face. It’s going to less impressive technically than any of the currently shipping products. And that’s good, because you don’t make fundamental changes by tweaking existing technologies. A Nomad audio player was a tweak. An Oculus headset is a tweak.

Everything we’ve seen to date with VR has been an attempt to bring information to a 3D world. Headsets are just a means to project that 3D environment so our eyes can see it.

I think Apple’s approach with AR will be completely different: they will bring 3D to an information world.

We all have the greatest source of information humankind has ever known in our pocket or purse. Much of that information relates to the world around us: weather, transportation, shopping, dining, etc. Relating that data to our physical space will be a powerful tool.

All the AR examples we’ve seen on Apple’s devices hint at this direction. They take the information on our phone and place it at derived 3D coordinates. Where people get tripped up in these demos is where the results are shown: on a standard screen.

I don’t think that’s Apple’s final goal, because any current screen technology will block your view of the real world. It’s also why I think 3D headsets will remain a niche technology: people have innate need to see what’s going on around them.

Our current  screens also use a lot of power. And that means batteries. And that means weight. Not what I want on my face, for sure.

Apple knows this and that’s why I think a new display system is the thing they’re taking time to get right. We may or may not see this new display at WWDC. I can remember a time when all we had for an iPad was a simulator.

The changes caused by a new display will be incremental. There will certainly be technical limitations in the product that are imposed by size and weight: Apple will improve on those things as components allow.

Other changes will happen because no one, including Apple, really knows how this display will be used by normal folks (we, I should note, are not normal folks). The first Apple Watch tried to do a lot of things: iteration got rid of things no one used, and improved the things everyone wanted.

It’s likely that a first iteration will also be a “satellite device” where the iPhone does the heavy lifting. Much like the original iPod relied on a Mac. The realityOS could be nothing more than widgets for a new display on your face.

That will feel lame until you realize something else: after two decades, the basic form factor and functionality of that first iPod is now an essential part of our lives and we call it an iPhone. Don’t underestimate Apple’s ability to iterate.

Consistency Sin

This past summer we narrowly avoided a major user interface regression on Apple devices. The story ended well, but I think it’s important to look back on the situation and ask a simple question:

Why did this happen in the first place?

My answer is something I call “consistency sin”. Understanding the cause lets us avoid similar situations in the future.

Your first reaction to this nomenclature may be, “Isn’t consistency a good thing in user interfaces?”

Absolutely! Colors, fonts, and other assets should be similar within an app. Combined they help give the user a sense of place and act as a guide through an interface. And in many, cases these similarities should be maintained across platforms. There’s no sin there.

But you can get into trouble when this consistency starts to affect the user experience.

Design is not how it looks, it’s how it works.

Steve Jobs said a lot of smart things, but I use this advice most often.

The roots of consistency sin take hold when folks disregard the inherent differences between platforms. A greater importance is placed on making sure things match visually: how a person uses that design takes a back seat.

Platform controls and interface elements can differ at a fundamental level. The mouse is optimized for indirect interaction while a screen is optimized for direct interaction.

For the most part, developers are shielded from these details through the use of standard components that conform to the Human Interface Guidelines.

Enter Safari

Higher level interactions are driven by the type and quantity of information the user is working with.

To use Safari as an example, I can have hundreds of tabs open while I work on my Mac; on iOS it’s usually less than a dozen. Safari on iOS is also a full screen experience, while multiple windows and interactions with other applications are common on macOS.

Safari’s new tab design on iOS works great for me: swiping between tabs of fullscreen content is a better interaction for a limited number of pages. The grid of pages as a fallback for selection also works well for managing what I want to keep around.

The consistency sin in Safari was to come up with a good design for iOS and assume that it would also work well on iPadOS and macOS. It practice, these new tabs were difficult to use in a different work environment.

Luckily the folks working on Safari did the smartest thing possible: they listened to feedback and fixed the issues before shipping. That’s an important thing for a product that every Apple customer uses every day on every device.

It’s one thing to make a mistake, it’s a wholly different thing to deny that anything’s wrong. So let’s take a look at another example.

Notifications, Too

Notifications also suffer from consistency sin.

I look at my iPhone Lock Screen dozens of times each day, and sometimes just to just view a reminder or some other short notification. It’s quick, simple, and minimizes distractions.

On my Mac I see the Lock Screen only once or twice per day for just a few seconds as I enter a password. That means notifications occur while I’m actively working.

Again, consistency sin looks for a single solution that ignores my needs. On macOS I don’t want a minimal solution that is suitable for a mobile device. I want options that let me quickly dismiss or defer an item that’s interrupting my work. (And I certainly don’t want to hunt around in the window for a hidden control that lets me access a function.)

Placement is also an issue: on iOS controls tend toward the bottom of the screen (for reachability). The opposite is true on macOS where they tend  toward the top of the screen so they’re closer to the menu bar and window controls. Consistency sin says that notifications should always be at the top of the screen.

With iOS, there’s a nice visual and functional separation between app interactions in the lower half of the screen and notifications in the upper half. On the Mac, notifications are just another thing fighting for real estate at the top of the screen.

We’ve Only Just Begun

The bad news is that we’re likely to have more consistency sin in our future, thanks to Electron and other cross platform frameworks.

While development teams try to attain feature parity, experience parity will suffer. Everyone who’s used an iOS app and immediately thought “this is a web page” will know what that means.

There is a long history of user interface frameworks that make work easier for a product team. Every time, these solutions end up being a least common denominator that makes it more difficult for customers. Don’t be surprised when they complain: as they did for Safari, and as they do for Notifications.

If you’re a designer or developer, it’s your job to push back on the notion of consistency when it begins to affect a user’s experience. Remember design is how it works, and work is not the same on every device. 

jsc: My New Best Friend

A friend of mine recently pointed me at a well hidden command line tool. In the JavaScript framework used by Safari and other parts of Apple’s products, there is a tool called jsc. It’s a command line interface for JavaScript that uses the same code as the rest of the system.

You can find the binary at /System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Helpers/jsc. That path is unwieldy, so I have an alias set up that lets my just type jsc in the Terminal.

So what can you do with jsc? Pretty much anything you can do with JavaScript in a browser with the caveat that there aren’t document and window instances.

If you run jsc -h, you’ll see a lot of options for testing and profiling JavaScript. It’s clear that the WebKit team uses this internally for running tests. But we can also use it for trying out ideas and running simple utilities.

A picture is worth a thousand words, so let me show you how it can be used to solve a simple problem. I recently needed to convert some strings in our Turkish localization of Frenzic to uppercase: the lowercase “i” was getting converted to the dotless version.

JavaScript’s toLocaleUpperCase() function is the perfect way to do this, so I pulled jsc out of my tool bag and got to work. The first challenge was getting input.

Luckily, there is a readline() function that takes keyboard input and returns a value. Unluckily, that input isn’t in the encoding you’d expect it to be: characters are returned in ISO-8859-1 (Latin-1), not UTF-8. Remember, there’s no document instance so the default encoding is used.

To workaround this limitation, you can percent escape the characters to UTF-16 and then decode them back into UTF-8 with this technique:

var text = decodeURIComponent(escape(readline()));

(If any WebKit engineers are reading this, it would be nice to have a command line option like --encoding=utf-8.)

Generating output is a bit different than a browser, too. You’ll be using print() instead of console.log(). To convert the text input and display it, I used this:

print(text.toLocaleUpperCase('tr-TR'));

There are a few more built-in functions that may prove useful, but so far, I’ve only needed to read and write text. It’s undocumented, but jsc also takes standard input and can be used as a shebang:

$ echo "print(1+2);" | jsc
3

Since this is likely code I’ll have to use again, I created a Turkish.js file:

while (true) {
    print('Turkish text?');
    var text = decodeURIComponent(escape(readline()));
    print(text.toLocaleUpperCase('tr-TR'));
    print('-------------');
}

I can now run this any time with jsc Turkish.js. And you also get to see how having JavaScript in a command line can be handy. Enjoy!

Codesign: The Saga Continues

I have a long history of writing about code signing in macOS. When Big Sur was released, I thought “Finally!”

I was wrong.

This time around I was tripped up by Safari, of all things. It doesn’t open app archives like other parts of macOS.

This story began with customer reports of xScope being a “damaged app” on Big Sur. This was surprising because I had been downloading and testing the app on Big Sur for several months without issue.

I was also doing all this work on Apple Silicon using the DTK. And since Google Chrome wasn’t yet working on this device, all my testing was limited to Safari. Safari’s default setting is to open “safe” files after download, so I left that alone (as most customers would).

This is where I shot myself in the foot. At no point did my downloads touch the Archive Utility. And I had no idea that Safari’s code is different than the system utility.

When I checked the signature of the app downloaded with Safari, everything looks good:

$ codesign -vvvv ~/Downloads/xScope.app
...
/Users/CHOCK/Downloads/xScope.app: valid on disk
/Users/CHOCK/Downloads/xScope.app: satisfies its Designated Requirement

Things were very different when using Google Chrome:

$ codesign -vvvv ~/Downloads/xScope.app
/Users/CHOCK/Downloads/xScope.app: unsealed contents present in the root directory of an embedded framework
In subcomponent: /Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework

$ codesign -vvvv ~/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework
/Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework: a sealed resource is missing or invalid
file added: /Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework/Versions/Current/Resources/._fr_CA.lproj
file added: /Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework/Versions/Current/Resources/Updater.app/Contents/Resources/._fr_CA.lproj
file added: /Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework/Versions/Current/Resources/Updater.app/Contents/Resources/._pt.lproj
file added: /Users/CHOCK/Downloads/xScope.app/Contents/Frameworks/Sparkle.framework/Versions/Current/Resources/._pt.lproj

It turns out all these folks complaining about a “damaged app” were either using Chrome or had Safari’s “safe” file handling turned off. The damaged archive wasn’t getting repaired automatically by Safari.

The root of the problem is localization in the Sparkle framework. There are two symlinks with extended attributes (the “._” is where macOS stores things like Finder information). The intent of the symlink was to say that French Canadian is the same as French, and Portuguese is the same as Brazilian Portuguese.

Since macOS automatically makes this inference, it’s safe to just delete the scripts that create the symlinks. In your Sparkle project, find any occurrences of “Run Script: Link fr_CA to fr” and “Run Script: Link pt to pt_BR” in your Target Build Phases and remove them. I had them in “Sparkle”, “SparkleCore”, and “Installer Progress”.

After you build and notarize, you’ll see that your app is “valid on disk” no matter how it’s unarchived.

I’ve also submitted this information to Apple’s Product Security group. As I said in my email, the biggest problem here is expectations:

The reason I’m writing is because Safari’s implementation for opening “safe” files is somehow bypassing a code signing check or repairing the downloaded package. The Archive Utility does not. Customer and developer expectations for unzipping archives is that they are not modified and behave the same way across all Apple products.

If you’re a Mac developer who’s using Sparkle and distributing your product via a web download, now’s a good time to check how things work in a variety of browsers. I’ve heard that we’re not the only ones affected.

And if you encounter a download that’s damaged because of these Sparkle symlinks, this quick fix in the Terminal will set things right:

$ ditto xScope.app xScopeFixed.app
$ rm -rf xScope.app
$ mv xScopeFixed.app xScope.app

The ditto command strips the extended attributes that are causing the issue. This may, in fact, be what Safari is doing for “safe” files.

All that’s left to do now is wonder what surprises codesign has in store for next year’s release of macOS…

Triode and Internet Radio

Today, the Iconfactory is happy to introduce a product that’s made with three different apps: Triode. Simply put, it’s the best way to enjoy all of your favorite Internet radio stations, wherever you go.

So why did I make these apps?

A Brief History of Internet Radio

Before I can answer this question, it helps to know a bit about where I live. Laguna Beach is surrounded on three sides by a coastal range that blocks any broadcast from the Los Angeles and San Diego metro areas. A signal’s only clear shot is from the Pacific Ocean, where there are no radio towers.

I moved to Laguna in the mid-1990’s — as you probably know, that was a time when the Internet was starting to take off.

Progressive and independent radio stations immediately saw the Internet as way to extend their reach: WXYC was the first station to broadcast a stream in 1994. Many others soon followed.

I saw it as a way to overcome the physical limitations of where I lived. And in the process, I discovered a lot of interesting sounds from all over the world.

In short, Internet radio is nothing new and I’ve been listening to it since the late 1990’s.

Old Meets New

What is new are the array of devices we can use to listen to radio streams. In the beginning, my only option was a desktop computer, but now I also carry a powerful device in my pocket and have another one attached to my TV.

These devices also have advanced capabilities like multi-room audio with AirPlay 2, a CarPlay interface while driving, automation with Siri shortcuts, and iCloud data syncing. They’re also highly accessible thanks to VoiceOver.

Yet many radio stations provide free apps that are cross-platform and can’t take advantage of the features that Apple provides its developers. In fact, many of these apps are basically a view that use the same JavaScript-based player on the station’s website.

I saw an opportunity to do something better.

Always With Me

Early in 2018, Triode got its start on tvOS as a way to listen to music while relaxing on the couch. I loved that so much, I then wanted it while working on my Mac. Then, while out and about with my iPhone and AirPods. A simple idea was blossoming.

The challenge quickly became creating a unified experience across three different devices and input mechanisms. Building a user interface that worked natively with a TV remote, a mouse, and your finger was a challenge!

Overcoming this hurdle had a real benefit: it’s pretty awesome to find a great station on your Mac while you work, and have it automatically show up on your iPhone during a commute, or on your TV as you relax in the evening.

Early in the development process it became apparent that great artwork was an important part of the experience. Seeing a beautiful new album cover has always been essential part of discovering an artist or track. It became a central element of our unification effort.

This effort also helped refine my thoughts on the future of interaction on Apple’s platforms. And, as it turns out, SwiftUI.

A Place of Discovery

With the help of my colleagues, the user interface and interaction started shaping up. The task then became how to introduce folks to Internet radio.

Most existing players have another flaw: they promote quantity over quality. It sounds impressive to tout tens of thousands of stations — until you realize that a lot of them aren’t very good. Clearly, some curation was needed.

Over the years, I had accumulated a collection of favorite stations. This formed the core of the “Our Picks” in Triode.

While thinking about how to extend this group of stations, it became clear that they all had one thing in common: they were independent and listener-supported.

These stations have have long been a place to discover new music. They’re often associated with a college or university that has a student body looking for the latest and greatest. The lack of commercial pressures also provides an environment where new acts can get their first bread and established artists can do something unexpected.

Every major artist used to be someone you had never heard of, and Internet radio played a major part in that transformation.

That spirit guided our curation of “Our Picks”, while “Find a Station” provides quick access to thousands of stations from around the world. Both will become “Your Favorites” and automatically get synced to all your devices.

Privacy First Design

Last, but certainly not least, Triode was designed with your privacy in mind.

Unfortunately, today’s Internet is full of trackers. If you’re using a web-based stream or app, its likely that your listening habits and other metrics are being collected.

In Triode, the only information provided to broadcasters is your IP address. And since we don’t display any ads in the app, that’s one less thing for marketeers to know about you.

And More

The Triode website has more details and this review at MacStories goes into great detail about how the app looks and works on the different platforms.

It’s been a blast making these apps — I hope you enjoy them!