The Forensic Shit Show

It turns out someone at the FBI advised another law enforcement officer in San Bernardino to reset the iPhone that the government wants Apple to unlock.

This is just another episode in a complete forensic shit show.

Remember, this is the same case where the media was allowed to roam freely through a crime scene. One of the photos in that gallery shows a computer without an Ethernet connection on the wall (the age of the apartment also suggests that there would be no wired Internet.)

What are the chances that there was a wireless network in that apartment? What are the chances that there are IP logs on that router? Or maybe some kind of data backed up to a disk on the router? Here’s another wild guess: maybe that router was used to connect to an online backup service.

Yep, someone did the equivalent of a “restore factory defaults” on a device under active investigation.

What we’re seeing here is law enforcement’s complete lack of understanding of how digital devices store and transmit data. This new evidence is much more intricate than smoking guns or blood splatters. The important stuff is what you don’t see: it’s a hard problem where the people dealing with it are untrained. Shit, I work in this business and trying to decipher what’s going on makes my head spin.

Yet law enforcement is asking Apple to not only provide data, but also to create a forensic instrument that allows them to extract information from any device. And by its very nature, this tool would be made widely available throughout the forensic and law enforcement community.

Basically, the government is asking Apple to hand over a golden key that can defeat the security of any device to folks that can’t even secure a wireless network. Worse, this whole process is being overseen by politicians that think the problem is predators getting access to their grandkid’s Playstation.

This is why the entire tech community is saying “No fucking way.”

Updated February 21st, 2016: Several people have commented about my use of “restore factory defaults” in the post above. My intention was figurative, not literal.

The folks involved with the investigation were pressing buttons without understanding the consequences of their actions. To me, it feels like a “reboot to fix” approach. The password reset did not damage any data, it just made automatic backups stop working because iCloud information on the device needed to be updated, and that can’t be done without a passcode.

Others have reminded me that the FBI had cleared the crime scene. That’s true, but since the Wi-Fi equipment was not collected as evidence, it still shows that the investigators were out of their league. In an electronic investigation, a router is a key piece of the puzzle.

Both of these things are details in a bigger picture: the FBI wants to hold the private keys to a public key encryption system that affects the privacy of hundreds of millions people. If they can’t get the details of an online backup service right, how the hell do we expect them to guard a back door?

There’s also a possibility that the iCloud password reset was intentional. If this is the case, we have a government that is extorting Apple by essentially planting evidence. Imagine what they could do with a private key.


Every Mac developer that uses iCloud has a dirty little secret:

They don’t fully test their software before they ship it to customers on the Mac App Store. It’s because Apple won’t let them.

iOS developers, on the other hand, can upload a build to TestFlight and use the app with the iCloud production servers to make sure everything is working great before it gets sent to the App Store for review.

TestFlight has been available to internal developers since iOS 8 was announced in 2014. The system was opened up to external testers who have an iTunes account in the early part of 2015.

Mac developers have never had access to TestFlight, either internally or externally. It’s “coming soon”, and until that day comes, there’s no way to test apps that use the iCloud servers. Which sucks for both the developer and the customer.

But wait, there’s more.

Apple is touting analytics as an awesome new feature for developers that use the App Store to distribute their creations. It’s a huge benefit to our businesses, but only when you’re selling solely on iOS. This feature is nowhere to be found on the Mac App Store. Again, it’s “coming soon”.

Just yesterday, Apple did something great for developers. They now block reviews on beta OS releases. Unless that operating system is for the Mac.

Let me guess: it’s “coming soon”.

It doesn’t take a genius to see that Apple is doing something it rarely does: a half-assed job.

As developers, we completely understand how much work it is to announce these kinds of initiatives and get them working on multiple platforms. It’s not easy and takes a lot of resources. But it’s clear that these precious resources are not being allocated.

Apple needs to change its priorities for the Mac App Store or just shut the whole thing down. As it now stands, developers who are tired of being second-class citizens are making that decision for them and leaving on their own.

This is a pity because the Mac App Store is a great way for customers to download and purchase software. No one benefits from this half-assed job.

Updated July 23rd, 2015: I think the thing that bothers me most about this situation is the inequality. Mac developers aren’t getting the same value from the App Store as their counterparts on iOS. We all pay Apple 30% of our earnings to reach our customers, we should all get the same functionality for that fee.

Dupe this Radar if you agree.

discoveryd Clusterfuck

I usually keep things fairly clean on this site. I have a simple metric: would I be embarrassed if my Mom read this post? As you’ve probably guessed from the title, this post is going to be different.

So, Mom, it’s time to stop reading. I’m pissed off and you know how I get when that happens.

In case you’re wondering what I’m talking about, look at this shit. A network process using 100% of the CPU, WiFi disconnecting at random times, and names, names (1), names (2), names (4). All caused by a crappy piece of software called discoveryd.

I started reporting these issues early in the Yosemite beta release and provided tons of documentation to Apple engineering. It was frustrating to have a Mac that lost its network connection every few days because the network interfaces were disabled while waking from sleep (and there was no way to disable this new “feature”.)

Regardless of the many issues people were reporting with discoveryd, Apple went ahead and released it anyway. As a result, this piece of software is responsible for a large portion of the thousand cuts. Personally, I’ve wasted many hours just trying to keep my devices talking to each other. Macs that used to go months between restarts were being rebooted weekly. The situation is so bad that I actually feel good when I can just kill discoveryd and toggle the network interface to get back to work.

Only good thing that’s come of this whole situation is that we now have more empathy for the bullshit that folks using Windows have suffered with for years. It’s too bad that Apple only uses place names from California, because OS X Redmond would be a nice homage.

It’s no secret in the tech community that discoveryd is the root cause of so many problems. There are even crazy workarounds. With so many issues, you’d expect some information from Apple explaining ways to mitigate the problems.


The only explanation I can come up with for this astounding lack of information is that there’s some mid-level product manager at Apple who’s covering their ass. I hope this person who’s responsible for withholding advice feels good about themselves, because the rest of us hate them with the burning passion of a thousand suns. Being stingy with knowledge in an engineering organization is a fucking stupid career move.

To give you an idea of how helpful a tiny piece of information is towards people’s productivity, let me give you a simple example that’s already saved me hours of frustration.

For months, I’ve seen bullshit like this in Bonjour:


That shows the xScope service on the Mac that provides data for the Mirror on iOS. In that screenshot, the service is being shown as available on three devices: one with just an IPV6 address, one with no IP addresses, and one with a duplicate IPv6 address and a valid IPv4 address. The name “CedarX” was the only way I could find to prevent names from incrementing (and breaking things that use the host name of that device.)

The “funny” thing is that this Mac is running the latest version of 10.10 with fixes for “WiFi issues”. And after tweeting about it in frustration, I got this response:

I followed Hendrik’s advice and guess what? No more network issues.

Bonjour keeps a cache that’s shared amongst devices on the network. This is so that if the device is asleep, another one that’s awake can provide the necessary information. I suspect that a device running an older version of discoveryd poisoned this cache. For some reason, the invalid cache information couldn’t be corrected by a newer version of the software which screwed things up in the first place.

But this is all just conjecture because Apple hasn’t written that fucking tech note.

This situation also shows another important aspect of the discoveryd clusterfuck: this code is all over the place. It’s in use by iOS, OS X and presumably whatever is running on the Apple Watch. As such, any one of those devices can poison Bonjour for everything else on your network.

This workaround is fairly simple if you’re on a home network where you have direct physical access to the all the devices. But as we all know, wireless networking is essential in places like an office, an airport or a coffee shop. Good luck rebooting everything in that kind of environment. And what happens when someone running an older version of OS X connects to that network and poisons it? Time to reboot!

You also can’t rely on software updates to fix everything: I have both an Airport Express and Apple TV that are no longer receiving fixes. Having to buy new hardware because of crappy software adds insult to injury.

Ironically, these issues are most likely to affect Apple’s best customers. The more devices you have, and the longer you have them, the more likely you are to get an unstable network. The only advice I can offer is to restart your entire network.


China Attacks

Robert Graham built a tool to analyze the traffic causing the DDoS attack on GitHub. He then used it to prove that the machine is “located on or near the Great Firewall of China”.

One has to wonder when politicians will take these virtual attacks as seriously as physical ones. As Robert points out, this attack is against a key part of the United States’ Internet infrastructure. What would we do if China took out all the Interstate highways leading into California?