That time I demoed a porn filter for Apple and it didn’t go well

So this would have been in the fall of 2006 while working on Mac OS X Leopard (10.5). It was during a time when Apple was spending two years each on Mac OS X upgrades¹ instead of the normal one we’re now used to. That meant bigger features with more breadth, so even though we had twice the time to get things done, it felt like more work.

Marketing really wanted some better parental controls² for the Mac, since up till then we had just had some basic controls shoehorned into Accounts. The list of new parental controls features marketing wanted was pretty extensive for a single release:

• App restrictions
• Website restrictions
• Automatic content filtering
• Email permission requests
• Email & Chat limits
• Activity logs
• Time restrictions

I think this was the first major tentpole feature Apple had thrown at me since taking the SWE job a few years prior. Before that the biggest thing I had worked on was probably the Sync part of the .Mac pref pane or maybe the Mosaic screen saver (whose untimely death I still shed a tear for).

I still remember my first interaction with the PM on how robust the parental controls needed to be. He thought as long as it wasn’t hackable by most kids then it would be fine. I thought if even one kid in a school figured out how to bypass it, then they’d share the hack with all the kids. I should have kept my mouth shut.

We had a team of me as DRI³, two newer engineers, and a QA. I codenamed the project “Poppins” (clever, huh?). We took over an office nearby and taped up the windows so passersby wouldn’t get an eyeful.

Of all the features, the content filter was the most daunting. At this point I didn’t know how to do almost any of it. I’d never worked with machine learning, my networking skills were meager, and I knew next to nothing about Python, which we’d use to wrangle the vast amounts of training and testing data into usable form.

“Content filter” of course was basically a euphemism for porn filter, although it did (and still does) include categories for hate, gambling, substance abuse, and violence. Apple made me sign a legal contract stating that if I sustained emotional or mental damage during the course of looking at all this porn, I wouldn’t sue them. I still have it! (The contract, not the emotional damage…)

And the threat was real. Thankfully fellow engineer Giovanni Donelli had a great idea: let’s get ahold of the Safari source code and write our own browser that never shows images. This made working much less awkward, especially considering our QA was a young woman and we were constantly stuck in that office together.

My first big decision was whether to try to write all of this from scratch, or seek out a deal with a third-party. At the time, Apple Engineering was often criticized for having “NIH Syndrome”, or “not invented here syndrome”, meaning that if it wasn’t made at Apple then it’s just not good enough, which is obviously short-sighted.

I took that to heart as I played with the third-party solutions already on the market. Most were pretty much garbage — either they didn’t work or they were easily bypassable — but one did stand out as promising. I managed to get the owner on the phone and we talked, deciding it might be a good fit.

They gave me the source code under NDA and I forked a version for our purposes. After some tuning on our test/training material, it seemed to work pretty well! I was optimistic — the most daunting part of the project might just turn out to be the easiest. As we polished and fixed more bugs, the feature was eventually deemed ready for an “HI demo”.

“HI demos” are hard to explain. HI of course refers to the Human Interface group at Apple, but at that time these demos always included Kevin Tiene, director of Mac engineering, Greg Christie, director of HI, Scott Forstall, VP of Platform Engineering, Bertrand Serlet (sometimes), as well as all the EPMs, managers, and designers (a young Mike Matas was the HI designer for parental controls).

The demos would take place in a large open studio hallway space in a wing of IL3. The entire wing was then on secure lockdown because of some secret project that the rest of us weren’t supposed to know about. That project of course was iPhone, then known as “Purple”. (Every time there was a meeting or demo there we had to wait forever while they cleared out all the incriminating whiteboards and such from the open area.)

At this particular demo, there were about 40 people in attendance and I was to show off for the first time how the new parental controls worked, including the content filter. Because the hall was so large with so many people, the demo Mac was projected onto a large wall at the end that was probably 16 feet tall.

I worked through my script and everything went well. The content filter part of course was a little awkward, but I tried to load “penthouse.com” in Safari and the content filter successfully blocked it. Whew! The Demo Gods are with me!

But then suddenly Scott Forstall comes over and shoos me out of my chair. He looks around the room and I still remember his exact words: “OK everyone, I’m serious. Is anyone going to be offended if something shows up on the screen? I’m totally serious about this. Anyone? Anyone? I’m serious.” We all knew exactly what he meant but no one budged or said anything.

I was now freaking out a little. This isn’t part of the script! And what the hell did he even plan to do? The system worked by refreshing a HUGE blocklist of porn sites every few days. Did Scott Forstall know of such niche porn sites that they wouldn’t be on the blocklist?

Scott went to Safari and typed into the address bar: google.com. Embarrassingly, I still had no idea what he was up to. Then he went to Google preferences, then Safe Search. My heart started to sink. One more time from Scott, “I’m serious y’all, is everyone OK with this?” The hall was quiet. He switched Safe Search to Off.

The problem of course was that the software used a blocklist of bad domains, and obviously google.com was not on any blocklist! Scott googled “porn” and within seconds, projected onto a 16-foot wall in front of 40 people, was a large grid of hardcore porn. He left it up for a beat longer than I thought was necessary as if to make a point, then closed the window.

There was a little bit of chuckling but nothing was said. That was the end of the demo.

Over the next six months I wrote the content filter from scratch, if you can believe it, as an Apache module (because Apache web servers shipped on every Mac at that time) configured as a reverse proxy. And instead of massive blocklists of bad sites, we used this new technology called “machine learning”. It actually worked pretty well!⁴

As a final coda to my experience bringing parental controls to Mac OS X Leopard, it turns out that the content filter wasn’t actually the hardest part. During a demo to Steve Jobs that I wasn’t present for, 3 weeks before the WWDC announcement and a mere 3 months before ship, SJ casually remarked, “Oh, and I don’t want to have to walk upstairs to Reed’s bedroom every time I want to change something.” So the word came down: oh, by the way, all of this needs to work remotely. That was the hardest part.