Why Google knows our 'darkest secrets' not Facebook

John Herrman

Google, bruised as it has been by the rise of social media, still runs the most important utility on the internet.

I use Google search for fewer things than I used to - I rarely use it to find something to just read, for example - but it's still the best index we have of the world’s vast and scattered public network.

I feed it some words, it guesses what I want. It's a little dumb, but in a way I can work with. While our relationship has evolved over the years, its core pillars have held.

Simple search is destined to become a secondary feature, if not for one of Google's competitors than for Google itself. More and more of what it does for users will be ceded to social networks: people searching, news searching. Social networks have effortlessly built a more relevant searchable index of the internet's most recent history, leaving Google frozen out and scrambling awkwardly to catch up. But there are some needs social networks can never serve.

Twitter knows what you tell it, and a tiny bit more. Facebook presents a mile-high view of your online social history, which while unsettling in sheer size and scope, is selective and containable. The kinds of secrets it knows are secrets you're sharing, even if with just one person. They seem worse because the site reads them back to you endlessly.

It's Google, the social also-ran that knows your real secrets. It knows the things you wouldn't ask your friends. It knows things you can't ask your spouse. It knows the things you haven't asked your doctor yet. It knows things that you can't ask anyone else and that might not have been asked at all before Google existed. Google's servers are a repository of the developed world's darkest and most heartbreaking secrets, a vast closet lined with millions of digital skeletons that, should they escape, would spare nobody.

In 2006, Aol (then AOL) released a large database of search logs to researchers. The logs, which contained a month's worth of searches from about 650,000 users, were anonymised clumsily: each unique user was given a numeric tag that itself provided no information but tied all a user's searches together. This made it easy to identify at least some of them - imagine a search for "hot free porn" followed days later by a self-search - a fact made far more horrifying by the actual contents of the logs.

AOL issued a warning with the data:

Pornography is prevalent on the web and unfiltered search engine logs contain queries by users who are looking for pornographic material.

This was an understatement.

SomethingAwful users scoured the logs for the most shocking content back in 2006. The resulting posts are some of the most powerful documents on the internet. It's worth looking through all three, but here's the kind of stuff you'll find:

Some were downright horrifying:

Others were humiliating:

Some were more relatable funny:

Some were deeply sad:

Via: somethingawful.com

This one, cut for length, is probably the hardest to read of the bunch:

The fallout from the leak was immediate and harsh, at least internally. Multiple employees, including the CTO, were canned. The New York Times was able to identify one of the users, who said, "We all have a right to privacy. Nobody should have found this all out.”

Yet the event was widely written off as just another data leak, a result of lazy disregard for established rules. What it was, really, was a defining characterisation of search - what it is, what it's for, what the engines know.

It's tempting to say that these kinds of unsavvy searches are a relic from the days when AOL search was a going concern, and the general public didn't know better (for reference, AOL had just made its mail service free for the first time in history. It was a while ago). But I think that would be a mistake: if anything, the rise of social networks has reinforced, by force of contrast, the perception of the search box as a safe place.

This will be a significant part of what becomes of Google search. It's disturbing, and almost completely valueless to advertisers, anonymous to all but Google and the user, yet tied directly to your above-ground Google life - your Gmail, your Google+, your Google Drive - in a voluntarily created account. It's also, in its own sad, unmonetisable and sometimes beautiful way, incredibly important - a confessional booth, a digital back-alley doctor, an incidental diary, or a pillow to scream in.

Google is no longer organising the world's public information; that's doing an increasingly fine job of organising itself. Google is keeping the world's secrets.

*John Herrman writes for Buzz Feed.com.