Speaking of data – Cory Doctorow (of BoingBoing) has a neat piece of dystopian fiction up at Radar magazine – called “Scroogled”. The subtitle:

Google controls your e-mail, your videos, your calendar, your searches… What if it controlled your life?

A few quotes:

“It started in China,” she went on, finally. “Once we moved our servers onto the mainland, they went under Chinese jurisdiction.”

Greg sighed. He knew Google’s reach all too well: Every time you visited a page with Google ads on it, or used Google maps or Google mail – even if you sent mail to a Gmail account – the company diligently collected your info. Recently, the site’s search-optimization software had begun using the data to tailor Web searches to individual users. It proved to be a revolutionary tool for advertisers. An authoritarian government would have other purposes in mind.

“They were using us to build profiles of people,” she went on. “When they had someone they wanted to arrest, they’d come to us and find a reason to bust them. There’s hardly anything you can do on the Net that isn’t illegal in China.”

Greg shook his head. “Why did they have to put the servers in China?”

and

“We’re drafting a team for Building 49…”

“There is no Building 49,” Greg said automatically.

“Of course,” the guy said, flashing a tight smile. “There’s no Building 49. But we’re putting together a team to revamp the Googlecleaner. Maya’s code wasn’t very efficient, you know. It’s full of bugs. We need an upgrade. You’d be the right guy, and it wouldn’t matter what you knew if you were back inside.”

“Unbelievable,” Greg said, laughing. “If you think I’m going to help you smear political candidates in exchange for favors, you’re crazier than I thought.”

“Greg,” the man said, “we’re not smearing anyone. We’re just going to clean things up a bit. For some select people. You know what I mean? Everyone’s Google profile is a little scary under close inspection. Close inspection is the order of the day in politics. Standing for office is like a public colonoscopy.” He loaded the cafetière and depressed the plunger, his face screwed up in solemn concentration. Greg retrieved two coffee cups—Google mugs, of course – and passed them over.

“We’re going to do for our friends what Maya did for you. Just a little cleanup. All we want to do is preserve their privacy. That’s all.”

Greg sipped his coffee. “What happens to the candidates you don’t clean?”

Read the whole thing, as they say. It’s about more than the specific topic of social graph data – but it’ll get you thinking a bit.

I’ve spent the day in the discussion forums on the social graph/network (the hot issue of the day is what to call it) and while I see the word ‘privacy’ a lot, I haven’t yet found the nugget explaining how privacy is going to work in an era of social graph as exportable, universally mineable data. I’ll keep looking, and as always, welcome pointers to what I may be missing.

Welcome, Instapundit readers…and everyone check this out as well…

Let me pull something over from my professional life for a second, because I think it’s consequential enough that you folks ought to know about it. It’s not something I’ve done but something I’ve been reading about.

Google and other companies (Six Apart, among them) are going to open their API for social graphs.

The short version: Google will announce a new set of APIs on November 5 that will allow developers to leverage Google’s social graph data. They’ll start with Orkut and iGoogle (Google’s personalized home page), and expand from there to include Gmail, Google Talk and other Google services over time.

What’s a ‘social graph’, you ask…it’s a map of the connections between people and between people and content.These graphs are deeply meaningful because they are not random; they are full of meaning because they track our behavior, contacts and interests.

When I was designing my version of Pajamas Media, I wanted to construct a social graph connecting readers and blogs and use that data to predict what new blogs people might like, and to cluster ad placements to targeted groups of readers. I thought – and still think – there’s a ridiculous amount of value in that data. There is also a series of problems with that data, and they are at root problems of privacy.

Because the tradeoff for the usefulness of having someone suggest new blogs I’d find interesting is that someone has to know what blogs I read. And to the extent that I read blogs that I don’t want people to know about – blogs about sex, psychological issues, political positions antithetical to my public persona – that’s potentially a problem.

I thought I had an elegant solution to the problem for my PJ’s design (hey, I can’t tell all my secrets), but still saw (and see) this as the crux issue a system like this will have to get around.

But now much bigger fish are stepping into the market, and they are doing it with data much more serious and personal than your blog-surfing habits.

Google is announcing that it will create a series of open API’s (data interfaces) that will allow other people to write systems that will allow them access to an undetermined set of Google’s social graph data. What data does Google have? Well, pretty much everything. My email from my Gmail accounts; my search history; my blog posting and links if I keep them on Google tools; the links to and from my blog if the Google spider is set to capture them; the YouTube videos I watch, and so on…

Now the folks doing this are serious and smart people. They have explicitly talked about the issues of privacy. here’s Six Apart’s David Recordon:

An open social graph is just as important as an open identity.

* You should own your social graph

* Privacy must be done right by placing control in your hands

* It is good to be able to find out what is already public about you on the Internet

* Everyone has many social graphs, and they shouldn’t always be connected

* Open technologies are the best way to solve these problems

* We’re going to release code and demos soon

The privacy and security implications of this are pretty staggering, if I’m interpreting this correctly. They are solvable – I did a baby solution as noted when I designed the system for PJ’s, and I don’t doubt that the horsepower of this group can propose useful solutions.

But I’d be a helluva lot happier of they had started with the basic principles and mechanisms for ensuring privacy and announced those first – before releasing working code modules.

I’ll be digging into this more deeply, and may have some posts here to talk about it. Meanwhile, discuss among yourselves, and personally note that I welcome our new Redwood City overlords…

Mary Mapes has seized on Dan Rather’s Quixotic attack on CBS at a bloody shirt (enough metaphors yet?) to wave in defense of the truthiness of their journalism about President Bush.

Go read the whole thing, but move the drinks away from the keyboard when you do.

It has been three years since we aired our much-maligned story on President Bush’s National Guard service and reaped a whirlwind of right-wing outrage and talk radio retaliation. That part of the assault on our story was not unexpected. In September 2004, anyone who had the audacity to even ask impertinent questions about the president was certain to be figuratively kicked in the head by the usual suspects.

What was different in our case was the brand new and bruising power of the conservative blogosphere, particularly the extremists among them. They formed a tightly knit community of keyboard assault artists who saw themselves as avenging angels of the right, determined to root out and decimate anything they believed to be disruptive to their worldview.

To them, the fact that the president wimped out on his National Guard duty during the Vietnam War — and then covered it up — was no big deal. Our having the temerity to say it on national TV was unforgivable and we had to be destroyed. They organized, with the help of longtime well-connected Republican activists, and began their assault.

Actually, we had done a straightforward, well-substantiated story. We presented former Texas Lt. Governor Ben Barnes in his first ever interview saying that he had pulled strings to get the future president into the National Guard after a Bush family friend requested help in keeping the kid out of Vietnam.

And we showed for the first time a cache of documents allegedly written by Bush’s former commander. The documents supported a mountain of other evidence that young Bush had dodged his duty and not been punished. They did not in any way diverge from the information in the sketchy pieces of the president’s official record made available by the White House or the National Guard. In fact, to the few people who had gone to the trouble of examining the Bush record, these papers filled in some of the blanks.

We reported that since these documents were copies, not originals, they could not be fully authenticated, at least not in the legal sense. They could not be subjected to tests to determine the age of the paper or the ink. We did get corroboration on the content and support from a couple of longtime document analysts saying they saw nothing indicating that the memos were not real.

Instantly, the far right blogosphere bully boys pronounced themselves experts on document analysis, and began attacking the form and font in the memos. They screamed objections that ultimately proved to have no basis in fact. But they captured the argument. They dominated the discussion by churning out gigabytes of mind-numbing internet dissertations about the typeface in the memos, focusing on the curl at the end of the “a,” the dip on the top of the “t,” the spacing, the superscript, which typewriters were used in the military in 1972.

It was a deceptive approach, and it worked.

These critics blathered on about everything but the content. They knew they would lose that argument, so they didn’t raise it. They focused on the most obscure, most difficult to decipher element of the story and dove in, attacking CBS, Dan Rather, me, the story and the horse we rode in on — without respite, relentlessly, for days.

Oh my Freaking God.

Look, Mary, let me try and explain it to you. I’ll make it simple, I don’t have a lot of time.

I don’t for a second doubt that Bush pulled strings to get beneficial treatment. Similarly, I don’t doubt that Kerry gamed the system to leave Vietnam before his tour was up. People game systems all the time – and in both cases, it’s a legitimate issue to raise when someone is running for office.

But the fact that Bush may have used pull doesn’t justify lying – or careless assertion of facts that can be easily disproved – to do a hit piece just in time for an election. Let me put it another way:

Captain Dudley Smith: Would you be willing to plant corroborative evidence on a suspect you knew to be guilty, in order to ensure an indictment?

Ed Exley: Dudley, we’ve been over this.

Captain Dudley Smith: Yes or no, Edmund?

Ed Exley: No!

There’s a reason why we let guilty suspects go free when cops plant evidence.

Mapes and Rather got a hard-on for Bush, and were blinded by the glee that they’d be able to lay a hard hit on him just in time for the election. Sadly for them, they stopped paying attention to the details.

Mapes, delusionally, claims that the typographic facts about the documents they hung their case on are themselves ‘deceptive’ and that the documents are ‘fake, but accurate’. If Mapes and Rather had been honest journalists, they would have run with the provable facts about Bush’s military history – they would have has a much less sexy story, but a real one.

And Mapes is just off her freaking rocker when she claims that the fact that attacks on Bush were met with brickbats while ignoring that attacks on Clinton were equally met with personal vendetta.

But then zombies were never very smart.

Instapundit linked to this farewell (literally, sadly) lecture by Carnegie Mellon videogame and 3-D presentation professor Randy Pausch (creator of Alice, which I think is a really cool tool). He has terminal pancreatic cancer, and gave a farewell lecture which was stupendously moving – go read the whole thing.

But he said one thing that kind of riveted me.

Flashing his rejection letters on the screen, he talked about setbacks in his career, repeating: “Brick walls are there for a reason. They let us prove how badly we want things.”

As someone who’s experienced more than his share of setbacks, let me tell you the truth of that message. I’d hate to have lived a life with no walls for me to push against.

The best teachers, as always, teach us as much about life as anything else.

I have a piece up in the Examiner on my son’s enlistment. It ends:

I fed him and paid his tuition. He’s taken those materials, and now the man he’s used them to become has set out to navigate that wide, risky world. I watched his back again as he walked down our driveway last night.

From now on, I’ll do my best to see that our leaders are watching his back as well.

My blogging will be light to nonexistent until next week; Biggest Guy reports Monday, and to be blunt – I have more important things to do while he’s here.

Don’t kill each other or blow anything up while I’m away, please, and enjoy the work of the coauthors here…

Here’s what I see from my client’s window:

Here’s what I think about every time I look:

And here’s what I’m thinking about today…

May this anniversary be one of hope – hope for a year with less hate, a year where we look back on shared tragedy and realize that we also share a future, and that we all must share a planet.

Watching the Petraus/Crocker hearings and responses had me thinking about what they reminded me of…

…and I realized that it was really Noh drama:

By tradition, Noh actors and musicians never rehearse for performances together. Instead, each actor, musician, and choral chanter practices his or her fundamental movements, songs, and dances independently or under the tutelage of a senior member of the school. Thus, the tempo of a given performance is not set by any single performer but established by the interactions of all the performers together.

What we’re really watching isn’t any kind of real debate – it’s really a drama, preplanned out on both sides, and really much more about style and a kind of formal abstraction than about anything resembling reality.

Here’s the Center For American Progress on Petraeus’ testimony. Prepared in advance, they argue against points Petraeus doesn’t make.

Here’s Petraeus’ testimony; read it for yourself.

It’s really about positioning public arguments – seeding the public discourse.

The reality is – what, exactly? We’re lost in a forest of statistics of dubious provenance and facts rooted in rhetorical claims – on both sides, no doubt. We make a decision based largely on faith, and we choose who and what to have faith in.

And watching the hearings on my laptop, and reading about Moveon’s arrogant and stupid ad (…bought at a discount? Bob Owens wonders…) and then watching the disrespectful and foolish Code Pink demonstrators who fancy themselves the voice of the antiwar movement – no, the voice of the American people – I am wondering just who the dramaturg on the other side really is, and exactly what play they think this is and why they think I should have faith in them.

You can support the Victory Caucus by signing the online petition.

And you can support VictoryPAC by making a donation here:

VictoryPAC is a nonaligned PAC – which I control – that will support Congressional candidates who don’t support immediate withdrawal in Iraq; it will support both candidates who support the Administration position as well as those who propose interesting alternatives that don’t involve simply giving up and going home.

Do both – it’s one of those moments when you can feel the balance shifting.