Sunday January 23 2011

Exquisite Whispers

A while ago I started a website called Exquisite Tweets that makes it easier to collect and link to a series of posts on Twitter.

On January 19th, the site received a surge of requests to one collection in particular, which I’ve archived here for posterity. The collection appears to portray a comedy of errors leading to the mistaken belief that a gunman was on the loose at Oxford Circus. It’s a great example of the kind of stories that bubble up from Twitter, and a text book use case for Exquisite Tweets.

Except this one is better described as based on a true story.

The list was collected by James Darling (full disclosure: friend) from the jumbled search results for the phrase “Oxford Circus”. It was an attempt to illustrate the confusion, as much his own as anyone else’s.

But it doesn’t represent a “widespread panic”, this was no “news hoax”. The “hysteria” lasted for half an hour at the most, and people wandering around Oxford Circus were mostly oblivious to the “chaos.” It seems that the rumours weren’t even caused by a misheard tweet in the first place, according to diligent old media reports.

But the story stuck and its message became: “Behold the dangers of the real time web, it’s chinese whispers gone mad!”

The only danger here was in people falling over each other to point and laugh at the credulous, while they themselves had been deceived by a selective telling of the facts.

Their reaction is certainly understandable, and after 1,717 tweets and 93,523 hits at the time of writing, this was obviously something that resonated with people.

But it was a false resonance, and the only tweet to really “spiral out of control” was the one in which James shared the link with his own followers.

“I thought it was pretty obvious to me and my followers that it probably wasn’t anyone’s fault. If I’d realised it was going to hit the masses I’d have upped the state-the-obvious. But then again, if we spent as much time making sure everything we write on the internet isn’t misconstrued by anyone in the world, then we’d be as good to the world as a PR company.”

In the end, no one got hurt, and people had a good chuckle, so who cares? For me it’s an interesting example of how easy it is to bend the truth, even if all you’re doing is presenting facts. Something to consider for those of us dealing with archival, curation and history.

A brief ode to Varnish

BEWARE: nerd stuff follows.

Here’s one for those of you with a graph fetish:

Varnish request rates 19 January 2011

Varnish requests per second — 19 January 2011

After I realised I was sending Cache-control: no-cache headers on every page load and fixed that nasty red patch, pretty much every request was served from the static cache, saving my web server, poorly optimised PHP and database from the full brunt of the load.

This was great until I had to uncache the page, because people who’d deleted their tweets were upset that they were still showing up in James’s collection.

Uncaching a URL from the Varnish cache

You might find this useful if you’re running Varnish on debian:

Based on these example docs, I ended up adding the following to my /etc/varnish/default.vcl file to support a wildcard matching HTTP PURGE method for adding to the Varnish purge list. Make sure you don’t override any existing vcl_recv sections you’ve already defined.

sub vcl_recv {
    if (req.request == "PURGE") {
        if (!req.http.X-Real-IP ~ "72.249.20.114") {
            error 405 "Not allowed.";
        }
        purge("req.url ~ " req.url " && req.http.host == " req.http.host);
        error 200 "Purged.";
    }
}

I don’t want to allow anyone to purge entries in the cache. Usually you’d add an access control list (acl) entry for localhost and check it against client.ip as in the examples, but Varnish sits behind nginx on my system so every request comes from localhost. So I’m checking the X-Real-IP header set in my nginx proxy.conf against my server’s IP.

The debian startup script for varnish gives you an easy way to reload the config file without restarting Varnish: /etc/init.d/varnish reload. After running that, I can now purge that page with curl:

curl -X PURGE http://www.exquisitetweets.com/collection/abscond/152

And everyone’s happy.

Comments welcome via email or Twitter