AWS as a Twitter Alternative

I originally drafted this piece on August 17, 2012 but didn't publish it until speaking with Dave Winer, today.



After yesterday's API announcement from Twitter, do we really need an open alternative? Probably not. Your average Twitter user doesn't care and won't notice, but here's the litmus test - it's one that App.net is currently going through. Let's build it and see if they come.

A tweet is, at most, 160 charters of text. That's 140 charters prefaced by a username. Of course it can be longer when we start adding in metadata. But, my point is that it's just short messages full of text characters. The big challenge is that Twitter is a system that's both a static blog and a dynamic communications system. It's one of the first main stream systems to serve those two purposes, similar to Usenet.

You only need to look at a couple Amazon web services to see that they've not only implemented a scalable infrastructure for a Twitter alternative, but they documented the API so thoroughly that any third party could reimplement the API and decouple themselves from AWS, if they wanted to.

How Would It Work?
If I were to implement a simple Twitter alternative it would behave like RSS with push notifications. They key is that you'd have to be able to read, post, mention, and follow via a web browser without needing to set up your own server. This is not as hard as it seems thanks to JavaScript. If we can make this work while running inside of a web browser, then building a desktop app or server to augment the basic service would be simple.

Overview
Notification is the easy part. Simply use AWS SNS to post updates to followers who are subscribing (following you). SNS updates would be posted to each follower's SQS queue.

The hard part is where and how to store static tweets. Originally, I proposed storing them as linked lists in DNS TXT records. Dave Winer pointed out that, although DNS could scale like this in theory, it probably couldn't in reality. He said that we already had the scalable solution with AWS S3. So, my proposal is to store each tweet as either standalone text or as an HTML webpage in an S3 bucket. Each S3 object contains the entire tweet with a link to the previous tweet and the next tweet. Any third party app could access these tweets and the links and meta-data could be nicely stored in the the S3 object HTML headers. (S3 HTML headers begin with x-amz-meta- and they're beautiful for associating meta-data with S3 objects.)

The Rules
1. Users would sign up for SNS and SQS.
Users, in this context, would be users of the service who could be the developers. They, in turn, could build apps that abstract this so the layperson user wouldn't need to sign up for SNS or SQS.

2. If you want to follow someone, you subscribe to their SNS notification and use your SQS as the endpoint for message delivery.

3. If you want to message (@) someone, you post to their queue.

AWS already has APIs to handle permissions so each user can block followers.

4. User's AWS credentials can be stored in their own web browser database. This eliminates the need for a third party database server that could be compromised.

________
January 30, 2013: This is still a work in progress with some unanswered questions, but, since it was tweeted about, yesterday, I dug it out of my drafts folder and published it.