Monday, August 31, 2009

Twitter's "Track" Command: Gone But Not Forgotten.

Twitter used to have a fantastic real time search command: Track

Sometime in 2008 it seems to have been turned off - probably because it generated too much SMS (text messaging) traffic.

Fire!
During the San Diego Wildfires of 2007 I was splitting my time between San Diego (Carlsbad) and Santa Cruz (Capitola). The wildfires broke out over the weekend when I was in the Bay Area. By Monday morning two separate fires were threating our home in Carlsbad. I went to sleep, Monday night, trying to prepare myself, mentally, for what it was going to be like once our home burned down.

Tuesday, as I drove down to Carlsbad, I wanted every piece of information I could find about the fires. Listening to XM channel 247 (emergency channel - a wordplay on 24/7) helped, but it was too broad since it was covering all the fires burning in San Diego, Orange County, and L.A.

This is where Twitter's Track command was a saviour (keep in mind that there were no iPhone apps back then). I simply texted some keywords to Twitter and every time someone's tweet contained one of those words it was relayed to me via SMS. I had Twitter track "Carlsbad" and the major road near my home, "Palomar".

Retweeting wasn't as popular back then as it is now so I received very few duplicate tweets. Nearly every tweet that I received - and I was receiving a new tracking tweet every five minutes - was helpful:
"It doesn't look like the fire's reached Carlsbad."
"Winds dying down and reversing direction - I can see flames from Carlsbad."
"Voluntary evacuation south of Palomar Airport Road."
"KPBS reports that fire in San Marcos, near Carlsbad, is 20% contained."
etc.

Luckily, the closest both fires got to our home was about four miles. Now, if only Twitter could bring back the Track command and ignore retweets.

Saturday, August 29, 2009

My Experiences with DNS Hosting

Overview
The Domain Name System, better known as DNS, is probably the most critical part of the Internet. DNS converts domain names, such as google.com, into IP addresses like 74.125.45.100. Since it's so important it's also the most robust and redundant Internet infrastructure in place. Attacks against this system usually go unnoticed by the public. If an attack were to successfully bring down all 13 root name servers then Internet traffic would, for all practical purposes, be unroutable - and the Internet would stop working. Luckily, each root server is actually a farm of servers which appear, from the outside world, as a single server.

Taking Down the Internet
Taking down all 13 root servers at the same time would have the effect of removing every street sign on every road in the world. Unless you know where you're going, and you've been there very recently, then your network packets used for web browsing, e-mail, etc, won't know how to reach their destination.

The Root
Top Level Domains (TLDs) are the last portion of a fully qualified domain name (i.e. .com, .net, .us, etc). To be completely correct, all TLDs end with the same character ("." pronounced "dot"). If you have a decent web browser then the following link should work: http://www.cnn.com./ (include the ending .) If this example doesn't work, then try pinging it from the command line. Think of the . as the root of DNS.

Domain Name Registration
When you purchase a domain name the registrar usually configures your DNS with some default settings. Generally, it'll point your domain to a generic landing page until you either upload your own web page or reconfigure the DNS to point to either another DNS server or web site. Once you've changed a DNS record, it can take some time until ISPs are updated. How long these updates take to propagate is configurable when creating a DNS record - the typical range is from an hour to a day.

DNS Configuration
You have two options when configuring DNS. Either you can configure it through your registrar or you can run your own DNS server. Over the past decade I've tried both methods, extensively.

DNS Self-hosting: QuickDNS Manager
In the beginning, domain registrars did not have sophisticated DNS management tools so I ran my own DNS server using QuickDNS Manager from Men & Mice (They no longer sell this great product, under this name, anymore). QuickDNS made it extremely simple to configure DNS using the QuickDNS Manager's GUI.

Click to enlarge

In this example, the TTL (time to live) column sets how long, in seconds, third party DNS servers (i.e. ISPs) should cache this information before going back to the the registrar. The defaults in the upper right are used when the TTL column is blank for a particular record. Therefore, this DNS configuration tells third party DNS servers to cache the www.example.com and example.com records for 300 seconds (five minutes).

Although self-hosting my own DNS server gave me a huge amount of flexibility the biggest draw back was that it requires a dedicated server machine. Since running a DNS server doesn't require heavy lifting by the server's CPU, I was successful in running my own DNS server for business purposes on an old 233 MHz (Wall Street) and then later a 500 MHz PPC G3 (Pismo) PowerBook with no problem at all. The beauty of using an old laptop as a server is that its battery acts like an internal UPS. As a matter of fact, about five years ago, I used to run e-commerce web servers, mail servers, DNS servers, etc., "on the cheap" using a farm of laptop servers.

There are other many other DNS server software options, but I particularly liked QuickDNS due to its ease of use.

GoDaddy's DNS in the Cloud
These days, it's hard to beat using a DNS service that's hosted in the cloud - especially when, in the case of GoDaddy, it's free. For the cost of registering your domain name (about $10/year), you can configure your domain's DNS either through a web browser or through a text file that can be uploaded and downloaded to/from GoDaddy.

GoDaddy UIs
GoDaddy's DNS notations deviate slightly from the DNS BIND standard, but it still works as expected. Specifically, they have eliminated the need for each domain to end with a dot - after all, it's implicit. Also, when you want to reference the domain's root name (i.e. example.com) you use the @ symbol.

Here's a screenshot of how I've configured AdjixSucks.com to be a static web site hosted on Amazon's S3 (more about hosting websites on S3 can be found here):



Here's the text file, from GoDaddy, which can be downloaded, edited, and then uploaded (Be sure not to upload duplicated DNS records. If there's a duplicate record then GoDaddy will not apply any changes and return an error. This is a great safety mechanism to prevent accidents which could bring down a website.)


Using GoDaddy's web interface, you can configure your DNS record's TTL for 30 minutes, one hour, 12 hours, one day, or one week. To configure with a finer level of granularity, i.e. 300 seconds, you'll have to upload the updates to GoDaddy via a text file.

Out Source or In-house?
While there are other DNS hosting options, and some cost a small amount of money, it makes a lot of sense to use a professional DNS hosting solution instead of running your own DNS server. If you don't own the hardware then you don't have to support it. (While software may have bugs, it never fails in the manner that hardware can.) Due to the critical nature of DNS, third party hosting solutions do an excellent job at supporting this service.