Hacks and Tricks of the Trade at Facebook

Software engineers love hacks that become tricks of the trade.

A hack is regarded as something that gets the job done in a clever way, but it's usually brittle. It's an inelegant, but effective, solution to a computing problem, sometimes referred to as a kludge or jury rig.

Any software engineer who's coded on a daily basis has written a hack. Every so often, a hack rises to the level of innovative breakthrough and is recognized by a prominent person in the industry.

As I've said before, innovation is anything that reduces the cost of a transaction in terms of time or money. The best hacks are the ones that already use the current infrastructure in an innovative way.

Until a decade ago, most every time a website served up a webpage it would go to the database to refetch data. For example, an e-commerce store would look up the products that are on sale each time a user requested that page. The problem was that many pages were served up at the same time, with redundant hits to the database. After all, the list of items on sale isn't going to change from one minute to the next.

About ten years ago, as servers needed to handle more load, there was a rise in open source caching technologies to minimize the number of trips to a database. This was a big gain for read only data, which doesn't change often. Storing data in memory (as a cache does) reduces the overhead of interacting with a database. A database's job is to ensure data reliability by running many checks, formally know as ACID properties. But many of these checks are unnecessary if the data doesn't change very often. So, rather than make a trip to the database, the data is simply stored in memory so it's retrieved much faster.

Facebook Tricks

I recently heard about a brilliantly simple trick that Facebook uses to speed up their site. When a Facebook user logs into their account, their data is fetched from the database. While fetching the data, the user has to wait. The amount of wait time could be imperceptible to the user, or it could be a noticeably long time if the website is under a heavy load. "Heavy load" is a relative term, but Facebook services more than one billion users per day, so saving any amount of time makes a noticeable difference.

Wouldn't it be great if Facebook's servers knew what data a user needed before the user formally requested it? Well, that's effectively what Facebook's done with their little trick that simply involves sending an encrypted UDP (datagram) ahead of the formal TCP/IP request. UDP requests are fire-and-forget, meaning there's a small chance they might not arrive at their destination. TCP/IP, on the other hand, guarantees delivery (or notice of a failed delivery). TCP/IP is the reason that webpages render perfectly compared to the BBS's of the 1980s that used unreliable dial-up modems where static and interference would be misinterpreted as data and displayed as garbage text.

So, the UDP datagram arrives well ahead of the TCP/IP request which enables Facebook's servers to pre-fetch the data and load it in its cache before the formal TCP/IP request arrives. A simple yet elegant way to optimize a website for speed.