The Single Most Important Step to Securing a Web Site

I've worked as an employee and consultant, in a technical capacity, for tiny companies through multi-billion dollar corporations. With the exception of the companies where I've been a founder, I've noticed that the technical teams at these companies would have a hard time immediately detecting an attack or abuse unless it caused obvious damage to the functioning of their website.

For small to medium size websites (< 5-10 requests/second/web server) the single most important defense against attacks is to continuously monitor logs. The simplest way to do this is to "tail" logs for all instances of the web server, app server, and security logs (ssh, DB, etc) 24/7 on either a system admin's or developer's computer. It would be great if this could be automated, but, many times, it's going to take a human to notice something abnormal.

Most abuse/attacks on a website begin with probing the website itself (port 80/443) or via SSH (port 22). Once abuse or an attack is noticed from a centralized source, the first step should be to block that IP address and then contact the ISP of the attacker/abuser ideally with a phone call to their network operations center or via their abuse@example.com e-mail address which is usually listed when conducting a whois lookup. In either case, the offending ISP will ask for a copy of your server logs which should be cleansed of any third party data. In other words, only send the offending ISP a copy of the logs that specifically pertain to the attacker/abuser.

Typical web server logs should contain at least the following info:
Requester's Host Name/IP: ec2-75-101-189-255.compute-1.amazonaws.com
Timestamp with time zone offset: [20/Dec/2012:13:39:40 -0800]
Request: "GET /index.html HTTP/1.1"
HTML Status: 200
Response size (bytes): 16844
Referrer: "http://google.com/com"
Time taken to server the request (seconds): 1

Also, don't include too many log parameters on web requests lest you suffer from scrolling blindness. The logs have to be readable, in real time, by a human. For unusual periods of peak traffic, it helps to tail and grep the logs in real time to focus on specific log entries.

You'll be amazed at how much you'll learn by simply monitoring these server logs on a daily basis, for just a couple weeks; it will give you a sense for what is normal traffic. Errors, unusual requests, and long response times should be investigated. Some of these issues will be web app bugs or areas requiring optimization which may have never been detected (such as an internal report taking minutes to respond when a typical request/response takes less than a second).

Frequent attacks go unnoticed until there is some ramification. If you don't know what normal behavior is then you won't be able to detect abnormal behavior.
 

How to Squeeze More Than 140 Characters Into A Tweet

Summary
How to squeeze more than 140 characters into a tweet with indexable text.

Background
Last Friday, I half jokingly sent out a tweet demonstrating how to squeeze more than 140 characters into a tweet.

Basically, I tweeted that it was possible to take a screen shot of text longer than 140 characters which could then be attached to a tweet as an image.

I say half jokingly because, even though it works, it's a bit of a hack. It wasn't until Dave Winer – who actually didn't support this technique – retweeted me that I saw a surprising interest in it. The interest probably wasn't so much the solution as it was the part that was a joke. One person called it "twaxing" (tweeting + faxing). And Dave half-joked, "the web cries" for me. But I noticed, over the weekend, that a number of people have started adopting this technique.

It's handy that Twitter now, organically, hosts images on their servers which means that a link to an image in a tweet is much less likely to break when compared to hosting an image with a third party.

A couple people pointed out the obvious shortcomings to the twax technique which was that an image of text isn't indexable (searchable). That got me thinking… what if a twax was indexable? Would there be any benefit for those times when you needed, say, 400 or 500 characters in a tweet? Perhaps.

Searchable Solutions
My initial twax search solution was to embed the tweet's meta-data in the text of the tweet but that is too ugly, inelegant, and incomplete.

After some more thought, I realized a very workable solution: embed the tweet's meta-data as a QR code inside the tweet. Any third party server could simply scan the image and decode the QR code while any human could read the text.

Technically speaking, this will work. But, would anyone use it and how would it work?

I can easily see a mobile app or Twitter web feature that would notify you once you exceeded 140 characters while continuing to let you type. When you clicked the Tweet button, to publish your tweet, the tweet would contain the first 140 characters of the tweet while the entire tweet's text would be displayed in the image, followed by a QR code of the tweet's text and meta-data.

The nice thing about this technique is that a single QR code can encode thousands of characters of text. Plus, the tweet text and its image with the QR code could stand alone without network connectivity.

When A Constitutional Amendment Outlives Its Usefulness

We have completely failed the dead children of Connecticut today. –Ric Arthur

"Why should a private American citizen own an assault rifle?" asks the city girl.

Good question.

"Because it's my Second Amendment right," answers the farmer.

Good answer.

Does the U.S. Constitution contain amendments that have outlived their purpose?

Certainly. The best example is the 21st Amendment which repealed the 19th Amendment on Prohibition.

What about the Bill of Rights? No Amendment in the Bill of Rights has ever been repealed.

The Third Amendment of the Bill of Rights, which prevents soldiers from being quartered in a home without the owner's consent, has certainly outlived its purpose. The U.S. Military is so large and well funded that there's no need to quarter soldiers in private homes. Repeal the Third Amendment and America doesn't skip a beat.

Surprisingly, yesterday's shooting isn't the largest K-12 massacre. In 1927, a disgruntled tax payer used dynamite to kill more than 40 people, including 38 elementary school children in Bath Township, Michigan.

So, what about the Second Amendment which gives American citizens the right to keep and bear arms? Could that, too, have outlived its usefulness? What were our forefathers thinking when they drafted this amendment?

The Second Amendment serves several purposes. In 18th Century America, a typical citizen may own a firearm for hunting and protection. Constables didn't cruise the suburbs to maintain law and order 200 years ago. And there was no way to call 911 for help. Big chain supermarkets and grocery stores didn't exist with stocks of food that you could store in your refrigerator. Hunting was a big part of survival. 

But, beyond these reasons, there was another key purpose for 18th Century Americans to own firearms which was to keep the government in check. With the exception of the cannon, private American citizens were on even footing with the government when it came to weapons. Automatic weapons didn't back then, nor did weapons of mass destruction.

In colonial American, firearms didn't even have what we now think of as traditional bullets. In the 18th Century, firearms like handguns and rifles were single shot flintlocks with a hammer that held a rock mineral (flint). When the trigger was squeezed, the hammer hit a frizzen that threw sparks into a pan of fine gun powder which set off the weapon. If you turned a flintlock firearm sideways the FFFF gunpowder fell out of the pan making it unfireable. A flintlock firearm couldn't easily be concealed nor would it work in the rain (hence the phrase keep your powder dry.)

In the 18th Century, a battle between private citizens and government backed troops was nearly an even match in terms of firepower. It's clear that this is no longer the case. The U.S. Government has weapons of mass destruction and no reasonable person could make the case that private American citizens should own tanks, bombs, or missiles.

Question Without Answers

At this point, we only have complex questions without simple answers:

How does the current state of military hardware affect the Second Amendment? A matchup between the U.S. Government and its private citizens is not even close to a fair fight.

Should Americans own any firearms? Should private firearm ownership be better controlled? Could it be better controlled?

Even if private firearm ownership was eliminated, would people take to other forms of attack such as the Bath Township massacre or the Shoe Bomber? Bad people can always find big ways to hurt good people. Look no further than the 9/11 attacks to see how machines of peace can be turned into weapons of mass destruction.

Could schools be made more secure, similar to airports? Even if that's done, kids will still go outside to play which is where two elementary students were shot, two years ago, in my own town of Carlsbad.

Here's something important to consider… think critically of your ideas. If you think the Second Amendment should stand as is, consider yourself wrong. If you think the Second Amendment should be repealed, once again, consider yourself wrong as you think consider workable solutions. Not being able to critically examine your own thoughts and ideas leads to closed-mindedness. 

One final thought to ponder as the Constitution tries to keep pace with technology...
Which is more important: your privilege to drive a car or your right to keep and bear arms?

(Not that they're mutually exclusive nor am I suggesting the prohibition of all firearms. I'm merely making a point about the difference between rights and privileges. Driving, in Saudi Arabia, is a privilege that women don't have.)

There's no simple solution – but at least all sides agree that something needs to be done to prevent a repeat of yesterday's tragedy.

What If Money Didn't Matter?

Keep doing what you love and what you're good at.

 

A Bicycle for the Mind

Steve Jobs frequently referred to personal computers as a bicycle for the mind.

When Steve was about 12 years old, he read a study that measured the locomotion efficiency of different species. At the very top of the list was the condor, who expended the least amount of calories to travel. Humans, on the other hand, came in an unimpressive third of the way down the list.

Steve points out that someone at Scientific American took a look at the efficiency of locomotion of a human on a bicycle which blew away all other forms of animal powered locomotion. After all, the bicycle is the most efficient human powered machine for locomotion.

It's an interesting metaphor since it seems so accurate – it's the amplification of human ability.

iCloud Restore: Slow and Steady

I recently upgraded from my iPhone 4S to an iPhone 5. I've been using iCloud for backing up my iOS devices for the past year, so I was somewhat familiar with it.

The first time I used iCloud to restore my iPhone data, in March, it didn't work well – actually, I gave up on trying to recover my camera roll with 12,000+ photos since it was already backed up on my desktop. It was just taking too long.

For my iPhone 5 restore, I waited it out. It took about 60 hours to recover my 30 GBs of backups stored on iCloud. It seemed as if each photo took about 10 seconds to restore. Of course, there were about 8-12 hours in there when I wasn't on WiFi and the iCloud recovery process was paused. So, let's call it 50 hours.

Anticipation
Why so long? Because it requires a lot of time to download that much data from iCloud, regardless of how big the pipe is (bandwidth) on your end. Unlike iTunes movies and music downloads, which happen in a jiffy, iCloud restores do not gain any benefit from content deliver networks (CDNs) such as Akamai or Amazon's CloudFront. A CDN will cache data, like movies, on servers that are as close to you as possible. That way the bits are only traveling, say, 100 miles, instead of thousands of miles. Less hops means faster delivery.

Once the iCloud restore was complete my iPhone then downloaded my apps, music, movies directly from the iTunes store, which did benefit from a CDN. Rather than store this information in my own iCloud storage Apple just keeps it on their servers since this information isn't unique. My last step was to resync with iPhoto to recover my photo albums – after all, iCloud only backs up your camera roll.

In the end, I didn't lose anything, it just took a very long time. For my next iPhone upgrade, I will backup my data via iTunes on my computer. Just keep in mind... if you have a 64 GB iPhone, you should make sure that you have at least 64GB free on your computer before trying to back it up.

Response: The Few, the Proud, the Infantilized

This post is a response to Professor Fleming's article, The Few, the Proud, the Infantilized, published earlier this week.

It is by no means enough that an officer of the Navy should be a capable mariner. He must be that, of course, but also a great deal more. He should be as well a gentleman of liberal education, refined manners, punctilious courtesy, and the nicest sense of personal honor.

--- John Paul Jones

I read Bruce Fleming piece entitled The Few, The Proud, the Infantilized. The knee jerk reaction of many of my fellow service academy graduates might be, as was said in A Few Good Men, "I would rather you just said thank you, and went on your way. Otherwise, I suggest you pick up a weapon, and stand a post." However, I want to add some deeper insight. Some of Fleming's premises are right, but many of his conclusions are wrong, or, at least, incomplete.

As a Naval Academy graduate, I can tell you that Fleming is spot-on with his facts regarding the perception of midshipmen. From his articles and his book, Annapolis Autumn, I'm amazed at how much he gets right. He has the pulse of the Brigade of Midshipmen but without adequate context. Annapolis needs smart people who think differently, and, as painful as a pill it is to swallow, Professor Fleming's presence at the Naval Academy helps to keep us thinking critically about that institution.

Costs and Insights

Mr. Fleming has extraordinary insight into the inner workings of the U.S. Naval Academy's academic machine. He has been a professor at Annapolis far longer than any superintendent, commandant, or academic dean has been in charge. He has probably seen a few mistakes made at my beloved alma mater; no system is perfect. However, the downside of his overexposure to the Brigade is that Fleming has become a perpetual midshipman.

Without a doubt, a four year education at a service academy costs more than NROTC or OCS. However, his conclusion to eliminate or "repair" the academies, is shortsighted. I see Professor Fleming as a liberal thinker at a conservative institution; yet his propositions are too conservative in nature, while other comments, such as "Most of what the Naval Academy's PR machine disseminates is nonsense," are exaggerated.

Fleming needs to take a holistic view of the service academies and their function within the military. I greatly disliked the Naval Academy when I attended it; but, you would be hard pressed to find a graduate who, after completing their military obligation, still maintained the same view of their respective service academy that they held when the attended it.

The U.S. military has many pipelines to become a combat officer. As Lieutenant General Victor H. “Brute” Krulak wrote, in 1957, to the Commandant of the Marine Corps:

...in terms of cold mechanical logic, the United States does not need a Marine Corps. However, for good reasons which completely transcend cold logic, the United States wants a Marine Corps. Those reasons are strong; they are honest, they are deep rooted and they are above question or criticism.

Replace the words Marine Corps with service academy in the above quote and it is equally valid.

Things are not always what they appear. From the hottest fire comes the strongest steel is an old cliche with a deep truth. As a midshipman, we are put through trials and tribulations. Military indoctrination is the most extreme physical, academic, and emotional roller coster ride that one can experience at such a young age this side of combat or being a POW. On the surface, Plebe Summer may look like sadistic hazing. Every plebe is systematically set up for failure. They are scorned, yelled at, and punished. More stress and negativity is compressed into several weeks than is imaginable. Can this humiliation be productive? Just ask a graduate. Analyzing this and drawing negative conclusions, without experiencing it, is incomplete.

What other academic institution of comparable size has such laser focus on the product it produces? None come to mind other than private military schools. More generals and admirals now ascend to flag officer rank not because the service academies are worse at what they do, but, rather, it is, I believe, because the other commissioning pipelines have vastly improved over the decades.

Learning the Untaught

At first, I took exception to Fleming characterization of service academies as a military Disneyland. But, then I reflected on my time as a plebe when I, too, thought the same thing and nearly quit so I could return to being a corporal in the Marines. That would have been shortsighted. I am much better off for sticking it out.

The problem is that Fleming is standing too close to only one side of the equation without being able to visualize the bigger picture. He only sees the academic portion of the pipeline without participating in the full experience that extends past graduation. I cannot convey strongly enough the importance of what I learned at Annapolis that was never explicitly taught. Rather, it has to be experienced; and not in a classroom or on campus.

For example, Fleming comments on sleep deprivation. Sleep deprivation taught me my capabilities and limitations of what I can and cannot accomplish when physically, mentally, and emotionally exhausted. In a civilian school, I would have simply slept in and skipped class. Midshipmen who skip class are punished for being AWOL. Most civilian school graduates who did not experience what I did in this area had to learn their reactions later in their careers; perhaps at follow on training after they were commissioned or when they were in an operational environment. It is much better to learn it in a safe academic setting where the repercussion of failure is minimal. Learning from failure is a huge part of success.

My entire view on ethics was shaped as a result of what I learned in leadership classes at the Naval Academy. Mandatory leadership and ethics classes of this caliber are few and far in between at civilian schools. For example, I was explicitly taught when it is, in fact, okay to lie and be deceptive; and I try to remind others of the same: http://web.joemoreno.com/ethics.pdf

"But the students I respect the most tell me that those who succeed do so despite the institutions, not because of them." Ahh, no truer words were spoken by a midshipman. I know the sentiment. Even as a "firstie" (senior) preparing for graduation, I too resented the Academy. I didn't just mark the days until graduation, I memorized them. But, alas, midshipmen are held to a higher standard than their peer college students while giving up more liberties than could be imagined. We don't do it because it is easy; we do it because it is hard and, ironically, we end up taking pride in what we hate. It's a great honor for an active duty academy graduate to return to their alma mater to serve in uniform five, ten, or twenty years after graduation. I would be eager to see professor Fleming anonymously interview these returning active duty alumni a decade after graduation.

Mark Twain's sentiment about his father can be similarly applied to a midshipman:

When I was a boy of 14, my father was so ignorant I could hardly stand to have the old man around. But when I got to be 21, I was astonished at how much the old man had learned in seven years.

Better or Worse?

Is a service academy graduate better than an OCS or NRTOC graduate? No, but, the individual service academy graduate is, without a doubt, a better officer for having attended a service academy.

Many standards at the service academies are higher than both civilian schools and the military, in general. I've seen midshipmen expelled from Annapolis for academic or conduct issues only to embark on a long, honorable career in the Navy or Marine Corps. I have also seen other students voluntarily decide that Annapolis wasn't for them, only to attempt, and fail, to follow another pipeline to commissioning. I would speculate that the tolerance of wrongdoings which Fleming witnessed at Annapolis were failures of the institution's human leaders, not the principles of the institution.

Just like Fleming, I am personally very close to this issue. If it were not for Annapolis, I would not have become a Marine officer. In high school, and three years later when I applied to Annapolis, my SATs were 950, combined, out of 1,600, both times. That score put me in the bottom 36th percentile academically. When I sought admission, in 1988, the Naval Academy was the most competitive school in the nation with SATs averaging around 1,200 (if my memory serves). I graduated in the bottom half of a so-so high school. I was rejected by all the four year universities and two year community colleges I applied to. I had no motivation for college. Yet, despite this lack of self-discipline, I excelled in a military environment. Had it not been for the Naval Academy, I would not have graduated with a bachelor's degree from any other institution within four years.

For many of us, the structure provided by the service academies is so important that it literally means the difference between success and failure. In this area, I frequently think about the success of Buzz Aldrin, West Point class of 1951. Once his disciplined structure was removed, after leaving NASA and retiring from the military, his depression demons surfaced. It's amazing what one can accomplish when he or she has the full force and good faith of the U.S. government behind him or her.

I cannot stress to Fleming strongly enough that he is only seeing the tip of the iceberg. He is looking at a newly born baby and concluding that it is less capable than any other animal which can walk or swim right after birth. Judging the product of a service academy primarily by interacting with cadets or midshipmen is like judging a person's career at high school graduation. Midshipmen can be misinformed and hold on to those misbeliefs for a long time. Unlearning mislearned lessons usually requires a paradigm shift – one that midshipmen won't experience until years after graduation.

Service Academy Flaws?

It is easy to follow leaders. It is harder for leaders to lead followers. But, there is no harder leadership task than leading your own peers. Your peers see all of you, 24/7. Your motivations and beliefs must be pure in order for them to follow you. I think Fleming completely misses this experience. In the short term, an officer may fool both a senior or subordinate but it is nearly impossible to fool one's own peers.

All will agree with Fleming that the service academies should reduce their worst flaws. But, it is not obvious what those flaws actually are, especially from the view point of a midshipman. And we need to approach change very cautiously. Regardless of national politics, no other country, or the U.N., maintains the same global commitments and responsibilities as the United States. A mistake in training our military service members cannot be fixed with a software update. We all know what is at risk. We can make the right decision for the right reasons and still get the wrong result if poorly executed. 

Perhaps, one day, the service academies will outlive their usefulness; but it will be obvious as applications decline and interest is lost. When the service academies become the telegram or slide rule for combat officer commissions, it will be apparent. But, in the mean time, we should keep in mind, as Fleming points out, that the service academies are a means to an end, not an end itself. 

Childhood Communications

Yesterday, I read Ernest Hemingway's A Day's Wait. It's about a boy who is waiting to die from the flu. He has a temperature of 102°. He heard that any temperature over 44° was deadly. For an entire day, he simply waits to die, not understanding the difference between Fahrenheit and Celsius. Once he learns the difference between the two temperature scales and that he's not going to die, he simply goes back to his every day routine, but he's a bit more sensitive to the world. Children are resilient like that.

This story reminds me of two events in my life. The resiliency part brings back a specific memory when I was five years old. It was the first time my parents took me from Brooklyn to Huntington to show me the new house that we were going to be moving into. When my father told me that this would be our new home, I cried as we stood on the front lawn.

I asked him, "What about my friends?" I was obviously upset that I'd be leaving my "life long" friends of just a couple years.

My father hugged me and said, "You'll make new ones."

That thought hadn't occurred to me. I could simply make new friends. As soon as he said it, it all made perfect sense and I was completely over the trauma of moving to a new world.

Miscommunications
When I was about seven years old, I experienced the biggest miscommunication of my childhood. My parents were born in the mid 1930s and they grew up listening to radio programs like The Shadow, The Lone Ranger, etc. In the early 1970s, when I was about seven years old, they bought some cassette tape recordings of the old radio shows. One night, I sat with my entire family in our downstairs basement, which was laid out for entertainment. It was a game room with a wet bar, piano, organ, pool and ping pong table as well as a quadraphonic sound system.

One of the recordings that we listened to was Orson Welles' radio drama of H.G. Wells' novel The War of The Worlds. The broadcast talked about Martians who had landed in Grover's Mill, NJ. I knew that we were listening to a recording of a radio broadcast from decades ago. And I knew that New Jersey wasn't very far away – I had been there frequently. But, I also knew if Martians had attacked us in the past, so close to our home, that it could happen again.

So, when I asked my parents, several times, if these events had really transpired, they unknowingly answered, "Yes."

Obviously, they were answering the wrong question. I was asking if Martians had really attacked us. They heard me asking if this performance had really been broadcast over the radio.

I was reliving the panic of October 30, 1938, when many in the public thought the dramatic broadcast was real and fled their homes. Many listeners sued the network for "mental anguish" and "personal injury," but nearly all were dismissed. The newspapers may have played up the extend of the panic, but to me, and those before me, it was real.

I was terrified and, unlike the resilience of finding new friends, it took me a while to get over the fear of Martians.

Contextual Menus

When I worked at Apple, my boss's boss was discussing UI/UX design with Steve Jobs in meeting about the Apple Online Store. Steve commented that contextual menus had become the "dumping ground" for the lazy designer.

After upgrading to OS X 10.8.2, last night, I noticed two things. They weren't new things, but they jumped out at me.

First was that Mac OS X is no longer Mac OS X, it's now just OS X. I had heard something about this rebranding earlier this year, but this was the first time that I actually noticed it. In 2001, when I spoke at WWDC, it was beaten into us that we had to refer to it as Mac OS X.

The second thing - and this one's minor - is that the order of items on Mail's contextual menus could be better arranged - much better arranged.

Specifically, when right clicking on an e-mail account the Erase Deleted Items/Junk Mail should be near the very top. In other words, they should be one of the first choices since they're the most common options selected. Personally, I think the New/Rename/Delete/Export Mailbox options probably don't even belong in this contextual menu since they're so rarely needed.

But, that's just my 2¢. There may be a good reason that I'm overlooking.

Yet Another Aviation Lesson Learned

This past weekend, I flew down to Norfolk, VA. I was surprised that it was actually cool when I left Morristown, NJ but the forecast called for highs around 90° F so I knew that it would be warming up.

I had never flown into Norfolk International Airport and I hadn't printed out a hard copy of the approach procedures or taxi diagram since that information was on my iPad.

It was very pleasant at my cruising altitude of 6,000'. But, I could feel the heat and humidity when ATC stepped me down to 3,000' as I approached my destination. I realized that it would be an uncomfortable ten minutes, once I landed, until I reached my tie down spot and opened the door. Most small planes don't have air conditioners – it's all about the weight – so the only air movement inside the plane happens when it's flying.

As I touched down, the tower told me to expedite a turn onto the taxiway because there was a jet on short-final approach behind me.

I had already brought up the taxi diagram on my iPad so I knew exactly where I was and where I needed to go. Then, within 30 seconds of landing, the iPad display went black and it shutdown. It had overheated.

Obviously, this isn't an emergency situation but my heart did skip a beat as I evaluated my options before ground control told me to move. I figured, worst case, I could inform ground control that I needed progressive taxi instructions to the terminal, but that would be hard on my ego. After all, I didn't want to seem unprepared. Luckily, I remembered that I also had the taxi diagram on my iPhone. The electronic taxi diagrams are very handy since they not only display the layout of the airport's runways and taxiways, but they also mark your exact location, via GPS, within a couple meters.

As I brought up the taxi diagram on my iPhone, I wondered if it, too, would shutdown due to the heat. Luckily, it didn't, but this is a scenario I will be better prepared for, next time.