Can't Send E-mail More Than 500 Miles

I do love great storytelling...

-------------------------

From: Trey Harris

Here's a problem that sounded impossible... I almost regret posting the story to a wide audience, because it makes a great tale over drinks at a conference. :-) The story is slightly altered in order to protect the guilty, elide over irrelevant and boring details, and generally make the whole thing more entertaining.

I was working in a job running the campus email system some years ago when I got a call from the chairman of the statistics department.

"We're having a problem sending email out of the department."

"What's the problem?" I asked.

"We can't send mail more than 500 miles," the chairman explained.

I choked on my latte. "Come again?"

"We can't send mail farther than 500 miles from here," he repeated. "A little bit more, actually. Call it 520 miles. But no farther."

"Um... Email really doesn't work that way, generally," I said, trying to keep panic out of my voice. One doesn't display panic when speaking to a department chairman, even of a relatively impoverished department like statistics. "What makes you think you can't send mail more than 500 miles?"

"It's not what I think," the chairman replied testily. "You see, when we first noticed this happening, a few days ago--"

"You waited a few DAYS?" I interrupted, a tremor tinging my voice. "And you couldn't send email this whole time?"

"We could send email. Just not more than--"

"--500 miles, yes," I finished for him, "I got that. But why didn't
you call earlier?"

"Well, we hadn't collected enough data to be sure of what was going on until just now." Right. This is the chairman of statistics. "Anyway, I asked one of the geostatisticians to look into it--"

"Geostatisticians..."

"--yes, and she's produced a map showing the radius within which we can send email to be slightly more than 500 miles. There are a number of destinations within that radius that we can't reach, either, or reach sporadically, but we can never email farther than this radius."

"I see," I said, and put my head in my hands. "When did this start A few days ago, you said, but did anything change in your systems at that time?"

"Well, the consultant came in and patched our server and rebooted it. But I called him, and he said he didn't touch the mail system."

"Okay, let me take a look, and I'll call you back," I said, scarcely believing that I was playing along. It wasn't April Fool's Day. I
tried to remember if someone owed me a practical joke.

I logged into their department's server, and sent a few test mails. This was in the Research Triangle of North Carolina, and a test mail to my own account was delivered without a hitch. Ditto for one sent to Richmond, and Atlanta, and Washington. Another to Princeton (400 miles) worked.

But then I tried to send an email to Memphis (600 miles). It failed. Boston, failed. Detroit, failed. I got out my address book and started trying to narrow this down. New York (420 miles) worked, but Providence (580 miles) failed.

I was beginning to wonder if I had lost my sanity. I tried emailing a friend who lived in North Carolina, but whose ISP was in Seattle. Thankfully, it failed. If the problem had had to do with the geography of the human recipient and not his mail server, I think I would have broken down in tears.

Having established that -- unbelievably -- the problem as reported was true, and repeatable, I took a look at the sendmail.cf file. It looked fairly normal. In fact, it looked familiar.

I diffed it against the sendmail.cf in my home directory. It hadn't been altered -- it was a sendmail.cf I had written. And I was fairly certain I hadn't enabled the "FAIL_MAIL_OVER_500_MILES" option. At a loss, I telnetted into the SMTP port. The server happily responded with a SunOS sendmail banner.

Wait a minute... a SunOS sendmail banner? At the time, Sun was still shipping Sendmail 5 with its operating system, even though Sendmail 8 was fairly mature. Being a good system administrator, I had standardized on Sendmail 8. And also being a good system administrator, I had written a sendmail.cf that used the nice long self-documenting option and variable names available in Sendmail 8 rather than the cryptic punctuation-mark codes that had been used in Sendmail 5.

The pieces fell into place, all at once, and I again choked on the dregs of my now-cold latte. When the consultant had "patched the server," he had apparently upgraded the version of SunOS, and in so doing downgraded Sendmail. The upgrade helpfully left the sendmail.cf alone, even though it was now the wrong version.

It so happens that Sendmail 5 -- at least, the version that Sun shipped, which had some tweaks -- could deal with the Sendmail 8 sendmail.cf, as most of the rules had at that point remained unaltered. But the new long configuration options -- those it saw as junk, and skipped. And the sendmail binary had no defaults compiled in for most of these, so, finding no suitable settings in the sendmail.cf file, they were set to zero.

One of the settings that was set to zero was the timeout to connect to the remote SMTP server. Some experimentation established that on this particular machine with its typical load, a zero timeout would abort a connect call in slightly over three milliseconds.

An odd feature of our campus network at the time was that it was 100% switched. An outgoing packet wouldn't incur a router delay until hitting the POP and reaching a router on the far side. So time to connect to a lightly-loaded remote host on a nearby network would actually largely be governed by the speed of light distance to the destination rather than by incidental router delays.

Feeling slightly giddy, I typed into my shell:

$ units
1311 units, 63 prefixes

You have: 3 millilightseconds
You want: miles
    * 558.84719
    / 0.0017893979

"500 miles, or a little bit more."

Tiny House: The Story Behind the Story

This past Tuesday, I was tickled when a local news TV station asked me to share my thoughts about a 200 sq ft furnish shed which was being offered for rent for more than $1,000/month.



Throughout this week, my interview was syndicated to about 100 news outlets. Friends and colleagues, some of whom I hadn't had contact with for many years, reached out to me to say that they saw it on CNN or San Jose Mercury News to name a few.

The Story Behind the Story

About six years ago, I learned that the story, behind the story, is sometimes as interesting as the story itself. Friends who contacted me wanted to know how I ended up on the news. Unlike last time, I did not contact the news. I was simply walking home from a local bakery, with some fresh bread, when I saw a guy recording himself on a video camera. Initially, since he was standing in front of a house with a "For Sale" post, I thought he was a real estate agent.

As I walked by, he asked me if I knew about the shed that was being offered for rent for more than $1,000/month. I told him that I heard about it, a day earlier, when the posting went viral on Reddit. He then introduced himself and asked if he could interview me for his story. Of course, I said yes. He placed a mic on my shirt, turned on the camera, and stood next to it while we had a casual conversation. And then, viola, I was on the Channel 10 Six O'Clock News.

Security Quotes

My favorite security quotes from Bruce Schneier.

Security, when it is working, is often invisible not only to those being protected, but to those who plan, implement, and monitor security systems.

Every one of us, every day of our lives, makes security trade-offs. Even when we’re not thinking of threats or dangers or attacks, we live almost our entire lives making judgments about security, assessments of security, assumptions regarding security, and choices about security.

Security is both a feeling and a reality. We’re secure when we feel protected from harm, free from dangers, and safe from attack. In this way, security is merely a state of mind. But there’s the reality of security as well, a reality that has nothing to do with how we feel. We’re secure when we actually are protected.

Security is always a trade-off, and to ignore or deny those trade-offs is to risk losing basic freedoms and ways of life we now take for granted.

Perfect security is impractical because the costs are simply too high; we would have to treat the whole world as a threatening place and all the people in it as evildoers, when in fact the real threats are not nearly so pervasive. We’d have to create an extremely oppressive regime. But freedom is security. Openness is security. If you want proof, look around you. The world’s liberal democracies are the safest societies on the planet. Countries like the former Soviet Union, the former East Germany, [former] Iraq, North Korea, and China tried to implement large-scale security systems across their entire populaces. Would anyone willingly trade the dangerous openness of the U.S. or most countries in Europe for the security of a police state or totalitarian government?

All security is, in someway, about prevention.

Security is about preventing adverse consequences from the intentional and unwarranted actions of others.

Protecting assets from unintentional actions is safety, not security.

Technology is generally an enabler, allowing people to do things. Security is the opposite: It tries to prevent something from happening, or prevent people from from doing something, in the face of someone actively trying to defeat it.

Five step process to analyze and evaluate security systems, technologies, and practices.
1. What assets are you trying to protect?
2. What are the risks to these assets?
3. How well does the security solution mitigate those risks?
4. What other risks does the security solution cause?
5. What costs and trade-offs does the security solution impose?

A threat is a potential way an attacker can attack a system.
Risk[: to] take into consideration both the likelihood of the threat and the seriousness of a successful attack.

Risk management is about playing the odds. It’s figuring out which attacks are worth worrying about and which ones can be ignored.

Threats determine the risks, and the risks determine the countermeasures.

Insurance ... allows a store to take its risk and, for a fee, pass it off to someone else. It allows the store to convert a variable-cost risk into a fixed-cost expense.

People underestimate risks they willingly take and overestimate risks in situations they can’t control.

In America, automobiles cause 40,000 deaths every year; that’s the equivalent of a full 727 crashing every day and a half - 225 total in a year. As a society, we effectively say that the risk of dying in a car crash is worth the benefits of driving around town. But, if those same 40,000 people died each year in fiery 727 crashes instead of automobile accidents, you can be sure there would be significant changes in the air passengers systems.

People make security decisions based on perceived risks instead of actual risks.

More people are killed every year by pigs than sharks, which shows you how good we are at evaluating risk.

Security systems are never value-neutral; they move power in varying degrees to one set of players from another.

Sometimes it seems those in charge - of governments, of companies - need to do something in reaction to a security problem. Most people are comforted by action, whether good or bad.

At the most basic level, a system is a collection of simpler components that interact to form a greater whole. A machine is is a simple thing, even though it may have different pieces. A hammer is a machine; a table saw is a system. A pulley is a machine; an elevator is a system. A tomahawk is a machine; a Tomahawk cruise missile is a complex system.

The only reliable way to measure security is to examine how it fails - in the context of the assets and functionality it is protecting.

If you can think about security systems in terms of how individual failures affect the whole, you’ll have gone a long way to understanding how security works.

Security usually fails at the seams - at the points where two systems interact - seams between security systems and other systems, seams between parts of a security system.

Security systems can fail in two completely difference ways. The first way is that they can fail in the face of an attack. The door lock fails to keep the burglar out, the airport face-scanner fails to identify the terrorist, or the car alarm is bypassed by a thief. These are passive failures. The system fails to take action when it should. A security system can also fail by doing what it’s suppose to do, but at the wrong time. The door lock successfully keeps the legitimate homeowner out, the airport face-scanner incorrectly identifies an honest citizen as a terrorist, or the car alarm rings when no one is trying to steal the car. These are active failures: The system fails by taking action when it shouldn’t.

The most common security mistake of all is to expend considerable effort combating outsiders while ignoring the insider threat.

A terrorist is someone who employs physical or psychological violence against noncombatants in an attempt to coerce, control, or simply change a political situation by causing terror in the general populace.

The U.S. government has tried to address it [the 9/11 Attacks] by demanding (and largely receiving) new powers of surveillance and data collection. This completely misses the point. The problem isn’t obtaining data, it’s deciding which data is worth analyzing and then interpreting it. So much data is collected - organizations like the NSA suck up an almost unimaginable quantity of electronic communications, the FBI gets innumerable leads and tips, and U.S. allies pass along all sorts of information - that intelligence organization can’t possibly analyze it all.

Basically, there are three ways to authenticate someone: by something he knows, by something he has, and by something he is. All these ways have been used from prehistory until the present day, and they all have different security properties and trade-offs.

When the city of London began putting up house numbers and street signs in the 1760s, people rioted because they didn’t want strangers to be able to navigate through their neighborhoods.

A security protocol is a series of steps that some trusted person carries out, steps designed to enforce some sort of security rules.

Like protocols, procedures are steps that a trusted person carries out. But in security lingo, procedures are exceptions; they’re the things that people do when a security event occurs.

Protocols are the routines trusted people follow day to day; procedure are what they do in response to an anomaly.

Sensible security does not result from fear. Just because anomalies happen doesn’t mean security has failed. The risk of a terrorist attack before 9/11 wasn’t appreciable smaller than the risk of a terrorist attack after 9/11. Before 9/11, European countries mostly had an accurate assessment of their risks. In the U.S., the risks were largely underestimated; many people thought it couldn’t happen there.

To summarize: Prevention is impossible. Mitigation is important. Intelligence and counterattack are critical. And none of this is as effective as addressing the root causes of terrorism.

Spending more money on intelligence and investigation is far more cost-effective, because it targets the attackers, rather than waiting for the attackers to come to the defensive systems.

When you examine the details, only two effective antiterrorism countermeasures were taken in the wake of 9/11: strengthening cockpit doors and passengers learning they need to fight back. Everything else - let me repeat that:  everything else - was only minimally effective, at best, and not worth the trade-offs.

The color-coded threat alerts issued by the Department of Homeland Security are useless today, but may become useful in the future. The U.S. military has a similar system; DEFCON 1-5 corresponds to the five threat alerts levels: Green, Blue, Yellow, Orange, and Red. The difference is that the DEFCON system is tied to particular procedures; military units have specific actions they need to perform every time the DEFCON level goes up or down. The color-alert system, on the other hand, is not tied to any specific actions. People are left to worry, or are given nonsensical instructions to buy plastic sheeting and duct tape.

There’s no way to prevent all future terrorist attacks.

Ironically, the the two years since 9/11, we’ve got the security level mostly right but the costs wildly wrong. The security we’re getting against terrorism is largely ineffective, although it’s probably commensurate with the minimal level of risk that actually exists.

Pundit after pundit has talked about the balance between privacy and security, discussing whether various increases of security are worth the privacy and civil liberty losses. The discussion seems odd to me, because linking the two is just plain wrong.

Security and privacy, or security and liberty, are not two sides of a teeter-totter.

Arming pilots, reinforcing cockpit doors, and teaching flight attendants karate are all examples of security measures that have no effect on individual privacy or liberties.

Unfortunately, the Department of Homeland Security is far more likely to increase the country’s vulnerability to terrorism. Centralizing security responsibility will create a commonality of approach and a uniformity of thinking; security will become more brittle. Unless the new department distributes security responsibility even as it centralizes coordination, it won’t improve the nation’s security.
The dual requirements that security decisions need to be made as close to the problem as possible, and that security analysis needs to happen as far away from the sources as possible make the problem subtle. Security works better if it is centrally coordinated but implemented in a distributed manner.

If you think technology can solve your security problems, then you don’t understand the problems and you don’t understand the technology.

Antique Row's Last Shop

One does not simply walk into Antique Row's last remaining antique store, in North Park, without receiving an appropriate history on a purchase.

Last summer, I stopped by Zac's Attic and bought a couple demitasse spoons. Last week, when I made espresso, I noticed that I didn't have the proper cups to serve it in.

As I drove past Zac's Attic, I stopped in and immediately saw a fine looking set of espresso cups. When I brought them to the counter, the proprietor who's owned the business for 40 years, Dave McPeeters, told me about the century old cups.

The items I bought were from the Palace Hotel, in San Francisco, complete with the hotel's logo on the Buffalo China cups. The Palace Hotel is know for where opera tenor, Enrico Caruso, performed as Don Jose, in Carmen, the night before 1906 San Francisco Earthquake, which gutted the hotel.

Today, the hotel houses the opulent Palm Court Restaurant, which looks like a palace; it's also know for its Pied Piper Bar with its tiled floor and historic artwork behind the bar.

And, now, I get to sip espresso, with anisette, from this little piece of history.

Apple Card Credit Card Signup

I signed up to receive an Apple Card, today. The entire process, from applying, through approval, to receiving the new card in my Apple Wallet took less than ten minutes.

Once I received it in my wallet it was active to use with Apple Pay. I also requested to have the physical titanium card sent to me which should take about a week.

Customer support via text message

I was wondering if the Apple Card had a virtual account number similar to my Citi Mastercard. There was a button to call or text for help which was great. Nothing better than getting customer support via text message – no wait or hold time.

Unfortunately, the Apple Card can only generate one number at a time which can be easily changed by pressing a button. So, at this time, it seems that I can only have a single credit card number active with Apple Card.

Another disappointment was that Apple Card interest rate depends on credit worthiness, somewhere between 12.99% - 23.99%. My FICO score is over 810, so I would have hoped for something lower than 17.99%. But it's been more than a decade or two since I've carried a balance from, month to month, on my credit card. So, this shouldn't be an issue for me.

A nice thing about the Apple Card was that I could tap one button any all of my Apple bills (iCloud, iTunes, etc) were switched to my Apple Card which returns 3% cash, daily. Let's see how it works.

It Can Always Be Worse

Noon meal formation at the US Naval Academy.

Plebe Summer, at the Naval Academy, is about sacrifice. It's about learning how to deal with failure as a team. Even though I knew it was a training environment, it didn't make it any less stressful. We were constantly tested on more knowledge than anyone could learn. That was the point – learning how to deal with failure while being part of a team. Sometimes... many times... that meant sacrificing your comfort, enjoyment, or pleasure in order to help someone else suffering worse than you. As Plebes, we were all in it together.

We learned what it meant to have honor. We learned to never sacrifice others for our own well being, even when we thought we'd never get caught. This isn't something you do sometimes or most of the time, it's something you do all the time. It only takes one dishonorable act to forever taint your honor. We were taught a lot, at the Academy, even when it's okay to lie.

Squash Practice

The Naval Academy challenges all of us in three key areas: military discipline, academics, and sports (all Midshipmen are required to play a sport).

During Plebe Summer, I was playing squash. Every afternoon, we went to our sport. Our squash practice consisted only of Plebes, and an old, retired, Navy captain who was our coach.

One Plebe, who was struggling to learn his required knowledge, didn't practice squash; instead, he'd sit up against the wall of a squash court and study his handbook. I could tell, by looking at him, he was stressed out.

After a couple days in a row, the squash coach told him to put down his book and play squash, which he reluctantly did. The next day, he was back at studying instead of playing squash. The squash coach snapped at him to put down his book and practice.

"Put down your damn Reef Points, pick up a racquet and get on the court," said the coach to the overstressed Plebe.

The Plebe appealed to the coach, thinking that this gentle old man would understand his predicament since he had once been a Plebe in the 1950s.

"Sorry, sir, I'm very stressed out," said the Plebe as a group of us watched the exchange. 

The squash coach's patience had run out and he shot back, "Try spending six years in a POW camp and see how that stresses you out." Then he walked off.

Our eyes were as wide as saucers. This old squash coach had been a pilot, who was shot down in Vietnam, and spent six years as a POW. We scattered like roaches, onto the squash courts, and resumed playing squash. In that one sentence, we realized the indelible lesson that it can always be worse.

Yesterday, I called 911 – it was a mistake.

Yesterday, I was at Balboa Park during a parade which had a couple hundred thousand attendees. Whenever that many people get squeezed into a tight venue, such as a stadium, cell phone connectivity becomes an issue – and yesterday was no exception.

At one point, I launched an app on my iPhone which was taking a long time to open up since it needed Internet access. Just as I closed the app it popped up three modal dialog windows in a row. The app was closed, but the modal windows blocked all other iOS interactions as if the phone was frozen.

Restarting the iPhone

To restart the iPhone, all I needed to do was press the volume up, volume down, and then the sleep/wake button. This brings up a slider at the top of the screen, "slide to power off." But, the problem was this slider was blocked by the modal window.

I tried pressing all three buttons and holding them down. Unfortunately, this activated the phone's emergency mode which dialed 911 and then sent out SOS text messages to my emergency contacts with a map of my location. The police called me back with an automated message stating that they received a hangup call but couldn't pinpoint my location. However, even though my phone had location services turned off, it seems the phone was able to know my location and sent a pin-drop to my emergency contacts along with an SOS message.

Force Restarting the iPhone

What I needed to do was force restart my iPhone, but I didn't know how. Luckily, my buddy had his iPhone (also with spotty connectivity) and we did a Google search. A force restart is similar to a typical restart, except you hold down the sleep/wake button until you see the Apple logo:

1. Press the volume up button.

2. Press the volume down button.

3. Hold down the sleep/wake button until you see the Apple logo (5-10 seconds).

Jesta.io: Like Uber for Dining at a Restaurant

Today, I had lunch with the founders of Jesta. I have been wanting a restaurant customer experience like Jesta for years. It's like Uber for dining in at a restaurant. 

I simply pointed my iPhone at the QR code on the table and the Jesta app displayed the current menu for the restaurant, with photos of the items I could order. I tapped on each item and then sent my order to the kitchen, along with my payment and tip.

The best part is there was no need for me to wave down the food server when I was done eating. I could simply leave since my bill was already settled. Simple!

It all worked exactly expected. Jesta is in about half a dozen restaurants in San Diego, and growing. I hope they're able to raise a round of funding to put them over the top for marketing. Innovation at its best.

Where to Open a Restaurant?

I was talking to a savvy restaurateur who told me how he decided on the location to open an Indian restaurant in a particular neighborhood in San Diego.

He found several viable locations for his Indian cuisine. The problem was, how could he know which one was the best? So, he tested the market. For each potential location, he created his restaurant menu and direct-mailed it to the surrounding neighborhoods. Each menu had a different phone number to call, with the address of the potential restaurant. He immediately started getting calls from people who loved Indian food and were grateful that one was opening nearby. After a several weeks, he had enough calls to see which location had the most interest.

Testing a market, like this, is a very simple technique, but it's often the most overlooked step in the process for many entrepreneurs. Be sure to listen to your customers before you make up your mind to avoid confirmation bias. 

No Nonsense Marketing

The Marines are excellent at no-nonsense marketing. It's about being direct and setting the expectation. 


In June 1985, there was a TWA terrorist hijacking which was my Pearl Harbor moment. That was the moment when I pledged to join the Marines. I knew nothing about the military; not even the difference between the enlisted and officer ranks. But I wanted to do my part to make a difference. 

The local recruiting office housed all four military services. The Marines' office was in the back, so I had to pass by the Army, Navy, and Air Force offices on my way. As soon as I walked in the front door, a soldier stopped to offer me help.

"I'm looking to join the Marines or something," I said, shrugging my shoulders as I said the last word.

"Or something? Have you considered the Army?" he asked as he guided me into his office. He could tell I was looking for a challenge so he fired up a 12" LaserDisc to show me exciting clips of Ranger and Airborne training. For about two hours, that afternoon, this Army recruiter told me about what the Army could be. He convinced me take the ASVAB military entrance exam, later that week.

After we finished, I left the Army office and headed to the Marines' recruiting office where I met SSgt Meehan; a Marine I remained in touch with to this very day. The SSgt, who, at 27 years of age seemed to have the wisdom and experience of a senior citizen. He sat me down next to his desk, lit his pipe, and said, "I don't have any fancy LaserDiscs to show you videos. At this point, I have no idea what you're qualified to do, so I can't make any promises. First you need to take the ASVAB. Before you do that, you have to take my 30 minute practice exam."

SSgt Meehan led me to a small room where a couple other potential recruits were taking exams. I don't recall the details of the exam, but it wasn't too difficult. When I completed it, the SSgt reviewed my answers and told me that we could proceed to official ASVAB as soon as he could schedule it.

"Can the same ASVAB exam results be used for all the military services?" I asked the SSgt.

"Yes."

I explained to him about my soft commitment with the Army.

"If you want to be a Marine then I would like you to schedule that test with me," replied SSgt Meehan.

As I headed out of the building, I stopped by the Army's office and gently backed out of my ASVAB commitment.

"I can tell," said the soldier I had spent two hours with, earlier that day. "You're gonna be a Marine."

In my mind, I was committed to joining the Marines and the SSgt's direct and practical approach was the icing on the cake.