Rolling Out a New Version of Your Website

Tricks I Learned at Apple: Steve Jobs Load Testing is an excellent precursor to this post.

When launching a completely new version (update) of a website, it's best to have a rollout and a rollback plan. Very few brand new websites will have the problems that HealthCare.gov had in 2013 because new websites typically start with zero traffic. HealthCare.gov was a unique case since it went from zero to millions of users, overnight. 

Typically, as a website grows, servers will be added and optimized to handle the additional traffic. But, if growth happens too quickly, then the company can prevent new users from creating new accounts on the website while they manage their growth and scale up their infrastructure. Facebook was able to manage their growth by rolling out across college campuses, one at a time, whereas Twitter had no way to control their growth since they were open to the public, resulting in the fail whale. Again, these are rare cases; the typical problem with websites occur when rolling out a major update.

Rolling out the New Website Version

While growing from zero to millions of users is a high quality problem, it's actually very rare. A more likely problem is encountered when an entirely new version of a website is rolled out since it will probably have critical bugs or scaling issues. 

When I worked at Apple and Wyndham, we had to handle both bugs and scaling issues. At Apple, we switched from using RDBMs to memory caches for read-only data. At Wyndham, we had to roll out more than a dozen different websites at once for brands like Days Inn, Ramada, Howard Johnson's, Super 8, Hawthorn Suites, etc.

Managing Risk

Initially, Wyndham wanted to switch from the old website to the new one, all at once. My boss, who's a particularly sharp guy, had enough experience to immediately recognize the risk of doing this. Specifically, what if the new website was broken (what if it had too many bugs, preventing customers from booking rooms)? Instead, he suggested a very simple plan. Rather than making the switch, overnight, he suggested we keep the old version of the website running while rolling out the new website over the course of a week or so.

Since both the old and new versions of the website talked to the same database, it was a simple process, at a high level. We'd have an all-hands meeting, on Monday morning, in our war room (dedicate conference room). During Monday's meeting, all of the departments (marketing, product management, development, and QA) would give a thumbs up to move forward. Then, we'd have our load balancers begin to randomly send 1% of the traffic to the new version of our website. We'd place a cookie on the customer's browser so, if they came back later, they'd automatically be directed to the new version of the website otherwise they'd end up the old version. 

Staging the Rollout 

Just before the close of business on Monday, we'd meet again to confirm that everything was running as expected. On Tuesday morning, we'd meet and give a thumbs up to increase the traffic to the new website to 5%, etc. It looked like this:
Monday: 1%
Tuesday: 3% – 5% (based on Monday's performance)
Wednesday: 10%
Thursday: 50%
Friday: 100%

The beauty of starting at 1% and then 3 % – 5% is that's the most revenue you'll risk losing (in theory) if something goes wrong.

By using this week-long rollout process, we all kept our jobs. I only recall one time, when there was a major bug, that we had to stop after the first day or two, which wasn't a big deal; we simply sent all traffic to the old website while the new one was fixed and we got it right on our next rollout.

Interesting Apple ID Issue

Problem

A friend called me this morning because he was having an issue with apps on his iPhone. After resetting his Apple ID password, the App Store wouldn't authenticate him so he couldn't update any apps on his iPhone. My first suspicion was, since this problem happened soon after he updated to iOS 12.1.3, there was a bug preventing his credentials from propagating to all of his devices.

Solution

The actual problem is my buddy originally downloaded some apps with a different Apple ID than the one he's currently using on his iPhone. The apps continued to run fine until it came time to update them. The solution was simple. The current apps only needed to be deleted on his iPhone and then re-downloaded. An easy fix, especially since all of his apps were free. If his apps weren't free, it would have required a bit of help from Apple's billing department.

Lesson

The lesson learned is be very cautious of changing Apple IDs. I've been using the same one since I worked at Apple when Steve Jobs introduced iTools and every employee was automatically assigned our original Apple ID. The second lesson I learned many years ago is if you've ever worked for Apple then you will be the first call that a friend or family member makes for Apple tech support. A calling that I do enjoy. 

Engineers Turned Entrepreneurs, Part 3

The more I mentor engineers-turned-entrepreneurs, the more I've noticed it requires the proper attitude, more so than the raw skills. I call it the entrepreneur's attitude. When starting off, it's OK if a new entrepreneur doesn't know a whole lot about startups, but they do need to be coachable without being overly impressionable. 

When I speak with wannabe entrepreneurs, who are coming from an individual contributor background, I frequently quote Steve Jobs's comments from WWDC '97.

You got to start with the customer experience and work backwards to the technology. You can't start with the technology and try to figure out where you're going to try to sell it. And I've made this mistake probably more than anyone else in this room and I've got the scar tissue to prove it.

I see two key parts to the entrepreneur's attitude that are important.

The first key part of the entrepreneur's attitude is they need to be focused outward, on customers, and think in terms of benefits before features. Don't lead off with your wants (i.e. I want our company to be the best at blah, blah, blah... save that pitch for investors.). Instead, lead off with the benefits you provide to your customers. Try to eliminate words like "I" and "we" in your pitches and marketing.

The second key to a successful entrepreneur's attitude is recognizing and embracing opportunity. I recently had a friend from NY stay with me at my home. He's made millions of dollars selling companies he founded and ran. Interestingly enough, he doesn't consider himself an entrepreneur. Rather, he prefers to be labeled as a software developer. Regardless of his title, he is constantly seeking new experiences, knowledge, and opportunities. His default position, when experiencing something new, is to immediately investigate it and give it a try.

Opportunities can be found most anywhere. Many times, opportunities first present themselves as uninvited inconveniences. With the federal government currently shut down, some people are seeing it as an opportunity.

Engineers need to think like entrepreneurs. As Steve Jobs said, begin with the customer experience and then work your way backwards.

1. Sales: How and where will your customers acquire your offerings?

2. Marketing: How will customers learn about your product?

3. Development: How do you know what features to put into your product that will benefit your customers?

Many entrepreneurs will fumble #3, from the get-go. They'll either fail to get feedback from potential customers or they'll try to bake every possible feature, under the sun, into their product. To deal with the former challenge, I recommend following the lean startup methodology. For the latter, I recommend a notional press release.

Part 1 in this series:
http://blog.joemoreno.com/2018/08/engineers-turned-entrepreneurs.html 

Part 2 in this series:

http://blog.joemoreno.com/2018/12/engineers-turned-entrepreneurs-part-2.html

Two Funny Bird Stories

Tiny Bird Nest

Bird is a dockless electric scooter-sharing company that's been around just over. In 2018, Time magazine listed Bird as one of its 50 Genius Companies. 

Bird has an innovated business model by paying most anyone to recharge the scooters. Once they're recharged, Bird directs where the scooters should be dropped off, called a Bird Nest.

I spoke with one person who tried to use a Bird scooter, for the first time, but couldn't figure out where to insert the money. That's funny once you realize that the Bird process is exactly the same as Lyft and Uber in that you use an app to begin, end, and pay for a ride (Bird charges $1 to unlock the scooter and 15¢/minute).

Another person told me an interesting situation she ended up in when the battery died on the smartphone that was used to start the ride. The smartphone app is how a scooter ride begins and ends, so, with a dead smartphone there was no way to end the ride. Solution: Load the the Bird into a car, drive home, recharge the phone, and end the ride. 

Purchasing an Airplane

A Marine buddy put me in touch with a former Air Force C-5 Galaxy pilot who's interested into purchasing a plane for himself. He wanted to know my experience, so here is the crux of it...

I own a 1966 Cessna 182 Skylane that I bought in 2011 (I learned to fly in 2010).

For starters, when looking for a plane, checkout Controller.com. It is one of the best ways to find a plane for sale. When purchasing a plane, obviously costs are one of the biggest issues.

Since I bought an older plane, it was one tenth the cost of a brand new plane from the factory (I then added in avionics to turn it into a glass cockpit panel).

You’ll want to know how many hours are on the engine since its last major overall to compare costs. For example, two of the same make/model planes for sale, with the same number of flight hours on the airframe, might cost very different amounts if one plane has 100 hours since its last major overall and the other has 1,700 hours since it last major overhaul. Piston engines typically last 1,800 – 2,100 hours before needing an overhaul. And pricing is straight forward. For example, a new engine might be worth $40,000, and at 1,000 hours, it would be worth $20,000, at 1,500 hours it would be worth $10,000, etc. (i.e. I’m figuring $10,000/500 hours of use).

Annual inspections will be your biggest cost. Over the years, my annual inspections have run anywhere from about $2,500 to just over $20,000 when I had a cylinder overhauled and my propeller and windscreen replaced. Keep in mind with a twin engine plane, you’ll be paying close to 1.5x - 2x more for your annual inspections since two engines need to be inspected.

Ideally, you’ll want an aircraft that has all of its logbooks back to the beginning, when the plane shipped from the manufacture. This will tell you about issues and accidents. My plane was in two crashes/hard landings around 1988 and 1992 which have never been an issue for me. My opinion is if a plane was in an accident many years ago, and it’s been flying regularly since then, then all should be well. I would be very hesitant to buy a plane that was in an accident within the past couple years for fear of an unknown surprise.

Don’t forget about partial ownership, which is very common. Rather than purchasing the plane yourself, you could go in on it with 1 - 3 others and split the purchase costs and maintenance.

When I found the plane I wanted to buy, I took it to a maintenance shop to conduct a pre-sales inspection. A couple issues came out of that inspection which gave me the ability to further negotiate down the sales price.

After buying my plane, I had to pay use tax (sales tax) and an annual property tax. In San Diego, I pay about $550/year in property tax on the plane.

Surprisingly, insurance isn’t required for owning or operating an airplane as a private pilot (part 91). I got my insurance through AOPA and I pay about $1,500/year for a “smooth million” of insurance across the board (I’ve never needed it, so far).

These are some of my thoughts, off the top of my head.

Needed: 55 - 60 Person Venue in Silicon Valley

Apple Park

Ask

I'm looking for a presentation venue in Silicon Valley that can hold 55 – 60 people. An ideal venue would be a college classroom with A/V.

Details

For nearly four years, I have been giving my speaking engagements in Silicon Valley. My talk focuses on Apple's design and marketing philosophy and why that makes Apple different.

The venue I've used for my presentation is no longer available. I found a similarly priced venue, but it only holds about 40 people. So, now, I need a new space which could be any professional location such as a large conference room, classroom, auditorium, etc. Do you know of a space that's available which isn't publicly advertised?

Requirements

1. Seat 55 – 60 people in a classroom setup with a screen and laptop projector.

This great venue, that I used for years, is no longer available

2. Located within 15 – 20 minutes of Cupertino for our visit and warm welcome at Apple Park.

A group photo during my group's warm welcome at Apple Park

3. Competitively priced with my previous venue ($200 – $250 per 1/2 day).

I had a great time presenting to billionaire Lu Junqing
and his philanthropic daughter, Jennifer.
Since I'm not a billionaire, I need an inexpensive venue.

Caveat

Over the years, I have looked at many venues near Cupertino, so I'm aware of hotel ballrooms and salons, but their costs can easily be 3x – 5x what I'm currently paying after adding in service fees, tourism and city taxes, A/V equipment rental, setup costs, etc. So, please let me know if you're aware of any large space that typically remains unused since that might be a good fit.


Plan B
What if I can't find a Silicon Valley venue that meets my requirements? Well, then, it's on to Plan B, which isn't all that bad... a full-time job in tech, ideally located in San Diego.
This is me: http://joemoreno.com.

True Talent

True talent is a force multiplier that can’t be taught. Talented people aren’t driven by discipline. They are driven by passion and love. Discipline can be a part of it (Tiger Woods has to practice) but it's the talent that's most important.  

As kids, we’re taught that skills are what we need to success in life, and that’s correct for the average person. But truly talented people are the ones who earn millions of dollars.

In 1996, the Chief of Naval Operations, who’s the most senior member of the U.S. Navy, committed suicide. The Navy didn't skip a beat. Michael Jackson dies, and hundreds of millions of dollars are lost in concert revenue. That's the difference between talent and skill.

Happiness: The Unalienable Right

In September, I spoke at a memorial service for my fallen Naval Academy classmates. While writing down my thoughts, I speculated what our dearly departed would want for those they left behind. My conclusion was happiness.

USNA 1993 Reunion Brunch Following Our Memorial Service.

As my first piece of 2019, I thought it appropriate to talk about happiness this New Year's Day. It may seem like a minor thing, but it is an unalienable right proclaimed in the Declaration of Independence.

We have life. We have liberty. Those rights were given to us. But it's up to us to pursue our individual happiness.

We all want to be happy. But the trick is figuring out how to achieve it. I've spoken a lot about simplicity, but that pertains to things like systems and products, not people.

While the recipe for happiness is simple, it does require some focus and attention.

Meaning

A life of happiness begins with making meaning, which is a very personal process. Making meaning and being happy requires a few things.

1. Belonging

In order to belong, you'll need candid relationships with others where you can be yourself, not your beliefs.

2. Purpose

Purpose is simply using your strengths to serve others. While a person can have multiple purposes, it's a personal choice regarding which ones to pursue.

3. Transcendence

Transcendence is simply something that lifts you to a higher calling. In its basic sense, transcendence is an existence or experience beyond typical. I don't mean it to represent metaphysical, paranormal, or supernatural.

A higher calling is something that drives a person beyond what's typical, due to their devotion to duty or expectation. It could be writing, religion, military, medicine, parenting, etc. It involves giving up personal gains for the greater good. As one example, it could be pro bono work like open-source coding or free legal advice.

4. Storytelling

Storytelling is the story you tell yourself, about yourself (and, perhaps, others). The beauty of telling your story is that you are the author and you can edit and change the story as you live it.

Armed with this knowledge, I encourage you to go out and perform all manner of things thereunto pertaining in order to be happy and live a meaningful life with a health dose of love.

Carpe diem.

⁂

The following is my memorial service speech.

U.S. Naval Academy Chapel.

We are here today to remember our classmates who are no longer with us.

SQUEEZE
And we are reminded that they each had to squeeze their entire life into a shorter period of time than we have been given. We’ve outlived them.

SAD
We sit here and allow ourselves to be sad.
And that’s OK.
We cannot separate our memory of them from the empty sadness it brings us.
To do otherwise would not be human or compassionate.
It’s OK to be sad.

INSPIRE
But we didn’t come here, this morning, to only be sad as we remember them.
When we look back at their lives, it should inspire us to enjoy our own life more.
It should remind us to live in the present.
To enjoy the moment.
To enjoy the simple things that we encounter every day.

DASH
We know the date our fallen classmates were born.
And we know the date that they left us.
And that their entire life;
All of our lives;
Is represented by that dash in between those two dates that define us.

LIFE IS SHORT
It's not only that life is so short, but also that we’re dead for so long.
So, what advice might our fallen classmates give us, today, after we leave our reunion and go home, back to our daily routines?

HAPPINESS
And my answer is happiness.
Whatever makes you happy while maintaining a responsibility to the long-term.
We don’t pay enough attention to our own happiness.
But it’s important.
We shouldn’t forget that our country was built for it, literally.
Life, liberty, and the pursuit of happiness.
We have life.
We have liberty.
It’s up to each of us to pursue our happiness.

So, we remember our fallen classmates, today, with fondness, respect, and love; and with the sadness that they left us too early. And, as I mentioned earlier, it’s okay to allow yourself to be sad, this morning, and then pursue your own happiness while we live the dash between the two most important dates that define our lives.

Thank you and carpe diem.