Survivorship bias and startups

Many founders new to tech entrepreneurship in 2020/2021 may not have an appropriate perception of the failure rate as they head into 2023.

Heading into 2023, the future looks grim for a lot of startups that enjoyed the aberration that was the fundraising environment of 2020 and 2021.

Abundant and cheap venture capital led to skyhigh valuations on companies in the earliest stages of market validation and repeatable growth. While fun at the time for aspiring entrepreneurs and new VC, undoubtedly the rubber would meet the road at some point, and that started to happen in early 2022.

Venture capitalist Bill Gurley summarized it best in a Tweet:

A must read Twitter thread from Bill Gurley

“An entire generation of entrepreneurs & tech investors built their entire perspectives on valuation during the second half of a 13-year amazing bull market run. The “unlearning” process could be painful, surprising, & unsettling to many. I anticipate denial.”

I would add that the 13 year bull run ended in a two year fireworks finish for startups, venture capital, and private equity in a manner that hadn’t been seen since the .com bubble days. 2022 has brought the subsequent correction, as shown in this fundraising data from law firm Cooley:

Cooley: Series D Deals Dropped 78% in Just Q3 Alone. Late Stage Has Fallen Off a Cliff

The concern that prompted this short essay is that many founders new to tech entrepreneurship may not have an appropriate perception of the failure rate. Those reliant on venture capital for survival are about to meet an unforgiving market. In this situation, having an understanding of “survivorship bias” is perhaps a helpful tool for founders in the “unlearning process” that Bill is referring to.

Here’s a quick explanation of survivorship bias courtesy of Wikipedia:

“Survivorship bias or survival bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data. Survivorship bias is a form of selection bias that can lead to overly optimistic beliefs because multiple failures are overlooked.”

An interesting case study from World War II demonstrates this “selection bias”. Bomber planes returning from missions were assessed for damage so they could be reinforced in vulnerable areas. Initially, they were reinforced where bullet holes were visible, but a statistician realized the flaw in this approach; it only considered the planes that had returned, and assumed the areas where the plane was hit were the problem areas. The answer lay in opposite of those data; they needed to consider the planes that didn’t return, and reinforce the areas that hadn’t been hit on the surviving planes.

This hypothetical pattern of damage of surviving aircraft shows locations where they can sustain damage and still return home. If the aircraft was reinforced in the most commonly hit areas, this would be a result of survivorship bias because crucial data from fatally damaged planes was being ignored; those hit in other places presumably did not survive. Martin Grandjean (vector), McGeddon (picture), Cameron Moll (concept).

Bringing this back to the context of startups: survivorship bias is the tendency to view the performance of successful startups as a representative and comprehensive sample, without regarding those startups that have failed.

I hear the “9 out of 10 startups fail” stat a lot, and I think it’s actually referring to SMBs, not startups. Startups are defined by their ability to redefine and dominate a considerable market; rapidly reaching immense scale. Tech markets are very aggressive in my experience. My guess is the fail rate number is much closer to 999 out of 1,000 than 9 out of 10.

Companies like Dropbox, Airbnb, Shopify, Twitch — these are massive outliers. If you’re only looking at the startups in the news, even the ones that are failing; you’re missing 99%+ of the startup population that never made it out of their garage office.

If you’re a founder or team member feeling down because your startup’s future is looking grim right now; push yourself, but also be kind to yourself. Remember that very few startups reach any type of successful outcome, and even strong operators are often beaten by the market.

And for those who have made the (common) mistake of tying their personal identity to their startup; if you’re feeling quite low, please talk to someone. You’re not alone, people care about you, and your startup is not your personal identity.

---