“Go to ABC University. Though the fee is a little expensive, you’re guaranteed a great job after that. Just look at Rajesh. He has got such a great job at XYZ Company after university.”

We often hear these kinds of statements in our day to day lives. We should always be wary of such statements and check if they are nothing but a case of survivorship bias.

Survivorship bias is a logical fallacy that results in people focusing on successful cases while overlooking unsuccessful cases, usually because of their lack of visibility. No one likes to focus on a loser, right?

Forget about going to university, don’t we also hear stories about how college drop-outs like Mark Zuckerberg and Steve Jobs went on to become extremely successful multi-millionaires. This article by The Atlantic beautifully illustrates the dangers in believing these seductive stories. Here’s an extract,

“Like any myth, this story has a kernel of truth: There are exceptional individuals whose hard work, determination, and intelligence make up for the lack of a college degree. If they could do it, one might think, why can’t everybody? Such a question ignores the outlier status of these exceptional drop-out entrepreneurs and innovators. Those who are able to achieve such success often rely on a set of skills already developed before they get to college….But what happens to young people without access to these important resources? For them, skipping college to pursue business success is like investing their savings in lottery tickets in the hopes they will be a multimillion-dollar winner… The reality is that the next college dropout will not be LeBron James, James Cameron, or Mark Zuckerberg. He will likely belong to the millions of college drop-outs you don’t hear the press singing about.”

 

In Economics and Finance

Survivorship bias can lead to severe flaws not just in our day to day logic but also in economic and financial analyses. Ajay Shah noted Indian Economist, on his blog, points out how people tend to over-estimate corporate earnings by using the current crop of current Nifty 50 companies as a proxy for Indian corporations. His logic is simple – the index undergoes periodic revisions where some companies are dropped out while some new ones are instated. Since the new companies usually tend to be doing better than the ones that drop out, any analysis that looks at the latest crop of companies to analyze earnings over a long time period will tend to over-estimate earnings growth.

Similarly, studies have shown how mutual fund companies tend to over-estimate their returns over time as they drop their under-performing funds.

 

Econometric Models

We also should be careful while dealing with econometric models. Economists tend to prefer balanced panel data, as they are conducive to a clean analysis. A balanced panel data is a data set where each individual has the same number of observations through time. However, in the real world, we often get unbalanced panels. Data can be missing for various reasons, and though we are often tempted to balance the panel and delete the unbalanced individuals, we should be careful. In fact, unless the causes behind the missing observations are entirely random, we should not balance the panel. We may end up deleting valuable information.

Take for example we have a dataset of firms over a 40 year time period. We see that some firms do not have observations over the entire period as they have closed down in the interim. Now if we delete these observations we will probably end up with a selection bias as there may be certain characteristics that these firms posses because of which they had to shut down; something we will miss out in our analysis when we delete them.

As an example, I used NYU Stern’s Swiss Railways data set, an unbalanced panel with 49 companies’ observations across 13 years. I estimated the following model:

Ln_Costs = a + B*(Network) + C*(Staff) + D*(Stops) + E*(Tunnel)

I wanted to test how firms’ costs were related to the network(total length of their railway network), Staff (number of employees), Stops (Number of stations), and Tunnel (A dummy for whether their network had tunnels of a large length).

Out of the 49 firms in question, only 37 were present for the entire period. Most of the other firms had exited before the final year of the data set.

I first conducted a balanced panel analysis and got the following results:

              Estimate Std. Error t-value  Pr(>|t|)
(Intercept) 1.0546e+01 1.1908e-01 88.5607 < 2.2e-16 ***
NETWORK    3.6950e-06 2.6079e-06  1.4168   0.15719
STAFF      2.2411e-03 4.6429e-04  4.8270  1.87e-06 ***
STOPS      9.6047e-03 3.7608e-03  2.5539   0.01096 *
TUNNEL     1.9544e-01 2.9842e-01  0.6549   0.51283

 

I then conducted an analysis on the unbalanced panel, using all the companies:

              Estimate Std. Error t-value  Pr(>|t|)
(Intercept) 1.0551e+01 1.0727e-01 98.3622 < 2.2e-16 ***
NETWORK     5.1834e-06 1.8577e-06  2.7902 0.0054354 **
STAFF       1.5593e-03 2.7549e-04  5.6603 2.345e-08 ***
STOPS       1.1440e-02 3.4306e-03  3.3348 0.0009062 ***
TUNNEL      2.7512e-01 2.4753e-01  1.1115 0.2668046

 

You can see how different the results are. NETWORK was not a significant coefficient for the balanced set, while it was significant and more positive for the unbalanced set which meant that it plays a larger role in total costs. Similarly, the coefficient on STOPS and STAFF reduced from a large coefficient to a smaller coefficient. The results probably indicate that the firms that ceased to exist had large networks but not too many stops and staff hired.

To conclude, we must always be careful before falling into the trap of survivorship bias – whether it’s in our daily lives or while conducting economic analyses.

 

References :

  1. Elton E, Gruber M, and Black C (1996), Survivorship Bias and Mutual Fund Performance, The Review of Financial Studies.
    Available at: http://www.jstor.org/stable/2962224
    (Accessed 24 September 2017)
  2. Shah A (2017), Indian Corporations have Weak Earnings Growth, Ajay Shah’s Blog.
    Available at: https://ajayshahblog.blogspot.in/2017/08/indian-corporations-have-weak-earnings.html
    (Accessed 15 September 2017)
  3. Shermer M (2014), How the Survivorship Bias Distorts Reality, Scientific American.
    Available at: https://www.scientificamerican.com/article/how-the-survivor-bias-distorts-reality/
    (Accessed 29 September 2017)
  4. Survivorship Bias, Investopedia
    Available at: http://www.investopedia.com/terms/s/survivorshipbias.asp
    (Accessed 1 October 2017)
  5. Swiss Railways, Panel Data Sets, Prof W. Greene – NYU Stern.
    Available at: http://people.stern.nyu.edu/wgreene/Econometrics/PanelDataSets.htm
    (Accessed 28 September 2017)
  6. Zimmer R (2013), The Myth of the Successful College Dropout: Why it Could Make Millions of Young Americans Poorer, The Atlantic.
    Available at: https://www.theatlantic.com/business/archive/2013/03/the-myth-of-the-successful-college-dropout-why-it-could-make-millions-of-young-americans-poorer/273628/
    (Accessed 1 October 2017)