Whenever I read about Buffett and other great managers, what I tend to see all the time are things like, “xx has beaten the market y out of z years; the odds of that happening are 1 in 5,000!” or some such thing. Not too long ago, there was an article about managers with outstanding performance and the screen was based on who beat the market five years in a row, ten years in a row or something like that.

But for me, I tend not to care about that at all. In fact, I would rather invest with someone who only beat the market seven out of the last ten years but with a wider and more consistent margin than someone that beat the market ten years in a row, and only with a small margin.

So that got me thinking about what I should look at. Well, when I say that, I don’t mean that I would use this stuff to choose investment managers since I don’t really invest in funds at all. What I mean, I guess, is that if I don’t like the above ‘beat the market x out of y years’, what is a better indicator?

__Tax Digression__

But before that, I just happened to be reading the 1986 Berkshire Hathaway letter to shareholders and came across this comment about taxes. Trump is expected to do something about taxes and I heard Buffett or Dimon mention somewhere recently that any tax cut will be competed away by the market implying that it won’t make a difference to investors. Anyway, this is what he wrote about it back in 1986 after the last big tax change:

TaxationThe Tax Reform Act of 1986 affects our various businesses in important and divergent ways. Although we find much to praise in the Act, the net financial effect for Berkshire is negative: our rate of increase in business value is likely to be at least moderately slower under the new law than under the old. The net effect for our shareholders is even more negative: every dollar of increase in per-share business value, assuming the increase is accompanied by an equivalent dollar gain in the market value of Berkshire stock, will produce 72 cents of after-tax gain for our shareholders rather than the 80 cents produced under the old law. This result, of course, reflects the rise in the maximum tax rate on personal capital gains from 20% to 28%. Here are the main tax changes that affect Berkshire:oThe tax rate on corporate ordinary income is scheduled to decrease from 46% in 1986 to 34% in 1988. This change obviously affects us positively - and it also has a significant positive effect on two of our three major investees, Capital Cities/ABC and The Washington Post Company. I say this knowing that over the years there has been a lot of fuzzy and often partisan commentary about who really pays corporate taxes - businesses or their customers. The argument, of course, has usually turned around tax increases, not decreases. Those people resisting increases in corporate rates frequently argue that corporations in reality pay none of the taxes levied on them but, instead, act as a sort of economic pipeline, passing all taxes through to consumers. According to these advocates, any corporate-tax increase will simply lead to higher prices that, for the corporation, offset the increase. Having taken this position, proponents of the "pipeline" theory must also conclude that a tax decrease for corporations will not help profits but will instead flow through, leading to correspondingly lower prices for consumers. Conversely, others argue that corporations not only pay the taxes levied upon them, but absorb them also. Consumers, this school says, will be unaffected by changes in corporate rates. What really happens? When the corporate rate is cut, do Berkshire, The Washington Post, Cap Cities, etc., themselves soak up the benefits, or do these companies pass the benefits along to their customers in the form of lower prices? This is an important question for investors and managers, as well as for policymakers. Our conclusion is that in some cases the benefits of lower corporate taxes fall exclusively, or almost exclusively, upon the corporation and its shareholders, and that in other cases the benefits are entirely, or almost entirely, passed through to the customer. What determines the outcome is the strength of the corporationâ€™s business franchise and whether the profitability of that franchise is regulated. For example, when the franchise is strong and after-tax profits are regulated in a relatively precise manner, as is the case with electric utilities, changes in corporate tax rates are largely reflected in prices, not in profits. When taxes are cut, prices will usually be reduced in short order. When taxes are increased, prices will rise, though often not as promptly. A similar result occurs in a second arena - in the price- competitive industry, whose companies typically operate with very weak business franchises. In such industries, the free market "regulates" after-tax profits in a delayed and irregular, but generally effective, manner. The marketplace, in effect, performs much the same function in dealing with the price- competitive industry as the Public Utilities Commission does in dealing with electric utilities. In these industries, therefore, tax changes eventually affect prices more than profits. In the case of unregulated businesses blessed with strong franchises, however, itâ€™s a different story: the corporation and its shareholders are then the major beneficiaries of tax cuts. These companies benefit from a tax cut much as the electric company would if it lacked a regulator to force down prices. Many of our businesses, both those we own in whole and in part, possess such franchises. Consequently, reductions in their taxes largely end up in our pockets rather than the pockets of our customers. While this may be impolitic to state, it is impossible to deny. If you are tempted to believe otherwise, think for a moment of the most able brain surgeon or lawyer in your area. Do you really expect the fees of this expert (the local "franchise-holder" in his or her specialty) to be reduced now that the top personal tax rate is being cut from 50% to 28%? Your joy at our conclusion that lower rates benefit a number of our operating businesses and investees should be severely tempered, however, by another of our convictions: scheduled 1988 tax rates, both individual and corporate, seem totally unrealistic to us. These rates will very likely bestow a fiscal problem on Washington that will prove incompatible with price stability. We believe, therefore, that ultimately - within, say, five years - either higher tax rates or higher inflation rates are almost certain to materialize. And it would not surprise us to see both.

OK, the last paragraph is kind of interesting too. Buffett said he bought $12 billion in stocks after the election so I guess he is not so worried about the fiscal position of the U.S.

Back to fund performance stuff…

__Comparing Two Distributions__

I said that I don’t care for the ‘beat the market x out of y years’ idea. So that got me thinking about the simple high school statistics problem of comparing two normal distributions. I am aware of the argument against using normal distributions in finance, but I don’t really care about that here. I am just looking for some simple descriptive statistics. I’m not creating a derivatives pricing model to price an exotic option for a multi-billion dollar book where modeling errors can cause huge losses. So in that sense, who cares. Normal distribution is fine for this purpose.

Plus, I am not so interested in factor models that try to assess fund manager skill. Some people use factor models and whatever is left over is what they define as ‘skill’. Well, say the model cancels out ‘quality’ as a factor and doesn’t give the manager credit for it; what if the manager intentionally focused on quality investments? Should he not get credit for it? Having said that, I don’t know much about these models so whatever… I don’t get into that here. Whatever factor exposures these managers have, I assume the manager intentionally assumed those risk factors to gain those returns.

Basically I just want to compare two distributions and see how far apart they are. It’s basically the question, is distribution A, with 99% confidence, the same as distribution B? In other words, are the two distributions different with any degree of statistical significance? Or are we just looking at a bunch of noise resulting from totally random chance?

The simple comparison of two distributions is:

standard deviation of the difference between two means (Std_spd) =

Sqrt[(Vol_A^2/n) + (Vol_B^2/n)]

where: Vol_A = standard deviation of distribution A and

n = number of samples

So the z-score would be:

(mean_A – mean_B) / Std_spd

And then you can just calculate or look up the probability from this z-score.

Looks good. This would tell me how significantly different a manager’s return is versus the market.

But the problem is that these two distributions are not independent. In your old high school statistics text book, the example is probably something like number of defective parts in factory A versus factory B. Obviously, those distributions would be independent.

This is not so in the stock market. A fund manager’s returns and the stock market’s return are not independent. Hmm… Must account for that.

The answer to that goes back to my derivative days; calculating tracking error. Sometimes fund managers or futures traders wanted to use one index to hedge against another. An example might be (in the old days!) an S&P 100 index option trader wanting to hedge their delta using the S&P 500 index futures. Does this make sense? What is the tracking error between the two indices? Does it matter? Is the tracking error too big for it to be an effective delta hedge? How about using the S&P 500 futures to hedge a Dow 30 total return swap? TOPIX index swap with the Nikkei 225 futures?

Anyway, the calculation for tracking error simply makes an adjustment by making a deduction for correlation (getting square root of the covariance).

So, the above formula becomes:

Sqrt[(Vol_A^2/n) + (Vol_B^2/n) – ((2 * Vol_A * Vol_B * correlation(A,B)) / n)]

Using this formula, I calculated all this stuff for the superinvestors, just for fun.

I just wanted to know simple things like, is it harder to outperform an index by 10% per year over 10 years, or by 3% per year over 20? Or something like that. The Buffett partnership was only 13 years, and Greenblatt’s Gotham returns in the Genius book is only 10 years. But the spread is so wide that it is yugely anomalous to achieve, or is it? This is sort of what I wanted to know. It normalizes the outperformance spread versus the length of time the outperformance lasted.

__Few Standouts__

A few of the standouts looking at it this way, not surprisingly:

- Buffett Partnership 1957-1969: a
**6.0**sigma event,**1 in 1 billion**chance of occurring (yes, that b is not a typo!) - Walter J. Schloss 1956-1984:
**5.2**std,**1 in 9.4 million** - BRK 1965-2015:
**4.8**std,**1 in 1.3 million** - Greenblatt (Gotham 1984-1994):
**3.8**std,**1 in 14,000** - Tweedy Brown 1968-1983:
**3.7**std,**1 in 9,300**

For the Graham and Doddsville superinvestors, I looked first at the “beat the market x out of y years” to see the probability of that happening assuming a 50% chance of beating the market in any given year. And then I’ll compare the two distributions as described above. At the end, I also added Lou Simpson’s returns from the 2004 Berkshire letter.

Keep in mind that just because a manager is not in the 4-5 sigma range, that doesn’t make them bad managers. Some of these numbers are just insanely off-the-charts and can’t be expected to happen often.

Anyway, take a look!

__Buffett Partnership (1957-1969)__

Beat the market 13 out of 13 times: Chance of occuring: 0.012% or 1 in 8,192.

Given that Buffett partnership gained 29.5%/year with a 15.7% standard deviation while the DJIA returned 7.4%/year with a 16.7% standard deviation and the Partnership had a 0.67 correlation, the partnership returns is** 6.0 **standard deviations away from the DJIA. 6 standard deviations make the partnership returns a** 1 in 1 billion event**.

What’s astounding is that the standard deviation of Buffett’s returns is actually lower than the DJIA.

__BRK 1965-2015__

Beat market 40 out of 51 years: 0.003% chance or 1 in 35,000

__BRK__ __S&P500__

Return 19.3% 9.7%

std 14.3% 17.2%

correl 0.61

4.8 std, **1 in 1.3 million**

This uses book value, which may not be fair as not everything in BPS is marked to market (over 51 years). Using BRK stock price, it would be a 3.2 std event, or 1 in 1,455. But this too may not be fair as the volatility of the price of BRK is more a function of Mr. Market than Mr. Buffett. This may be true of all superinvestor portfolios, but in the case of BRK, there is a penalty in that we are looking at the volatility of a single stock (BRK), and not the underlying portfolio. Single stock volatility is usually going to be much higher than that of a portfolio.

__Munger 1962-1975__

Beat the market 9 out of 14 years: 21% chance or 1 in 5

__Munger__ __DJIA__

return 19.8% 5.0%

std 33.0% 18.5%

correl: 0.73

#years: 14

2.4 std, **1 in 122**.

__Sequoia 1970-1983__

Beat the market 8 out of 14 years: 40% chance or 1 in 2.5

__Sequoia __ __S&P 500__

return 17.2% 10.0%

std 25.0% 18.1%

correl 0.65

1.4 std or **1 in 12.**

This is the in-sample period; the period included in the Superinvestors essay.

__Sequoia 1970-2016__

Beat the market 26 out of 47 years, 28% chance or 1 in 3.6

__Sequoia__ __S&P500__

return +13.7% +10.9%

std 19.3% 17.1%

corr 0.67

1.3 std or **1 in 10**

__Sequoia 1984-2016__

This is the out of sample period; the period after the essay.

Beat the market 18 out of 33 years, 36% chance or 1 in 3

__Sequoia__ __S&P500__

return 11.9% 10.9%

std 16.0% 16.6%

corr 0.73

0.5 std or **1 in 3**

__Sequoia 2000-2016__

And just for fun, a recent through-cycle period starting in 2000. They have been underperforming the market since 2007, though.

Beat 9 out of 17 years, 50% chance or 1 in 2.

__Sequoia__ __S&P500__

return 7.3% 4.5%

std 13.7% 18.1%

correl 0.69

0.9 std or **1 in 5**

__Walter J. Schloss 1956-1983__

Beat the market 22 out of 28 years, 0.2% chance, or 1 in 540

__WJS__ __S&P500__

return 21.3% 8.4%

std 19.6% 17.2%

corr: 0.75

5.2 std or **1 in 9.4 million**

__Tweedy, Browne Inc. 1968-1983__

Beat the market 13 of 16, 1.1% chance or 1 in 94

__Tweedy__ __S&P500__

return 20.0% 7.0%

std 12.6% 19.8%

corr: 0.71

3.7 std or **1 in 9,300**

__Pacific Partners Ltd. 1965-1983__

Beat the market 13 of 19 years, 8% chance or 1 in 12

__Pacific__ __S&P500__

returns 32.9% 7.8%

std 60.2% 17.2%

corr: 0.37

1.9 std or **1 in 35**

__Gotham 1985-1994__

Beat the market 9 out of 10 times: 1.1% or 1 in 93 chance

3.8 std, **1 in 14,000**

__Lou Simpson (GEICO: 1980-2004)__

18 out of 25 years. 2.2% chance or 1 in 46.

__GEICO__ __S&P__

return 20.3% 13.5%

std 18.2% 16.3%

corr: 0.74

2.7 std, 0.4%, **1 in 288**

__Conclusion__

So that was kind of interesting. It just reaffirms how much of an outlier Buffett really is. There is a lot to nitpick here too, so don’t take these numbers too seriously. I used standard deviation of annual returns, for example. I suspect some of these correlations may be higher if monthly or quarterly returns were used.

This sort of thing may be useful in picking/tracking fund managers. At least it can be one input. For example, it gives you more information than the Sharpe ratio; whereas the Sharpe ratio doesn’t care how long the fund has been performing, the above analysis takes into account how long someone has been performing as well as by how much. But yeah, Sharpe ratio is trying to measure something else (return per unit of risk taken).

Anyway, as meaningless as it may be, it’s one way of seeing if it’s harder to create a long term record like Buffett (1965-2015) or a shorter super-outperformance like Greenblatt (1984-1994). This analysis says that Buffett’s 1965-2015 performance is a lot more unlikely to be repeated (well, at least on a BPS basis; using BRK stock price, Greenblatt’s performance is more unlikely!).

I sliced up Sequoia Fund’s return into various periods for fun as it is the only continuous data (other than BRK) out of the Graham and Doddsville Superinvestors. I was going to look into their performance since 1984 a little more deeply, but this took a little more time than planned (despite the automation of a lot of it; well, debugging and fixing takes time, lol…).

So maybe I will revisit the Sequoia Fund issue in a later post. My hunch is that the Superinvestor returns were achieved on a much lower capital base so the universe of potential investments were much larger than what Sequoia (and others) are looking at now despite their efforts to keep AUM manageable.

Also, you will notice that comparing the two distributions gives a more nuanced or accurate picture of the performance than just looking at how many years someone has outperformed; it incorporates the spread, correlation, volatility etc…

Anyway, I guess that’s enough for now…

Great post as always! Are you aware of any superinvestors with recent & consistent standout performance say over 10/15 years?

Not really… I don't know of anyone that isn't already covered by the value investing community. But I bet there are a lot of smaller funds putting up huge numbers even today… but they would be really small and we just don't hear about them (cuz their too small!)….

I have been looking at whalewisdom.com which tracks all the 13-Fs out there (4,000+ with AUM >$100mn), and I can say those who can deliver 15% CAGR are rare (let's assume 13-Fs are good proxies for low turnover long-only value funds)….

Excellent post as always… by any chance did you calculate the distribution for Markel ??

Check out for instance Turtle Creek in Canada. Spectacular results…

Ran Turtle Creek's 13-F on Bloomberg's PORT analytics. Up a total of 24% from 9/30/14 to 2/17/17. But top 4 holdings accounted for over half of its portfolio.

This is because their can. holdings are not listed. They typically have about 25 holdings.

Great post. As much of an outlier as Buffett is (and the statistics suggest) I think the numbers are misleading. The Partnership performance came at a time of substantially less sophistication in the investment world. Put another way, those same performance numbers today would in my mind be a far, far greater achievement yet would show up with the same statistical likelihood. Secondly, BRK's performance is not comparable to the returns on traditional funds. A large part of this has come from the performance of the insurance businesses and the free money it provides the insurance operation. Just my initial thoughts so would be interested in your view.

I don't disagree. And yes, BRK may not be a fair comparison to traditional funds also because of the book value issue (I used BPS).

If WEB's performance came during a time of less sophistication why weren't there more people outperforming in the 60s then now? Or were there? I am asking in all seriousness. If the distribution of active mgrs doing better in the 60s vs now was higher, that claim would carry more weight. the lack of bloombergs and big data shouldn't invalidate what he did.

BRK's float gets him I think 5% of performance when factoring in the low cost plus leverage. AQR did a great paper on this, buffets alpha i think. If you can get a hold of Cliff Asness on Barry Ritholt'z masters in business i think in early 2016, great interview. Talks quant + some buffett but the idea as Cliff said is you can't eat risk adj returns and WEB wanted to so levered at 1.5x or so which is consistent with AQR's own research – ie portfolio leverage on low beta = good.

Cool post, and I like the discussion on the taxes (and forgot it was in the 1986 letter). It'll be interesting to see if the market is getting those second order effects correct. I suspect that it will get it more or less right in lots of cases but maybe there are some opportunities out there (TDG should probably be a beneficiary and isn't necessarily expensive even at current tax rates, for example).

One thing that I hope is that it stops the practice of keeping money offshore. That absolutely drives me nuts. I understand that if you're running, say, Apple, then if you keep the money offshore you can report higher earnings and get a higher stock price, but with a 10% or so cost of capital leaving the money stranded overseas means that if it stays more than 2-3 years then you're destroying value. Obviously companies like Google and Apple do it, but also Coke and Nike and a ton of other ones. Honestly, if tax reform just fixes that then I'd be happy.

Very interesting analysis.

The impact of time is astonishingly on the probability. Schloss vs Tweedy really jumped out at me. Both have similar return outperformance (+13% p.a.) but TB ran 0.64x index volatility vs. Schloss at 1.14x. Just looking at those two statistics I would suggest TB's outperformance is much more remarkable, but the additional 12 years of outperformance by Schloss moves the odds against him astronomically.

Coming from a background analysing non-equity funds with much lower volatility, someone who can outperform an equity index with significantly less vol appeals much more than the same outperformance but with greater than index volatility. But that is undoubtedly some bias on my part by way of background.

Why would you not analyze the difference in 2 populations (investor's results vs index) to eliminate interdependence of results in the same year? It would simplify calculations.

Hi,

So, financials have been rallying for quite awhile. This includes JPM, GS, BAC, WFC, all the big boys, and they are not as cheap as they have been. Do you still see JPM as table thumping? I still hold on JPM and GS and even bought some WFC recently after their "thing", but it's getting really difficult to make a judgement here from a value perspective because it is seriously getting into macro impact. Thanks for your toughts.

Hi,

I think they will do well over time. They are definitely not as cheap as they have been when I was really pounding the table on them. I wouldn't pound the table on them now, but consider them good solid holdings for sure.

That absolutely drives me nuts. I understand that if you're running, say, Apple, then if you keep the money offshore you can report higher earnings and get a higher stock price, but with a 10% or so cost of capital leaving the money stranded overseas means that if it stays more than 2-3 years then you're destroying value. Edward Gelber, M.D., Board-Certified Psychiatrist in Brooklyn … Click on Edward Gelber

there have very essential and useful info about problem is that these two distributions are not independent. In your old high school statistics text book, the example is probably something like number of defective parts in factory A versus factory B.

Professional Tax adviser in the UK

Professional Tax specialist in the UK

This idea is mind blowing. I think everyone should know such information like you have described on this post. Thank you for sharing this explanation.if anyone looking for six sigma certification

training join us

your post is Excellent. thank for shearing ……..