Here’s a long excerpt from Byrne Hobart’s appearance on Patrick McKenzie’s Complex Systems Podcast that explains modern alpha-seeking so well:
Hedge funds in their modern incarnation are machines for looking for deficiencies in other people’s model of the world that can be expressed through trades. That model has, has very much evolved – at least for the largest hedge funds, it’s evolved towards a setup where, if you look at asset classes, you can see that they have different risk and return characteristics, and then within those asset classes, you can make other judgments.
You can say things like, let’s say, very low-rated bonds are much more sensitive to recession risk, and highly-rated bonds are more sensitive to interest rate risk; you can say that, typically, best-performing stocks will actually continue to outperform, as will worst-performing stocks; that typically statistically cheap companies do a bit better over time than statistically expensive ones; that industries correlate and industry membership explains a large share of a given stock’s performance, et cetera.
You can enumerate all of these factors that are just broad statistical explanations for where returns come from, and that allows you to look at someone’s investing track record and identify, how much of this was that you picked good stocks? How much of this was that your career happened to span a bull market? How much of this was, not only was it a bull market, but the first job you got happened to be analyst at a tech fund, and tech did unusually well in that bull market?
The subtext of this next part is that skill has a fuzzy component
We run these regressions and find out, “okay, you, you beat the market by five points a year. It turns out that 5.5 of that was luck, and the other negative 0.5 was skill.” Someone actually did this with George Soros’s investment record and found that his skill contributed negative two points, and that following trends in currencies was just a really, really good trade to run at that time.
I think there’s still, there’s still something valuable in having implicitly done the regression in your head and actually somehow instinctively identified this systematic signal and executed well.
Patrick McKenzie: I think there’s probably a sort of unscored pregame in which you look at every opportunity available in the world and somehow through “luck,” select an opportunity where, go figure, the beta in that opportunity, the returns to the market exceed returns available in other markets during those years.
Maybe you sub-optimized with respect to how you executed on that opportunity before you, but you picked very well on which opportunity to spend a portion of your professional career going after.
[Patrick notes: I often feel this way about commentary on how geeks are “lucky” to have had their special interest be extremely valuable to e.g. tech industry.]
But that fuzziness will probably shrink as our ability to measure improves
Byrne Hobart: Yeah, I think that is a reasonable way to look at it, especially in earlier history, but As we have more data now, at least within financial markets…
It is very hard to time these factor performances – very hard to time, “when will this industry do well or worse, when will momentum work unusually well or worse” – I’m sure people try to do it, I’m sure some people are good at it, but if you construct a portfolio where you’re netting out exposure to all of those factors, what you have done is you’ve created a portfolio that is just a measure of someone’s skill at identifying the idiosyncratic return drivers of individual stocks. So if they bought NVIDIA, they also had to short a corresponding amount of other large companies, other growth companies, other tech companies, etc. such that, if they make money on NVIDIA after all that hedging, it’s because they actually knew something right about NVIDIA.
What that ends up meaning is that the hedge fund – we’ve actually made the full circle. People used to knock hedge funds as “a compensation scheme masquerading as an asset class,” and as they’ve gotten better at building these hedge funds, market-neutral, factor-neutral portfolios, they are increasingly a method of measuring investment skill masquerading as an asset class.
Because, what do you want? In theory it makes sense that you should be able to charge a lot of money for skill, and you should not be able to charge very much money for, “you happen to get a job analyzing an industry that happened to do well over the time when you were a portfolio manager.”
So it means that as hedge funds have gotten better at just just delivering that idiosyncratic return, and [with] the accumulation of a bunch of different portfolio managers, who are finding a bunch of different ways to extract the “idio” from a bunch of different sets of companies, you can charge a lot more for that – which means you can pay people a lot more, and so you can bring in more talent to the industry.
That model keeps on growing, but it does become a model where you as an analyst or as the trader or as the portfolio manager, you are constantly asking yourself questions like, “why do I deserve to be right about this?” If you have a reason to think this is a good company, what is the reason that someone else looking at the same evidence didn’t think so?
Sometimes the reason is you looked at more evidence than they did – they talked to five people at private companies that order lots of GPUs, you talked to eight people, you have a slight edge on the person who worked less.
Sometimes you just have a signal where you identify, not really why didn’t someone else exploit this, but why does this happen in the first place?
So you’re always trying to build this model of the world, and of what you know, what you know relative to other people, what mistakes other people might be making, how persistent those mistakes are, how much competition there is to exploit those mistakes, and you’re trying to measure the degree to which your returns are being competed away…
You’re always doing this kind of introspection and always trying to rigorously measure your own skill as an empiricist. It is basically this exercise in just being a rationalist. It is like they are mentally reinventing the entire LessWrong corpus all the time.
Why measuring VC skill will always be a hard problem
Byrne: Measuring venture investor skill is one of the hard problems in finance, and may never be solved because, if you are in a power law kind of investing situation, you have these long lags between when you write the check and when you get a wire for a much larger amount to your account. Because of that, not only do you have a fairly small sample of successes, but the more successful you are, the more likely it is to be from one really, really big thing you did, which means the more successful someone is, the easier it is to claim that they were lucky. And that just makes it a really frustrating business to analyze and understand.
It also means that it’s very hard for a venture investor to think about the marginal cost of doing one more interview or buying one more data set or something like that – whereas with a hedge fund or a prop trading firm… I don’t know that any of them explicitly measure things like “what is the marginal value of this analyst spending the next 30 minutes reading a transcript of an interview or editing the scraper that we’re using to track the inventory in this API that the company does not realize is actually exposed to the public internet.”
They don’t measure it quite that granulated, but they have a pretty good sense of what is the incremental return on the next action, and they have pretty high confidence in that.
You don’t have that. You don’t know if the next call that you take is from someone who’s starting the next Stripe – the odds are very, very low, but the odds are non-zero, and you will never actually have enough data to be anything like confident in that.