How to Compare Venture Returns & The Curious Case of a16z

The Wall Street Journal published an article shining a light on Andreessen Horowitz, the vaunted VC firm that seemingly came out of nowhere in 2009 to become one of the most prestigious venture firms in Silicon Valley. The article indicates that a few of a16z’s funds are not performing as well as their age-matched peers.

Basically, one evaluates venture fund performance by comparing a fund raised in a certain year to the set of other funds raised in that same year. Much like wine enthusiasts, investors in venture funds use the term “vintage” as a way to talk about the relative performance of funds.

For example, one can compare a given fund to the broader market (ex. “2009 was not a good vintage. The top 5% of 2009 funds only generated 2.6x ROI, much lower than other years.”) or compare a given fund to the set of funds managed by the firm. A real-world example of this latter usage can be found in a post by Fred Wilson: “Our first USV fund, our 2004 vintage, has turned out to be the single best VC fund that I have ever been involved in. We made 21 investments. We made money on twelve of those investments. We lost money on nine of them. And we lost our entire investment on most of those nine failed investments.”

But this is where the tidy explanations end and a lot of methodological hand-wringing begins. a16z managing partner Scott Kupor published a rebuttal on the firm’s site stating that the WSJ journalist misinterpreted the data that investment advisory firm Cambridge Associates sent their way.

But Kupor wasn’t alone in finding fault with the Journal. Adam Marx, fellow freelancer for Mattermark, shared a summary of the venture capital community’s reaction to the WSJ piece on his own site, and it’s worth checking out if you haven’t been following this story too closely. In general, the investment community sides with Andreessen Horowitz.

Here is my take on the whole thing.

Scott Kupor is right in one respect. I learned the hard way that ROI and IRR numbers do not really matter when making comparisons between venture fund performance. As Brooklyn Bridge Ventures’s Charlie O’Donnell explained in a very thorough rebuttal to a piece I wrote, two metrics matter:

1. In a fund that’s still operational, cash distributions matter most.
2. To a fund that’s run its 10–12 year course, ROI after management and carry fees matters most.

Considering that all of Andreessen Horowitz’s funds are currently in progress, the Journal writer should have focused on raw cash distributions and cash distributions as a function of the initial fund size. If Kupor is to be believed, had the journalist made their comparisons based on cash and stock distributions, then Andreessen Horowitz would “match” elite venture capital firms rather than “trail” them as the WSJ headline suggests. However, since I don’t have the data from Cambridge Associates, I can’t make that judgment myself. (If I get said data, it’ll be the subject of another piece.)

However, despite Kupor’s best efforts to educate readers on the different accounting methods used to mark unrealized gains to market conditions, he fails to mention how Cambridge Associates calculates its figures around unrealized gains. If it’s by way of survey, then his critique is valid. Different valuation methods produce significant variance in results despite all being technically, theoretically and legally valid. But if Cambridge Associates is performing a real apples-to-apples analysis based on knowledge of venture firms’ positions in various companies (which would be somewhat challenging given the opacity of venture investing, but I digress) then a16z has some explaining to do. 

To conclude, I think it’s generally good that the WSJ piece came out. Wether you think the journalist’s narrative is any more or less valid than the firm’s is beside the point. It re-kindled a longstanding discussion about the economics and metrics of venture capital and highlighted the analytical challenges the particular structure of the market for private equity imposes on analysts and researchers.

This post was originally shared in my weekly newsletter. If you want to read more about startups, venture capital and similar subjects every week, consider subscribing.