The need to acknowledge the existence of errors in performance attribution

I found the above graphic today on Facebook, from the “I Love Mathematics” group. If you haven’t figured it out, yet, I love mathematics.

It occurred to me that a similar graphic could be prepared for performance attribution; perhaps something like:Please don’t interpret the pie sizes to represent the degree of representativeness of each source, as (a) some are mutually exclusive, (b) they all won’t always apply, and (c) sizes will, in reality, vary from application to application (organization to organization).

But let’s briefly review these sources of errors in performance attribution

We begin with two that deal with the handling of residuals.

We know that arithmetic models are “linking challenged.” That is, when we link effects across time we get residuals. The good news is that these residuals can be “smoothed out.” The challenge is in getting them done correctly. There are a variety of models available: some work well, some don’t. I’ll conduct a webinar on this topic on February 6.

Geometric handles linking fine: it’s the single period where it has challenges. And so, residuals arise. These, too, need to be “smoothed out.” Some model designers (e.g., Carl Bacon, CIPM) assign the residual to a single effect, while others (e.g., Jose Menchero, PhD, CFA) allocate the residual across all effects.

Holdings-based models have issues. I will soon be writing an article on this topic, that will elaborate on the degree to which these errors can arise, and why firms who employ such models should seriously consider switching to a transaction-based approach.

Using the wrong model can be a problem, in that the effects don’t actually relate to the investment process, or at least in a proper manner. The worst case is probably when we see the employment of an equity model for fixed income.

And finally we have data problems. Data continues to be a huge issue, one that continues to gain attention. Having sufficient controls in place is important to reduce this source’s contribution to errors.

Have other thoughts on the sources of errors in performance attribution?

Please let us know what you think about this topic. I’ll confess that I put this together fairly quickly, so perhaps failed to consider one or more sources. And so, please chime in!