I was teaching a custom version of our Fundamentals of Performance Measurement class for a university foundation earlier this month, when I stumbled upon this slogan: “store less, calculate more.” It applies to performance software development, and is probably worthy of a bit of an explanation.

Background

We were talking about the unit value method to derive rates of return. Recall that this approach is similar to how the net asset value approach works:

• For the unit value method, we begin by defining a “unit value” for the portfolio (similar to defining an initial “NAV for a fund”; e.g., 100).
• We next divide our portfolio’s market value by the unit value to determine the number of units we hold in the portfolio (similar to the number of shares for a mutual fund;
  • e.g., if our market value is $1 million, then we divide $1,000,000 by $100 to get 10,000,
  • meaning we have 10,000 units in our portfolio, where each unit is worth $100).
• Once this is done, it’s up to us to maintain the unit value (by taking the market value and dividing it by the number of units on a daily basis) and the number of units (to account for contributions and withdrawals from the portfolio; similar to the purchase or redemption of shares in a fund).

One specially attractive feature of the unit value method is that we can easily derive the rate of return from any date to any date, by simply dividing the ending date’s unit value by the beginning dates, and subtracting one.

The problem with unit value approach is that we have to store a LOT of data: for each day, a unit value and number of units.

But storage is cheap!

Very true, very true.

I’m a bit “old school.” I first started building software in the 1970s, when storage wasn’t so cheap. We were forced to be very efficient in what data we stored, and how we stored it; this is no longer the case. That being said,

why waste storage when you don’t have to?

Think about it, if you store every day’s unit value and number of units for a year (along with all the other details), what’s the likelihood that you’ll use any given day? VERY, VERY low. Month-ends, yes! But any day during the month? Unlikely. And, these values have no other meaning; they serve no other purpose.

And so, you’re storing, year-after-year, data that you will never, ever, ever have to call upon. But, for the unit value method to work, you store this data. Granted, after some time, you might decide to do away with the days that aren’t month-ends, which would probably be a good idea, but you’d need some sort of a “back up plan” (no pun intended) in case you need to resurrect one of the non-month-end days.

We don’t use the unit value method; so, what’s your point?

This applies to performance software development, too!

One issue that performance software development folks typically run into is whether to calculate returns “on the fly” or store lots of data so that you can calculate them when necessary.

We are occasionally called upon to either design, participate in the design, or review the design of performance systems. I generally recommend not storing daily data.

But, you might say, we want to calculate daily rates of return! Well, you can. Just don’t store all those days’ data: just store month-end data.

But, what if we need to do a “point-to-point” return calculation? E.g., from June 2, 2013 to March 25, 2015?

Performance software development tip for point-to-point returns when you don’t store daily data

All you need is:

• Month-end holdings
• Daily prices
• Daily transactions.

If you have this data, you can get to any date in the month. I’ve designed logic that can get you to any day, in a pretty efficient fashion. It’s really not complicated. You essentially back into the day’s holdings, using one of the month-ends that bound the day you’re interested in (e.g., for June 2, 2013 use May 31, 2013; for March 25, 2015, use March 30, 2015), and price the day’s holdings from your pricing file. You can then revalue any days between June 2, 2013 and June 30, 2013 that have cash flows, so that you can derive the returns in between; likewise between April 30, 2015 and March 25, 2015.

Performance software development slogan: store less, calculate more

Yes, it’s true that storage is cheaper; but computers are much faster, too. And for those rare occasions when you need to find the starting or ending value that’s not a month-end for a return request, derive it!

Why be wasteful? If you store daily data, eventually you’ll need to buy more and more storage units.

Again, you can purge days in between month-ends at some point, but why not be efficient from the start? That’s my thinking on one aspect of performance software development.

I probably haven’t given the topic all the details it warrants, but wanted to essentially touch on the tradeoff between storage and calculations.

Again, storage is cheap, computers are fast; we know that. But, it’s still a worthy exercise to consider these alternatives and their respective “pro’s and cons.”

What do you think?