# The Key Problem with Monte Carlo Software - The Need for Better Performance Metrics

Recent articles in *Advisor Perspectives* by Wade Pfau and David Blanchett have explained the benefits and limitations of using Monte Carlo simulations in financial planning (see here and here). Pfau and Blanchett talked about the power and limitations of the technique, compared Monte Carlo to the similar approach of using rolling historical periods and addressed the importance of return assumptions.

I'll add to the discussion by examining this technique from a different perspective, focusing on the metrics used to evaluate the outputs from Monte Carlo simulations. Popular financial-planning software packages have shortcomings in this respect, and other metrics can provide more useful information. I will address how to measure the performance of financial plans when variable investment returns and longevity are introduced and demonstrate that the most-commonly used measures have weaknesses.

### Measuring performance

When we move from using deterministic projections of financial-planning outcomes to using Monte Carlo simulations, we need performance measures that can handle the variability of outcomes. The best measures will be those that appropriately summarize the outcomes so advisors and clients can communicate effectively in choosing withdrawal levels and asset allocations and deciding whether to purchase financial products such as annuities.

I'll use an example to show how the measures used with Monte Carlo simulations differ from those used with deterministic forecasts and how some of the commonly used measures can be improved. I'll also discuss measures that can be applied in more complex planning, when withdrawal amounts vary as a function of investment performance.

### Example

This article is based on a 65-year-old female with a 25-year life expectancy who has reached retirement with $1 million in savings that she will use to generate retirement income. Her goal is to be able to withdraw $40,000 in the first year with inflation increases each year thereafter, following the classic 4% rule. Her investment options include stocks with an arithmetic average real return after inflation and expense charges of 6.85% and a standard deviation of 20%, and bonds with a 0.45% return and 5.5% standard deviation. She also has the option of using a portion of savings to purchase an inflation-adjusted single-premium immediate annuity (SPIA) that will pay 4.5% of the purchase price in the first year, with increases based on actual inflation each year thereafter.

My assumed fixed-income investment returns are lower than historical averages, reflecting current lower bond yields. For stocks, I've assumed the same return premium over bonds as the historical averages. The SPIA pricing reflects the current level of rates from direct-purchase sites such as Income Solutions® or the Thrift Savings Plan available to federal employees. The analysis is pre-tax.

### Deterministic forecasting

With deterministic forecasting, probabilities do not come into play when measuring success — either a plan works or it doesn't. The success measures are whether the plan depletes savings over a set retirement period, and if not, how much savings remains. In the example, the plan involving 4% inflation-adjusted withdrawals will provide retirement income for 25 years as long as the compound real return is greater than zero. However, with zero return, the plan will fail if the retiree lives to more than 90 years old. A compound real return greater than 4% means that savings will actually grow over the course of retirement, no matter how long retirement lasts.

Deterministic forecasts can give rise to overconfidence. It might not seem that much of a challenge to beat 4%, although that is a real rate, not a nominal rate. However, when we apply Monte Carlo simulations, we get a clearer picture of the actual risks.