A Comparison of Risk Tolerance Products

FinaMetrica

FinaMetrica has changed hands twice recently. In 2017, the Australian company was purchased by PlanPlus, a Canadian company that added a number of new features and capabilities. And then, earlier in April, Morningstar purchased the enhanced Suitability Pro program, which includes FinaMetrica plus those new features. The Suitability Pro enhancements include mapping a prospect’s portfolios to FinaMetrica’s famous 1-99 scale of risk tolerance, and identifying a more suitable portfolio if there is a mismatch. The program also creates guard rails on the riskiness of a proposed portfolio based on, among other things, the client’s time horizon for a particular bucket of money, the client’s self-reported investment sophistication and his or her behavior during past downturns.

I’ve started with FinaMetrica, not because it has the highest market share in the U.S. market (our survey indicated that it is a distant second to Riskalyze), but because an arbitration panel is likely to be overwhelmed by the amount of scientific data validating FinaMetrica’s methodology, so much so that you could drop it with a loud thud on the exhibit table. It starts with a 41-page PDF technical manual co-authored by Dr. Myrsini Katsikatsou, research fellow at the London School of Economics department of statistics, with commercial economist Stuart Erskine. The document features 36 tables and 11 figures, with normative evaluations of the scoring algorithms and distributions – a technical way of saying: Are the (so far collected) 541,549 scores normally distributed, consistent across time, consistent across countries (primarily the U.S., Australia, New Zealand and the U.K., but with some data collected from investors in Continental Europe, India, Canada and Ireland) and across gender. The charts show results based on ages, marital status, income ranges, net worth, educational attainment and number of family members, and they measure consistency of results across bull and bear markets.

Some of the results that I found interesting: The average risk tolerance score for the aforementioned 500,000+ tests consistently were almost exactly on the 50 level through all the market turmoil that we’ve experienced. Men, particularly young men, tended to have a higher level of risk tolerance than the average, and men overall scored about five points higher than women. The average score of the highest income category is about 9 points higher than the average score of the lowest income cohort. People of both genders age 20-29 have, on average, scored about 7 points higher than people age 70-80. The most educated people taking the risk tolerance instrument scored 6 points higher, on average, than those with the least education. People taking the test in India and South Africa (a small minority of the overall FinaMetrica test-takers) reported the highest risk tolerance scores among all countries, with a 53.5 average. The lowest average was among U.K. investors: 47.8.

The arbitration panel can peruse means and medians by year, standard deviations, skewness and kurtosis – and basically the report shows that the test has produced remarkably consistent results for the last decade and a half. If they have the stomach to dig deeper, the panelists can ponder over the meaning of “eigenvalue,” “Guttman’s Lambda 1-6,” the Spearman-Brown Coefficient” and “Cronbach’s alpha.”

FinaMetrica’s 25-question test (there are three additional tests that measure other factors than risk tolerance) is grounded in the science of psychometrics, which is basically the study of whether a questionnaire accurately measures what it is intended to. To help me understand psychometrics and how they relate to FinaMetrica, the company provided me with a Google Scholar link, which lists 250 research reports that have studied FinaMetrica and its analytical processes in some detail – published in a variety of academic journals like the Journal of Behavioral and Experimental Psychology; Journal of Accounting and Finance; the International Journal of Accounting and Finance; the Journal of Economic Psychology; the Journal of Business & Economics; the Journal of Investment Consulting; and the International Journal of Information Technology & Decision Making.

Beyond that, the Suitability Pro component of the software shows clients a graph, based on the expected return and standard deviation of the portfolio they and the advisor have selected, with bands going into the future showing what would happen at one, two and three standard deviations on both sides of the expected return. The advisor discusses this with the client, and types in notes based on the conversation. It is not hard to envision those notes, from that conversation, finding their way into evidence.

My “defensibility in arbitration” score for FinaMetrica and the Suitability Pro features built around it: A+.

Riskalyze

Riskalyze is by far the market leader in the U.S. risk tolerance space, and is more properly described as a full-featured platform based on all the different directions the firm’s developers have taken it. You can use Riskalyze to create portfolios, or choose from its model marketplace, or automate trading activities – all in the name of (company slogan) “fearless investing.”

In conference presentations, company founder/CEO Aaron Klein recommends that advisors use the program as a prospecting tool. You invite prospects to take the Riskalyze instrument and identify where they fall on a scale of 1-99. Then the program will analyze the prospect’s existing portfolio and give it a score from 1-99. If there’s a mismatch, the advisor can point out the shortcomings of the prospect’s current portfolio, propose to create a new one that will be a better match, and take over the client relationship.

When I reached out to Riskalyze, Nicole Keene, chief of staff to CIO Michael McDaniel, replied that “none of our users has ever had to provide scientific basis behind their risk tolerance process… Our process requires the investor to confirm, ‘Yes that feels like me,’ and advisors must complete their firms’ new account paperwork accordingly, which also contains a client signature and approval of the results.”

In other words, when you go to arbitration, you are relying on the fact that your client signed off on the portfolio you recommended.

Keene also pointed me to a 2015 research paper by three Ph.D.s and another researcher, which, to the best of my knowledge, has not been published in any journal. The authors tell us that “Riskalyze has commissioned us to examine their technology and place it in the broad context of academic research on this topic.”

The paper explains the importance of evaluating clients’ risk tolerance, the reasons why clients may make irrational portfolio choices, and explains the Riskalyze methodology: a series of questions that ask people to choose between different risk/reward options (possible high-end returns and losses over the next six months).

Then the authors propose an ideal way to measure risk preferences (lottery choice questions, where you can choose a 50-50 gamble of having $1,000 or $1 million, or $300,000 in your pocket right now), and then talk about the Riskalyze instrument toward the end. A plaintiff attorney might be interested in the place where the report notes that the Riskalyze questions, “are, in the strictest sense, hypothetical, since the exact probabilities and payoffs specified in the questionnaire are not the ones that his eventual portfolio will face.”

Later it says “Ideally, the number of indifference points obtained would be much more than the number of parameters in the utility curve to be estimated; in Riskalyze’s case, the number of parameters is five, while the number of indifference points is five to seven. In theory, Riskalyze’s technology could provide the repeated sampling that would be needed to reduce noise in the measurement. However, Riskalyze has chosen not to exercise this capability…”

Comparing Riskalyze’s methodology to psychometrics, the authors suggest that Riskalyze’s “utility-based” instrument is more flexible and potentially more precise. The rest of the paper evaluates whether it is possible to determine the risk score of a portfolio (rather than an individual) – the other function of the Riskalyze prospect marketing process. Here, we are told that “Riskalyze’s current method for estimating portfolio variance, although it conforms to industry norms, has clear drawbacks,” and proposes three alternatives that might work better.

At the end of the report, the authors suggest that the best way to measure the worth of a risk tolerance questionnaire is the market test: “if a technology helps advisors retain their clients, it is a good one.” It concludes, “We believe (without having a thorough knowledge of all the products that exist) that Riskalyze’s technology is likely to prove to be a good one. However, there are changes that could improve it.” Among them: more sophisticated methods to estimate covariance matrices between various asset classes or individual investments, and better guidance on when clients should be re-tested to see if their proclivity for risk has shifted in the interim.

Compared with the voluminous research around FinaMetrica’s psychometric approach, this one paper is thin; it would hardly make a sound when dropped on the exhibit table. The plaintiff attorney is sure to point out the periodically tepid endorsements in this paper, and would almost certainly note to the arbitration panel that it was commissioned and paid for by Riskalyze itself.

My “defensibility in arbitration” score for Riskalyze: C.

Tolerisk Pro

Tolerisk Pro differs from both FinaMetrica and Riskalyze in that it not only measures risk tolerance (using a 30-question psychometric-based questionnaire), but also risk capacity – that is, the client’s ability to take risk based on current portfolio size and future needs. By combining the two, Tolerisk comes up with a score that evolves over time, as the client becomes wealthier compared to future needs – or, quite possibly, as the market drops dramatically, imperiling the client’s financial future and necessitating a scaling back of portfolio risk.

To assess risk capacity, the program projects cash flows into and out of the client portfolio, and also looks at a couple’s second-to-die probabilities of financial success. It also customizes all of these risk capacity factors based on age, health and heredity. The software uses IRS mortality tables to calculate projected lifespans.

The output, once again, is a score ranging from 1-99, and this number is mapped to actual client portfolios in the most direct possible way; the system recommends that clients who get a 50 score would have 50% of their portfolio in risk assets (equities), while a 75% score would imply 75% of the assets invested in equities.

So what would the advisor using Tolerisk Pro lay down on the exhibit table in an arbitration hearing? The two-dimensional risk tolerance process (propensity for risk and risk capacity) is endorsed by Michael Kitces. Tolerisk CEO Mark Friedenthal also sent me a white paper which shows the output for a hypothetical client. In Section 1, there is an explanation of how the information is utilized and validated. In larger type, the paper shows the Tolerisk “willingness” and “ability” scores, a recommended portfolio based on certain inputs, calculations showing the probability of running out of money, and an interesting graph that predicts how the client’s Tolerisk risk score will evolve over time based on projected returns and asset allocation.

At the end, the client acknowledges that he or she has read and understands the contents of the report, which is by far the most important thing to show the arbitrators.

My “defensibility in arbitration” score for Tolerisk Pro: C+.

Totum Risk

Totum Risk measures risk capacity along with risk tolerance (or preference), but the instrument leans heavily in the direction of risk capacity. Instead of using a normal distribution, it prefers to account for fat tails in return distributions by replacing the standard deviation measurement with the Sortino ratio. Like FinaMetrica and Riskalyze, Totum Risk will also calculate a prospect or client’s portfolio’s risk exposure – all on the by-now-familiar 1-99 scale. The brevity stands out: There are only 11 questions, some of which relate to a client’s health and career – and you can use a shorter nine-question instrument on mobile devices. The system creates a portfolio proposal that you can give to clients.

The firm also touts the fact that its questionnaire has been reviewed and approved by FINRA as a workable know-your-customer (KYC) tool, but this is handled with just four questions taken from FINRA’s 2011 guidelines that relate to the following: the client’s age, income, number of people in the household, annual expenses, net worth, investment amount and time horizon. This is followed by four questions relating to risk capacity and another four covering risk preferences, including a seemingly naive question that the plaintiff attorney is sure to ask about: How much are you willing to lose over the next 12 months for future gains? This is measured using a slider that the client can move up or down, showing potential gains and losses based on historical market behavior.

The Totum Risk methodology was built by PhDs using research from (according to the website) “Noble [sic] Economic Prize winning psychologists and economists.”

I asked the company to provide research and reports to back up the Ph.D and Nobel Prize claims, and was told they would get back to me. They never did.

My “defensibility in arbitration” score for Totum Risk: C-.

Andes Wealth Technologies

Andes Wealth is a brand new program that, like Riskalyze, is not exclusively a client risk tolerance assessment instrument. The program helps analyze and build portfolios based on research by Prof. Andrew Lo at MIT. Andes Wealth CEO, Helen Yang, co-authored a paper with Lo that won the Harry Markowitz Award in 2011.

The risk tolerance assessment process that Andes Wealth uses is simplicity itself: It shows bar charts, each graphically illustrating potential returns versus downside risk, based on a normal distribution around an expected return for different portfolio mixes. (You can insert your own expected returns, but if you do, you need a way to defend them; the defaults are based on more-easily-defended historical returns.) In all cases, of course, the green bar showing potential returns is higher than the red bar is low. The advisor can adjust the time period and initial investment, so the client can see one-, five- and 10-year possible returns versus possible losses. If the client chooses to input an initial investment amount, then the maximum expected (80% confidence level, roughly two standard deviations) gains and losses are listed in dollar figures at the top and bottom of the bars.

Once the client chooses a portfolio, looking at different time periods, the system evaluates his or her current portfolio and identifies how to move that current portfolio to the efficient frontier at the point the client chose. There is no 1-99 scale here; the client has chosen a portfolio and gets what he or she asked for.

But that’s not all.

Yang has also created an “investor behavior analysis,” based on another of Professor Lo’s papers. This provides an additional dimension to the assessment. Among the profiles: the passive investor who prefers you to handle all the investment mechanics; the trend follower and the safety seeker (who tends to freak out in down markets). Three other profiles are provided– contrarians, risk seekers and adaptive investors – but are less common.

Andes Wealth is the only program in this survey that maps a client to a place on the efficient frontier; the theory being that a risk tolerance instrument might assess a client as a “53” on the 1-99 scale, and then provide a “53” portfolio that might be undiversified or otherwise inefficient. The program lets you show in a colorful graph the return profile of the portfolio compared to the ever-shifting efficient frontier over any time period, allowing clients to see where they are versus the most efficient version of their risk profile over the past six months or one, two or five years and so forth – so the program becomes an ongoing instrument to spur dialogue and provide explanation in the face of extreme market conditions.

The program also offers a robust investment policy statement generator, which might catch the interest of advisors who once relied on IPS AdvisorPro before the Broadridge acquisition of Fi360.

How would you answer a plaintiff’s attorney who was curious about the academic justification of the Andes Wealth assessment? Yang says that the answer is pretty simple: There’s a lot of research on the distributions of risk and return. The client has signed off on one such distribution, with the knowledge that 20% of the time a portfolio might fall somewhere outside of that expected band of returns, either on the upside or the downside. You have their signature and hundreds of studies on the distribution of asset class returns over long and short time periods.

My “defensibility in arbitration” score for Andes Wealth: B+.

Before we finish, I need to add a couple of important caveats. The "defensibility" scores are not intended to evaluate the quality of the programs themselves. If I were trying to design the perfect risk tolerance instrument, I would include the FinaMetrica guard rails that impose a risk limit on short-term investments or on clients with limited investment experience, and add Riskalyze's deep integrations with just about the entire universe of planning software, plus the platform components that are extensions of the risk score. I would include Tolerisk Pro's and Totum Risk's risk capacity measurements, because a client’s bravado doesn't necessarily imply that he or she should embrace a high-risk mix of assets. And until I saw Andes Wealth, I didn't think about the fact that the traditional risk tolerance process could give clients a portfolio that matches their risk level, but is still an extremely inefficient (or badly designed) mix of asset classes. I'd want to map client portfolios not only on the risk spectrum, but also on the efficient frontier at the point of client preference.

Another important caveat: These are not the only risk tolerance instruments on the market. They are, however, the most popular or, in the case of Andes Wealth, among the most feature-rich. They all come with essentially the same lesson: The advisor should not rely entirely on a multiple-choice questionnaire to determine what asset mix to recommend to clients. Each of the instruments profiled here will facilitate a potentially deep conversation about risk and return, up and down markets, and a commitment not to overreact when the ride gets bumpy. Having clear documentation of that conversation, and what the client ultimately decided, is a good defense against the charge that you put a client in a harmfully wrong combination of assets.

Bob Veres' Inside Information service is the best practice management, marketing, client service resource for financial services professionals. Check out his blog at: www.bobveres.com.

More Volatility/Downside Protection Topics >