Advisor Perspectives welcomes guest contributions. The views presented here do not necessarily represent those of Advisor Perspectives.
In medicine, likelihood ratios improve patient outcomes and refine drug regimens by assessing the reliability of common diagnostic tests. In finance, likelihood ratios can quantify the reliability of an economic indicator such as one designed to identify recessions.
Specifically, using likelihood ratios, I will show that the ECRI coincident index is a poor tool for recognizing recessions, while their weekly leading index is a better indicator.
In medicine, likelihood ratios are used to estimate how much the probability that a patient has a particular disease changes from before a diagnostic test is given to after its result is known. (For those interested, precise definitions of the various terms, calculation methods and application to medicine are described here and here.) One can use the same concept to calculate the probability of the economy being in recession when an indicator is positive or negative.
At any particular time a recession indicator can be sending one of four messages, depending on the actual state of the economy and where it is to the recession “trigger” value that causes its prediction to change. The possibilities are a correct recession call (true positive), a false recession call (false positive), a valid all-is-well (true negative), or that the indicator is missing the existence of a recession (false negative). How often any one of the conditions occurs, together with the length of the observation period and the length of the recessions, are the raw data for my analysis.
Likelihood ratios for the ECRI coincident index
Let’s assess the value of testing for a recession using the year-on-year growth rate of the ECRI coincident index (ECRI-COg) and applying the concept of likelihood ratios. Specifically, I will calculate the probability that this indicator has correctly identified recessions when it declines to or below a level of 2%. This indicator and criterion was used by ECRI’s CEO, Lakshman Achuthan, to support his February 2012 claim that a recession was imminent. (The ECRI-COg is currently at a level of 2.48%, above the recession trigger line of 2%.) A graph of this indicator can be found in this commentary.
The period for which I analyzed this indicator is from January 1968 to February 2012, totaling 2,298 weeks. There were seven recessions during this period which lasted a total of 360 weeks. An additional 14 weeks (the three-month period prior to each of the seven recessions) were also counted as recession, in order not to penalize for early recession warnings, which a good leading indicator should provide. Therefore the total period with recession was 458 weeks (condition positive) and the total period without recession was 1840 weeks (condition negative).
Figure 1 shows a diagrammatic representation of the input data for the calculation. The actual data for this indicator is in the table 1 below.
From the data we can determine the sensitivity and the specificity of the test. The former is the ratio of true positives to the total period with recession; the later is the ratio of true negatives to the total period without recession, as shown in table 1.
Table 1 |
|
Recession
|
|
|
Exists |
Doesn’t exist |
|
Positive |
True Positive (TP) |
False Positive (FP) |
Test |
345 |
399 |
Result |
Negative |
False Negative (FN) |
True Negative (TN) |
|
113 |
1441 |
|
|
Sensitivity |
Specificity |
|
|
= TP / (TP + FN) |
= TN / (FP + TN) |
|
|
= 345 / (345 + 113) |
= 1441 / (399 + 1441) |
|
|
= 0.7533 |
= 0.7832 |
We can now calculate two important measures. First, the likelihood ratio positive (LR+), which is the probability of a test being positive when the indicator signals recession, divided by the probability of a test being positive when the indicator is not in recession territory.
Second, the likelihood ratio negative (LR–), which is the probability of a test being negative when the indicator is in recession territory divided by the probability of a test being negative when the indicator is not in recession territory.
The likelihood ratio positive (LR+) and likelihood ratio negative (LR–) for this indicator are:
- LR+ = sensitivity / (1 − specificity) = 0.7533 / (1 − 0.7832) = 3.4737
- LR– = (1 − sensitivity) / specificity = (1 − 0.7533) / 0.7832 = 0.3150
In evidence-based medicine, a good test is one that has a likelihood ratio positive of about 10 or a likelihood ratio negative of about 0.1. A recession test using ECRI-COg as the recession indicator would not, therefore, be considered a good test. Its LR+ and LR– of 3.474 and 0.315, respectively, do not even come close to meeting the medical criteria.
Estimation of probabilities
When the ECRI-COg signals a recession (a positive test), the chance that the economy is actually in or near recession increases, but only to 46.37% (up from 19.93% before the test is applied), which is approximately equal to the probability of winning when betting on black or red playing roulette in a casino. Similarly, when ECRI-COg signals no recession (a negative test) the chance of the economy being in recession declines from 19.93% to 7.27%. (See appendix A for the calculation of probabilities.)
Additionally, my own recession indicator evaluation system calculated a very low score of -4.176 for this indicator, confirming the implications of the likelihood ratios.
Other indicators
A better indicator to identify recessions is the year-on-year growth rate of the 10-week rolling average of ECRI’s weekly leading index (WLI-yoy), which signals recession when, exponentially smoothed, it is below 3%. This indicator has been signaling recession since the middle of October 2011, when ECRI made its initial recession call. The indicator is shown in figure 2, with 3.0% added to make zero the recession trigger line.
The likelihood ratios for this indicator are LR+ = 17.35 and LR– = 0.33. The high value of LR+ (greater than 10) makes WLI-yoy a good indicator when it tests positive for recession. From LR+ one can determine that, when WLI-yoy signals recession (when the indicator is below the recession trigger line), the chances of the economy being in or near recession is 81.2%. Similarly, the high value of LR– (greater than 0.1) makes WLI-yoy a bad indicator when it tests negative for recession. Therefore this indicator does not provide useful results when it signals no recession (when the indicator is above the recession trigger line).
My own recession indicator evaluation system calculated a better score – though still not a good one – of -0.433 for this indicator. Note that this indicator is firmly in recession territory now.
Conclusion
Visually observing the indicator is not an adequate way to assess its usefulness. It is more rigorous – but nonetheless still quite easy – to calculate likelihood ratios to determine the effectiveness of a recession indicator to signal a recession. And this is a technique that applies not just to recession calling but to any numerical test used for forecasting! Likelihood ratios are a valuable addition indeed for any advisor’s toolkit.
Appendix A
The positive post-test probability after testing with ECRI-COg is calculated as follows:
- Pre-test probability = (TP + FN) / 2298 = 458 / 2298 = 0.1993 or 19.93%
- Pre-test odds = 0.1993 / (1 – 0.1933) = 0.2489
- Post-test odds = 0.2489 * LR+ = 0.2489 * 3.4737 = 0.8646
- Positive post-test probability = 0.8646 / (0.8646 + 1) = 0.4637 or 46.37%
The negative post-test probability after testing with ECRI-COg is calculated as follows:
- Post-test odds = 0.2489 * LR– = 0.2489 * 0.3150 = 0.0784
- Negative post-test probability = 0.0784 / (0.0784 + 1) = 0.0727 or 7.27%
Appendix B
There are a few recession indicators which provide high probabilities that a recession exists when the indicator is in recession territory and also provide low probabilities for the economy to be in recession when the indicator does not signal recession.
Index |
SuperIndex Coincident |
ADS +0.83 Coincident |
SuperIndex Leading |
CB-LEIg +2.35% Leading |
Condition Positive |
360 |
360 |
458 |
458 |
True Positive |
356 |
291 |
414 |
387 |
False Positive |
39 |
33 |
85 |
90 |
Condition Negative |
1938 |
1938 |
1840 |
1840 |
True Negative |
1899 |
1905 |
1755 |
1750 |
False Negative |
4 |
69 |
44 |
71 |
Sensitivity |
0.9889 |
0.8083 |
0.9039 |
0.8450 |
Specificity |
0.9799 |
0.9830 |
0.9538 |
0.9511 |
Likelihood Ratio +ve |
49.1402 |
47.4712 |
19.5674 |
17.2751 |
Likelihood Ratio -ve |
0.0113 |
0.1950 |
0.1007 |
0.1630 |
Pos post-test probability |
90.13% |
89.81% |
82.97% |
81.13% |
Neg post-test probability |
0.21% |
3.50% |
2.45% |
3.90% |
Georg Vrba is a professional engineer who has been a consulting engineer for many years. In his opinion, mathematical models provide better guidance to market direction than financial “experts.” He has developed financial models for the stock market, the bond market and the yield curve, all published in Advisor Perspectives. The models are updated weekly. If you are interested to receive theses updates at no cost, send email request to .
Read more articles by Georg Vrba, P.E.
