The Comprehensive Capital Analysis and Review (CCAR) is an annual exercise by Federal Reserve to ensure that bank holding companies (BHCs) have robust, forward-looking capital planning processes that account for their unique unique risks, and whether they have sufficient capital to continue operations throughout times of economic stress. Forecasting losses under varying macro-economic scenarios is a key and integral part of this exercise.
The accurate assessment of a bank’s loss forecast at an account level would require a deep rooted understanding of the multifaceted problem statement and the dynamic interplay of a host of factors, which need to be judiciously incorporated. The data complexity and the limitations of the conventional approaches, which are inherently designed to only classify risky customers, give rise to the necessity for a new methodology.
The stress testing models required for forecasting losses for CCAR exercise are expected to accurately predict losses for each quarter (or month) separately for the forecast horizon (nine quarters), for baseline and various stress scenarios. Various approaches have been used by banks/bank holding companies to develop these models.
Segment level models conventionally used for stress testing purposes assume macro-economic variables to be the major driver of changes in the portfolio of a BHC across various time periods. However, internal portfolio characteristics also play a significant role in explaining the losses as well.
With an increasing focus towards robustness of the stress testing procedures and availability of granular data, account level models provide a competing approach to segment level models and have certain inherent benefits in terms of capturing the correct risk drivers with advanced levels of precision.
In this paper, we present the benefits of the Account Level Mixed Effects modeling approach in the case of loss forecasting, which enables incorporating macroeconomic variables and behavioral variables in a single model equation to forecast losses for next nine quarters under various macro-economic scenarios.
This approach captures the varying predictive power for each predictor through the forecasting horizon, as Mixed Effects models have coefficients of covariates which are time variant.
This approach can be used for estimating losses for various levels of economic stress. Additionally, it provides computational and maintenance advantages of one model vis-à-vis multiple models. This paper highlights the evolution, the current practices of loss forecasting and the future direction in this space.
Evolving Regulatory Space
Post the subprime mortgage crisis in 2008, the Federal Reserve conducted the Supervisory Capital Assessment Program (SCAP) for the 19 largest US BHCs, to address the uncertainty in financial market conditions and their corresponding impact on the health of these banks. SCAP served as a significant contributor in stabilization of the financial system. In addition, for the first time bank by bank results were publically disclosed so as to re-instill the confidence of the investors.
Based on the learnings from SCAP, the Federal Reserve initiated CCAR in 2010. Initially, CCAR covered the same 19 BHCs that had participated in SCAP. However, in November 2011 the Board of Governors of the Federal Reserve issued a rule which required all BHCs with total consolidated assets of $50 billion or more to submit capital plans to the Federal Reserve annually. The Federal Reserve would then assess whether the capital plans submitted are robust, forward looking and whether the banks have sufficient capital to continue their banking operations during stress periods.
The tier 1 capital ratios, which measure a bank’s financial health, of almost all major banks in US have increased since the implementation of CCAR . The main reasons as to why banks have a greater capital cushion are
• Supervisory scenarios complemented with BHC scenarios have helped in capturing idiosyncratic risks associated with each bank
• The banks need to maintain a capital buffer to make sure they have enough capital even in case of the most severely stressed scenario for each of the next nine quarters
• Public disclosures of the CCAR makes the banks vulnerable to market sentiments associated with their health
In 2016, the Fed assessed the capital plans of 33 BHCs and objected on qualitative grounds to capital plans of two banks.
The results demonstrated two overarching messages: The quality of the capital planning process is a more prominent aspect of the Fed’s focus (versus just the quantity of capital), and that the bar continues to rise, especially for the largest firms. Therefore, BHCs must continue to improve their capital planning processes while meeting quantitative capital requirements. To ensure a qualitatively sound capital planning process, a robust model development process is an extremely crucial step.
Going forward as the regulatory stringency increases, the banks would continuously need to enhance their database and model sophistication and granularity so as to stay abreast with the supervisory expectations around capital planning.
Most banks today have been using the segment level PD-EAD-LGD framework for developing loss forecasting models for CCAR purposes. These segment level models provide accurate forecasts at a segment/portfolio level, but are difficult to integrate with business practices as most business decision models are developed at an account level. As a result, the regulatory and business modeling have remained segregated till date.
Account level models address the shortcomings of segment level models and are generally more accurate as compared to segment level models even at a portfolio level. Moreover, segment level models are prone to higher prediction errors in case the portfolio mix changes rapidly, since the internal risk characteristics are generally not included in these models. Account level models incorporate individual account level characteristics in the models and are able to forecast better at granular levels as well. Since account level models can forecast losses at the most granular levels, these models can also facilitate the integration of CCAR models into all other business processes.
However, developing account level models for CCAR has several challenges. The data required to develop these models should be present at an equally granular level, which might not be the case with all banks. Account level methodologies for scorecard development are in practice, but replicating them in the CCAR modeling domain is not straight forward. Since scorecards are inherently designed to predict the credit worthiness of the customers, the most important parameter becomes rank ordering (for example, defining a cut-off for accepting or rejecting the applications) while for CCAR models it is accuracy of the model prediction.
Incorporating macro-economic variables in account level models is another challenge that the banks face today. Developing models which meet all these requirements would require significant research in the way regulatory models are developed today. However, as banks try to stay abreast with the latest trends in regulatory processes, account level models are gaining importance as far as robust model development techniques are concerned.
Prevalent Account Level Approaches
The following are two account level approaches that can be used for CCAR models:
• Transition matrix approach
• Hazard models
These approaches have also been discussed in the Federal Reserve papers. The choice of modeling technique depends on the business requirements, which includes ease of implementation and availability of data across economic cycles. The basic difference is that while a transition matrix approach can be used to predict all delinquency states along with current and charge-off states, hazard models are used to forecast the percentage of population getting charged off after a certain period of time. The approaches have been have been discussed in greater detail below:
Transition Matrix Approach: The delinquency transition models forecast the movement of loans between delinquency buckets given specific economic scenarios. These models offer a significant amount of granularity as they quantify the probability of movement between the delinquency buckets even before the charge-off state is reached. Sound transition models require robust time series of the data containing sufficient sample for each transition to establish statistically significant relationships. Data availability may therefore inhibit the development of granular transition models. In the absence of sufficient data samples, certain transition probabilities need to be based on empirical factors, which may reduce the model accuracy. Hence, although transition models may be able to provide high levels of sophistication, the data might not be able to support such levels of granularity.
"In the absence of a sufficient data samples, certain transition probabilities need to be based on empirical factors, which may reduce the model accuracy."
Moreover, since the target variable in the model involves more than two possible states, this can be achieved either by using multinomial modeling technique or by developing multiple binomial models separately for each delinquency state. This may require significant investment in terms of time and the number of resources involved.
Hazard Models: In this approach, survival and hazard functions are estimated to describe the distribution of time to charge off. The survival function gives the probability of not charging off until that particular instant of time, and the hazard function gives instantaneous failure rate at time T, given that the subject has survived up to time T.
There are multiple types of hazard models (parametric, semi-parametric and non-parametric) which can be used to develop a model, each with its own set of assumptions. Therefore, it becomes crucial to analyze the model development data to determine the most appropriate approach.
Mixed Effects models (discussed in the next section) have coefficients of covariates that are time variant, which is difficult to achieve in traditional hazard models.
Mixed Effects Modeling Approach
The Y14-A CCAR stress test requires the losses to be forecasted over a nine quarter horizon, which can translate into developing separate quarterly models. The Mixed Effects modeling approach enables incorporating macro-economic variables and behavioral variables in a single model equation to forecast losses for next nine quarters for various macro-economic scenarios by allowing for varying coefficients for the predictors. The objective of the models is to predict PD/EAD at an account level in each of the quarters in the forecasting horizon under a given macro-economic scenario. The Mixed Effects modeling approach enables a single model equation capturing the relationship between the dependent variable and the customer behavioral attributes and macro-economic factors. This single equation captures the dynamic relationship between behavior attributes and macroeconomic attributes to predict PD/EAD. Theoretically, the equation for PD/EAD will contain different coefficients for each quarter and it will then use the relevant coefficient while making all others equal to zero. The framework was developed to enable the model to simultaneously capture a) Impact of macro-economic factors and b) Impact of behavioral (on-us and bureau) risk drivers This approach can be used for estimating losses for various levels of economic stress. Additionally, it provides computational and maintenance advantages of one single model vis-à-vis quarterly models.
Accuracy of Prediction
The account level models developed with the above approach have resulted into higher accuracy as compared to existing segment level models. The following table illustrates the comparison of the accuracy between the two models.
The model accuracy was also evaluated at other granular levels (e.g. at sub-portfolio level). It was observed that account level models are accurate even at granular levels as well.
The following table shows the comparison of Mixed Effects modeling approach as compared to other account level approaches:
The fundamental feature which distinguishes the Account Level Mixed Effects modeling approach from the conventionally used CCAR approaches is its ability to predict nine quarters using a single model equation for various macroeconomic scenarios. This approach offers the benefit of having a single equation that gives more control to measure the relationship of variables across different quarters, capturing the continual transition effect of behavioral and macro-economic variables.
The noteworthy benefit of this methodology is its ability to incorporate the impact of both behavioral and macro-economic variables through-the-cycle data to capture stress accurately. Developing these CCAR models for our clients has not only led us to providing a more easy way to manage a one equation solution, but has also resulted in higher model accuracy. One of the other major benefits of developing account level models for regulatory purpose is their ability to get integrated with business models.
Considering the granularity which account level models provide, banks should continue to explore new account level methodologies. To accurately forecast losses, banks should constantly encourage their teams to explore more advanced techniques and strive to replace and refine the existing models to ensure that use of regulatory models is expanded for business purposes as well. This would lead to banks’ enhanced ability to successfully sustain themselves during recessions and would entail faith in regulators and the public in general towards these banks; which at one point were considered to be one of the major forces in driving the US economy towards recession.
- Federal Reserve Release on Comprehensive Capital Analysis and Review (2015) http://www.federalreserve.gov/newsevents/ pres/bcreg/bcreg20150311a1.pdfl
- Federal Reserve Release on Comprehensive Capital Analysis and Review (2014) http:// www.federalreserve.gov/newsevents/press/ bcreg/ccar_20140326.pdf
- Federal Reserve Release on Comprehensive Capital Analysis and Review (2013) http:// www.federalreserve.gov/newsevents/press/ bcreg/ccar-2013-results-20130314.pdf
- Federal Reserve Release on Comprehensive Capital Analysis and Review (2012) http:// www.federalreserve.gov/newsevents/press/ bcreg/bcreg20120313a1.pdf
- History of Basel committee http://www.bis. org/bcbs/history.htm
- Basel II description https://www.princeton. edu/~markus/teaching/Eco467/10Lecture/ Basel2_last.pdf
- History of CCAR http://www.federalreserve. gov/bankinforeg/ccar-and-stress-testing-ascomplementary-supervisory-tools.htm
- An Introduction to CCAR http://riskarticles.com/
- Stress Testing http://docplayer.net/20892839- Stress-testing-a-look-into-the-fed-s-blackbox.html
- Federal Reserve Release on Comprehensive Capital Analysis and Review (2016) http:// www.federalreserve.gov/newsevents/press/ bcreg/bcreg20110318a1.pdf
- All domestic BHCs with year-end 2008 assets exceeding $100 billion
- http://www.federalreserve.gov/bankinforeg/ ccar-and-stress-testing-as-complementarysupervisory-tools.htm
- Probability of Default (PD), Exposure at Default (EAD) and Loss given Default (LGD)