"Stress Testing: What's Inside the Black Box?"

Chairman Barr, Ranking Member Foster, Members of the Subcommittee, thank you for the opportunity to testify today. Stress tests have played an important role in our country's financial history. In 2009, Federal Reserve supervisors used the Supervisory Capital Assessment Program to evaluate how well the largest banks would fare if the Global Financial Crisis got even worse. The results forced a handful of banks to raise capital. Perhaps more importantly, publication of the results assuaged the worst fears of market participants. A couple years later, Congress and regulators had created a mandate of regular stress tests for the largest financial institutions, and stress testing had become a central part of bank oversight across the world.

At the time of the crisis-era stress test, I was a Federal Reserve analyst covering several banks subject to the stress test. Two of my fellow panelists today were also involved. I think they would agree with me that the 2009 stress test was a revelation and a revolution in the practice of banking supervision. With mandatory stress testing, bank supervisors supplemented our traditional point-in-time analysis of balance sheets and risk-management practices with a forward-looking, analytical approach to bank risks. An all-around good thing.

I would like to make four points about the state of U.S. stress testing today.

* Supervisory stress tests should be countercyclical.

* Stress tests should use multiple scenarios.

* Stress tests are part of a broader regulatory and supervisory toolkit.

* Transparency can be a double-edged sword.

First, supervisory stress tests should be countercyclical.

In a 2019 paper, my coauthor and I discussed the difference between stress testing in "war time," during a financial crisis, and "peace time," when conditions have eased. n1 At the time, we observed that stress tests that influence capital requirements will inevitably introduce some volatility into capital planning. And bankers will always raise that concern. But today, five years later, after various changes to the tests and a significant increase in disclosures, we face the opposite risk--that the stress tests have become too predictable.

Regulators must keep their guard up during peace time--during those long periods of financial calm when the possibility of a banking crisis and its associated costs appear remote. Stress scenarios can appear fanciful to bankers and regulators whose careers may have begun some time since the last crisis--or, more cynically, whose remuneration is linked to short-term profit metrics as opposed to long-run viability and hedging tail risks. To be effective, stress test scenarios must be sufficiently rigorous and they must have countercyclical elements.

Certain aspects of our political process tend to favor procyclicality, the letting down of our guard during peace time. When bad things happen, we tighten rules, we strengthen risk monitoring, we chastise supervisors. But over some years, economies recover, financial metrics recover, people in responsibility turn over. The regulated are empowered and voices of caution are dimmed. There is the risk that stress testing could become less rigorous over time, or the risk that the rigor of stress testing could vary from one administration to the next. That's what happened in 2018. With regulatory tailoring, the number of banks subject to the stress tests was significantly reduced. Other aspects of the process were also eased.

As an aside, regulatory tailoring of course eased a number of other prudential standards for large banks, not just stress tests. Along those lines, I published a series of blogs last year analyzing the question: Did regulatory tailoring have anything to do with the failure of Silicon Valley Bank, or SVB, the first large bank to fail last year, or the costs of that failure? The answer was mixed. For example, we concluded that SVB would likely have chosen a different funding strategy, or held different liquid assets, if it had been subject to the liquidity coverage ratio. n2 This could have made a difference. We also noted that the Federal Deposit Insurance Corporation could have saved billions of dollars if SVB and other failed large banks had held more senior debt that could have borne losses under a standard that regulators then applied only to larger banks. n3

It's probably no coincidence that the global financial crisis of 2008-09 is now around the same distance from us as the savings and loan crisis was in the mid-2000s, the last time there was so much pressure to ease bank regulation.

Second, stress tests should use multiple scenarios.

Financial crises tend to emanate from unexpected places. Many have argued for multiple scenarios in Federal Reserve stress tests. My colleagues and I at the Office of Financial Research made this suggestion a decade ago. n4 When the Federal Reserve sought feedback on its Comprehensive Capital Analysis and Review (CCAR) program in 2018, we argued for improvements to the supervisory stress tests to evaluate the interaction between funding and solvency and to address interest rate risk. n5

Other advanced-economy supervisors have introduced diverse scenarios and methodologies. The Bank of England and European Central Bank conduct "exploratory" scenarios every other year. This year, the BoE put the annual stress test on hold in order to conduct a desk-based exercise with multiple scenarios and a broader test involving both banks and nonbanks. n6 This year, the ECB is evaluating the impact of a cyber attack. In contrast, the core scenario structure in the Federal Reserve's stress testing has remained little changed after a full decade of use. However, this year, for the first time, the Federal Reserve has also introduced non-binding exploratory scenarios, as well as a climate scenario analysis. n7

In short, the US innovated the use of supervisory stress tests during the 2008-09 crisis. Early efforts in other jurisdictions didn't measure up to ours. But in later years the innovation was elsewhere, as ours until recently remained static.

Stress tests are part of a broader regulatory and supervisory toolkit.

Of course, stress tests aren't the right tool for every risk. Some have argued the Federal Reserve should have included SVB in the stress test; and that the stress test should have included a rising interest-rate scenario, since that's what sank SVB. That probably would have been a useful exercise. Still, it might not have done the trick. Francisco Covas of the Bank Policy Institute, who is here today, has done analysis showing that subjecting SVB to a stress test with an interest-rate scenario would not necessarily have resulted in greater capital for SVB. Rather, he says, rising rates would have boosted SVB's net interest income, making up to some extent for the decline in asset values. n8

Francisco may have been talking his book, but he has a point. In 2020, the International Monetary Fund made that same point--that stress tests may not be the best approach to interest-rate risk--in its periodic review of US financial stability oversight. n9 Instead, the IMF has argued repeatedly for the Fed to implement a Basel standard for the management of interest-rate risk in the banking book (IRR-BB) with quantitative thresholds, as many other countries have done. I made a similar argument in a blog last year. n10

US regulators initially worked with the Basel Committee on Banking Supervision, a global group of supervisors, to develop the IRR-BB standard, ironically at the behest of Congress in the FDIC Improvement Act of 1991 after the U.S. savings and loan crisis. The committee agreed on the standard in 2004 and made it more rigorous in 2016, when supervisors globally noted that the very low interest-rate environment at the time wouldn't last forever. Yet, US financial regulators never implemented the standard. Thus we find our banking sector again with large unrealized securities losses relative to GDP. There's no doubt that SVB, First Republic, and other failed banks would have been out of compliance with the Basel interest-rate risk standard, some catastrophically, had US regulators implemented the standard in the years leading up to March 2023 (see Appendix 1, The Basel IRR-BB standard and Silicon Valley Bank).

The lesson is that supervisors need many tools in their toolkit. A lesson that was again driven home recently when supervisors rejected several big banks' resolution plans.

Fourth, transparency can be a double-edged sword.

Too much transparency can be a bad thing. My colleague and I wrote a little about that in our 2019 paper. We noted first that there are two types of transparency: transparency of outputs-- how did each bank do in the stress scenarios?--and transparency of inputs--what models did the Fed use? For both, our message was that transparency in stress testing, as in banking supervision generally, is a double-edged sword. For outputs, the goal would be to reveal just enough about banks to help market participants evaluate their risks. But not so much as to undermine confidence when it should not be undermined, and especially not to create widespread panic. For inputs, the goal would be to reveal just enough to help banks develop their models and manage their risks, and to inform regulators and markets about bank resiliency. But not so much that stress tests become a predictable "compliance exercise" or become easy to game.

I'll make two observations about those tradeoffs.

First, US supervisors are already revealing a lot about the stress test methodologies to the regulated industry, which may allow banks to merely optimize to the stress test rather than build resiliency. The Federal Reserve's disclosures about its models and methods, along with the Bank of England's, are far more transparent than other authorities across the world (see Appendix 2, Ex Ante Disclosure of Stress Testing Inputs for Selected Large Jurisdictions). n11

Second, look what happened to the failed government-sponsored enterprises, Fannie Mae and Freddie Mac. They were subject to Congressionally mandated stress tests from 2002 to 2008, but those tests did not detect the growing risk. In fact, they showed strong capital compliance.

What happened? In retrospect, the law creating the stress-test mandate had important weaknesses. It codified the inflexibility of the central stress scenario. An analysis by three Federal Reserve economists, published after the GSEs' failure, noted that the supervisor, the Office of Federal Housing Enterprise Oversight (OFHEO), never tweaked its model estimates nor introduced new variables, despite well-documented changes in mortgage underwriting practices during this time. The authors argued that the problem, in part, was that details were published in advance for public comment. The paper found,

"One potential reason for this static approach was that OFHEO was required by law to fully disclose the stress test model and went so far as to publish all stress scenarios, empirical specifications, and parameter estimates in the Federal Register. Hence, it would have been administratively cumbersome to make any material changes to the underlying models." n12

Conclusion

In short, supervisory stress tests are very important for bank risk management and supervisory oversight. I'm concerned, as are many others, that the tests have become too routinized and bureaucratized, and that further disclosures from regulators will simply make them more so.

With thanks to outstanding research support from Vincient Arnold.

n1 Greg Feldberg and Andrew Metrick, "Stress Tests and Policy," Journal of Financial Crises, Yale Program on Financial Stability, Vol. 3 (1), pp. 1-19, available at https://papers.ssrn.com/sol3/papers.cfm7abstract id=3424327.

n2 Greg Feldberg, "Lessons from Applying the Liquidity Coverage Ratio to Silicon Valley Bank," March 27, 2023, available at https://som.vale.edu/storv/2023/lessons-applving-liquiditv-coverage-ratio-silicon-vallev-bank.

n3 Greg Feldberg and Carey Mott, "The 2023 Banking Crisis: Lessons about Bail-in," July 6, 2023, available at https://som.vale.edu/storv/2023/2023-banking-crisis-lessons-about-bail.

n4 Rick Bookstaber, Jill Cetina, Greg Feldberg, Mark Flood, and Paul Glasserman, "Stress Tests to Promote Financial Stability: Assessing Progress and Looking to the Future," Journal of Risk Management in Financial Institutions, Vol. 7 (1), Winter 2013/14, pp. 16-25, available at https://papers.ssrn.com/sol3/papers.cfm7abstract id=2438285.

n5 Jill Cetina, "Incorporating Liquidity Shocks and Feedbacks in Bank Stress Tests," Office of Financial Research Working Paper 2015-06, July 22, 2015, available at https://www.financialresearch.gov/briefs/files/OFRbr-2015-06-Incorporating-Liquidity-Shocks-and-Feedbacks-in-Bank-Stress-Tests.pdf.

n6 Bank of England, "Record of Financial Policy Committee meetings on Sept 26 and Oct 5 2023," pp. 17-18, available at https://www.bankofengland.co.uk/-/media/boe/files/financial-policv-summarv-and-record/2023/fpc-summary-and-record-october-2023.pdf.

n7 Michael Barr, "Multiple Scenarios in Stress Testing," October 19, 2023, available athttps://www.federalreserve.gov/newsevents/speech/barr20231019a.htm; Dan Tarullo, "Reconsidering Regulatory Uses of Stress Testing," Brookings, Hutchins Center Working Paper #92, May 2024, available at Reconsidering Regulatory Uses of Stress Testing.

n8 Francisco Covas, "How Did Regulatory Tailoring Affect SVB's Capital Requirements?," Bank Policy Institute, May 3, 2023, available at https://bpi.com/how-did-regulatory-tailoring-affect-svbs-capital-requirements/.

n9 International Monetary Fund, "United States: Financial Sector Assessment Program--Technical Note on Banking Supervision and Regulation," IMF Country Report No. 20/248, August 2020, pages 37-38, available at https://www.imf.org/en/Publications/CR/Issues/2020/08/07/United-States-Financial-Sector-Assessment-Program-Technical-Note-Banking-Supervision-and-49657.

n10 Greg Feldberg, "US Banks' Interest-Rate Risk Reporting and Regulation: A Comparative Context," April 26, 2023, available at https://som.vale.edu/storv/2023/us-banks-interest-rate-risk-reporting-and-regulation-comparative-context.

n11 Financial Stability Institute, Online appendix - Stress-testing banks - a comparative analysis, page 22, available at https://www.bis.org/fsi/publ/insights12 appendix.pdf.

n12 Scott Frame, Kristopher Gerardi, and Paul S. Willen, "The Failure of Supervisory Stress Testing: Fannie Mae, Freddie Mac, and OFHEO," Federal Reserve Bank of Boston Working Paper No. 15-4, page 3, available at https://www.bostonfed.org/publications/research-department-working-paper/2015/the-failure-of-supervisory-stress-testing-fannie-mae-freddie-mac-and-ofheo.aspx.

Read this original document at: https://docs.house.gov/meetings/BA/BA20/20240626/117459/HHRG-118-BA20-Wstate-FeldbergG-20240626.pdf