The Rise of AI-Powered Operational Risk: A New Era of Scenario Analysis
Banks are facing increasing pressure to not just identify operational risks, but to truly understand their potential impact. Traditional scenario analysis, while valuable, has often struggled with defending the assumptions behind extreme loss events. Now, a convergence of factors – more ambitious risk management goals, the demise of the Advanced Measurement Approach (AMA), and advancements in artificial intelligence – is driving a shift towards a more structured, data-driven approach. Experts at Elseware and JPMorgan Chase are leading the charge, exploring how AI can “industrialize” the challenge process and build more robust risk models.
From Capital Modeling to Forward-Looking Risk Management
The role of scenario analysis has evolved significantly. Previously, under the AMA framework, scenarios primarily served capital modeling, adding synthetic data points for regulatory purposes. With AMA’s removal, the focus has shifted to proactive risk management – encompassing internal capital adequacy assessment, operational resilience, and recovery planning. This transition demands scenarios that aren’t just plausible narratives, but “parameterized mechanisms that generate losses,” according to Patrick Naim, CEO of Elseware.
Nedim Baruh, Managing Director and Head of Operational Risk Measurement and Analytics at JPMorgan Chase, emphasizes the increased ambition within the industry. Practitioners are raising their own expectations, recognizing the potential of forward-looking risk assessments. However, translating this ambition into widespread acceptance remains a challenge. While there’s growing interest from management and risk owners, skepticism persists regarding the robustness of the results.
The XOI Approach: Structuring Operational Risk
The demand for structure in operational risk analysis isn’t new. Over 15 years ago, Elseware developed the Exposure, Occurrence, and Impact (XOI) approach to address a paradox: institutions were expected to identify events with probabilities exceeding 0.1%, yet relied on backward-looking modeling techniques. XOI draws parallels to credit and market risk modeling, focusing on the mechanisms that generate losses. Exposure, represents the resources at risk – people, systems, transactions, or products.
Baruh highlights a key problem with earlier scenario analysis: reliance on subjective expert opinions. Without a structured framework, validating these opinions and understanding the factors driving potential billion-dollar losses proved difficult. Structured scenarios, aim to identify risk factors and describe the loss-generating mechanism – how those factors interact to produce a loss.
Improving Practice and Remaining Limitations
Structured scenarios have demonstrably improved the challenge process, enabling stronger discussions with validation teams. However, limitations remain in evidencing the assumptions behind risk factors. While some assumptions are data-driven, many still rely on subject matter expertise, making it difficult to demonstrate robustness, particularly when dealing with rare, extreme events.
Naim points to the importance of granularity when defining exposures. They must be defined as independently exposed units, which can be challenging in practice. For example, assessing cyber risk at the workstation level or market manipulation at the desk level is insufficient; a broader perspective – such as product or product-country combinations – is required.
AI’s Emerging Role: Automating the Challenge
The recent surge in interest surrounding AI, particularly large language models (LLMs), stems from their potential to leverage vast knowledge and identify risks. However, Naim cautions that LLMs primarily interpolate, building on existing knowledge rather than exploring entirely new territory. Their non-deterministic outputs require safeguards to avoid confusion in automated applications.
The most promising application of AI lies in automating aspects of the challenge process. Instead of directly generating or quantifying scenarios, AI can be used to find evidence that challenges existing assumptions. This approach, explored through Elseware’s TrustAgent, starts with a human assessment and then leverages AI to identify potentially contradictory information.
Baruh emphasizes that the goal isn’t to replace human judgment, but to structure the challenge process, making it more systematic and less biased. This could lead to more transparent documentation for model validation teams and a more efficient use of resources.
“Structured scenarios have clearly improved the execution of scenario analysis, particularly the ability to challenge assumptions. Discussions with validation teams are much stronger than under earlier, less structured approaches.”
Nedim Baruh, JPMorganChase
Industrializing the Challenge: A Three-Step Process
“Industrializing the challenge process” involves a three-step approach: identifying potential sources that challenge an assumption, comparing those sources to the original assessment, and determining whether the information supports, contradicts, or is irrelevant. This process, while still requiring human judgment, is enhanced by AI’s ability to efficiently analyze large volumes of data.
The ultimate aim is to shift the focus of operational risk governance from defending numbers to documenting and challenging the reasoning behind them. By providing automated support for the challenge process, discussions can center on logic and evidence, rather than debating specific figures.
FAQ
Q: What is the XOI approach?
A: It’s a framework for structuring operational risk analysis, focusing on Exposure (resources at risk), Occurrence (probability of an event), and Impact (cost of the event).
Q: How can AI help with scenario analysis?
A: AI can automate the challenge process by identifying evidence that supports or contradicts existing assumptions, making the process more robust, and transparent.
Q: What are the limitations of using AI in operational risk?
A: LLMs primarily interpolate, not extrapolate, and their non-deterministic outputs require careful management.
Q: What is TrustAgent?
A: It’s a research framework developed by Elseware to automate the challenge process in operational risk scenario analysis.
Pro Tip: Don’t rely solely on AI. Human judgment and expertise remain crucial throughout the scenario analysis process.
Did you know? The shift away from the AMA framework has fundamentally changed the purpose of scenario analysis, moving it from capital modeling to proactive risk management.
To learn more about structured operational risk modeling, visit Elseware.
What are your thoughts on the future of AI in operational risk? Share your insights in the comments below!
