Day One | March 22
8:50 Chair’s opening remarks
PANEL DISCUSSION
Session details
- Scope of model risk and validation work expanding
- Depth of analysis required from regulators
- Trend towards outsourcing aspects of model risk
- Inclusion of validation in outsourcing work
- Maintaining resources with increasing demand
- Drivers behind resource constraints and scaling up internally
- Inclusion of qualitative models under model risk
Justyna van Barneveld, Head of Framework and Reporting, Model Risk Management, ING
Olga Collins, ED, Global Head of Model Risk Infrastructure and Reporting, Morgan Stanley
Irfan Kazi, Director, Advanced Analytics and Data Management, CIBC
QUANTIFYING MODEL RISK
Session details
- Application across model types
- Common approach to quantify model risk capital
- Expectations from senior management
- Communicating to the board
- Impact on books with model risk uncertainty
- Quantifying rare but significant events with limited data
- Managing extreme uncertainty around scenarios
- Building volatility into models
- Managing long term change
- Determining what changes were permanent vs. reactive
- Lessons learnt from rapid change and adaptation
10:30 Morning refreshment break and networking
EXTENDED SESSION
Session details
- Speed of development and deployment
- Application beyond traditional model scope
- Impact on control standards with increased use
- Recognition of process constraints and risks
- Accepting lower control and development standards to stay ahead of competition
- Validation requirements
- Validation frequency
- Automating validation
- Regulatory framework for treatment of AI/ML models
- Application of old concepts to new models
- Model validation and model lifecycle: Developing standardised methodology
- Dynamic adaptation to new data
- Managing reputational risk of model output
- Scaling up model production in a fast paced environment
David Palmer, Senior Supervisory Financial Analyst, Federal Reserve Board
Irfan Khan, Director, Advanced Analytics and Data Management, CIBC
ETHICS AND BIAS
Session details
- Understanding how models are trained
- Imitation of human decisioning
- Feeding unconscious bias into machines
- Managing risk of bias in models
- Managing bias to protected characteristics
- Developing tests and structures into models to demonstrate to regulators
- Ethics of decision making
- Transparency around decision making and ethics
- Producing socially acceptable outcomes
Imir Arifi, Head of Methodologies and Models in the Americas, UBS
12:40 Lunch break and networking
Session details
- New and traditional applications within BD
- Regulatory requirements in the industry across Europe and the US
- Expectations for model risk management
- Successful strategies for model automation
- Outlook and challenges ahead
Julian Horky, Head of Risk Controlling, Berenberg Capital Markets
EVOLUTION – PANEL DISCUSSION
Session details
- Fintech as a driver within model risk
- Behavioural models and uses within areas including financial crime
- Direction of model risk towards machine learning
- Definitions and scope of global regulation
- Evolution of economies and political environments and impact on models
- Foundational capabilities to evolve quickly
- Impact of legacy infrastructures on evolution
- Impact of remote working and resourcing skill gap
- Maintaining proprietary models in remote environment
- Impact of Covid on modeling
- Treatment of extreme events
- Developing models for future resilience
Chris Smigielski, Model Risk Director, Arvest Bank
Saqib Jamshed, MD, Model Risk Management, The OCC
Liming Brotcke, Senior Director, Head of Model Validation, Ally
Stephen Hsu, SVP, Head of Model Risk Management, Pacific Western Bank
3:10 Afternoon refreshment break and networking
COMPUTER VISION
Session details
- Moving away from traditional banking models
- Leveraging remote capabilities to serve customers
- Using selfies and passport photos for authentication
- Move to digital phone services
- Use of bots on WhatsApp for customer service
ESG
Session details
- Literature review of 2021 FRB NY Climate Stress Testing paper
- Case studies of practical approaches in incorporating climate risk in:
- Credit Underwriting
- Portfolio Stress Testing and ACL CECL
Arsa Oemar, Director, MUFG Union Bank
PANEL DISCUSSION
Session details
- Identifying models to use
- Determining approach with internal or external models
- Feedback and expectation from regulators
- Building independent models
- Defining climate risk to advance model processes
Meet Shah, Executive Director, Operational Risk, UBS
5:40 Chair’s closing remarks
5:50 End of day 1 and drinks reception
Day Two | March 23
8:50 Chair’s opening remarks
CULTURE
Session details
- Culture to manage development and deploying of models
- Developing culture to attract talent
- Implementing a technology culture
- Evolution of models and treatment within financial services
- Incorporating model risk at the heart of risk management
Liming Brotcke, Senior Director, Head of Model Validation, Ally
INTERPRETABILITY
Session details
- Regulatory expectation of explainability
- Conceptual soundness evaluation
- Explainable machine learning
- Designing inherently interpretable models
- Incorporating constraints
- Adverse action reason code
Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo
10:20 Morning refreshment break and networking
VALIDATION – PANEL DISCUSSION
Session details
- Recalibrating AI models
- Frequency of validation
- Managing model risk to accommodate changes to validation
- Understanding models in order to validate them
- Increased use across risk siloes
- Parameter and algorithm tuning
- Testing bias in data sets and model inputs
Seyhun Hepdogan, Director, Model Risk Management, Discover Financial Services
Michael Jacobs, Lead Modeling & Quantitative Analytics Expert, PNC Financial Services
Shawn Tumanov, Director, BMO Financial Group
WORKFORCE/TALENT RETENTION
Session details
- Changes to work force with remote and hybrid environments
- Limitations of institution dependent policies
- Attracting academic talent with machine learning expertise
- Mitigating talent migration and staying competitive
- Quantifying value added by workforce
- Talent to understand and use tools available in the market
Wei Ma, Head of Model Risk and Validation, Sumitomo Mitsui Banking Corp
12:20 Lunch break and networking
FAIR LENDING
Session details
- Evolution of model governance structure
- Convergence of fair lending and model risk teams
- Skill migration towards quantitative roles
- Heightened focus on fair lending and fraud
- Managing protected attributes to remove from data elements
- Uses of unstructured data and compliance with privacy laws
REGULATION
Session details
- Introduction and motivation: The importance of quantifying model risk for stress testing
- Review of relevant literature and industry practice
- Hazard rate model framework for stress testing of wholesale obligor PDs
- The principle of relative entropy for measurement of model risk
- Modeling data and econometric results
- Model risk bounds on CECL forecast scenarios
Michael Jacobs, Lead Quantitative Analytics and Modeling Expert, PNC
2:40 Afternoon refreshment break and networking
DATA AND PRIVACY
Session details
- Ensuring unstructured data uses don’t violate privacy laws
- Defining permissible data formats
- Capabilities to use within current models
- Building models to leveraging unstructured data
- Validation requirements
- Social media uses and considerations
AUTOMATION – PANEL DISCUSSION
Session details
- Automating validation on an ongoing basis
- Model governance to monitor models
- Ongoing performance monitoring of models
- Enhancing efficiency of model risk teams
- Reducing human intervention on manual tasks
- Leveraging AI to automate tasks whilst retaining human oversight
- Identifying areas that can be automated effectively
Jing Zou, Managing Director, Model Risk Management, Royal Bank of Canada
Mitchell Button, SVP, Model Validation, US Bank
Florentino Rico, Senior Manager, Model Risk Management, Discover Financial Services
Wei Ma, Head of Model Risk and Validation, Sumitomo Mitsui Banking Corp
4:40 Chair’s closing remarks and end of Summit