Senior Analyst, Model Risk Management

Toast creates technology to help restaurants and local businesses succeed in a digital world, helping business owners operate, increase sales, engage customers, and keep employees happy. As Senior Analyst in our Model Risk Management team, you will help manage the Model Risk Program including completing model validation reviews, maintaining the model risk inventory, and monitoring model performance particularly for Fraud and Gen AI Models. You will work closely with the Data Science Team, architects, engineers and product managers to assess the risk of model design, implementation, and use across product lines. A day in the life (Responsibilities) Support the implementation and day-to-day execution of the Second Line of Defense (2LOD) Model Risk Management (MRM) program for high-risk models, with particular focus on Fraud detection models (Transaction Fraud & Merchant Fraud) and Generative AI / LLM-based systems deployed across Toast. Assist in maintaining and enhancing the Model Risk Management framework, including policies, procedures, validation standards, governance documentation, templates, and best practices aligned with evolving regulatory and industry expectations. Enforce model lifecycle standards across development, implementation, use, monitoring, recalibration, change management, governance, and decommissioning, ensuring appropriate controls for traditional ML models as well as GenAI systems (e.g., RAG architectures, copilots, AI-assisted decision tools). Contribute to the development, risk-tiering, and ongoing maintenance of a comprehensive model inventory, including assessment of model impact, intrinsic risk (complexity and methodology), reliance on model outputs, and emerging AI-specific risks. Perform independent model validation reviews under the guidance of senior leadership, covering conceptual soundness, data integrity, model methodology, performance metrics (e.g., AUC, precision/recall, calibration), stability, bias/fairness risk, explainability, and monitoring frameworks. Produce validation reports and track issue remediation plans through closure. For Fraud models, evaluate class imbalance handling, threshold optimization, cost-sensitive performance metrics, operational overlays, rule-based controls, and portfolio-level impact analyses. For Generative AI systems, validate systems and evaluate risks related to hallucination, prompt injection, adversarial vulnerabilities, data privacy and leakage, model explainability limitations, bias, guardrails, output monitoring, jailbreak testing, regression testing, and human-in-the-loop controls. Partner with Data Science, Data Engineering, Product/Engineering, Information Security, Legal/Compliance, Finance, Credit Risk and Business teams to obtain documentation, perform effective challenge, conduct validation and oversee performance monitoring Prepare reports and executive materials summarizing model risk issues, validation findings, monitoring insights, and remediation status for leadership review, risk committees, audit committees, and internal audit engagements. Research and stay informed on industry developments in fraud analytics, machine learning, Generative AI governance, and regulatory guidance (e.g., SR 11-7, OCC 2011-12, NIST AI RMF). Propose enhancements to strengthen Toast’s Model Risk and AI Governance framework. Contribute to the development of model risk training materials and support delivery of training sessions to key stakeholders to enhance awareness of fraud model risk and AI-related risks. Support ad-hoc initiatives related to model risk governance, AI oversight, regulatory compliance, and enterprise risk management enhancements. What you'll need to thrive (Requirements) Advanced degree in Data Science, Statistics, Applied Mathematics, Computer Science, Engineering, or a related quantitative discipline 5+ years of relevant industry experience in Model Risk Management, Model Validation, Data Science, or Machine Learning within fintech, banking,