Evaluation Lead

Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world-class team, we're unlocking the next era of autonomous transportation with technology that's powering commercial autonomous trucks and robotaxis. Waabi is backed by and partners with world leaders in AI, automotive, logistics, and deep tech. With offices in Toronto, San Francisco, Dallas, and Pittsburgh, Waabi is growing quickly and looking for diverse, innovative and collaborative candidates who want to impact the world in a positive way. To learn more visit: www.waabi.ai We are looking for a hands-on leader to build a new centralized Evaluation team. This team will be responsible for providing comprehensive and holistic analysis on all aspects of performance of the autonomy system. In this role, you will collaborate closely with the systems & safety team, responsible for defining the requirements & evaluation criteria, as well as the autonomy teams to understand their evaluation needs. You will get to work with Waabi World, our highly realistic closed-loop simulation engine built with the latest in generative AI technologies to deliver the evaluation capabilities needed to support the safe development of the next generation of autonomous vehicles! You will... - Lead and build a cross functional team of software engineers, data analysts, and data scientists supporting automated workflows that provide high signal on autonomy performance. - Design scalable production frameworks for sampling evaluation sets, developing and improving metrics, and systematically measuring the performance of both autonomy and the eval ecosystem itself. - Design pipelines, tools, and dashboards to characterize autonomy performance for technical teams and executive leadership, collaborating closely with platform teams on implementation, and autonomy, systems and safety and product teams on requirements. - Work closely with simulation and software teams to build solutions that leverage our data, metrics and simulation platforms effectively. - Lead technical projects; contributing as an IC while also managing the team. - Participate and share ideas in technical and architecture discussions, collaborating with researchers and engineers. - Conduct regular one-on-one meetings to offer guidance and constructive feedback to direct reports. Qualifications: - Minimum of 6+ years of autonomous vehicle industry experience including at least 2+ years managing high performing teams - Experience evaluating AI or machine learning models, ideally in self-driving or related fields - MS/PhD or Bachelors degree in Computer Science, Data Science, Robotics and/or similar technical field(s) of study - Strong statistical background - Experience working with internal cross-functional partners/stakeholders - Experience with system design/architecture and algorithms - Open-minded and collaborative team player with willingness to help others - Passionate about self-driving technologies, solving hard problems, and creating innovative solutions. Bonus/nice to have: - Previous experience leading Autonomy Evaluation teams - Experience with large scale databases and analytics