Staff Engineer, Autonomy Evaluation (VLA)

<p id="isPasted"><span style="font-family: helvetica, arial, sans-serif;">We are <strong>CARIAD</strong>, an automotive software development team with the Volkswagen Group. Our mission is to make the automotive experience safer, more sustainable, more comfortable, more digital, and more fun. To achieve that we are building the leading tech stack for the automotive industry and creating a unified software platform for over 10 million new vehicles per year. We’re looking for talented, digital minds like you to help us create code that moves the world. Together with you, we’ll build outstanding digital experiences and products for all Volkswagen Group brands that will transform mobility. Join us as we shape the future of the car and everyone around it.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Role Summary:</u></strong></span></p> <p><span style="font-family: helvetica, arial, sans-serif;">The Staff Engineer, Autonomy Evaluation (VLA) leads the development of evaluation methods, metrics, and tooling for VLA / VLAM-based automated driving at CARIAD.</span><br><br><span style="font-family: helvetica, arial, sans-serif;">This role turns model behavior into clear performance evidence across route-level, scenario-level, and regression testing. You will build the evaluation loop that connects data, simulation, model iteration, and in-vehicle performance, helping the team improve driving quality quickly and make confident technical decisions.</span><br><br><span style="font-family: helvetica, arial, sans-serif;">You will work closely with ML, systems, and integration engineers to define how progress is measured for both demo performance and future productization.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Role Responsibilities:</u></strong></span></p> <p><span style="font-family: helvetica, arial, sans-serif;"><strong>Evaluation Strategy &amp; Metrics</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Define and evolve evaluation strategies for VLA / VLAM-based driving performance</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Design metrics and success criteria spanning safety, comfort, robustness, and generalization</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Create benchmarks for route-level, scenario-level, and long-tail/edge-case behavior</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong>Evaluation Pipelines &amp; Tooling</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Develop scalable evaluation pipelines across offline logs, simulation, and in-vehicle test results</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Build tooling for dataset slicing, scenario curation, and automated report generation</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Enable reproducible evaluation with clear versioning of data, models, and metrics</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong>Regression Detection &amp; Failure Analysis</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Detect and diagnose regressions across model iterations and software releases</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Establish a structured forensic / root-cause analysis workflow for driving failures and edge cases</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Translate findings into actionable improvements for model training, data collection, and system integration</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong>Cross-Functional Partnership &amp; Decision Support</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Partner with ML and integration engineers to assess model quality and prioritize fixes</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Support release and demo readiness decisions with clear, quantitative evidence</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Communicate results and trade-offs to technical and non-technical stakeholders</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong>Technical Leadership &amp; Standards</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Define best practices for evaluation methodology, metrics hygiene, and reporting</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Mentor engineers and raise the bar for rigor and speed of iteration in the evaluation loop</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Contribute to long-term evaluation architecture for productization</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>General Skills:</u></strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Systems-level thinking and ability to translate ambiguous driving behavior into measurable outcomes</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong analytical skills in benchmarking, regression analysis, and experiment design</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Ability to connect quantitative findings to practical engineering decisions and release readiness</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong cross-functional communication skills; comfortable working across teams and time zones</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">High ownership mindset with rigor, curiosity, and a bias toward action</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Required Specialized Skills:</u></strong></span></p> <ul id="isPasted"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Deep understanding of autonomy evaluation for ADAS / AD, robotics, or embodied AI systems</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong experience with performance metrics, benchmarking, regression analysis, and scenario-based validation</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Strong software engineering skills in Python; experience building robust evaluation or data pipelines</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience working with simulation, large-scale driving data, or model-in-the-loop evaluation</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Desired Skills:</u></strong></span></p> <ul id="isPasted"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience with foundation models, VLMs / VLAMs, or end-to-end driving models</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Familiarity with learned evaluators, LLM/VLM-assisted evaluation, or agentic workflows for analysis and triage</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Experience in autonomous driving, robotics, or other safety-critical systems</span></li> <li style="font-family: helvetica, arial, sans-serif;">&nbsp;</li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Workplace Flexibility:</u></strong></span></p> <ul id="isPasted"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Collaborate across time zones; occasional early/late meetings to align with global partners</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Occasional travel as needed for vehicle testing, integration workshops, or demos</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Years of Relevant Experience:</u></strong></span></p> <ul id="isPasted"> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">8+ years of experience in autonomy, robotics, machine learning, or evaluation for complex technical systems</span></li> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">3+ years of experience building metrics, validation frameworks, or evaluation pipelines for ML-based systems</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Required Education:</u></strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">Master’s degree in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Desired Education:&nbsp;</u></strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif;"><span style="font-family: helvetica, arial, sans-serif;">PhD in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field</span></li> </ul> <p><span style="font-family: helvetica, arial, sans-serif;"><strong><u>Compensation</u></strong></span></p> <p><span style="font-family: helvetica, arial, sans-serif;">Salary range is dependent on factors such as geographical differentials, credentials or certifications, industry-based experience, qualification and training. In the city of Mountain View, California, the salary range for this position is $191,267 - $269,203.</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">CARIAD, Inc. provides performance based merits and annual bonus along with a competitive benefits package. Benefits include medical, dental, vision, 401k with employer match and defined contribution plan, short and long term disability, basic life and AD&amp;D insurance, employee assistance program, tuition reimbursement and student loan repayment plans, maternity and non-primary caregiver leave, adoption assistance, employee referral program and vacation and paid holidays. We also offer a unique vehicle lease program that covers registration and insurance fees.&nbsp;</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">CARIAD is an Equal Opportunity Employer.  We welcome and encourage applicants from all backgrounds, and do not discriminate based on race, sex, age, disability, sexual orientation, national&nbsp;origin, religion, color, gender identity/expression, marital status, veteran status, or any other characteristics protected by applicable laws.&nbsp;</span></p> <p><span style="font-family: helvetica, arial, sans-serif;">Employment with CARIAD Inc. is subject to export control and sanctions compliance. Some positions may involve access to technology and/or software source code subject to U.S. legal restrictions on release to certain foreign persons based on citizenship or permanent residence. To ensure compliance, applicants will be required to provide information for screening. Employment may be contingent on the outcome, including verification of U.S. citizenship or lawful permanent resident status, or confirmation that a license, exemption, or exception applies.&nbsp;CARIAD retains the discretion to decline to obtain a required license in any case.&nbsp;By applying, you acknowledge and agree to participate in this process.</span></p>