Senior Software Quality Engineer - AI

<div class="content-intro"><p><strong>About Boomi and What Makes Us Special</strong></p> <p><span style="font-weight: 400;"><span style="font-weight: 400;">Are you ready to work at a fast-growing company where you can make a difference? Boomi aims to make the world a better place by connecting everyone to everything, anywhere. Our award-winning, intelligent integration and automation platform helps organizations power the future of business. At Boomi, you’ll work with world-class people and industry-leading technology. We hire trailblazers with an entrepreneurial spirit who can solve challenging problems, make a real impact, and want to be part of building something big. If this sounds like a good fit for you, check out </span><a href="http://www.boomi.com/"><span style="font-weight: 400;">boomi.com</span></a><span style="font-weight: 400;">&nbsp; or visit our&nbsp;</span><a href="https://boomi.com/company/careers/"><span style="font-weight: 400;">Boomi Careers</span></a><span style="font-weight: 400;"> page to learn more.</span></span></p> <p>&nbsp;</p></div><p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"><strong>How You'll Make An Impact</strong></span></p> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">As a Senior Software Quality Engineer, you will bridge traditional software quality engineering with cutting-edge AI quality practices. You will design and implement comprehensive testing strategies that ensure both the reliability of backend infrastructure and the quality of Generative AI features powering our agentic applications. This role combines deep technical expertise in automated testing &amp; AI/ ML focused quality engineering, enabling you to establish quality standards that span from microservices architecture to AI/ML models. You'll be instrumental in building the testing frameworks and quality processes that ensure our products are robust, reliable, and production-ready.</span></p> <h2><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">What You Will Do</span></h2> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Design and execute comprehensive test plans for backend services, APIs, and microservices architectures.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Partner with Backend Engineers, AI Engineers, and Product teams to understand requirements and identify quality risks early in the development cycle.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Develop automated testing frameworks using tools like Pytest, Playwright, unittest, and integration testing libraries.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Build end-to-end test suites that validate the integration between backend systems, AI agents, databases &amp; services.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Advocate for quality-first practices, influencing architectural decisions and embedding testing into the development lifecycle.</span></p> </li> </ul> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Design and implement automated evaluation frameworks for Generative AI features, including LLM / SLM model testing, prompt/ output validation, and behavioral assessment of agentic workflows.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Develop quality metrics and evaluation methodologies for LLM-based applications, assessing accuracy, consistency, reliability, quality of AI/ML models.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Create and maintain curated evaluation datasets and synthetic test data that cover edge cases, adversarial scenarios, and real-world variability.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Leverage AI evaluation tools and frameworks &amp; establish observability and monitoring requirements to quality assessment.</span></p> </li> </ul> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Establish observability and monitoring requirements including structured logging, metrics collection, and tracing for both backend services and AI agent behavior.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Implement quality gates and acceptance criteria and create test documentation including test plans, automation architecture, quality reports, and runbooks.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Build strategies for testing in cloud environments &amp; containerized execution.</span></p> </li> </ul> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Implement performance testing, load testing, and reliability testing for production backend services and AI inference pipelines.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Define and champion quality standards, best practices, and testing methodologies for both traditional backend systems and AI applications.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Conduct code reviews with a focus on testability, quality patterns, and maintainability.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Mentor junior quality engineers, sharing expertise in automation frameworks, AI testing approaches, and quality engineering principles.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Communicate quality insights, risk assessments, and test results effectively to technical and non-technical stakeholders using tools like Jira and Confluence.</span></p> </li> </ul> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"><strong>The Experience You Bring</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">4+ years of experience in software quality engineering, test automation, or software development with strong QA focus.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Deep expertise in test automation frameworks and tools such as pytest, unittest, Selenium, Playwright etc.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Strong programming skills in Python or Java with ability to write production-quality test code and automation frameworks.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Proven track record building automated testing infrastructure for backend services and APIs.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Experience with various testing methodologies: unit testing, integration testing, end-to-end testing, contract testing, performance testing, and test-driven development (TDD).</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Strong understanding of CI/CD pipelines and experience integrating automated tests into deployment workflows.</span></p> </li> </ul> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Comfortable working in Linux/Unix environments for test execution and debugging.</span></p> </li> </ul> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Familiarity with testing AI/ML applications or LLM-based systems, with understanding of unique quality challenges in non-deterministic systems.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Knowledge of Generative AI concepts, agentic frameworks (ReAct, chain-of-thought, tool use), and LLM capabilities/limitations.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Exposure to agentic AI patterns and knowledge of how to test tool orchestration, multi-turn conversations, and RAG systems.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Excellent written and verbal communication skills with ability to document findings clearly and comprehensively.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Strong collaboration skills working across cross-functional teams in Agile/Scrum environments.</span></p> </li> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Experience with collaboration tools like Jira, Confluence, and Git for team workflows.</span></p> </li> </ul> <p><br><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"><strong>Education&nbsp;</strong></span></p> <ul> <li style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Bachelor's degree in Computer Science, Software Engineering, or related field.</span></p> </li> </ul> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;"><em>Aren’t sure if you’re a match? We know that impostor syndrome and the confidence gap can prevent us from meeting spectacular candidates — so don’t hesitate to apply; you could be the perfect fit!</em></span></p> <p><strong><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Location</span></strong></p> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Vancouver (Hybrid)</span></p> <p><strong><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Compensation and Benefits</span></strong></p> <p><span style="font-family: helvetica, arial, sans-serif; font-size: 12pt;">Boomi is committed to fair and equitable compensation practices. Base compensation for this position in our Vancouver hub ranges from $89,923 - $112,404 CAD annually + applicable bonus. Final compensation will be determined by various factors including the candidate’s knowledge, skills, experience, and geographic location.&nbsp;</span></p><div class="content-conclusion"><p>&nbsp;</p> <p><span style="font-weight: 400;"><strong>Be Bold. Be You. Be Boomi.</strong> We take pride in our culture and core values and are committed to being a place where everyone can be their true, authentic self. Our team members are our most valuable resources, and we look for and encourage diversity in backgrounds, thoughts, life experiences, knowledge, and capabilities.&nbsp;&nbsp;</span></p> <p><span style="font-weight: 400;">All employment decisions are based on business needs, job requirements, and individual qualifications.</span></p> <p><em>Boomi strives to create an inclusive and accessible environment for candidates and employees. If you need accommodation during the application or interview process, please submit a request to&nbsp;<a href="mailto:talent@boomi.com">talent@boomi.com</a>. This inbox is strictly for accommodations, please do not send resumes or general inquiries.&nbsp;</em></p></div>