Quality Assurance Lead

Role Overview

We are looking for an experienced QA Lead to drive quality strategy for an AI-powered platform. This role focuses on API automation, LLM evaluation frameworks, and regulated product lifecycle validation. You will design test infrastructure, define measurable quality gates, and ensure AI outputs meet reliability, safety, and performance standards.

The position requires deep backend testing expertise combined with emerging AI validation practices in a structured, compliance-driven environment.

Key Responsibilities

• Design, automate, and execute API test suites
• (functional, contract, performance, resilience)
• Define LLM evaluation strategies including:
• golden datasets, hallucination checks, guardrails, latency KPIs
• Build automated evaluation pipelines tied to release cycles
• Create PDLM deliverables:
• verification plans, protocols, traceability matrices, risk-based coverage
• Integrate test infrastructure with Azure DevOps pipelines
• Implement quality gates and dashboard reporting
• Collaborate with Product, Clinical, and R&D teams
• Validate AI outputs across workflows and user roles
• Ensure compliance with regulated product lifecycle processes

Required Skills

• 8+ years in backend/API automation testing
• REST/JSON testing and authentication workflows
• Performance testing experience
• Strong CI/CD integration background
• Python or TypeScript automation
• Postman/Newman, REST-assured
• PyTest or Jest
• k6 or JMeter performance testing
• Docker-based test environments
• Hands-on validation of ML/LLM systems
• Experience with AI evaluation datasets and guardrails
• Strong documentation and traceability discipline (PDLM/QMS)

Good to Have

• Azure DevOps pipeline expertise
• Healthcare standards knowledge (FHIR / HL7)
• Data privacy & security awareness
• Frontend testing exposure

Behavioral Expectations

• Ownership mindset in quality leadership
• Strong cross-functional collaboration
• Structured thinking in regulated environments
• Ability to define scalable QA frameworks
• Adaptability to rapidly evolving AI technologies