AI QA Lead @ Onsite(Full time)

AI QA Lead

Location: Bangalore- India

FTE

AI QA Lead to provide hands-on technical leadership for AI evaluation and test execution across GenAI systems.

This is a playercoach role. You will actively design, execute, and review AI evaluation scenarios while guiding AI QA engineers and ensuring consistency, rigor, and quality of outcomes.

This is not a traditional manual QA lead role. The role requires a strong understanding of GenAI behavior, failure modes, and evaluation techniques, combined with disciplined execution and mentoring ability.

Key Responsibilities

• Lead day-to-day AI QA execution across client and internal projects
• Design, execute, and review:
• AI test cases and scenarios
• Prompt-based evaluations
• Adversarial and edge-case testing
• Regression and repeatability checks
• Guide AI QA engineers on:
• Test design quality
• Evaluation depth and coverage
• Correct interpretation of AI outputs
• Review and validate evaluation results and reports before client delivery
• Work closely with:
• AI QA Program Managers on planning and delivery
• Product and ML teams on testability and feedback loops
• Continuously improve AI QA methodologies and templates

Must-Have Skills & Experience

• 58+ years of experience in QA or testing roles
• Hands-on experience testing AI, ML, or GenAI systems
• Strong understanding of:
• Prompt engineering basics
• AI failure modes (hallucinations, inconsistency, bias, drift)
• Ability to mentor and review work of junior engineers
• Strong attention to detail and quality

Good-to-Have

• Experience with automated or semi-automated evaluation tools
• Exposure to security, safety, or adversarial testing
• Experience in client-facing QA roles
• Familiarity with scripting or data analysis

What Success Looks Like (612 Months)

• AI QA execution is consistent, repeatable, and trusted
• Junior engineers produce higher-quality evaluations
• Evaluation reports require minimal rework
• Feedback from QA meaningfully improves AI systems

Working Style & Expectations

• Hands-on and detail-oriented
• Comfortable working with evolving AI behavior
• Calm, structured, and methodical