Quality Assurance Engineer

Job Title: QA Engineer

Location: Bengaluru, Karnataka (On-Site)

Experience: 2-8 Years

Role Overview

We are looking for a QA Engineer to lead validation of AI use cases and solution approaches, ensuring that proposed solutions are robust, reliable, and aligned with problem requirements before productionization.

Key Responsibilities

• Define evaluation frameworks for AI/LLM-based use cases
• Design and execute advanced test scenarios for model behavior and edge cases
• Validate solution approaches during prototyping and experimentation phases
• Work closely with stakeholders to ensure solution correctness and feasibility
• Establish metrics for evaluating model performance and output quality
• Guide teams on best practices for AI validation and testing
• Mentor junior QA engineers in analytical and evaluation approaches

Required Skills

• Strong understanding of AI/ML systems and LLM behavior
• Experience in testing complex, non-deterministic systems
• Ability to define evaluation frameworks and validation strategies
• Strong analytical and problem structuring skills

Good to Have

• Experience with LLM evaluation tools and techniques
• Exposure to prompt engineering and model tuning
• Familiarity with Python and experimentation workflows

What Were Looking For

• Strong judgment and critical thinking
• Ability to validate is this the right solution? vs just does it