AI QA Test Engineer

POSITION SUMMARY

We are seeking a skilled QA Engineer with 4+ years of experience to drive end-to-end quality assurance for AI-powered applications. The ideal candidate will have strong foundations in both manual and automation testing, along with hands-on exposure to evaluating LLM-based systems. This role will own the quality lifecycle for AI-driven features, ensuring accuracy, reliability, safety, and performance across releases.

KEY RESPONSIBILITIES:

Manual Testing

Create and execute test plans, test cases, and scenarios.

Perform functional, UI, integration, regression, and exploratory testing for AI and non-AI modules. Validate end-to-end user journeys.

Log defects with clear reproduction steps and collaborate with engineering teams for triage and resolution. Automation Testing

Develop and maintain automated test suites. Build and enhance automation frameworks using Selenium, Playwright, or PyTest.

Automate test execution for AI workflows where feasible.

Integrate automated tests into CI/CD pipelines for continuous validation.

Improve automation code quality, reliability, and coverage. AI / LLM Quality Assurance Test GenAI applications powered by LLMs.

Evaluate LLM outputs for correctness, factual accuracy, and hallucinations.

Develop structured evaluation datasets and golden test cases.

Perform regression testing for prompt changes, dataset updates, and model upgrades. Quality Governance Ensure adherence to QA processes and documentation standards.

Participate in sprint planning, requirement discussions, and design reviews.

Support root-cause analysis for production defects and contribute to preventive actions.

REQUIRED SKILLS AND QUALIFICATIONS:

27 years of hands-on QA experience

(manual and automation).

Strong understanding of QA methodologies, SDLC, and defect lifecycle.

Experience testing AI/ML applications.

Hands-on experience with test automation tools

Experience with Git and Agile methodologies.

Strong analytical, communication, and problem-solving skills.

EDUCATION:

Bachelors degree in computer science, Engineering, or related field.