AI Quality Assurance

Responsibilities

• Own end-to-end testing strategy for AI-driven web applications and backend services
• Design and maintain automated test frameworks for scalable AI systems
• Execute smoke, functional, regression, integration, and performance testing
• Evaluate LLM/model performance using structured evaluation frameworks
• Conduct backend and REST API testing using tools like Swagger and Postman
• Perform load and performance testing (e.g., JMeter)
• Identify, document, and track defects through resolution
• Contribute to CI/CD pipelines to strengthen automated testing workflows
• Validate deployments in Kubernetes and AWS environments

Requirements

Must-Have

• 57 years of experience in software testing with strong automation expertise
• Hands-on experience testing AI-based applications or LLM-powered systems
• Experience in model evaluation and analyzing LLM performance
• Strong backend/API testing experience
• Proficiency in Python, JavaScript, or TypeScript
• Experience with automation tools and frameworks
• Working knowledge of Kubernetes (log inspection/debugging)
• Experience using GitHub and JIRA in Agile environments

Strong Plus

• Experience with RAGAS, DeepEval, or similar LLM evaluation frameworks
• Experience with pytest or modern automation tooling
• Exposure to open-source AI/ML models
• Experience contributing to CI/CD pipelines (e.g., GitHub Actions)
• Experience testing applications deployed across multiple geographies
• Hands-on AWS experience