Quality Assurance Tester

Role: AI Tester

Location: Indore

Experience: 6+ Yrs.

Companys Website: http://www.yash.com/

• Skillset: Gen AI + LLM + Agentic AI
• Key Responsibilities
• Design and implement test strategies for text-based AI agents and conversational systems
• Understands LLM responses, reasoning accuracy, and hallucination risks AI Response Validation.
• Manually verify that AI agent responses are accurate, non-toxic, and free of hallucinations.
• Functional & Agentic Testing: Test the agent's ability to call external tools (MCP, APIs) and execute complex workflows correctly.
• Scenario Testing: Develop and execute manual test cases for diverse, edge-case user queries, including adversarial testing (prompt injection).
• Defect Documentation: Identify, document, and report bugs in AI reasoning, UI, and performance in tools like JIRA.
• UAT & Usability: Perform User Acceptance Testing to ensure the AI interaction is intuitive and meets end-user needs.
• Dataset Curation: Collaborate with developers to create high-quality datasets for training and testing.
• Required Skills & Qualifications:
• Testing Expertise: 4+ years of experience in functional, regression, and usability testing.
• Additional weightage will be given to candidates worked on AI Product Testing.
• AI Familiarity: Understanding of LLM behavior, chatbots, or agentic systems.
• API Testing: Hands-on experience with tools like Postman to validate AI API calls.
• Analytical Skills: Strong ability to analyze unstructured conversational data and identify patterns.
• Technical Aptitude: Basic knowledge of Python or SQL is a plus for analyzing logs.
• Must-Have Skills AI Testing & Evaluation
• Experience testing AI chatbots Prompt testing and response evaluation Hallucination detection and output validation Testing RAG pipelines and knowledge retrieval