AI Workflow QA Lead & Pod Manager

Company Description

Humentrix enables AI teams to efficiently scale the human work required for tasks such as data annotation, dataset cleaning, human feedback, model evaluation, and production monitoring. By providing managed human teams, not freelancers, Humentrix ensures QA-driven execution aligned with client requirements and flexible scaling to meet evolving operational needs. Serving AI startups, labs, and enterprise teams, the company focuses on deploying reliable and secure human infrastructure that supports AI model development and optimization. Humentrix helps alleviate bottlenecks in AI workflows caused by growing data volumes, evaluation demands, and quality standards, allowing teams to focus on modeling and product innovation.

Role Description

This is a full-time on-site role in Mumbai for an AI Workflow QA Lead & Pod Manager. The role involves overseeing AI-related workflows, ensuring high-quality execution of processes like data annotation, evaluation, and quality monitoring. Responsibilities include managing a team, implementing QA guidelines, optimizing workflow efficiencies, and collaborating with stakeholders to achieve project goals. The role also requires monitoring key performance metrics and providing detailed reporting to internal and external teams.

Key Responsibilities

• Lead and manage a pod of 612 AI workflow specialists
• Ensure adherence to client-defined annotation and evaluation guidelines
• Design and maintain QA sampling processes
• Monitor accuracy metrics and performance benchmarks
• Identify recurring model failure patterns and edge cases
• Coordinate onboarding of new workflows
• Serve as escalation point for ambiguous or high-risk tasks
• Produce structured quality and performance reports

Qualifications

• Experience in Quality Assurance, monitoring workflows, and maintaining operational standards
• Proficiency in managing and motivating teams to achieve project objectives
• Understanding of AI and ML workflows, including data annotation, curation, model evaluation, and human feedback
• Strong organizational, communication, and problem-solving skills
• Capability to adhere to and enforce client-defined guidelines for maintaining workflow quality
• Prior experience in stakeholder management and professional aptitude in providing detailed reporting
• Relevant academic or professional qualifications such as a degree in operations management, quality assurance, or a related technical field
• Familiarity with project management tools and methodologies would be an advantage