AI Test Engineer / 4 to 9 years experience / multiple positions / Bangalore / Immediate to 15 days joiners

About the job

About Senzcraft:

Founded by IIM Bangalore and IEST Shibpur Alumni, Senzcraft is a hyper-automation company. Senzcraft vision is to Radically Simplify Today's Work. And Design Business Process For The Future. Using intelligent process automation technologies.

We have a suite of SaaS products and services, partnering with automation product companies.

Please visit our website - https://www.senzcraft.com for more details

Our AI Operations SaaS platform https://MagicOps.ai

Senzcraft on linkedin -> https://www.linkedin.com/company/senzcraft

Senzcraft is awarded by Analytics India Magazine in its report State of AI in India as a Niche AI startup "Agentic AI Service Provider" | Senzcraft is also recognized by NY based SSON as a top hyper-automation solutions provider.

About the Role (AI Test Engineer Data & AI Products) :

We are looking for a Senior QA / Test Engineer Data & AI to ensure the quality, reliability, and trustworthiness of data-driven, ML, and LLM-powered products. This role combines hands-on testing with quality leadership, focusing on data pipelines, AI/ML models, APIs, and GenAI/LLM systems in an enterprise and regulated environment.

You will work closely with Data Scientists, Data Engineers, Product Managers, and R&D teams to define test strategies, automate validation, and embed quality gates across the AI product lifecycle.

Key Responsibilities:

• Define test strategies for Data, Analytics, ML, and LLM-based systems
• Design and execute test cases across data ingestion, transformation, storage, and consumption layers
• Validate data quality (accuracy, completeness, consistency, timeliness) across pipelines
• Perform ML model validation including training, inference, retraining workflows, and output correctness
• Define and implement LLM evaluation strategies:
• Golden datasets
• Prompt/response validation
• Hallucination, relevance, consistency, latency checks
• Guardrails and safety testing
• Design, automate, and execute API test suites (functional, contract, performance, resilience)
• Build Python/SQL-based automation frameworks for data, API, and AI testing
• Monitor and validate model performance, drift, and data drift in production-like environments
• Integrate testing into CI/CD pipelines (Azure DevOps), enforcing quality gates and dashboards
• Create and maintain PDLM/QMS deliverables:
• Verification & Validation plans
• Test protocols and reports
• Risk-based coverage
• Traceability matrices mapped to requirements and use cases
• Log, track, and manage defects related to data pipelines, APIs, and AI outputs
• Collaborate with Product, Clinical/R&D, and Engineering teams to refine requirements and acceptance criteria
• Mentor junior testers and promote best practices in Data & AI Quality Engineering

Required Skills & Experience:

Education

• Bachelors degree in Computer Science, Data Engineering, AI, or related field

Experience

• 58+ years of experience in QA / Test Engineering
• Strong experience testing Data platforms, Analytics systems, ML models, and AI services
• Hands-on experience with API automation, performance testing, and CI/CD
• Experience working in Agile/Scrum environments
• Exposure to regulated or enterprise-grade product lifecycles is a strong plus

Technical Skills

Data & Analytics Testing

• Advanced SQL for complex data validation
• Testing ETL/ELT pipelines (Airflow, Informatica, or similar)
• Knowledge of data warehousing platforms (Snowflake, BigQuery, Databricks)
• BI/analytics validation (Power BI, Tableau)

AI / ML / LLM Testing

• Strong understanding of ML lifecycle (data prep, training, evaluation, inference)
• Validation of ML model outputs using statistical and business metrics
• Experience testing AI/ML/LLM services via REST APIs
• LLM evaluation frameworks, prompt testing, safety & guardrails

Automation & DevOps

Strong scripting skills in Python (TypeScript is a plus)

Test frameworks: PyTest, unittest, Jest

API testing: Postman/Newman, REST Assured

Performance testing: k6, JMeter

CI/CD integration using Azure DevOps

Containerization: Docker

Version control: Git

Cloud exposure: Azure (preferred), AWS, or GCP

Soft Skills & Competencies

• Strong analytical and critical-thinking skills
• Clear communication with both technical and non-technical stakeholders
• Quality-first mindset with attention to detail
• Collaborative approach when working with Data Science and Engineering teams
• Ability to adapt quickly in fast-evolving AI and GenAI environments
• Strong documentation and traceability discipline

Experience: 5-8+ years

Location: Bangalore

Work Mode: Hybrid

• Notice Period: Immediate joiners preferred, up to 15 days notice accepted