Experiment Engineer

Experiment Engineer

Location: Mumbai Santacruz (Kalina) / Mahape (Navi Mumbai)

Experience: 38 Years

Function: AI Scaling Platform

About the Role

Were looking for a highly analytical and hands-on Experiment Engineer to design, build, and scale experimentation platforms that power data-driven decision-making. This role focuses on A/B testing infrastructure, deployment automation, and experimentation pipelines that enable teams to validate hypotheses, optimize models, and ship winning variations with confidence.

Key Responsibilities

• Design and build scalable A/B testing frameworks, including data splitting strategies and champion vs. challenger testing models
• Develop infrastructure for running experiments and canary deployments on Kubernetes or managed cloud services (EKS, GKE, AKS)
• Containerize application variations using Docker and orchestrate them via Kubernetes for reliable testing environments
• Formulate hypotheses, conduct trials, analyze outcomes, and document iteration results
• Integrate experimentation workflows into CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions) for automated rollout of variations
• Define metrics and implement monitoring/logging systems to track experiment performance and troubleshoot issues in real time
• Collaborate with product managers, developers, data scientists, and stakeholders to define objectives and interpret results
• Enable policy or decision teams to evaluate model outputs and approve production rollouts
• Debug distributed system issues, optimize Docker images, and ensure efficient resource utilization

Required Skills & Qualifications

• 38 years of experience, with at least 2 years in A/B testing platforms, MLOps, or similar production environments
• Strong hands-on experience with at least one major cloud provider (AWS, GCP, or Azure)
• Expertise in Docker and Kubernetes
• Proven experience building and maintaining CI/CD pipelines
• Strong Python programming skills and familiarity with ML frameworks (scikit-learn, TensorFlow, PyTorch, XGBoost)
• Experience with data pipeline tools (Airflow, Prefect) and model/data versioning tools (Artifact Registry, Git LFS)
• Experience implementing observability solutions for infrastructure and ML performance
• Strong shell scripting skills

What Makes You a Great Fit

• Strong problem-solving mindset and analytical thinking
• Excellent communication and collaboration abilities
• Ability to explain complex technical concepts to diverse stakeholders
• Proactive, self-driven, and comfortable working in fast-paced environments