GenAI Platform Engineer (Data Ops Automation)

We are looking for a highly skilled GenAI Platform Engineer to build AI-driven Accelerators and Autonomous Agents that automate the complete lifecycle of our Data Platform from legacy migration and pipeline development to incident management and ML lifecycle automation.

The ideal candidate should have strong expertise in Generative AI, Agentic Frameworks, Data Engineering, dbt, Spark, and Cloud AI Platforms, with hands-on experience in building intelligent automation systems.

Key Responsibilities

Migration & Construction Accelerators

• Build AI agents to convert legacy SQL, Stored Procedures, and ETL workflows into optimized dbt models.
• Develop AI-powered assistants for automated Spark/Python code generation and dbt transformations.

Autonomous Incident & Problem Solver

• Design self-healing AI agents that monitor Spark, dbt, and cloud-native platform logs to identify failures, explain root causes, and automate fixes.
• Build AI-driven assistants to handle data quality alerts and identify upstream/downstream impacts.

Machine Learning (MML) Construction

• Develop AI agents to accelerate the ML lifecycle, including feature engineering suggestions, training pipeline creation, and deployment automation.

Required Skills

• Strong experience with LangChain, CrewAI, AutoGPT, or similar AI agent frameworks.
• Hands-on experience in building RAG (Retrieval-Augmented Generation) systems.
• Proven experience using LLMs for code generation and SQL-to-dbt/Python conversion.
• Strong expertise in dbt, including Jinja, Macros, and dbt Mesh.
• Experience with Apache Spark/PySpark, Data Pipelines, and Lakehouse architecture.
• Strong Python programming skills.
• Experience with GCP (Vertex AI) or equivalent cloud AI platforms.

Preferred Qualifications

• Bachelors or Masters degree in Computer Science, Information Technology, Data Science, Artificial Intelligence, or a related field.
• 46 years of experience in Data Engineering, AI/ML Engineering, or GenAI Platform development.
• Hands-on experience with modern Data Engineering and AI ecosystem tools such as Databricks, Airflow, Delta Lake, or MLflow.
• Prior experience building AI-powered automation solutions, autonomous agents, or intelligent developer tools is highly preferred.

Success Metrics

• Migration Efficiency: % reduction in manual effort for legacy-to-dbt migrations.
• MTTR (Mean Time to Resolution): Reduction in time spent diagnosing and fixing pipeline incidents.
• Development Velocity: Speed increase in deploying new pipelines and ML models via AI assistants