Data engineer Machine learning

Job Title: Senior Databricks Data Engineer (Machine Learning Mandatory)

Location: Hyderabad/Pune/Bangalore

Work mode: Work from Office (4 Days per week)

Experience: 5+ Years

Job Summary

We are seeking a Databricks Data Engineer with strong expertise in Azure Data Factory (ADF), PySpark, SQL, and Machine Learning. The ideal candidate must have hands-on experience building data pipelines and working directly on Machine Learning projects, including data preparation, feature engineering, model deployment support, and ML lifecycle management.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
• Build and optimize ETL/ELT processes for large-scale data processing.
• Develop complex SQL queries, data models, and performance optimizations.
• Prepare, cleanse, and transform data for Machine Learning use cases.
• Implement feature engineering and ML data pipelines in Databricks.
• Work with MLflow and Databricks Machine Learning capabilities for experiment tracking and model management.
• Ensure data quality, governance, security, and reliability across data platforms.

Mandatory Skills

• Strong hands-on experience with Databricks and PySpark.
• Expertise in Azure Data Factory (ADF).
• Advanced SQL skills for data transformation and optimization.
• Hands-on Machine Learning experience is mandatory, including:
• Data preprocessing and feature engineering.
• Model training and evaluation.
• ML pipeline development.
• Model deployment and monitoring.
• Experience with MLflow and Databricks ML.
• Experience with Azure Data Lake, Delta Lake, and cloud-based data platforms.
• Proficiency with Git and CI/CD practices.

Regards,

Anil Kumar

anil.katta@valuemomentum.com