Data Engineer (Azure ML ,Databricks, Data lakes, Python)

We are seeking highly skilled Data Engineers with 4+ years of hands-on experience in Databricks and strong expertise in data engineering, ML pipelines, and cloud-based data platforms. The ideal candidate will design, build, and maintain large-scale data processing systems, enabling high-quality data flow and analytics for business insights.

• Design, develop, and optimize ML/data transformation pipelines using Python,Pandas , PySpark, and Apache Spark.
• Build and manage scalable data architectures for handling large and complex datasets.
• Work extensively on Azure Databricks and Delta Lake, ensuring efficient data storage and processing.
• Collaborate with cross-functional teams to deliver data solutions across multiple domains.
• Implement best practices in data modeling, data warehousing, and data governance for both on-prem and cloud environments.
• Continuously evaluate emerging technologies and tools to enhance data engineering capabilities.
• Troubleshoot, optimize, and ensure performance of data pipelines and analytical systems.
• Take ownership of technical deliverables, contributing to project leadership and independent execution.

Mandatory:

• Strong hands-on experience with Databricks and Delta Lake.
• Proficiency in Python and Pandas (compulsory).
• Strong experience in PySpark and Apache Spark.
• Experience in ML pipelines, data integration, and data pipeline optimization.
• Solid understanding of data modeling, data warehousing, and distributed systems.
• Ability to manage and transform large-scale, multi-dimensional datasets.
• Strong problem-solving, analytical, and process optimization skills.
• Up-to-date with modern data engineering tools and cloud data ecosystems.