Databricks Lead Engineer

We are looking for an experienced Databricks Lead with strong expertise in building scalable Lakehouse solutions and enterprise-grade ELT pipelines for BFSI domain projects.

Location: Pune / Mumbai / Bengaluru

Experience: 610 Years

Notice Period: Immediate to 30 Days Preferred

Key Responsibilities :

Design and implement Medallion Architecture (Bronze, Silver, Gold layers) using Databricks.

Develop and optimize scalable ELT pipelines using PySpark and Spark SQL.

Work on Delta Lake, Unity Catalog, and Lakehouse architecture implementation.

Build orchestration and CI/CD workflows using Databricks Workflows and Azure Data Factory (ADF).

Ensure data governance, RBAC, data masking, and compliance standards.

Collaborate with architects, stakeholders, and mentor development teams.

Key Skills Required :

Strong hands-on experience in Databricks and Apache Spark (PySpark & Spark SQL).

Expertise in Delta Lake, Unity Catalog, and Lakehouse Architecture.

Experience with Azure Data Factory (ADF) and cloud-based data engineering solutions.

Good understanding of performance tuning and optimization techniques in Spark.

Experience with Kafka or Azure Event Hubs for streaming data pipelines.

Knowledge of CI/CD pipelines and data governance best practices.

Preferred Certifications :

Databricks Associate / Professional Certification

Microsoft Azure DP-203