Databricks Lead

Job Title: Databricks Lead

Location: Chennai/Hyderabad/Bangalore

Experience: 8-10 years of overall experience and 5+ years of hands-on leadership in data engineering projects

Budget: Open to discuss

Notice Period : Immediate joiner/ Serving notice with less than 60 days/Notice is less than 60 days

About the Role

We are looking for a highly skilled Databricks Lead with 810 years of overall experience and 5+ years of hands-on leadership in data engineering projects. The ideal candidate will lead teams in designing, building, and optimizing scalable data pipelines and Lakehouse solutions on Databricks.

Key Responsibilities

• Lead the development and delivery of end-to-end data pipelines using Databricks , Spark, Delta Lake, and PySpark.
• Drive implementation of Lakehouse/Medallion architectures (Bronze/Silver/Gold).
• Guide and mentor data engineers, ensure best coding and architectural practices.
• Optimize Spark jobs, Databricks clusters, and workflows for performance and cost.
• Implement CI/CD pipelines, version control, and DevOps practices for Databricks (Repos, Workflows, Terraform).
• Collaborate with architects, data scientists, and business teams to deliver scalable data solutions.
• Ensure data quality, governance, security, and compliance standards are followed (Unity Catalog preferred).

Required Skills & Experience

• 810 years in Data Engineering or Cloud Data Platforms.
• 5+ years leading data engineering teams or technical delivery.
• Strong hands-on experience with Databricks , Spark, PySpark, SQL, and Delta Lake.
• Experience with cloud platforms (AWS/Azure/GCP) and related data services.
• Solid understanding of data modeling, ingestion frameworks, orchestration, and performance tuning.
• Experience with CI/CD, Git, Terraform, Databricks Repos, and Workflows.
• Strong problem-solving, leadership, and communication skills.

Preferred Qualifications

• Databricks certifications (Associate or Professional).
• Experience with streaming technologies (Kafka, Kinesis, Event Hubs).
• Familiarity with MLflow and data science integration.

What We Offer

• Opportunity to lead cutting-edge data engineering initiatives.
• Collaborative, high-growth environment.
• Competitive compensation and professional development opportunities.