Databricks Data Engineer Lead

Were Hiring: Databricks Data Engineer Lead Level

Location: Hyderabad, India

Work Model: Hybrid (3 days/week in office mandatory)

Experience: 1012+ Years

Paves Technologies is looking for a highly experienced Databricks Data Engineer (Lead Level) to drive enterprise-scale data architecture, governance, and advanced analytics initiatives on the Azure Cloud. This leadership role requires deep expertise in Azure Databricks, PySpark, and distributed data systems, along with strong architectural and mentoring capabilities.

Role Overview

The ideal candidate will have 1012+ years of data engineering experience, including strong hands-on expertise in Azure Databricks, PySpark, Python, and Azure data services. You will lead the design and implementation of scalable data platforms, drive modernization initiatives, and implement Medallion Architecture (Bronze, Silver, Gold layers) to support enterprise analytics and business intelligence.

Key Responsibilities

Lead end-to-end architecture and implementation of enterprise data platforms using Azure Databricks

Design and implement Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake best practices

Build scalable PySpark-based ETL/ELT pipelines across ingestion, transformation, and analytics layers

Develop advanced data transformations using Python, PySpark, Spark SQL, and advanced SQL

Architect robust data models (dimensional, star schema, normalized models) for analytics and reporting

Drive adoption of Databricks capabilities including Unity Catalog, Declarative Pipelines, and Delta Lake optimization

Implement performance optimization techniques such as partitioning strategies, file compaction, Z-ordering, caching, and broadcast joins

Define best practices, CI/CD frameworks, reusable templates, and infrastructure automation

Work with Azure ecosystem tools such as Azure Data Factory (ADF), ADLS, Azure DevOps, networking, and security services

Ensure data quality, RBAC, governance, lineage tracking, and production stability

Provide technical leadership and mentorship to data engineering teams

Required Skills & Experience

1012+ years of experience in Data Engineering

3+ years of hands-on experience with Databricks

Strong expertise in PySpark, Python, and distributed Spark processing

Experience implementing Medallion Architecture (Bronze/Silver/Gold)

Advanced knowledge of Data Modeling, Data Analysis, and complex SQL (CTEs, window functions, execution plan tuning)

Strong understanding of Delta Lake architecture, schema evolution, partitioning strategies, and performance optimization

Experience working with Azure Data platforms and cloud-native architectures

Strong leadership, stakeholder management, and communication skills

Mandatory Certifications

Databricks Certified Data Engineer Associate

Databricks Certified Data Engineer Professional

How to Apply

If you are interested and meet the required qualifications, please share your resume at:

contracts@pavestechnologies.com

hr@pavestechnologies.com

Rakesh.k@pavestechnologies.com

#Hiring #Databricks #DataEngineer #Azure #BigData #PySpark #DataEngineering #DeltaLake #CloudEngineering #HyderabadJobs #TechHiring #Analytics #DataArchitecture