Senior Data Engineer / Databricks Engineer (Hybrid / Remote) || Chennai || Immediate Joiners only || 5+ years

Job title

Senior Data Engineer / Databricks Engineer

Work mode - Hybrid / Remote

Location - Chennai

Experience- 5+ years overall; 35+ years handson with *Databricks + PySpark / Spark SQL*

Key responsibilities

- Design and build *scalable ETL/ELT pipelines* on *Databricks (Azure/AWS/GCP)* using *PySpark, Spark SQL, and Scala/Python*.

- Implement *Delta Lake* patterns (bronze/silver/gold layers, schema evolution, CDC, Zorder, optimized storage) for highquality, governed data assets.

- Integrate data from multiple sources (databases, APIs, files, streaming) into cloud data lakes/warehouses and orchestrate via *Databricks Workflows, ADF, Airflow, or similar*.

- Monitor pipeline performance, tune Spark jobs, optimize cluster configs, and ensure costefficient lakehouse operations.

- Collaborate with data analysts, data scientists, and business stakeholders to translate requirements into productionready data models on Databricks.

- Follow DevOps practices (Git, CI/CD) and security/governance standards (Unity Catalog, RBAC, datalineage and monitoring).

Mandatory skills

- 5+ years in data engineering / data warehousing.

- 35+ years in *Databricks* (notebooks, clusters, jobs, workflows, Delta Lake).

- Strong handson with *PySpark / Spark SQL* and *Python / Scala*.

- Experience with at least one cloud platform: *Azure (ADLS, ADF, Synapse) or AWS (S3, Glue, EMR)* or GCP.

- Good understanding of *relational datawarehouse concepts, dimensional modeling, and SQL*.

- Familiarity with *ETL/ELT tools* (e.g., Azure Data Factory, Informatica, Airflow) is a plus.