Certified Data Engineer (Databricks)

Job Title: Data Engineer

Location: Coimbatore

Experience: 35 years

Job Summary

We are seeking a Data Engineer with strong experience in building scalable data pipelines and modern lakehouse solutions using Databricks. The role involves designing ETL/ELT workflows, processing large datasets using Apache Spark, and working across Amazon Web Services and Microsoft Azure environments.

Key Responsibilities

Develop and maintain scalable ETL/ELT pipelines

Build Lakehouse architecture using Databricks and Delta Lake

Implement batch and streaming data processing

Optimize performance of queries and data models

Automate workflows using CI/CD and orchestration tools

Support BI and analytics teams with reliable datasets

Required Skills

Strong experience in Python and SQL for data engineering tasks

Hands-on experience with PySpark and Apache Spark for large-scale data processing

Experience developing ETL/ELT pipelines in Databricks

Good knowledge of Delta Lake, Snowflake, Redshift, or Azure Synapse

Experience working with AWS services such as S3, Glue, Lambda, and Redshift

Experience working with Azure Data Factory and Azure Data Lake Storage

Ability to build both batch and streaming pipelines

Experience with Apache Airflow or similar workflow orchestration tools

Familiarity with Git and CI/CD for deployment automation

Understanding of data modeling concepts like Star and Snowflake schema

Preferred Qualifications

Databricks Certified Data Engineer (Professional)

Experience with dbt and Spark Structured Streaming

Exposure to analytics, reporting, and AI-driven data solutions