Senior Data Engineer

Role Overview :

We are seeking a Senior Data Engineer with strong developer DNA someone who codes daily in PySpark, SQL, and Python, builds end-to-end data solutions, and leads design of robust, production-grade data platforms on Microsoft Fabric, Azure Databricks, and ADF. This is a hands-on leadership role for engineers who can think architecturally and code efficiently.

Key Responsibilities:

Design, develop, and optimize complex data pipelines and lakehouse architectures.

Lead development of batch and streaming ingestion frameworks using ADF, Databricks, and Event Hubs.

Write optimized PySpark/SQL transformations for performance, scalability, and cost efficiency.

Build reusable data frameworks and automation scripts for continuous integration and deployment.

Establish unit testing, code review, and versioning standards across engineering teams.

Integrate with DevOps CI/CD pipelines for automated releases and environment management.

Mentor junior engineers on coding practices and architecture design.

Collaborate with data architects on Fabric-based Lakehouse implementations.

Skills & Qualifications:

612 years of strong hands-on experience in data engineering and development.

Expertise in:

o PySpark, SQL, Python, and performance tuning of large-scale transformations.

o Databricks, ADF, ADLS, Fabric Data Pipelines, and Delta Lake.

Deep understanding of data architecture, schema design, job orchestration, and data governance. Strong knowledge of CI/CD (Azure DevOps) and infrastructure-as-code for data pipelines.

Optional: Exposure to GCP BigQuery, Dataproc, or Dataflow as secondary platforms.

Preferred Certifications: Databricks Certified Data Engineer Professional Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) Microsoft Certified: Azure Solutions Architect Expert (AZ-305) Optional: Google Cloud Certified Professional Data Engineer