Principal Data Engineer

Job Title : Principal Data Engineer

Job location : Bengaluru

Exp : 15-25 years

Position Overview :

We are seeking a highly accomplished Principal Data Engineer with deep hands-on engineering capabilities and strong architectural expertise in the Azure data ecosystem. This role involves designing, building, and optimizing large-scale data platforms while influencing the long-term data & AI strategy. Given the visibility of the role with senior stakeholders, who can confidently represent the data engineering function in discussions with leadership teams.

Key Responsibilities

Architecture & Leadership

Architect end-to-end data platforms including ingestion, storage, processing, and consumption on Azure.

Define best practices, data architecture standards, and reusable engineering frameworks.

Provide technical leadership and mentorship to engineering teams.

Hands-on Engineering

Design and implement sophisticated data pipelines using Azure Data Factory (ADF).

Build ingestion, transformation, and orchestration workflows for both structured and unstructured datasets.

Develop scalable distributed data-processing solutions using Azure Databricks (ADB) with PySpark.

Work extensively with Azure Data Lake (ADLS) to engineer high-performance storage layers.

Own data quality, pipeline reliability, and performance across the data ecosystem.

Operational Excellence

Implement CI/CD using Git and Azure DevOps following DevOps best practices.

Drive cost, performance, security, and scalability optimization across Azure data services.

Troubleshoot production issues, conduct root cause analyses, and implement tuning improvements.

Collaborate cross-functionally with data architects, analysts, and business stakeholders.

Required Skills & Experience

15+ years of deep hands-on experience in data engineering, with at least several years in architect-level roles.

Expertise in Azure Data Factory (ADF), Azure Data Lake (ADLS), and Azure Databricks (ADB) with PySpark.

Strong understanding of data warehousing, data modeling, and ELT/ETL patterns.

Proficiency in SQL, performance tuning, and query optimization.

Experience with CI/CD, Git-based workflows, Azure DevOps.

Familiarity with Azure Synapse Analytics (preferred).

Good exposure to modern Data & AI practices, including ML-ready pipelines.

Excellent communication skills to present engineering solutions to leadership.

Good to Have

Experience in the Retail domain.

Strong stakeholder management experience in high-visibility environments.

Experience working with distributed/global engineering teams.