Data Engineering lead

Data Engineer lead

Role Overview

The Senior Data Engineer will design and implement scalable, secure, and high-performance data solutions using the Azure Data Stack . The role focuses on building reliable data pipelines, ensuring data quality, and enabling downstream analytics and AI workloads through well-structured, governed, and performant data models.

Key Responsibilities

• Design, build, and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines .
• Develop and manage data lakes and data warehouses on Azure Data Lake Storage (ADLS) and Azure Synapse Analytics .
• Integrate data from multiple sources including core banking, CRM, trading, and partner systems into unified data models.
• Build parameterized pipelines , implement data validation , and automate ingestion and transformation workflows.
• Collaborate with Cloud Architects , Data Scientists , and BI Teams to enable high-quality, reusable data assets.
• Establish data partitioning , indexing , and incremental load strategies for performance optimization.
• Enable pipeline orchestration , logging , and error handling using ADF features and Azure Monitor.
• Integrate CI/CD workflows via Azure DevOps for deployment automation and version control.
• Ensure data security, encryption, masking, and retention policies aligned with RBI / SEBI guidelines.
• Contribute to data governance, cataloging, and lineage tracking using Azure Purview or equivalent tools.

Technical Skills

• Mandatory:
• Azure Data Stack ADF , Synapse , ADLS , Azure SQL , Event Hub / Service Bus .
• Strong SQL skills and ETL design principles.
• Experience with Azure DevOps CI/CD and ARM templates / Terraform for infrastructure automation.
• Good understanding of data modelling , data quality , and incremental processing techniques.
• Good to Have:
• Experience with Python-based ETL scripting and API-driven data ingestion .
• Knowledge of Power BI datasets , Azure Monitor , and Log Analytics .
• Familiarity with DataOps and data governance frameworks .

Experience

• 812 years of experience in data engineering, with at least 45 years on Azure Cloud .
• Proven experience in data lake / data warehouse implementation using Azure-native services.
• Exposure to financial data models (Customer, Loans, Investments, Transactions) preferred.

Job Location : Bangalore (initially will have to work from our Kochi Office, as Bangalore office is yet to open)