Sr Technical Lead-Data Engg

Area(s) of responsibility

Azure Cloud Data Lead – 5B

 

Experience: 8 to 10 Years  - 5B

 

•            Develop scalable data processing solutions using MS Fabric, Azure Databricks, ensuring efficient data workflows and optimized performance.

•            Work closely with data scientists and analysts to understand data needs and integrate data from multiple sources.

•            Build and maintain ETL/ELT pipelines, leveraging PySpark/Spark and SQL to transform and process large datasets.

•            Implement data security, access controls, and governance standards, ensuring data quality and integrity throughout ETL processes

•            Cloud Lead with experience in Azure ADF, Databricks, PySpark

•            Responsible for designing and implementing secure, scalable, and highly available cloud-based solutions and estimation on AWS and Azure Cloud

•            Experince in Azure Databricks and ADF, Azure Synapse and PySpark

•            Experience with integration of different data sources with Data Warehouse and Data Lake is required

•            Experience in creating Data warehouse, data lakes for Reporting

•            Understanding of data modelling and data architecture concepts

•            To be able to clearly articulate pros and cons of various technologies and platforms

•            Define and implement cloud governance and best practices.

•            Identify and implement automation opportunities to increase operational efficiency.

 

Mandatory Skillset: MS Fabric, Databricks, PySpark, ADF, SQL