Key Skills: Azure Databricks, Apache Spark, ETL, Python, Data Engineering, Data Pipelines, CI/CD, Data Governance

Roles & Responsibilities:

Develop, maintain, and optimize ETL pipelines on Azure Databricks using Spark and Python.
Design scalable data processing workflows to support analytics and reporting for BFSI use cases.
Implement data quality checks, validation frameworks, and monitoring aligned with regulatory standards.
Integrate data from multiple sources including legacy systems, cloud platforms, and third-party APIs.
Support data migration, transformation, and reporting initiatives across financial products and services.
Identify and resolve performance bottlenecks in data pipelines to ensure efficiency and reliability.
Collaborate with data scientists, analysts, and stakeholders to deliver production-ready solutions.
Contribute to CI/CD, DevOps practices, and automation of data pipelines.
Ensure adherence to data governance, security, and compliance standards.
Work in Agile environments and participate in sprint planning, reviews, and continuous improvement.

Experience Required:

5-7 years of experience in data engineering and big data technologies.
Strong hands-on experience with Azure Databricks and Apache Spark.
Proficiency in Python for data processing and pipeline development.
Experience in designing and implementing ETL/data pipelines at scale.
Knowledge of data modeling, data warehousing, and distributed systems.
Experience working with cloud platforms, preferably Azure.
Familiarity with CI/CD pipelines and DevOps practices.
Strong problem-solving skills with the ability to troubleshoot performance issues.
Experience in BFSI domain and understanding of regulatory/data governance requirements is preferred.

Education: B.Tech, B.E., M.Sc., or Master's degree in a relevant field

Databricks Data Engineer_97913