Senior Data Engineer

About the Role

We are looking for a Senior Data Engineer with 5+ years of experience to build scalable enterprise-grade data platforms and distributed data processing systems using Azure and Databricks.

The ideal candidate should have strong hands-on expertise in Databricks, PySpark, SQL optimization, and end-to-end data pipeline development. We are specifically looking for engineers who take complete ownership of development, deployment, debugging, performance tuning, and production support in large-scale data environments.

Work from Office Minimum 4 Days/Week

Immediate Joiners or Candidates with Notice Period up to 15 Days Preferred

Key Responsibilities

• Build and maintain scalable ETL/ELT pipelines using PySpark, Python, and Databricks
• Design and optimize distributed data processing workflows on Azure
• Develop enterprise-scale data models and lakehouse architectures
• Work extensively with Delta Lake, Unity Catalog, Databricks Workflows, and Medallion Architecture
• Perform Spark job optimization, SQL tuning, and storage performance improvements
• Debug production pipeline failures and resolve distributed system issues
• Collaborate with API, UI/UX, Data, and Engineering teams across complete project lifecycles
• Write modular, reusable, production-grade code following engineering best practices
• Support deployment validation, monitoring, and production support activities

Required Skills

• 5+ years of Data Engineering experience
• Minimum 4 years of hands-on experience with Databricks and Azure ecosystem
• Strong expertise in PySpark and Python
• Advanced SQL skills including joins, CTEs, window functions, and query optimization
• Deep understanding of Delta Lake, Unity Catalog, and Databricks architecture
• Experience building large-scale ETL/data pipelines
• Strong debugging and distributed systems troubleshooting skills
• Experience with performance tuning and Spark optimization
• Familiarity with Agile development and collaborative engineering workflows

Good to Have

• Exposure to AI/ML pipelines, RAG architectures, Azure OpenAI, or LangChain
• Experience with Supply Chain, ERP, or SAP data domains
• Understanding of end-to-end data flow across APIs, backend services, and frontend applications