Lead Data Engineer

Were Hiring: Lead Data Engineers ( Immediate to 30 days)

Location: Whitefield, Bangalore

Platform Specialization: Databricks

Project Scope: Complex DWH & Migration initiatives across the full data engineering lifecycle

Are you passionate about building scalable data pipelines, optimizing performance, and solving real-world data challenges? Were looking for hands-on Data Engineers who thrive in fast-paced environments and bring deep expertise in SQL, Python, Spark, Databricks, and cloud-native data platforms.

Lead Data Engineers (8+ years):

• Build and optimize robust data pipelines for ingestion, transformation, and analytics
• Work across structured and semi-structured data sources, ensuring performance and reliability
• Collaborate with cross-functional teams to deliver scalable solutions on Databricks
• Strong coding skills in SQL, Python, and Spark are a must
• Experience in data validations, reconciliation, and quality checks across ingestion and transformation stages

Must-Have Technical Skills:

• Advanced SQL (window functions, CTEs, performance tuning)
• Python (data manipulation, modular design, error handling)
• Advanced Apache Spark Programming
• Databricks (Notebooks, Delta Lake, Job Orchestration, Unity Catalog, Delta Live Tables, DB Asset Bundles, Lakehouse Architecture, Workspace Management, Cluster Configuration, MLflow Integration, and Role-Based Access Control)
• CI/CD pipelines, version control (Git), and testing frameworks
• Experience with cloud platforms (Azure, AWS, or GCP)

Software Engineering Fundamentals:

• Clean code practices and modular design
• Agile delivery and sprint planning
• Debugging, logging, and exception handling
• Code reviews and mentoring junior engineers
• Understanding of system design and scalability

Bonus Skills:

• Exposure to data modeling (dimensional, star/snowflake schemas)
• Familiarity with data governance, lineage, and cataloging tools
• Prior experience in large-scale data migration projects and validation frameworks