Data Engineer

We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.

• Design, develop, and optimize scalable data pipelines using Databricks (PySpark/Spark SQL).
• Implement and tune data workloads using Azure Data Factory, Azure Synapse Analytics, and Microsoft Fabric.
• Apply best practices in Delta Lake, including performance tuning, partitioning, indexing, and caching.
• Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
• Implement data governance, lineage, and security using Unity Catalog and Azure capabilities.
• Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
• Collaborate with data scientists, analysts, and business teams to deliver clean and reliable data.
• Monitor platform health, optimize cluster configurations, and ensure cost-efficient scaling.

• 35 years of data engineering experience.
• Strong hands-on development skills on Databricks in production environments.
• Expertise in PySpark / Spark SQL and distributed data processing.
• Professional experience with:

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

• Strong knowledge of:
• Delta Lake ACID transactions & schema evolution
• ETL/ELT architecture and data warehousing concepts
• Performance tuning of Spark jobs and pipeline optimization
• Cloud platform experience Azure preferred.
• Proficiency with Git-based version control and DevOps practices.

• Databricks Certified Data Engineer Associate or Professional Mandatory
• Azure Data Engineering certifications (DP-203 or equivalent) Advantageous

• Strong analytical and root-cause troubleshooting abilities.
• Excellent communication, documentation, and stakeholder coordination.
• Ability to work independently and within cross-functional agile teams.

• Work with leading-edge Azure and Databricks technologies.
• Drive impactful data initiatives with high autonomy.
• Growth-oriented environment with continuous learning opportunities.