Senior Manager Data Engineering

Job Description:

Description

We are seeking a highly experienced Senior Manager, Information Technology to lead our Data Engineering function, overseeing the design and implementation of scalable, highperformance data solutions. This role requires a balance of handson technical leadership, people management, and strategic delivery to drive scalable, enterprisewide data solutions.

You will play a critical role in shaping the enterprise data strategy, providing technical and people leadership to engineering teams, and driving innovation across our data platforms to support business growth and analytics at scale.

Responsibilities

Key Responsibilities

Technical Leadership & Architecture

• Architect, implement, and oversee scalable data pipelines using Apache Spark, Databricks, and Azure Data Factory.
• Design and optimize ETL/ELT workflows for both structured and unstructured data.
• Build and maintain distributed data platforms using Databricks, Delta Lake, Cassandra, and other modern data stores.
• Leverage Delta Live Tables (DLT) to deliver reliable, maintainable, and testable batch and streaming pipelines.
• Design and maintain data models for cloud data warehouses such as Databricks and/or Snowflake.
• Apply advanced performanceoptimization techniques including partitioning, indexing, and compression.
• Develop and maintain disaster recovery and backup strategies to ensure business continuity.

Platform Integration & DevOps

• Integrate Databricks with Azure Machine Learning, Azure Synapse, and other Azure services.
• Implement and govern CI/CD pipelines using Azure DevOps and Terraform.
• Collaborate with data scientists to deploy and manage ML models using MLflow.
• Ensure high standards for data quality, security, compliance, and governance across all platforms.
• Troubleshoot and resolve complex issues related to data pipelines, infrastructure, and workflows.

People Leadership & Delivery

• Lead, manage, and mentor a team of data engineers and technical leads, fostering a collaborative and highperformance culture.
• Define work assignments, provide technical guidance, and review deliverables for both direct and indirect reports.
• Partner with global Agile teams to deliver highquality, scalable solutions.
• Collaborate with HR and recruitment vendors to hire talent for new and evolving business initiatives.
• Stay current with emerging data technologies and recommend strategic adoption aligned with business goals.

Required Experience and Skills

• Bachelors degree with 12+ years of data engineering experience, including 3+ years in people management.
• Strong expertise inAzurebased data platforms, including Databricks, Spark, Kafka, and Cassandra.
• Proficient inPython, SQL, and/or Scala, with experience in largescale distributed systems.
• Solid background indata modeling, data warehousing, and cloudnative architectures.
• Experience leadingcrossfunctional teams and delivering complex, enterprise data solutions.
• Familiarity withML pipelines and MLOps; Azure or Databricks certifications preferred.