Sr. ETL Developer

Microsoft Fabric Senior Developer — ADF and Medallion Architecture

Role Summary

We are seeking a seasoned Senior Data Engineer / Microsoft Fabric Developer to design, build, and operate scalable data platforms using Microsoft Fabric, Azure Data Factory (ADF), and Medallion Architecture patterns. This role leads end-to-end data platform initiatives, translates business requirements into robust data pipelines and lakehouse solutions, and mentors engineers to deliver reliable, performant, and governed data products.

Key Responsibilities

Architect and implement end-to-end data solutions on Microsoft Fabric, applying Medallion Architecture (Bronze, Silver, Gold) to organize ingestion, cleansing, transformation, and serving layers.

Design and build scalable ETL/ELT pipelines using Azure Data Factory, Fabric Dataflows, and Fabric Notebooks; optimize for throughput, latency, and cost.

Develop data models and transformations using Spark, SQL, and Fabric OneLake; implement incremental processing, schema evolution, and data quality checks.

Implement data governance and security: access controls, encryption, lineage, cataloging, and compliance with organizational policies.

Operationalize pipelines with monitoring, alerting, automated testing, DevOps CI/CD setup and Deployment automation, and runbook documentation to ensure high availability.

Collaborate cross-functionally with product owners, data scientists, BI teams, and platform engineers to translate requirements into production-ready data products.

Mentor and review code, design patterns, and best practices for junior and mid-level engineers.

Drive performance tuning for Spark jobs, Fabric compute, and storage; manage cost optimization strategies.

Evaluate and adopt new Fabric features, Azure services, and third-party tools to improve platform capabilities.

Required Qualifications

Experience: 5+ years in data engineering or related roles, Fabric data pipelines,  Azure Data Factory or equivalent orchestration tools.

Microsoft Fabric: Hands-on experience designing and delivering solutions on Microsoft Fabric or strong experience with lakehouse architectures and willingness to adopt Fabric.

Medallion Architecture: Proven experience implementing Bronze/Silver/Gold layers, data partitioning, and incremental processing patterns.

Big Data Technologies: Strong Spark experience (PySpark or Scala), SQL, and familiarity with distributed compute concepts.

Data Storage: Experience with OneLake, ADLS Gen2, Delta Lake or similar lakehouse storage formats.

DevOps and CI/CD: Experience with Git, Azure DevOps or GitHub Actions and deployment pipelines.

Data Governance: Knowledge of data cataloging, lineage, RBAC, and compliance controls.

Soft Skills: Strong communication, stakeholder management, and mentoring ability.

Technical Skills and Tools

Core: Azure Data Factory; Microsoft Fabric components (Data Engineering, Data Factory integration, Notebooks, OneLake); Spark (PySpark/Scala); SQL.

Orchestration and Scheduling: ADF pipelines, Fabric pipelines, event-driven triggers.

CI/CD and Automation: Git, Azure DevOps, GitHub Actions

Value Added Skills and Tool: Azure Monitor , Databricks experience; Power BI , Data Quality Great Expectations , Purview

Preferred Qualifications

Batchers degree in Computer Science, Engineering, Data Science, or related field.

Value Added Certifications : Microsoft Certified: Azure Data Engineer Associate or Microsoft Fabric certifications.