Senior Data Engineer with Microsoft Fabric

Job Title: Senior Data Engineer with Microsoft Fabric

Location: Hyderabad / Bengaluru (Tech Mahindra Office)

Work Mode: 3 days from office

Experience: 710 years

About the Role

We are seeking a highly skilled and motivated Senior Data Engineer to architect and implement scalable ETL and data storage solutions using Microsoft Fabric and the broader Azure technology stack . This role will be pivotal in building a metadata-driven data lake that ingests data from over 100 structured and semi-structured sources, enabling rich insights through canned reports, conversational agents, and analytics dashboards.

Key Responsibilities

• Design and implement ETL pipelines using Microsoft Fabric (Dataflows, Pipelines, Lakehouse, Warehouse, SQL) and Azure Data Factory .
• Build and maintain a metadata-driven Lakehouse architecture with threaded datasets to support multiple consumption patterns.
• Develop agent-specific data lakes and an orchestration layer for an uber agent that can query across agents to answer customer questions.
• Enable interactive data consumption via Power BI , Azure OpenAI , and other analytics tools.
• Ensure data quality, lineage, and governance across all ingestion and transformation processes.
• Collaborate with product teams to understand data needs and deliver scalable solutions.
• Optimise performance and cost across storage and compute layers.

Required Qualifications

• 710 years of experience in data engineering with a focus on Microsoft Azure and Fabric technologies .
• Experience with structured and semi-structured data (JSON, XML, CSV, Parquet).
• Proven ability to build metadata-driven architectures and reusable components.
• Familiarity with agent-based architectures and conversational AI integration is a plus.
• Strong understanding of data modelling, governance, and security best practices .

Strong expertise in:

• Microsoft Fabric (Lakehouse, Dataflows Gen2, Pipelines, Notebooks)
• Azure Data Factory, Azure SQL, Azure Data Lake Storage Gen2
• Power BI and/or other visualisation tools
• Azure Functions, Logic Apps, orchestration frameworks
• SQL, Python, PySpark/Scala

Preferred Qualifications

• Experience with Azure OpenAI , Copilot Studio , or similar conversational platforms.
• Knowledge of CI/CD pipelines and DevOps practices for data engineering.
• Experience in building multi-tenant data platforms or domain-specific lakes.

Apply now and be part of building next-gen data platforms!