Data Engineer

Summary

A hands-on Full Stack Data Engineer responsible for designing, building, and optimizing scalable Microsoft Fabric-based data and analytics solutions. The role requires expertise across data engineering, cloud integration, AI-assisted development, and lightweight application integration, with a focus on rapid delivery, reliability, scalability, and business impact.

Full Stack Data Engineer (Microsoft Fabric) 5+ Years experience

Location- Chennai, Pune

Experience 5+ Years

Overview

We are seeking a highly motivated and hands-on Full Stack Data Engineer with strong experience in Microsoft Fabric and modern Azure-based data platforms. The ideal candidate should be capable of working across the full stack of data engineering from ingestion and transformation to Gold-layer curation, analytics enablement, API integration, and AI-assisted application workflows.

This role requires engineers who can work independently, leverage AI-assisted development for rapid delivery, and collaborate across data, cloud, and lightweight application layers. Exposure to MCP (Model Context Protocol), ReactJS integration, and modern AI-enabled engineering practices is highly preferred.

1. Fabric Ecosystem & Full Stack Data Engineering

Work with organizational OneLake structures, creating and managing shortcuts for efficient enterprise-scale data access

Design and maintain scalable Lakehouse solutions using Medallion Architecture principles (Bronze, Silver, Gold)

Build and optimize Delta Lake tables for reporting, analytics, and AI workloads

Develop and manage pipelines using Fabric Data Factory, Notebooks, and Spark workloads

Build ingestion and transformation workflows supporting structured and semi-structured data

Implement orchestration, scheduling, monitoring, and recovery mechanisms for enterprise data pipelines

Implement dimensional models (Star Schema/Snowflake Schema) to support BI, reporting, and semantic layer requirements

Build curated Gold-layer datasets for downstream analytics and AI consumption

Support integration with Power BI semantic models and reporting platforms

2. Azure, Integration & Full Stack Development

Develop batch and incremental pipelines from ADLS Gen2, APIs, Azure SQL, Blob Storage, and external systems

Support ETL/ELT orchestration using Fabric Pipelines and Azure Data Factory

Integrate Fabric-based data platforms with APIs, AI services, and enterprise applications

Support MCP (Model Context Protocol) integration and AI-enabled workflows where required

Collaborate on AI-assisted development and rapid prototyping initiatives

Work with lightweight ReactJS-based applications and frontend integrations

Support development of internal dashboards, data-driven applications, and AI-enabled user experiences

Support automation using Azure Functions, Logic Apps, and event-driven workflows

Work with Git, CI/CD pipelines, Azure DevOps, and deployment automation processes

3. Data Processing, Optimization & AI-Assisted Development

Develop and optimize PySpark notebooks for transformation, cleansing, and enrichment

Build efficient SQL queries, views, and stored procedures in Fabric Warehouse / Azure SQL

Implement optimization techniques including partitioning, caching, and query tuning

Monitor pipeline performance, troubleshoot failures, and improve system reliability

Implement logging, alerting, and operational best practices

Utilize AI-assisted development tools such as GitHub Copilot and modern AI coding assistants

Rapidly prototype and deliver scalable engineering solutions with minimal guidance

4. Governance, Security & Collaboration

Implement RBAC and secure data access across Fabric workspaces and Azure environments

Apply data quality validations and governance best practices

Support metadata management and lineage using Microsoft Purview

Collaborate with Data Architects, Analysts, BI Developers, Product Teams, and Business Stakeholders

Translate business requirements into scalable data and application solutions

Participate in Agile delivery processes, code reviews, and pull request workflows

Must-Have Skills

Microsoft Fabric (Fabric Data Factory, OneLake, Lakehouse, Spark Notebooks)

Azure Data Services: ADLS Gen2, Azure SQL, Blob Storage

Strong SQL skills (joins, aggregations, optimization)

Python / PySpark for data transformation

ETL/ELT pipeline development

Medallion Architecture (Bronze/Silver/Gold)

Exposure to REST APIs and backend integrations

Basic to intermediate ReactJS knowledge

Understanding of AI-assisted development workflows

Git, Azure DevOps, CI/CD

Good to Have

MCP (Model Context Protocol) exposure

Azure OpenAI / AI integration concepts

Azure Functions / Logic Apps

Event Hubs / streaming concepts

Microsoft Purview

Cosmos DB

Power BI semantic models

Experience

5+ years of experience in Data Engineering

Hands-on experience with Microsoft Fabric preferred

Strong Azure Data Engineering background with willingness to work across full-stack and AI-enabled engineering workflows

Key Competencies

Strong problem-solving and debugging skills

Ability to work independently with minimal guidance

Strong ownership mindset and delivery focus

Good understanding of ETL/ELT and cloud-native data platforms

Ability to collaborate across engineering, analytics, and application teams

Adaptability in fast-paced AI-enabled development environments

Certifications (Preferred)

DP-203: Azure Data Engineer Associate

DP-700: Fabric Data Engineer Associate