Senior Data Engineer

Senior Data Engineer

Position Overview

We are seeking an experienced Senior Data Engineer with 8-12 years of hands-on experience to join our team on-site. This role requires a self-starter who can architect and implement robust data foundation layers while providing technical leadership and guidance to the engineering team. The ideal candidate will have deep expertise in modern data platforms, particularly Databricks/Delta Lake, and a strong understanding of how to build scalable, reliable data infrastructure that powers enterprise analytics and AI initiatives.

Key Responsibilities

Data Foundation & Architecture

Design, build, and maintain scalable data foundation layers that serve as the backbone for analytics, reporting, and AI/ML initiatives

Architect end-to-end data solutions including storage, integration, pipelines, and modeling components

Establish best practices and standards for data architecture across the organization

Evaluate and recommend new technologies and tools to enhance the data platform

Data Storage & Management

Design and implement data storage solutions using modern cloud-based platforms

Build and optimize data warehouses, data lakes, and lakehouse architectures

Implement data partitioning, indexing, and optimization strategies for performance at scale

Ensure data availability, reliability, and disaster recovery capabilities

Data Pipelines & Integration

Develop robust, scalable ETL/ELT pipelines for batch and real-time data processing

Build data integration frameworks connecting diverse data sources (APIs, databases, files, streaming)

Implement data quality checks and monitoring throughout the pipeline

Optimize pipeline performance and troubleshoot production issues

Automate data workflows and orchestration processes

Data Modeling

Design conceptual, logical, and physical data models aligned with business requirements

Implement dimensional modeling (star/snowflake schemas) for analytics use cases

Create and maintain data dictionaries and documentation

Ensure data models support both operational and analytical workloads

Team Leadership & Collaboration

Mentor and guide junior and mid-level data engineers on technical best practices

Conduct code reviews and ensure high-quality, maintainable code standards

Lead technical design discussions and architectural decisions

Collaborate with data scientists, analysts, and business stakeholders to understand requirements

Drive continuous improvement initiatives within the data engineering team

Required Qualifications

Experience & Education

8-12 years of experience in data engineering or related roles

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience)

Proven track record of building enterprise-scale data platforms

Experience in building intelligent AI agents and RAG-based applications along with good exposure to Prompt engineering

Expertise in implementing and optimizations with Databases such as SQL Server, Postgres SQL, Clickhouse etc.,

Core Technical Skills

Must Have:

Data Foundation Layer Expertise: Deep understanding and practical implementation of:

o Data storage architectures (data lakes, warehouses, lakehouse)

o Data pipeline development and orchestration

o Data integration patterns and frameworks

o Data modeling techniques and best practices

Programming: Strong proficiency in Python and SQL

Cloud Platforms: Experience with Azure (or comparable)

Big Data Technologies: Spark, Kafka, or similar distributed processing frameworks

AI & Modern Tools

Working knowledge of AI tools and their integration with data platforms

Experience supporting ML/AI workflows and feature engineering pipelines

Soft Skills

Self-Starter: Ability to work independently, take initiative, and drive projects to completion

Leadership: Proven ability to guide and mentor team members

Communication: Excellent verbal and written communication skills to collaborate with technical and non-technical stakeholders

Problem-Solving: Strong analytical and troubleshooting abilities

Adaptability: Comfortable working in fast-paced environments with evolving requirements