Senior Data Engineer – Data Ingestion & Pipelines (5-9 Years)

Senior Data Engineer Data Ingestion &

Pipelines

Location: India (Bangalore)

Experience: 59 years

About the Role

We are looking for a Senior Data Engineer who is deeply comfortable working with data

building ingestion pipelines, writing efficient transformations, exploring different databases

technologies, and ensuring reliable data movement across the organization.

You will primarily work on our Kafka ingestion flow, distributed data systems, and multiple

storage layers (SQL, NoSQL, and graph databases).

The ideal candidate enjoys working with raw data, optimizing queries, and building stable

pipelines using Python and Spark.

This role is perfect for someone who is data-heavy, curious about the data world.

Key Responsibilities

Data Engineering (Primary Focus)

Build and maintain robust ETL/ELT workflows using Python and Apache Spark.

Design and optimize transformations across SQL, NoSQL, and graph database ecosystems.

Implement data quality checks, schema handling, and consistency across pipelines.

Deal with complex, high-volume datasets and real-world production data.

Streaming & Ingestion

Work on Kafka ingestion pipelines topics, partitions, consumer logic, schema evolution.

Monitor ingestion performance, throughput, and reliability.

Build connectors/utilities for integrating various sources and sinks.

Querying & Multi-Database Work

Write and optimize complex queries across relational and NoSQL systems.

Exposure to graph databases and graph query languages (e.g., Cypher, Gremlin).

Understand indexing, modeling, and access patterns across different DB types.

Python & DevOps Awareness

Write clean and modular Python code for ETL jobs and data utilities.

Basic understanding of CI/CD, job orchestration, logging, and monitoring.

Ability to debug production ETL issues end-to-end.

Required Skills

Strong Python ETL, utilities, data processing.

Strong SQL query optimization and data modeling fundamentals.

Mandatory Spark experience batch transformations, performance tuning.

Kafka basics producers, consumers, offsets, schema handling.

Hands-on experience with relational and NoSQL databases.

Exposure to graph databases and graph query languages.

Strong debugging and data exploration skills.

Nice-To-Have

Retail or e-commerce domain experience.

Familiarity with Terraform or basic infra automation.

Experience with nested JSON, denormalized structures, and semi-structured data.

Understanding of distributed storage concepts.

Soft Skills

High ownership and end-to-end problem solving.

Enjoys working with complex or messy data.

Curious mindset comfortable exploring new databases and data flows.

Clear communication and ability to collaborate with backend and product teams.