Data Engineer

Role: Data Engineer

Location: Coimbatore

We are looking for a Senior Data Engineer with 5+ years of hands-on experience in building, scaling, and operating batch and event-driven data platforms across cloud environments.

Key Responsibilities

- Design and build scalable batch and real-time data pipelines

- Implement parallel and distributed data processing for large datasets

- Develop event-driven architectures for near real-time ingestion and processing

- Handle high-volume, high-velocity structured and semi-structured data

- Optimize pipeline performance, latency, and cloud cost

- Ensure data quality, fault tolerance, and observability

- Collaborate with product, analytics, and platform teams on data architecture

Core Technical Skills

- Strong SQL, data modeling, and query optimization

- Distributed processing frameworks:

- Apache Spark or Flink or Presto/Trino or Hadoop MapReduce

- Databricks or AWS EMR or GCP Dataproc

- Cloud data integration & orchestration:

- AWS Glue or AWS Data Pipeline

- Azure Data Factory (ADF) or Synapse Pipelines

- GCP Dataflow or Cloud Composer

- Data warehouses & lakehouses:

- Snowflake or BigQuery or Redshift or Azure Synapse

- Streaming & messaging systems:

- Kafka or AWS Kinesis or GCP Pub/Sub or Azure Event Hubs

- EventBridge or AWS SNS or equivalent event platforms