Data Engineer

Location- Chennai

Experience- 5 to 12 years

Data Engineers / Ingestion StreamSets

We are looking for a Data Engineer (Ingestion/ELT) with strong experience in building and maintaining data ingestion and ELT pipelines, especially using StreamSets. The ideal candidate will have strong SQL skills, experience with pipeline design and orchestration, and hands-on capability in implementing data quality checks and validations across ingestion workflows.

Key Responsibilities

• Design, develop, and optimize ELT pipelines to ingest data from multiple sources into target data platforms.
• Build and maintain StreamSets pipelines, including ingestion patterns, error handling, and performance tuning.
• Work on pipeline design and migration/modernization efforts leveraging tools such as Talend / Informatica / ODI (as applicable).
• Write and optimize complex SQL for transformations, validations, reconciliation, and troubleshooting.
• Implement data quality validation checks (completeness, accuracy, consistency, duplicates, null checks, referential integrity) and ensure ingestion meets defined quality thresholds.
• Collaborate with downstream consumers (analytics, reporting, data science) and upstream source teams to ensure reliable and timely data delivery.
• Document pipeline design, operational runbooks, and validation logic for supportability and audit readiness.

Required Skills & Qualifications

• Strong experience in ELT & pipeline development.
• Hands-on expertise in StreamSets pipeline design and development.
• Exposure to at least one ETL/ELT tool such as Talend, Informatica, ODI (design/implementation/maintenance).
• Strong SQL skills (complex joins, window functions, query tuning, validation queries).
• Experience in data quality validation and reconciliation approaches within ingestion/ELT pipelines.