Lead Associate Data Engineer

Job Title

Lead Associate - Data Engineer (1+ Years Experience)

Location

Gurgaon

About the Role

We are looking for a detail-oriented Lead Associate Data Engineer to support and maintain production automation and data processing workflows. This role involves monitoring Python-based automation pipelines, troubleshooting production issues.

This role is ideal for professionals with 1+ years of experience in automation, data engineering, or AI(good to have) who are comfortable working in production environments and collaborating with cross-functional teams.

Key Responsibilities

Automation & Production Support

Monitor and support production automation pipelines developed using Python and FastAPI

Perform L1/L2 troubleshooting for automation scripts, APIs, and data workflows

Identify and resolve script failures, dependency conflicts, API errors, and logic edge cases

Analyze logs, error traces, and performance issues in distributed systems

Data Processing & Validation

Process and validate structured and semi-structured data from multiple sources (Excel, CSV, APIs)

Perform data normalization, deduplication, and quality checks

Investigate and resolve data inconsistencies impacting downstream workflows

DevOps, Tools & Collaboration

Use Git for version control (branching, pull requests, conflict resolution)

Perform basic Docker operations such as container inspection and log analysis

Test and debug APIs using Postman and cURL

Prepare incident reports, root cause analysis (RCA), and escalation summaries

Collaborate with business, data, and engineering teams

AI / LLM (Good To Have)

Support LLM-based automation workflows including prompt optimization and token management

Assist in RAG systems by validating embeddings and debugging semantic search relevance

Review and improve LLM output quality

Skillset

Core skills (Python, PySpark, SQL, Excel, Azure/cloud platform, Power BI).

Operational independence (monitoring, troubleshooting, ownership, proactiveness).

Growth opportunities (AI, Gen AI, LLM workflows, RAG, prompt engineering).

Business impact (bridging stability with AI innovation).

Qualifications

Bachelors degree in Engineering, Computer Science, Data Science, AI or related field