Technical Lead

Technical Lead

Location: Pune

Experience: 8+yrs

Work Model: Work from Office

We are looking for a highly skilled Technical Lead (individual contributor) who has experience supporting and guiding a team of 78 engineers by solving complex technical challenges in a production support environment. The ideal candidate must be proactive in identifying opportunities for system improvements, performance enhancements, and automation across our data ecosystem.

Key Responsibilities

• Act as the primary technical expert, supporting a team of 78 members by resolving complex Big Data and data engineering issues.
• Work in a 24/7 or high-availability production support environment, ensuring stability, scalability, and performance of Big Data platforms.
• Troubleshoot performance bottlenecks across Spark, Hadoop, EMR, SQL queries, and data pipelines.
• Identify, propose, and implement improvements in data pipelines, architecture, performance tuning, and cost optimization.
• Collaborate with cross-functional stakeholders (Data Engineering, DevOps, Architecture, etc.) to deliver scalable data solutions.
• Own incident management, root cause analysis (RCA), preventive actions, and optimization initiatives.
• Develop and review SQL queries, ETL logic, and data transformation code.
• Ensure adherence to best practices in data quality, reliability, and security.
• Monitor and optimize workloads running on AWS EMR, Redshift, and other Big Data components.
• Drive automation to reduce manual intervention and improve operational efficiency.

Required Skills & Experience

• 710 years of total experience in Big Data technologies, with strong hands-on technical leadership.
• Prior experience supporting or guiding a 78 member team on technical problem-solving (IC role, not people management).
• Strong experience with AWS EMR, Redshift, Hadoop ecosystem, and distributed computing frameworks.
• Expertise in Spark SQL, Hive, Presto, ETL workflows, and performance tuning.
• Must have excellent SQL development skills with the ability to write complex and optimized queries.
• Strong knowledge of job orchestration, scheduling, monitoring, and log analysis.
• Experience working in a production support or DevOps-oriented environment.
• Good understanding of data modeling, partitioning, and optimization strategies.
• Ability to diagnose issues across large-scale data pipelines and distributed systems.

Good to Have

• Experience with Python, Airflow, or similar orchestration tools.
• Knowledge of AWS cost optimization for Big Data workloads.
• Familiarity with CI/CD pipelines and automation tools.
• Exposure to data migration or modernization projects.