Data Engineer

Immediate Joiner

Hybrid work mode

Hyderabad

6+ Years

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows using AWS Glue.
• Develop serverless data processing solutions using AWS Lambda.
• Design and manage data lakes using Amazon S3.
• Build and optimize data warehouse solutions using Amazon Redshift.
• Write complex and optimized queries using SQL.
• Implement and manage table formats using Apache Iceberg.
• Ensure data quality, integrity, governance, and performance tuning.
• Collaborate with data analysts, data scientists, and cross-functional teams.
• Implement CI/CD pipelines and automation for data workflows.
• Monitor, troubleshoot, and optimize data jobs and pipelines.

Required Skills

• Strong hands-on experience with AWS (Glue, Redshift, Lambda, S3).
• Proficiency in Python and PySpark.
• Strong command of SQL (query optimization, performance tuning).
• Experience implementing Apache Iceberg in data lake environments.
• Experience building scalable ETL/ELT pipelines.
• Knowledge of data modeling and warehouse design.
• Understanding of distributed data processing concepts.

Preferred Skills

• Experience with data lake architecture and lakehouse concepts.
• Knowledge of partitioning strategies and performance tuning in Iceberg.
• Familiarity with DevOps practices and CI/CD pipelines.
• Experience with data governance and security best practices in AWS.