Data Engineer

Job Title: Data Engineer (48 Years)

Location: Pune | Bangalore | Chennai

Employment Type: Full-time

Work Mode: Work from Office (3 days/week at client office mandatory)

Availability: Immediate joiners only

About the Role:

We are looking for experienced Data Engineers with 4 to 8 years of experience to join our team. You will design and build scalable data pipelines on AWS using Databricks, Apache Airflow, and Apache Iceberg. This role involves working on enterprise-grade data platforms with a focus on performance, governance, and reliability.

Note: A detailed background verification (BGV) will be conducted. A valid LinkedIn profile is mandatory for application consideration.

Key Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Databricks, PySpark, and Apache Airflow.
• Implement data lakehouse architectures using Apache Iceberg for ACID transactions, time travel, and schema evolution.
• Build and maintain cloud-native data solutions on AWS using services like S3, Glue, Lambda, EventBridge, SNS, SQS, and Lake Formation.
• Implement fine-grained data access controls with AWS Lake Formation.
• Develop event-driven and serverless data workflows for scalability and cost efficiency.
• Collaborate with data scientists, analysts, and software engineers to deliver high-quality data products.
• Senior-level candidates will also lead technical initiatives and mentor junior team members.

Required Skills

Must-Have for All Candidates:

• Databricks
• Apache Airflow
• Python and PySpark
• Apache Iceberg
• AWS services: S3, Glue/Crawler, Lambda, Lake Formation

Additional Skills Based on Experience:

• For 46 Years Experience:
• Hands-on experience with EventBridge, SNS, SQS, and Kafka. Terraform knowledge is a bonus.
• For 68 Years Experience:
• Advanced proficiency in Databricks, Airflow, Python, Iceberg, and core AWS services (S3, Glue, Lambda, Lake Formation). Ability to lead technical design and mentor team members.

Preferred Qualifications

• Experience with Terraform for infrastructure as code.
• Familiarity with Kafka or other streaming platforms.