Hadoop Data Engineer

Dentsu

2 - 5 years

Bengaluru

Posted: 05/02/2025

Job Description

The purpose of this role is to maintain, improve, clean and manipulate data in the business’s operational and analytics databases. The Data Engineer works with the business’s software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse performance, and troubleshoot any existent issues.

Job Description:

Responsibilities:

• Design, develop, and maintain scalable data pipelines and systems for data processing.

• Utilize Hadoop and related technologies to manage large-scale data processing.

• Perform data ingestion using Kafka & spark, Sqoop and various file formats and process data into Hive using Beeline/spark.

• Develop and maintain shell scripts for automation of data processing tasks.

• Implement full and incremental data loading strategies to ensure data consistency and availability.

• Orchestrate and monitor workflows using Apache Airflow.

• Ensure code quality and version control using Git.

• Troubleshoot and resolve data-related issues in a timely manner.

• Stay up-to-date with the latest industry trends and technologies to continuously improve our data infrastructure.

Requirements:

• Proven experience as a Data Engineer (ETL, data warehousing).

• Strong knowledge of Hadoop and its ecosystem (HDFS, YARN, MapReduce, Tez and spark).

• Proficiency in Kafka & spark, Sqoop and Hive.

• Experience with shell scripting for automation.

• Expertise in full and incremental data loading techniques.

• Excellent problem-solving skills and attention to detail.

• Ability to work collaboratively in a team environment and communicate effectively with stakeholders.

Good to have:

• Understanding of PySpark and its application in real-time data processing.

• Hands-on experience with Apache Airflow for workflow orchestration.

• Proficiency with Git for version control

• Experience on Postgres SQL or SQL server or MSBI

Location:

Bengaluru

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent

About Company

Dentsu is a global advertising and digital marketing agency headquartered in Tokyo, Japan. It is part of the Dentsu Group Inc., one of the largest advertising and communications groups in the world. Dentsu specializes in providing integrated marketing solutions, including digital marketing, media planning, content creation, data analytics, public relations, and customer experience management. The company operates across numerous industries and markets, serving clients with innovative strategies to drive brand growth and engagement. With a strong focus on technology and creativity, Dentsu aims to deliver impactful, data-driven campaigns tailored to modern consumer behavior.

Services you might be interested in

One-Shot Campaign

Reach out to ideal employees in one shot!

The intelligent campaign for reaching out to the ideal audience to whom you can ask for help (guidance or referral).