Big Data Developer

TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and theres nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project.

Role- Big Data Engineer

Location- Pune

Experience- 4+ years

Please find the job description below:

About the Role:

We are looking for an experienced Big Data Engineer with proven hands-on expertise in Apache Spark. The role involves designing, developing, and optimizing large-scale data processing pipelines. Proficiency in Scala is preferred; however, experience in any JVM-based language for Spark development is acceptable.

Key Responsibilities:

1. Design, develop, and maintain data processing pipelines using Apache Spark.
2. Optimize Spark jobs for performance and scalability in distributed environments.
3. Work with large datasets to perform ETL (Extract, Transform, Load) operations.
4. Collaborate with data scientists, analysts, and engineers to deliver robust data solutions.
5. Ensure adherence to best practices for data quality, security, and governance.
6. Troubleshoot and resolve issues related to data processing and performance tuning.

Required Skills & Qualifications:

1. Proficiency in Spark Core, Spark SQL, and Spark Streaming.
2. Strong programming skills in Scala (preferred), or Python/Java.
3. Experience with distributed computing and parallel processing concepts.
4. Familiarity with the Hadoop ecosystem, Hive, or related technologies.
5. Solid understanding of data structures, algorithms, and performance optimization.