Senior Data Engineer

Greetings from TCS!

Job Title: Senior Data Engineer

Required Skillset: Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda)

Experience Range: 8+ years

Job Description

ROLE AND MISSION DESCRIPTION

Main tasks

• Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL
• Maintain the platform in operational condition in production (analyze and correct incidents and defects)
• Development of Python data ingestion and transformation jobs with Spark on large volumes of data
• Provide a long-term vision, both operationally and in terms of data platform strategy
• Support and promote best practices
• Participate in technical and functional design workshops
• Write and update technical documentation

Technical skills required in order of priority :

• MUST
• Python
• Spark
• Databricks
• SQL
• SHOULD
• AWS (S3, Glue, AirFlow, Cloudwatch, Lambda, IAM)
• COULD
• Big DATA
• WOULD
• GIT

Methodologies :

• CI/CD avec Gitlab
• JIRA / Confluence
• Scrum

Thanks & Regards,

Ria Aarthi A.