Data Engineer

TCS is hiring for Data Engineer (Data brick) role!!!

Experience: 4 to 12 years

Location: Kolkata, Ahmedabad

Skills: Databricks, Spark and Data Migration experience

Job Summary:

Strong Databricks, Spark and Data Migration experience

Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.

Key Responsibilities:

a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design & Implement scalable data pipelines in Databricks, SQL Script conversion, etc.

b) Development & Optimization: Build Delta Lake tables and Implement ACID-complaint data models, manage metadata, schema evolution and lineage for migrated datasets

c) Collaboration & Governance: Work with Data Architect to define Target data models and migration approaches, collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team

d) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.

Interested Candidate can mail on -

Regards,

Laharika-TCS HR