Locations: Kolkata

Experience: 8+ Years

Role: Data Engineer/Developer

Must-Have:

Python

Spark

Databricks

AWS (S3, Glue, Airflow, CloudWatch, Lambda)

Responsibility:

Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL
Maintain the platform in operational condition in production (analyze and correct incidents and defects)
Development of Python data ingestion and transformation jobs with Spark on large volumes of data
Provide a long-term vision, both operationally and in terms of data platform strategy
Support and promote best practices
Participate in technical and functional design workshops
Write and update technical documentation

Methodologies:

Python Developer