Skill: - Databricks
Experience: 6 to 14 years
Location: - Kochi (Walk in on 22nd March)
Design, develop, and maintain scalable and efficient data pipelines using Azure Databricks platform.
Have work experience in Databricks Unity catalog
– Collaborate with data scientists and analysts to integrate machine learning models into production pipelines.
– Implement data quality checks and ensure data integrity throughout the data ingestion and transformation processes.
– Optimize cluster performance and scalability to handle large volumes of data processing.
– Troubleshoot and resolve issues related to data pipelines, clusters, and data processing jobs.
– Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
– Conduct performance tuning and optimization for Spark jobs on Azure Databricks.
– Provide technical guidance and mentorship to junior data engineers.