Lead Consultant-Databricks Platform Engineer

Responsibilities

·         Design and implement data architecture leveraging technologies such as AWS.

·         Set up Unity Catalog,- Should have specific experience setting up Unity Catalog

·         Develop, optimize, and manage data pipelines for ETL processes using azure Databricks, with a focus on data integrity and quality.

·         Design and maintain data models and schemas, incorporating Unity Catalog data governance practices.

·         Ensure data security and compliance with regulations using Databricks.

·         Establish a robust data governance strategy, defining standards, metadata management, lineage, and quality practices.

·         Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups.

·         Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions.

Qualifications
Minimum qualifications
years of experience in Data engineering or related field

Core Skills:
Data Platform Engineer with Databricks and Unity Catalog.