Lead Consultant- Databricks Developer with experience in Unity Catalog + Python , Spark , Kafka for ETL!
Genpact
5 - 10 years
Hyderabad
Posted: 06/06/2025
Job Description
Responsibilities
Develop and maintain scalable ETL pipelines using Databricks with a focus on Unity Catalog for data asset management.
-
Implement data processing frameworks using Apache Spark for large-scale data transformation and aggregation.
-
Integrate real-time data streams using Apache Kafka and Databricks to enable near real-time data processing.
-
Develop data workflows and orchestrate data pipelines using Databricks Workflows or other orchestration tools.
-
Design and enforce data governance policies, access controls, and security protocols within Unity Catalog.
-
Monitor data pipeline performance, troubleshoot issues, and implement optimizations for scalability and efficiency.
-
Write efficient Python scripts for data extraction, transformation, and loading.
-
Collaborate with data scientists and analysts to deliver data solutions that meet business requirements.
-
Maintain data documentation, including data dictionaries, data lineage, and data governance frameworks.
Qualifications we seek in you!
Minimum qualifications
Bachelors degree in Computer Science, Data Engineering, or a related field.
experience in data engineering with a focus on Databricks development.
About Company
Genpact is a global professional services firm that offers a wide range of digital transformation services and solutions. With a presence in over 30 countries, Genpact leverages its deep domain expertise in operations and analytics to help businesses transform their operations, improve efficiency, and enhance customer experience. The company combines digital technology, data science, and operational excellence to deliver business outcomes across various industries, including banking, insurance, manufacturing, and healthcare. Founded in 1997 as a subsidiary of GE, Genpact has grown into an independent, NYSE-listed company with a diverse workforce of over 90,000 employees globally.
Services you might be interested in
One-Shot Campaign
Reach out to ideal employees in one shot!
The intelligent campaign for reaching out to the ideal audience to whom you can ask for help (guidance or referral).