Principal Consultant - Databricks Developer + AWS
Genpact
5 - 10 years
Noida
Posted: 9/3/2024
Job Description
Responsibilities
• Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.
• Work with architect and lead engineers for solutions to meet functional and non-functional requirements.
• Demonstrated knowledge of relevant industry trends and standards.
• Demonstrate strong analytical and technical problem-solving skills.
• Must have experience in Data Engineering domain .
Minimum Qualifications
• Bachelor’s Degree or equivalency (CS, CE, CIS, IS, MIS, or engineering discipline) or equivalent work experience.
• Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.
• Work with architect and lead engineers for solutions to meet functional and non-functional requirements.
• Demonstrated knowledge of relevant industry trends and standards.
• Demonstrate strong analytical and technical problem-solving skills.
• Must have excellent coding skills either Python or Scala, preferably Python.
• Must have experience in Data Engineering domain .
• Must have implemented at least 2 project end-to-end in Databricks.
• Must have at least experience on databricks which consists of various components as below
o Delta lake
o dbConnect
o db API 2.0
o Databricks workflows orchestration
• Must be well versed with Databricks Lakehouse concept and its implementation in enterprise environments.
• Must have good understanding to create complex data pipeline
• Must have good knowledge of Data structure & algorithms.
• Must be strong in SQL and sprak-sql.
• Must have strong performance optimization skills to improve efficiency and reduce cost.
• Must have worked on both Batch and streaming data pipeline.
• Must have extensive knowledge of Spark and Hive data processing framework.
• Must have worked on any cloud (Azure, AWS, GCP) and most common services like ADLS/S3, ADF/Lambda, CosmosDB/DynamoDB, ASB/SQS, Cloud databases.
• Must be strong in writing unit test case and integration test
• Must have strong communication skills and have worked on the team of size 5 plus
• Must have great attitude towards learning new skills and upskilling the existing skills.
Preferred Qualifications
• Good to have Unity catalog and basic governance knowledge.
• Good to have Databricks SQL Endpoint understanding.
• Good To have CI/CD experience to build the pipeline for Databricks jobs.
• Good to have if worked on migration project to build Unified data platform.
• Good to have knowledge of DBT.
• Good to have knowledge of docker and Kubernetes.
About Company
Genpact is a global professional services firm delivering digital transformation by putting digital and data to work to create competitive advantage.
Services you might be interested in
One-Shot Campaign
Reach out to ideal employees in one shot!
The intelligent campaign for reaching out to the ideal audience to whom you can ask for help (guidance or referral).