Lead Consultant-Databricks Engineer

Responsibilities

·         Work closely with Software Designers to ensure best practices are being followed and offer suggestions for improvements to code proficiency and maintainability.

·         Occasionally interface with customers to assist with analyzing user needs to determine technical requirements.

·         Design, build, and maintain scalable and reliable data pipelines using DataBricks

·         Develop high-quality code with an emphasis on performance, scalability, and security.

·         Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.

·         Implement data transformations and complex algorithms within the DataBricks environment.

·         Optimize data processing and refine data architecture to improve system efficiency and data quality.

·         Mentor junior engineers and contribute to the development of best practices within the team.

·         Stay up to date with emerging trends and technologies in data engineering and cloud computing.