Senior Data Engineer (Unity Catalog)

Job Description:

KPI Partners is seeking a skilled Senior Data Engineer with expertise in Databricks, Unity Catalog, PySpark, Python, and SQL. As a member of our dynamic team, you will be responsible for designing, developing, and maintaining scalable data pipelines and solutions that support our data-driven decision-making processes.

Key Responsibilities:

• Develop and maintain data pipelines using Databricks and PySpark to ensure efficient data processing and transformation.
• Utilize Unity Catalog to manage and govern data assets and ensure data quality and security.
• Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver actionable insights.
• Write efficient and optimized queries using SQL to extract and manipulate data from various sources.
• Monitor and troubleshoot data pipeline performance, implement improvements, and provide support for data-related issues.
• Document data engineering processes, architecture, and workflows to ensure knowledge sharing within the team.

Qualifications:

• Bachelors degree in computer science, Data Science, or a related field.
• Proven experience as a Data Engineer or in a similar role with a strong understanding of data engineering principles.
• Proficiency in Databricks, Unity Catalog, PySpark, Python, and SQL.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Strong problem-solving skills and the ability to work collaboratively in a team environment.
• Excellent communication skills to convey technical concepts to non-technical stakeholders.

What We Offer:

• Competitive salary and benefits package.
• Opportunities for professional development and career growth.
• A collaborative and innovative work environment.
• The chance to work with cutting-edge technologies in the data engineering field.

If you are passionate about data engineering and eager to contribute to a fast-paced organization, we encourage you to apply for the Data Engineer position at KPI Partners.