Senior Data Engineer

Key Skills Required

• Azure Data Services Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure DevOps
• Databricks
• SQL
• NoSQL Databases MongoDB (preferred)
• CI/CD tools Azure DevOps or GitHub

Educational Qualification

• Bachelors Degree in Computer Science, Computer Engineering, or related STEM disciplines (Science, Technology, Engineering, Mathematics).

Roles & Responsibilities

1. Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Databricks.
2. Work with structured and unstructured data stored in Azure Data Lake Storage (ADLS).
3. Develop and optimize complex SQL queries for data transformation and reporting.
4. Implement code versioning, CI/CD pipelines, and deployment processes using Azure DevOps or GitHub.
5. Collaborate with cross-functional teams in an Agile/Scrum environment.
6. Integrate and manage data from NoSQL databases such as MongoDB.
7. Ensure data quality, reliability, and performance optimization across data platforms.

Required Skills & Experience

• Strong hands-on experience with Databricks and Azure Data Factory (ADF).
• Expertise in SQL and database concepts.
• Strong understanding of Azure Data Lake Storage (ADLS) and cloud-based data architectures.
• Experience with code management and CI/CD tools such as Azure DevOps or GitHub.
• Good understanding of Agile delivery methodologies.
• Familiarity with NoSQL databases (MongoDB preferred).
• Excellent problem-solving, multitasking, and prioritization skills.
• Strong written and verbal communication skills.

Preferred / Added Advantage

• Experience working with IoT Data Standards such as:
• Project Haystack
• Brick Schema
• Real Estate Core