Azure DevOps Data Engineer

Azure DevOps (Data) Engineer - 6+ Month Rolling Contract

Market Day Rate on offer

Remote - (Sunday to Thursday working days)

Location: Remote - 1 Week per quarter in Qatar

Paritas is working with a global IT Consultancy & leading Energy client who are seeking a skilled and innovative DevOps Engineer to join our team, focused on implementing Continuous Integration and Continuous Deployment (CI/CD) for Azure data services, including Azure Databricks and Azure Data Factory (ADF). This role will play a critical part in modernizing our data operations and ensuring seamless integration and automation within our Azure environment.

Responsibilities

• Design, build, and maintain CI/CD pipelines for Azure Databricks and Azure Data Factory.
• Collaborate with data engineers and software development teams to integrate code changes into the deployment pipeline.
• Automate deployment processes and workflows to enhance efficiency and reduce manual intervention.
• Monitor and troubleshoot issues in the CI/CD pipeline, ensuring minimal downtime and quick resolution.
• Implement best practices for asset bundling and version control within Azure DevOps.
• Maintain and optimize Azure DevOps environment for scalability and performance.
• Ensure security and compliance standards are met in all deployment processes.
• Document processes, procedures, and configurations for CI/CD pipelines and Azure DevOps.

Skillset

• Proven experience with Azure DevOps and implementing CI/CD pipelines.
• Proficiency in Databricks Asset Bundles (DABs).
• Strong knowledge of Azure data services, specifically Azure Databricks and Azure Data Factory.
• Experience with scripting and automation tools (e.g., PowerShell, Python).
• Experience in implementing selective deployment for ADF data pipelines.
• Understanding of containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.
• Excellent problem-solving skills and ability to troubleshoot complex CI/CD issues.
• Strong communication skills and ability to collaborate effectively with cross-functional teams.
• Knowledge of security best practices for CI/CD pipelines and Azure environments.
• Detail-oriented with a focus on delivering high-quality, reliable automation solutions.
• Bachelor's degree in Computer Science, Information Technology, or related field.
• Minimum of 5 years of experience in DevOps or related roles.