Staff Software Engineer - Cloud Platform

This is a remote based in Bangalore, India

The Cloud Platform Engineering team is responsible for the Platforms, Tools, and CI/CD pipelines at Calix.

We are seeking a skilled and experienced Staff Engineer – Cloud Platform with expertise in Google Cloud Platform (GCP) Composer and Apache Airflow to join our team. The ideal candidate will have a strong background in cloud infrastructure, automation, and CI/CD pipelines, with a focus on managing and optimizing workflows using GCP Composer and Airflow. This role will involve collaborating with development, data engineering, and operations teams to design, implement, and maintain scalable and reliable data pipelines and infrastructure.

Responsibilities: 

• Design/rearchitect our infrastructure platform components to be highly available, scalable, reliable, and secure. 
• Design, deploy, and manage GCP Composer environments for orchestrating Apache Airflow workflows.
• Optimize Airflow DAGs (Directed Acyclic Graphs) for performance, scalability, and reliability.
• Monitor and troubleshoot Airflow workflows, ensuring smooth execution and timely resolution of issues.
• Implement best practices for Airflow task scheduling, dependency management, and error handling.
• Manage and optimize GCP infrastructure, including Compute Engine, Cloud Storage, Big Query, and other relevant services.
• Implement monitoring and logging solutions using GCP Stackdriver or similar tools to track the health and performance of Airflow workflows and cloud infrastructure.
• Set up alerts and notifications for critical issues and performance bottlenecks.
• Analyze logs and metrics to identify and resolve potential issues proactively.
• Implement IaC using tools like Terraform/Terragrunt
• Integrate CI/CD pipelines with GCP services and Airflow workflows to enable seamless deployment and testing
• Collaborate extensively with cross functional teams to understand their requirements; educate them through documentation/trainings and improve the adoption of the platforms/tools.  

Qualifications: 

• 8+ years of experience in building large scale distributed system in an always available production environment. 
• 5+ years of experience in building Infrastructure Platforms and CI/CD pipelines in a major public cloud provider – GCP preferred; hands-on expertise in commonly used Cloud infrastructure/platforms services tools and services.
• Proven experience as a Platform Engineer or similar role, with a focus on GCP.
• Hands-on experience with GCP Composer or Apache Airflow, including DAG creation and management.
• Experience with GCP Dataplex, including Data zone configuration would be a plus.
• Experience with infrastructure as code (IaC) tools such as Terraform or Deployment Manager.
• Proficiency in scripting languages such as Python, Bash, or similar.
• Experience with CI/CD tools such as Jenkins, GitLab CI, or CircleCI.
• Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes.
• Knowledge of monitoring and logging tools such as Stackdriver, Prometheus, or ELK stack.
• Understanding of security best practices and compliance requirements in cloud environments.
• GCP certification would be added advantage 
• Bachelor’s degree in computer science or equivalent. 

Location:

• India – (Flexible hybrid work model - work from Bangalore office for 20 days in a quarter)