Staff Data Engineer

Organization: - At CommBank, we never lose sight of the role we play in other peoples financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things.

Job Title: - Staff Data Engineer

Location: Bangalore

Business: Corporate Technology

Impact and Contribution:

Reduce the deployment time of models by leveraging advanced tools, automation, and modern engineering practices. This includes optimising CI/CD processes, improving container build efficiency, and ensuring reliable, repeatable deployment patterns across environments. contributed to the execution of strategic initiatives aimed at improving team effectiveness and engineering performance. This includes uplifting shared platform services, enhancing automation maturity, strengthening engineering guardrails, and improving deployment reliability.

Roles and Responsibilities:

• Strong experience in building and maintaining data pipelines, automating workflows, and integrating AI-driven solutions.
• Design, develop, and maintain scalable ETL pipelines using AWS Glue, PySpark, and Spark.
• Orchestrate workflows using Apache Airflow to automate data processing tasks.
• Write efficient SQL queries for data extraction, transformation, and loading.
• Work with large-scale data systems such as Hadoop or Teradata to manage and process data.
• Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and performance.
• Build and implement data pipelines in distributed data platforms including warehouses, databases, data lakes and cloud lakehouses to enable data predictions and models, and reporting and visualisation analysis via data integration tools and frameworks.
• Implement data quality checks and ensure compliance with data governance policies.
• Implement security best practices and ensure compliance with organizational standards.

Essential Skills:

• 12+ years of experience into Data Engineering.
• Deep expertise in Apache Airflow
• Strong experience into AWS Glue, Lambda, Redshift, S3 etc
• Proven experience into PySpark, Hadoop or Teradata.
• Programming Languages: Python
• Excellent problem-solving and critical thinking abilities.
• Strong communication and documentation skills.
• Ability to thrive in cross-functional teams and dynamic environments

Education Qualification:

• Bachelors degree/masters degree in engineering in Computer Science/Information Technology