Data Engineer

Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things

Job Title:  Data Engineer

Location: Bangalore-Manyata Tech Park (Hybrid)

Business & Team: Financial Crime Technology – Enterprise Data Platform

 Impact & contribution:

You will work with Product Owner /lead engineer in the squad working towards impact of Enterprise Transformation Program across key applications.

Roles & Responsibilities:

• In your role as a Data Engineer, you will be reporting to a Data Engineering Manager
• Utilizing Big Data technologies to solve our customer data centric problems.
• Working closely with the Data engineer, Solution Designers and Business Analysts to understand and perform Data Ingestion, Enrichment and Egression
• Working across the Group Data Warehouse (GDW), Big Data Platform (Omnia) & AWS Cloud Platform
• Perform testing activities involved with End-to-End testing, System Testing and UAT testing.
• Analyse current state of testing within a specific platform and identify opportunities for initiating or increasing automation.
• To drive and optimize the Continuous Integration framework, using test automation against builds across multiple development, test, staging and production environments.
• Develop Automation feasibility reports for different applications and systems based on the technical analysis, complexity and reusability.
• Participate in the full lifecycle of agile projects (ideation through implementation).
• Building group data products by integrating diverse data from hundreds of internal and external sources
• Adopting tools, programming languages and templates to improve our data quality and efficiency
• Building and optimizing Big Data pipelines, architecture and data sets
• End to end ownership and accountability of the business requirements, identify technical / performance issues and solve them.
• Learn and adapt to new technologies to solve problems and improve existing solutions
• Required to maintain and build excellent relationships with CDAO peers and stakeholders as well as actively contribute to business outcomes and team culture

Essential Skills:

• 5+ Years of Experience in ETL/Bigdata Automation testing.
• In-depth knowledge of Testing principle, methodologies, and best practices.
• Sound understanding of Automation Framework - Using Python or JAVA.
• Minimum of 3+ years of hands-on experience in Software Engineering or development to build solution in Hadoop, Hive and Spark technologies.
• Minimum 3+ years of hands-on experience in any of the language (Scala/Python) with spark ETL development.
• Minimum 3+ years of hands-on experience in any of them (Teradata / Oracle / SQL Server / MySQL) with strong SQL expertise
• Minimum 3+ years of hands-on experience in any of the CI/CD tools like Intellij teamcity & Github cloud action.
• Minimum 3+ years of hands-on experience in any of the code maintenance repository like Github & bit bucket
• Minimum of 3+ years of experience on Shell Scripting.
• Good Understanding of AWS services.
• Good to have hands-on experience in any ETL Tool like Ab initio GDE/BRE
• Good Understanding of Distributed system and computing
• Good to have hands on experience on Performance Tuning
• Should be aware of AGILE processes and methodologies
• A natural ability to effectively communicate, educate and influence the different stakeholders/ peers, to solve challenging problems, while working in collaboration with other cross-functional engineering and solution design teams
• A positive desire to contribute towards initiatives, along with a strong ‘can- do’ attitude in juggling between constantly changing priorities
• Flexible to cross trained to organisational adopted tools & technologies
• Good to have hands-on experience in any ETL Tool Ab initio GDE/BRE
• Prior experience on Banking Domain / Financial Services related projects would be a great value add
• Hands on experience in any cloud platform like AWS/Azure/GCP is a great value add

Educational Skills:

• A degree in Engineering / Computer Science or a related discipline