Senior Data Engineer

Organization:

At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things.

Job Title: Senior Data Engineer

Location: Bangalore

Business & Team: Ops Tech – Financial Crime Technology

Impact & contribution

Here, you’ll get to turn your passion into reality, working with new technology on state-of-the-art projects to enable a seamless and unrivalled customer experience. You’ll build high quality, reliable solutions that determine how millions of people access their money and define future industry trends and maintain compliance with regulatory and community expectations.

We’re constantly striving to be the best in financial services, pushing the boundaries to provide a seamless user experience across every product. Leveraging cloud technologies, we provide highly standardised platforms that push us toward rapid deployment and greater resilience.

Roles & Responsibilities:

As part of Financial Crimes Technology (FCT) we’re focussed on delivering efficient and timely business processing for regulatory obligations, an essential component of great and compliant customer service.

We’re anchored by open communication, quality work and a culture of excellence, and we offer a collaborative and supportive space to share knowledge.

You will be joining a Data Delivery Team, responsible for designing, building and maintaining scalable data pipelines, optimizing data processing, and ensuring efficient data storage and retrieval. Here, you’ll experience flexibility, great rewards and the opportunity to drive your own career growth. 

We’re interested in hearing from people who

• Design, develop and optimize ETL/ELT pipelines for processing large-scale structed and unstructured data.
• Implement data ingestion, transformation, and storage solutions using Hadoop, Spark and AWS Services(S3,Glue,EMR,Redshift,Athen,Lambda etc)
• Good with Data Analysis, Data warehousing, SQL Queries.
• Knowledge of performance tuning strategies, data quality, defect/issues analysis, and providing optimal resolutions
• Possesses hands-on technical experience working in a public cloud environment ideally using AWS.
• Are experienced in developing and optimising data integration workflows, ensuring efficient data movement and transformation between various systems and data sources.
• Can implement data quality checks, data validation, and error handling mechanisms to maintain data integrity and accuracy.
• Have familiarity with GITHUB, confluence, and JIRA.
• Are capable to attend business meetings with stakeholders, understand the requirements and convert them into technical design

Essential Skills

We use a broad range of tools, languages, and frameworks. We don’t expect you to know them all but experience or exposure with some of these (or equivalents) will set you up for success in this team;

• 8+ years of relevant experience
• Should strong in SQL queries, performance tuning, Batch Job creation.
• Experience with Apache Spark (Pyspark/Scala) for distributed data processing.
• Experience with Hadoop Ecosystem (HDFS, Hive, Sqoop etc)
• AWS Cloud Data Implementation. Implement data solutions leveraging AWS cloud services such as Amazon S3, AWS Lambda, Redshift, Amazon EMR, StepFunctions , Cloudwatch Events and others.
• Knowledge on workflow Orchestration Tools (Apache Airflow,AWS Step Function, Autosys Workflow Automation)
• Familiarity with CICD pipelines. GitHub, Confluence, Jira.
• Sound understanding of cloud native (AWS/Azure) application principles and the whys behind them.
• Great verbal and written skills to communicate your designs and strategy to peers and management.

Education Qualification:

• Bachelor’s degree/master’s degree in engineering in Computer Science/Information Technology