Big Data- Hadoop Spark-SQL - Scala - Data Engineering

Role description

Skill Big Data Spark SQL Scala

Mandotory Certification any one

Cloudera CCA Spark and Hadoop Developer CCA175

Databricks Certified Developer Apache Spark 2X

Hortonworks HDP Certified Apache Spark Developer

Job Description

Experience in Scala programming languages

Experience in Big Data technologies including Spark Scala and Kafka

Who have a good understanding of organizational strategy architecture patterns Microservices Event Driven and technology choices and coaching the team in execution in alignment to these guidelines

Who can apply organizational technology patterns effectively in projects and make recommendations on alternate options

Who have handson experience working with large volumes of data including different patterns of data ingestion processing batch realtime movement storage and access for both internal and external to BU and ability to make independent decisions within scope of project

Who have a good understanding of data structures and algorithms

Who can test debug and fix issues within established SLAs

Who can design software that is easily testable and observable

Who understand how teams goals fit a business need

Who can identify business problems at the project level and provide solutions

Who understand data access patterns streaming technology data validation data performance cost optimization

Strong SQL skills

ETL TalenD preferred Any other ETL tool

Experience with Linux OS user level

Python or R programming skills good to have but not mandatory

Skills

Mandatory Skills : Apache Spark,Scala,SparkSQL,Big Data Hadoop Ecosystem