Job Description:

Must have:

• 8-11 years of hands on Experience on the Big Data space (Like Spark, hadoop, Hive etc.)
• Should be able to communicate with customer in functional and technical aspect.
• Should be able to own module and guide team for functional and technical issues.
• Good hands on exposure on Big Data technologies – pySpark (Dataframe and sparkSQL), Hadoop, and Hive
• Expert-level proficiency in python and Bash Scripts
• Expert-level proficiency in writting Glue jobs
• Good hands on exposure of Orchestration with Airflow and job scheduler experience
• Hands-on experience with Cloud Platform provided Big Data technologies (i.e. Glue, EMR, RedShift, S3, Kinesis)
• Good understanding of SQL and data warehouse tools like (Redshift)
• having good understanding on performance optimization on spark and data warehouse
• Strong analytical, problem-solving, data analysis and research skills
• Demonstrable ability to think outside of the box and not be dependent on readily available tools
• Excellent communication, presentation and interpersonal skills are a must
• Capable of providing the design and Architecture for typical business problems.

Good to have:

• Experience in migrating workload from on-premise to cloud and cloud to cloud migrations
• AWS certifications

Skills Required:

Big Data,Pyspark,Python, AWS

Role:

• Provides analytical expertise in the process of model development, refining, and implementation in a variety of analytics problems.
• Designing /architecting complex , highly available , distributed , failsafe compute systems dealing with considerable amount (GB/TB) of data
• Develop efficient ETL pipelines through pySpark or Glue
• Work closely with business and product management teams to develop and implement analytics solutions.
• Collaborate with data engineers & architects to implement and deploy scalable solutions.
• Provides strong technical leadership in adopting and contributing to open-source technologies related to Big Data across multiple engagements
• Perform code reviews to ensure our applications comply with best practices.
• Provision of different services and use those in the optimized manner with pricing understanding    
• Identify and work upon incorporating non-functional requirements into the solution (Performance , scalability , monitoring etc.)

Education/Qualification:

BE / B.Tech / MCA / M.Tech / M.Com