Pyspark Developer

Must-Have**

(Ideally should not be more than 3-5)

1. Minimum 6-8  years of experience in build & deployment of Bigdata applications using SparkSQL, SparkStreaming in Python;

2.  Minimum 5 years of extensive experience in design, build and deployment of Python-based applications;

3.  Design and develop ETL integration patterns using Python on Spark. Develop framework for converting existing PowerCenter mappings and to PySpark(Python and Spark) Jobs.

4.  Create Pyspark frame to bring data from multiple databases to Amazon S3. Expertise on graph algorithms and advanced recursion techniques. Hands-on experience in generating/parsing XML, JSON documents, and REST API request/responses.

Good-to-Have

1.      Hands-on experience writing complex SQL queries, exporting and importing large amounts of data using utilities.