Job Title: Data Engineer

Job Type: Full Time

Who can apply: Anyone

Location: Hyderabad and Chennai

Work regime: Hybrid (3 days/week onsite)

Keywords: AWS services - Glue, Redshift, RDS, CloudWatch, CloudFormation, S3, ETL, Python, Pyspark,

What you'll do:

Duties and Responsibilities:

Designing, implementing, deploying and maintaining highly robust, scalable and flexible data pipelines using AWS Services Glue, Redshift, Lambda, RDS, S3, IAM, CloudFormation etc.
Experience with development of ETL solutions on Spark framework.
Automating workflows leveraging DevOps framework (CI/CD/CT) where applicable.
Experience in understanding Dimensional data models and Data mapping documents.
Development of automated ETL solutions with Microsoft SSIS and SQL Server
Maintain, troubleshoot and modify complex SSIS packages handling high-volume of data
Ensuring quality of technical solutions as data moves across Healthfirst environments.
Working with solution and data architects to develop, construct, test and maintain architectures, prototypes and design solutions.

What you bring:

6-8+ years of Data Engineering experience in a Production cloud infrastructure AWS services - Glue, Redshift, RDS, CloudWatch, CloudFormation, S3, IAM etc.
4+ years of experience in development in Python
4+years Experience with developing ETL processes using SQL server and SSIS ensuring all data pipelines
3+years Experience with SSIS performance tuning and troubleshooting
Experience in translation of Data mapping documents into PySpark ETL code compatible to be able to run on Glue.
Hands-on work with Amazon Web Services, including creating Redshift and PostgreSQL data structures, accessing them with Spectrum and storing data in S3.
Proficiency in Python and SQL is a must.
Hands-on experience of .Net, SQL SERVER, T-SQL, and database objects (procedure, functions, views)
Hands-on of Unix/Shell scripting

Data Engineer