GCP Data engineer Python/PySpark

Exp- 6 to 13 years

Location- PAN INDIA

Required Technical Skill Set

Responsibilities

Perform data analytics on big data to solve and propose solutions to complex business problems
Develop and enhance end-to-end data pipelines using a big data processing technology stack (e.g. Hadoop, Cloud, Spark ecosystem technologies)
Troubleshoot complex data structures and perform root cause analysis to proactively resolve product and operational issues
Develop and document test cases for new features, execute testing, and document and share business UAT testing results as a part of agile sprints
Perform end-to-end data pipeline integration testing in adherence to change management standards and processes
Understand and adhere to coding and application standards and design (e.g. Python, SQL, etc.)
Utilize DevOps best practices to deploy new features into production systems, including continuous integration/continuous delivery (CICD), test-driven development, and proper version control (e.g. Jenkins, Jira, GIT)
Collaborate with business partners to define product objectives, business requirements, system design, and key performance indicators

Skills-

Strong in SQL, Python, PySpark
Experience with cloud platforms (GCP)
Experience working in Agile delivery environment.
Experience with orchestration tools like Airflow.
Experience with GCP Services (i.e. GCS Bucket, Cloud Functions, Dataproc, Dataflows, Pub-Sub).
Experience building automated data pipelines.
Experience performing data analysis and data exploration.
Experience working in multi-developer environment, using version control like Git.
Strong critical thinking, communication, and problem-solving skills.
Understanding DevOps best practice and CICD
Understanding of containerization (i.e. Kubernetes, Docker)

GCP Data engineer Python/PySpark