Senior Big Data Developer

The Applications Development Senior Programmer Analyst (data engineering senior programmer analyst) is an intermediate level position responsible for participation in the establishment and implementation of new or revised data platform eco systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to data engineering scrum team to implement the business requirements:

Requirement:

• Looking for well- rounded Big Data Professional with deep technical expertise, strong troubleshooting capabilities and ability to work in high-pressure situations to ensure the production environments runs smoothly.
• Flexible in extending support during critical issue
• Positive attitude

Technical skills:

• Hadoop, Spark, Hive, YARN, HBase or Cassandra, Presto/Trino, Flink, Kafka
• Python, Scala, SQL, Oracle, UNIX

Production Support ,Troubleshooting and Root Cause Analysis:

• Strong understanding of log management
• Expertise in identifying and troubleshooting production issues related to data pipelines, system bottleneck or job failure
• Ability to quickly diagnose problems in distributed system and apply fixes under pressure.

Performance Tuning:

• Optimize Big Data workflow( eg Spark job tuning, Query optimization in Hive )
• Knowledge of partitioning strategies and indexing techniques in distributed storage system

Operational Skills:

• Proficiency in Bitbucket
• Experience with tools like Jenkins
• Knowledge of how to handle batch jobs, real time streaming pipelines and their production deployment ( Autosys )

Collaboration & Communication:

• Experience with incident management system like JIRA, ServiceNow, SAAS.
• Ability to work closely with data engineers, DBA, System admins to resolve issues
• Clear documentation skills to maintain runbooks, troubleshooting guides and best practice.

Soft Skills

• Ability to handle shifting priorities and manage multiple urgent issues at same time.
• Ensuring all aspects of data pipeline or systems are running smoothly even during high stress incidents
• Ability to explain complex technical issues to non-technical stakeholders

Preferred Experience:

• Experience with Data Governance and understanding of security protocols( SSL, encryption) in big data environments.
• Previously involvement in post-mortem analysis and continuous improvement of production process.
• Familiarity with Agile methodologies and working scrum team