Big Data Developer - Spark

Position: Senior Data Engineer Experience: 69 Years

Role Summary

We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality pipelines under guidance from internal leads.

Core Technical Skills (Must-Have)

1. Apache Spark (PySpark / Scala)

2. SQL (Spark SQL / Trino / HiveQL)

3. Apache Iceberg

4. Apache Airflow (Orchestration)

5. DBT (Data Build Tool)

6. Kubernetes (SKE / AKS)

7. ADO / CI-CD

8. Linux & Shell Scripting

Nice-to-Have (Secondary) Skills

1. Python

2.Grafana / Prometheus

3. Kafka