Staff Data Engineer

.Job Title: - Staff Data Engineer

Location: Bangalore

Business: Corporate Technology

Interview Process - Screening >> AI Interview >> L1 Discussion

Impact and Contribution:

Reduce the deployment time of models by leveraging advanced tools, automation, and modern engineering practices. This includes optimizing CI/CD processes, improving container build efficiency, and ensuring reliable, repeatable deployment patterns across environments. Contribute to the execution of strategic initiatives aimed at improving team effectiveness and engineering performance. This includes uplifting shared platform services, enhancing automation maturity, strengthening engineering guardrails, and improving deployment reliability.

Roles and Responsibilities:

• Strong experience in building and maintaining data pipelines, automating workflows, and integrating AI-driven solutions.
• Design, develop, and maintain scalable ETL pipelines using AWS Glue, PySpark, and Spark.
• Orchestrate workflows using Apache Airflow to automate data processing tasks.
• Write efficient SQL queries for data extraction, transformation, and loading.
• Work with large-scale data systems such as Hadoop or Teradata to manage and process data.
• Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and performance.
• Build and implement data pipelines in distributed data platforms including warehouses, databases, data lakes and cloud lakehouses to enable data predictions and models, and reporting and visualisation analysis via data integration tools and frameworks.
• Implement data quality checks and ensure compliance with data governance policies.
• Implement security best practices and ensure compliance with organizational standards.

Essential Skills:

• 12+ years of experience into Data Engg.
• Deep expertise in Apache Airflow
• Strong experience into AWS Glue
• Programming Languages: Python
• Excellent problem-solving and critical thinking abilities.
• Strong communication and documentation skills.
• Ability to thrive in cross-functional teams and dynamic environments

Education Qualification:

• Bachelors degree/masters degree in engineering in Computer Science/Information Technology