Data Engineer

Job Description Data Engineer

Role Overview

We are seeking a highly skilled Data Engineer with strong expertise in Python, PySpark, AWS services (Glue, Lambda), and Snowflake, DBT. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions that support data integration, analytics, reporting, and business intelligence initiatives across the organization.

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines using Python, PySpark, and AWS Glue.
• Build and maintain data ingestion frameworks to process structured and unstructured data from multiple sources.
• Develop and manage data transformation and integration processes to ensure data accuracy and consistency.
• Utilize Snowflake for data modeling, warehouse design, performance tuning, and query optimization.
• Automate workflows and data processing tasks using AWS Lambda and other AWS-native services.
• Ensure high standards of data quality, security, governance, and compliance across all data platforms.
• Collaborate with Data Scientists, Business Analysts, and cross-functional teams to deliver robust and scalable data solutions.
• Monitor, troubleshoot, and enhance the performance, reliability, and scalability of data pipelines and workflows.
• Implement best practices for code quality, deployment, documentation, and operational support.
• Maintain comprehensive documentation of data architecture, processes, and technical solutions.

Required Skills & Qualifications

• 49 years of experience as a Data Engineer or in a similar data-focused role.
• Strong programming expertise in Python with hands-on experience in PySpark development.
• Extensive experience with AWS services, including:
• AWS Glue
• AWS Lambda
• Amazon S3
• CloudWatch
• IAM
• Other relevant AWS data and integration services
• Strong proficiency in Snowflake, including:
• Data modeling
• Warehouse design and management
• Query optimization and performance tuning
• Solid understanding of ETL/ELT frameworks, data warehousing concepts, and large-scale data processing.
• Advanced SQL skills with experience in query optimization and performance improvement.
• Experience with Git, CI/CD pipelines, and modern deployment practices.
• Knowledge of data governance, security standards, and regulatory compliance requirements.
• Strong analytical, troubleshooting, and problem-solving abilities.
• Excellent communication and collaboration skills with the ability to work effectively in cross-functional teams.

Preferred Qualifications

• Experience working in cloud-native data engineering environments.
• Familiarity with data orchestration and workflow management tools.
• Exposure to data lake and modern data platform architectures.
• Experience supporting analytics, reporting, and business intelligence initiatives.

Key Competencies

• Strong ownership and accountability.
• Ability to work in a fast-paced, collaborative environment.