Data Engineer

Responsibilities & Duties

• Lead the design, development, and scaling of data platforms and pipelines using Snowflake on AWS
• Build and optimize data ingestion frameworks using StreamSets and AWS S3 for structured, semi-structured, and unstructured data
• Develop and manage robust ETL/ELT pipelines leveraging AWS services (S3, Glue, Lambda, DMS) and Python-based data processing
• Design and enhance database schemas (MySQL/PostgreSQL) with a focus on performance, scalability, and maintainability
• Collaborate with development teams to optimize queries and schema design for new features prior to production deployment
• Support and implement data lake and data warehouse architectures, ensuring efficient data flow into Snowflake
• Lead database migration initiatives (e.g., MySQL to PostgreSQL) with minimal downtime and high data integrity
• Write and maintain Python (or Go) scripts for data ingestion, transformation, and automation
• Proactively identify improvements to enhance data platform scalability, resiliency, and performance
• Troubleshoot production issues, perform root cause analysis, and ensure rapid recovery with no data loss
• Implement data security and governance best practices including access control, encryption, and compliance standards
• Work closely with cross-functional teams (engineering, analytics, business) to deliver scalable data solutions
• Mentor junior engineers and contribute to best practices, coding standards, and architectural decisions
• Maintain technical documentation, data flow diagrams, and operational procedures

________________________________________

Required Qualifications & Skills

• Bachelors degree in Computer Science, Engineering, or related field
• 6 - 8 years of experience in data engineering, ETL development, or data platform engineering
• Strong hands-on experience with Snowflake (data modeling, performance tuning, virtual warehouses)
• Deep expertise in AWS ecosystem, especially AWS S3, Glue, Lambda, DMS, and IAM
• Proven experience with StreamSets for data ingestion and pipeline orchestration
• Strong proficiency in Python for data engineering, automation, and pipeline development
• Advanced experience in SQL, schema design, and performance optimization (MySQL/PostgreSQL)
• Solid understanding of data lakes, data warehouses, and ETL/ELT best practices
• Experience with Apache tools such as Airflow, Kafka, or Superset
• Familiarity with Linux systems, networking basics, and system performance tuning
• Strong problem-solving skills with experience in incident management and root cause analysis
• Excellent communication and collaboration skills

________________________________________

Preferred Qualifications

• Experience with Kubernetes and containerized data platforms
• Exposure to real-time data streaming and event-driven architectures
• Experience with CI/CD pipelines and DevOps practices
• Knowledge of data governance, security, and compliance frameworks
• Experience in regulated industries (finance, healthcare, insurance)
• Certifications in AWS or Snowflake (e.g., SnowPro)