Data Engineer

Key Responsibilities:

• Data Pipeline Development: Design, build, and optimize scalable ETL/ELT pipelines to process and transform data from various sources into Snowflake.
• Data Modeling: Build and maintain data models within Snowflake for optimized querying and reporting.
• Cloud Infrastructure: Leverage AWS services (e.g., S3, Redshift, Lambda, Glue) for data storage, processing, and orchestration.
• Automation & Infrastructure as Code: Use Terraform to automate and manage cloud infrastructure deployments and ensure scalability, reliability, and efficiency.
• Reporting & Visualization: Collaborate with BI teams to integrate data with Tableau for reporting, dashboards, and analytics.
• Data Quality & Governance: Implement best practices for data quality, governance, and security in line with company policies.
• Performance Optimization: Continuously monitor and improve the performance of data systems and pipelines, ensuring low-latency and high-availability.
• Collaboration: Work closely with cross-functional teams (data scientists, analysts, product managers) to deliver actionable insights and products.
• Troubleshooting & Support: Provide ongoing support to ensure that data systems and pipelines are running smoothly and addressing issues as they arise.

Skills & Qualifications:

• Experience with Snowflake: Proficiency in Snowflake for data warehousing, including data loading, transformation, and optimization.
• AWS Expertise: Hands-on experience with AWS tools such as S3, Redshift, Lambda, Glue, and others for data processing and storage.
• Data Pipeline Development: Experience building and maintaining end-to-end data pipelines using tools like Apache Airflow, DBT, or similar.
• Tableau: Solid experience in integrating and visualizing data in Tableau for reporting and dashboard creation.
• Terraform: Experience in Infrastructure as Code (IaC) using Terraform to manage cloud resources.
• SQL Proficiency: Strong SQL skills for data querying, transformation, and troubleshooting.
• Programming: Familiarity with Python or other programming languages for building custom data pipelines and automation.
• Data Governance & Security: Understanding of data governance principles, security best practices, and compliance requirements.
• Communication: Strong communication skills to collaborate with technical and non-technical teams.

Nice to Have:

• Experience with containerization (Docker, Kubernetes).
• Knowledge of machine learning models and integration into data pipelines.
• Agile or Scrum methodology experience.
• Familiarity with CI/CD processes for data engineering workflows.