Data Engineer

Job Title: Data Engineer

Experience: 8+ Years

Notice period-Immediate to 15 days

Work Mode: Remote

Drop your CV - 8668072276

Experience Required

• 8+ years of overall IT experience
• Minimum 4+ years of hands-on experience in building end-to-end data engineering pipelines using Python

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for data extraction, transformation, and storage
• Integrate large-scale data from multiple sources (FTP, APIs, databases, web scraping) using Python
• Develop and maintain Python-based ETL frameworks and reusable packages
• Work with large datasets ensuring performance, scalability, and reliability
• Troubleshoot data pipelines and ensure data consistency and integrity
• Collaborate with cross-functional teams in Agile/Scrum environments

Technical Skills Required

Programming & Data Engineering

• Strong proficiency in Python (Core concepts: Data Types, Data Structures, OOP, Multiprocessing)
• Hands-on experience with SQL, Normalization, and Performance Optimization
• Experience handling large datasets using Python
• Strong understanding of design patterns and data structures
• Knowledge of PySpark and distributed data processing (Good to have)

Google Cloud Platform (GCP)

• BigQuery
• Cloud Storage (GCS)
• DataProc
• Workflows

DevOps & Version Control

• Experience with CI/CD pipelines
• Containerization (Docker/Kubernetes)
• DevOps best practices
• GitHub or other version control systems

Software Engineering Practices

• Unit testing and integration testing
• Writing clean, efficient, and scalable code
• Strong problem-solving skills and attention to detail

Soft Skills

• Excellent communication skills
• Ability to work collaboratively in team environments
• Experience working in Agile/Scrum methodology