Data Engineer

Job Summary

We are looking for a Data Engineer with 3+ years of experience in designing, developing, and maintaining scalable data pipelines. The ideal candidate should have strong hands-on experience in Python, PySpark, and Databricks for building modern data engineering solutions.

Key Responsibilities

• Develop, maintain, and optimize ETL/ELT data pipelines.
• Build scalable data processing solutions using Python and PySpark.
• Design and implement data workflows in Databricks.
• Perform data ingestion, transformation, and validation from multiple data sources.
• Optimize Spark jobs for performance and cost efficiency.
• Work with structured and unstructured datasets.
• Collaborate with business analysts, data architects, and stakeholders to understand data requirements.
• Ensure data quality, governance, and security standards are maintained.
• Troubleshoot and resolve data pipeline issues.

Required Skills

• 3+ years of experience in Data Engineering.
• Minimum 2 years of hands-on experience with Python.
• Strong experience with PySpark.
• Mandatory experience with Databricks.
• Experience with SQL and database concepts.
• Understanding of ETL/ELT processes and data warehousing concepts.
• Experience working with cloud-based data platforms.
• Knowledge of Git and CI/CD practices.

Preferred Skills

• Experience with Azure Data Lake, AWS S3, or similar storage platforms.
• Exposure to Delta Lake.
• Knowledge of data modeling and performance tuning.
• Familiarity with Azure services.

Education

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.