AWS Databricks Data Engineer_97831

Company:IT Services Organization

Key Skills: PySpark, AWS, Databricks, Python, SQL, AWS Cloud Engineering, MySQL

Roles & Responsibilities:

• Develop and maintain end-to-end data pipelines using PySpark and Python on AWS Databricks.
• Write and optimize SQL queries for data extraction, transformation, and validation to support analytics and reporting.
• Build scalable data processing workflows in Databricks to ingest, transform, and curate datasets for downstream use.
• Design and implement robust data engineering solutions on AWS, ensuring scalability, security, and reliability.
• Apply cloud engineering best practices for deployment, monitoring, and maintenance of data platforms.
• Ensure high data quality through validation, error handling, and consistent data processing standards.
• Collaborate with stakeholders to gather requirements and translate them into efficient data pipeline designs.
• Support integration with MySQL or other databases for data ingestion and storage (if applicable).
• Contribute to continuous improvement initiatives and team delivery processes.

Experience Required:

• 4 - 8 years of experience in data engineering or cloud data platforms.
• Strong hands-on experience with PySpark, Python, and SQL for data processing.
• Experience working with Databricks and building scalable data pipelines.
• Solid experience with AWS services and cloud-based data architectures.
• Understanding of data modeling, ETL/ELT processes, and big data concepts.
• Experience with data quality, validation, and performance optimization techniques.
• Familiarity with CI/CD, version control, and deployment practices in data engineering.
• Ability to work with large datasets and optimize processing performance.
• Exposure to MySQL or other relational databases is an added advantage.

Education: Any Graduation