Databricks Engineer

Databricks Engineer

Experience - 5+ Years

Location - Pune/Nagpur (WFO)

We are seeking an experienced Data Engineer with strong hands-on expertise in Databricks on any cloud platform (AWS, Azure, or GCP), along with proven skills in SQL, Python, and PySpark. The ideal candidate will have a solid background in database design, data processing, and building scalable data pipelines in a cloud-based environment.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and PySpark
• Work with structured and semi-structured data across various cloud platforms (AWS, Azure, or GCP)
• Write optimized and complex SQL queries for data extraction and transformation
• Implement data ingestion from various sources into the data lake/warehouse
• Collaborate with data analysts, business stakeholders, and other engineering teams to understand requirements and deliver solutions
• Monitor and improve the performance of existing data pipelines
• Ensure data quality, security, and governance across the data lifecycle

Required Skills & Qualifications:

• 57 years of professional experience in Data Engineering or related roles
• Strong experience with Databricks on any cloud platform (AWS, Azure, or GCP)
• Proficiency in SQL, Python, and PySpark
• Solid understanding of database structures, data modeling, and data warehousing concepts
• Experience with cloud data services and storage solutions
• Familiarity with CI/CD practices and version control tools like Git
• Excellent problem-solving and communication skills

Good to Have:

• Exposure to Delta Lake, Lakehouse architecture, or MLflow
• Knowledge of Apache Airflow or other workflow orchestration tools
• Experience in performance tuning and job optimization in Spark environments