Data Engineer - Consultant || BIG 4

Job Title: Data Engineer

Experience: 47 Years

Location: Bengaluru || Hybrid

Role Overview

We are looking for a skilled Data Engineer to design, build, and optimize scalable data pipelines and platforms for enterprise analytics and advanced data use cases. The role involves working with large-scale datasets, modern cloud data platforms, and distributed processing frameworks to support business intelligence, reporting, and data science initiatives.

Key Responsibilities

• Design, develop, and maintain robust, scalable data pipelines using Databricks and Apache Spark
• Build and optimize ETL/ELT workflows for structured and semi-structured data
• Work with cloud-native data services to ingest, process, and store large volumes of data
• Collaborate with data analysts, data scientists, and business stakeholders to translate requirements into technical solutions
• Ensure data quality, reliability, performance, and security across data platforms
• Optimize Spark jobs for performance, cost, and scalability
• Implement data models to support analytics and reporting use cases
• Adhere to data governance, security, and compliance standards
• Participate in code reviews, technical design discussions, and best-practice implementation

Technical Skills

• Strong experience with Databricks and Apache Spark (PySpark/Scala)
• Hands-on experience with cloud platforms:
• Azure (preferred) ADLS, Azure Data Factory, Synapse
• AWS (S3, Glue, EMR, Redshift) or
• GCP (BigQuery, Dataflow, GCS)
• Proficiency in SQL and data modeling concepts
• Experience with data warehousing and lakehouse architectures
• Familiarity with CI/CD pipelines and version control tools (Git)

Good to Have

• Experience in Big 4 or consulting-driven delivery models
• Exposure to real-time or streaming data processing
• Knowledge of data governance, lineage, and metadata management
• Understanding of performance tuning and cost optimization in cloud environments