GCP Data Engineers with Dataplex | 4 to 8 years | Bangalore | 18 to 20 LPA | Big4

Job Title: GCP Data Engineer with Dataplex

Experience: 4 to 8 Years

Budget: 18 to 20 LPA

Location: Bangalore

Type: Full-Time

Responsibilities:

Job Summary: As a skilled GCP Data Engineer with hands-on experience in Google Cloud Platform (GCP) and Dataplex, you will be responsible for building, maintaining, and optimizing data pipelines and data lake infrastructure. You will work closely with data architects, governance teams, and business stakeholders to deliver reliable, high-quality, and governed data products on GCP.

Core Responsibilities:

Design, develop, and maintain scalable data ingestion, transformation, and orchestration pipelines using GCP-native services such as Dataflow, Dataproc, Cloud Composer, and Pub/Sub

Implement and manage GCP Dataplex lakes and zones configuring data discovery, metadata tagging, data quality rules, and governance policies

Build and manage data lake and lakehouse architectures on Cloud Storage and BigQuery, following best practices for partitioning, clustering, and cost optimisation

Collaborate with Data Governance and Architecture teams to ensure data quality, lineage, and cataloging standards are enforced across data pipelines

Develop and maintain ELT/ETL workflows using Cloud Composer (Apache Airflow), Dataflow (Apache Beam), and Dataproc (Spark/Hadoop)

Implement data access controls and security policies including IAM roles, column/row-level security, and VPC Service Controls on GCP

Monitor, troubleshoot, and optimise pipeline performance and reliability in production environments

Participate in code reviews, technical documentation, and contribute to data engineering best practices and standards

Support data migration and modernisation initiatives by moving on-premise or legacy data workloads to GCP cloud-native solutions

Mandatory skill sets:

Hands-on experience with GCP Dataplex creating and managing lakes, zones, assets, data quality rules, and metadata discovery

Strong proficiency in BigQuery writing complex SQL, data modelling, performance tuning, partitioning, and clustering

Experience building data pipelines using Cloud Dataflow (Apache Beam) and/or Dataproc (Apache Spark)

Hands-on experience with Cloud Composer (Apache Airflow) for pipeline orchestration and scheduling

Proficiency in Python and SQL for data transformation and pipeline development

Experience with Cloud Storage for data lake design raw, refined, and curated layer management

Working knowledge of data governance concepts data quality, metadata management, data lineage, and cataloging