Data Platform Software Engineer — Lead

Job Title: Data Platform Software Engineer Lead

Location: Bangalore, Karnataka India

Minimum Experience: 8-13 month contract with multiple extensions

ROLE OVERVIEW

We are looking for a Data Platform Lead to own the design, build, and governance of our enterprise data platform. This is a role within the Data COE, responsible for driving platform maturity across a federated, multi-cloud environment. You will set engineering standards, eliminate data duplication and drift across domains, and enable reliable, governed data access without creating centralisation bottlenecks.

KEY RESPONSIBILITIES

• Design and evolve the enterprise data platform spanning Snowflake and Databricks across Azure and AWS
• Build and maintain production-grade ELT/ETL pipelines using Spark, dbt, Airflow, and Kafka
• Define and enforce platform standards for ingestion, transformation, storage, and access across federated domain teams
• Drive data quality, data contract adoption, and lineage visibility across the estate
• Own cloud storage architecture across S3 and ADLS; lead CI/CD practices for all platform assets
• Lead containerized workload deployment using Kubernetes for data platform services
• Coordinate with domain teams to align on standards; represent platform engineering in architecture forums
• Mentor mid-level engineers and conduct technical design reviews for platform-impacting changes

MUST-HAVE SKILLS

• Snowflake expert-level: data modelling, performance tuning, role-based access, zero-copy sharing, dynamic data masking
• Databricks production experience: Delta Lake, Unity Catalog, cluster management, Databricks Workflows
• Python and SQL strong proficiency; Scala advantageous
• Apache Spark distributed data processing at scale
• Apache Kafka real-time event streaming, pipeline design, topic management
• Apache Airflow pipeline orchestration, DAG design, dependency management
• dbt transformation layer modelling, testing, documentation
• Cloud storage S3 and ADLS; familiarity with Parquet, Delta, and partitioning strategies
• Kubernetes container orchestration for data platform services
• CI/CD GitHub Actions or equivalent for data platform assets (pipelines, schemas, infra)
• 8+ years of data engineering experience, with 3+ years in a lead or staff IC capacity

GOOD-TO-HAVE SKILLS

• Data quality frameworks Great Expectations or equivalent, embedded in pipeline execution
• Feature Store platforms Feast, Tecton, or equivalent; ML data infrastructure design
• Data catalogue and lineage tooling Amundsen, DataHub, or equivalent
• Data contracts as an engineering practice schema registries, versioning, producer/consumer agreements
• Open table formats Apache Iceberg; lakehouse architecture patterns
• Cross-cloud experience Snowflake (Azure), Databricks, and BigQuery integrations

WHAT WE'RE LOOKING FOR

• Systems thinker able to see how pipeline, storage, and governance decisions interact at scale
• Able to influence without authority across autonomous, federated domain teams
• Opinionated on standards and quality, but pragmatic in how they are adopted across an existing estate
• Strong communicator able to translate platform complexity to both engineering peers and business stakeholders
• Ownership-driven, with a track record of improving platform reliability, cost efficiency, and developer experience