Lead Big Data Developer - VP - PUNE

We are seeking a highly experienced, strategic, and technically proficient Senior Lead Big Data Developer (C13) to drive the development, standardization, and governance of data across our Big Data Portfolio. This role is crucial in enabling enterprise-wide data consistency, ensuring adherence to industry data standards, and optimizing data pipelines for high-performance applications.  

The ideal candidate will have extensive experience in data standardization, metadata management, lineage tracking, and data governance, along with deep expertise in modern Big Data technologies and distributed computing frameworks. This role demands a thought leader who can work closely with business stakeholders, data scientists, and engineers to drive data strategy, architecture, and operational efficiencies.

 Key Responsibilities:
- Lead the design, development, and optimization of scalable, standardized, and high-performance data architectures.
- Define and implement data standardization frameworks, metadata management, and data governance principles across multiple projects.
- Collaborate with business, compliance, risk, and regulatory teams to ensure data meets industry standards (ISO, FIBO, BCBS 239, etc.).
- Drive data lineage, data quality, and reconciliation strategies to improve data reliability and accessibility.
- Architect and optimize data pipelines (ETL/ELT) with streaming and batch processing methodologies.
- Partner with cross-functional teams to automate data reconciliation, validation, and anomaly detection.
- Implement data security and privacy controls in compliance with internal policies and external regulations.
- Mentor and guide junior and mid-level engineers while fostering a culture of engineering excellence and continuous learning.
- Own end-to-end data lifecycle management, ensuring seamless ingestion, processing, and storage of structured and unstructured data.

Qualifications:
- 10+ years of experience in data engineering, software development, and Big Data platforms.
- Expertise in Java , J2EE, Springboot, Micro Services, Big Data technologies (Spark, Hadoop, Hive, Impala, Kafka, Scala, Cloudera, Databricks).
- Hands-on experience in data standardization, data governance, and metadata management.
- Strong SQL and NoSQL database expertise (Oracle, MongoDB, Couchbase).
- Proficiency in Python, Java, and Shell scripting for data processing and automation.
- Deep understanding of data modeling, data lineage, and ETL frameworks.
- Knowledge of cloud-based data solutions (AWS, GCP, or Azure).
- Familiarity with machine learning-based anomaly detection and automated reconciliation frameworks.
- Experience in Capital Markets, Risk, Finance, and Regulatory Reporting is a plus.
- Agile leadership experience, with the ability to drive data initiatives in an Agile/DevOps environment.
- Strong analytical and problem-solving skills with a data-driven mindset.

Preferred Skills:
- Expertise in Canonical Data Models and Data Mesh Architecture.
- Experience working with ISO standards for financial data interoperability.
- Ability to influence senior stakeholders and drive data-driven decision-making.
- Strong documentation and technical storytelling skills for presenting data strategies.

This role offers the opportunity to lead enterprise-wide data initiatives, enhance data quality and standardization, and drive long-term strategic improvements in Big Data ecosystems.