Data Architect (Data Engineering & AI/ML Platforms)

Role Overview

We are looking for a Data Architect with a strong foundation in data engineering and hands-on experience designing scalable data platforms and intelligent pipelines. This role focuses on architecting end-to-end data ecosystems, including batch/streaming pipelines, machine learning workflows, and next-generation AI systems powered by LLMs and agent-based frameworks.

You will define data architecture standards, guide platform evolution, and enable advanced analytics and AI use cases across the organisation.

Key Responsibilities:

Generative AI & LLM Systems

• Architect AI solutions using OpenAI APIs and open-source models like LLaMA.
• Build RAG (Retrieval-Augmented Generation) pipelines for enterprise knowledge systems.
• Develop agentic workflows using frameworks such as LangChain and LangGraph.
• Design and deploy AI agents capable of multi-step reasoning and tool integration.
• Implement Model Context Protocol (MCP)-based architectures for tool interoperability.

Data Architecture & Platform Design

• Design and implement scalable, secure, and high-performance data architectures (data lakes, lakehouses, warehouses).
• Define data modelling standards (dimensional, data vault, and domain-driven design).
• Architect batch and real-time data pipelines using technologies like Apache Spark and Apache Kafka.
• Ensure data governance, quality, lineage, and compliance across systems.

Data Engineering & Pipelines

• Build and optimise ETL/ELT workflows using orchestration tools like Apache Airflow.
• Develop scalable ingestion pipelines for structured, semi-structured, and unstructured data.
• Implement CI/CD pipelines and infrastructure automation using Terraform.
• Containerise and deploy services using Docker and orchestrate via Kubernetes.

Machine Learning & AI Pipelines

• Design and operationalise ML pipelines, including training, evaluation, and deployment workflows.
• Build and manage feature stores and model lifecycle pipelines.
• Integrate ML systems with production-grade data platforms.

.Cloud & Modern Data Stack

• Architect solutions on cloud platforms (AWS, Azure, or GCP).
• Design cost-efficient, scalable storage and compute layers.
• Enable real-time analytics and streaming data use cases.

Leadership & Strategy

• Collaborate with stakeholders, data scientists, and engineering teams to define data strategy.
• Establish best practices for data architecture, security, and AI governance.
• Mentor engineers and guide technical decision-making.

Required Qualifications

• Bachelors/Masters degree in Computer Science, Engineering, or related field.
• 8+ years of experience in data engineering, with 23+ years in data architecture roles.
• Strong hands-on experience with:
• Distributed data processing (Spark)
• Streaming systems (Kafka)
• Workflow orchestration (Airflow)
• Experience designing ML pipelines and deploying models in production.
• Solid understanding of data modelling, data warehousing, and big data ecosystems.
• Experience with containerization and orchestration (Docker, Kubernetes).
• Proficiency in Python, SQL, and/or Scala.

Preferred Qualifications

• Experience with RAG pipelines, vector databases, and semantic search.
• Hands-on experience with LLMs, prompt engineering, and AI agents.
• Familiarity with LangChain, LangGraph, and MCP-based integrations.
• Experience with DevOps/DataOps practices and CI/CD pipelines.
• Knowledge of data governance frameworks and security best practices.

Nice to Have

• Experience with real-time analytics and event-driven architectures.
• Exposure to MLOps platforms (e.g., MLflow, Kubeflow).
• Certifications in cloud platforms (AWS/Azure/GCP).

Key Competencies

• Strong system design and architectural thinking
• Problem-solving and analytical skills
• Communication and stakeholder management
• Ability to translate business needs into scalable technical solutions