Senior Backend Developer - LLM Agent Platform

About the Role

Seeking a Senior Backend Developer to build and scale a production-grade LLM agent platform. Youll work with advanced AI orchestration (LangGraph, LangChain), high-performance async Python services, real-time streaming, multi-provider LLM integrations, and enterprise-grade safety/guardrails. This is not prototype workthe platform is fully tested, observable, documented, and built for scale.

Core Tech Stack

Backend & Services

• Python 3.11+, async/await, mypy (strict typing)
• FastAPI for async APIs (OpenAPI docs)
• Pydantic v2
• UV package manager
• Docker (optimized multi-stage builds)

AI & Orchestration

• LangGraph + LangChain workflows
• Multi-provider LLM support (OpenAI, Anthropic, Azure OpenAI, OpenRouter)
• Dynamic model switching
• Human-in-the-loop (HITL) patterns
• Guardrails AI (PII, toxicity, prompt injection protection)
• MCP for external tool integrations

Infrastructure / DevOps

• PostgreSQL / SQLite / Redis for checkpointer
• Server-Sent Events (SSE) streaming
• OpenTelemetry, LangSmith
• GitHub Actions CI/CD
• Health/readiness checks + structured logging

Key Responsibilities

Architecture & Development

• Build LLM agents using LangGraph
• Implement MVCS architecture (Handler Service Gateway)
• Develop internal/MCP tools and multi-provider LLM gateways
• Build streaming endpoints with guardrails
• Optimize async workflows for high concurrency

Code Quality & Testing

• Write unit/integration/E2E tests (pytest)
• Enforce strict typing (mypy) + code standards (Ruff)
• Review PRs, maintain architecture boundaries
• Document APIs, decisions, troubleshooting

Production & Operations

• Deploy horizontally scalable, stateless services
• Implement observability: tracing, logging, metrics
• Improve LLM provider performance/cost
• Implement safety guardrails
• Troubleshoot across FastAPI, LangGraph, and LLM integrations

Collaboration

• Participate in API design
• Mentor junior engineers
• Contribute to documentation & architecture decisions

Required Qualifications

• 5+ years Python experience
• Strong async/await, FastAPI (or similar async frameworks)
• Experience with LLM APIs
• Docker + cloud deployment
• Strong understanding of REST, layered architecture, type systems
• Expertise in testing, CI/CD, security, and performance tuning
• Experience with streaming (SSE/WebSockets) and scalable systems