Senior Product Manager – AI Infrastructure

About the Role

We are building a next-generation AI inference platform that virtualizes the full AI stack from GPU infrastructure and runtime optimization to multi-tenant model serving.

We are looking for a highly technical Product Manager who can operate at the intersection of distributed systems, GPU infrastructure, and AI runtime engineering.

This role requires deep systems understanding. You will partner directly with platform and runtime engineers to define architecture, platform capabilities, and economic strategy for our AI execution fabric.

What You Will Own

Product strategy for AI inference and GPU platform services

Roadmap for:

• Multi-tenant inference
• GPU orchestration and GPU scheduling
• Inference runtime capabilities
• Model sharding and dynamic batching
• KV cache optimization
• Kubernetes-based resource scheduling

Define SLOs for latency (p95/p99), throughput, and availability

Drive GPU utilization improvements and cost-per-token optimization

Shape autoscaling and resource scheduler strategies

Define pricing models (usage-based, token-based, throughput tiers)

Partner with engineering on control plane vs data plane design decisions

Ensure enterprise-grade capabilities (IAM, isolation, audit, observability)

What Were Looking For

Required

• 612+ years of product/ engineering experience in building cloud infrastructure, compute, or platform products

Strong understanding of distributed systems and multi-tenant architectures

Familiarity with GPU platforms (CUDA, MIG, NVLink, NCCL)

Exposure to AI inference runtimes (TensorRT, ONNX Runtime, Triton, vLLM)

Experience with Kubernetes, autoscaling, and resource schedulers

Ability to reason about:

• Model sharding
• Dynamic batching
• KV cache management
• GPU memory constraints

Experience defining platform SLOs and capacity planning

Experience with usage-based or infrastructure pricing models

Preferred

• Experience at a hyperscaler or cloud infrastructure company
• Experience with AI inference platforms or ML infrastructure
• Exposure to Kubernetes, scheduling systems, or multi-tenant platforms
• Experience defining usage-based pricing models
• Background in engineering (CS degree or prior software engineering role)

What This Role Is Not

• Not an AI research role
• Not a consumer AI product role
• Not an LLM application PM role
• Not a purely commercial or marketing product role
• Not a AI implementation PM role