Exploring Full Stack AI Engineering in a Serverless Environment: A Production Approach Using Next.js, OpenAI Agents SDK, Ingest, and Vercel

As AI engineers, how do we deploy zero-ops resilient agent-powered user-ready apps today? Let's dive into the modern way of running production workloads in a full-stack and serverless environment.

  • * Full stack AI engineering in a serverless environment involves building client apps, using agent frameworks, an orchestration layer, and deploying everything on a serverless platform.
  • * There are many options for each component of this infrastructure.
  • * Client app examples include Remix, Astro, Svelte, Next.js.
  • * New agent frameworks are emerging regularly, like Langchain, Verscel's AI SDK, Flowwise agents, OpenAI's agents SDK, etc.
  • * Orchestration layers include Temporal, AWS Step Functions, Langmith ingest, and many more.
  • * Serverless environments to deploy to include Lambda, API Gateway, Bedrock on AWS, Google Cloud's Vertex AI, Azure's AI Studio, Verscel, etc.
  • * With so many options and combinations available, it can be challenging to know what works best together.
  • * The speaker has found success using Next.js for client apps, OpenAI's agents SDK for agent framework, Ingest for orchestration layer, and Verscel for serverless deployment.
  • * This combination offers features like streaming first-class server actions, file-based routing, built-in tracing, easy model interchangeability, event-driven orchestration, and more.
  • * The recommended architecture involves a Next.js app that triggers AI workflows through Ingest services, which manage connections to Python serverless functions hosting the OpenAI agents SDK.
  • * The speaker has built an example app using these tools to create a newsletter with AI agents focused on scalability and local developer experience.
  • * To run the example app, users need to clone the repository and open three terminals for Nex.js, Python agents, and Ingest dev server.
  • * Users can input topics in the app's homepage, and it will generate a newsletter using data from OpenAI agents powered by FastAPI.
  • * The architecture ensures cost-efficiency and reliability with continuous deployment through Verscel and full type safety with Pyantic and TypeScript across Nex.js.
  • * Users can explore and contribute to the example app on GitHub, where they can see how the pieces fit together in detail.
  • * Ingest provides lightweight orchestration by breaking tasks into individual steps that scale up and down according to Verscel's cloud function limits.
  • * The code for this architecture includes an ingest endpoint, newsletter API endpoints, Python agents directory, and an ingest folder defining different workflows.
  • * AI agents are defined in a FastAPI app, with each agent having its own endpoint, and Verscel automatically hosts the functions when it detects the top-level API directory.
  • * Users can define custom environment variables like OpenAI API key and Verscel blob storage token for their project.
  • * Using this favorite combination of tools, users can achieve scalability, resilience, and powerful agentic capabilities in their full stack AI engineering projects.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!