Building Reliable AI Customer Support Systems with Effect: Handling Complexity and Uncertainty

Building Reliable AI Customer Support Systems with Effect: Handling Complexity and Uncertainty

As CTO of 14.ai, I'm passionate about building reliable AI-native customer support platforms with Effect, a TypeScript library that empowers us to write robust, type-safe, and composable code at scale.

• 1. Michael is co-founder and CTO of 14.ai, which builds an AI-native customer support platform.
• 2. Their systems interact directly with end users and use large language models (LLMs) in production, requiring reliability under uncertain conditions.
• 3. They use the Effect TypeScript library to manage complexity, as TypeScript alone falls short when dealing with unreliable APIs, complex dependencies, non-deterministic model outputs, or long-runnin
• 4. Effect provides strong type guarantees, composition primitives, built-in concurrency, streaming, interruptions, retry mechanisms, structured error modeling, and a clean dependency injection system
• 5. The architecture includes:
• * A React front-end for dashboards, agent UI, knowledge management, insights, analytics, SDKs, and more.
• * An internal RPC server handling app logic using Effect RPC and a modified version of TanStack Query on the front-end.
• * A public API server using Effect HTTP with OpenAPI docs auto-generated from annotated schemas.
• * A data processing engine syncing data from CRM, docs, and databases for real-time analytics and reporting.
• * Agent workflows written in a custom DSL built on Effect, allowing deterministic and non-deterministic behavior mixing.
• * PostgreSQL for both data and vector storage, with Effect SQL handling queries.
• 6. They use Effect schemas for runtime validation, encoding, decoding, type-safe input and output handling, and auto-generated documentation across the stack.
• 7. Agents are "planners" that take user input, create a plan, choose appropriate action workflows or sub-agents, execute them, and repeat until the task is complete.
• 8. Actions are small units of execution like fetching payment info or searching through logs, while workflows are deterministic multi-step processes.
• 9. Sub-agents group related actions and workflows into larger domain-specific modules like a billing agent or log retrieval agent to manage complexity.
• 10. They built a domain-specific language for workflows using Effect's functional pipe-based system, enabling clear and composable expressions of branching, sequencing, retries, state transitions, and
• 11. For reliability, they model fallback strategies with retry policies that track state to avoid retrying failed providers often.
• 12. They duplicate token streams for analytics purposes, storing one directly for the user and one on their side.
• 13. Effect helps manage dependencies using dependency injection (DI) to mock LLM providers and simulate failure scenarios for testing.
• 14. The developer experience with Effect is excellent due to its schema-centric approach, strong typing, auto-generated documentation, DI, modularity, and composability.
• 15. Effect provides guard rails to help prevent common mistakes, especially for engineers new to TypeScript.
• 16. Using Effect effectively requires discipline, as it can create a false sense of safety if not careful with error handling and dependency tracing.
• 17. The learning curve is real, but the benefits compound after getting past the initial bump.
• 18. Effect helps build predictable and resilient systems without being an all-or-nothing solution; it can be adopted incrementally.
• 19. Effect is particularly useful for LLM and AI-based systems where reliability and coping with non-determinism matter.
• 20. Effect brings the rigor of functional programming into real-world TypeScript while remaining practical for production use.
• 21. Start small with Effect to let benefits build up over time, making your systems predictable, observable, and reliable.
• 22. 14.ai is open to discussing their experience and lessons learned when building with agents, LLMs, or exploring Effect.

Source: AI Engineer via YouTube

❓ What do you think? What are some potential downsides or limitations of using Effect, despite its many benefits, that you've experienced or anticipate as your platform evolves? Feel free to share your thoughts in the comments!