Scaling Systems with Compute: The Rise of AI Agents

Unlocking the power of AI, I'm here to share a groundbreaking idea that could revolutionize how we build systems that scale with compute.

1. Speaker has been working on Learning Management Systems (LMS) for four years.
2. Started focusing on AI agent company when ChatGPT was released.
3. Initially, they used GPD2 models but found them to be frustratingly stupid with small context windows and poor reasoning capabilities.
4. They wrote lots of code around these models to make them work reliably.
5. As models became smarter, they had to delete more of the custom code written earlier.
6. Observed patterns in building agents that can scale with intelligence.
7. Built a structure extraction library called Jsonformer.
8. Wrote code around models as they were too stupid to work with JSON.
9. Core agenda: convey one idea through various examples and demos.
10. Idea: systems that scale with compute beat systems that don't.
11. Rigid, fixed, deterministic systems can think or use more compute when needed.
12. Exponentials are rare and should be taken advantage of when found.
13. Examples from history: chess, go, computer vision, Atari games.
14. General method always ends up winning as it scales out search capabilities.
15. Ramp is a finance platform that helps businesses manage expenses, payments, procurement, and travel efficiently.
16. AI is used extensively in the product to automate routine tasks.
17. Discussing an agent called Switching Report: a checklist for transactions from third-party card providers when people onboard to Ramp.
18. Want to support arbitrary CSV formats from other platforms with crazy schemas.
19. Three approaches to solving the problem: manual code, LMS with scripting, and giving the entire CSV to an LLM for interpretation.
20. The third approach works well when run 50 times in parallel, generalizing across different formats.
21. Although it requires significantly more compute, engineer time is scarce and the cost of running the system is lower than manual processing per transaction.
22. Different approaches can be represented as arrows: Black Arrow (classical compute), Blue Arrows (fuzzy land with neuron nets and weird matrix multiplication).
23. Ramp codebase is moving towards an approach where the LLM decides when to break into classical land and write some code or use pandas/Python code.
24. More blue arrow usage in a company's codebase can help directly by leveraging exponential trends without much effort from their end.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!