Building AI Agents for Automating Unstructured Data: A Focus on Knowledge Work

Join Jerry, Co-founder & CEO of Llama Index, as he explores the power of AI agents in automating knowledge work, highlighting use cases and architectures that can revolutionize the way we process unstructured data.

  • 1. Jerry is the co-founder and CEO of Llama Inducts.
  • 2. The talk is about building AI agents that automate knowledge work.
  • 3. Knowledge workers spend a lot of time reviewing unstructured data in various forms such as PDFs, Powerpoints, Word documents, and Excel files.
  • 4. AI agents can reason and act over massive amounts of unstructured context tokens, analyze, research, synthesize insights and take actions end-to-end.
  • 5. Use cases for AI agents that automate knowledge work fall into two main categories: assistive agents and automation type agents.
  • 6. Assistive agents are more like a standard chat interface that helps humans get more information faster.
  • 7. Automation type agents automate routine tasks, can run in the background and require less human intervention.
  • 8. The stack required to build AI agents includes nice tools to interface with the external world and an agent architecture with MCP 80A.
  • 9. Building a document toolbox is essential for AI agents to interact with massive amounts of unstructured documents.
  • 10. A pre-processing layer is necessary to create tool interfaces, including data connectors, document parsing, extraction, and indexing.
  • 11. The document MCP server is a generalization of the idea of rag (retrieval augmented generation) that equips an AI agent with tools to understand and reason over unstructured documents.
  • 12. Agent orchestration ranges from more constrained architectures to unconstrained architectures, and there are two main categories of UXs: assistant-based UXs and automation interface UXs.
  • 13. Assistant-based UXs surface information and help a human produce some unit of knowledge work through a chat-based interface, while automation interface UXs process routine tasks in a multi-step en
  • 14. Automation agents serve as a backend for data ETL transformation, while assistant agents are more front-end facing and provide research user-facing interfaces.
  • 15. Document agents help automate different types of knowledge work, such as financial due diligence, enterprise search use cases, and technical data sheet injection.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!