Building LinkedIn's JI Platform: A Journey into Compound AI Systems

Join me, Shad, as we embark on a journey to build LinkedIn's Ji platform, exploring its development, key components, and lessons learned.

  • 1. Sh is the ad manager of J Foundation and will discuss building LinkedIn's JI platform.
  • 2. The JI product experience is supported by their platform, starting with Collaborative Articles in 2023.
  • 3. Collaborative Articles were a simple GI feature using ChatGPT (GPT-4 model) to create long content.
  • 4. LinkedIn's team built key components such as the gateway for centralized access and some Pyon notebooks for prompt engineering.
  • 5. Two different tech stacks were used: Java online, Python backend.
  • 6. Limitations of the simple approach led to developing the second generation JI product, called Co-Pilot or Coach.
  • 7. The new feature provided personalized recommendations based on user profiles and job descriptions.
  • 8. A Python SDK was built on a popular framework for orchestrating API calls and integrating with large scale infrastructure.
  • 9. The text stack was unified to reduce costs associated with transferring prompts between Java and Python.
  • 10. Prompt management, a submodule for prompt versioning and structure, was introduced.
  • 11. Conversational memory infrastructure helps track and retrieve content from LLM interactions.
  • 12. LinkedIn launched the first real multi-agent system, LinkedIn H Assistant, in 2024 to help recruiters automate tasks.
  • 13. The Python SDK support was extended for distributed agent orchestration.
  • 14. A skill registry was created to handle skill discovery and invocation, making it easy for developers to call APIs for tasks.
  • 15. Experiential memory extracts, analyzes, and infers textural knowledge from interactions between agents and users.
  • 16. Operability is essential as agents have autonomous behavior; an in-house solution was built on top of a toolkit to track low-level granularity of telemetry data.
  • 17. The platform consists of four layers: orchestration, prompt engineering tools, skills invocation, and memory management.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!