Exploring the Emergence of Open Language Models: The Story of Open Router

Exploring the dawn of multimodal AI, where language models like Open Router are revolutionizing software development.

1. The speaker started Open Router in early 2023 to explore the potential of the AI market.
2. They wanted to answer the question: "Will this market be winner-take-all?"
3. Open AI was the leading model, with a few others close behind.
4. The speaker built prototypes and investigated open source options.
5. In January, they noticed users wanting different types of models for moderation.
6. Next month, the open-source race began with models like Bloom 176b and OPT by Facebook.
7. Llama 1, launched in February by Meta, outperformed GPT3 on most benchmarks.
8. In March, Alpaca was successfully distilled from Llama 1, making it possible to transfer style and knowledge from large models to small ones for less than $600.
9. Open Router started as a place to collect these language models but evolved into a marketplace over time.
10. In April, the speaker launched Window AI, an open-source Chrome extension that lets users choose their model and web apps to "suck it in."
11. Open Router now has 400+ models from over 60 active providers, with crypto as a payment method.
12. Initially, there were only one or two providers for each open-source model, but this quickly changed.
13. The ecosystem of language models became "out-of-control heterogeneous," leading to the creation of Open Router as a marketplace.
14. Aggregating providers helped developers increase uptime and access real-world data on latency and throughput.
15. Open Router helps users compare models, use their own prompts, have fine-grain privacy controls, and see usage across all models.
16. The speaker believes the AI market will not be winner-take-all; instead, it will be multimodel.
17. Many customers use different models for various purposes, realizing significant gains by doing so.
18. Inference is becoming a dominant operating expense, and selecting and routing will be crucial.
19. Open Router aims to make the "wild ecosystem" more homogeneous and easier to work with for developers.
20. The speaker shared their idea for middleware within Open Router, enabling new features and abilities like web searching and PDF parsing for all models.
21. Middleware is optimized for inference and works similarly to the way it does in web development frameworks.
22. Plugins can call MCps (Model Cards for Model Reporting) from inside a plugin and augment results on the way back to the user.
23. Open Router focuses on low latency, with custom cache work bringing latency down to 30 milliseconds.
24. The company plans to add more modalities, like generating images through transfusion models, and improve routing capabilities.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!