Co-founder of PyTorch: The Power Behind AI APIs & Building Personal Local Agents

Co-founder of PyTorch: The Power Behind AI APIs & Building Personal Local Agents

As AI becomes increasingly pervasive, personal local agents are crucial for augmenting our daily lives - but can we really trust these autonomous entities to make decisions and take actions on our behalf?

• 1. The speaker works on PyTorch, an open-source machine learning library, and is involved with Llama to some extent.
• 2. They are interested in the concept of personal agents that can take action on behalf of the user, as opposed to just providing information or context.
• 3. A key characteristic of a highly intelligent agent is its ability to have context and act in the world.
• 4. Without the right context, even a highly intelligent agent can be as useless as a "bag of rocks."
• 5. The speaker gives the example of a personal agent that has access to their Gmail, WhatsApp, and calendar, but fails to inform them that they have not renewed a prescription because it did not have
• 6. The speaker argues that personal agents should be local and private, for several reasons:
• * Centralized services can be a single point of failure or attack.
• * Users may not want certain types of data (e.g., intimate conversations) to be stored in the cloud.
• * Local agents can provide more control and customization for the user.
• 7. The technical challenges of building local agents include:
• * Slow and limited local model inference.
• * Lack of good open multimodal models for computer vision and other tasks.
• * Lack of robust catastrophic action classifiers to prevent agents from taking harmful actions.
• 8. The speaker is optimistic about the potential of open models, which are improving at a faster rate than closed models due to their collaborative nature.
• 9. There are ongoing efforts to address the challenges of local agents, including improving open multimodal models and developing better catastrophic action classifiers.
• 10. The speaker mentions several projects they are involved with, including Galactica, an open science model, and PyTorch, which is working on enabling local agents. They also invite the audience to a

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!