Microsoft AI Engineers Showcase AI Red Teaming Tools at AI Conference" (19 words)

Welcome to the AI Engineer Conference, where I'm KJ Kanazawa from Microsoft, discussing the importance of AI red teaming in building trustworthy AI systems.

  • * KJ Kanazawa from Microsoft works in product and AI foundry
  • * Nakumar is an engineer on the team
  • * They're at the AI engineer conference
  • * The goal of the team is to get AI into the hands of people and help them with end users and internal users
  • * There are concerns about AI being easy to trick or misuse
  • * For example, chatbots can be manipulated into revealing sensitive information or performing actions they shouldn't
  • * AI models are built on an ecosystem of different things, including Python packages and other services
  • * Self-driving cars in San Francisco have been known to drive past school buses with stop signs, highlighting the potential risks of AI
  • * It is important for AI engineers to consider these risks and build trustworthy systems
  • * Microsoft has a team called the Microsoft AI red team that helps identify and mitigate AI risks
  • * They have partnered with the Azure AI foundry to offer a solution that makes it easy for AI engineers to use the AI red team's expertise
  • * The solution is a hosted Python package called pyate, wrapped in an easy-to-use SDK and dashboard
  • * It allows AI engineers to test their systems for vulnerabilities and ensure they are secure
  • * Nakumar demonstrates how to use the tool by running a simple RAG on Postgres SQL app with an Azure sample
  • * He shows how the semantic kernel, which is a key part of the system, can be used to understand and respond to natural language inputs
  • * The system is able to detect and prevent potential attacks or misuse, such as trying to access sensitive data or performing malicious actions
  • * Overall, the tool helps AI engineers build more secure and trustworthy systems by identifying and addressing potential vulnerabilities.

Source: AI Engineer via YouTube

❓ What do you think? What are your thoughts on the ideas shared in this video? Feel free to share your thoughts in the comments!