In-House AI Strategy: Proof, Recipe, & $Millions

In-House AI Strategy: Proof, Recipe, & $Millions - Our Success Story

Discover how our AI strategy went from 'paying for truffles to garnish instant noodles' to generating millions of dollars in revenue, and learn the 5 key lessons that will help you avoid common pitfalls and achieve similar results.

1. Many companies order AI strategies "off the shelf" without fully understanding their needs, potentially wasting resources on features they don't require.
2. The speaker's company chose to build their own system, taking roughly two developers and 10-12 sprint weeks, which resulted in significant revenue growth and a group level award.
3. "Off the shelf" solutions can be expensive and overkill for smaller companies or specific use cases; it is essential to consider when it makes sense to build versus buy.
4. The speaker recommends building an in-house system once you've identified your workflow, as this allows for customization and cost savings.
5. Focus on a single, painful job to be done rather than attempting a comprehensive solution; choose something with a clear dollar value outcome.
6. Identify the specific "value event" that you are working towards to measure success.
7. Instrument everything to track actual revenue moved by your AI system; offline evaluations and metrics like F1 score or NDCG are secondary.
8. Build your revenue funnel from start to finish, tracking each step back to the value event.
9. Your users should be your guide rails for running ambitious experiments and making decisions/prioritization.
10. Anticipate user needs and provide solutions proactively; don't wait for users to request help.
11. The speaker's company created a daily digest system, which increased engagement by an order of magnitude compared to their chat app.
12. Once you have a working AI system, focus on guiding action rather than just delivering information; ensure time saved is used wisely.
13. A proactive system can surface valuable insights users might not have considered otherwise.
14. Invest development resources in high-quality data over great models; good data is less expensive and more effective in the long run.
15. The best results come from simple additions to alert users, rather than changing complex models.
16. Building for what your users need, rather than chasing model benchmarks, can create a powerful feedback loop leading to improvements and increased adoption.
17. Tight feedback loops with users encourage them to provide ideas for improvement, creating a flywheel effect for revenue growth.
18. Focusing on the basics and building simple solutions based on user needs pays off in the long run.
19. The speaker encourages starting small, focusing on revenue impact, and letting users guide the AI strategy.
20. Revenue impact should always take priority over evaluation metrics; don't be swayed by "shiny" models or features that don't directly contribute to your bottom line.
21. The speaker emphasizes the importance of tracking dollar-based outcomes and making decisions based on actual revenue moved, not just offline evaluations or other metrics.
22. Proactively anticipating user needs and providing solutions can lead to increased engagement and higher value for users.
23. High-quality data is crucial for building an effective AI system; investing in good data is more cost-effective than investing in great models.
24. Building an AI strategy should be a collaborative effort between developers, managers, and users, with everyone working together to prioritize revenue growth and user value.

Source: AI Engineer via YouTube

❓ What do you think? What is the most significant factor preventing organizations from achieving their AI goals, and how can they overcome this barrier to achieve meaningful revenue growth? Feel free to share your thoughts in the comments!