Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google & Amazon

Aishwarya Naresh Reganti and Kiriti Badam have helped build and launch more than 50 enterprise AI products across companies like OpenAI, Google, Amazon, and Databricks. Based on these experiences, they’ve developed a small set of best practices for building and scaling successful AI products. The goal of this conversation is to save you and your team a lot of pain and suffering. *We discuss:* 1. Two key ways AI products differ from traditional software, and why that fundamentally changes how they should be built 2. Common patterns and anti-patterns in companies that build strong AI products versus those that struggle 3. A framework they developed from real-world experience to iteratively build AI products that create a flywheel of improvement 4. Why obsessing about customer trust and reliability is an underrated driver of successful AI products 5. Why evals aren’t a cure-all, and the most common misconceptions people have about them 6. The skills that matter most for builders in the AI era *Get 15% off Aishwarya and Kiriti’s Maven course, Building Agentic AI Applications with a Problem-First Approach, using this link:* https://bit.ly/3V5XJFp *Brought to you by:* Merge—The fastest way to ship 220+ integrations: https://merge.dev/lenny Strella—The AI-powered customer research platform: https://strella.io/lenny Brex—The banking solution for startups: https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs *Episode transcript:* https://www.lennysnewsletter.com/p/what-openai-and-google-engineers-learned *Archive of all Lenny's Podcast transcripts:* https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=4jmgl11w&dl=0 *My biggest takeaways (for paid newsletter subscribers):* https://www.lennysnewsletter.com/i/183007822/referenced *Where to find Aishwarya Naresh Reganti:* • LinkedIn: https://www.linkedin.com/in/areganti • GitHub: https://github.com/aishwaryanr/awesome-generative-ai-guide • X: https://x.com/aish_reganti *Where to find Kiriti Badam:* • LinkedIn: https://www.linkedin.com/in/sai-kiriti-badam • X: https://x.com/kiritibadam *Where to find Lenny:* • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ *In this episode, we cover:* (00:00) Introduction to Aishwarya and Kiriti (05:03) Challenges in AI product development (07:36) Key differences between AI and traditional software (13:19) Building AI products: start small and scale (15:23) The importance of human control in AI systems (22:38) Avoiding prompt injection and jailbreaking (25:18) Patterns for successful AI product development (33:20) The debate on evals and production monitoring (41:27) Codex team’s approach…