We wanted to have a deep dive into LLMs and AI as they exist in May 2024 and were lucky to get one of the pre-eminent experts in the space – founder of MosaicML and VP of gen AI at Databricks – Naveen Rao. In this we cover:
- the state of AI in May 2024
- reactions to gpt4o and the Google IO Gemini launches
- Small models vs large models
- open source models – the state of the art, biz models for open source
- are transformers the end state of LLM architecture
- Agentic behavior
- How do we get more data for training?
- And much more!
5:32 What do enterprises think of AI
8:00 ChatGPT-4o and Google I/O reactions
9:38 Has generative AI scaling hit a wall?
16:34 AI pretraining difficulties
22:49 General AI vs. specialized AI
25:55 Biggest opportunities for generative AI startups
29:15 AI hardware landscape
33:23 How open source AI companies can compete with closed models like OpenAI? 38:59 Why Naveen doesn’t use AI in his home
40:22 Is open-sourcing AI dangerous?
43:46 Advice for founders
47:00 Naveen sold to Intel too early
51:09 The pace of AI innovation
