High Agency: The Podcast for AI Builders

Jeff Huber of Chroma: Building the open-source toolkit for AI Engineering Oct 24, 2024

This week on High Agency, Raza Habib is joined by Chroma founder Jeff Huber. They cover the evolution of vector databases in AI engineering, challenge common assumptions about RAG and share insights from Chroma's journey. Jeff shares insights from Chroma's development, including their focus on developer experience and observations about real-world usage patterns. They also get into whether or not we can expect a super AI any time soon and what is over and under hyped in the industry today.

00:00 - Introduction
02:30 - Why vector databases matter for AI
06:00 - Understanding embeddings and similarity search
12:00 - Chroma early days
15:45 - Problems with existing vector database solutions
19:30 - Workload patterns in AI applications
23:40 - Real-world use cases and search applications
27:15 - The problem with RAG terminology
31:45 - Dynamic retrieval and model interactions
35:30 - Email processing and instruction management
39:15 - Context windows vs vector databases
42:30 - Enterprise adoption and production systems
45:45 - The journey from GPT-3 to production AI
48:15 - Internal vs customer-facing applications
51:00 - Advice for AI engineers

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

How to Create AI Strategy in Enterprises with Peter Gostev from Moonpig Oct 16, 2024

In this episode of High Agency podcast, Peter Gostev shares his experiences implementing LLMs at NatWest and Moonpig. He discusses creating an AI strategy, talks about challenges in deploying LLMs in large organizations, and shares thoughts on underappreciated AI developments.

00:00 - Introduction
00:44 - OpenAI dev day reactions
03:47 - Using AI to automate customer service
10:43 - Impact of AI products
13:41 - Who are the users of LLMs
14:47 - Challenges building with AI in a large enterprise
21:22 - AI use cases at Moonpig
24:34 - How to create an AI strategy
28:10 - Underappreciated AI developments

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an LLM evals platform for enterprises. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

Ex-Coinbase CPO's Next Big Thing: AI Employees I Surojit Chatterjee Oct 02, 2024

In this episode of High Agency, we're joined by Surojit Chatterjee, former CPO of Coinbase and now CEO of Ema. Surojit unveils his audacious plan to create universal AI employees and revolutionize Fortune 1000 workforce. Drawing from his career at tech giants like Google and Coinbase, he shares how these experiences fueled his vision for Ema. Surojit dives into the challenges of building AI agents, explores the concept of artificial humans, and predicts how this technology could transform the future of SaaS

(00:00:00) Introduction and Surojit’s background

(00:03:00) Founding story of Ema (Universal AI Employee)

(00:04:53) How the Universal AI Employee works

(00:08:39) Ema’s data integration and security

(00:12:57) AI employee use cases in enterprises

(00:15:02) Challenges with building AI agents

(00:16:45) Evaluations, hallucinations, customizing models

(00:19:52) Artificial human metaphor

(00:25:42) AI employee vs humans

(00:31:25) Advice for AI builders

(00:37:14) Is AI overhyped or underhyped?

(00:39:28) How the business model of SaaS will change

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

Why Your AI Product Needs Evals with Hamel Husain and Swyx Sep 25, 2024

Hamel Husain is a seasoned AI consultant and engineer with experience at companies like GitHub, DataRobot, and Airbnb. He is a trailblazer in AI development, known for his innovative work in literate programming and AI-assisted development tools. Shawn Wang (aka Swyx) is the host of the Latent Space podcast, the author of the essay 'Rise of the AI Engineer,' and the founder of the AI Engineer World Fair. In this episode, Hamel and Swyx share their unique insights on building effective AI products, the critical importance of evaluations, and their vision for the future of AI engineering.

Chapters
00:00 - Introduction and recent AI advancements

06:14 - The critical role of evals in AI product development

15:33 - Common pitfalls in AI product development

26:33 - Literate programming: A new paradigm for AI development

39:58 - Answer AI and innovative approaches to software development

51:56 - Integrating AI with literate programming environments

58:47 - The importance of understanding AI prompts

01:00:37 - Assessing the current state of AI adoption

01:07:10 - Challenges in evaluating AI models

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

How AI is Changing Product Management with Raz Nussbaum from Gong AI Sep 17, 2024

Raz Nussbaum is a Senior Product Manager in AI at Gong — the leading AI platform for revenue teams. He is an absolute legend when it comes to building and scaling AI products that genuinely deliver value. In this episode, he opens up about what it takes to build successful AI products in an era where things change at lightning speed.

Chapters
00:00 - Introduction
01:16 - How LLMs Changed Product Development at Gong AI
08:32 - Including Product Managers in Development Process
13:05 - Testing and Monitoring Pre vs Post-deployment
17:53 - New Challenges in the Face of Generative AI
19:39 - Shipping Fast and Interacting with the Market
23:25 - What's Next For Gong AI
25:13 - The Psychology of Trusting AI
28:19 - Is AI Overhyped or Underhyped?

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

From Fiction to Reality: Sudowrite's Journey in AI-Assisted Creative Writing Sep 11, 2024

In this episode, we dive deep into the world of AI-assisted creative writing with James Yu, founder of Sudowrite. James shares the journey of building an AI assistant for novelists, helping writers develop ideas, manage complex storylines, and avoid clichés. James gets into the backlash the company faced when they first released Story Engine and how they're working to build a community of users.

00:00 - Introduction and Background of Sudowrite
02:26 - The Early Days: Concept, Skepticism, and User Adoption
05:20 - Sudowrite's Interface, Features, and User Base
10:23 - Developing and Iterating Features in Sudowrite
17:29 - The Evolution of Story Bible and Writing Assistance
24:27 - Challenges in Maintaining Coherence and AI-Assisted Writing
29:12 - Evaluating AI Features and the Role of Prompt Engineering
33:35 - Handling Tropes, Clichés, and Fine-Tuning for Author Voice
40:43 - The Controversy and Future of AI in Creative Work
51:37 - Predictions for AI in the Next Five Years

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

Building the Nervous System for AI with Russ d'Sa from LiveKit Sep 04, 2024

In this episode, LiveKit CEO Russ d'Sa explores the critical role of real-time communication infrastructure in the AI revolution. From building voice demos to powering OpenAI's ChatGPT, he shares insights on technical challenges around building multimodal AI on the web and what new possibilities are opening up.

00:00 - Introduction and Background
01:34 - The Evolution of AI and Lessons for Founders
05:20 - Timelines and Technological Progress
10:32 - Overview of LiveKit and Its Impact on AI Development
13:39 - Why LiveKit Matters for AI Developers
19:08 - Partnership with OpenAI
21:25 - Challenges in Streaming and Real-Time Data Transmission
30:07 - Building a global network for AI communication
37:21 - Real-world applications of LiveKit in AI systems
40:55 - Future of AI and the Concept of Abundance
43:38 - The Irony of Wealth in an Age of AI

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

From PyTorch to Fireworks AI: Lin Qiao on Building AI Infrastructure Aug 28, 2024

This week we’re talking to Lin Qiao, former PyTorch lead at Meta and current CEO of Fireworks AI. We discuss the evolution of AI frameworks, the challenges of optimizing inference for generative AI, the future of AI hardware, and open-source models. Lin shares insights on PyTorch design philosophy, how to achieve low latency, and the potential for AI to become as ubiquitous as electricity in our daily lives.

Chapters:
00:00 - Introduction and PyTorch Background
04:28 - PyTorch's Success and Design Philosophy
08:20 - Lessons from PyTorch and Transition to Fireworks AI
14:52 - Challenges in Gen AI Application Development
22:03 - Fireworks AI's Approach
24:24 - Technical Deep Dive: How to Achieve Low Latency
29:32 - Hardware Competition and Future Outlook
31:21 - Open Source vs. Proprietary Models
37:54 - Future of AI and Conclusion

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

How Paras Jain is building the future of AI video creation Aug 21, 2024

In this episode of High Agency, we are speaking to Paras Jain who is the CEO of AI video generation startup Genmo. Paras shares insights from his experience working on autonomous vehicles, why he chose academia over an offer from Tesla, and the research-minded approach that has lead to Genmo's rapid success.

Chapters:
(00:00) Introduction
(01:52) Lessons from selling an AI company to Tesla
(07:01) Working within GPU constraints and transformer architecture
(11:18) Moving from research to startup success
(14:36) Leading the video generation industry
(16:05) Training diffusion models for videos
(19:36) Evaluating AI video generation
(24:06) Scaling laws and data architecture
(28:34) Issues with scaling diffusion models
(33:09) Business use cases for video generation models
(36:43) Potential and limitations of video generation
(40:59) Ethical training of video models

AI at Scale: Lessons from Gusto's $9.5 billion journey with Eddie Kim & Ali Rowghani Aug 16, 2024

In this week’s episode of the High Agency podcast, Humanloop Co-Founder and CEO Raza Habib sat down with Eddie Kim, co-founder and Head of Technology at Gusto and guest host Ali Rowghani to discuss how Gusto has applied AI to revolutionize ops-heavy processes like payroll and HR admin. Eddie also elaborates why Gusto is choosing to build, and not buy, the majority of their GenAI tech stack.

Chapters
00:00 - Introduction and Background
02:15 - Overview of Gusto's Business
05:59 - Operational Complexity and AI Opportunities
08:51 - Build vs. Buy: Internal vs. External AI Tools
10:07 - Prioritizing AI Use Cases
13:53 - Human-in-the-Loop Approach
19:39 - Centralized AI Team and Approach
22:53 - Measuring ROI from AI Initiatives
32:25 - AI-Powered Reporting Feature
38:46 - Code Generation and Developer Tools
42:52 - Impact of AI on Companies and Society
47:22 - AI Safety and Risks
49:54 - Closing Thoughts

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com/podcast

Building the first LLM-based search engine for developers with Michael Royzen Aug 02, 2024

In this episode, we sit down with Michael Royzen, CEO and co-founder of Phind. Michael shares insights from his journey in building the first LLM-based search engine for developers, the challenges of creating reliable AI models, and his vision for how AI will transform the work of developers in the near future.

Tune in to discover the groundbreaking advancements and practical implications of AI technology in coding and beyond.

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

Contrarian Guide to AI: Jason Liu on Betting Against Agents while Doubling Down on RAG & Fine-Tuning Jul 24, 2024

Jason Liu is a true Renaissance Man in the world of AI. He began his career working on traditional ML recommender systems at tech giants like Meta and Stitch Fix and quickly pivoted into LLMs app development when ChatGPT opened its API in 2022. As the creator of Instructor, a Python library that structures LLM outputs for RAG applications, Jason has made significant contributions to the AI community. Today, Jason is a sought-after speaker, course creator, and Fortune 500 advisor.

In this episode, we cut through the AI hype to explore effective strategies for building valuable AI products and discuss the future of AI across industries.

Chapters:
00:00 - Introduction and Background
08:55 - The Role of Iterative Development and Metrics

10:43 - The Importance of Hyperparameters and Experimentation

18:22 - Introducing Instructor: Ensuring Structured Outputs
20:26 - Use Cases for Instructor: Reports, Memos, and More
28:13 - Automating Research, Due Diligence, and Decision-Making
31:12 - Challenges and Limitations of Language Models
32:50 - Aligning Evaluation Metrics with Business Outcomes
35:09 - Improving Recommendation Systems and Search Algorithms
46:05 - The Future of AI and the Role of Engineers and Product Leaders
51:45 - The Raptor Paper: Organizing and Summarizing Text Chunks

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

What comes after Open AI? Logan Kilpatrick on how you should prepare for the future of LLMs Jul 17, 2024

If you need to understand the future trajectory of AI, Logan Kilpatrick will help you do just that. Having seen the frontier at both OpenAI and Google.

Logan led developer relations at OpenAI before leading product on the Google AI Studio. He's been closer than anyone to developers building with LLMs and has seen behind the curtain at two frontier labs.

Logan and I talked about:
🔸 What it was like joining OpenAI the day ChatGPT hit 1 million users
🔸 What you might expect from GPT5
🔸 Google's latest innovations and the battle with OpenAI
🔸 How can you stay ahead and achieve real ROI
🔸 Logan's insights into the form factor of AI and what will replace chatbots

Chapters:
00:00 - Introduction
01:50 - OpenAI and the Release of ChatGPT
07:43 - Characteristics of Successful AI Products and Teams
10:00 - The Rate of Change in AI
12:22 - The Future of AI and the Role of Systems
13:47 - ROI in AI and Challenges with Cost
18:07 - Advice for Builders and the Potential of Fine-Tuning
20:52 - The Role of Prompt Engineering in AI Development
25:27 - The Current State of Gemini
34:07 - Future Form Factors of AI
39:34 - Challenges and Opportunities in Building AI Startups

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com

AI's Memory Upgrade: Max Rumpf on how to build advanced RAG systems Jul 11, 2024

I'm excited to share this conversation with Max Rumpf the founder of Sid.AI. I wanted to speak to Max because Retrieval Augmented generation has become core to building AI applications and he knows more about RAG than anyone I know.

We get deep into the challenges of building RAG systems and the episode is full of technical detail and practical insights.

We cover:
00:00 - Introduction to Max Rumpf and SID.ai
03:39 - How SID.ai's RAG approach differs from basic tutorials
07:30 - Challenges of document processing and chunking strategies
13:07 - Retrieval techniques and hybrid search approaches
15:06 - Discussion on knowledge graphs and their limitations
20:58 - Reranking in RAG systems and performance improvements
32:14 - Impact of longer context windows on RAG systems
35:10 - The future of RAG and information retrieval
39:47 - Recent research papers on AI and hallucination detection
42:04 - Value-augmented sampling for language model alignment
43:11 - Future trends and investment opportunities in AI
43:50 - SEO optimization for LLMs and its potential as a business
45:20 - Closing thoughts and wrap-up

I hope you enjoy the conversation and if you do, please subscribe!

Building AI Products at Scale: Lessons from Zapier's CEO Jul 02, 2024

In this episode, I had the pleasure of speaking with Wade Foster, the founder and CEO of Zapier. We discussed Zapier's journey with AI, from their early experiments to the company-wide AI hackathon they held in March. Wade shared insights on how they prioritize AI projects, the challenges they've faced, and the opportunities they see in the AI space. We also talked about the future of AI and how it might impact the way we work

Inventing the AI Engineer Jun 24, 2024

In this episode, I chatted with Shawn Wang about his upcoming AI engineering conference and what an AI engineer really is. It's been a year since he penned the viral essay "Rise of the AI Engineer' and we discuss if this new role will be enduring, the make up of the optimal AI team and trends in machine learning.

The Rise of the AI Engineer Blog Post: https://www.latent.space/p/ai-engineer

Chapters
00:00 - Introduction and background on Shawn Wang (Swyx)
03:45 - Reflecting on the "Rise of the AI Engineer" essay
07:30 - Skills and characteristics of AI Engineers
12:15 - Team composition for AI products
16:30 - Vertical vs. horizontal AI startups
23:00 - Advice for AI product creators and leaders
28:15 - Tools and buying vs. building for AI products
33:30 - Key trends in AI research and development
41:00 - Closing thoughts and information on the AI Engineer World Fair Summit

Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to https://hubs.ly/Q02yV72D0

Building an AI coding assistant with Beyang Liu CTO of Sourcegraph Jun 18, 2024

Sourcegraph have built the most popular open source AI coding tool in both the dev community and the Fortune 500. I sat down with Beyang Liu their CTO and cofounder to find out how they did it.

We dive into the technical details of Cody's architecture, discussing how Sourcegraph handles the challenges of limited context windows in LLMs, why they don't use embeddings in their RAG system, and the importance of starting with the simplest approach before adding complexity.

We also touch on the future of software engineering, open-source vs closed LLM models and what areas of AI are overhyped vs underhyped

I hope you enjoy the conversation!

Chapters
- 00:00:00 - Introduction
- 00:02:30 - What is Cody, and how does it help developers?
- 00:04:15 - Challenges of building AI for large, legacy codebases
- 00:07:30 - The importance of starting with the simplest approach
- 00:11:00 - Sourcegraph's multi-layered context retrieval architecture using RAG
- 00:15:30 - Adapting to the evolving landscape of LLMs and model selection
- 00:19:00 - The future of software engineering in the age of AI
- [00:23:00 - Advice for individuals navigating the AI wave
- 00:26:00 - Predictions for the future of AI in software development
- 00:30:00 - Is AI overhyped, underhyped, or both?
- 00:33:00 - Exciting AI startups to watch
--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to https://hubs.ly/Q02yV72D0

Evaluating LLMs the Right Way: Lessons from Hex's Journey Jun 10, 2024

I recently sat down with Bryan Bischof, AI lead at Hex, to dive deep into how they evaluate LLMs to ship reliable AI agents. Hex has deployed AI assistants that can automatically generate SQL queries, transform data, and create visualizations based on natural language questions. While many teams struggle to get value from LLMs in production, Hex has cracked the code.

In this episode, Bryan shares the hard-won lessons they've learned along the way. We discuss why most teams are approaching LLM evaluation wrong and how Hex's unique framework enabled them to ship with confidence.

Bryan breaks down the key ingredients to Hex's success:
- Choosing the right tools to constrain agent behavior
- Using a reactive DAG to allow humans to course-correct agent plans
- Building granular, user-centric evaluators instead of chasing one "god metric"
- Gating releases on the metrics that matter, not just gaming a score
- Constantly scrutinizing model inputs & outputs to uncover insights

For show notes and a transcript go to:
https://hubs.ly/Q02BdzVP0
-----------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to https://hubs.ly/Q02yV72D0

Building reliable AI agents with Cai GoGwilt CTO of Ironclad Jun 03, 2024

50% of AI contracts at Ironclad’s largest customers are now automatically negotiated with the help of generative AI. Ironclad were one of the earliest adopters of LLMs, starting when the best model was still GPT-3. There’s a lot of hype around AI agents without many successful examples but Ironclad had successfully deployed them in one of the most sensitive industries imaginable.

In this episode Cai explains how they achieved this. Why they had to build their own visual programming language to make agents reliable and shares his advice for AI leaders starting to build products today.

Where to find us: https://hubs.ly/Q02z2J6v0

Welcome to the High Agency podcast! Jun 03, 2024

Welcome to very first episode of the High Agency podcast! High Agency is a new podcast from Humanloop.

Every week, I (Raza Habib) will interview leaders from companies, who have already succeeded with AI in production. We'll share their stories, lessons and playbooks to help you build with LLMs more quickly and with confidence.

To get notified of the first episodes with Cai Gogwilt or Ironclad, Bryan Bishof of Hex, Beyang Liu of Sourcegraph and Wade Foster of Zapier please subscribe on youtube, spotify or apple podcasts! (just search for High Agency podcast)
----------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models.
It enables product teams to develop LLM-based applications that are reliable and scalable.

Principally, it lets you rigorously measure and improve LLM performance during development and in production. The evalutation tools are. combiined with a collaborative workspace where engineers, PMs and subject matter experts improve prompts, tools and agents together.

By adopting Humanloop, teams save 6-8 engineering hours each week through better workflows and they feel confident that their AI is reliable.

To find out more go to https://hubs.ly/Q02z2J6v0

Our TOPPODCAST Picks

Follow Us

Stay Connected

High Agency: The Podcast for AI Builders

Related Podcasts

Links

Stay Connected