Agentic Conversations (formally mlops.community)

Latest Episodes:

Managing Small Knowledge Graphs for Multi-agent Systems // Tom Smoker // #236 May 28, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Tom Smoker is the cofounder of an early-stage tech company empowering developers to create knowledge graphs within their RAG pipelines. Tom is a technical founder and owns the research and development of the knowledge graphs tooling for the company.

Managing Small Knowledge Graphs for Multi-agent Systems // MLOps podcast #236 with Tom Smoker, Technical Founder of whyhow.ai.

A big thank you to @latticeflow for sponsoring this episode!

LatticeFlow - https://latticeflow.ai//

/ Abstract

RAG is one of the more popular use cases for generative models, but there can be issues with repeatability and accuracy. This is especially applicable when it comes to using many agents within a pipeline, as the uncertainty propagates. For some multi-agent use cases, knowledge graphs can be used to structurally ground the agents and selectively improve the system to make it reliable end-to-end.

// Bio

Technical Founder of WhyHow.ai. Did a Master's and a PhD in CS, specializing in knowledge graphs, embeddings, and NLP. Worked as a data scientist to senior machine learning engineer at large resource companies and startups.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models: https://arxiv.org/abs/2401.01313

Understanding the type of Knowledge Graph you need — Fixed vs Dynamic Schema/Data: https://medium.com/enterprise-rag/understanding-the-type-of-knowledge-graph-you-need-fixed-vs-dynamic-schema-data-13f319b27d9e

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Tom on LinkedIn: https://www.linkedin.com/in/thomassmoker/

Timestamps:

[00:00] Tom's preferred coffee

[00:33] Takeaways

[03:04] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:23] Academic Curiosity and Knowledge Graphs

[05:07] Logician

[05:53] Knowledge graphs incorporated into RAGs

[07:53] Graphs & Vectors Integration

[10:49] "Exactly wrong"

[12:14] Data Integration for Robust Knowledge Graph

[14:53] Structured and Dynamic Data

[21:44] Scoped Knowledge Retrieval Strategies

[28:01 - 29:32] LatticeFlow Ad

[29:33] RAG Limitations and Solutions

[36:10] Working on multi-agents, questioning agent definition

[40:01] Concerns about the performance of agent information transfer

[43:45] Anticipating agent-based systems with modular processes

[52:04] Balancing risk tolerance in company operations and control

[54:11] Using AI to generate high-quality, efficient content

[01:03:50] Wrap up

Just when we Started to Solve Software Docs, AI Blew Everything Up // Dave Nunez // #235 May 27, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

David Nunez, based in Santa Barbara, CA, US, is currently a Co-Founder and Partner at Abstract Group, bringing experience from previous roles at First Round Capital, Stripe, and Slab.

Just when we Started to Solve Software Docs, AI Blew Everything Up // MLOps Podcast #235 with Dave Nunez, Partner of Abstract Group, co-hosted by Jakub Czakon.

Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/.

// Abstract

Over the previous decade, the recipe for making excellent software docs mostly converged on a set of core goals: Create high-quality, consistent content. Use different content types depending on the task. Make the docs easy to findFor AI-focused software and products, the entire developer education playbook needs to be rewritten.

// Bio

Dave lives in Santa Barbara, CA, with his wife and four kids. He started his tech career at various startups in Santa Barbara before moving to San Francisco to work at Salesforce. After Salesforce, he spent 2+ years at Uber and 5+ years at Stripe, leading internal and external developer documentation efforts.

In 2021, he co-authored Docs for Developers to help engineers become better writers. He's now a consultant, advisor, and angel investor for fast-growing startups. He typically invests in early-stage startups focusing on developer tools, productivity, and AI. He's a reading nerd, Lakers fan, and golf masochist.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://www.abstractgroup.co/

Book: docsfordevelopers.com

About Dave: https://gamma.app/docs/Dave-Nunez-about-me-002doxb23qbblme?mode=doc https://review.firstround.com/investing-in-internal-documentation-a-brick-by-brick-guide-for-startupshttps://increment.com/documentation/why-investing-in-internal-docs-is-worth-it/

Writing to Learn paper by Peter Elbow: https://peterelbow.com/pdfs/Writing_for_Learning-Not_just_Demonstrating.PDF

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Dave on LinkedIn: https://www.linkedin.com/in/djnunez/

Connect with Kuba on LinkedIn: https://www.linkedin.com/in/jakub-czakon/?locale=en_US

Timestamps:

[00:00] Dave's preferred coffee

[00:13] Introducing this episode's co-host, Kuba

[00:36] Takeaways

[02:55] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:23] Good docs, bad docs, and how to feel them

[06:51] Inviting Dev docs and checks

[10:36] Stripe's writing culture

[12:42] Engineering team writing culture

[14:15] Bottom-up tech writer change

[18:31] Strip docs cult following

[24:40] TriDocs Smart API Injection

[26:42] User research for documentation

[29:51] Design cues

[32:15] Empathy-driven docs creation

[34:28 - 35:35] Zilliz Ad

[35:36] Foundational elements in documentation

[38:23] Minimal infrastructure of information in "Read Me"

[40:18] Measuring documentation with OKRs

[43:58] Improve pages with Analytics

[47:33] Google-branded doc searches

[48:35] Time to First Action

[52:52] Dave's day in and day out and what excites him

[56:01] Exciting internal documentation

[59:55] Wrap up

Open Standards Make MLOps Easier and Silos Harder // Cody Peterson // #234 May 21, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Cody Peterson has diverse work experience in the field of product management and engineering. Cody is currently working as a Technical Product Manager at Voltron Data, starting in May 2023. Previously, they worked as a Product Manager at dbt Labs from July 2022 to March 2023.

MLOps podcast #234 with Cody Peterson, Senior Technical Product Manager at Voltron Data | Ibis project // Open Standards Make MLOps Easier and Silos Harder.

Huge thank you to Weights & Biases for sponsoring this episode.

WandB Free Courses - http://wandb.me/courses_mlops

// Abstract

MLOps is fundamentally a discipline of people working together on a system with data and machine learning models. These systems are already built on open standards we may not notice -- Linux, git, scikit-learn, etc. -- but are increasingly hitting walls with respect to the size and velocity of data.

Pandas, for instance, is the tool of choice for many Python data scientists -- but its scalability is a known issue. Many tools make the assumption that data fits in memory, but most organizations have data that will never fit in a laptop. What approaches can we take?

One emerging approach with the Ibis project (created by the creator of pandas, Wes McKinney) is to leverage existing "big" data systems to do the heavy lifting on a lightweight Python data frame interface. Alongside other open source standards like Apache Arrow, this can allow data systems to communicate with each other and users of these systems to learn a single data frame API that works across any of them.

Open standards like Apache Arrow, Ibis, and more in the MLOps tech stack enable freedom for composable data systems, where components can be swapped out, allowing engineers to use the right tool for the job to be done. It also helps avoid vendor lock-in and keeps costs low.

// Bio

Cody is a Senior Technical Product Manager at Voltron Data, a next-generation data systems builder that recently launched an accelerator-native GPU query engine for petabyte-scale ETL called Theseus. While Theseus is proprietary, Voltron Data takes an open periphery approach -- it is built on an interface through open standards like Apache Arrow, Substrait, and Ibis. Cody focuses on the Ibis project, a portable Python dataframe library that aims to be the standard Python interface for any data system, including Theseus and over 20other backends.

Prior to Voltron Data, Cody was a product manager at dbt Labs, focusing on the open source dbt Core and launching Python models (note: models is a confusing term here). Later, he led the Cloud Runtime team and drastically improved the efficiency of engineering execution and product outcomes.

Cody started his career as a Product Manager at Microsoft, working on Azure ML. He spent about 2 years on the dedicated MLOps product team and 2 more years on various teams across the ML lifecycle, including data, training, and inferencing.

He is now passionate about using open source standards to break down the silos and challenges facing real-world engineering teams, where engineering increasingly involves data and machine learning.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Ibis Project: https://ibis-project.org

Apache Arrow and the “10 Things I Hate About pandas”: https://wesmckinney.com/blog/apache-arrow-pandas-internals/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Cody on LinkedIn: https://linkedin.com/in/codydkdc

Retrieval Augmented Generation May 17, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Syed Asad is an Innovator, Generative AI & Machine Learning Engineer, and a Champion for Ethical AI

MLOps podcast #233 with Syed Asad, Lead AI/ML Engineer at KiwiTech // Retrieval Augmented Generation.

A big thank you to @ for sponsoring this episode! AWS -

// Abstract

Everything and anything around RAG.

// Bio

Currently Exploring New Horizons: Syed is diving deep into the exciting world of Semantic Vector Searches and Vector Databases. These innovative technologies are reshaping how we interact with and interpret vast data landscapes, opening new avenues for discovery and innovation. Specializing in Retrieval Augmented Generation (RAG): Syed's current focus also includes mastering Retrieval Augmented Generation Techniques (RAGs). This cutting-edge approach combines the power of information retrieval with generative models, setting new benchmarks in AI's capability and application.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://sanketgupta.substack.com/

Our paper on this topic, "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584

Sanket's blogs on Medium in the past: https://medium.com/@sanket107

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Syed on LinkedIn: https://www.linkedin.com/in/syed-asad-76815246/

Timestamps:

[00:00] Syed's preferred coffee

[00:31] Takeaways

[03:17] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:37] A production issue

[07:37] CSV file handling risks

[09:42] Embedding models are not suitable

[11:22] Inference layer experiments and use cases

[14:00] AWS service handling the issue

[17:35] Salad testing and insights

[22:12] OpenAI vs Customization

[24:30] Difference between Olama and VLLM

[27:16] Fine-tuning of small LLMs

[29:51] Evaluation framework

[32:04] MLOps for efficient ML

[37:12] Determining the pricing of tools

[39:35] Manage Dependency Risk

[40:27] Get in touch with Syed on LinkedIn

[41:46] ML Engineers are now all AI Engineers

[43:01] The hard framework

[43:53] Wrap up

RecSys at Spotify // Sanket Gupta // #232 May 16, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Sanket works as a Senior Machine Learning Engineer at Spotify, working on building end-to-end audio recommender systems. Models built by his team are used across Spotify in many different products, including Discover Weekly and Autoplay.

MLOps podcast #232 with Sanket Gupta, Senior Machine Learning Engineer at Spotify // RecSys at Spotify.

A big thank you to LatticeFlow for sponsoring this episode!

// Abstract

LLMs with foundational embeddings have changed the way we approach AI today. Instead of re-training models from scratch end-to-end, we instead rely on fine-tuning existing foundation models to perform transfer learning. Is there a similar approach we can take with recommender systems? In this episode, we can talk about: a) how Spotify builds and maintains large-scale recommender systems, b) how foundational user and item embeddings can enable transfer learning across multiple products, c) how we evaluate this system, and d) MLOps challenges with these systems

// Bio

Sanket works as a Senior Machine Learning Engineer on a team at Spotify building production-grade recommender systems. Models built by my team are being used in Autoplay, Daily Mix, Discover Weekly, etc.

Currently, my passion is how to build systems to understand user taste - how do we balance long-term and short-term understanding of users to enable a great personalized experience?

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksWebsite: https://sanketgupta.substack.com/

Our paper on this topic, "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584

Sanket's blogs on Medium in the past: https://medium.com/@sanket107

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sanket on LinkedIn: www.linkedin.com/in/sanketgupta107

Timestamps:

[00:00] Sanket's preferred coffee

[00:37] Takeaways[02:30] RecSys are RAGs

[06:22] Evaluating RecSys parallel to RAGs

[07:13] Music RecSys Optimization

[09:46] Dealing with cold start problems

[12:18] Quantity of models in the recommender systems

[13:09] Radio models

[16:24] Evaluation system

[20:25] Infrastructure support

[21:25] Transfer learning

[23:53] Vector database features

[25:31] Listening History Balance

[26:35 - 28:06] LatticeFlow Ad

[28:07] The beauty of embeddings

[30:13] Shift to real-time recommendation

[34:05] Vector Database Architecture Options

[35:30] Embeddings drive personalized

[40:16] Feature Stores vs Vector Databases

[42:33] Spotify product integration strategy

[45:38] Staying up to date with new features

[47:53] Speed vs Relevance metrics

[49:40] Wrap up

From A Coding Startup to AI Development in the Enterprise // Ryan Carson // #231 May 10, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Ryan Carson. CEO, Founder for 20 years, built and sold 3 startups, helping build a global community of AI devs with Intel.

MLOps podcast #231 with Ryan Carson, Senior AI Dev Community Lead at Intel

Huge thank you to Zilliz for sponsoring this episode.

// Abstract

Ryan shares his professional journey, tracing his transition from building Treehouse to joining Intel. The conversation evolves into a deep dive into Carson's aspiration to democratize access to AI development. Furthermore, he expounds on the exciting prospects of new technology like Gaudi three, a new ASIC for AI workloads. Ryan emphasizes the need for driving competition in computing to lower prices and increase access, underlining the importance of associating individual work with company-based OKRs or KPIs. There is also a reflection on the essentiality of forging quality relationships in professional settings and aligning work with top-level OKRs. Discussion on the potential benefits of AI in constructing and maintaining professional interactions is explored. Touching upon practical applications of AI, they also delve into smaller projects, the possibility of one-person companies, and the role of AI for daily interactions. The episode concludes with an expression of optimism about technological advances shaping the future and an appreciation for the enlightening conversation.

// Bio

Ryan has been a founder, entrepreneur, and CEO for 20 years, successfully building, scaling, and selling three companies. He's passionate about empowering people to become developers and then connecting them together in a global community.

After earning a degree in Computer Science in Colorado, Ryan moved to the UK and worked as a web developer. He then organized global tech conferences, hosting thousands of attendees and influential speakers such as Mark Zuckerberg, the founders of Android, Instagram, and Twitter, among others. His company also produced Twitter’s and Stack Overflow’s developer conferences.

Following that, Ryan started an online Computer Science school. Under his leadership, the team grew to over 100 employees, educating more than 1,000,000 students. During this period, he secured $23 million in venture capital and earned recognition as Entrepreneur of the Year.

Over the last two years, Ryan dove deep into AI and LLMs. He built an educational proof-of-concept called maple.coach, which focuses on teaching Sales. The platform is built using technologies like Next.js, TypeScript, GPT-4, and Vercel.

Outside of work, Ryan shares his life with his wife of 20 years and their two teenagers in Connecticut. They enjoy spending their free time sailing and taking walks with their Sheltie, Brinkley.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: ryancarson.com

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryancarson/

FedML Nexus AI: Your Generative AI Platform at Scale // Salman Avestimehr // #230 May 07, 2024

Show notes

Salman Avestimehr is a Dean's Professor, the inaugural director of the USC-Amazon Center for Secure and Trusted Machine Learning (Trusted AI), and director of the Information Theory and Machine Learning (vITAL) research lab. He is also the CEO and co-founder of FedML.

MLOps podcast #230 with Salman Avestimehr, CEO & Founder of FedML, FedML Nexus AI: Your Generative AI Platform at Scale.

A big thank you to FEDML for sponsoring this episode!

// Abstract

FedML is your generative AI platform at scale to enable developers and enterprises to build and commercialize their own generative AI applications easily, scalably, and economically. Its flagship product, FedML Nexus AI, provides unique features in enterprise AI platforms, model deployment, model serving, AI agent APIs, launching training/Inference jobs on serverless/decentralized GPU cloud, experimental tracking for distributed training, federated learning, security, and privacy.

// Bio

Salman is a professor, the inaugural director of the USC-Amazon Center for Secure and Trusted Machine Learning (Trusted AI), and the director of the Information Theory and Machine Learning (vITAL) research lab at the Electrical and Computer Engineering Department and Computer Science Department of the University of Southern California. Salman is also the co-founder and CEO of FedML. He received his Ph.D. in Electrical Engineering and Computer Sciences from UC Berkeley in 2008. Salman does research in the areas of information theory, decentralized and federated machine learning, secure and privacy-preserving learning, and computing.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.avestimehr.com/https://fedml.ai/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Salman on LinkedIn: https://www.linkedin.com/company/fedml/

Timestamps:

[00:00] AI Quality: First in-person conference on June 25

[01:28] Salman's preferred coffee

[01:49] Takeaways

[03:33] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:53] Challenges that inspired Salman's work

[06:20] Controlled ownership

[08:11] Dealing with data leakage and privacy problems

[10:45] In-house ML Model Deployment

[13:36] FEDML: Comprehensive Model Deployment

[17:27] Integrating FEDML with Kubernetes

[19:46] AI Evaluation Trends

[24:37] Enhancing NLP with ML

[25:48] FEDML: Canary, A/B, Confidence

[29:36] FEDML customers

[33:21] On-premise platform for secure data management

[37:16] Future prediction: data is crucial for better applications

[38:18] Maturity in evaluating and improving steps

[41:38] Focus on ownership

[45:12] Benefits of smaller models for specific use cases

[48:57] Verify sensitive tasks, trust quick, important mobile content creation

[51:50] Wrap up

What is AI Quality? // Mohamed Elgendy // #228 May 03, 2024

Show notes

Mohamed Elgendy is the Co-Founder & CEO at Kolena. Additionally, Mohamed Elgendy has had 1 past job as the Director of Product and Engineering at Synapse Technology Corporation.

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

MLOps podcast #228 with Mohamed Elgendy, Co-founder & CEO of Kolena Inc., What is AI Quality?

// Abstract

Delve into the multifaceted concept of AI Quality. Demetrios and Mo explore the idea that AI quality is dependent on the specific domain, equitable to the difference in desired qualities between a $1 pen and a $100 pen. Mo underscores the performance of a product being in sync with its intended functionality and the absence of unknown risks as the pillars of AI Quality. They emphasize the need for comprehensive quality checks and adaptability of standards to differing product traits. Issues affecting edge deployments, like latency, are also highlighted. A deep dive into the formation of gold standards for AI, the nuanced necessities for various use cases, and the paramount need for collaboration among AI builders, regulators, and infrastructure firms form the core of the discussion. Elgendy brings to light their ambitious AI Quality Conference, aiming to set tangible, effective, but innovation-friendly Quality standards for AI. The dialogue also accentuates the urgent need for diversification and representation in the tech industry, the variability of standards and regulations, and the pivotal role of testing in AI and machine learning. The episode concludes with an articulate portrayal of how enhanced testing can streamline the entire process of machine learning.

// Bio

Mohamed is the Co-founder & CEO of Kolena and the author of the book “Deep Learning for Vision Systems”. Previously, he built and managed AI/ML organizations at Amazon, Twilio, Rakuten, and Synapse. Mohamed regularly speaks at AI conferences like Amazon's DevCon, O'Reilly's AI conference, and Google's I/O.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: www.kolena.io

Deep Learning for Vision Systems book: https://www.amazon.com/Learning-Vision-Systems-Mohamed-Elgendy/dp/1617296198/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Mo on LinkedIn: https://www.linkedin.com/in/moelgendy/

Timestamps:

[00:00] Mo's preferred coffee

[00:07] Takeaways

[02:52] See you all in San Francisco on June 25!

[03:04] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:22] AI Quality in Mo's eyes

[08:36] Quality Standards for Software

[14:11] Common Chatbot Functionality

[19:20] The Birth of Innovation

[24:27] Transforming Insights into Standards

[30:27] Testing: One step to quality

[34:58] Two different data points to be harmonized

[37:29] Model cards

[39:12] Test Coverage Democratizes Collaboration

[42:55] Representation matters

[44:50] Wrap up

Handling Multi-Terabyte LLM Checkpoints // Simon Karasik // #228 Apr 30, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com

Simon Karasik⁠ is a proactive and curious ML Engineer with 5 years of experience. Developed & deployed ML models at WEB and Big scale for Ads and Tax.Huge thank you to Nebius AI for sponsoring this episode. Nebius AI - https://nebius.ai/

MLOps podcast #228 with Simon Karasik, Machine Learning Engineer at Nebius AI, Handling Multi-Terabyte LLM Checkpoints.

// Abstract

The talk provides a gentle introduction to the topic of LLM checkpointing: why is it hard, and how big are the checkpoints? It covers various tips and tricks for saving and loading multi-terabyte checkpoints, as well as the selection of cloud storage options for checkpointing.

// Bio

Full-stack Machine Learning Engineer, currently working on infrastructure for LLM training, with previous experience in ML for Ads, Speech, and Tax.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Simon on LinkedIn: https://www.linkedin.com/in/simon-karasik/

Timestamps:

[00:00] Simon's preferred beverage

[01:23] Takeaways

[04:22] Simon's tech background

[08:42] Zombie models garbage collection

[10:52] The road to LLMs

[15:09] Trained models Simon worked on

[16:26] LLM Checkpoints

[20:36] Confidence in AI Training

[22:07] Different Checkpoints

[25:06] Checkpoint parts

[29:05] Slurm vs Kubernetes

[30:43] Storage choices lessons

[36:02] Paramount components for setup

[37:13] Argo workflows

[39:49] Kubernetes node troubleshooting

[42:35] Cloud virtual machines have pre-installed mentoring

[45:41] Fine-tuning

[48:16] Storage, networking, and complexity in network design

[50:56] Start simple before advanced; consider model needs.

[53:58] Join us at our first in-person conference on June 25, all about AI Quality

Leading Enterprise Data Teams // Sol Rashidi // #227 Apr 26, 2024

Show notes

Sol Rashidi is an esteemed executive, leader, and influencer within the AI, Data, and Technology space. Having helped IBM launch Watson in 2011 as one of the earliest world applications of Artificial Intelligence, Sol has pioneered some of the early advancements in space.

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Huge thank you to @WeightsBiases for sponsoring this episode. WandB Free Courses - http://wandb.me/courses_mlops

MLOps podcast #227 with Sol Rashidi, CEO & Co-Founder of ExecutiveAI, Leading Enterprise Data Teams.

//Abstract

In the dynamic landscape of MLOps and data leadership, Sol shares invaluable insights on building successful teams and driving impactful projects. In this podcast episode, Sol delves into the importance of prioritizing relationships, introduces a pragmatic "Wrong Use Cases Formula" to streamline project prioritization, and emphasizes the critical role of effective communication in data leadership. Her wealth of experience and practical advice provides a roadmap for navigating the complexities of MLOps and leading data-driven initiatives to success.

// Bio

With eight (8) patents granted, 21 filed, and received awards that include:

"Top 100 AI People" 2023

"The Top 75 Innovators of 2023"

"Top 65 Most Influential Women in 2023"

"Forbes AI Maverick of the 21st Century" 2022

“Top 10 Global Women in AI & Data”, 2023

"Top AI 100 Award", 2023

“50 Most Powerful Women in Tech”, 2022

“Global 100 Power List” - 2021, 2022, 2023

“Top 20 CDOs Globally” - 2022

"Chief Analytics Officer of the Year" - 2022

"Isomer Innovators of the Year" - 2021, 2022, 2023

"Top 100 Innovators in Data & Analytics” - 2020, 2021, 2022, 2023

"Top 100 Women in Business" - 2022

Sol is an energetic business executive and a goal-oriented technologist, skilled at coupling her technical acumen with storytelling abilities to articulate business value with both startups and Fortune 100s who are leaning into data, AI, and technology as a competitive advantage while wanting to preserve the legacy upon which they were founded. Sol has served as a C-Suite member across several Fortune 100 & Fortune 500 companies, including:

Chief Analytics Officer - Estee LauderChief Data & Analytics Officer - Merck Pharmaceuticals

EVP, Chief Data Officer - Sony MusicChief Data & AI Officer - Royal Caribbean Cruise Lines

Sr. Partner leading the Digital & Innovation Practice- Ernst & Young

Partner leading Watson Go-To-Market & Commercialization - IBM

Sol now serves as the CEO of ExecutiveAI LLC. A company dedicated to democratizing Artificial Intelligence for Humanity, and is considered an outstanding and influential business leader who is influencing the space, traveling the world as a keynote speaker, and serving as the bridge between established Gen

1.0 markets and those evolving into 4.0.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Sol's Book will be out on April 30, 2024. Your AI Survival Guide: Scraped Knees, Bruised Elbows, and Lessons Learned from Real-World AI Deployments: https://www.amazon.com/Your-Survival-Guide-Real-World-Deployments/dp/1394272634?ref_=ast_author_mpb

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sol on LinkedIn: https://www.linkedin.com/in/sol-rashidi-a672291/