Agentic Conversations (formally mlops.community)

Latest Episodes:

Meta GenAI Infra Blog Review // Special MLOps Podcast Jul 03, 2024

Show notes

Meta GenAI Infra Blog Review // Special MLOps Podcast episode by Demetrios. // Abstract Demetrios explores Meta's innovative infrastructure for large-scale AI operations, highlighting three blog posts on training large language models, maintaining AI capacity, and building Meta's GenAI infrastructure. The discussion reveals Meta's handling of hundreds of trillions of AI model executions daily, focusing on scalability, cost efficiency, and robust networking. Key elements include the Ops planner work orchestrator, safety protocols, and checkpointing challenges in AI training. Meta's efforts in hardware design, software solutions, and networking optimize GPU performance, with innovations like a custom Linux file system and advanced networking file systems like Hammerspace. The podcast also discusses advancements in PyTorch, network technologies like Roce and Nvidia's Quantum 2 Infiniband fabric, and Meta's commitment to open-source AGI. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Building Meta’s GenAI Infrastructure blog: https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Timestamps:

[00:00] Meta handles trillions of AI model executions

[07:01] Meta creating AGI, ethical and sustainable

[08:13] Concerns about energy use in training models

[12:22] Network, hardware, and job optimization for reliability

[17:21] Highlights of Arista and Nvidia hardware architecture

[20:11] Meta's clusters optimized for efficient fabric

[24:40] Varied steps, careful checkpointing in AI training

[28:46] Meta is maintaining huge GPU clusters for AI

[29:47] AI training is faster and more demanding

[35:27] Ops planner orchestrates a million operations and reduces maintenance

[37:15] Ops planner ensures safety and well-tested changes

AI Agents for Consumers // Shaun Wei // #244 Jun 28, 2024

Show notes

Sean Wei, the CEO and co-founder of RealChar, shares his journey from working in the autonomous vehicle industry to creating an open-source voice assistant project called RealChar, which eventually evolved into Rivia, a voice AI assistant focused on managing personal phone calls.

The Future of AI and Consumer Empowerment // MLOps podcast #244 with Shaun Wei, CEO & Co-Founder of RealChar.

A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/

// Abstract

Explore the groundbreaking work RealChar is doing with its consumer application, Rivia. This discussion focuses on how Rivia leverages Generative AI and Traditional Machine Learning to handle mundane phone calls and customer service interactions, aiming to free up human time for more meaningful tasks. The product, currently in beta, embodies a forward-thinking approach to AI, where the technology offloads day-to-day burdens like scheduling appointments and making calls.

// Bio

Shaun Wei is a well-connected technology professional with a rich background in developing and analyzing artificial intelligence systems. In 2018, Shaun played a pivotal role in the advent and deployment of Google Duplex, a remarkable AI capable of handling natural conversations and performing tasks such as booking hair salon appointments and restaurant reservations via telephone. His involvement wasn't just limited to the developmental side; Shaun also uniquely positioned himself on the receiving end, gathering insights by interviewing users directly impacted by the technology. This dual perspective has enabled Shaun to grasp both the technical underpinnings and the human-centric applications of AI, making him a valuable asset in the tech industry.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.rivia.tech/

https://realchar.ai/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Shaun on LinkedIn: https://www.linkedin.com/in/shaunwei/

Timestamps:

[00:00] Shaun's preferred coffee

[00:28] Takeaways

[03:30] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:57] AI in Production: Challenges & Insights

[06:13] AI Scheduling and Assistance

[08:00] Technical Challenges in AI

[12:36 - 14:06] LatticeFlow Ad

[14:09] Handling Challenges in AI

[15:52] Learning driving and technical aspects

[19:04] Self-Driving Cars: Multimodal Integration

[23:41] Processing data with Transformers

[26:46] Real-time phone data gathering

[30:49] Real-time observability in AI

[35:09] Time to first token

[37:26] Preferred vs. Dynamic Model Selection

[40:12] Event-driven architecture basics

[42:06] Navigating challenges together

[44:02] Challenges with Inconsistent Responses

[45:40] Importance of product reliability

[47:47] Training Data and Model Performance

[50:02] Exploring AI in Customer Service

[51:34] Navigating challenges in AI

[53:15] Excited Launch Strategy Advice

[57:10] Wrap up

ML and AI as Distinct Control Systems in Heavy Industrial Settings // Richard Howes // #243 Jun 25, 2024

Show notes

Join us at our first in-person conference today, all about AI Quality: https://www.aiqualityconference.com/

ML and AI as Distinct Control Systems in Heavy Industrial Settings // MLOps podcast #243 with Richard Howes, CTO of Metaformed.

Richard Howes is a dedicated engineer who is passionate about control systems, whether it be embedded systems, industrial automation, or AI/ML in a business application.

Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract

How can we balance the need for safety, reliability, and robustness with the extreme pace of technology advancement in heavy industry? The key to unlocking the full potential of data will be to have a mixture of experts, both from an AI and human perspective, to validate anything from a simple KPI to a Generative AI Assistant guiding operators throughout their day. The data generated by heavy industries like agriculture, oil & gas, forestry, real estate, civil infrastructure, and manufacturing is underutilized and struggles to keep up with the latest and greatest - and for good reason. They provide the shelter we live and work in, the food we eat, and the energy to propel society forward. Compared to the pace of AI innovation, they move slowly, have extreme consequences for failure, and typically involve a significant workforce. During this discussion, we will outline the data ready to be utilized by ML, AI, and data products in general, as well as some considerations for creating new data products for these heavy industries. To account for complexity and uniqueness throughout the organization, it is critical to engage operational staff, ensure safety is considered from all angles, and build adaptable ETL needed to bring the data to a usable state.

// Bio

Richard Howes is a dedicated engineer who is passionate about control systems, whether it be embedded systems, industrial automation, or AI/ML in a business application. All of these systems require a robust control philosophy that outlines the system, its environment, and how the controller should function within it. Richard has a bachelor's of Electrical Engineering from the University of Victoria, where he specialized in industrial automation and embedded systems. Richard is primarily focused on the heavy industrial sectors like energy generation, oil & gas, pulp/paper, forestry, real estate, and manufacturing. He works on both physical process control and business process optimization using the control philosophy principles as a guiding star.

Richard has been working with industrial systems for over 10 years, designing, commissioning, operating, and maintaining automated systems. For the last 5 years, Richard has been investing time into the data and data science-related disciplines, bringing the physical process as close as possible to the business, taking advantage of disparate data sets throughout the organization. Now with the age of AI upon us, he is focusing on integrating this technology safely, reliably, and with distinct organizational goals and ROI.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

AWS Trainium: https://aws.amazon.com/machine-learning/trainium/

AWS Inferentia: https://aws.amazon.com/machine-learning/inferentia/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Richard on LinkedIn: https://www.linkedin.com/in/richardhowes/

Accelerating Multimodal AI // Ethan Rosenthal // #242 Jun 21, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Accelerating Multimodal AI // MLOps podcast #241 with Ethan Rosenthal, Member of Technical Staff of Runway.

Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract

We’re still trying to figure out systems and processes for training and serving “regular” machine learning models, and now we have multimodal AI to contend with! These new systems present unique challenges across the spectrum, from data management to efficient inference. I’ll talk about the similarities, differences, and challenges that I’ve seen by moving from tabular machine learning to large language models to generative video systems. I’ll also talk about the setups and tools that I have seen work best for supporting and accelerating both the research and productionization process.

// Bio

Ethan works at Runway Building Systems for media generation. Ethan's work generally straddles the boundary between research and engineering without falling too hard on either side. Prior to Runway, Ethan spent 4 years at Square. There, he led a small team of AI Engineers training large language models for Conversational AI. Before Square, Ethan freelanced, consulted, and worked at a couple of e-commerce startups. Ethan found his way into tech by way of a Physics PhD.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksWebsite: https://www.ethanrosenthal.com

Ethan's mangum opus: https://www.ethanrosenthal.com/2020/08/25/optimal-peanut-butter-and-banana-sandwiches/

Real-time Model Inference in a Video Streaming Environment // Brannon Dorsey // Coffee Sessions #98: https://youtu.be/TNO6rYwP3yg

Feature Stores for Self-Service Machine Learning: https://www.ethanrosenthal.com/2021/02/03/feature-stores-self-service/

Gen-1: The Next Step Forward for Generative AI: https://research.runwayml.com/gen1

Machine Learning: The High Interest Credit Card of Technical Debt by D. Sculley et al.: https://research.google/pubs/machine-learning-the-high-interest-credit-card-of-technical-debt/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Ethan on LinkedIn: https://bsky.app/profile/ethanrosenthal.com

Timestamps:

[00:00] Ethan's preferred coffee

[00:11] Takeaways

[02:07] Falling into LLMs

[03:16] Advanced AI Tech Capabilities

[04:40] AI-powered video editing tool

[06:56] Transition to AI: Diffusion Models

[09:09] Multimodal Feature Store breakdown

[15:33] Multimodal Feature Stores Evolution

[18:09] Benefits of Multimodal Feature Store

[25:09] Centralized Training Data Repository

[27:33] Large-scale distributed training

[32:37 - 33:39] AWS Ad

[33:45] Dealing with researchers on productionizing

[43:52] Infrastructure for Researchers and Engineers

[47:04] Generative DevOps movement

[49:21] Structuring teams

[52:06] Multimodal Feature Stores Efficiency

[54:02] Wrap up

Navigating the AI Frontier: The Power of Synthetic Data and Agent Evaluations in LLM Development // Boris Selitser // #241 Jun 18, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Navigating the AI Frontier: The Power of Synthetic Data and Agent Evaluations in LLM Development // MLOps podcast #241 with Boris Selitser, Co-Founder and CTO/CPO of Okareo.

A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/

// Abstract

Explore the evolving landscape of building LLM applications, focusing on the critical roles of synthetic data and agent evaluations. Discover how synthetic data enhances model behavior description, prototyping, testing, and fine-tuning, driving robustness in LLM applications. Learn about the latest methods for evaluating complex agent-based systems, including RAG-based evaluations, dialog-level assessments, simulated user interactions, and adversarial models. This talk delves into the specific challenges developers face and the tradeoffs involved in each evaluation approach, providing practical insights for effective AI development.

// Bio

Boris is the Co-Founder and CTO/CPO at Okareo. Okareo is a full-cycle platform for developers to evaluate and customize AI/LLM applications. Before Okareo, Boris was Director of Product at Meta/Facebook, leading teams building internal platforms and ML products. Examples include a copyright classification system across the Facebook apps and an engagement platform for over 200K developers, 500K+ creators, and 12M+ Oculus users. Boris has a bachelor’s in Computer Science from UC Berkeley.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://docs.okareo.com/blog/data_loop https://docs.okareo.com/blog/agent_eval

The Real E2E RAG Stack // Sam Bean // MLOps Podcast #217 - https://youtu.be/8uZst7pgOw0

RecSys at Spotify // Sanket Gupta // MLOps Podcast #232 - https://youtu.be/byH-ARJA4gk

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Boris on LinkedIn: https://www.linkedin.com/in/selitser/

Timestamps:

[00:00] Boris' preferred coffee

[00:37] Takeaways

[02:32] Please like, share, leave a review, and subscribe to our MLOps channels!

[02:48] Software Engineering and Data Science

[06:01] AI Transformative Potential Explained

[10:31] Prompt Injection Protection Strategies

[17:03] Agent's metrics for Jira

[24:11] Data and Metrics Evolution

[27:54] Evaluation Focus Enhances Systems

[31:22 - 32:52] LatticeFlow AD

[32:55] Custom Evaluation and Synthetic Data

[36:23] Synthetic data for expansion, evaluation, and map

[41:06] Diverse agents' personalities for readiness

[44:25] Agent functions

[46:17] Optimizing Routing Agents

[50:04] Adapting to tool output for decision-making

[52:56] Agent framework evolution

[55:41] Agent framework for delivering value

[57:03] Wrap up

How to Build Production-Ready AI Models for Manufacturing // [Exclusive] LatticeFlow Roundtable Jun 14, 2024

Show notes

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ MLOps Coffee Sessions Special episode with LatticeFlow, How to Build Production-Ready AI Models for Manufacturing, fueled by our Premium Brand Partner, LatticeFlow. Deploying AI models in manufacturing involves navigating several technical challenges such as costly data acquisition, class imbalances, data shifts, leakage, and model degradation over time. How can you uncover the causes of model failures and prevent them effectively? This discussion covers practical solutions and advanced techniques to build resilient, safe, and high-performing AI systems in the manufacturing industry. // Bio Pavol Bielik Pavol earned his PhD at ETH Zurich, specializing in machine learning, symbolic AI, synthesis, and programming languages. His groundbreaking research earned him the prestigious Facebook Fellowship in 2017, representing the sole European recipient, along with the Romberg Grant in 2016. Following his doctorate, Pavol's passion for ensuring the safety and reliability of deep learning models led to the founding of LatticeFlow. Building on a more than a decade of research, Pavol and a dynamic team of researchers at LatticeFlow developed a platform that equips companies with the tools to deliver robust and high-performance AI models, utilizing automatic diagnosis and improvement of data and models. Aniket Singh Vision Systems Engineer AI Researcher Mohan Mahadevan Mohan Mahadevan is a seasoned technology leader with 25 years of experience in building computer vision (CV) and machine learning (ML) based products. Mohan has led teams to successfully deliver real world solutions spanning hardware, software, and AI based solutions in over 20 product families across a diverse range of domains, including Semiconductors, Robotics, Fintech, and Insuretech. Mohan Mahadevan has led global teams in the development of cutting-edge technologies across a range of disciplines including computer vision, machine learning, optical and hardware architectures, system design, computational optimization and more. Jürgen Weichenberger 20+ years of advanced analytics, data science, database design, architecture, and implementation on various platforms to solve Complex Industry Problems. Industrial Analytics is the fusion of manufacturing, production, reliability, integrity, quality, sales- and market-analytics and covering 10 Industries. By combining skills and experience, we are creating the next-generation AI & ML Solutions for our clients. Leveraging a unique formula which allows us to model some of the most challenging manufacturing problems while building, scaling, and enabling the end-user to leverage the next generation data products. The Strategy & Innoation Team at Schneider is specialising on Industrial-Grade Challenges where we are applying ML & AI methods to achieve state of the art results. Personally, I am driving my team and my own education to extend the limits of AI & ML beyond the current possible. I hold more than 15 patents and I am working on new innovations. I am working with our partner eco-system to enrich our accelerators with modern ML/AI techniques and integrating robotic equipment allows me to create next generation solutions. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://latticeflow.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Timestamps: [00:00] Demetrios' Intro [00:48] Announcements [01:57] Join us at our first in-person conference on June 25 all about AI Quality! [03:39] Speakers' intros [06:00] AI ML uncommon use cases [10:14] Challenges in Implementing AI and ML in Heavy Industries [11:41] Optimizing AI use cases [18:07] Moving from PoC to Production [20:53] Hybrid AI Integration for Safety [28:28] Training AI for Defect Variability [33:18] Challenges in AI Integration [35:39] Metrics for Evaluating Success [37:27] Challenges in AI Integration [44:39] Usage of LLMs [50:34] Fine-tuning AI Models [53:20] Trust Dynamics: TML vs LLM [55:23] Wrap up

From Robotics to Recommender Systems // Miguel Fierro // #240 Jun 11, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Miguel Fierro is a Principal Data Science Manager at Microsoft and holds a PhD in robotics.

From Robotics to Recommender Systems // MLOps Podcast #240 with Miguel Fierro, Principal Data Science Manager at Microsoft.

Huge thank you to Zilliz for sponsoring this episode. Zilliz - https://zilliz.com/.

// Abstract

Miguel explains the limitations and considerations of applying ML in robotics, contrasting its use against traditional control methods that offer exactness, which ML approaches generally approximate. He discusses the integration of computer vision and machine learning in sports for player movement tracking and performance analysis, highlighting collaborations with European football clubs and the role of artificial intelligence in strategic game analysis, akin to a coach's perspective.

// Bio

Miguel Fierro is a Principal Data Science Manager at Microsoft Spain, where he helps customers solve business problems using artificial intelligence. Previously, he was CEO and founder of Samsamia Technologies, a company that created a visual search engine for fashion items, allowing users to find products using images instead of words, and founder of the Robotics Society of Universidad Carlos III, which developed different projects related to UAVs, mobile robots, humanoid robots, and 3D printers. Miguel has also worked as a robotics scientist at Universidad Carlos III of Madrid (UC3M) and King’s College London (KCL) and has collaborated with other universities like Imperial College London and IE University in Madrid. Miguel is an Electrical Engineer from UC3M, PhD in robotics by UC3M in collaboration with KCL, and graduated from MIT Sloan School of Management.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://miguelgfierro.com

GitHub: https://github.com/miguelgfierro/

RecSys at Spotify // Sanket Gupta // MLOps Podcast #232 - https://youtu.be/byH-ARJA4gk

Recommenders joins LF AI & Data as new Sandbox project: https://cloudblogs.microsoft.com/opensource/2023/10/10/recommenders-joins-lf-ai-data-as-new-sandbox-project/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Miguel on LinkedIn: https://www.linkedin.com/in/miguelgfierro/

Timestamps:

[00:00] Miguel's preferred coffee

[00:11] Takeaways

[02:25] Robotics

[10:44] Simpler solutions over ML

[15:11] Robotics and Computer Vision

[19:15] Basketball object detection

[22:43 - 23:50] Zilliz Ad

[23:51] Mr. Recommenders and Recommender systems' common patterns

[31:35] Embeddings and Feature Stores

[42:34] Experiment ROI for leadership

[47:17] Hi ROI investments

[51:13] LLMs in Recommender Systems

[54:51] Wrap up

Uber's Michelangelo: Strategic AI Overhaul and Impact // #239 Jun 07, 2024

Show notes

Uber's Michelangelo: Strategic AI Overhaul and Impact // MLOps podcast #239 with Demetrios Brinkmann.

Huge thank you to Weights & Biases for sponsoring this episode. WandB Free Courses - http://wandb.me/courses_mlops

// Abstract

Uber's Michelangelo platform has evolved significantly through three major phases, enhancing its capabilities from basic ML predictions to sophisticated uses in deep learning and generative AI. Initially, Michelangelo 1.0 faced several challenges, such as a lack of deep learning support and inadequate project tiering. To address these issues, Michelangelo 2.0 and subsequently 3.0 introduced improvements like support for Pytorch, enhanced model training, and integration of new technologies like Nvidia’s Triton and Kubernetes. The platform now includes advanced features such as a Genai gateway, robust compliance guardrails, and a system for monitoring model performance to streamline and secure AI operations at Uber.

// Bio

At the moment, Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios constantly learns and engages in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether analyzing the best paths forward, overcoming obstacles, or building Lego houses with his daughter.

// MLOps Jobs

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

From Predictive to Generative – How Michelangelo Accelerates Uber’s AI Journey blog post: https://www.uber.com/en-JP/blog/from-predictive-to-generative-ai/

Uber's Michelangelo: https://www.uber.com/en-JP/blog/michelangelo-machine-learning-platform/

The Future of Feature Stores and Platforms // Mike Del Balso & Josh Wills // MLOps Podcast # 186: https://youtu.be/p5F7v-w4EN0

Machine Learning Education at Uber // Melissa Barr & Michael Mui // MLOps Podcast #156: https://youtu.be/N6EbBUFVfO8

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Timestamps:

[00:00] Uber's Michelangelo platform evolution analyzed in podcast

[03:51 - 4:50] Weights & Biases Ad

[05:57] Uber creates Michelangelo to streamline machine learning

[07:44] Michelangelo platform's tech and flexible system

[11:49] Uber Michelangelo platform adapted for deep learning

[16:48] Uber invests in ML training for employees

[19:08] Explanation of blog content, ML quality metrics

[22:38] Michelangelo 2.0 prioritizes serving latency and Kubernetes

[26:30] GenAI gateway manages model routing and costs

[31:35] ML platform evolution, legacy systems, and maintenance

[33:22] Team debates maintaining outdated tools or moving on

[34:41] Please like, share, leave feedback, and subscribe to our MLOps channels!

[34:57] Wrap up

AWS Tranium and Inferentia // Kamran Khan and Matthew McClean // #238 Jun 04, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/

Matthew McClean is a Machine Learning Technology Leader with the leading Amazon Web Services (AWS) cloud platform. He leads the customer engineering teams at Annapurna ML, helping customers adopt AWS Trainium and Inferentia for their Gen AI workloads.

Kamran Khan, Sr Technical Business Development Manager for AWS Inferentina/Trianium at AWS. He has over a decade of experience helping customers deploy and optimize deep learning training and inference workloads using AWS Inferentia and AWS Trainium.

AWS Tranium and Inferentia // MLOps podcast #238 with Kamran Khan, BD, Annapurna ML, and Matthew McClean, Annapurna Labs Lead Solution Architecture at AWS.

Huge thank you to AWS for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract

Unlock unparalleled performance and cost savings with AWS Trainium and Inferentia! These powerful AI accelerators offer MLOps community members enhanced availability, compute elasticity, and energy efficiency. Seamlessly integrate with PyTorch, JAX, and Hugging Face, and enjoy robust support from industry leaders like W&B, Anyscale, and Outerbounds. Perfectly compatible with AWS services like Amazon SageMaker, getting started has never been easier. Elevate your AI game with AWS Trainium and Inferentia!

// Bio

Kamran Khan

Helping developers and users achieve their AI performance and cost goals for almost 2 decades.

Matthew Mc

CleanLeads the Annapurna Labs Solution Architecture and Prototyping teams, helping customers train and deploy their Generative AI models with AWS Trainium and AWS Inferentia

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksAWS Trainium: https://aws.amazon.com/machine-learning/trainium/

AWS Inferentia: https://aws.amazon.com/machine-learning/inferentia/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Kamran on LinkedIn: https://www.linkedin.com/in/kamranjk/

Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewmcclean/

Timestamps:

[00:00] Matt's & Kamran's preferred coffee

[00:53] Takeaways

[01:57] Please like, share, leave a review, and subscribe to our MLOps channels!

[02:22] AWS Trainium and Inferentia rundown

[06:04] Inferentia vs GPUs: Comparison

[11:20] Using Neuron for ML

[15:54] Should Trainium and Inferentia go together?

[18:15] ML Workflow Integration Overview

[23:10] The Ec2 instance

[24:55] Bedrock vs SageMaker

[31:16] Shifting mindset toward open source in enterprise

[35:50] Fine-tuning open-source models, reducing costs significantly

[39:43] Model deployment cost can be reduced innovatively

[43:49] Benefits of using Inferentia and Trainium

[45:03] Wrap up

Build Reliable Systems with Chaos Engineering // Benjamin Wilms // #237 May 31, 2024

Show notes

Join us at our first in-person conference on June 25, all about AI Quality: https://www.aiqualityconference.com/.

Benjamin Wilms is a developer and software architect at heart, with 20 years of experience. He fell in love with chaos engineering. Benjamin now spreads his enthusiasm and new knowledge as a speaker and author – especially in the field of chaos and resilience engineering.

Retrieval Augmented Generation // MLOps podcast #237 with Benjamin Wilms, CEO & Co-Founder of Steadybit.

Huge thank you to Amazon Web Services for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract

How to build reliable systems under unpredictable conditions with Chaos Engineering.

// Bio

Benjamin has over 20 years of experience as a developer and software architect. He fell in love with chaos engineering 7 years ago and shares his knowledge as a speaker and author. In October 2019, he founded the startup Steadybit with two friends, focusing on developers and teams embracing chaos engineering. He relaxes by mountain biking when he's not knee-deep in complex and distributed code.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://steadybit.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Benjamin on LinkedIn: https://www.linkedin.com/in/benjamin-wilms/

Timestamps: