Agentic Conversations (formally mlops.community)

Latest Episodes:

Building in Production Human-centred GenAI Solutions // Mohamed Abusaid & Mara Pometti// #177 Sep 05, 2024

Show notes

MLOps Coffee Sessions #177 with Mohamed Abusaid and Mara Pometti, Building in Production Human-centred GenAI Solutions sponsored by QuantumBlack, AI by McKinsey. // Abstract Trust is paramount in the adoption of new technologies, especially in the realm of education. Mohamed and Mara shed light on the importance of AI governance programs and establishing AI governance boards to ensure safe and ethical use of technology while managing associated risks. They discuss the impact on customers, potential risks, and mitigation strategies that organizations must consider to protect their brand reputation and comply with regulations. // Bio Mara Pometti Mara is an Associate Design Director at McKinsey & Company, where she helps organisations drive AI adoption through human-centered methods. She defines herself as a data-savvy humanist. Her practice spans across AI, data journalism, and design with the overarching objective of finding the strategic intersection between AI models and human intents to implement responsible AI systems that move organisations forward. Previously, she led the AI Strategy practice at IBM, where she also developed the company’s first-ever data storytelling program. Yet, by background, she is a data journalist. She worked as a data journalist for agencies and newsrooms like Aljazeera. Mara lectured at many universities about how to humanize AI, including the London School of Economics. Her books and writing explore how to weave a humanistic approach to AI development. Mohamed Abusaid Am Mohamed, a tech enthusiast, hacker, avid traveler, and foodie all rolled into one individual. Built his first website when he was 9 and fell in love with computers and the internet ever since. Graduated with computer science from university although dabbled in electrical, electronic, and network engineering before that. When he's not reading up on the latest tech conversations and products on Hacker News, Mohamed spends his time traveling to new destinations and exploring their cuisine and culture. Mohamed works with different companies helping them tackle challenges in developing, deploying, and scaling their analytics to reach its potential. Some topics he's enthusiastic about include MLOps, DataOps, GenerativeAI, Product thinking, and building cross-functional teams to deliver user-first products. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links QuantumBlack, AI by McKinsey: https://www.mckinsey.com/capabilities/quantumblack/how-we-help-clients --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/ Connect with Mara on LinkedIn: https://www.linkedin.com/in/mara-pometti Connect with Mohamed on LinkedIn: https://www.linkedin.com/in/mabusaid/

Visualize - Bringing Structure to Unstructured Data // Markus Stoll // #258 Sep 03, 2024

Show notes

Markus Stoll is the Co-Founder of Renumics and the developer behind the open-source interactive ML dataset exploration tool, Spotlight. He shares insights on:

AI in Engineering and Manufacturing
Interactive ML Data Visualization
ML Data Exploration

Follow Markus for hands-on articles about leveraging ML while keeping a strong focus on data.

Visualize - Bringing Structure to Unstructured Data // MLOps Podcast #258 with Markus Stoll, CTO of Renumics.

A huge thank you to SAS for their generous support!

// Abstract

This talk is about how data visualization and embeddings can support you in understanding your machine-learning data. We explore methods to structure and visualize unstructured data like text, images, and audio for applications ranging from classification and detection to Retrieval-Augmented Generation. By using tools and techniques like UMAP to reduce data dimensions and visualization tools like Renumics Spotlight, we aim to make data analysis for ML easier. Whether you're dealing with interpretable features, metadata, or embeddings, we'll show you how to use them all together to uncover hidden patterns in multimodal data, evaluate the model performance for data subgroups, and find failure modes of your ML models.

// Bio

Markus Stoll began his career in the industry at Siemens Healthineers, developing software for the Heavy Ion Therapy Center in Heidelberg. He learned about software quality while developing a treatment machine weighing over 600 tons. He earned a Ph.D., focusing on combining biomechanical models with statistical models, through which he learned how challenging it is to bridge the gap between research and practical application in the healthcare domain. Since co-founding Renumics, he has been active in the field of AI for Engineering, e.g., AI for Computer Aided Engineering (CAE), implementing projects, contributing to their open-source library for data exploration for ML datasets (Renumics Spotlight), and writing articles about data visualization.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://renumics.com/

MLSecOps Community: https://community.mlsecops.com/

Blogs: https://towardsdatascience.com/visualize-your-rag-data-evaluate-your-retrieval-augmented-generation-system-with-ragas-fc2486308557 : https://medium.com/itnext/how-to-explore-and-visualize-ml-data-for-object-detection-in-images-88e074f46361

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Markus on LinkedIn: https://www.linkedin.com/in/markus-stoll-b39a42138/

Timestamps:

[00:00] Markus' preferred coffee

[00:15] Takeaways

[01:41] Please like, share, leave a review, and subscribe to our MLOps channels!

[01:50] Register for the Data Engineering for AI/ML Conference now!

[02:27] Current focus and updates

[04:43] 3D Embeddings Visualization Explained

[07:07] Question Embeddings vs Retrieval

[08:24] Using heat maps effectively

[10:30] User insights visualization RAG

[16:59] 3D Crash Simulation Analysis

[20:33] Simulation purpose clarification

[22:34] Evaluating test data use cases

[24:22] Real-world car testing

[29:48] Identifying data issues early

[33:33] Multimodal data integration

[37:42] Custom vs Fine-tuned models

[39:45] Data processing challenges

[45:58] Use case-driven MVP

[48:26 - 50:08] SAS Ad

[50:09] Wrap up

AI Testing Highlights // Special MLOps Podcast Episode Sep 01, 2024

Show notes

MLOps for GenAI Applications // Special MLOps Podcast episode with Demetrios Brinkmann, Chief Happiness Engineer at MLOps Community.

// Abstract

Demetrios explores common themes in ML model testing with insights from Erica Greene (Yahoo News), Matar Haller (ActiveFence), Mohamed Elgendy (Kolena), and Catherine Nelson (Freelance Data Scientist). They discuss tiered test cases, functional testing for hate speech, differences between AI and traditional software testing, and the complexities of evaluating LLMs. Demetrios wraps up by inviting feedback and promoting an upcoming virtual conference on data engineering for AI and ML.

// Bio

At the moment, Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps Community Podcasts. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Balancing Speed and Safety // Panel // AIQCON - https://youtu.be/c81puRgu3Kw

AI For Good - Detecting Harmful Content at Scale // Matar Haller // MLOps Podcast #246 - https://youtu.be/wLKlZ6yHg1k

What is AI Quality? // Mohamed Elgendy // MLOps Podcast #229 - https://youtu.be/-Jdmq4DiOew

All Data Scientists Should Learn Software Engineering Principles // Catherine Nelson // Podcast #245 - https://youtu.be/yP6Eyny7p20

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Timestamps:

[00:00] Exploring common themes in MLOps community

[00:49] Common patterns about model output and testing

[01:34] Tiered test case strategy

[03:05] Functional testing for models

[05:24] Testing coverage and quality

[07:47] Evaluating LLMs challenges

[08:35] Please like, share, leave a review, and subscribe to our MLOps channels!

MLSecOps is Fundamental to Robust AISPM // Sean Morgan // #257 Aug 30, 2024

Show notes

Sean Morgan is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons. Learn more about the platform for end-to-end AI Security at https://protectai.com/.

MLSecOps is Fundamental to Robust AI Security Posture Management (AISPM) // MLOps Podcast #257 with Sean Morgan, Chief Architect at Protect AI.

// Abstract

MLSecOps, which is the practice of integrating security practices into the AIML lifecycle (think infusing MLOps with DevSecOps practices), is a critical part of any team’s AI Security Posture Management. In this talk, we’ll discuss how to threat model realistic AIML security risks, how you can measure your organization’s AI Security Posture, and most importantly, how you can improve that security posture through the use of MLSecOps.

// Bio

Sean Morgan is the Chief Architect at Protect AI. In prior roles, he's led production AIML deployments in the semiconductor industry, evaluated adversarial machine learning defenses for DARPA research programs, and most recently scaled customers on interactive machine learning solutions at AWS. In his free time, Sean is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Sean's GitHub: https://github.com/seanpmorgan

MLSecOps Community: https://community.mlsecops.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sean on LinkedIn: https://www.linkedin.com/in/seanmorgan/

Timestamps:

[00:00] Sean's preferred coffee

[00:10] Takeaways

[01:39] Register for the Data Engineering for AI/ML Conference now!

[02:21] KubeCon Paris: Emphasis on security and AI

[05:00] Concern about malicious data during the training process

[09:29] Model builders, security, pulling foundational models, nuances

[12:13] Hugging Face research on security issues

[15:00] Inference servers exposed; potential for attack

[19:45] Balancing ML and security processes for ease

[23:23] Model artifact security in enterprise machine learning

[25:04] Scanning models and datasets for vulnerabilities

[29:23] Ray's user interface vulnerabilities lead to attacks

[32:07] ML Flow vulnerabilities present significant server risks

[36:04] Data ops essential for machine learning security

[37:32] Prioritized security in model and data deployment

[40:46] Automated scanning tool for improved antivirus protection

[42:00] Wrap up

MLOps for GenAI Applications // Harcharan Kabbay // #256 Aug 27, 2024

Show notes

Harcharan Kabbay is a Data Scientist & AI/ML Engineer with Expertise in MLOps, Kubernetes, and DevOps, Driving End-to-End Automation and Transforming Data into Actionable Insights.

MLOps for GenAI Applications // MLOps Podcast #256 with Harcharan Kabbay, Lead Machine Learning Engineer at World Wide Technology.

// Abstract

The discussion begins with a brief overview of the Retrieval-Augmented Generation (RAG) framework, highlighting its significance in enhancing AI capabilities by combining retrieval mechanisms with generative models. The podcast further explores the integration of MLOps, focusing on best practices for embedding the RAG framework into a CI/CD pipeline. This includes ensuring robust monitoring, effective version control, and automated deployment processes that maintain the agility and efficiency of AI applications. A significant portion of the conversation is dedicated to the importance of automation in platform provisioning, emphasizing tools like Terraform. The discussion extends to application design, covering essential elements such as key vaults, configurations, and strategies for seamless promotion across different environments (development, testing, and production). We'll also address how to enhance the security posture of applications through network firewalls, key rotation, and other measures. Let's talk about the power of Kubernetes and related tools to aid a good application design. The podcast highlights the principles of good application design, including proper observability and eliminating single points of failure. I would share strategies to reduce development time by creating templates for GitHub repositories by application types to be reused, also templates for pull requests, thereby minimizing human errors and streamlining the development process.

// Bio

Harcharan is an AI and machine learning expert with a robust background in Kubernetes, DevOps, and automation. He specializes in MLOps, facilitating the adoption of industry best practices and platform provisioning automation. With extensive experience in developing and optimizing ML and data engineering pipelines, Harcharan excels at integrating RAG-based applications into production environments. His expertise in building scalable, automated AI systems has empowered the organization to enhance decision-making and problem-solving capabilities through advanced machine-learning techniques.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Harcharan's Medium - https://medium.com/@harcharan-kabbay

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Harcharan on LinkedIn: https://www.linkedin.com/in/harcharankabbay/locale=en_US

Timestamps:

[00:00] Harcharan's preferred coffee

[00:21] Takeaways

[01:03] Against local LLMs

[02:11] Creating bad habits

[02:42] Operationalizing RAG from the CI/CD perspective

[09:39] Kubernetes vs LLM Deployment

[12:12] Tool preferences in ML

[14:39] DevOps perspective of deployment

[17:44] Terraform Licensing Controversy

[22:47] PR Review Template Guidance

[27:32] People process tech order

[29:22] Register for the Data Engineering for AI/ML Conference now!

[30:00] ML monitoring strategies explained

[39:39] Serverless vs Overprovisioning

[44:43] Model SLA's and Monitoring

[51:04] LLM to App transition

[52:42] Ensuring Robust Architecture

[58:53] Chaos engineering in ML

[1:04:43] Wrap up

BigQuery Feature Store // Nicolas Mauti // #255 Aug 23, 2024

Show notes

Nicolas Mauti is an MLOps Engineer from Lyon (France), Working at Malt.

BigQuery Feature Store // MLOps Podcast #255 with Nicolas Mauti, Lead MLOps at Malt.

// Abstract

Need a feature store for your AI/ML applications but overwhelmed by the multitude of options? Think again. In this talk, Nicolas shares how they solved this issue at Malt by leveraging the tools they already had in place. From ingestion to training, Nicolas provides insights on how to transform BigQuery into an effective feature management system.

We cover how Nicolas' team designed their feature tables and addressed challenges such as monitoring, alerting, data quality, point-in-time lookups, and backfilling. If you’re looking for a simpler way to manage your features without the overhead of additional software, this talk is for you. Discover how BigQuery can handle it all!

// Bio

Nicolas Mauti is the go-to guy for all things related to MLOps at Malt. With a knack for turning complex problems into streamlined solutions and over a decade of experience in code, data, and ops, he is a driving force in developing and deploying machine learning models that actually work in production. When he's not busy optimizing AI workflows, you can find him sharing his knowledge at the university. Whether it's cracking a tough data challenge or cracking a joke, Nicolas knows how to keep things interesting.

// MLOps Jobs board jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksNicolas' Medium - https://medium.com/@nmauti

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Nicolas on LinkedIn: https://www.linkedin.com/in/nicolasmauti/?locale=en_US

Timestamps:

[00:00] Nicolas' preferred beverage

[00:35] Takeaways

[02:25] Please like, share, leave a review, and subscribe to our MLOps channels!

[02:57] BigQuery end goal

[05:00] BigQuery pain points

[10:14] BigQuery vs Feature Stores

[12:54] Freelancing Rate Matching issues

[16:43] Post-implementation pain points

[19:39] Feature Request Process

[20:45] Feature Naming Consistency

[23:42] Feature Usage Analysis

[26:59] Anomaly detection in data

[28:25] Continuous Model Retraining Process

[30:26] Model misbehavior detection

[33:01] Handling model latency issues

[36:28] Accuracy vs The Business

[38:59] BigQuery cist-benefit analysis

[42:06] Feature stores cost savings

[44:09] When not to use BigQuery

[46:20] Real-time vs Batch Processing

[49:11] Register for the Data Engineering for AI/ML Conference now!

[50:14] Wrap up

Design and Development Principles for LLMOps // Andy McMahon // #254 Aug 20, 2024

Show notes

Design and Development Principles for LLMOps // MLOps Podcast #254 with Andy McMahon, Director - Principal AI Engineer at Barclays Bank.

A huge thank you to SAS for their generous support!

// Abstract

As we move from MLOps to LLMOps, we need to double down on some fundamental software engineering practices, as well as augment and add to these with some new techniques. In this case, let's talk about this!

// Bio

Andy is a Principal AI Engineer, working in the new AI Center of Excellence at Barclays Bank. Previously, he was Head of MLOps for NatWest Group, where he led their MLOps Center of Excellence and helped build out their MLOps platform and processes across the bank. Andy is also the author of Machine Learning Engineering with Python, a hands-on technical book published by Packt.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksAndy's book - https://packt.link/w3JKL

Andy's Medium - https://medium.com/@andrewpmcmahon629

SAS: https://www.sas.com/en_us/home.html

SAS® Decision Builder: https://www.sas.com/en_us/offers/23q4/microsoft-fabric.html

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

Harnessing MLOps in Finance // Michelle Marie Conway // MLOps Podcast Coffee #174: https://youtu.be/nIEld_Q6L-0

The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, by Eric Ries: https://www.amazon.co.jp/-/en/Eric-Ries/dp/0307887898

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Andy on LinkedIn: https://www.linkedin.com/in/andrew-p-mcmahon/

Timestamps:

[00:00] Andy's preferred coffee

[00:09] Takeaways

[02:04] Andy's book is an Oxford curriculum

[06:13] Register for the Data Engineering for AI/ML Conference now!

[07:04] The Life Cycle of AI Executives Course

[09:55] MLOps as a term

[11:53] Tooling vs Process Culture

[15:01] Open source benefits

[17:15] End goal flexibility

[20:06] Hybrid Cloud Strategy Overview

[21:11] ROI for tool upgrades

[25:41] Long-term projects comparison

[29:02 - 30:48] SAS Ad

[30:49] AI and ML Integration

[35:40] Hybrid AI Integration Insights

[42:18] Tech trends vs Practicality

[44:39] Gen AI Tooling Debate

[51:57] Vanity metrics overview

[55:22] Tech business alignment strategy

[58:45] Aligning teams for ROI

[1:01:35] Communication mission effectively

[1:03:45] Enablement metrics

[1:06:38] Prioritizing use cases

[1:09:47] Wrap up

Data Quality = Quality AI // AIQCON Panel Aug 16, 2024

Show notes

// Abstract Data is the foundation of AI. To ensure AI performs as expected, high-quality data is essential. In this panel discussion, Chad, Maria, Joe, and Pushkar hosted by Sam Partee will explore strategies for obtaining and maintaining high-quality data, as well as common pitfalls to avoid when using data for AI models. // Panelists - Samuel Partee: Principal Applied AI Engineer @ Redis - Chad Sanderson: CEO & Co-Founder @ Gable - Joe Reis: CEO/Co-Founder @ Ternary Data - Maria Zhang: CEO Cofounder @ Proactive AI Lab Inc - Pushkar Garg: Staff Machine Learning Engineer @ Clari Inc.

The Variational Book // Yuri Plotkin // #253 Aug 13, 2024

Show notes

Yuri Plotkin is a Biomedical Engineer and Machine Learning Scientist, and the author of The Variational Book.

The Variational Book // MLOps Podcast #253 with Yuri Plotkin, an ML Scientist.

// Abstract

Curiosity has been the underlying thread in Yuri's life and interests. With the explosion of Generative AI, Yuri was fascinated by the topic and decided he needed to learn more. Yuri pursued learning by reading, deriving, and understanding seminal papers within the last generation. The endeavors culminated in the writing of a book on the topic, The Variational Book, which Yuri expects to release shortly in the coming months. A bit of detail about the topics he covers can be found here: www.thevariationalbook.com.

// Bio

Evolved from biomedical engineer to wet-lab scientist, and more recently transitioned Yuri's career to computer science with the last 10+ years developing projects at the intersection of medicine, life sciences, and machine learning. Yuri's educational background is in Biomedical Engineering, at Columbia University (M.S.) and the University of California, San Diego (B.S.). Current interests include generative AI, diffusion models, and LLMs.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://plotkiny.github.io/

The Variational Book: www.thevariationalbook.com

SAS: https://www.sas.com/en_us/home.html

SAS® Decision Builder: https://www.sas.com/en_us/offers/23q4/microsoft-fabric.html

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Yuri on LinkedIn: http://www.linkedin.com/in/yuri-plotkin/

Timestamps:

[00:00] Yuri's preferred coffee

[00:23] Takeaways

[01:56] Register for the Data Engineering for AIML Conference now!

[02:47] Yuri's background

[06:13] The Variational Book

[10:25] Not including LLMs in the book

[12:14] Diffusion models

[16:37] Evolution within diffusion models

[20:55] Diffusion models for video

[25:43] Evolution and optimization of algorithms

[28:53] Markovian

[33:06 - 34:52] SAS Ad

[34:53] Understanding Markovian vs Non-Markovian models

[40:28] Visualizing model evolution

[43:46] Models through time

[44:53] The Variational Book inspiration

[47:53] Influencing LLM latent space

[51:07] Understanding ML Architectures

[52:56] Balancing AI complexity

[55:05] Wrap up

Vision and Strategies for Attracting & Driving AI Talents in High Growth // Panel // AIQCON Aug 09, 2024

Show notes

// Abstract Attracting and retaining top AI talent is essential for staying competitive. This panel will explore crafting and communicating a compelling vision that aligns with the organization's evolving needs, inspiring potential hires and motivating current employees. The discussion will offer actionable strategies for sourcing top talent, adapting to changing needs, and maintaining company alignment. Attendees will learn best practices for attracting AI professionals, creating an attractive employer brand, and enhancing talent acquisition and retention strategies. Lastly, the panel will cover structuring and organizing the AI team as it grows to ensure alignment with business goals. This includes optimal team configurations, leadership roles, and processes that support collaboration and innovation, enabling sustained growth and success. // PANELISTS Ashley Antonides: Associate Research Director, AI/ML @ Two Six Technologies Olga Beregovaya: VP, AI @ Smartling Shailvi Wakhlu: Founder @ Shailvi Ventures LLC A big thank you to our Premium Sponsors Google Cloud & Databricks for their generous support!

Our TOPPODCAST Picks

Follow Us

Stay Connected

Agentic Conversations (formally mlops.community)

Related Podcasts

Links

Stay Connected