Agentic Conversations (formally mlops.community)

Latest Episodes:

[EXCLUSIVE EPISODE!] LLM Key Results Apr 13, 2023

Show notes

This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month.

For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing

Sign up for our LLM in-production conference happening on April 13th (TODAY) here:
https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13

Multilingual Programming and a Project Structure to Enable It // Rodolfo Núñez // MLOps Podcast #153 Apr 10, 2023

Show notes

MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan.

// Abstract

It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about which language is better, but rather what language solves the given section of your problem better for you.

// Bio

Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Scientist at Entel (a Chilean telecommunications company) and is now a Senior Machine Learning Engineer at the same company, where I also lead three mini teams dedicated to internal cybersecurity; design/promote continuous training for the entire Analytics team and also the whole company; and ensure the improvement of programming practices and code cleanliness standards.

Rodo is currently in charge of helping the team put models into production and define the tools that we will use for it. He specializes in R, but he's language/tool agnostic: you should use the tool that best solves your current problem.

Rodo studied Mathematical Engineering and MSc in Applied Mathematics at the University of Chile, in addition to General Engineering at the École Centrale Marseille.

Rodo really likes to share knowledge (bi-directionally) in whatever he thinks he can contribute. Some things that Rodo likes teaching are Data Science, Math, Latin Dances, and whatever he thinks he can give to people.

Rodo's other interests are computer games (especially Vermintide and Darktide), board games, and dancing to Latin rhythms. Also, he streams some games and Data science-related topics on Twitch.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.twitch.tv/en_coders

https://www.youtube.com/@en_coders

https://www.twitch.tv/rodonunez

https://github.com/rodo-nunez

https://github.com/en-coders-cl

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Rodo on LinkedIn: https://www.linkedin.com/in/rodonunez/

Timestamps:

[00:00] Rodo's preferred coffee

[00:16] Project structure

[00:34] Introduction to Rodolfo Núñez

[01:20] Takeaways

[04:34] Check out our Meetups, podcasts, newsletters, TikTok, and blog posts!

[05:50] Why data scientists should know how to code and code properly

[10:32] Becoming a team player

[14:02] Cookie-cutter project

[17:50] Markdown and Quarter over Jupyter notebooks

[23:18] Data scientists' templates

[30:06] Significance of scripts

[33:30] Monolith to Microservices

[34:33] Reproducibility

[36:37] Entire event processing scripts

[40:44] In-House cataloging solution

[42:08] Data flows

[46:00] Bonus topics!

[47:23] Elbow methodology

[50:17] Idea behind cross sampling

[50:51] Machine Learning and MLOps Security at Entel

[58:04] Wrap up

[Bonus Episode] Practical AI x MLOps // Demetrios Brinkmann, Mihail Eric, Daniel Whitenack and Chris Benson Apr 07, 2023

Show notes

Worlds are colliding! This week, we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc.

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community:

https://go.mlops.community/slack

https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more:

https://mlops.community/

How A Manager Became a Believer in DevOps for Machine Learning // Keith Trnka // MLOps Podcast #152 Apr 04, 2023

Show notes

MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning.

// Abstract

Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime.

Keith's background includes directing data science at 98.6, improving language models at Swipe and Nuance, and completing a Ph.D. thesis in language modeling for assistive technology. His work in these areas has contributed to the development of technology applications for healthcare, including telemedicine visits using natural language processing and machine learning.

// Bio

Keith has been in the industry for about 11 years. Most recently, he was the Director of Data Science at 98point6, where we made telemedicine visits easier for doctors using natural language processing, machine learning, backend engineering, AWS, and frontend engineering. Prior to that, Keith improved the language models used in mobile phone keyboards at Swype and Nuance. And before that, He did his Ph.D. thesis in language modeling for assistive technology.

Currently, Keith is traveling, mentoring, and doing a side project on machine translation.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Keith on LinkedIn: https://www.linkedin.com/in/keith-trnka/

Timestamps:

[00:00] Keith's preferred coffee

[00:14] Introduction to Keith Trnka

[02:28] Getting into Machine Learning

[03:24] Learning applications

[07:26] Data scientists' pain working together

[12:32] Build: An Unorthodox Guide to Making Things Worth Making by Tony Fadell

[15:00] Context meaning

[17:08] Determining Healthcare workers' certain passions

[20:24] From Engineer to User Advocate

[23:13] Balancing the drive to explore and cutting losses

[27:12] Simplifying Machine Learning Process

[32:59] Hiring senior engineers

[38:30] High productivity team tips

[40:38] Inspire people not to dictate

[45:46] Code Review Aggressivity Check

[48:25] Zero Downtime Migration

[52:40] Choosing between Lambda and EKS

[55:23] Wrap up

ML in Production: A DS from Ubisoft Perspective // Jean-Michel Daignan // MLOps Podcast #151 Mar 28, 2023

Show notes

MLOps Coffee Sessions #151 with Jean-Michel Daignan, ML in Production: A DS from Ubisoft Perspective, co-hosted by Abi Aryan.

// Abstract

As a data scientist himself, Jean-Michel has a unique perspective on the needs of data scientists when it comes to platform development. He talks about the non-invasive approach his team is taking to bring people onto the platform and their SDK, Merlin. The team is focused on tying machine learning products back to business use cases and the ROI they provide. Abby and Jean-Michel also discuss the use of generative AI and the importance of balancing delivering value and building things quickly. Jean-Michel's blog posts on the topic are recommended for further reading.

// Bio

The author of the blog "the-odd-dataguy.com" has been a data scientist for over 4.5 years at Ubisoft. Prior to joining the video game industry, Jean-Michel had a background in engineering from France and had previously worked in the energy sector. The blog focuses on topics related to data and machine learning, showcasing the author's expertise in the field.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Blog page: https://www.the-odd-dataguy.com/

Bringing Machine Learning to Production at Ubisoft (PydataMTL June 22): https://www.the-odd-dataguy.com/2022/12/29/recap_pydata_mtl_june22/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Jean-Michel on LinkedIn: https://www.linkedin.com/in/jeanmicheldaignan/

Timestamps:

[00:00] Jean-Michel's preferred beverage

[00:19] Jean-Michel Daignan's background

[00:28] Takeaways

[04:30] Rate us and share the podcasts with your friends!

[05:37] Jean-Michel's projects at Ubisoft

[07:48] Jean-Michel's success as a Data Scientist

[09:45] Ubisoft basics

[10:40] Jean-Michel's success from the downfalls of being a data scientist

[12:18] Building for data scientists' considerations

[13:57] Differences in designing for data scientists in general

[16:35] End twin pipelines and their functions

[19:35] Major problems doing maintenance

[20:53] Data quality ownership

[22:33] Monitoring levels

[24:25] Locomotive systems

[26:14] Merlin

[29:12] DS storage systems

[31:09] Feature stores batch or streaming?

[32:19] Bringing Machine Learning to Production at Ubisoft blog post

[35:10] Features and recommendation systems

[37:03] Playing games

[38:21] Play data = play personalities

[39:42] Deep learning in all the diffusion models or the foundation models

[43:06] Servicing data scientists' needs

[45:28] Ubisoft's data volume

[48:00] Wrap up

Large Language Models in Production Round-table Conversation Mar 23, 2023

Show notes

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian.

// Abstract

Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event!

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

Virtual LLMs in Production Conference registration:

https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13

Chinchilla papers:

https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/

Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/

Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/

Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/

Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/

Timestamps:

[00:00] Round table success to Virtual LLM in Production Conference on April 13th!

[00:18] Register for the Virtual LLM in Production Conference now!

[00:44] LLM in Production survey

[01:40] Lightning round of introduction of speakers

[04:34] Large Language Models definition

[09:17] What do we consider large?

[10:35] Thought process in use cases production

[14:30] LLM open source huge movements

[16:50] Problems with qualifications

[19:25] Production use cases frameworks directions

[25:25] Open-source language models tokenizer

[26:25] Language models democratization

[29:25] Three categories for LLMs in Production

[31:22] Latency at 2 levels

[33:27] Defining production

[34:57] Hitting the latency problems

[38:20] Fundamental latency barrier

[40:39] Latency use case requirement

[44:25] Costs and the use cases

[48:12] Product management involvement in costing

[49:38] LLMs Hallucination definition

[52:05] Building deterministic systems trust

[55:21] Wrap up

The Future of Search in the Era of Large Language Models // Saahil Jain // MLOps Podcast #150 Mar 21, 2023

Show notes

MLOps Coffee Sessions #150 with Saahil Jain, The Future of Search in the Era of Large Language Models, co-hosted by David Aponte.

// Abstract

Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when working with large language models.

Saahil also discusses the intersection of traditional information retrieval and generative models and the trade-offs in the type of outputs they produce. He suggests occupying users' attention during long wait times and the importance of considering how users engage with websites beyond just performance.

// Bio

Saahil Jain is an engineer at You.com. At You.com, Saahil builds searching and ranking systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. His research work has been published in machine learning conferences such as EMNLP, NeurIPS Datasets & Benchmarks, and ACM-CHIL, among others. He has publicly released various machine learning models, methods, and datasets, which have been used by researchers in both academic institutions and hospitals across the world, as part of an open-source movement to democratize AI research in medicine. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science from Columbia University and Stanford University, respectively.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: http://saahiljain.me/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/

Connect with Saahil on LinkedIn: https://www.linkedin.com/in/saahiljain/

Timestamps

[00:00] Saahil's preferred coffee

[04:32] Saahil Jain's background

[04:44] Takeaways

[07:49] Search Landscape

[12:57] Use cases exploration

[14:51] Differentiating what to give to users

[17:19] Search key challenges

[20:05] Search objective relevance

[23:22] MLOps Search and Recommender Systems

[26:54] Addressing Latency Issues

[29:41] Throughput presenting results

[32:20] Compute challenges

[34:24] Working at a small start-up

[36:10] Citations critics

[39:17] Use cases to build

[40:40] Integrating to Leveraging You.com

[42:26] Open AI

[46:13] Interfacing with bugs

[49:16] Staying focused

[52:05] Retrieval augmented models

[52:32] Closing thoughts

[53:47] Wrap up

The Challenges of Deploying (many!) ML Models // Jason McCampbell // MLOps Podcast #149 Mar 14, 2023

Show notes

MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo.

// Abstract
In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-style cloud-edge, local servers, or running on connected devices.

// Bio
Jason McCampbell is the Director of Architecture at Wallaroo.ai and has over 20 years of experience designing and building high-performance and distributed systems. From semiconductor design to simulation, a common thread is that the tools have to be fast, use resources efficiently, and "just work" as critical business applications.

At Wallaroo, Jason is focused on solving the challenges of deploying AI models at scale, both in the data center and at "the edge". He has a degree in computer engineering as well as an MBA and is an alum of multiple early-stage ventures. Living in Austin, Jason enjoys spending time with his wife and two kids and cycling through the Hill Country.

// MLOps Jobs board

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Website: https://wallaroo.ai
MLOps at the Edge Slack channel: https://mlops-community.slack.com/archives/C02G1BHMJRL

--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Jason on LinkedIn: https://www.linkedin.com/in/jasonmccampbell/

Timestamps:
[00:00] Jason's preferred coffee
[01:22] Takeaways
[06:06] MLOps at the Edge Slack channel
[06:36] Shoutout to Wallaroo!
[07:34] Jason's background
[09:54] Combining Edge and ML
[11:03] Defining Edge Computing
[13:21] Data transport restrictions
[15:02] Protecting IP in Edge Computing
[17:48] Decentralized Teams and Knowledge Sharing
[20:45] Real-time Data Analysis for Improved Store Security and Efficiency
[24:49] How to Ensure Statistical Integrity in Federated Networks
[26:50] Architecting ML at the Edge
[30:44] Machine Learning models adversarial attacks
[33:25] Pros and cons of baking models into containers
[34:52] Jason's work at Wallaroo
[38:22] Navigating the Market Edge
[40:49] Last challenges to overcome
[44:15] Data Science Use Cases in Logistics
[46:27] Vector trade-offs
[49:34] AI at the Edge challenges
[50:40] Finding the Sweet Spot
[53:46] Driving revenue
[55:33] Wrap up

Intelligence & MLOps // Karl Fezer // MLOps Podcast # 148 Mar 07, 2023

Show notes

MLOps Coffee Sessions #148 with Karl Fezer, Intelligence & MLOps co-hosted by Abi Aryan.

// Abstract
This conversation explores various topics, including biases, defining intelligence, and the future of large language models and MLOps. Karl discusses his paper on defining intelligence and how it relates to the increasing interest in Artificial Intelligence. Karl shares his thoughts on the overlap between foundational models and MLOps, emphasizing the importance of making high-impact tasks more efficient and easier. The conversation touched on philosophical tangents but ultimately circled back to practical applications of these concepts.

// Bio
Karl graduated in 2014 from the University of Georgia with a Master's in Science in Artificial Intelligence. Since then, he has continued to stay on top of the latest iterations of Machine Learning and loves trying new open-source frameworks.
For the last 6 years, he has been purely focused on AI Developer Relations. First at Mycroft, the voice assistant startup, then Intel, Arm, briefly at Wallaroo.ai, and now at Lockheed Martin.
He currently lives in Seattle and spends his free time writing, reading, sailing, and camping.

// MLOps Jobs board
jobs.mlops.community

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
Intelligence Primer paper by Karl Fezer and Andrew Sloss: https://arxiv.org/abs/2008.07324
Computing Machinery and Intelligence paper by Alan Turing: https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Karl on LinkedIn: https://www.linkedin.com/in/karlfezer/

Timestamps:

[00:00] Karl's preferred coffee

[00:52] Takeaways

[04:05] Subscribe to our newsletters, join our Slack channel, and keep listening to the podcasts

[04:49] Key to a successful podcast

[10:01] Open source toolkit

[13:04] Evangelizing

[16:46] Living in an adjusted simulator

[17:26] Intelligence Primer paper

[19:51] Intelligence vs Artificial Intelligence

[23:52] Biomimicry

[28:04] Dualism

[29:23] Conscious learning vs Unconscious learning

[34:00] Artificial Intelligence mismatch

[37:07] Infrastructure around machine learning

[41:30] Cloud primitives and current databases collaboration

[43:33] MLOps tools explosion and their duration

[45:04] Open-source tools investment

[48:05] Management service trade-offs

[49:48] Key winning factors for companies with MLOps in the next 5 years

[51:05] Wrap up

The Rise of Serverless Databases // Alex DeBrie // MLOps Podcast #147 Feb 28, 2023

Show notes

MLOps Coffee Sessions #147 with Alex DeBrie, Something About Databases, co-hosted by Abi Aryan.

// Abstract
For databases, it feels like we're in the middle of a big shift. The first 10-15 years of the cloud were mostly about using the same core infrastructure patterns, but in the cloud (SQL Server, MySQL, Postgres, Redis, and Elasticsearch).

In the last few years, we're finally seeing data infrastructure that is truly built for the cloud. Elastic, scalable, resilient, managed, etc. Early examples were Snowflake + DynamoDB. The most recent ones are all the 'NewSQL' contenders (Cockroach, Yugabyte, Spanner) or the 'serverless' ones (Neon, Planetscale). Also seeing improvements in caching, search, etc. Exciting times!

// Bio
Alex is an AWS Data Hero and a self-employed AWS consultant and trainer. He is the author of The DynamoDB Book, a comprehensive guide to data modeling with DynamoDB. Previously, he worked for Stedi and for Serverless, Inc., creators of the Serverless Framework. He loves being involved in the AWS & serverless community, and he lives in Omaha, NE with his family.

// MLOps Jobs board
jobs.mlops.community

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related
Links Key Takeaways from the DynamoDB Paper: https://www.alexdebrie.com/posts/dynamodb-paper/
Understanding Eventual Consistency in DynamoDB: https://www.alexdebrie.com/posts/dynamodb-eventual-consistency/
Two Scoops of Django 1.11: Best Practices for the Django Web Framework: https://www.amazon.com/Two-Scoops-Django-1-11-Practices/dp/0692915729CAP or no CAP?
Understanding when the CAP theorem applies and what it means: https://www.alexdebrie.com/posts/when-does-cap-theorem-apply/
Stop fighting your database/ The DynamoDB book: https://dynamodbbook.com/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/
Connect with Alex on LinkedIn: https://www.linkedin.com/in/alex-debrie/

Timestamps:
[00:00] Alex's preferred coffee
[00:27] Introduction to Alex DeBrie and DynamoDB
[01:05] Takeaways
[03:47] Please write down your reviews, and you might have a book by Alex!
[04:57] Alex's journey from being an Attorney to becoming a Data Engineer
[07:31] Why engineering?
[10:07] Serverless Data
[12:54] Before Airflow
[15:41] Batch vs streaming
[17:03] Difficulties in the Batch and Streaming sides
[19:45] Modern data infrastructure databases
[24:37] Cloud Ecosystem Maturity
[27:59] New generation type of Snowflake
[29:47] Comparing databases
[30:58] What's next on connectors from 2 perspectives?
[34:25] Management services at the MLOps level
[36:38] DynamoDB
[39:32] Why do you like DynamoDB?
[41:00] Data used in DynamoDB and size limits
[43:46] Comparison of tradeoffs between DynamoDB and Redis
[45:52] Preferred opinionated databases
[48:43] CAP or no CAP? Understanding when the CAP theorem applies and what it means
[52:10] The DynamoDB book
[56:17] Chapter you want to expand on in the book
[57:43] Next book to write
[59:25] ChatGPT iterations
[1:01:59] Data modeling book wished to be written
[1:03:27] Wrap up

Our TOPPODCAST Picks

Follow Us

Stay Connected

Agentic Conversations (formally mlops.community)

Related Podcasts

Links

Stay Connected