Agentic Conversations (formally mlops.community)

Latest Episodes:

Using Large Language Models at AngelList // Thibaut Labarre // #171 Aug 15, 2023

Show notes

MLOps Coffee Sessions #171 with Thibaut Labarre, Using Large Language Models at AngelList, co-hosted by Ryan Russon.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

Thibaut innovatively addressed previous system constraints, achieving scalability and cost efficiency. Leveraging AngelList investing and natural language processing expertise, they refined news article classification for investor dashboards. Central is their groundbreaking platform, AngelList Relay, automating parsing and offering vital insights to investors. Amid challenges like Azure OpenAI collaboration and rate limit solutions, Thibaut reflects candidly. The narrative highlights prompt engineering's strategic importance and empowering domain experts for ongoing advancement.

// Bio

Thibaut LaBarre is an engineering lead with a background in Natural Language Processing (NLP). Currently, Thibaut focuses on unlocking the potential of Large Language Model (LLM) technology at AngelList, enabling everyone within the organization to become prompt engineers on a quest to streamline and automate the infrastructure for Venture Capital. Prior to that, Thibaut began his journey at Amazon as an intern, where he built Heartbeat, a state-of-the-art NLP tool that consolidates millions of data points from various feedback sources, such as product reviews, customer contacts, and social media, to provide valuable insights to global product teams. Over the span of seven years, he expanded his internship project into an organization of 20 engineers. He received a M.S. in Computational Linguistics from the University of Washington.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

Website: https://www.angellist.com/venture/relay

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryanrusson/

Connect with Thibaut on LinkedIn: https://www.linkedin.com/in/thibautlabarre/

Timestamps:

[00:00] Thibaut's preferred beverage

[00:50] Takeaways

[04:05] Please like, share, and subscribe to our MLOps channels!

[04:44] A huge fan of Isaac Asimov

[07:20] Thibaut Labarre background

[09:13] AngelList as an organization

[10:50] AI sense of building

[12:29] System trade-offs

[15:20] OpenAI's limitation

[16:31] Human in the loop

[17:22] Classifying relevance

[18:09] Fight for value

[19:37] Added value

[22:10] Exploring efficient ways to automate tasks.

[24:20] Investing in off-the-shelf models

[27:56] AngelList Relay

[30:49] News article and investment document classification technology

[32:39] Back-end tech

[34:09] Prompt layer

[35:28] Prompt layer as a living

[37:04] Foreseeing no human intervention

[39:00] Blocking hallucinations

[40:33] Challenges

[43:49] Investments in other models besides OpenAI

[45:20] Integration with other models

[46:28] Ethical concerns when

[48:37] OpenAI breaking Prompts

[50:46] Wrap up

All the Hard Stuff with LLMs in Product Development // Phillip Carter // #170 Aug 11, 2023

Show notes

MLOps Coffee Sessions #170 with Phillip Carter, All the Hard Stuff with LLMs in Product Development.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

Delve into challenges in implementing LLMs, such as security concerns and collaborative measures against attacks. Emphasize the role of ML engineers and product managers in successful implementation. Explore identifying leading indicators and measuring ROI for impactful AI initiatives.

// Bio

Phillip is on the product team at Honeycomb, where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are, if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

Website: https://phillipcarter.dev/

https://www.honeycomb.io/blog/improving-llms-production-observability

https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm https://phillipcarter.dev/posts/how-to-make-an-fsharp-code-fixer/

The "hard stuff" post: https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm

Our follow-up on iterating on LLMs in prod: https://www.honeycomb.io/blog/improving-llms-production-observability

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Phillip on LinkedIn: https://www.linkedin.com/in/phillip-carter-4714a135/

Timestamps:

[00:00] Phillip's preferred coffee

[00:33] Takeaways

[01:53] Please like, share, and subscribe to our MLOps channels!

[02:45] Phillip's background

[07:15] Querying Natural Language

[11:25] Function calls

[14:29] Pasting errors or traces

[16:30] Error patterns

[20:22] Honeycomb's Improvement cycle

[23:20] Prompt boxes rationale

[28:06] Prompt injection cycles

[32:11] Injection Attempt

[33:30] UI undervalued, charging the AI feature

[35:11] ROI cost

[44:26] Bridging ML and Product Perspective

[52:53] AI Model Trade-offs

[56:33] Query assistant

[59:07] Honeycomb is hiring!

[1:00:08] Wrap up

MLOps at the Age of Generative AI // Barak Turovsky // #169 Aug 08, 2023

Show notes

MLOps Coffee Sessions #169 with Barak Turovsky, MLOps at the Age of Generative AI.

Thanks to Weights & Biases for sponsoring this episode. Check out their new course on evaluating and fine-tuning LLMs at wandb.me/genai-mlops.course

// Abstract

The talk focuses on MLOps aspects of developing, training, and serving Generative AI/Large Language models

// Bio

Barak is an Executive in Residence at Scale Venture Partners, a leading Enterprise venture capital firm. Barak spent 10 years as Head of Product and User Experience for Languages AI and Google Translate teams within the Google AI org, focusing on applying cutting-edge Artificial Intelligence and Machine Learning technologies to deliver magical experiences across Google Search, Assistant, Cloud, Chrome, Ads, and other products. Previously, Barak spent 2 years as a product leader within the Google Commerce team. Most recently, Barak served as Chief Product Officer, responsible for product management and engineering at Trax, a leading provider of Computer Vision AI solutions for the Retail and Commerce industries. Prior to joining Google in 2011, Barak was Director of Products in Microsoft’s Mobile Advertising, Head of Mobile Commerce at PayPal, and Chief Technical Officer at an Israeli start-up. He lived more than 10 years in 3 different countries (Russia, Israel, and the US) and speaks three languages. Barak earned a Bachelor of Laws degree from Tel Aviv University, Israel, and a Master’s of Business Administration from the University of California, Berkeley.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Bio and links about Barak's work: https://docs.google.com/document/d/1E4Yrmt_Y57oTEYHQQDvt71XzSJ8Ew5WvscAQbHV4K3U/edit

Framework for evaluating Generative AI use cases: https://www.linkedin.com/pulse/framework-evaluating-generative-ai-use-cases-barak-turovsky/?trackingId=%2BMRxEZ9WTPCNH2JscILTeg%3D%3D

The Great A.I. Awakening: https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Barak on LinkedIn: https://www.linkedin.com/in/baraktur/

Timestamps:

[00:00] Barak's preferred coffee

[00:23] Barak Turovsky's background

[03:10] Please like, share, and subscribe to our MLOps channels!

[04:09] Getting into tech

[08:39] First wave of AI

[12:39] Building a product at scale and the challenges

[15:59] Framework for evaluating Generative AI use cases

[24:33] Machine trust adoption

[29:13] Wandb's new course

[31:10] Focus on achievable use cases for LLMs

[36:36] User feedback

[38:23] Disruption of entertainment and customer interactions

[46:14] Get new tools or work with your own distribution?

[47:57] Importance of data engineers

[53:28] ML Engineers Collaborate with Product[

56:13] Wrap up

Experiment Tracking in the Age of LLMs // Piotr Niedźwiedź // #168 Aug 01, 2023

Show notes

MLOps Coffee Sessions #168 with Piotr Niedźwiedź, Experiment Tracking in the Age of LLMs, co-hosted by Vishnu Rachakonda.

// Abstract

Piotr shares his journey as an entrepreneur and the importance of focusing on core values to achieve success. He highlights the mission of Neptune to support ML teams by providing them with control and confidence in their models. The conversation delves into the role of experiment tracking in understanding and debugging models, comparing experiments, and versioning models. Piotr introduces the concept of prompt engineering as a different approach to building models, emphasizing the need for prompt validation and testing methods.

// Bio

Piotr is the CEO of neptune.ai. Day to day, apart from running the company, he focuses on the product side of things. Strategy, planning, ideation, getting deep into user needs and use cases. He really likes it.
Piotr's path to ML started with software engineering. Always liked math and started programming when he was 7. In high school, Piotr got into algorithmics and programming competitions and loved competing with the best. That got him into the best CS and Maths program in Poland which funny enough today specializes in machine learning.
Piotr did his internships at Facebook and Google and was offered to stay in the Valley. But something about being a FAANG engineer didn’t feel right. He had this spark to do more, build something himself. So with a few of his friends from the algo days, they started Codilime, a software consultancy, and later a sister company Deepsense.ai machine learning consultancy, where he was a CTO.
When he came to the ML space from software engineering, he was surprised by the messy experimentation practices, lack of control over model building, and a missing ecosystem of tools to help people deliver models confidently.
It was a stark contrast to the software development ecosystem, where you have mature tools for DevOps, observability, or orchestration to execute efficiently in production. And then, one day, some ML engineers from Deepsense.ai came to him and showed him this tool for tracking experiments they built during a Kaggle competition (which they won btw), and he knew this could be big.
He asked around, and everyone was struggling with managing experiments. He decided to spin it off as a VC-funded product company, and the rest is history.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://neptune.ai/blog/author/piotr-niedzwiedz

https://www.youtube.com/playlist?list=PLKePQLVx9tOfKFbg9GY2Anl41Be4T1-m5
https://thesequence.substack.com/p/-piotr-niedzwiedz-neptunes-ceo-on

https://open.spotify.com/episode/2KEqTMAHODbPKdUEtlrhm7?si=ed862b2ac7534e39
https://www.linkedin.com/in/piotrniedzwiedz/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Piotr on LinkedIn: https://www.linkedin.com/in/piotrniedzwiedz/

Timestamps:

[00:00] Introduction to Piotr Niedźwiedź

[01:35] Please like, share, and subscribe to our MLOps channels!

[01:58] Wojciech Zaremba

[05:20] The Olympiad

[06:31] Building own company

[12:16] Talking outside Poland with the same passion

[13:45] Adapting with Neptune

[19:35] Core values focus

[22:02] Product Vision changes with advances

[29:36] Control and confidence

[30:05] Experiment tracking existing use cases

[37:25] Control pane

[38:59] Piotr's prediction

[43:20] WiFi issues around the world

[44:09] Wrap up

Treating Prompt Engineering More Like Code // Maxime Beauchemin // #167 Jul 25, 2023

Show notes

MLOps Coffee Sessions #167 with Maxime Beauchemin, Treating Prompt Engineering More Like Code.

// Abstract

Promptimize is an innovative tool designed to scientifically evaluate the effectiveness of prompts. Discover the advantages of open-sourcing the tool and its relevance, drawing parallels with test suites in software engineering. Uncover the increasing interest in this domain and the necessity for transparent interactions with language models. Delve into the world of prompt optimization, deterministic evaluation, and the unique challenges in AI prompt engineering.

// Bio

Maxime Beauchemin is the founder and CEO of Preset, a Series B startup supporting and commercializing the Apache Superset project. Max was the original creator of Apache Airflow and Apache Superset when he was at Airbnb. Max has over a decade of experience in data engineering at companies like Lyft, Airbnb, Facebook, and Ubisoft.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Max's first MLOps Podcast episode: https://go.mlops.community/KBnOgN

Test-Driven Prompt Engineering for LLMs with Promptimize blog: https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-introducing-promptimize-for-test-driven-prompt-bffbbca91535

https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-Test-Driven Prompt Engineering for LLMs with Promptimize podcast: https://talkpython.fm/episodes/show/417/test-driven-prompt-engineering-for-llms-with-promptimize

Taming AI Product Development Through Test-driven Prompt Engineering // Maxime Beauchemin // LLMs in Production Conference lightning talk: https://home.mlops.community/home/videos/taming-ai-product-development-through-test-driven-prompt-engineering

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Max on LinkedIn: https://www.linkedin.com/in/maximebeauchemin/

Timestamps:

[00:00] Max introduces the Apache Superset project at Preset

[01:04] Max's preferred coffee

[01:16] Airflow creator

[01:45] Takeaways

[03:53] Please like, share, and subscribe to our MLOps channels!

[04:31] Check Max's first MLOps Podcast episode

[05:20] Promptimize

[06:10] Interaction with API

[08:27] Deterministic evaluation of SQL queries and AI

[12:40] Figuring out the right edge cases

[14:17] Reaction with Vector Database

[15:55] Promptomize Test Suite

[18:48] Promptimize vision

[20:47] The open-source blood

[23:04] Impact of open source

[23:18] Dangers of open source

[25:25] AI-Language Models Revolution

[27:36] Test-driven design

[29:46] Prompt tracking

[33:41] Building Test Suites as Assets

[36:49] Adding new prompt cases to new capabilities

[39:32] Monitoring speed and cost

[44:07] Creating own benchmarks

[46:19] AI feature adding more value to the end users

[49:39] Perceived value of the feature

[50:53] LLMs costs

[52:15] Specialized model versus Generalized model

[56:58] Fine-tuning LLMs use cases

[1:02:30] Classic Engineer's Dilemma

[1:03:46] Build exciting tech that's available

[1:05:02] Catastrophic forgetting

[1:10:28] Prompt-driven development

[1:13:23] Wrap up

Eliminating Garbage In/Garbage Out for Analytics and ML // Roy Hasson & Santona Tuli // #166 Jul 18, 2023

Show notes

MLOps Coffee Sessions #166 with Roy Hasson & Santona Tuli, Eliminating Garbage In/Garbage Out for Analytics and ML.

// Abstract

Shift left data quality ownership and observability that makes it easy for users to catch bad data at the source and stop it from entering your analytics/ML stack.

// Bio

Santona Tuli

Santona Tuli, Ph.D., began her data journey through fundamental physics—searching through massive event data from particle collisions at CERN to detect rare particles. She’s since extended her machine learning engineering to natural language processing, before switching focus to product and data engineering for data workflow authoring frameworks. As a Python engineer, she started with the programmatic data orchestration tool, Airflow, helping improve its developer experience for data science and machine learning pipelines. Currently, at Upsolver, she leads data engineering and science, driving developer research and engagement for the declarative workflow authoring framework in SQL. Dr. Tuli is passionate about building, as well as empowering others to build, end-to-end data and ML pipelines, scalably.

Roy Hasson

Roy is the head of product at Upsolver, helping companies deliver high-quality data to their analytics and ML tools. Previously, Roy led product management for AWS Glue and AWS Lake Formation.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://royondata.substack.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community:

https://go.mlops.community/slack

https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more:

https://mlops.community/

Connect with Demetrios on LinkedIn:

https://www.linkedin.com/in/dpbrinkm/

Connect with Roy on LinkedIn:

https://www.linkedin.com/in/royhasson/

Connect with Santona on LinkedIn:

https://www.linkedin.com/in/santona-tuli/

Timestamps:

[00:00] Santona's and Roy's preferred coffee

[01:05] Santona's and Roy's background

[03:33] Takeaways

[05:49] Please like, share, and subscribe to our MLOps channels!

[06:42] Back story of having Santona and Roy on the podcast

[09:51] Santona's story

[11:37] Optimal tag teamwork

[16:53] Dealing with stakeholder needs

[26:25] Having mechanisms in place

[27:30] Building for data Engineers vs building for data scientists

[34:50] Creating solutions for users

[38:55] User experience holistic point of view

[41:11] Tooling sprawl is real

[42:00] LLMs' reliability

[45:00] Things I would have loved to learn five years ago

[49:46] Wrap up

Python Power: How Daft Embeds Models and Revolutionizes Data Processing // Sammy Sidhu // #165 Jul 11, 2023

Show notes

MLOps Coffee Sessions #165 with Sammy Sidhu, Python Power: How Daft Embeds Models and Revolutionizes Data Processing.

// Abstract

Sammy shares his fascinating journey in the autonomous vehicle industry, highlighting his involvement in two successful startup acquisitions by Tesla and Toyota. He emphasizes his expertise in optimizing and distilling models for efficient machine learning, which he has incorporated into his new company, Eventual. The company's open-source offering, daf, focuses on tackling the challenges of unstructured and complex data. Sammy discusses the future of MLOps, machine learning, and data storage, particularly in relation to the retrieval and processing of unstructured data. The Eventual team is developing Daft, an open-source query engine that aims to provide efficient data storage solutions for unstructured data, offering features like governance, schema evolution, and time travel. The conversation sheds light on the innovative developments in the field and the potential impact on various industries.

// Bio

Sammy is a Deep Learning and systems veteran, holding over a dozen publications and patents in the space. Sammy graduated from the University of California, Berkeley, where he did research in Deep Learning and High Performance Computing. He then joined DeepScale as the Chief Architect and led the development of perception technologies for autonomous vehicles. During this time, DeepScale grew rapidly and was subsequently acquired by Tesla in 2019. Staying in Autonomous Vehicles, Sammy joined Lyft Level 5 as a Senior Staff Software Engineer, building out core perception algorithms as well as infrastructure for machine learning and embedded systems. Level 5 was then acquired by Toyota in 2021, adopting much of its work. Sammy is now CEO and Co-Founder at Eventual Building Daft, an open-source query engine that specializes in multimodal data.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://sammysidhu.com/

Check out Daft, our open-source query engine for multimodal data! https://www.getdaft.io/

Here are some talks/shows we have given about it:- PyData Global (Dec 2022): Large-scale image processing: https://www.youtube.com/watch?v=ol6IQUbyeDo&ab_channel=PyData

- Ray Meetup (March 2023): Distributed ML preprocessing + training on Ray https://www.youtube.com/watch?v=1MpEYlIlu7w&t=2972s&ab_channel=Anyscale

- The Data Stack Show (April 2023): Self-Driving Technology and Data Infrastructure with Sammy Sidhu https://datastackshow.com/podcast/the-prql-self-driving-technology-and-data-infrastructure-with-sammy-sidhu-co-founder-and-ceo-of-eventual/

Chain of Thought for LLMs: https://cobusgreyling.medium.com/chain-of-thought-prompting-in-llms-1077164edf97

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes: https://arxiv.org/abs/2305.02301

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sammy on LinkedIn: https://www.linkedin.com/in/sammy-sidhu/

Open Source and Fast Decision Making // Rob Hirschfeld // #164 Jul 04, 2023

Show notes

MLOps Coffee Sessions #164 with Rob Hirschfeld, Open Source and Fast Decision Making. This episode is brought to you by.

// Abstract

Rob Hirschfeld, the CEO and co-founder of Rack N, discusses his extensive experience in the DevOps movement. He shares his notable achievement of coining the term "the cloud" and obtaining patents for infrastructure management and API provision. Rob highlights the stagnant progress in operations and the persistent challenges in security and access controls within the industry. The absence of standardization in areas such as Kubernetes and single sign-on complicates the development of robust solutions. To address these issues, Rob underscores the significance of open-source practices, automation, and version control in achieving operational independence and resilience in infrastructure management.

// Bio

Rob is the CEO and Co-founder of RackN, an Austin-based start-up that develops software to help automate data centers, which they call Digital Rebar. This platform helps connect all the different pieces and tools that people use to manage infrastructure into workflow pipelines through seamless multi-component automation across the different pieces and parts needed to bring up IT systems, platforms, and applications. Rob has a background in Scale Computing, Mechanical and Systems Engineering, and specializes in large-scale complex systems that are integrated with the physical environment. He has founded companies and been in the cloud and infrastructure space for nearly 25 years, and has done everything from building the first Clouds using ESXi betas to serving four terms on the OpenStack Foundation Board. Rob was trained as an Industrial Engineer and holds degrees from Duke University and Louisiana State University.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://rackn.com/

https://robhirschfeld.com/about/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Rob on LinkedIn: https://www.linkedin.com/in/rhirschfeld/

Timestamps:

[00:00] Rob's preferred coffee

[00:17] Rob Hirschfeld's background

[01:42] Takeaways

[02:36] Please like, share, and subscribe to this channel!

[03:09] Creation of Cloud

[08:38] Changes in Cloud after 25 Years

[10:54] Pros and cons of microservices

[13:06] Secure Access Provisioning

[15:46] Parallelism with ads

[18:08] Redfish protocol

[20:21] Impact of using open source vs using a SAS provider

[26:15] Automation

[32:39] Embrace Operational Flexibility

[35:08] Automating infrastructure inefficiently

[41:26] Legacy code and resiliency

[43:39] Collection of metadata

[45:50] RackN

[51:23] Granular Cloud Preferences

[54:35] Reframing of perceived complexity

[57:32] Generative DevOps

[58:50] Wrap up

Democratizing AI // Yujian Tang // #163 Jun 27, 2023

Show notes

MLOps Coffee Sessions #163 with Yujian Tang, Democratizing AI, co-hosted by Abi Aryan.

// Abstract

The popularity of ChatGPT has brought large language model (LLMs) apps and their supporting technologies to the forefront. One of the supporting technologies is vector databases. Yujian shares how vector databases like Milvus are used in production and how they solve one of the biggest problems in LLM app building - data issues. They also discuss how Zilliz is democratizing vector databases through education, expanding access to technologies, and technical evangelism.

// Bio

Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published at conferences, including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Yujian on LinkedIn: https://www.linkedin.com/in/yujiantang

Timestamps:

[00:00] Yujian's preferred coffee

[02:40] Takeaways

[05:14] Please share this episode with your friends!

[06:39] Vector databases trajectory

[09:00] 2 start-up companies created by Yujian

[09:39] Uninitiated Vector Databases

[12:20] Vector Databases trade-off

[14:16] Difficulties in training LLMs

[23:30] Enterprise use cases

[27:38] Process/rules not to use LLMs unless necessary

[32:14] Setting up returns

[33:13] When not to use Vector Databases

[35:30] Elastic search

[36:07] Generative AI apps' common pitfalls

[39:35] Knowing your data

[41:50] Milvus

[48:28] Actual Enterprise use cases

[49:32] Horror stories

[50:31] Data mesh

[51:06] GPTCash

[52:10] Shout out to the Seattle Community!

[53:44] Wrap up

From Arduinos to LLMs: Exploring the Spectrum of ML // Soham Chatterjee // #162 Jun 20, 2023

Show notes

MLOps Coffee Sessions #162 with Soham Chatterjee, From LLMs to TinyML: The Dynamic Spectrum of MLOps, co-hosted by Abi Aryan.

// Abstract

Explore the spectrum of MLOps from large language models (LLMs) to TinyML. Soham highlights the difficulties of scaling machine learning models and cautions against relying exclusively on OpenAI's API due to its limitations. Soham is particularly interested in the effective deployment of models and the integration of IoT with deep learning. He offers insights into the challenges and strategies involved in deploying models in constrained environments, such as remote areas with limited power, and utilizing small devices like Arduino Nano.

// Bio

Soham leads the machine learning team at Sleek, where he builds tools for automated accounting and back-office management. As an electrical engineer, Soham has a passion for the intersection of machine learning and electronics, specifically TinyML/Edge Computing. He has several courses on MLOps and TinyMLOps available on Udacity and LinkedIn, with more courses in the works.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Soham on LinkedIn: https://www.linkedin.com/in/soham-chatterjee

Timestamps: