Agentic Conversations (formally mlops.community)

Latest Episodes:

[Exclusive] Tecton Round-table // Get your ML Application Into Production Dec 07, 2023

Show notes

Join our conference: https://home.mlops.community/public/events/llms-in-production-part-iii-2023-10-03

MLOps Coffee Sessions Special episode with Tecton, Get your ML Application Into Production, sponsored by Tecton. // Abstract Getting an ML application into production is more difficult than most teams expect—but with the right preparation, it can be done efficiently! Join us for this exclusive roundtable, where 4 machine learning experts from Tecton will discuss some of the most common challenges and best practices to avoid them. With over 35 years of combined experience in MLOps at companies like AWS, Google, Lyft, and Uber, and 15 years of experience at Tecton spent helping customers like FanDuel, Plaid, and HelloFresh getting ML models into production, the presenters will share how factors like organizational structure, use cases, tech stack, and more, can create different types of bottlenecks. They’ll also share best practices and lessons learned throughout their careers on how to overcome these challenges. // Bio Kevin Stumpf Kevin co-founded Tecton where he leads a world-class engineering team that is building a next-generation feature store for operational Machine Learning. Kevin and his co-founders built deep expertise in operational ML platforms while at Uber, where they created the Michelangelo platform that enabled Uber to scale from 0 to 1000's of ML-driven applications in just a few years. Prior to Uber, Kevin founded Dispatcher, with the vision to build the Uber for long-haul trucking. Kevin holds an MBA from Stanford University and a Bachelor's Degree in Computer and Management Sciences from the University of Hagen. Outside of work, Kevin is a passionate long-distance endurance athlete. Derek Salama Derek is currently a Senior Product Manager at Tecton, where he is responsible for security, collaboration experience, and Feature Platform infrastructure. Prior to Tecton, Derek worked at Google and Lyft across both ML infrastructure and ML applications. Eddie Esquivel Eddie Esquivel is a Solutions Architect at Tecton, where he helps customers implement feature stores as part of their stack for operational ML. Prior to Tecton, Eddie was a Solutions Architect at AWS. He holds a Bachelor’s Degree in Computer Science & Engineering from the University of California, Los Angeles. Isaac Cameron Isaac Cameron is a Consulting Architect at Tecton. Prior to Tecton, he was a Principal Solutions Architect at Slalom Build, focusing on data and machine learning, where he built his own feature platform for a large U.S. airline and has enabled many organizations to build intelligent products leveraging operational ML. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Kevin on LinkedIn: https://www.linkedin.com/in/kevinstumpf/ Connect with Derek on LinkedIn: https://www.linkedin.com/in/dereksalama/ Connect with Eddie on LinkedIn: https://www.linkedin.com/in/eddie-esquivel-2016/ Connect with Isaac on LinkedIn: https://www.linkedin.com/in/isaaccameron/ Timestamps: [00:00] Introduction to Kevin Stumpf, Derek Salama, Eddie Esquivel, and Isaac Cameron [02:48] Challenges of traditional classical ML into production [10:21] Infrastructure cost [16:50] Bridging Business and Tech [19:23] ML Infrastructure Essentials [29:38] Integrated Batch and Stream [35:12] Scaling AI from Zero [36:23] Stacks red flags [45:53] Tecton: Features Quality Monitoring [49:06] Building Recommender System Tools [53:19] Quantify business value in ML [54:40] Wrap up

DSPy: Transforming Language Model Calls into Smart Pipelines // Omar Khattab // #194 Dec 05, 2023

Show notes

MLOps podcast #194 with Omar Khattab, PhD Candidate at Stanford, DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines.

// Abstract

The ML community is rapidly exploring techniques for prompting language models (LMs) and for stacking them into pipelines that solve complex tasks. Unfortunately, existing LM pipelines are typically implemented using hard-coded "prompt templates", i.e., lengthy strings discovered via trial and error. Toward a more systematic approach for developing and optimizing LM pipelines, we introduce DSPy, a programming model that abstracts LM pipelines as text transformation graphs, i.e., imperative computational graphs where LMs are invoked through declarative modules. DSPy modules are parameterized, meaning they can learn (by creating and collecting demonstrations) how to apply compositions of prompting, finetuning, augmentation, and reasoning techniques. We design a compiler that will optimize any DSPy pipeline to maximize a given metric. We conduct two case studies, showing that succinct DSPy programs can express and optimize sophisticated LM pipelines that reason about math word problems, tackle multi-hop retrieval, answer complex questions, and control agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot prompting and pipelines with expert-created demonstrations. On top of that, DSPy programs compiled to open and relatively small LMs like 770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely on expert-written prompt chains for proprietary GPT-3.5. DSPy is available as open source at https://github.com/stanfordnlp/dspy

// Bio

Omar Khattab is a PhD candidate at Stanford and an Apple PhD Scholar in AI/ML. He builds retrieval models as well as retrieval-based NLP systems, which can leverage large text collections to craft knowledgeable responses efficiently and transparently. Omar is the author of the ColBERT retrieval model, which has been central to the development of the field of neural retrieval, and author of several of its derivative NLP systems like ColBERT-QA and Baleen. His recent work includes the DSPy framework for solving advanced tasks with language models (LMs) and retrieval models (RMs).

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://omarkhattab.com/DSPy https://github.com/stanfordnlp/dspy

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Omar on Twitter: https://twitter.com/lateinteraction

Timestamps:

[00:00] Omar's preferred coffee

[00:26] Takeaways

[06:40] Weight & Biases Ad

[09:00] Omar's tech background

[13:35] Evolution of RAG

[16:33] Complex retrievals

[21:32] Vector Encoding for Databases

[23:50] BERT vs New Models

[28:00] Resilient Pipelines: Design Principles

[33:37] MLOps Workflow Challenges

[36:15] Guiding LLMs for Tasks

[37:40] Large Language Models: Usage and Costs

[41:32] DSPy Breakdown

[51:05] AI Compliance Roundtable

[55:40] Fine-Tuning Frustrations and Solutions

[57:27] Fine-Tuning Challenges in ML

[1:00:55] Versatile GPT-3 in Agents

[1:03:53] AI Focus: DSP and Retrieval

[1:04:55] Commercialization plans

[1:05:27] Wrap up

Fireside Chat with LLM Startups // LLMs in Production Conference 3 Dec 01, 2023

Show notes

// Abstract Martian is focused on building a model router to dynamically route every prompt to the best LLM for the highest performance and lowest cost. Corti, the Al Co-Pilot for health care uses Al to improve patient care, demonstrating the potential of Al in healthcare and medical decision-making. They recently raised $60M, with Prosus being one of the lead investors. Transforms is pioneering in synthetic entertainment, showing how Al can transform the way we create and consume media. Moderator: Paul van der Boor // Speakers Sandeep Bakshi Head of Investments, Europe @prosusgroup3707 Shriyash Upadhyay Founder @Martian Lars Maaløe Co-Founder & CTO at Corti | Adj. Assoc. Professor of Machine Learning @ Corti Pietro Gagliano President & Founder @Transitional Forms // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

LLMs in Biomaterials Production // Pierre Salvy // #193 Nov 28, 2023

Show notes

MLOps podcast #193 with Pierre Salvy, Head of Engineering at Cambrium, LLM in Material Production, co-hosted by Stephen Batifol.

// Abstract

Delve into the world of proteins, genetic engineering, and the intersection of AI and biotech. Pierre explains how his company is using advanced models to design proteins with specific properties, even creating a vegan collagen for cosmetics. By harnessing the potential of AI, they aim to revolutionize sustainability, uncovering a future of lab-grown meats, molecular cheese, and less harmful plastics, confronting regulatory barriers, and decoding the syntax and grammar of proteins.

// Bio

Head of Engineering at Cambrium, a biotech company utilising genAI to design sustainable protein biomaterials for the future. Pierre spent the last decade researching ways to make computers calculate better biological systems. This is a critical step to engineering more sustainable ways to make the products we use every day, which is their mission at Cambrium.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: cambrium.bio

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/

Connect with Pierre on LinkedIn: https://www.linkedin.com/in/psalvy/

Timestamps:

[00:00] Pierre's preferred coffee

[00:10] Takeaways

[05:10] Please like, share, and subscribe to our MLOps channels!

[05:25] Weights and Biases ad

[07:52] Ski story

[09:54] Pierre's career trajectory

[13:35] From employee #2 to hiring a team

[14:42] From employee #2 to head of engineering

[15:50] Uncomfortable things to say are essential for growth and effectiveness

[18:27] From biotech to engineering

[21:10] LLMs at Cambrium

[24:26] Slackbot

[25:43] Quick and Easy Solutions

[26:47] Products created at Cambrium

[31:56] Impact of EU Regulation on Cambrium

[35:39] 2nd Biotech Winter

[36:35] Cost of error vs service not working

[38:00] Protein Synthesis and Mutations

[40:03] Large-Scale System Engineering Challenges

[43:28] Expensive Factors in Experiments

[44:39] LLMs vs Protein Models

[47:03] Protein Design with LLMs

[49:43] Eco-Friendly Product Vision

[53:28] Space glue

[54:00] Wrap up

Product Engineering for LLMs // LLMs in Production Conference Part III // Panel 2 Nov 24, 2023

Show notes

// Abstract A product-minded engineering perspective on UX/design patterns, product evaluation, and building with AI. // Bio Charles Frye Charles teaches people how to build ML applications. After doing research in psychopharmacology and neurobiology, he pivoted to artificial neural networks and completed a PhD at the University of California, Berkeley in 2020. He then worked as an educator at Weights & Biases before joining @Full Stack Deep Learning, an online community and MOOC for building with ML. Sahar Mor Sahar is a Product Lead at @stripe with 15y of experience in product and engineering roles. At Stripe, he leads the adoption of LLMs and the Enhanced Issuer Network - a set of data partnerships with top banks to reduce payment fraud. Prior to Stripe he founded a document intelligence API company, was a founding PM in a couple of AI startups, including an accounting automation startup (Zeitgold, acq'd by Deel), and served in the elite intelligence unit 8200 in engineering roles. Sahar authors a weekly AI newsletter (AI Tidbits) and maintains a few open-source AI-related libraries (https://github.com/saharmor). Sarah Guo Sarah Guo is the Founder and Managing Partner at @Conviction, a venture capital firm founded in 2022 to invest in intelligent software, or "Software 3.0." Prior, she spent a decade as a General Partner at Greylock Partners. She has been an early investor or advisor to 40+ companies in software, fintech, security, infrastructure, fundamental research, and AI-native applications. Sarah is from Wisconsin, has four degrees from the University of Pennsylvania, and lives in the Bay Area with her husband and two daughters. She co-hosts the AI podcast "No Priors" with Elad Gil. Shyamala Prayaga Shyamala is a seasoned conversational AI expert. Having led initiatives across connected home, automotive, wearables - just to name a few, she's put her work on research into usability, accessibility, speech recognition, multimodal voice user interfaces, and has even been published internationally across publications like Forbes. Outside of her research, she's spent the last 18 years designing mobile, web, desktop, and smart TV interfaces and has most recently joined @NVIDIA to work on deep learning product suites. Willem Pienaar Willem is the creator of @Feast, the open-source feature store and a builder in the generative AI space. Previously Willem was an engineering manager at Tecton where he led teams in both their open source and enterprise initiatives. Before that Willem built the core ML systems and created the ML platform team at Gojek, the Indonesian decacorn. // Sign up for our Newsletter to never miss an event: https://mlops.community/join/ // Watch all the conference videos here: https://home.mlops.community/home/collections // Check out the MLOps Community podcast: https://open.spotify.com/show/7wZygk3mUUqBaRbBGB1lgh?si=242d3b9675654a69 // Read our blog: mlops.community/blog // Join an in-person local meetup near you: https://mlops.community/meetups/ // MLOps Swag/Merch: https://mlops-community.myshopify.com/ // Follow us on Twitter: https://twitter.com/mlopscommunity //Follow us on Linkedin: https://www.linkedin.com/company/mlopscommunity/

Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines // Chris Van Pelt // #192 Nov 21, 2023

Show notes

MLOps podcast #192 with Chris Van Pelt, CISO and co-founder of Weights & Biases, Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines sponsored by Weights & Biases.

// Abstract

Chris provides insights into his machine learning (ML) journey, emphasizing the significance of ML evaluation processes and the evolving landscape of MLOps. The conversation covers effective evaluation metrics, demo-driven development nuances, and the complexities of ML Ops pipelines. Chris reflects on his experience with Crowdflower, detailing its transition to Weights and Biases and stressing the early integration of security measures. The discussion extends to the transformative impact of ML on the tech industry, challenges in detecting subtle bugs, and the potential of open-source models and multimodal capabilities.

// Bio

Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career to optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://wandb.ai/site

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/

Timestamps:

[00:00] Chris' preferred coffee

[00:33] Takeaways

[03:50] Huge shout-out to Weights & Biases for sponsoring this episode!

[04:15] Please like, share, and subscribe to our MLOps channels!

[04:25] CrowdFlower

[07:02] Difference between CrowdFlower and Trajectory

[09:13] Transition from CrowdFlower to Weights & Biases

[13:05] Excel spreadsheets being passed around via email

[15:45] Evolution of Weights & Biases

[19:24] CISO role

[22:23] Advise for easy wins

[25:32] Transition into LLMs

[27:36] Prompt injection risks on data

[29:42] LLMs for New Personas

[34:42] Iterative Value Evaluation Process

[36:36] Iterating on New Release

[39:31] Evaluation survey

[43:21] Landscape of LLMs and its evolution

[45:40] Conan O'Brien

[46:48] Wrap up

Building Defensible AI Apps // Gregory Kamradt // #191 Nov 17, 2023

Show notes

MLOps podcast #191 with Gregory Kamradt, Founder of Data Independent, Building Defensible AI Apps sponsored by Milvus Vector Database.

// Abstract

Demetrios engages in a captivating conversation with Gregory Kamradt, an AI visionary deeply immersed in technology and product development. The discussion spans various challenges businesses encounter in implementing AI, the transformative potential of AI in revolutionizing business processes, and the growth and possibilities associated with OpenAI. Gregory shares insights into his latest project, a smart companion app designed to analyze and summarize startup pitches. The episode unfolds as a rich source of knowledge, exploring diverse topics such as AI experimentation, the concept of an AI gateway, the future of finely tuned models for niche applications, and insights into the intricate landscape of AI within big tech, including Google's strategic direction and OpenAI's copyright protection measures.

// Bio

Greg has mentored thousands of developers and founders, empowering them to build AI-centric applications. By crafting tutorial-based content, Greg aims to guide everyone from seasoned builders to ambitious indie hackers. Greg partners with companies during their product launches, feature enhancements, and funding rounds. His objective is to cultivate not just awareness, but also a practical understanding of how to optimally utilize a company's tools. He previously led Growth @ Salesforce for Sales & Service Clouds in addition to being early on at Digits, a FinTech Series-C company.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://gregkamradt.com/

Greg Kamradt (Data Indy): https://www.youtube.com/@DataIndependent

Milvus Vector Database: https://zilliz.com/what-is-milvus

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkamradt/

Timestamps:

[00:00] Greg's preferred coffee

[00:12] Takeaways

[02:56] Quick word from our sponsor

[04:22] DevDay

[06:19] YouTube's unique perspective on the technological revolution

[09:34] GPT assistance

[13:36] AI Streamlining Fax Orders

[18:13] AI Marketplace Dynamics: GPT vs. Specialized

[22:04] Data Tooling Platform Challenges

[27:17] The Shield against copyright

[29:27] Llama Index vs OpenAI

[31:56] DS Pie and Compiler Tangent

[34:31] Orchestration Layer is dead!

[36:49] Personalized AI Models: Understanding Integration

[38:00] AI Defensibility

[43:00] Green Field AI Opportunities

[46:57] LLMs for live event pitch

[53:38] Exciting content creation process

[58:03] New context window benchmark

[1:02:23] AI Gateway

[1:04:35] Wrap up

Guarding LLM and NLP APIs: A Trailblazing Odyssey for Enhanced Security // Ads Dawson // #190 Nov 14, 2023

Show notes

MLOps podcast #190 with Ads Dawson, Senior Security Engineer at Cohere, Guarding LLM and NLP APIs: A Trailblazing Odyssey for Enhanced Security.

// Abstract

Ads Dawson, a seasoned security engineer at Cohere, explores the challenges and solutions in securing large language models (LLMs) and natural language programming APIs. Drawing on his extensive experience, Ads discusses approaches to threat modeling LLM applications, preventing data breaches, defending against attacks, and bolstering the security of these critical technologies. The presentation also delves into the success of the "OWASP Top 10 for Large Language Model Applications" project, co-founded by Ads, which identifies key vulnerabilities in the industry. Notably, Ads owns three of the top 10 vulnerabilities, including Training Data Poisoning, Sensitive Information Disclosure, and Model Theft. This OWASP Top 10 serves as a foundational resource for stakeholders in AI, offering guidance on using, developing, and securing LLM applications. Additionally, the session covers insider news from the AI Village's 'Hack the Future' | LLM Red Teaming event at Defcon31, providing insights into the inaugural Generative AI Red Teaming showdown and its significance in addressing security and privacy concerns amid the widespread adoption of AI.

// Bio

A mainly self-taught, driven, and motivated proficient application, network infrastructure & cyber security professional holding over eleven years experience from start-up to large-size enterprises leading the incident response process and specializing in extensive LLM/AI Security, Web Application Security and DevSecOps protecting REST API endpoints, large-scale microservice architectures in hybrid cloud environments, application source code as well as EDR, threat hunting, reverse engineering, and forensics. Ads have a passion for all things blue and red teams, be that offensive & API security, automation of detection & remediation (SOAR), or deep packet inspection, for example. Ads is also a networking veteran and loves a good PCAP to delve into. One of my favorite things at Defcon is hunting for PWNs at the "Wall of Sheep" village and inspecting malicious payloads and binaries.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://github.com/GangGreenTemperTatum

OWASP Top 10 for Large Language Model Applications Core Team Member and Founder - https://owasp.org/www-project-top-10-for-large-language-model-applications/CoreTeam

Fork for OWASP Top 10 for Large Language Model Applications - https://github.com/GangGreenTemperTatum/www-project-top-10-for-large-language-model-applicationsSecurity project: llmtop10.com

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Ads on LinkedIn: https://www.linkedin.com/in/adamdawson0/

Timestamps:

[00:00] Ads' preferred coffee

[00:46] Takeaways

[02:52] Please like, share, and subscribe to our MLOps channels!

[03:11] Security and vulnerabilities

[05:24] Work at Cohere and OWASP

[08:11] Previous work vs LLMs Companies

[09:46] LLM vulnerabilities

[10:38] Good qualities to combat prompt injection problems

[13:26] Data lineage

[16:03] Red teaming

[19:39] Freakiest LLM vulnerabilities

[22:17] Severe Autonomy Concerns

[25:13] Hallucinations

[27:59] Prompt injection

[29:15] Vector attacks to be recognized

[32:02] LLMs being customized

[33:18] Security changes due to maturity

[38:17] OWASP Top 10 for Large Language Model Applications

[44:31] Gandalf game

[46:06] Prompt injection attack

[49:46] Overlapping security

[53:26] Data poisoning

[56:57] Toxic data for LLMs

[58:50] Wrap up

Designing for Forward Compatibility in Gen AI // Rohit Agarwal // #189 Nov 10, 2023

Show notes

MLOps podcast #189 with Rohit Agarwal, CEO of Portkey.ai, Designing for Forward Compatibility in Gen AI.

// Abstract

For two whole years of working with a large LLM deployment, I always felt uncomfortable. How is my system performing? Are my users liking the outputs? Who needs help? Probabilistic systems can make this really hard to understand. In this talk, we'll discuss practical & implementable items to secure your LLM system and gain confidence while deploying to production.// BioRohit is the Co-founder and CEO of portkey.ai, which is an FMOps stack for monitoring, model management, compliance, and more. Previously, he headed Product & AI at Pepper Content, which has served ~900M generations on LLMs in production. Having seen large LLM deployments in production, he's always happy to help companies build their infra stacks on FM APIs or Open-source models.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://portkey.ai

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Rohit on LinkedIn: https://www.linkedin.com/in/1rohitagarwal/

Timestamps:

[00:00] Rohit's preferred coffee

[00:15] Takeaways

[03:22] Please like, share, and subscribe to our MLOps channels!

[05:16] Rohit's current work

[06:37] The Portkey landscape

[09:13] Compute unit is no longer a Cloud resource; it's a Foundational Model

[11:09] Hang-ups at high-scale models and how to combat them

[15:22] Complexity of the Apps evolving

[19:54] Rohit's working relationships with the agents

[22:52] Fine-tuning reliability

[24:38] Small language models can outperform larger ones

[26:38] Market map at Portkey

[34:37] AI Gateway

[37:59] Worker Bee and Queen Bee

[39:27] Security and Compliance

[43:11] Idea of Data Mesh

[45:57] Forward compatibility

[49:59] Decoupling AI Gateway from the code

[56:05] Hardest design decisions to make since creating Portkey

[58:52] Wrap up

Impact of LLMs on the Tech Stack and Product Development // Anand Das // #188 Nov 07, 2023

Show notes

MLOps podcast #188 with Anand Das, Co-founder and CTO of Bito, Impact of LLMs on the Tech Stack and Product Development.

// Abstract

Anand and his team have developed a fascinating Chrome extension called "explain code" that has garnered significant attention in the tech community. They have expanded their extension to other platforms like Visual Studio Code and Chat Brains, creating a personal assistant for code generation, explanation, and test case writing.

// Bio

Anand Das is the co-founder and CTO of Bito. Previously, he served as the CTO at Eyeota, which was acquired by Dun & Bradstreet for $165M in 2021. Anand also co-founded and served as the CTO of PubMatic in 2006, a company that went public on NASDAQ in 2020 (NASDAQ: PUBM). Anand has also held various engineering roles at Panta Systems, a high-performance computing startup led by the CTO of Veritas, as well as at Veritas and Symantec, where he worked on a variety of storage and backup products. Anand holds seven patents in systems software, storage software, advertising, and application software.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://bito.ai/

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Anand on LinkedIn: https://www.linkedin.com/in/ananddas/

Timestamps: