AI Engineering Podcast

Latest Episodes:

Shedding Light On Silent Model Failures With NannyML Sep 14, 2022

Show notes

Summary
Because machine learning models are constantly interacting with inputs from the real world they are subject to a wide variety of failures. The most commonly discussed error condition is concept drift, but there are numerous other ways that things can go wrong. In this episode Wojtek Kuberski explains how NannyML is designed to compare the predicted performance of your model against its actual behavior to identify silent failures and provide context to allow you to determine whether and how urgently to address them.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today!
Your host is Tobias Macey and today I’m interviewing Wojtek Kuberski about NannyML and the work involved in post-deployment data science

Interview

Introduction
How did you get involved in machine learning?
Can you describe what NannyML is and the story behind it?
What is "post-deployment data science"?
- How does it differ from the metrics/monitoring approach to managing the model lifecycle?
- Who is typically responsible for this work? How does NannyML augment their skills?
- What are some of your experiences with model failure that motivated you to spend your time and focus on this problem?
What are the main contributing factors to alert fatigue for ML systems?
What are some of the ways that a model can fail silently?
- How does NannyML detect those conditions?
What are the remediation actions that might be necessary once an issue is detected in a model?
Can you describe how NannyML is implemented?
- What are some of the technical and UX design problems that you have had to address?
- What are some of the ideas/assumptions that you have had to re-evaluate in the process of building NannyML?
What additional capabilities are necessary for supporting less structured data?
Can you describe what is involved in setting up NannyML and how it fits into an ML engineer’s workflow?
- Once a model is deployed, what additional outputs/data can/should be collected to improve the utility of NannyML and feed into analysis of the real-world operation?
What are the most interesting, innovative, or unexpected ways that you have seen NannyML used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on NannyML?
When is NannyML the wrong choice?
What do you have planned for the future of NannyML?

Contact Info

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

How To Design And Build Machine Learning Systems For Reasonable Scale Sep 10, 2022

Show notes

Summary
Using machine learning in production requires a sophisticated set of cooperating technologies. A majority of resources that are available for understanding how to design and operate these platforms are focused on either simple examples that don’t scale, or over-engineered technologies designed for the massive scale of big tech companies. In this episode Jacopo Tagliabue shares his vision for "ML at reasonable scale" and how you can adopt these patterns for building your own platforms.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you.
Your host is Tobias Macey and today I’m interviewing Jacopo Tagliabue about building "reasonable scale" ML systems

Interview

Introduction
How did you get involved in machine learning?
How would you describe the current state of the ecosystem for ML practitioners? (e.g. tool selection, availability of information/tutorials, etc.)
- What are some of the notable changes that you have seen over the past 2 – 5 years?
- How have the evolutions in the data engineering space been reflected in/influenced the way that ML is being done?
What are the challenges/points of friction that ML practitioners have to contend with when trying to get a model into production that isn’t just a toy?
You wrote a set of tutorials and accompanying code about performing ML at "reasonable scale". What are you aiming to represent with that phrasing?
- There is a paradox of choice for any newcomer to ML. What are some of the key capabilities that practitioners should use in their decision rubric when designing a "reasonable scale" system?
- What are some of the common bottlenecks that crop up when moving from an initial test implementation to a scalable deployment that is serving customer traffic?
How much of an impact does the type of ML problem being addressed have on the deployment and scalability elements of the system design? (e.g. NLP vs. computer vision vs. recommender system, etc.)
What are some of the misleading pieces of advice that you have seen from "big tech" tutorials about how to do ML that are unnecessary when running at smaller scales?
You also spend some time discussing the benefits of a "NoOps" approach to ML deployment. At what point do operations/infrastructure engineers need to get involved?
- What are the operational aspects of ML applications that infrastructure engineers working in product teams might be unprepared for?
What are the most interesting, innovative, or unexpected system designs that you have seen for moderate scale MLOps?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on ML system design and implementation?
What are the aspects of ML systems design that you are paying attention to in the current ecosystem?
What advice do you have for additional references or research that ML practitioners would benefit from when designing their own production systems?

Contact Info

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Building A Business Powered By Machine Learning At Assembly AI Sep 09, 2022

Show notes

Summary
The increasing sophistication of machine learning has enabled dramatic transformations of businesses and introduced new product categories. At Assembly AI they are offering advanced speech recognition and natural language models as an API service. In this episode founder Dylan Fox discusses the unique challenges of building a business with machine learning as the core product.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
Your host is Tobias Macey and today I’m interviewing Dylan Fox about building and growing a business with ML as its core offering

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Assembly is and the story behind it?
- For anyone who isn’t familiar with your platform, can you describe the role that ML/AI plays in your product?
What was your process for going from idea to prototype for an AI powered business?
- Can you offer parallels between your own experience and that of your peers who are building businesses oriented more toward pure software applications?
How are you structuring your teams?
On the path to your current scale and capabilities how have you managed scoping of your model capabilities and operational scale to avoid getting bogged down or burnt out?
How do you think about scoping of model functionality to balance composability and system complexity?
What is your process for identifying and understanding which problems are suited to ML and when to rely on pure software?
You are constantly iterating on model performance and introducing new capabilities. How do you manage prototyping and experimentation cycles?
- What are the metrics that you track to identify whether and when to move from an experimental to an operational state with a model?
- What is your process for understanding what’s possible and what can feasibly operate at scale?
Can you describe your overall operational patterns delivery process for ML?
What are some of the most useful investments in tooling that you have made to manage development experience for your teams?
Once you have a model in operation, how do you manage performance tuning? (from both a model and an operational scalability perspective)
What are the most interesting, innovative, or unexpected aspects of ML development and maintenance that you have encountered while building and growing the Assembly platform?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Assembly?
When is ML the wrong choice?
What do you have planned for the future of Assembly?

Contact Info

@YouveGotFox on Twitter
LinkedIn

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Update Your Model's View Of The World In Real Time With Streaming Machine Learning Using River Aug 26, 2022

Show notes

Summary
The majority of machine learning projects that you read about or work on are built around batch processes. The model is trained, and then validated, and then deployed, with each step being a discrete and isolated task. Unfortunately, the real world is rarely static, leading to concept drift and model failures. River is a framework for building streaming machine learning projects that can constantly adapt to new information. In this episode Max Halford explains how the project works, why you might (or might not) want to consider streaming ML, and how to get started building with River.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Your host is Tobias Macey and today I’m interviewing Max Halford about River, a Python toolkit for streaming and online machine learning

Interview

Introduction
How did you get involved in machine learning?
Can you describe what River is and the story behind it?
What is "online" machine learning?
- What are the practical differences with batch ML?
- Why is batch learning so predominant?
- What are the cases where someone would want/need to use online or streaming ML?
The prevailing pattern for batch ML model lifecycles is to train, deploy, monitor, repeat. What does the ongoing maintenance for a streaming ML model look like?
- Concept drift is typically due to a discrepancy between the data used to train a model and the actual data being observed. How does the use of online learning affect the incidence of drift?
Can you describe how the River framework is implemented?
- How have the design and goals of the project changed since you started working on it?
How do the internal representations of the model differ from batch learning to allow for incremental updates to the model state?
In the documentation you note the use of Python dictionaries for state management and the flexibility offered by that choice. What are the benefits and potential pitfalls of that decision?
Can you describe the process of using River to design, implement, and validate a streaming ML model?
- What are the operational requirements for deploying and serving the model once it has been developed?
What are some of the challenges that users of River might run into if they are coming from a batch learning background?
What are the most interesting, innovative, or unexpected ways that you have seen River used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on River?
When is River the wrong choice?
What do you have planned for the future of River?

Contact Info

Email
@halford_max on Twitter
MaxHalford on GitHub

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Using AI To Transform Your Business Without The Headache Using Graft Aug 16, 2022

Show notes

Summary
Machine learning is a transformative tool for the organizations that can take advantage of it. While the frameworks and platforms for building machine learning applications are becoming more powerful and broadly available, there is still a significant investment of time, money, and talent required to take full advantage of it. In order to reduce that barrier further Adam Oliner and Brian Calvert, along with their other co-founders, started Graft. In this episode Adam and Brian explain how they have built a platform designed to empower everyone in the business to take part in designing and building ML projects, while managing the end-to-end workflow required to go from data to production.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Your host is Tobias Macey and today I’m interviewing Brian Calvert and Adam Oliner about Graft, a cloud-native platform designed to simplify the work of applying AI to business problems

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Graft is and the story behind it?
What is the core thesis of the problem you are targeting?
- How does the Graft product address that problem?
- Who are the personas that you are focused on working with both now in your early stages and in the future as you evolve the product?
What are the capabilities that can be unlocked in different organizations by reducing the friction and up-front investment required to adopt ML/AI?
- What are the user-facing interfaces that you are focused on providing to make that adoption curve as shallow as possible?
  - What are some of the unavoidable bits of complexity that need to be surfaced to the end user?
Can you describe the infrastructure and platform design that you are relying on for the Graft product?
- What are some of the emerging "best practices" around ML/AI that you have been able to build on top of?
  - As new techniques and practices are discovered/introduced how are you thinking about the adoption process and how/when to integrate them into the Graft product?
- What are some of the new engineering challenges that you have had to tackle as a result of your specific product?
Machine learning can be a very data and compute intensive endeavor. How are you thinking about scalability in a multi-tenant system?
- Different model and data types can be widely divergent in terms of the cost (monetary, time, compute, etc.) required. How are you thinking about amortizing vs. passing through those costs to the end user?
Can you describe the adoption/integration process for someone using Graft?
- Once they are onboarded and they have connected to their various data sources, what is the workflow for someone to apply ML capabilities to their problems?
One of the challenges about the current state of ML capabilities and adoption is understanding what is possible and what is impractical. How have you designed Graft to help identify and expose opportunities for applying ML within the organization?
What are some of the challenges of customer education and overall messaging that you are working through?
What are the most interesting, innovative, or unexpected ways that you have seen Graft used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Graft?
When is Graft the wrong choice?
What do you have planned for the future of Graft?

Contact Info

Brian
- LinkedIn
Adam
- LinkedIn

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/[CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&utm_medium=rss

Accelerate Development And Delivery Of Your Machine Learning Projects With A Comprehensive Feature Platform Aug 06, 2022

Show notes

Summary
In order for a machine learning model to build connections and context across the data that is fed into it the raw data needs to be engineered into semantic features. This is a process that can be tedious and full of toil, requiring constant upkeep and often leading to rework across projects and teams. In order to reduce the amount of wasted effort and speed up experimentation and training iterations a new generation of services are being developed. Tecton first built a feature store to serve as a central repository of engineered features and keep them up to date for training and inference. Since then they have expanded the set of tools and services to be a full-fledged feature platform. In this episode Kevin Stumpf explains the different capabilities and activities related to features that are necessary to maintain velocity in your machine learning projects.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you.
Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today!
Your host is Tobias Macey and today I’m interviewing Kevin Stumpf about the role of feature platforms in your ML engineering workflow

Interview

Introduction
How did you get involved in machine learning?
Can you describe what you mean by the term "feature platform"?
- What are the components and supporting capabilities that are needed for such a platform?
How does the availability of engineered features impact the ability of an organization to put ML into production?
What are the points of friction that teams encounter when trying to build and maintain ML projects in the absence of a fully integrated feature platform?
Who are the target personas for the Tecton platform?
- What stages of the ML lifecycle does it address?
Can you describe how you have designed the Tecton feature platform?
- How have the goals and capabilities of the product evolved since you started working on it?
What is the workflow for an ML engineer or data scientist to build and maintain features and use them in the model development workflow?
What are the responsibilities of the MLOps stack that you have intentionally decided not to address?
- What are the interfaces and extension points that you offer for integrating with the other utilities needed to manage a full ML system?
You wrote a post about the need to establish a DevOps approach to ML data. In keeping with that theme, can you describe how to think about the approach to testing and validation techniques for features and their outputs?
What are the most interesting, innovative, or unexpected ways that you have seen Tecton/Feast used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Tecton?
When is Tecton the wrong choice?
What do you have planned for the future of the Tecton feature platform?

Contact Info

LinkedIn
@kevinmstumpf on Twitter
kevinstumpf on GitHub

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/[CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&utm_medium=rss

Build Better Models Through Data Centric Machine Learning Development With Snorkel AI Jul 29, 2022

Show notes

Summary
Machine learning is a data hungry activity, and the quality of the resulting model is highly dependent on the quality of the inputs that it receives. Generating sufficient quantities of high quality labeled data is an expensive and time consuming process. In order to reduce that time and cost Alex Ratner and his team at Snorkel AI have built a system for powering data-centric machine learning development. In this episode he explains how the Snorkel platform allows domain experts to create labeling functions that translate their expertise into reusable logic that dramatically reduces the time needed to build training data sets and drives down the total cost.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today!
Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
Your host is Tobias Macey and today I’m interviewing Alex Ratner about Snorkel AI, a platform for data-centric machine learning workflows powered by programmatic data labeling techniques

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Snorkel AI is and the story behind it?
What are the problems that you are focused on solving?
- Which pieces of the ML lifecycle are you focused on?
How did your experience building the open source Snorkel project and working with the community inform your product direction for Snorkel AI?
- How has the underlying Snorkel project evolved over the past 4 years?
What are the deciding factors that an organization or ML team need to consider when evaluating existing labeling strategies against the programmatic approach that you provide?
- What are the features that Snorkel provides over and above managing code execution across the source data set?
Can you describe what you have built at Snorkel AI and how it is implemented?
- What are some of the notable developments of the ML ecosystem that had a meaningful impact on your overall product vision/viability?
Can you describe the workflow for an individual or team who is using Snorkel for generating their training data set?
- How does Snorkel integrate with the experimentation process to track how changes to labeling logic correlate with the performance of the resulting model?
What are some of the complexities involved in designing and testing the labeling logic?
- How do you handle complex data formats such as audio, video, images, etc. that might require their own ML models to generate labels? (e.g. object detection for bounding boxes)
With the increased scale and quality of labeled data that Snorkel AI offers, how does that impact the viability of autoML toolchains for generating useful models?
How are you managing the governance and feature boundaries between the open source Snorkel project and the business that you have built around it?
What are the most interesting, innovative, or unexpected ways that you have seen Snorkel AI used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Snorkel AI?
When is Snorkel AI the wrong choice?
What do you have planned for the future of Snorkel AI?

Contact Info

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Declarative Machine Learning For High Performance Deep Learning Models With Predibase Jul 21, 2022

Show notes

Summary
Deep learning is a revolutionary category of machine learning that accelerates our ability to build powerful inference models. Along with that power comes a great deal of complexity in determining what neural architectures are best suited to a given task, engineering features, scaling computation, etc. Predibase is building on the successes of the Ludwig framework for declarative deep learning and Horovod for horizontally distributing model training. In this episode CTO and co-founder of Predibase, Travis Addair, explains how they are reducing the burden of model development even further with their managed service for declarative and low-code ML and how they are integrating with the growing ecosystem of solutions for the full ML lifecycle.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today!
Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you.
Your host is Tobias Macey and today I’m interviewing Travis Addair about Predibase, a low-code platform for building ML models in a declarative format

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Predibase is and the story behind it?
Who is your target audience and how does that focus influence your user experience and feature development priorities?
How would you describe the semantic differences between your chosen terminology of "declarative ML" and the "autoML" nomenclature that many projects and products have adopted?
- Another platform that launched recently with a promise of "declarative ML" is Continual. How would you characterize your relative strengths?
Can you describe how the Predibase platform is implemented?
- How have the design and goals of the product changed as you worked through the initial implementation and started working with early customers?
- The operational aspects of the ML lifecycle are still fairly nascent. How have you thought about the boundaries for your product to avoid getting drawn into scope creep while providing a happy path to delivery?
Ludwig is a core element of your platform. What are the other capabilities that you are layering around and on top of it to build a differentiated product?
In addition to the existing interfaces for Ludwig you created a new language in the form of PQL. What was the motivation for that decision?
- How did you approach the semantic and syntactic design of the dialect?
- What is your vision for PQL in the space of "declarative ML" that you are working to define?
Can you describe the available workflows for an individual or team that is using Predibase for prototyping and validating an ML model?
- Once a model has been deemed satisfactory, what is the path to production?
How are you approaching governance and sustainability of Ludwig and Horovod while balancing your reliance on them in Predibase?
What are some of the notable investments/improvements that you have made in Ludwig during your work of building Predibase?
What are the most interesting, innovative, or unexpected ways that you have seen Predibase used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Predibase?
When is Predibase the wrong choice?
What do you have planned for the future of Predibase?

Contact Info

LinkedIn
tgaddair on GitHub
@travisaddair on Twitter

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Stop Feeding Garbage Data To Your ML Models, Clean It Up With Galileo Jul 14, 2022

Show notes

Summary
Machine learning is a force multiplier that can generate an outsized impact on your organization. Unfortunately, if you are feeding your ML model garbage data, then you will get orders of magnitude more garbage out of it. The team behind Galileo experienced that pain for themselves and have set out to make data management and cleaning for machine learning a first class concern in your workflow. In this episode Vikram Chatterji shares the story of how Galileo got started and how you can use their platform to fix your ML data so that you can get back to the fun parts.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you.
Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started!
Your host is Tobias Macey and today I’m interviewing Vikram Chatterji about Galileo, a platform for uncovering and addressing data problems to improve your model quality

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Galileo is and the story behind it?
Who are the target users of the platform and what are the tools/workflows that you are replacing?
- How does that focus inform and influence the design and prioritization of features in the platform?
What are some of the real-world impacts that you have experienced as a result of the kinds of data problems that you are addressing with Galileo?
Can you describe how the Galileo product is implemented?
- What are some of the assumptions that you had formed from your own experiences that have been challenged as you worked with early design partners?
The toolchains and model architectures of any given team is unlikely to be a perfect match across departments or organizations. What are the core principles/concepts that you have hooked into in order to provide the broadest compatibility?
- What are the model types/frameworks/etc. that you have had to forego support for in the early versions of your product?
Can you describe the workflow for someone building a machine learning model and how Galileo fits across the various stages of that cycle?
- What are some of the biggest difficulties posed by the non-linear nature of the experimentation cycle in model development?
What are some of the ways that you work to quantify the impact of your tool on the productivity and profit contributions of an ML team/organization?
What are the most interesting, innovative, or unexpected ways that you have seen Galileo used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Galileo?
When is Galileo the wrong choice?
What do you have planned for the future of Galileo?

Contact Info

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Build Better Machine Learning Models With Confidence By Adding Validation With Deepchecks Jul 06, 2022

Show notes

Summary
Machine learning has the potential to transform industries and revolutionize business capabilities, but only if the models are reliable and robust. Because of the fundamental probabilistic nature of machine learning techniques it can be challenging to test and validate the generated models. The team at Deepchecks understands the widespread need to easily and repeatably check and verify the outputs of machine learning models and the complexity involved in making it a reality. In this episode Shir Chorev and Philip Tannor explain how they are addressing the problem with their open source deepchecks library and how you can start using it today to build trust in your machine learning applications.
Announcements

Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.
Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you.
Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out!
Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today!
Your host is Tobias Macey and today I’m interviewing Shir Chorev and Philip Tannor about Deepchecks, a Python package for comprehensively validating your machine learning models and data with minimal effort.

Interview

Introduction
How did you get involved in machine learning?
Can you describe what Deepchecks is and the story behind it?
Who is the target audience for the project?
- What are the biggest challenges that these users face in bringing ML models from concept to production and how does DeepChecks address those problems?
In the absence of DeepChecks how are practitioners solving the problems of model validation and comparison across iteratiosn?
- What are some of the other tools in this ecosystem and what are the differentiating features of DeepChecks?
What are some examples of the kinds of tests that are useful for understanding the "correctness" of models?
- What are the methods by which ML engineers/data scientists/domain experts can define what "correctness" means in a given model or subject area?
In software engineering the categories of tests are tiered as unit -> integration -> end-to-end. What are the relevant categories of tests that need to be built for validating the behavior of machine learning models?
How do model monitoring utilities overlap with the kinds of tests that you are building with deepchecks?
Can you describe how the DeepChecks package is implemented?
- How have the design and goals of the project changed or evolved from when you started working on it?
- What are the assumptions that you have built up from your own experiences that have been challenged by your early users and design partners?
Can you describe the workflow for an individual or team using DeepChecks as part of their model training and deployment lifecycle?
Test engineering is a deep discipline in its own right. How have you approached the user experience and API design to reduce the overhead for ML practitioners to adopt good practices?
What are the interfaces available for creating reusable tests and composing test suites together?
What are the additional services/capabilities that you are providing in your commercial offering?
- How are you managing the governance and sustainability of the OSS project and balancing that against the needs/priorities of the business?
What are the most interesting, innovative, or unexpected ways that you have seen DeepChecks used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on DeepChecks?
When is DeepChecks the wrong choice?
What do you have planned for the future of DeepChecks?

Contact Info

Shir
- LinkedIn
- shir22 on GitHub
Philip
- LinkedIn
- @philiptannor on Twitter

Parting Question

From your perspective, what is the biggest barrier to adoption of machine learning today?

Closing Announcements

Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0