NerdOut@Spotify

Join us for Wizards, Portals, and Spotify for Backstage Apr 19, 2024

Register for Spotify’s roadmap webinar on April 30, 2024 — and see what’s coming next from Spotify for Backstage, the open source platform for building internal developer portals. We’ll show you our latest developer tools, including a sneak peek at new Spotify Plugins for Backstage and a first-look at Spotify Portal for Backstage — a full-featured developer portal that is quick and easy for any engineering org to adopt. See demos from Spotify’s team and learn how to apply for the private beta — work with us to build the next great developer portal: yours!

Host and principal engineer Dave Zolotusky has a quick chat with Helen Greul, head of engineering for Backstage at Spotify, about the event. They talk about the CNCF’s recent BackstageCon in Paris, the growing popularity of the Backstage platform, and why the roadmap webinar on April 30 isn’t one to miss for fans of developer experience and wizardry.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

27: Measuring Developer Productivity Apr 18, 2024

Host and principal engineer Dave Zolotusky talks with Kyle Buttner, a product manager on Spotify’s insights team, to discuss Spotify's journey in measuring developer productivity — from how we evaluate different frameworks (like DORA and SPACE) to what kind of data we collect, to the role Backstage plays in unifying our development practices. Can productivity metrics really draw an accurate picture of your engineering org and show you the way to happier and more productive developers?

Learn more about how we measure developer happiness and productivity at Spotify:

How Spotify measures the value of Backstage: Read about how Spotify’s frequent and less frequent Backstage users compare across three metrics: developer activity, development lifecycle, and developer retention. (Plus, a bonus metric on retention!)
How Backstage Made Our Developers More Effective: Here’s a metric that was starting to scare us: new employees were taking up to 60 days before they made their tenth pull request.
Register for the Spotify for Backstage roadmap webinar: Join us on April 30 to learn about our latest tools for improving developer happiness and productivity.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

26: A Trillion Events Feb 08, 2024

How did we learn to do event delivery at scale at Spotify? It’s been a journey. When you do something like tap the play button in the Spotify app, that’s an event. And getting that event data is fundamental to the Spotify experience. Without it, we wouldn’t be able to make music recommendations, pay artists fairly, or track down pesky, hard-to-find bugs. At the most basic level, this seems like a straightforward process: record an event, send that event data to a server somewhere, do something useful with it. Easy, right? But now, multiply that process by 50 million events per second. So, how do we make sure all that important data is delivered reliably, from our client apps to the cloud?

Host and principal engineer Dave Zolotusky talks with 9-year Spotify veteran Riccardo Petrocco about our journey building a event delivery system that can reliably handle a trillion events around the world, moving from Kafka to the cloud, building systems that are simple enough so that nobody tries to find a way around them and encourages “doing the right thing”, the definition of “quality data”, the value of moving up the stack and focusing less on the data pipes and more on what’s in them, and how Backstage makes it easier for our developers to discover, consume, produce, and manage data.

Learn more about Spotify’s data journey:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

25: Voice Translation *Release Notes* Jan 11, 2024

We’ve seen generative AI and large language models do some amazing things in the past year — but how do you go from a tech demo to a real shipping product? In this Release Notes episode of the NerdOut@Spotify podcast, we’ll hear about what it took to ship our Voice Translation pilot, which takes podcasts recorded in English and uses AI to generate the original podcaster’s voice speaking in Spanish (with German and French coming next).

Host Dave Zolotusky talks with senior machine learning engineering manager Sandeep Ghael about how we brought expertise from across the company in order to go from a weekend prototype to releasing fully translated episodes of Lex Fridman, Armchair Expert, and other podcasts — in just six weeks.

Read more about Voice Translation for podcasts:

Spotify’s AI Voice Translation Pilot Means Your Favorite Podcasters Might Be Heard in Your Native Language

Hear the results on Spotify:

Lex Fridman Podcast — “Interview with Yuval Noah Harari”
Armchair Expert — “Kristen Bell, by the grace of god, returns”
The Diary of a CEO with Steven Bartlett — “Interview with Dr. Mindy Pelz”
More episodes on the Voice Translation Hub

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

24: Tesla *Release Notes* Nov 09, 2023

Over the summer, Spotify helped Tesla engineers ship a major update to their built-in media player. In this Release Notes episode, host Dave Zolotusky talks with Spotify engineering manager Geetika Arora and senior product designer JC Chhim about collaborating with Tesla to improve the in-car listening experience, the value of having a familiar user experience across devices, and how there’s more to a great collaboration than just picking the right SDK for the job.

Introducing Release Notes — a new series of mini episodes on the NerdOut@Spotify podcast. There are hundreds of teams at Spotify working on so many different things — from playlists that change throughout the day, to realistic voice translations, to a smarter way to shuffle songs. In each episode of Release Notes, we focus on one thing we shipped and what went into building it. You’ll see these mini episodes from time to time in the main podcast feed right alongside our regular episodes.

Learn more about our SDKs on the Spotify for Developers site: developer.spotify.com

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

23: Searching for Neighbors with Voyager Oct 19, 2023

How do you get a machine to find a song that’s similar to another song? What properties of the song should it look for? And then does it just compare each track to every other track, one by one, until it finds the closest match? When you have a catalog of 100 million different music tracks, like we do at Spotify, that would take a long time. So, for these kinds of problems, we use a technique known as nearest neighbor search (NNS). This past summer at Spotify, we built a new library for nearest neighbor search: It’s called Voyager — and we open sourced it.

Host and principal engineer Dave Zolotusky talks with Peter Sobot and Mark Koh, two of the machine learning engineers who developed Voyager. They discuss using nearest neighbor search for recommendations and personalization, how to go from searching for vectors in a 2D space to searching for them in a space with thousands of dimensions, the relative funkiness and danceability of Mozart and Bach, how to find a place on a map when you don’t have the exact coordinates, tricky acronyms (Annoy: “Approximate Nearest Neighbor Oh Yeah”) and initialisms (HNSW: “Hierarchical Navigable Small World”), why we stopped using our old NNS library, why we open sourced the new one, how it works for use cases beyond music (like LLMs), and looking for ducks in grass.

Learn more about Spotify Voyager:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com
You should follow us on Twitter @SpotifyEng and on LinkedIn!

22: Declarative Infra and Beyond Aug 24, 2023

In the very old days, if you needed more storage for your database, you had to walk into the data center and install another server. Now you can just log into your cloud provider’s console and click a few buttons. Voilá, more storage. So easy! But what if you’re replicating that storage configuration for hundreds of databases at once? Suddenly, that’s a lot more clicking. Not so easy! (Plus, very tedious and very error prone.) So instead of living with this “ClickOps” approach, we developed a declarative infrastructure model — our very own “infrastructure as code” solution for managing cloud resources at Spotify scale. Instead of manually configuring each resource, developers just describe the state they want. And once we adopted declarative infra, we unlocked ways to improve not just how we manage resources, but also how we update policies, manage dependencies, and make other changes to code across our entire fleet of repos — quickly, safely, easily. In other words, programmatically.

Host Dave Zolotusky talks with David Flemström — who went from pushing the limits of Spotify’s infrastructure as a feature developer to working on the platform team in order to improve infrastructure for all of our developers. The two Daves discuss what declarative infrastructure means at Spotify, our journey to adopting it (going from Puppet to cloud consoles, to something better than both) and why we did it, how our model works (Kubernetes!), how it changed the relationship between our feature teams and our platform teams, how this shift helped enable Fleet Management at Spotify, and where we’re going next with abstracting infrastructure so that it helps our engineers do more, more easily.

Learn more about declarative infrastructure and Fleet Management:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

21: The Man Who Killed ABBA Aug 03, 2023

Last episode, we talked about ABBA, our first A/B testing tool. We used it to test UI changes, new features, content recommendations — anything and everything we could think of. ABBA was so good and worked so well for so long…that we decided to get rid of it. Years of using ABBA taught us what makes for good experimentation, and we eventually realized we needed a better tool, built from scratch. Listen to find out why we pulled the plug on ABBA and how Spotify’s Experimentation Platform was born. And in case you missed it, a version of our internal platform will be available to the public as Confidence, a new enterprise product for developer teams — read today’s announcement: “Coming Soon: Confidence — An Experimentation Platform from Spotify”.

But first, let’s talk buttons. Everyone always has so many questions about buttons. How do you know which color they should be? Or how big they should be? Or whether the corners should be round or square? The easy answer: an A/B test! But if only all product experimentation was as simple as testing buttons. Senior staff engineer Mark Grey returns to talk with host Dave Zolotusky, along with senior engineer Dima Kunin — he helped build Spotify’s Experimentation Platform and was the guy who had the honor of finally retiring ABBA. They discuss the ins and outs of enabling experimentation at scale, including targeting criteria, controlling eligibility, the importance of measuring exposure, using properties instead of feature flags, the advantages of separating your app configuration from your experiments, fallback states, sample ratio mismatches — and all the other questions you have to answer about your experimentation process before you can even ask something as simple as “what color should a button be” — let alone “will this machine learning model consistently provide recommendations users appreciate over the next year”.

Plus, did you definitely, positively, absolutely eat the bread? Or did you just buy the bread? And a bonus trick question: What’s the difference between “treatments”, “variants”, and “groups” — and why is it always so hard to name things?

Learn more about ABBA and its successor, Spotify’s Experimentation Platform:

20: The Rise and Fall of ABBA: Listen to our previous episode with Mark Grey talking about our very first A/B testing tool, ABBA.
Spotify’s New Experimentation Platform (Part 1): How we went from ABBA to building EP, the internal experimentation platform we use today and that Confidence is based on.
Spotify’s New Experimentation Platform (Part 2): More features of EP, including our custom “salt machine” and more.

Plus, find out lots more about how we do experimentation at Spotify on our engineering blog — including a little light reading on automated salting and bucket reuse, choosing sequential testing frameworks, comparing quantiles at scale, and how we scale other scientific best practices across the org.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

20: The Rise and Fall of ABBA Jun 22, 2023

Back in the day, Spotify built a custom A/B testing tool called ABBA. It was great. The platform enabled lots of teams to try out lots of ideas for new features to see what worked and what didn’t. With ABBA, we went from doing tens of experiments to hundreds of experiments. But we didn’t just learn what color button users liked better: the more tests we ran, the more we learned about our testing methods, including the limitations of ABBA itself — which eventually led us to a new, better way to test. Here’s the story of ABBA, our very first experimentation platform, and the lessons we learned about doing product experimentation at scale.

Host Dave Zolotusky talks with Mark Grey, a senior staff engineer and 10-year Spotify veteran. They discuss Spotify’s earliest efforts at product testing, our early infrastructure for data and data processing (using Hive and Hadoop), how migrating to the cloud unlocked more processing power (and more testing), the difference between using tests to design the color of a button and using tests to inform the very next user interaction via machine learning, feature flags and holdout groups, all the things we learned about conducting scientifically sound experiments, how we built a culture of experimentation among our software development teams, and what finally drove us to sunset ABBA and build its successor: a bigger, better internal experimentation platform. Plus, progress bars and lightsabers.

Read more about ABBA and how we do product experimentation at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

19: Engineering for Accessibility Jun 01, 2023

We’re often focused on features that improve the experience for users — letting them do something better, faster, smarter. But if you don’t consider accessibility issues, many users won’t be able to use the feature at all. From color contrast and text size to alt text for enabling screen readers and voice commands, accessibility issues can affect everyone, whether you have a permanent disability or just an armful of groceries. So how do you get developers and designers to adopt an accessibility mindset and make it a fundamental part of the development process?

Host Dave Zolotusky talks with Dani Devesa Derksen-Staats, an iOS engineer on Spotify’s accessibility team. We’ll hear how Dani went from a five-year computer science program, where accessibility wasn’t mentioned even once, to becoming so passionate about the topic, he wrote a book on it. They also talk about the basic things we forget to consider when we don’t consider accessibility; how we can all benefit from accessibility improvements, whether that’s getting up a curb in a wheelchair or while pushing a stroller; and ways to address accessibility issues into the development process, from adopting a multimodal approach to UX design to integrating accessibility tests into your CI/CD.

Spotify recently introduced an Accessibility Center. Have questions or concerns about accessibility? Contact us:

Email: accessibility@spotify.com
Write to us at: Spotify USA Inc., 150 Greenwich Street, Floor 62, New York, NY 10007, USA

More on accessibility from Dani:

His book: Developing Accessible iOS Apps (GitHub)
His 2023 talk for Accessibility NYC: Creating Accessible iOS Apps: An Action Plan
His 2022 talk at NSSpain X: 365 Days of Accessibility

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

18: Meet Your Personal AI DJ May 04, 2023

How do you make an AI-generated voice feel more like a real person? You give it a real personality. Spotify recently released a new feature in the US and Canada called DJ. Turn it on, and you hear a curated selection of music and recommendations, along with a fun, friendly, knowledgeable voice telling you more about what you’re listening to (you know, like a real DJ) — except everything you hear is personalized just for you. Learn what makes this DJ feel so realistic and meet the people behind the technology — including the DJ himself.

Host Dave Zolotusky talks with product director Zeena Qureshi and director of engineering John Flynn — together they lead Speak, the team at Spotify responsible for DJ’s realistic, expressive voice. Hear how the AI voice technology they pioneered for Hollywood blockbusters and triple-A video games now brings Spotify’s personalized DJ to life, how they record a range of emotions to create deeper datasets for modeling DJ’s unique personality, the technical challenges of generating and delivering these dynamic performances at the press of a button, and how having your own personal audio guide brings a totally new dimension to the Spotify listening experience. You’ll also hear from the person who provides the raw ingredients for the AI DJ’s voice and soul: the real-life Xavier “X” Jernigan.

Learn more about Spotify’s personalized DJ and the technology behind it:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

17: Building Apps at Spotify Scale Mar 16, 2023

What happens when the standard tooling for iOS and Android just doesn't cut it anymore? What’s it like to maintain an app when there are literally thousands of commits every week? How do you develop your feature without worrying about everybody else’s feature, when at the end of the day, you’re shipping a single, massive app? In other words, what’s it really like to build apps at Spotify’s scale?

Host Dave Zolotusky talks with Patrick Balestra, a staff engineer on our client platform team, whose job is to make sure every Spotify developer has a great experience building, no matter what Spotify app they’re building for. They talk about adopting Bazel, reducing local build times with remote caching, managing multiple monorepos and zillions of dependencies, doing big tech migrations (Objective-C to Swift, Java to Kotlin) without slowing down, open sourcing our tools, and contributing to the Mobile Native Foundation.

Learn more about how we build mobile apps at Spotify:

Kodeco: Building with Bazel: Online tutorial made in collaboration with Spotify
BazelCon 2022: Our BazelCon talk about driving architectural improvements with dependency metrics
XCRemoteCache: Open source remote cache tool for Xcode projects that reuses target artifacts generated on a remote machine, served from a simple REST server
XCLogParser: Open source tool to parse Xcode and xcodebuild logs stored in the xcactivitylog format
2022 Mobile Ecosystem: Data from the Mobile Native Foundation's first mobile ecosystem survey
My Beat: Patrick Balestra
Mobile development posts on the Spotify Engineering Blog

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

16: Backstage Adopters Speak Feb 23, 2023

Whether you’re working at a major financial institution or an audio streaming platform, it turns out engineering teams face a lot of the same challenges to doing their jobs effectively. In this episode, you’ll hear Spotify’s head of technology and platforms, Tyson Singer, discuss developer experience with engineering leaders from Expedia Group, Bank of the West, consulting firm Booz Allen Hamilton, and digital telecom TELUS — all adopters of Backstage, our open platform for building developer portals. This panel discussion took place on December 15, 2022 at the launch event for Spotify Plugins for Backstage, our new developer software bundle.

At the event, Tyson is joined on stage by Guillermo Manzo (Expedia Group), Boyan Vassilev (Bank of the West), Jason Miller (Booz Allen Hamilton), and Nate Axcell (TELUS) to talk about the costs of context switching, breaking down organizational silos, how to get your developers started and how to get them unstuck, open source, inner source, golden paths, and the importance of making sure the best way to do something is the easiest way to do something otherwise it just doesn’t get done.

Learn more about Spotify Plugins for Backstage:

Listen to more episodes about Backstage:

Or check out the links below:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

15: Your Very Own Developer Advocate Feb 02, 2023

There are thousands of developers building great Spotify experiences — who don’t actually work at Spotify. From smart speakers to game consoles to car infotainment systems, how do we support this diverse, creative community building on top of the Spotify platform? Along with providing APIs, SDKs, and other developer tools, it starts with something even more fundamental: listening to and understanding their needs. That’s where our developer advocates come in.

Host Dave Zolotusky talks with Serah Kiburu, a Spotify engineer and senior developer advocate. They discuss how and why Spotify supports third-party developers, the value of being “customer zero” for our own developer tools, our most popular APIs, the importance of listening before you start building, Serah’s journey to Spotify in Stockholm, and how she’s preparing for the robot apocalypse.

Learn more about the resources we provide third-party developers:

Spotify for Developers: Where to find our public APIs, SDKs, libraries, documentation, and community news
Serah Njambi Kiburu and Her Hack Week Team: Read more about Serah and her team’s work, including their project from last year’s Hack Week

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

14: For Love and Money? Jan 12, 2023

From Norway to Nebraska, there are individual developers keeping the open source projects that we all depend on going. Last year, Spotify started an annual fund to help support maintainers of free and open source software (FOSS) — the Spotify FOSS Fund. We earmarked 100,000 Euros and then gave it away to nine projects, no strings attached. One of those recipients was Byte Buddy, a project with millions of users, but just one volunteer maintainer — its creator, Rafael Winterhalter. Hear what it takes to sustain something that started as a passion project in 2014 and then grew to become much more.

Rafael talks with our host Dave Zolotusky and Per Ploug, Spotify’s open source tech lead, who helped set up our FOSS fund. They discuss how Spotify decided which projects to fund, why giving away (and accepting) free money is harder than you might think, and what it’s really like maintaining a popular open source project all on your own.

Learn more about the recipients of the the Spotify FOSS Fund and other ways we’re working to create a more sustainable open source ecosystem:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

13: Advent of Code Live! Dec 15, 2022

Recorded live at midnight EST, December 6, 2022, from Boston, Gothenburg, and Stockholm, via the Spotify Live app. In this special Advent of Code episode, we try to solve the Day 6 puzzle the moment it’s released. Host Dave Zolotusky and Spotify’s chief architect, Niklas Gustavsson (aka, ngn), play against Spotify’s reigning champ, the one and only, unstoppable Jimmy Mårdell. Entering today’s challenge behind ngn in the standings, will Jimmy reclaim his rightful place atop the Spotify leaderboard? Play along with us and find out!

Plus, Spotify engineers share what makes this friendly, low-stakes coding competition so much fun every year, Jimmy answers questions from the live audience (including offering some speed tips), and more.

The Advent of Code season is upon us once again:

Advent of Code 2022 — Day 6: The puzzle we’re attempting to solve in this episode. Play along with us — when you hear the boxing bell ring in the episode, click the link above and see how fast you can come up with a solution. Can you beat us to it?
05: Advent of Code: Listen to our previous episode about this annual Spotify tradition and learn more about the community that’s grown up around it.
adventofcode.com: It’s never too late to join in the fun. Start on Day 1 and see how far you can go.

Special thanks to Eric Wastl for creating such a fun thing.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

12: Fleet First Nov 17, 2022

Instead of performing 10 big software upgrades to our infrastructure every year, what if we did 10,000 small ones, in all our repos, all at once? That’s the idea behind Fleet Management: using automation to take away some of the everyday toil of being a developer, freeing our teams to focus on more interesting problems than migrating to the next version of a low-level software dependency. But for automation at this scale to succeed, our developers would have to be willing to give up some control over their codebases. Could we really get our squad-based culture to adopt a fleet-first mindset?

Spotify’s chief architect, Niklas Gustavsson (aka, ngn), returns to the podcast to talk to host Dave Zolotusky about how we can safely make changes to thousands of repos at once (without bothering the repos’ owners), why adopting Fleet Management at Spotify was a cultural shift as much as a technological one, how having this level of automation in place helped us mitigate the log4j security incident within a few hours, the intricacies of software dependencies, the benefits of tech standards, and more. Welcome to the Fleet.

Learn more:

Launching soon: Spotify Plugins for Backstage: Read about the Soundcheck plugin, which we use to promote consistency and quality across our 10,000 software components.
Large-Scale Automated Refactoring Using ClangMR: Read a research paper on Google’s system for maintaining large monorepos — an approach we considered as we formulated our Fleet Management strategy for a multi-repo environment.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

11: Open Source Work Is Work Oct 27, 2022

If you look at any of our open source projects, you’re bound to see the phrase “Made with 💚 at Spotify.” That’s because many open source projects start off as passion projects. But is passion enough? Now that open source code is so essential to how we all build, it’s important for us to treat that code more seriously. To be sustainable, open source can’t rely on passion alone. It needs a little tough love, too. For that, we got an OSPO — an open source program office — to help our developers (and Spotify) get the most out of their open source experience.

Host Dave Zolotusky talks with Per Ploug, our open source tech lead and head of Spotify’s OSPO. You’ll hear about what developers can learn from open sourcing their projects, why we shouldn’t treat open source like a side gig, how to measure the success of an open source project (by the number of stars on GitHub, of course), what open source at Spotify looked like in the past (the good, the bad, the so-so), and what we hope it looks like tomorrow.

Read more about open source at Spotify on the Spotify Engineering blog:

Listen to more episodes about open source at Spotify:

10: Basic Pitch: Our audio-to-MIDI converter uses a neural network to transcribe the recordings of almost any musical instrument🎺🎸🎻🎤🪗
06: Bits of Math at Scale: Pedalboard is a Python library for applying studio-quality audio effects at scale🎶
04: Backstage in the Wild: Why Netflix and American Airlines both adopted our platform for developer portals for their own engineering teams📺✈️
03: Community Builders: What’s it like when you suddenly find yourself with a popular open source project and a fast-growing, global community of contributors😬
02: Open Issues: How to go from Hack Week project to public repo in two weeks instead of six months😅
01: What Is Backstage?: Why we open sourced our homegrown developer portal and how it improves the everyday experience of developers🧑🏻‍💻

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

10: Basic Pitch Oct 06, 2022

What if you could create a guitar solo just by humming it? That’s Basic Pitch, a new open source project from Spotify’s Audio Intelligence Lab. Basic Pitch is a neural network that can analyze the recording of almost any instrument (including your voice) and then transcribe the notes that it detects into MIDI, the standard file format used for musical notation. It’s like speech-to-text, except it’s turning musical performances — whatever you hum, strum, pluck, peck, or tinkle — into a digital score you can edit on your computer.

Hear host Dave Zolotusky talk with Spotify researcher Rachel Bittner about what makes detecting musical notes an interesting machine learning problem. You’ll learn about how musicians use audio-to-MIDI converters to make music, the subtleties of pitch tracking, and why you want your model to capture the main pitch events in the audio as well as all the “wiggly stuff”. Plus, a live demo of the model in action and all the “Hot Cross Buns” you can handle.

Listen to the end of the episode to hear "Virgo", the single that artist-producer Bad Snacks composed using Basic Pitch.🎻🎻🎻🎻🎻🎻

Basic Pitch is open source software that musicians can use online for free and that ML developers and researchers can tinker with and contribute to.

Can you play an instrument or hum a tune? Try it yourself at BasicPitch.io
Contribute on GitHub in the Python repo or the TypeScript repo
Soundtrap is Spotify’s collaborative, online digital audio workstation (DAW) for musicians, podcasters, and audio producers
Read more on the Spotify Engineering blog: Meet Basic Pitch: Spotify’s Open Source Audio-to-MIDI Converter
Listen to a track composed using Basic Pitch: “Virgo” by Bad Snacks on Spotify
Watch the making-of video on her YouTube channel: Spotify Made an Audio to MIDI Converter (?!?) - Basic Pitch Overview
Listen to “Op37 Maximum Shimmer” and other demo tracks composed by the Spotify developers who made Basic Pitch

Want more open source goodness from Spotify’s Audio Intelligence Lab? Listen to Ep.06: Bits of Math at Scale to learn about Pedalboard, a Python library for applying studio-quality audio effects at scale.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

09: It's All About the Data Sep 15, 2022

In technology, as in life, bad experiences often start with bad data. This is as true with a misguided aunt setting you up on a blind date as it is with machine learning recommendations about which podcast you might want to listen to next. High quality data is essential to making sure every Spotify listener has a rewarding experience made just for them. But with a half trillion events happening on the platform every day, how do you even begin to sort all that data out?

Enter Laura Lake, senior director of Spotify’s Personalization Insights team. When she first arrived here, even simple questions were difficult to answer without teams having to “knit together 50 different data sources”. In this episode, she talks with host Dave Zolotusky about a critical point in Spotify’s growth and the yearslong journey that resulted in improving and ensuring the quality of the data that all our developers rely on.

Hear about the technological and cultural changes that led to both better quality data and better collaboration between our teams — and how we use the data to build the knowledge models that lead to Discover Weekly, Daily Mix, and a more personalized experience for every one of Spotify’s hundreds of millions of listeners. How do we know if our ML models are doing what we want them to? Did our listeners actually discover something new? Let’s dig into the data.

This is the second episode in our miniseries about machine learning and personalization at Spotify.

Listen to Ep.08 “Humans in the Loop” to learn what the research tells us about the long-term satisfaction of our listeners — and what makes the job of an ML engineer so different from other kinds of software engineers.

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

08: Humans in the Loop Aug 25, 2022

Get ready: we’re diving into machine learning. Hear how we’re improving personalization with reinforcement learning (RL), what makes ML engineering so different from other kinds of software engineering, and why machine learning at Spotify is really about humans on one side of an algorithm trying to better understand the humans on the other side of it.

Spotify’s director of research, Mounia Lalmas-Roelleke, talks with host Dave Zolotusky about how we’re using RL to optimize recommendations for future rewards, how listening to more diverse content relates to long-term satisfaction, how to teach machines about the difference between p-funk and g-funk, and the upsides of taking the stairs.

Then Dave goes deep into the everyday life of an ML engineer. He talks with senior staff engineer Joe Cauteruccio about what it takes to turn ML theory into code, the value of T-shapedness, the difference between inference errors and bugs, using proxy targets and developing your ML intuition, and why in machine learning something’s probably wrong if everything looks right.

Plus, an ML glossary: our guests educate us on the definitions for cold starts, bandits, and more. This episode is the first in a series about machine learning and personalization at Spotify.

Learn more about ML and personalization:

Recent publications from Spotify Research:

“Variational User Modeling with Slow and Fast Features” (Feb. 2022)
“Algorithmic Balancing of Familiarity, Similarity, & Discovery in Music Recommendations” (Nov. 2021)
“Leveraging Semantic Information to Facilitate the Discovery of Underserved Podcasts” (Nov. 2021)
“Shifting Consumption towards Diverse Content on Music Streaming Platforms” (Mar. 2021)

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

07: *Bonus!* Kelsey on How to Give a Great Talk Aug 11, 2022

In this special bonus episode, Google’s Kelsey Hightower is back to tell us what separates a good talk from a great one, the importance of storytelling, and the pitfalls of relying too much on your demo.

For more Kelsey:

Listen to Ep.07: “Kelsey Hightower on Engineering Culture” (38 mins.)
Follow Kelsey on Twitter @kelseyhightower (DMs are open!)

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

07: Kelsey Hightower on Engineering Culture Aug 04, 2022

The one and only Kelsey Hightower drops by the recording studios at Spotify’s global HQ in Stockholm to chat with host Dave Zolotusky about what it’s like being a software developer today: did it get harder or easier?

Hear Dave and Kelsey talk about what happens when tech becomes more accessible to more people, Kelsey’s role models and who he looks to for inspiration, when is the appropriate time to talk to your daughter about Docker containers, what comes after “move fast & break things”, what makes a good engineering culture — and how to recognize one when you’re interviewing. Plus, being a minimalist and how many pairs of pants do you really need.

Along with being a principal engineer and longtime developer advocate at Google Cloud, Kelsey is a legendary conference speaker, creator of Kubernetes The Hard Way, famous for never shying away from the hard topics on Twitter, mentor and role model to many, and one of the most respected voices in the engineering community today. We were very glad to nerd out with him.

You can find more Kelsey here:

@kelseyhightower on Twitter
Kubernetes The Hard Way
“From McDonald’s to Google: How Kelsey Hightower became one of the most respected people in cloud computing” by Tom Krazit (Protocol, October 2020)

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

06: Bits of Math at Scale Apr 28, 2022

Our Audio Intelligence Lab built the world’s beefiest effects pedal using machine learning and Python. Say hello to Pedalboard, an open source framework for adding studio-quality effects to audio files — at a speed and scale well beyond the capabilities of the tools you’d normally find in a music studio.

Host and principal engineer Dave Zolotusky talks with Peter Sobot (@psobot — not a robot🤖), one of the ML engineers on the team that built Pedalboard. They discuss the world of audio effects processing in both music and ML research, how we use Pedalboard in our own research, Python as ML glue (with some secret C++ under the hood), and the unexpected use cases that appear whenever you open source your software.

Plus, a live programming demo…on a podcast? “Tap, tap, tap, return…”? How interesting can that be? Listen to find out!

Learn more about Pedalboard, open source, and ML and audio research at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

05: Advent of Code Apr 07, 2022

What do coders do when they’re not coding for work? Sometimes they code for fun. (And sometimes they code for fun at work.) Hear why every December some of Spotify’s engineers become obsessed with the coding puzzles on Advent of Code, a website created in 2015 by an engineer named Eric Wastl.

Host and principal engineer Dave Zolotusky talks with Spotify’s chief architect and VP of engineering, Niklas Gustavsson (aka ngn), about competition and community, the utility of ugly hacks and garbage code, and how we can learn from play.

We also hear from Spotify engineers Sandra Dinsdale, Sophy Cao, and Tomas Aschan about who they’re really competing against, why they play, and why they can’t stop.

And lastly, Dave tries to find out the secrets to winning from the unbeatable Jimmy Mårdell — No. 35 on last year’s Advent of Code global leaderboard, the top player (by far) on Spotify’s internal leaderboard every year, and ngn’s nemesis (sort of).

We hope you enjoy this peek into one corner of Spotify’s engineering community and are inspired to join the global Advent of Code community in December. Until then, we leave you with this tweet from Advent of Code creator Eric Wastl (@ericwastl):

“If you're doing #AdventOfCode and learning stuff, but your code's a mess, you got the wrong answer, you're a few days behind, it took you several hours, you're in a language you're not familiar with…All I hear is that you're learning stuff. You're doing great!! 🌟❤️🎄🧑‍🎄”

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng

04: Backstage in the Wild Mar 16, 2022

Spotify believes open source is one way big tech companies and the wider developer community can solve big tech problems together. So how’s that going? In this episode, you’ll hear from our friends at Netflix and American Airlines — both early adopters of Backstage, our open platform for building developer portals.

At first, streaming video services and an airline may seem like two very different kinds of software engineering organizations. But as you’ll hear, we all have a lot in common when it comes to managing increasingly complex tech ecosystems and helping our fellow developers build great products.

Host and Spotify principal engineer Dave Zolotusky talks with Laurie Barth & Brian Leathem, both engineers on Netflix’s Platform Experience & Design team (PXD), and with Melinda Malmgren, principal tech lead for American Airlines’s DevOps team.

You’ll hear them discuss using Backstage to build their own developer portals and improve developer experience, how big problems require federated solutions, open source and inner source (and open sourcing inner source — stick with us, it’ll all make sense when you listen), plenty of fiddly implementation details (if you’re into that sort of thing), plus Lego and the state of competitive gymnastics (naturally!).

Learn more about Backstage at Netflix, aka “Spotlight”:

Learn more about Backstage at American Airlines, aka “Runway”:

Learn more about Backstage at Spotify:

Or check out the links below:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng

03: Community Builders Mar 03, 2022

Open source has become an essential part of software development, for both individuals and companies. Hear from Spotify engineers about what it’s like to build products in the open, what they’ve learned from developing Backstage with hundreds of friendly, helpful Internet strangers, and how participating in a community helps you grow as an individual, too.

Host and Spotify principal engineer Dave Zolotusky talks with Emma Indal and Himanshu Mishra, two engineers who work on Backstage, the platform for building developer portals that Spotify open sourced in 2020. Like the rest of Spotify’s Backstage team, Emma got a crash course in working with a rapidly growing community and discovered just how fast new features can be built when you have contributors from all around the world joining in to help out. Himanshu was one of those early contributors, working on the project from the outside before turning his weekend obsessions and longtime interest in open source into a new job at Spotify. For both, the opportunity to build with the open source community has been a pivotal part of their journey as developers.

Learn more about Backstage and open source at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng

02: Open Issues Feb 08, 2022

Backstage, Spotify’s homegrown developer portal, has become the most ambitious open source project in the company’s history. But for many on the team, it was the first open source project they’d ever launched. What were the technical decisions they considered/reconsidered/forgot to consider before releasing it into the wild? And what did they learn along the way?

Host and Spotify principal engineer Dave Zolotusky returns with Lee Mills, along with Patrik Oldsberg, both maintainers of Backstage open source. Hear why some on the Spotify team were actually relieved when given less time to get the job done, how perfection can be your enemy, and why Patrik is having trouble finding the right window blinds.

Learn more about Backstage and open source at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng

01: What is Backstage? Feb 07, 2022

Spotify has thousands of developers managing tens of thousands of software components. How do you even begin to manage all that complexity? If you’re Spotify, you build your very own developer portal — we call it Backstage. Wait, what’s a developer portal? Good question!

Host and Spotify principal engineer Dave Zolotusky will introduce you to Austin Lamon and Lee Mills, two members of the Backstage team at Spotify, who work on both the open source and internal versions of Backstage. Hear what it’s like to have your fellow developers as your “customers” — and also hear from two of those customers, Chantal Delfeld and Kana Abe, engineers who joined Spotify more recently. They tell us about their experience discovering all that is Backstage for the first time. Plus, hear what else the team is nerding out about in their spare time.

Learn more about Backstage and open source at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng

00: Introducing NerdOut@Spotify Feb 04, 2022

A tech podcast produced by the nerds at Spotify, made for the nerds inside all of us. We’ll try to find the answer to questions big and small, like: Is it harder or easier to be a developer today? Is TypeScript lovable? Should you run your home automation on a Kubernetes cluster? Subscribe to find out.

Read what else we’re nerding out about on the Spotify Engineering blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng!

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected