Show notes
Sometimes applications crash. Some other times applications crash because memory is exhausted. Such issues exist because of bugs in the code, or heavy memory usage for reasons that were not expected during design and implementation. Can we use machine learning to predict and eventually detect out of memory kills from the operating system? Apparently, the Netflix app many of us use on a daily basis leverage ML and time series analysis to prevent OOM-kills. Enjoy the show! Our Sponsors Explore the Complex World of Regulations. Compliance can be overwhelming. Multiple frameworks. Overlapping requirements. Let Arctic Wolf be your guide.Check it out at https://arcticwolf.com/datascience Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve. We provide solutions in AI/ML, Fintech, Healthcare/RWE, and Predictive maintenance. Transcript 100:00:04,150 --> 00:00:09,034And here we are again with the season four of the Data Science at Home podcast. 200:00:09,142 --> 00:00:19,170This time we have something for you if you want to help us shape the data science leaders of the future, we have created the the Data Science at Home's Ambassador program. 300:00:19,340 --> 00:00:28,378Ambassadors are volunteers who are passionate about data science and want to give back to our growing community of data science professionals and enthusiasts. 400:00:28,534 --> 00:00:37,558You will be instrumental in helping us achieve our goal of raising awareness about the critical role of data science in cutting edge technologies. 500:00:37,714 --> 00:00:45,740If you want to learn more about this program, visit the Ambassadors page on our website@datascienceathome.com. 600:00:46,430 --> 00:00:49,234Welcome back to another episode of Data Science at Home podcast. 700:00:49,282 --> 00:00:55,426I'm Francesco Podcasting from the Regular Office of Amethyx Technologies, based in Belgium. 800:00:55,618 --> 00:01:02,914In this episode, I want to speak about a machine learning problem that has been formulated at Netflix. 900:01:03,022 --> 00:01:22,038And for the record, Netflix is not sponsoring this episode, though I still believe that this problem is a very well known problem, a very common one across factors, which is how to predict out of memory kill in an application and formulate this problem as a machine learning problem. 1000:01:22,184 --> 00:01:39,142So this is something that, as I said, is very interesting, not just because of Netflix, but because it allows me to explain a few points that, as I said, are kind of invariance across sectors. 1100:01:39,226 --> 00:01:56,218Regardless of your application, is a video streaming application or any other communication type of application, or a fintech application, or energy, or whatever, this memory kill, out of memory kill still occurs. 1200:01:56,314 --> 00:02:05,622And what is an out of memory kill? Well, it's essentially the extreme event in which the machine doesn't have any more memory left. 1300:02:05,756 --> 00:02:16,678And so usually the operating system can start eventually swapping, which means using the SSD or the hard drive as a source of memory. 1400:02:16,834 --> 00:02:19,100But that, of course, will slow down a lot. 1500:02:19,430 --> 00:02:45,210And eventually when there is a bug or a memory leak, or if there are other applications running on the same machine, of course there is some kind of limiting factor that essentially kills the application, something that occurs from the operating system most of the time that kills the application in order to prevent the application from monopolizing the entire machine, the hardware of the machine. 1600:02:45,710 --> 00:02:48,500And so this is a very important problem. 1700:02:49,070 --> 00:03:03,306Also, it is important to have an episode about this because there are some strategies that I've used at Netflix that are pretty much in line with what I believe machine learning should be about. 1800:03:03,368 --> 00:03:25,062And usually people would go for the fancy solution there like this extremely accurate predictors or machine learning models, but you should have a massive number of parameters and that try to figure out whatever is happening on that machine that is running that application. 1900:03:25,256 --> 00:03:29,466While the solution at Netflix is pretty straightforward, it's pretty simple. 2000:03:29,588 --> 00:03:33,654And so one would say then why making an episode after this? Well. 2100:03:33,692 --> 00:03:45,730Because I think that we need more sobriety when it comes to machine learning and I believe we still need to spend a lot of time thinking about what data to collect. 2200:03:45,910 --> 00:03:59,730Reasoning about what is the problem at hand and what is the data that can actually tickle the particular machine learning model and then of course move to the actual prediction that is the actual model. 2300:03:59,900 --> 00:04:15,910That most of the time it doesn't need to be one of these super fancy things that you see on the news around chatbots or autonomous gaming agent or drivers and so on and so forth. 2400:04:16,030 --> 00:04:28,518So there are essentially two data sets that the people at Netflix focus on which are consistently different, dramatically different in fact. 2500:04:28,604 --> 00:04:45,570These are data about device characteristics and capabilities and of course data that are collected at Runtime and that give you a picture of what's going on in the memory of the device, right? So that's the so called runtime memory data and out of memory kills. 2600:04:45,950 --> 00:05:03,562So the first type of data is I would consider it very static because it considers for example, the device type ID, the version of the software development kit that application is running, cache capacities, buffer capacities and so on and so forth. 2700:05:03,646 --> 00:05:11,190So it's something that most of the time doesn't change across sessions and so that's why it's considered static. 2800:05:12,050 --> 00:05:18,430In contrast, the other type of data, the Runtime memory data, as the name says it's runtime. 2900:05:18,490 --> 00:05:24,190So it varies across the life of the session it's collected at Runtime. 3000:05:24,250 --> 00:05:25,938So it's very dynamic data. 3100:05:26,084 --> 00:05:36,298And example of these records are for example, profile, movie details, playback information, current memory usage, et cetera, et cetera. 3200:05:36,334 --> 00:05:56,086So this is the data that actually moves and moves in the sense that it changes depending on how the user is actually using the Netflix application, what movie or what profile description, what movie detail has been loaded for that particular movie and so on and so forth. 3300:05:56,218 --> 00:06:15,094So one thing that of course the first difficulty of the first challenge that the people at Netflix had to deal with was how would you combine these two things, very static and usually small tables versus very dynamic and usually large tables or views. 3400:06:15,142 --> 00:06:36,702Well, there is some sort of join on key that is performed by the people at Netflix in order to put together these different data resolutions, right, which is data of the same phenomenon but from different sources and with different carrying very different signals in there. 3500:06:36,896 --> 00:06:48,620So the device capabilities is captured usually by the static data and of course the other data, the Runtime memory and out of memory kill data. 3600:06:48,950 --> 00:07:04,162These are also, as I said, the data that will describe pretty accurately how is the user using that particular application on that particular hardware. 3700:07:04,306 --> 00:07:17,566Now of course, when it comes to data and deer, there is nothing new that people at Netflix have introduced dealing with missing data for example, or incorporating knowledge of devices. 3800:07:17,698 --> 00:07:26,062It's all stuff that it's part of the so called data cleaning and data collection strategy, right? Or data preparation. 3900:07:26,146 --> 00:07:40,782That is, whatever you're going to do in order to make that data or a combination of these data sources, let's say, compatible with the way your machine learning model will understand or will read that data. 4000:07:40,916 --> 00:07:58,638So if you think of a big data platform, the first step, the first challenge you have to deal, you have to deal with is how can I, first of all, collect the right amount of information, the right data, but also how to transform this data for my particular big data platform. 4100:07:58,784 --> 00:08:12,798And that's something that, again, nothing new, nothing fancy, just basics, what we have been used to, what we are used to seeing now for the last decade or more, that's exactly what they do. 4200:08:12,944 --> 00:08:15,222And now let me tell you something important. 4300:08:15,416 --> 00:08:17,278Cybercriminals are evolving. 4400:08:17,374 --> 00:08:22,446Their techniques and tactics are more advanced, intricate and dangerous than ever before. 4500:08:22,628 --> 00:08:30,630Industries and governments around the world are fighting back on dealing new regulations meant to better protect data against this rising threat. 4600:08:30,950 --> 00:08:39,262Today, the world of cybersecurity compliance is a complex one, and understanding the requirements your organization must adhere to can be a daunting task. 4700:08:39,406 --> 00:08:42,178But not when the pack has your best architect. 4800:08:42,214 --> 00:08:53,840Wolf, the leader in security operations, is on a mission to end cyber risk by giving organizations the protection, information and confidence they need to protect their people, technology and data. 4900:08:54,170 --> 00:09:02,734The new interactive compliance portal helps you discover the regulations in your region and industry and start the journey towards achieving and maintaining compliance. 5000:09:02,902 --> 00:09:07,542Visit Arcticwolves.com DataScience to take your first step. 5100:09:07,676 --> 00:09:11,490That's arcticwolf.com DataScience. 5200:09:12,050 --> 00:09:18,378I think that the most important part, though, I think are actually equally important. 5300:09:18,464 --> 00:09:26,854But the way they treat runtime memory data and out of memory kill data is by using sliding windows. 5400:09:26,962 --> 00:09:38,718So that's something that is really worth mentioning, because the way you would frame this problem is something is happening at some point in time and I have to kind of predict that event. 5500:09:38,864 --> 00:09:49,326That is usually an outlier in the sense that these events are quite rare, fortunately, because Netflix would not be as usable as we believe it is. 5600:09:49,448 --> 00:10:04,110So you would like to predict these weird events by looking at a historical view or an historical amount of records that you have before this particular event, which is the kill of the application. 5700:10:04,220 --> 00:10:12,870So the concept of the sliding window, the sliding window approach is something that comes as the most natural thing anyone would do. 5800:10:13,040 --> 00:10:18,366And that's exactly what the researchers and Netflix have done. 5900:10:18,488 --> 00:10:25,494So unexpectedly, in my opinion, they treated this problem as a time series, which is exactly what it is. 6000:10:25,652 --> 00:10:26,190Now. 6100:10:26,300 --> 00:10:26,754They. 6200:10:26,852 --> 00:10:27,330Of course. 6300:10:27,380 --> 00:10:31,426Use this sliding window with a different horizon. 6400:10:31,558 --> 00:10:32,190Five minutes. 6500:10:32,240 --> 00:10:32,838Four minutes. 6600:10:32,924 --> 00:10:33,702Two minutes. 6700:10:33,836 --> 00:10:36,366As close as possible to the event. 6800:10:36,548 --> 00:10:38,886Because maybe there are some. 6900:10:39,008 --> 00:10:39,762Let's say. 7000:10:39,896 --> 00:10:45,678Other dynamics that can raise when you are very close to the event or when you are very far from it. 7100:10:45,704 --> 00:10:50,166Like five minutes far from the out of memory kill. 7200:10:50,348 --> 00:10:51,858Might have some other. 7300:10:51,944 --> 00:10:52,410Let's say. 7400:10:52,460 --> 00:10:55,986Diagrams or shapes in the data. 7500:10:56,168 --> 00:11:11,310So for example, you might have a certain number of allocations that keep growing and growing, but eventually they grow with a certain curve or a certain rate that you can measure when you are five to ten minutes far from the out of memory kill. 7600:11:11,420 --> 00:11:16,566When you are two minutes far from the out of memory kill, probably this trend will change. 7700:11:16,688 --> 00:11:30,800And so probably what you would expect is that the memory is already half or more saturated and therefore, for example, the operating system starts swapping or other things are happening that you are going to measure in this. 7800:11:31,550 --> 00:11:39,730And that would give you a much better picture of what's going on in the, let's say, closest neighborhood of that event, the time window. 7900:11:39,790 --> 00:11:51,042The sliding window and time window approach is definitely worth mentioning because this is something that you can apply if you think pretty much anywhere right now. 8000:11:51,116 --> 00:11:52,050What they did. 8100:11:52,160 --> 00:12:04,146In addition to having a time window, a sliding window, they also assign different levels to memory readings that are closer to the out of memory kill. 8200:12:04,208 --> 00:12:10,062And usually these levels are higher and higher as we get closer and closer to the out of memory kill. 8300:12:10,136 --> 00:12:15,402So this means that, for example, we would have, for a five minute window, we would have a level one. 8400:12:15,596 --> 00:12:22,230Five minute means five minutes far from the out of memory kill, four minutes would be a level two. 8500:12:22,280 --> 00:12:37,234Three minutes it's much closer would be a level three, two minutes would be a level four, which means like kind of the severity of the event as we get closer and closer to the actual event when the application is actually killed. 8600:12:37,342 --> 00:12:51,474So by looking at this approach, nothing new there, even, I would say not even a seasoned data scientist would have understood that using a sliding window is the way to go. 8700:12:51,632 --> 00:12:55,482I'm not saying that Netflix engineers are not seasoned enough. 8800:12:55,556 --> 00:13:04,350Actually they do a great job every day to keep giving us video streaming platforms that actually never fail or almost never fail. 8900:13:04,910 --> 00:13:07,460So spot on there, guys, good job. 9000:13:07,850 --> 00:13:27,738But looking at this sliding window approach, the direct consequence of this is that they can plot, they can do some sort of graphical analysis of the out of memory kills versus the memory usage that can give the reader or the data scientist a very nice picture of what's going on there. 9100:13:27,824 --> 00:13:39,330And so you would have, for example, and I would definitely report some of the pictures, some of the diagrams and graphs in the show notes of this episode on the official website datascienceaton.com. 9200:13:39,500 --> 00:13:48,238But essentially what you can see there is that there might be premature peaks at, let's say, a lower memory reading. 9300:13:48,334 --> 00:14:08,958And usually these are some kind of false positives or anomalies that should not be t…
Full show notes at the publisher 
Our TOPPODCAST Picks 
Stay Connected
