Knowledge Graph vs. LLM – Bryon Jacob
Feb 29, 2024
Show notes
Years ago, I got to be an advisor for this company called data.world, and at the time, they were just getting started on helping figuring out how do you converge all the data sets that are in the world and help people work with them and combine them and share them. They built this thing that was kind of like GitHub for data.
I was interested in it because I could see at the time where the world was going and we’re going to need these much more advanced tools for being able to manage data. I tried to contribute my small way, but my favorite thing about it is that I got to know Bryon Jacob, who’s the CTO of data.world.
Brian is delightful guy. This is one of the guys who’s been thinking about the the nature of data, the structure of data, how we work with that in computers for his entire career. And he got onto a track that you could consider a little bit fringe, of using graph databases decades ago, the semantic models that we use to understand data from the thinking around RDF and the early semantic web.
And now what he’s built is the system that when ingests any kind of data, it parses that out, takes it in a graph database and makes it accessible through a query language called SPARQL, which you’ll hear us refer to. This is a kind of “advanced mode episode” and I know we’re going to lose some people We refer to a lot of technical stuff that probably only data nerds are really going to be interested in. I won’t be offended if you check out.
But, if you have any interest in data or the future of analyzing data and using data in AIS, you need to listen and understand this conversation. Brian is an expert. He’s built one of the most important king pin tools for using all the data in large-scale organizations or projects within the new generative AI context. If you are trying to use something like ChatGPT or another LLM as an interface to structured data, you’re doing it wrong, and I think you’ll be convinced about that as you start to understand what we’re discussing today.
So, hang in there. I promise this is a really REALLY valuable conversation for anybody who is trying to work at the forefront of using AIS for data analytics. I’m thrilled that we get to share this conversation with Bryon with you today.
Bryon Jacob is the CTO and co-founder of data.world – on a mission to build the world’s most meaningful, collaborative, and abundant data resource. Bryon is a recognized leader in building large-scale consumer internet systems and an expert in data integration solutions.
Bryon’s twenty years of academic and professional experience spans AI research at Case Western Reserve University, enterprise configuration software at Trilogy, and consumer web experience at Amazon and most recently in ten years building HomeAway.com. At HomeAway, Bryon oversaw platform development and the integration of thirty acquisitions while building the world’s largest online marketplace for vacation rentals.
AI Beyond Deep Learning – Ryad Benosman
Feb 16, 2024
Show notes
One of the things that’s on the verge of excitement and annoyance for me is the way that Artificial Intelligence work has all kind of converged around deep learning. Deep learning is amazing and super powerful, and we’ve gotten a lot out of it, but what it has done is, both attracted a lot of people to Artificial Intelligence, but also, steered all the research efforts away from other approaches into deep learning.
And you could say that makes some sense, because for a long time, we weren’t making a lot of advancement with these other approaches. But the truth is, most of the advancement in artificial intelligence really comes from the growth in computation and our ability to wrangle a lot of computation. If you took that same approach, with other algorithms, you may well get interesting results. If you could apply as much computation to them, the way we have for the large language models.
And so what really excites me is when I find people who are working on other approaches to artificial intelligence. My buddy, Dr. Ryad Benosman, has been working on different approaches to processing data for a long time, primarily in vision. And his worldview is highly neuromorphic. It’s about trying to understand. What is the brain doing with this data? How do I get a computer to do the things that the brain would do? And that’s hard because we don’t know exactly what a brain does, but one of the things we know is the way that eyeballs incorporate the signal that they get, and then try to turn that into something that the brain can put to use and that’s It’s obviously not done the way that deep learning works.
A lot of what Ryad has worked on has been time series machine learning, those are my words, not his, but basically what it means is, trying to process this data in real time in the order that you receive it and piece together, something meaningful.
That’s very applicable to computer vision. I think Ryad has been responsible for spinning about four companies out of labs, to develop these technologies.
Probably the most well-known is called, the Prophesee Camera, which, they developed and then sold to Sony. This is a camera is an event camera. Instead of just taking frames 30 times a second, aggregating all the signal on every pixel and sticking it into a frame, what an event camera can do is look at the signal that’s changing in any pixel , in the sensor, and this is very important for things like sensor fusion going forward. The work he’s done on algorithms to make that possible is super exciting.
Ryad was a professor in France for a long time. Most recently in Pittsburgh and at Carnegie Mellon. He’s been all over the place, on skunkworks at Meta, and now is doing exciting things we can’t talk about, but look, most people never get a chance to hang out with Ryad.
I’m so thrilled that I got to, get him on the podcast and share him with you.
Dr. Ryad Benosman is professor of Ophthalmology in the University of Pittsburgh School of Medicine. He is also an Adjunct Faculty Member in the Robotics Institute of Carnegie Mellon University. Prior to this appointment, Dr. Benosman was a full professor at Université Pierre et Marie Curie, Institut de la Vision, in France.
He is curently Director of Research at Meta (Neuromorphic and Event based Sensing and Computation)
He has worked on Event-based (Neuromorphic) Sensing and Computation, applied to develop novel Brain Inspired Machine Learning. His lab used to be the home of the event-based neuromorphic silicon retina ATIS and several other neuromorphic AI related platforms. He also has worked on brain implants and retina prosthetics and optogenetics stimulation.
Short episode where Ash & Pablos marvel at this ingenious invention called the GyroGlove that can ameliorate the jittery hands of folks with Parkinson’s disease.
If anyone can introduce us to the inventor, Dr. Faii Ong, we want to get him on the podcast!
Postmodernist Cuisine – Chris Young
Feb 08, 2024
Show notes
Well, if you ever got tired of listening to me, talk. Today’s the day when you just get to hear from my buddy, Chris Young, because I wound him up and clicked go, and he just talks, and it’s great. He has so much, interesting experience and amazing insights. So Chris Young, if you don’t know, I met him back when we started the Intellectual Ventures Lab, because he was the guy that Nathan Myhrvold hired to start the cooking projects.
We built an experimental kitchen there. Chris ran the project called Modernist Cuisine. Which ended up publishing a 2,400 page cookbook on the science of cooking. That won every award in the world. It’s literally a monument to modernist cooking. And these are new techniques for chefs and we talk about that a bunch today. Before that Chris had created the experimental lab at the Fat Duck and that’s Heston Blumenthal’s restaurant outside of London. Usually considered, if not the best restaurant on Earth, a contender.
Since publishing Modernist Cuisine, Chris started a company called ChefSteps to popularize sous vide, which is the most successful of the techniques so far invented by modernists. You have to learn to sous vide. It’s super easy. You can make everything you do come out perfectly.
After selling that company to Breville, Chris started a new company called Combustion and Combustion is really cool startup. There’s lots of lessons in here for entrepreneurs and folks who are making products. Chris is an amazing entrepreneur, very dedicated, really good at figuring out how to make everything work. Combustion is a difficult company to do because it’s hardware and software; and it’s in the kitchen; and it is hundreds of degrees, Fahrenheit. So it was just a lot to deal with. It’s great to learn these lessons and they’re shipping now and very successful with it. And then Chris has a YouTube channel called Chris Young Cooks, where he’s doing some of the cool stuff that we used to do on Modernist Cuisine. Cool photography, but doing it for video and sharing some of the insights that they have about cooking. So anyway, You’re going to have a blast listening to Chris.
Chris Young is a chef-scientist known for applying science and technology to create culinary experiences that earlier generations would never have imagined. Before becoming a chef, Young completed degrees in mathematics and biochemistry at the University of Washington. Unfulfilled with a life in the hard sciences, Young left his doctoral work behind for a job as a chef at one of Seattle’s top-rated restaurants, Mistral.
Young’s expertise wasn’t long secluded to the American Northwest. From 2003 to 2007, Young worked with the world-famous chef Heston Blumenthal to oversee development of some of his most innovative dishes. In 2004, Young opened The Fat Duck Experimental Kitchen, leading a team of more than six full-time chefs and coordinating the work of several consulting scientists. Beyond developing new dishes for The Fat Duck’s menu, Young was responsible for recipe development for the critically acclaimed first and second seasons of BBC’s “In Search of Perfection: With Heston Blumenthal.”
In 2007, Young was asked by the renowned technologist, inventor, and accomplished cook Nathan Myhrvold to return to Seattle to work at Intellectual Ventures. Alongside Myrhvold, Young helped research, experiment, and eventually coauthor the eagerly anticipated, industry game-changing Modernist Cuisine: The Art and Science of Cooking.
In 2012, along with Modernist Cuisine colleagues Chef Grant Lee Crilly and photographer Ryan Matthew Smith, Young co-launched an online-based culinary school ChefSteps, using an underground space beneath Seattle’s Pike Place Market. Their mission: teaching people how to utilize modern techniques in their cooking. He is the founder and owner at Combustion Inc., a company that builds nice things that make cooking more enjoyable. Like a thermometer that’s wireless, oven-safe, and uses machine learning to do what no other thermometer can: predict your food’s cooking and resting times with uncanny accuracy.
Mixtapes: a Lightweight Plan to Save the Internet – ØF
Feb 05, 2024
Show notes
Pablos: People are pissed off about social media all the time. They think that Facebook is making people vote for the wrong person. It’s still very difficult to find somebody who thinks they voted for the wrong person because of Facebook, but they think everyone else did. Never mind that, there’s this kind of, uh, very popular sensibility, which is to blame Facebook for all the problems in the world. They’re doing fake news, they’re doing, disinformation they’re doing , every possible thing that could be wrong. Everybody wants to blame Facebook for getting wrong or Twitter or, any of the other social platforms. So if you think about it, in one sense, , yeah, Facebook got everybody together. I’m just going to use them as the example, we can extrapolate. They got everybody together. They, ended up getting too much content. you and your friends are posting too much shit. Nobody has time to see all of it. So you need the magical algorithm, which you should do like triple air quotes every time I say algorithm. They’re like, the algorithm is supposed to figure out, okay, of all the shit that’s supposed to be showing up on your feed, what’s the coolest, or what’s the stuff that you’re gonna like the most? That’s the job of the algorithm. And of course, we all believe the algorithm is tainted. And so, it’s not really trying to find the things I care about the most or like the most. It’s just gonna find the things that piss me off the most so that I get my, outrage, dopamine hit and keep coming back. So, which may all be true. We don’t know. But, the point is, there’s a fundamental problem, which is you cannot see everything that gets posted from all the people you follow. So, there does have to be some ranking. And then the second, thing is that you want that ranking to be tuned for you. And I think the thing that people, are missing about this is that you’ve got to have, a situation where it is very personalized because, not everybody’s the same. Even if you and I followed the same thousand people, it doesn’t mean we have identical interests. There are other factors that need to play into determining like what I want to see and what you want to see. And then I think that there’s a whole bunch of things that, are classified as societal evils, that Facebook has to decide are not okay for anybody to follow. So if you have posts about Hitler, nobody should get to see those. Even if you’re a World War II historian, nope, you don’t get to see it. So there’s a kind of, problem here, which is that all of this flies in the face of actual diversity, actual multiculturalism, we have 190 countries in the world. We have a lot of different peoples, different cultures, you and I just had a huge conversation about, different cultures and how they drive, we don’t agree about these things. We have different ideas in different places in the world, even whole societies have different ideas about what’s okay, and what’s not okay, and that is the definition of Culture that is the definition of multiculturalism is valuing that that exists and letting everybody have their own ideas And and make let these different people operate in the way that suits them And when you travel, you get beaten over the head with that because, I can appreciate that people drive like this in Bangkok. That’s not how I want to do it , that’s kind of the fundamental point here. So anyway, what I’m trying to get at is you cannot create one set of rules for the entire world. That is not okay. Ash: 100% Pablos: And so what Facebook has chosen to do is try to create one set of rules for the entire world, at least the two billion people that are on Facebook. Ash: But then you become the government of Facebook. Pablos: You become the government of Facebook. And it’s and we’re all pissed off because they keep choosing rules that some people don’t like or whatever. And so I think this is untenable and I don’t think there’s a solution there. I think it is a fool’s errand and what I believe is, has gone wrong is that Facebook made the wrong choice long ago and they chose to control the knobs and dials and now they’re living with the flack that comes with, every choice they make about where to set those knobs and dials. And what they should have done is given the user the knobs and dials. They should let me have buried six pages deep in the settings, have control over. What do you want more of? What do you want less of? Ash: More or less rant. Pablos: Yeah, They try to placate you with the like button and unfollow and all that, but it’s not really control. So, contrast that with, the other fork in history that we didn’t take, go back to like 2006, in the years before Facebook, We had this beautiful moment on the internet, with RSS. So RSS, which stands for Really Simple Syndication, that hardly matters, RSS was an open standard that allowed any website to publish the content in the form of posts in a kind of machine readable way. And then you could have an RSS reader that could subscribe to any website. So we didn’t have the walled garden of Facebook, but, you remember all this, of course, but I’m just trying to break it down here. What we had was, this kind of open standard. , anybody in the world could publish on RSS using their website, all the blog software did this out of the box. WordPress does it out of the box. In fact, most websites, would support RSS. And then you had a reader app, that could be any reader app. This is again, open standards so get any reader you want. And if you just subscribe to any website in the world, you are following them directly. When they publish a post, it show up in your feed. And when you followed too many people, you could start making filters. So I’ve been making filters. I still do RSS. So by the way, all this machinery still works 15 years later. The machinery still works almost any website if you just put /RSS or / feed on the domain name you’ll see an RSS feed and you can subscribe to that so it goes into my reader app And then I’ve been building filters over the years. So I have filters like -Trump because I got sick and tired of all this bullshit about Trump regardless what you think about Trump I just wanted to think about other things and it was painful to have a feed filled with Trump during the election So I have also -Biden, I have -Kanye, I have -Disney, I have minus all kinds of shit that I don’t want to see, I still follow the publishers, but it’s weeding out articles that are about those things. And so I get this feed that’s pretty curated for me and my interests, and I get more of the stuff I like and less of the stuff I don’t like, but I’m responsible for the knobs and dials, I’m controlling the settings, and I get to have my own autonomy about what I think is cool and not cool. And if I don’t want Hitler, I can easily just -Hitler. And what we did instead is we kind of signed up for this sort of, babysitter culture of having Facebook make those choices for us. And people not, taking responsibility for their own choices has put us in this situation where we just have an internet full of people want to blame somebody else for everything that they think is going wrong. What we need to do is, figure out a way to, shift the world back to RSS. And out of the walled garden. So that’s my, that’s where I’m at, and I have ideas about that. Ash: And it’s interesting, go back to Delphi, So Delphi internet… Pablos: One of the first, before, before internet, this was like an ISP, like a, like AOL. Centralized ISP. Ash: Right. So, so Delphi was sold to Murdoch, to News Corp and, and then the founder, Dan Burns brought that back. He purchased it, he re acquired the company and then invited a couple of ragtag individuals, myself and, and Palle again, and Rusty Williams. Chip Matthes, and we had like, you know, a room with a VAX in the back. I was doing a lot of the stuff, but we were running forums. Dan had this crazy idea. It was like, Hey, what if you could just make your own forum? And this would be like way pre Facebook, it’s like 97, 98. And 98, we started supplying that ability to websites. And the first one we did was a guy named Gil . And like we said to him, it’s like, Hey Gil, like you guys really should have some forums, like, yeah, we totally should be. Wait, so how do we do that? And we wrote like a little contract, right? like the first, I think, business development contract that you could probably make. He was head of, , business development, eBay. Right. So he did that. I mean, he’s very well known sort of angel kind of lead syndicate guy. Now I like an angel is for like for, for ages. Pablos: Oh, Penchina. I know who you’re talking about. Yeah. Ash: We still have like the first document, you will do this. I will do this. I will give you a forum. You will use it for people to talk about, I don’t know, the, the, their beanie baby or whatever they were selling back then. And the, the reality was that that took off and then we started supplying this technology, which we then enabled, we RSS enabled it, by the way, of course, at some point, right. When it was, when the, when the XML feeds were like ready to go, we upgraded from XML And then we, we, we took that and we said, all right, let’s go, let’s go for it. And at some point we’re doing 30 million a month, 30 million people a month. Unique. We’re like on this thing and we never governed. You could, you could go hidden, right? Kind of like your locked Instagram page versus not, but we didn’t govern anything. Forums had moderators, they were self appointed moderators of that domain of, of madness. So if you didn’t like that person’s moderation, You know, like, all right, screw this guy. You know, like, I don’t, I don’t want to listen to you. You’re crazy. And what we found, and this was the piece of data that I think that was the wildest. Servers are expensive back then. You actually have to have servers. Or in our case we were beating everyone else. Cause we had a VAX that was locked in a, Halon secure room. No, because it came when we repurchased it for a dollar. Like the VAX was still there and Lachlan Murdoch’s, office became our like conference room. No, I’m not kidding. It was, it was really crazy. There was a, it was just a VAX sitting there and, Hey, look, you could run UNIX on it. We were good. We didn’t care. It loved threads and it was good. And it could do many, many, many, many threads. So we were running this, this thing highly efficiently. There’s six people in a company doing that much. That was the company, literally six. I look today and how many people we hire and I’m like, there were six of us. It was wild, the iceberg effect took place. So what ended up happening is the percent, and this is where I think Facebook can’t do or doesn’t want to do, is how do you advertise below the waterline? And when we were sitting there with the traffic, we’re like, dude, why is there so much traffic, but we can’t see it, right? It looked like we only had 20, 000 forums or something, and there was like all this mad traffic going on. And. It was something like the 80, 20 rule the other way. It was like 20 percent was indexable that you could see that you could join a forum. And it was 80 percent were, were insane things like Misty’s fun house. That by the way, is a legitimate. Forum at one point, right? It was Misty’s fun house. So I’m just saying, cause we’re trying to figure out what was going on. Where were the people chatting and talking? And that’s what we did. We let them bury themselves deeper and deeper and deeper. Usenet did that. If you just go back in time, what do you think BBSs were? It’s the same. Pablos: Exactly. Ash: We always love talking. Pablos: Yeah. People love talking. Ash: You just figure out which one you want to dial into. Pablos: Nobody’s pissed off about who they’re talking to really. Usually they’re pissed off about who other people are talking. They’re pissed off about some conversation they’re not really a part of. Or a conversation they can be a spectator on, but doesn’t match their culture. That’s one of the big problems with Twitter it’s like BBSs, and it’s BBS culture. Elon was the winner of the Twitter game long before he bought Twitter, because, that’s just BBS culture that he had in his mind, IRC or whatever. All kinds of people who are not part of that culture, are observing it and think that it’s a horrible state, of society that people could be trolling each other and shit. And that’s just part of the fun. You have this problem when you try to cram too many cultures into one place, it takes a lot of struggle to work that out if you’re in, Jamaica, Queens, then you’re gonna, you’re gonna work it out over time, with a lot of struggle, you’re going to work it out and the cultures are going to learn to get along. But in, but on Twitter, there’s no incentive. Ash: That’s why we still have states. The EU still has, like, how many languages? That’s why we have Jersey for New Yorkers. Pablos: The EU in their way has figured out how these cultures can get along. I think there’s a real simple fix to this. The big death blow to RSS in some sense was that the winning reader app was Google Reader. And so the vast majority, of the world that was using RSS was using Google Reader. And then I don’t totally have insight on how this happened, but, Google chose to shut down Google Reader. And I don’t know if they were trying to steer people into their, Facebook knockoff products or whatever at the time. in a lot of ways I think what it did is it just handed the internet over to Facebook. Because anybody who was being satisfied by that, and just ended up getting, into their Facebook news feed instead. So it just kind of ran into a walled garden. I don’t really blame Facebook for this, the way a lot of people want to. I blame the users. You’ve got to take some responsibility, make your own choice, choose something that’s good for you, and most people are not willing to do that. But, I think to make it easier for them, and there is a case to be made that , people got better things to do than architect their own rSS reader process, but we could kind of do it for them. And so I think there’s one, one big kingpin missing, which is you could make a reader app that would be like an iPhone app now. And you could think of it as like open source Instagram. It’s just an Instagram knockoff, but instead of following, other people on a centralized platform by Instagram, it just follows RSS. And then it only picks up RSS posts that have at least one picture, right? So any RSS post that has one picture and then the first time you post it automatically makes a WordPress blog for you, that’s free. And then, posts your shit as RSS compliant blog posts, but the reader experience is still just very Instagramesque. So now it’s completely decentralized in the sense that like you own your blog, yeah, WordPress is hosting it, but that’s all open source. You could download it, move it to Guam if you want, whatever you want to do. So now all publishers have their own direct feeds. All users are publishers, which is kind of the main thing that Facebook solved. Ash: Content is no longer handed over to someone, right? That’s the other big thing. Pablos: Exactly. The content is yours and then your followers are yours, right? When they follow you, they follow you at your URL. And so you can take them with you wherever you go. And then to make this thing more compelling, you just add a few tabs. You add the Twitteresque tab. You add the TikTokesque tab for videos. And, add, the podcast tab. So now, posts are just automatically sorted into the tab for the format that matches them. Because people have different modalities for, for consuming this shit. So, depending on what you’re in the mood for, you might want to just look at pictures because you’re on a conference call. Fine. Instagram. Or, you know, you might want to watch videos because you’re on a flight. Who…
Primer on Fusion Reactors — Bob Mumgaard & Steve Renter
Feb 01, 2024
Show notes
This is a conversation with Bob Mumgaard and Steve Renter, founders of Commonwealth Fusion Systems in Cambridge, Massachusetts. So these guys spun out of MIT. An incredible, ambitious, company to figure out how to commercialize fusion, and it’s really the first fusion company in the world that has been able to show publicly that they have no new science needed in order to make it work, which is a major milestone.
They also published a series of seven papers a couple of years ago now, showing exactly how they can build a fusion reactor. This is the kind of thing that has been a joke my whole life with physicists saying that fusion is only 20 years away and always will be. That might not be true anymore…
Commonwealth Fusion Systems had a major breakthrough when they figured out how to make a super magnet, using a new kind of super conductor that’s appropriate for the task, and that really changed the game.
Almost everything else they do, the science is the same as it was in 1970.
They make a plasma that floats inside of what is called a “tokamak”, which is a kind of donut shaped toroid that has plasma floating on a magnetic field inside. So all this is like crazy, hard, technical science. None of it would have been possible without decades of government funded research. Multiple governments funding research.
These guys were standing on the shoulders of a lot of giants who came before them. And they are very careful to acknowledge that all the time, which is great. And they’ve been very successful at showing that they can really make fusion happen. Now there’s a lot of engineering work that has to happen between here and there, but know they’re going at it with an extraordinary amount of ambition and they’ve hired a lot of really smart people.
They have probably the best shot at achieving fusion of anything that I have seen.
This would be the biggest upgrade for humans in our lifetime if we get there. And so it’s a very exciting project.
I’m thrilled that I’ve gotten to know these guys a little bit and got to hang out with them their operation that keeps growing.
I hope that you learned something from this conversation with Bob and Steve.
Bob is the co-founder and CEO of Commonwealth Fusion Systems (CFS), a fusion energy startup company spin-out of MIT. He drives the company’s vision to accelerate the path to commercial fusion energy by managing its strategic partnerships and technical approach. He also believes in the power of technical and organizational innovation to facilitate the breakthroughs needed to combat climate change. The CFS team and Bob are working in collaboration with MIT’s Plasma Science and Fusion Center (PSFC) to realize the holy grail of renewable energy: fusion.
Steve Renter is the Chief Growth Officer at Commonwealth Fusion Systems, designing and building commercial fusion systems to provide limitless, clean energy to the world. Commonwealth Fusion Systems (CFS) has the fastest, lowest cost path to commercial fusion energy.
The ultimate mission is to deploy fusion power plants to meet global decarbonization goals as fast as possible. CFS has assembled a team of leaders in tough tech, fusion science, and manufacturing with a track record of rapid execution. Supported by the world’s leading investors, CFS is uniquely positioned to deliver limitless, clean, fusion power to combat climate change.
LLMs are Superstition – ØF
Jan 29, 2024
Show notes
Pablos: So what happens right now in scientific research is, if you’re going to do a research study on something, like “are M&Ms is bad for you?” It’s impossible to do that study. You have to be very specific and ask a much more fine grained questions like ” how many M&Ms does it take to, Kill a mouse?” or to cause a mouse to vomit. You just have to be very specific cause that’s something testable. You could test that, you can get multiple mice, you can feed them enough M&M’s that they eventually vomit. The whole research study can be done that way.
And so when you read scientific research studies, that’s typically what you’re looking at is some very narrowly defined thing that they believe is correlated to a much more significant or bigger effect, but you can’t test the whole thing. You can’t ask questions like, “does this thing cause cancer?”
You can ask questions like, ” does this amount of exposure to this thing over this much time cause this specific, type of cancer in this type of rat?” Things like that. So that’s great and all because it means, we’re structuring, tests that we can actually perform, but the downside is that for most people, what they would actually like to know is ” do M&Ms cause cancer or how many of them is too much, things like that.
Getting those answers is often not straightforward from scientific literature. And so the way that we. usually try to compensate for that is to do what’s called a meta analysis. And a meta analysis is where somebody will go and dig up all of the studies on a given topic, combine them and try to say, “across a hundred studies involving M&Ms and cancer, this is kind of what happened” and, to just sort of give you a general sense of whether or not, the effect you’re interested in is happening. Good examples of this are like, chiropraction is largely, debunked.
A lot of people get pissed off at me talking about it because it can be a deluxe placebo, but in clinical trials, very few clinical trials are performed. It’s hard to do them. Different practitioners have, different effectivity levels anyway. And so the problem is it’s hard to run those studies, but even if you do, you can’t find any indication that chiropraction actually cures anything. So this is a case where we don’t have good research and the only way to try and get to the bottom of it is with a meta analysis where you find the studies that have been done and you sort of combine their results and try to say whether or not chiropraction works.
People, there’s no point arguing with me if you’re listening and you think chiropractic is great. Go nuts. I encourage you not to do that, but, whatever, do your own thing. But the point is the only way you could get a reasonable answer is with this kind of meta analysis. Now meta analysis is very time consuming and difficult to perform and often isn’t getting done, but what it really involves is just go read a bunch of studies. Well, it turns out that’s what an LLM is really fucking good at. So you, so right now we’re in a stunted position because one of the big problems with OpenAI and ChatGPT is they’ve crippled ChatGPT. It doesn’t read scientific literature and even if it does, it’s not really allowed to comment on it.
So they’ve crippled the thing to keep you from talking to it about anything that might be health related and stuff like that. What you would really want an LLM to do, and one of the things that would be really good at is doing ad hoc meta analysis. So you could just say, “Hey, I feel like I’m getting a cold, should I take zinc?”
There’s people marketing zinc for that purpose. We’ve all been told to take zinc, but I don’t fucking know if that’s an old wives tale,
Ash: It’s like echinacea, zinc, doesn’t matter, it’s all those things.
Pablos: I don’t have time to go read every scientific research study, but I bet you collectively we have that answer, and so if I could just ask an LLM.
Ash: Wasn’t wasn’t IBM’s Watson at some point pretty good? Watson Health actually had all this.
Pablos: That’s probably what they were trying to do.
Ash: They were doing it and they were doing pretty well. They weren’t they weren’t using a full LLM model. That’s that was the whole breakthrough.
Pablos: They were kind of in the pre LLM days. It was LM. It was just LM. It wasn’t LLM.
Ash: Just language models. And they were taking huge amounts of data. But what they had is they had their own normalized structures underneath. So that was the difference, right? They didn’t let the structure form itself. But what you’re saying is true.
Pablos: You’re right, and we could probably build like a Watson for health in a weekend now using, Stable Diffusion or something. It would be way better. You would just basically load it up with all the research and let it go nuts and then let people ask questions like, ” Hey, should I be taking zinc?”
Ash: The problem is reliability score.
Pablos: Oh no, it’d be terrible, but it’s already horrific. Right now, we’re just going off superstition. I mean, literally that question of, should I take zinc? You’re gonna get as many answers as people you ask because somebody’s Chinese grandmother said You should be, taking echinacea instead.
Ash: You should listen to my first class. The first part we were talking about is what is known as “triangulation of information truth.” What is provenance for data. Then you have to figure out, how do you weigh it? LLMs are fantastic, like you said, because they can take all your source inputs.
So if you go back to, to signal analysis, or analytics for like intelligence again. We’ll just lean on that for a moment. Truth is great if you’re playing with mathematics. You get QED and you call it a day for the most part. But for other things, truth, zinc, for example, like your zinc example. There’s some balance between like how much did it really? Was it an emotional support protocol? Did it help you because you were convinced that, your grandmother was right or whatever’s happening to, to actual physical actions internally, right?
We can be scientific about it, but it comes back to source and information. If you pick a really, really dangerous topic and we won’t go there, but let’s just pick Gaza for one second. How do you find what’s really happening? Well, you hear a lot and someone’s like, “well, I read it in the Wall Street Journal.”
I read it here. I heard it there. I took Al Jazeera. I did Briebert. Whatever you picked. The question was, did you do it in all the languages? Did you listen to a local radio station? Did you find someone’s signal data from nearby? What was happening? Did the bomb go off or did this happen?
If you look at information, just like you’re looking at these scientific papers, the question becomes the weighting factor. We as humans, I think one of the things we know how to maybe do, at least a good analyst should be able to do, is try to give weighting based on time and location and stuff.
And I think the large language models have to start to put in context again. I think they have to add one more dimension.
Pablos: For sure. And I think that you touched on the other thing, which is that right now, all this information is like floating around without, tracking provenance, and so, interestingly, like in scientific research, you at least have citations. which is a lightweight form of provenance. It’s a start, but ultimately, the way these things all need to be built, not only , the LLMs for doing meta analysis, but really every knowledge graph needs to be built off of assertions that are tracked. You keep track of provenance, okay the sky is blue, well, who said the sky is blue?
Where did you get that from? And that way, whenever you’re ingesting some knowledge, it’s coming with a track record. That’s how we’re going to solve news online, eventually.
Ash: Kind of like, the Google Scholar score or whatever.
I go back to my partner, to Palle, right? So Palé actually has a patent. It’s probably expiring soon, so for those of you who want to do this, we should go do it. He owns webprovenance.com. And he owns the patent on how you check provenance. One of the things that came out of the BlackDuck software stuff was that at BlackDuck, we needed to know who created something. So do you remember the Sun Microsystems, IBM, lawsuit, Java? If you’re a compiler theorist, then you know that, just because West Side Story takes place in New York, You could probably say, well, doesn’t it sound exactly like, Romeo and Juliet? So maybe you change the variables, but it’s the same stuff. And the idea was that when we were looking at, open source, with open source, the interesting thing is you’re trying to figure out, where did this little rogue piece of code, this little GPL or LGPL infection come from?
You need to find it. So it’s one thing to talk about the combinatorials, but the other was to find it. And then Palle was like, well, I can do something cooler., He said, if Brewster Kahle’s Way Back Machine, remember the original Alexa project? So If you could go in and take all that data, he’s like, I could pretty much tell you like who killed JFK.
You can find the provenance of almost any information. He wrote this wild algorithm for it. I’d love to see some of that incorporated into the LLM stuff because that algorithm, and again, we would happily, anyone out there if you’re willing, this has been a project we’ve been looking at for the better part of 15 years.
Pablos: Well Stability might pick it up. They love that kind of stuff, that would be a huge coup for them.
Ash: Well, we should, we should have this conversation offline, but it’s a, it’s interesting. It’s an incredibly cool algorithm. He was a compiler theorist anyway, an algorithmist, at Thinking Machines. So, he always wondered where the info came from.
And I sat there and said, hey, we should find a way. And I remember the stunt I wanted was like, to figure out if they were aliens. And he’s like, what do you mean? It’s like, well, who started that rumor? Like, where did it happen? Right? So, imagine you could take any rumor, and I can tell you how it started.
Pablos: That’s so cool.
Ash: Wouldn’t that be the coolest thing ever?
Pablos: So important.
Ash: Yeah, and we need that.
Pablos: That is super important. I’ve seen somewhere, a map that somebody made of where are all the UFO sightings reported? And like 98 percent of them are in the United States. I think the rest of the world doesn’t even have UFO as like a notion. it’s not even a, thing for them.
Ash: It’s cause we have no healthcare. Look, all I know is, years ago, we just didn’t have enough data. Years ago, we couldn’t. We were like, looking at the Wayback Machine, and we were like, I was like, well, who can we go to to get all the data?
Can we get the entire web? Today, large language models have already stolen all the data. They already have it. So if you have enough of the data, we could definitely help you figure out the algorithm to go backwards and it’s complicated.
Pablos: That’s super exciting.
Ash: He actually patented it himself because he was trying to figure out if he didn’t need a patent attorney. So that was his project, can I make a patent? And his patents on provenance. So I think it’s a big coup if they could pull it off. Can you imagine you could just type in who started, where did this first start?
Pablos: Dude, that’s crazy cool.
Ash: It’s super cool.
Pablos: I’m kind of always on a rant about this, but we need a variety of models. Like LLM is the beginning, not It’s a thing that you need, like the way we’re doing it now actually kind of sucks and requires a lot of brute force, but there’s so many things that it’s not good for.
Ash: And it’s so susceptible to the thing that, what did I do in my life? Psych warfare is all about information corruption. Dude, you corrupt a large language model, that thing is convinced that the sky is red at that point.
Pablos: Exactly, well, I’ve been thinking about that. Why don’t I just..
Ash: Corrupt it?.
We’re bad hackers.
Pablos: I can fire up, 100,000, blogs written by an LLM that all just talk about my, prowess with the ladies.
Ash: Exactly.
Pablos: And the next thing you know, all the future LLMs will be trained on a massive amount of data that indicates that, Pablos is the man.
Why wouldn’t we do that?
Ash: At the end of the day, the LLMs are basically superstition. There you go. I’ve just said it.
Pablos: Right. They’re superstition.
There you go.
Ash: LLMs are superstition.
They’re based on some concept of something that it derived because it took a whole lot of information from a lot of grandmothers.
Pablos: And that’s the thing, Like what’s posted on the internet is all that they know.
It’s driving me crazy.
Ash: Worse, it’s only the people who have given them permission, so the quality sources are going to start cutting them off. So, all they’ve got, all you’ve got are the people who are generating rumors that they’ve seen UFOs.
Pablos: Well, that’s all true for the LLMs made in America.
Ash: Yeah, so the American LLMs know where the UFOs are.
Pablos: Japan decided that copyright doesn’t apply to training LLMs. So the most powerful LLMs, for now, are gonna be in Japan.
Sign me up.
Ash: Even better, that means Japanese information…
Pablos: That’s probably true, learn Japanese.
Ash: Which, think of it, if I wanted to build, my 100,000 LLMs generating your prowess, I’m gonna do it all in Japanese. I’ll do kanji, hiragana, and katakana. I’ll give it to them in all three formats. You could crush it.
I I would love to see any of these. I think that’s, that should be our ask for everyone.
Pablos: Yeah.
Ash: if someone, someone wants to run with it, go build it.
Pablos: Yeah, people, build this shit.
Ash: Tell us. We can help you commercialize. We will find you.
Pioneering Computer Graphics & Animation – Richard Chuang
Jan 25, 2024
Show notes
Part of what I love about getting to create a podcast like this is sharing conversations I’ve had with some of these extraordinary people that you just never hear about otherwise or never get to meet. They’re working behind the scenes, inventing new technologies that become part of our lives.
And they’re out of the spotlight. I’ve been lucky enough to get to know Richard Chuang, who’s with us today, because we both served as board members at the University of Silicon Valley, where he is still a trustee. Richard is a pioneer in computer graphics. He’s been there since the beginning, since the moment we turned pixels into images on a computer screen and turned those images into animations.
What has later become, everything you see coming out of Hollywood now. He’s a real pioneer in computer graphics, having built some of the animation systems at PDI, 30 years ago, that ultimately became DreamWorks Animation, where the pioneering feature films animated on computers. Ants, Shrek, those kinds of things were first done.
Look, Richard, it makes me want to cry. How humble he is, the wisdom he has in this conversation. He’s sharing so much with us and, going back to, the seventies when he first learned about computers. there’s some computer history in here. His encounters with Steve Jobs over the years, in both working in animation with computers, his experience with, some of the other pioneers in the, in the industry.
Richard doesn’t take credit personally for anything, but the truth is he’s behind a lot of these things that change the world, and I’m so thrilled to be able to share him with all of you guys.
Pablos: Here’s one of the things I think is a critical area of invention that remains unsolved, but it’s definitely a part of the future. So if you’re using an iPhone anywhere in the world, cultures vary. I’ve been working with this guy in Venezuela on a project. I text him on WhatsApp and then he replies with a voice memo like every time and so his, culture and worldview is just like talking to the phone and probably because I know Venezuelans do a lot more talking or something. Whereas I never use voice memo. I’m texting, but a lot of that is like, I’m in public around other people and I don’t want to disturb them and, disturbing people is considered uncool where I come from, but in Venezuela, like everybody’s chattering all the time, probably because they’re all Latinos. Talking to your computer will become more and more common. And you can see that some people are more comfortable with it than others. I see it a lot more in people from other countries than I do in Americans. Right now, talking to Siri kind of sucks, and Alexa. These things are kind of stunted because, they’re very one shot oriented. If you take your iPhone and start using the voice interface for ChatGPT, wow, it gets pretty exciting. Because now you’re having this, two way, audible conversation that builds on itself over time. And if you haven’t done that, I think everybody should try it because that will give you a sense of where these things are going. Once you get that going and realize, oh, I can just do this while I’m driving or walking, and I don’t have to be staring at my phone. It starts to get compelling. And so it’s not hard to imagine being, a few years down the road where ChatGPT is just listening all the time and piping in when it has the answers for you . So that’s just laying the groundwork, hopefully all that makes sense. But where I think this goes is that we need to solve one really big problem that remains, which is sub vocal input. Ash: Okay. Pablos: And what that means is, right now, if I’m talking, I don’t want to talk to my phone, I don’t even want to dictate text messages or do voice memo, because there’s people around listening, I don’t want them here in my business. We’re in this situation where the eavesdropping potential, even if you’re not talking about something super secret, it could be private or whatever. I don’t want to play a message from you out loud and I want other people hearing things that I haven’t screened yet, who knows what you’re talking about. So, what sub vocal input would do is give you the ability to just essentially whisper and have your phone pick it up. People around you wouldn’t hear you, wouldn’t understand you but you would still use the same machinery that you have and we all have the ability to whisper, and and quietly. If you’re trying to whisper for someone else to hear you, maybe it gets kind of loud, but if you’re just trying to whisper to yourself, it can be super quiet. We know that this should be possible, and we know that because deaf people are able to train themselves to do lip reading pretty well. So a deaf person who’s, got nothing, bothering them audibly can sometimes, apply enough focus to the task of learning how to read lips that they can do a really good job of it. So there’s enough of a signal in what your phone could see. So you know with Face ID there’s a tiny little LiDAR sensor that’s doing depth, and it can see your face. It can see the, minute details about your face. That’s why it can tell, the difference between your face and a photo of you and your twin brother or sister, whatever. So it might be possible right now. With the hardware that’s in an iPhone, even though you probably don’t have access to the right APIs for this to work, but maybe in a equivalent Android phone or something, maybe this could be prototyped. Where you could just use that machinery, train a giant, model, just a machine learning model on, lip reading. Ash: Yeah. Pablos: And so you would be able to just look at your phone and whisper, and it would transcribe. Ash: There’s a couple of things on this. Three GSM world, before GSM, 2000 or so. So we’ll go back in time. One of the big conversations that we would have was, I was a proponent saying that we just don’t have enough bandwidth and People are like, “yeah, but we’re going to have 3G & 4G & 5G & 6G.” And I said, “no, no, you’re missing the point.” The bandwidth to your device is not the issue, it’s between the device and the human. It’s your conversation. It’s, this is where we’re stuck. We’re stuck because we type, we could try Dvorak, we could try QWERTY, we can pick the keyboard, we can have sideways keyboards, we can speak to it, but I still think all of these are terrible. Whispering, could be very interesting. There was a MIT headset, Alter Ego. So Alter Ego, if you look at this thing up, it’s a mind reading, reading device. Sub vocalization signals through EEG, brain activity. He can actually make it work. Pablos: Well, I’ve played with some of these things. I have NeuroSky headset emotive, but I think what you have to do with them… Ash: This one you wear. It’s bone conducting. It’s wild. You just put it on and say, Pablos: Oh, it’s bone conducting. So it’s picking up speech, it’s not EEG. Ash: No, no, no. The bone conducting is how it tells you things back. So it even whispers it back. Like, into your head. Pablos: Oh, but you could just do that with headphones. Ash: No, that’s how it whispers back. You think it and then it tells you things. Anyway, it’s called alter ego, we’ll link to Alter Ego. To me, it goes back to what you’re saying, which is, is there a way? Otherwise, we just look like, we’re murmuring to ourselves, right? We’ll just look completely crazy. Like sometimes I get a little bit annoyed with people on conversations with AirPods. You just have no idea what’s going on, right? There’s a little hairdryer sticking out of their head, and they’re like, just walking around, and we just are fully, we’re already like, isolating ourselves and now we’re, we’re conversing. I think what you’re saying though is that the sub vocalization stuff needs to be in a way where it’s, Almost so discreet that it is a relationship between you and a listening device, right? It’s almost like the pixie on your shoulder. Pablos: Yes. Ash: It’s like the little angel devils whatever the animated version was. Pablos: Yeah, and I think there could be other technologies. I don’t know if you could fit it in something like an AirPod. Maybe like a Compton backscatter detector, one of these terahertz imagers, like the thing at the airport that you do the HOVA signal to, and then it’s you. Without a lot of radiation, you know, those things are low impact. You could do something like that to see the tongue through the side of the mouth. Ash: My belief is closer to the way that you were trying to tackle this problem, which is, hey, it listens in and jumps in. But what if I could prompt it to jump in, right? So for example, let’s assume that instead of having to build anything new, it’s now just listening to me. Constant in real time. Imagine a natural language parsing system with a, engine underneath. We used to call these things While Aware. This was actually the name of our company from years ago. And While Aware was intercepting SMS messages in real time on the SMSC. And the idea was that, it would detect what the conversation was, but because it knows who you are, it would evoke different things at different moments, right? So let’s pick, for example, Bitcoin share price, Bitcoin’s falling as a price. And that message was coming to you or that data was somehow coming to you. It might say, do you want to open up, your trading account and you can go sell it. And for me, it might, immediately tell me, do you want to book, tickets to Belize in a non extradition country, because my capital call is too high,. Whatever it is, if I have a margin call, because it knows what’s happening. It’s contextual, understanding. And I think one of the big things that we’re missing in all of these little support things that you allude to that ChatGPT brings to the table is contextual. We fail because It doesn’t understand us. Siri doesn’t know. Pablos: This is a separate conversation. Fundamentally, you are right. The whole future of AI requires that it know you, it needs to know you, it needs to know every conversation you’ve had, not only every SMS but text message and email, it needs to have 100 percent of that so it understands you. It knows what you know, it knows what you care about, it sees what you do, it sees what you say, it has to have all that and I want the AI to have all that. We need to architect for that and right now we’re not doing that because we’re building giant centralized AI’s. Ash: That’s when you’re, different technologies, whether it’s the backscatter or it’s the, lip reader or the whisper detector. All of those become a lot easier when you have context. I don’t know if you remember Google’s evolution, 2009, 2010, Google suddenly, not as creepy as Facebook, but its searches were just better, its searches were just better. Why were they better? Oh, you’re standing in New York city. So obviously maybe it’s contextual to what’s around you. Maybe the weather is cold. So Google’s original cookie, which they’re now getting rid of, was so laden with data. If you could mine that sucker, you won. It knew all of the signals. And I used to call it, signal gathering in terms of the more signal you had, the more accurate you became. And the more you look like sort of a savant. So our AI, like you said, isn’t really smart and Siri’s terrible because it doesn’t know much. It doesn’t even know intent. So as humans, why is it that we can speak with somebody with a very heavy accent sometimes? Because we know the context of what’s happening and why we got there. It’s not just lip reading. It’s because when we’re with them, we do our own interpretive dance. I think that if you tie the two together, what you just said about, you know, these other little signal things, you could pull it off. Pablos: I assume we’re gonna get the latter for free. That’s gonna happen. AIs will be stunted until they start to have access to everything and know everything about me and my context in real time. So that’s all gonna happen anyway, and there’s such momentum around that. So I think we get that for free and even if you didn’t, having a conversation with ChatGPT right now will probably convince you that it’s, like, good enough that we’re going this direction one way or another. Ash: The reason I bring all this up is, can you imagine if, instead of having to whisper, what if all I have to do is have my phone out, and I just say yes or no, or I say more? Go back to my Starship Trooper obsession of, “would you like to know more?” What’s interesting is, imagine in your scenario, you’re having this sub vocal conversation, but instead of you having to have any conversation, ChatGPT has heard you and it’s like, ” oh, alter ego, Pablos: No, no, I get it. One of my friends, figured out that you could get through life with only four words, fuck, man, dude, and totally. If you just have those four words, you can get through life because you can express a multitude of things with just those four words. Totally. Ash: Totally. Your response, totally. Funny enough though, right? That may solve some of your problems because you could whisper a little Pablos: Yeah, yeah. Ash: And not have to do long things. Pablos: Yeah. Right. Exactly. No, you’re totally right. And that’s what you do with your friends. And the closer you are to your friends, like if you’re just hanging out with somebody you’ve known for a long time, you can have a lot of communication with very little actual content. If I watch my daughter and her best friend hanging out, they’re incomprehensible because they have like, shortcodes for memes, everything they see or talk about or discuss is related to some other thing that I wasn’t part of and like they’re foreign objects to me. I think that is kind of what you’re describing. Like at some point, Ash: So go back to your Venezuelan, right? If you go back to that conversation and they’re sending you a voice note. Now, let’s say that voice notes processed and parsed and read by our GPT friend, and it comes back and gives you a summary, five sentence. So you don’t even have to look. It just whispers it in your head. Like he wants to know, should he edit the podcast? I don’t know, whatever it is. And you could just go back and be like, just hit the yes button, right? I mean, you could go back and say, totally. You could do one of your four words. Pablos: Yeah, totally. No, you got to try it. I tried it. You can go for days without using any other words. But yeah, I think that gets more possible. Like with a human, the more shared experience you have, the more shared context, shared vocabulary, the more concise you can be in your interactions. And so it stands to reason that an AI that knows you really well could get to the point where. All you gotta do is nod or wink and you’re done, on a lot of things cause it knows how to set you up to make a quick decision. Ash: If it can formulate the outbound response in long form, and all you have to say is totally… Pablos: Mm hmm, yep. Ash: Then you’re good, right? That’s usually the problem with these voices, with getting those voices. I’ve got those too, where people, it is the Latin America thing. They just love, like, I don’t know what’s going on. It was Brazil too, just, people just go off. And they have a recording. I’m like, you do understand, if I could listen to this, I wouldn’t be texting you. That’s like, I would pick up the phone and just phone you if I can, if I could have a dialogue, I would have one. When I saw that, I was like, well, can you just tell me like what’s in the voice recording? That’s what we’re looking for. The other thing to think of, and I thought this is where you were going before, you were talking about the sub vocal thing, It’s almost like the Babelfish thing, for all the fans of Hitchhiker’s Guide. I just had this crazy problem happen, which was, I’d ordered an Uber, and I’m sending information to the Uber driver in English, and the Uber driver is replying in Spanish, but I have a little translate button, but I don’t think they had a translate button. And at some point they just simply just said, no hable ingles. I tried to give the directions to my house, finally, I had to run into the street. I sent my daughter out into the street, like someone went out and we’re trying to tell them like, go to the yellow house. And I’m like, does anyone remember the word yellow? I realized that I was getting translate and they just didn’t speak English. I think that maybe there’s this universal input concept. If someone sends you a voice message, it not just transcribes it, but maybe it automatically just dumps it into like concise format. Or to the other person, it reads it to them. So you pick your poison of consumption, like the way you like to consume it, and you just build a proxy in the sky that just It just takes care of all this. There’s like a universal proxy, like a little babble bot that sits in the world. And I think you could get pretty far with that. And then you use that to feed ChatGPT. And then you use that to go with the totally man, dude, fuck, right? That’s your sequence to that. And then you add your sort of exotic input mechanisms for your sub vocal and everything else. So I could like, you know. Whisper. Pablos: So job one is all the people making AIs need to figure out how to make them mine so that I have my own that I can love and trust and have for life. Job two is they need to make that thing know everything about me, I’m not just a lowest common denominator, I’m me and I need, I need my AI to really know me. Job three is we’ve got to come up with some clever hard…
Pablos: There’s this idea that was just published that you could produce concrete and make it stronger by adding charred coffee grounds to the mix. And this is some research out of Australia. So concrete, if it’s not obvious, is like the most used material on the entire planet, aside from oil, which we burn. Cement, is in everything, and it’s this like staggering scale problem. Partly because of its contribution to greenhouse gases, right? So when you make cement, you’re burning some shit to make a bunch of heat to make the cement and you need that heat and there are ideas to decarbonize cement by electrifying cement plants. But then there’s this chemical process going on, which is the bulk of the carbon emissions. And there’s just no way to get rid of that. So that’s kind of the lay of the land. Interestingly, about half of all the cement in the entire world is made in China. That country is basically made of cement. This is one of the major targets for trying to do reductions of carbon emissions. And these guys figured out how to use coffee grounds. It’s not totally clear to me that they’re using, uh, used coffee grounds, I presume that’s the case, because there’s 10 billion Kilograms of used coffee waste every year that mostly ends up as biomass rotting in landfills. So this is worth solving. I thought this was kind of interesting. You can’t just take the coffee and throw it in the cement because the oils and stuff in it will seep out and actually make the cement fall apart. They invented this pyrolyzing process where you basically heat up the coffee grounds to a specific, pretty high target temperature, around 500 C, I guess. That’ll get rid of the oils presumably, and makes it into an additive you can just throw into the cement mix and it makes it 30 percent stronger. So I got two things that are kind of interesting, related to this. We Have a company our fund backed called DMAT, and these guys figured out how to make cement that’s lower carbon, but the way they do it, is they solved this 2000 year old mystery in material science, which is, how did the Romans make cement? Ash: I was going to bring that up. Pablos: Yeah. Cause they made the, the Pantheon to like two millennia ago and it’s still there. It’s unreinforced concrete in a seismic zone. And then they, somehow got busy, watching Netflix or something and got bored and forgot all about how to make cement. And then nobody’s been able to figure it out ever since. Ash: They were just looking at the colosseum. They were like, Hey, I’d rather look at the lion. Maximus Aurelius or whomever. And then that’s it. They’re like, forget it. Pablos: Look at the cool lion. Oh shit. The lion ate the guy who knows how to make the cement. Ash: Literally probably what happened. Pablos: That is literally probably what happened. So anyway, I got this team at MIT that figured it out. Ash: It was self healing, right? Pablos: We figured that out a little while ago. It’s self healing because what happens with cement is it fractures, water seeps into the cracks and then destroys the cement from the inside out. And that’s what’s happening to our bridges and everything else we made. And so to make it stronger and handle that, we load it up with steel rebar. So it’s steel reinforced, and then it still only lasts 50 years. The Roman cements, apparently lasting at least 2000 years. And what happens is it just gets stronger because when it cracks, water seeps into the cracks and it activates these lime deposits that are trapped in there. And so then the lime fills the crack and seals it up and heals the cement. Presumably the colosseum is just getting stronger over time. Now we know how to do that. So we can make cement that lasts virtually forever, use less of it, use less steel, and the kicker is, it’s about 20 percent less CO2, out of the box without even trying. That’s pretty dramatic considering the, the scale of the problem and the lack of other practical ways of decarbonizing. So these might be compatible, right? You might be able to also use this coffee additive. What I like about this is that cement is such a big thing. Most people just take it for granted. They don’t know how. Intensive this is from a carbon emissions standpoint and the scale of it. this. You know like we can actually make things way, way better. with some of these ideas. Ash: And the way they were doing it, the Romans had volcanic rocks, so they had this ability to automatically have the little bubbles in it. But I think what’s interesting is that, some people are like, oh, can we put plastic? Isn’t that where we just got in trouble with microplastics? Let’s solve one problem and then really screw up something else. The idea I was thinking is maybe this is where the coffee ground becomes like the aeration, right? Cause the whole structure was that as the bubbles popped, that was how the lime. Seeped back in, right? The water combined. Pablos: I think that was one of the theories that was debunked. I’m not positive, but I think that was the, like the prevailing idea, or it was kind of a half baked idea of like how this happened. And I think that is not what actually, it’s nothing to do with the volcanic rock after all. Ash: It wasn’t the volcanic, right? They had a couple , right? One was like some guy was trying to do bacteria. five, six years ago. That was the other crazy one, which was like, we will just have a living organism inside. The other question is, during production, can you trap, can you use it to just trap the stuff? Like, if you look at, was it clean, right? If you look at those guys, Pablos: So that’s what DMAT solved. And they do it with this process called hot mixing. Which apparently was considered dumb for, I don’t know, centuries or something. And so nobody tried it. Apparently using hot mixing they can get the lime deposits optimally trapped in the cement. I don’t know all the details. Ash: I like it. Pablos: Yeah, so we’ll get them on the podcast sometime and have them explain all the all the ins and outs. But yeah, pretty cool stuff. Ash: The challenge with almost all of these carbon reduction technologies is scale. Oh, hey, we’re going to take carbon out of the sky. And it’s like, okay, what did, what was the impact? Well, it’s like half a car. Pablos: Right because the sky is like the most entropic source of carbon there is. Literally, the number 400 parts per million. Well, let’s see. If you had a haystack, and you had, 400 needles and, a million pieces of straw, good luck finding a needle. It’s literally, the hardest possible place to get carbon. If you want to, sequester carbon, the thing to do would be to just, leave the fucking coal in the ground. Where it’s, the highest density of carbon you could find. So yeah, it’s, it’s kind of idiotic. Most of these things kind of solve themselves if you solve energy. If you had like a shit ton of free energy, then yeah, you could go do carbon capture from the atmosphere, but, otherwise it’s pretty painful. Ash: The problem is, yeah, like you said, unless you can turn it back into like a diamond or something, like you said, put it back into coal. These magma guys are, are cranking. Maybe we can use those guys. You’ve heard of the magma guys? Pablos: What’s the magma guys? Ash: These guys were doing the near magma experiment. They’re like, we’re just going to go 6, 000 feet, like just a little over a mile. What’s a mile? 5,280 feet? So you just go a little bit into the mantle. Just tap into that hyper geothermal. Pablos: I don’t know what you’re talking about. Ash: Oh, so there’s a project, just came out a couple of days ago that they revealed that they have a timeline on 2025. They’re going to do two. One is an open magma bubble, it’s in Iceland and then they’re going to do another one on top of it. They’re going to build like a little station and they’re going to go straight down. This is poking the bear, I would say. Pablos: So they’re basically trying to do a man made volcano. Ash: Yes, yes, that’s the, that’s the way to think of it. Pablos: Iceland doesn’t have enough volcanoes. Ash: There’s not enough problems where you could just suddenly drill a hole and burst the pimple of God, right? I don’t Pablos: People are worried about AI, and here we are trying to make a cousin for Eyjafjallajökull. Ash: I like it because someone’s like, “there’s infinite heat.” And I’m like, “yeah, but it’s kind of down there for a reason.” Didn’t work out too well for a lot of people, right? Pablos: I don’t understand, I guess if you succeeded at drilling that hole, then I think you would have basically the same thing as the makings of a volcano. Ash: Yeah, but they’re trying to contain it, right? They somehow feel like, like they could drill in a place… Pablos: You’re going to have to cycle it because if it cools, even if the magma comes up and cools, it’s just going to plug your hole. Ash: So the point is that they have to get a turbine to magma, magma rotating. It’s wild. It’s going to be interesting. just liked the idea that, that someone’s literally poking the bear. Pablos: Oh, they definitely should try. Ash: Cause you know, we talk about fusion being risky, but this one I just feel has a lot more problems. Pablos: Yeah, I think they’re just gonna, the magma is just gonna plug the hole. Ash: No, they’ve got, they’ve got, some ideas. Yeah, well, it is pressure. It’s under pressure. That’s why I keep calling it a pimple. Pablos: Yeah, that’s why volcanoes get made, right? Ash: That’s why they gotta go to Iceland. But, the interesting thing is, if you could technically, if you could maintain pressurization all the way up to the top, right, then it can stay magmatic and you could technically build some sort of, high velocity magma drive. That’s, what they’re thinking of. And that will just keep cycling. Cooling, but just spinning this turbine. Pablos: What do you do with the magma that comes up hot? Ash: It becomes like a, a river. Pablos: You run the turbine, but then where does it go? You gonna pump it back down? Ash: Yeah, it’s as if you were in a magma flow, right? So magma continues to move. It continues, it has a lot of movement, which people don’t realize. Look, the minute I heard drill 6,000 feet into a thin crust lava magma I sort of went, Hmm, this cannot end well. That’s, that’s the way I looked at it. But who knows? Pablos: But it’s just Iceland, so you know, there’s only like 130,000 people there. They’re tough though. If anybody can handle it… Ash: Don’t you remember? Didn’t, they stop all transatlantic flights? You remember right? There’s like a little Ash: cloud and, so just Iceland, but it’s, it’s literally on the jet stream. We Have a few airplanes crossing right over Iceland. No more going to Europe or vice versa. Pablos: Yeah, well, we overdid it anyway. Europe is basically just like a suburb of the U.S. now. Ash: And Brexit. So, you know, Pablos: There’s a lot of people who are trying to figure out how to decarbonize cement and it stalls out in part because there’s like four or five thousand cement plants around the world, and they all cost $100 million to build in the first place. A lot of the ideas for decarbonizing cement require building a new plant. And even if you could build one, you’re not going to build 4,000 of them. They’re Just non starters. And that’s part of why I like DMAT is that they can integrate in any cement plant with basically zero capex. You can just go in and upgrade, turn some knobs, and make a new formula. So, that’s super cool, and hopefully this coffee based additive would have that property as well. Ash: I think what’s interesting is just the coffee part of all this conversation. Pablos: If I go back to that article, it says that there’s, 10 billion kilograms, which is 22 billion pounds of coffee waste a year. I presume this is post consumer grounds. Ash: This is probably commercial coffee grounds that they can track using, like, Starbucks. It doesn’t include what we take home. Pablos: So it’s at least something like three pounds of coffee grounds per human, for every man, woman, and child on Earth. I don’t even drink coffee. So somebody else is doing double. The other one that we, got excited about and backed is this, startup called Marvel Labs. What’s exciting there is they figured out how to use the used coffee grounds as an input material for 3D printers. That sounds like kind of a cute thing, but the truth is it’s staggering implications. And it’s because 3D printers, they’re called rapid prototypers because we used them in labs and they were very expensive and impractical for a long time. And then in 2007, one of my buddies helped start MakerBot, and I was an advisor for MakerBot, which was the first consumer 3D printer. And so we thought we were gonna eventually build farms of these things like AWS, you’d just have a data center full of MakerBots and you’d wire them up to the “buy now” button, and whenever you clicked “buy now,” a MakerBot would print your stuff and then print a box around it and then print a FedEx label on it. It would show up in the mail. Obviously that didn’t happen, and here we are 15 years later, and you don’t buy anything on Amazon that’s 3D printed. There’s two big reasons. One is they’re one pixel printers, so they’re super slow, and that makes it expensive. And then the other part of it is that the input materials are expensive, so you’ve got these high quality filaments, plastic filaments and things that are expensive. At the end of the day, you’re competing with injection molding, which is like the cheapest way of making anything on Earth. And so, it hasn’t worked out. There’s a couple of exceptions. So for example, with metals, 3d printing of metals has worked out pretty well for two reasons. One, they’re higher value parts. So you’re printing, you know, jet parts and rockets and stuff. But also the technique in the printers is it’s a powder bed, so you have this bin of powder, you run over it with a binder, like glue, from an inkjet head or a laser or something to sinter it together, and then, you pick up your part and shake it off, and you’ve got this part that was printed in a bed of dust. It’s actually a very elegant way of making a 3D printer, and it’s faster, because they’re more like layer at a time instead of pixel at a time. Anyway, so what Marvel Labs did is they adapted that style of printer, which is fast, but the input material is these used coffee grounds and what the effect of that is, is now they can print stuff out of coffee. They’re making all kinds of stuff. Sinks and light fixtures and bicycles and things. And the parts come out of the machine. They’re made of coffee and then they just powder coat them with paint or metalize them so they look like metal and you can’t even tell that it’s made of coffee. And so this whole thing works awesome, but the main reason that it’s important , and the reason that we invested, is that it flips the economics. So now, these parts that Marvel Labs is making, they’ve reshored manufacturing, they manufacture stuff in the U. S., they do it fully automated. And the parts are cheaper than doing it in Asia. That’s what’s exciting to me. They’re also printing with seaweed. They’re printing with sawdust. All the technologies they invented to make it work are about, printing with biomass in general. They’re kind of the kingpin. Now we can get this whole vision together of producing things on demand in 3D printers in the U. S. Ash: It’s interesting because several things, right? One is, like you said, it’s not just, the on demand. All of our strategic risk starts to change, right? Think of what happens when, we get to a point where we’re having another pandemic or, I don’t know, they go after Taiwan. Supply chain changes if you’re suddenly local, right? As long as we can get enough coffee into the system, we have enough of our own source material. Pablos: Ha, Ha, ha, ha. As a matter of national securit…
Our TOPPODCAST Picks 
Stay Connected
