The InfoQ Podcast

Latest Episodes:

Yuri Shkuro on Tracing Distributed Systems Using Jaeger Aug 28, 2019

Show notes

The three pillars of observability are logs, metrics, and tracing. Most teams are able to handle logs and metrics, while proper tracing can still be a challenge. On this podcast, we talk with Yuri Shkuro, the creator of Jaeger, author of the book Mastering Distributed Tracing, and a software engineer at Uber, about how the Jaeger tracing backend implements the OpenTracing API to handle distributed tracing. Why listen to the podcast: - Jaeger is an open-source tracing backend, developed at Uber. It also has a collection of libraries that implement the OpenTracing API. - At a high level, Jaeger is very similar to Zipkin, but Jaeger has features not available in Zipkin, including adaptive sampling and advanced visualization tools in the UI. - Tracing is less expensive than logging because data is sampled. It also gives you a complete view of the system. You can see a macro view of the transaction, and how it interacted with dozens of microservices, while still being able to drill down into the details of one service. - If you have only a handful of services, you can probably get away with logging and metrics, but once the complexity increases to dozens, hundreds, or thousands of microservices, you must have tracing. - Tracing does not work with a black box approach to the application. You can't simply use a service mesh then add a tracing framework. You need correlation between a single request and all the subsequent requests that it generates. A service mesh still relies on the underlying components handling that correlation. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZlvMyR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZlvMyR

Louise Poubel on the Robotic Operating System Aug 19, 2019

Show notes

ROS is the Robotic Operating System. It’s been used by thousands of developers to prototype and create a robotic application. ROS can be found on robotics in warehouses, self-driving car companies, and on the International Space Station. Louise Poubel is an engineer working with Open Robotics. Today on the podcast, she talks about what it takes to develop software that moves in physical space, including the Sense, Think, Act Cycle, the developer experience, and architecture of ROS. Why listen to this podcast: - Writing code for robot development, you use the Sense, Think, Act Cycle. - ROS is an SDK for robotics. It provides a communication layer that enables data to flow between nodes that handle sensors, logic, and actuation. - ROS has two versions and has been around for twelve years. ROS 1 was entirely implemented in C. ROS 2 offers is a common C layer with implementations in many different languages, including Java, JavaScript, and Rust. - Released on a six-month cadence, Dashing was the latest release (May 2019). Previous releases were supported for one year, Dashing is the first LTS and will be supported for two years. - ROS 2 builds on top of the standard Data Distribution Service (DDS) that you find in mission-critical systems like nuclear power and airplanes. - Simulation is an important step in robotics. It allows you to prototype a system before deploying to a physical system. - Rviz is a three-dimensional visualizer used to visualize robots, the environments they work in, and sensor data. It is a highly configurable tool, with many different types of visualizations and plugins. It allows you to put together all your data in one place and see it. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2ZbOvrO You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2ZbOvrO

Matt Klein on Envoy Mobile, Platform Complexity, and a Universal Data Plane API for Proxies Aug 09, 2019

Show notes

In this podcast we sit down with Matt Klein, software plumber at Lyft and creator of Envoy, and discuss topics including the continued evolution of the popular proxy, the strength of the open source Envoy community, and the value of creating and implementing standards throughout the technology stack. We also explore the larger topic of cloud natives platforms, and discuss the tradeoffs between using a simple and opinionated platform against something that is bespoke and more configurable, but also more complex. Related to this, Matt shares his thoughts on when and how to make the decision within an organisation to embrace technology like container orchestration and service meshes. Finally, we explore the creation of the new Envoy Mobile project. The goal of this project is to expand the capabilities provided by Envoy all the way out to mobile devices powered by Android and iOS. For example, most current user-focused traffic shifting that is conducted at the edge is implemented with coarse-grained approaches via by BGP and DNS, and using something like Envoy within mobile app networking stacks should allow finer-grained control. Why listen to this podcast: - The Envoy Proxy community has grown from strength-to-strength over the last year, from the inaugural EnvoyCon that ran alongside KubeCon NA 2018, to the increasing number of code contributions from engineers working across the industry - Attempting to create a community-driven “universal proxy data plane” with clearly defined APIs, like Envoy’s XDS API, has allowed vendors to collaborate on a shared abstraction while still allowing room for “differentiated success” to be built on top of this standard Google’s gRPC framework is adopting the Envoy XDS APIs, as this will allow both Envoy and gRPC instances to be operated via a single control plane, for example, Google Cloud Platform’s Traffic Director service. - There is a tendency within the software development industry to fetishise architectures that are designed and implemented by the unicorn tech companies, but not every organisation operates at this scale. - However, there has also been industry pushback against the complexity that modern platform components like container orchestration and service meshes can introduce to a technology stack. - Using a platform within these components provides the best return on investment when an organisation’s software architecture and development teams have reached a certain size. - Function-as-a-Service (Faas)-type platforms will most likely be how engineers will interact with software in the future. Business-focused developers often do not want to interact with the platform plumbing Envoy Mobile is building on prior art, and aims to expand the capabilities provided by Envoy all the way out to mobile devices using Android and iOS. Most current end user traffic shifting is implemented with coarse-grained approaches via BGP and DNS, and using something like Envoy instead will allow finer-grained control. - Using Envoy Mobile in combination with Protocol Buffers 3, which supports annotations on APIs, can facilitate working with APIs offline, configuring caching, and handling poor networking conditions. One of the motivations for this work is that small increases in application response times can lead to better business outcomes. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/33nlGMu You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/33nlGMu

Armon Dadgar on HashiCorp Research, the Evolution of Infrastructure Tooling, and Standardisation Aug 02, 2019

Show notes

On this podcast, we’re talking to Armon Dadgar, co-founder and CTO of HashiCorp. Alongside Mitchell Hashimoto, Armon founded HashiCorp over six years ago, and the company has gone from strength to strength, with their open source infrastructure product suite now consisting of Consul, Nomad, Vault and Terraform. We discuss the formation of the HashiCorp research division, and explore some of the computer science research underpinning Consul and Nomad. We also cover the challenges of supporting teams when they are looking to embrace new modes of working with dynamic infrastructure, and Armon introduces the new learn.hashicorp.com educational website and accompanying community and support forums. Why listen to this podcast: - There is a lot of fundamental computer science research that underpins the HashiCorp infrastructure workflow and configuration tooling. This helps to ensure that these mission-critical tools perform as expected, and enables sound reasoning about scaling these technologies. - The HashiCorp founders recognised the value of creating an industrial research-focused department within the company even when there were only 30 staff. - The Consul service mesh and distributed key value store leverages consensus and gossip algorithms from computer science research, Raft and SWIM, respectively. The HashiCorp team contributed a novel research-based improvement to SWIM -- Lifeguard: SWIM-ing with Situational Awareness -- that was presented at the DSN academic conference - Initially HashiCorp produced a new tool every 6-12 months, focusing on filling gaps within the infrastructure workflow tooling market. Now the focus is on refining the operator/user experience of the existing tools, creating more integrations with other platforms and tooling, and facilitating engineering teams adopting these tools, via the creation of educational resources and community forums. - Standardisation within computing technology can offer many benefits, especially where interoperability is required or technology switching costs are high. Care must be taken to ensure the correct interfaces are created, and that the time is right to create appropriate abstractions. - The HashiCorp team are focusing on "marching up the stack", with the goal that a lot of the underlying "plumbing" should be hidden from, or easily configurable by, application developers. This will allow developers to focus on adding value related to their business or organisation, rather than getting stuck with managing infrastructure. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2KptB3d You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2KptB3d

Kingsley Davies and Cat Swetel at QCon London about Ethics and Requisite Variety Jul 29, 2019

Show notes

In this episode recorded at QCon London 2019 Shane Hastie, Lead Editor for Culture & Methods, first spoke to Kingsley Davies about ethics and then with Cat Swetel about requisite variety and being mindful of the impact our decisions have for the future. Why listen to this podcast: • The need to explore the application of technology for good • The need for ethical standards in the technology industry • Data is the new oil and it is frequently used in ways that are not in the best interest of society • Other engineering professions have codes of conduct and ethical frameworks that are mandated as part of the education process, software engineering currently has very little • Ashby’s law of requisite variety – the more options that are available to a system, the more resilient the system is applies to all aspects of our socio-technical systems • We exist in the realm of ethics – we can’t just go to work and do what you’re told. Everything we do is a choice and our choices have a huge impact on the future

Thomas Wuerthinger on GraalVM and Optimizing Java With Ahead-of-Time Compilation Jul 19, 2019

Show notes

The promise of Java has always been, “write once, run anywhere.” This was enabled through just-in-time compilation, which allowed developers to target a platform at compilation. But, this flexibility has given rise to comments like, “Java is slow.” What if you could compile Java to Native Code? On this podcast, we’re talking to Thomas Wuerthinger, a senior research director at Oracle Labs. Leading programming language implementation teams for Java, JavaScript, Ruby, and R. He is the architect of the Graal compiler and the Truffle self-optimizing runtime. Why listen to the podcast: - The GraalVM project was initially just a replacement for the JVM C2 just-in-time compiler, but has evolved to include support for multiple languages, as well as an ahead-of-time compiling mode. - Support for multiple languages can provide better performance for some languages, as well as making direct calls without inter-process communication. - With GraalVM’s AOT compilation, you can statically link system libraries, which allows you to run a static binary on a bare-metal Docker image, without even a Linux distribution. - The major benefits of AOT are minimized startup time, memory footprint, and packaging size. This can come with a trade-off in reduced maximum throughput and higher latency. - The GraalVM roadmap includes supporting additional platforms, such as Windows and mobile, as well as performance improvements for both the JIT and AOT compilers. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2Y2hPk2 You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2Y2hPk2

Johnny Xmas on Web Security & the Anatomy of a Hack Jun 17, 2019

Show notes

On this podcast, Wes talks to John Xmas. Johnny works for Kasada, a company that offers a security platform to help ensure only your users are logging into your web applications. Johnny is a well-known figure in the security space. The two discuss common attack vectors, the OWASP Top 10, and then walk through what hackers commonly do attempting to compromise a system. The show is full of advice on protecting your systems including topics around Defense in Depth, Time-Based Security, two-factor authentication, logging/alerting, security layers, and much more. Why listen to this podcast: - While there are sophisticated web attacks out there that use things like PhantomJS or Headless Chome, the vast majority of the web application attacks are the same unsophisticated scripted attacks that you always hear about. These are simple scripts using tools like curl and BurpSuite with Python or JavaScript. These simple scripts are still incredibly effective. - OWASP Top 10 really hasn’t changed all that much in the last ten years. For example, despite being the number one approach used to educate defensive engineers on how to protect their apps, SQLI (SQL Injection) is still the most common attack. We continue to repeat the same mistakes that have exposed systems for a decade now. - Phishing is by and far the quickest way to compromise a system. Defensive in Depth, security boundaries, limiting local admin rights are all things that corporations can implement to minimize the blast radius. - Attackers have hundreds of gigs of actual username/password combinations that have been exposed from all the breaches over the past few years. These are often a first step when attempting to compromise a system. It’s more often likely that they will figure out a valid email pattern for a company and then feed actual names into that pattern to go after the username. From there, brute force attacks with those usernames against libraries of passwords is a common approach. - A common approach is to go after an email login. While the email can be a treasure trove of information, it’s more about using those credentials in other places. It’s pretty common, for example, to use those credentials to get into a network with a VPN. - Captcha/reCaptcha is not very effective and preventing these brute force attacks. There are a large number of bypasses and even Mechanical Turk companies that are available to bypass these tools. What can be effective is Time Based Security because it slows the attackers down. If you can slow them down, you can make the attack say long to succeed that they’ll go somewhere else. - Once inside the network, most companies often have little security on internal systems. Multi-factor authentication, not just on the front door, but on internal systems is a huge step in the right direction. Monitoring not only for failed login attempts but, in some situations, valid login attempts (such as when a domain admin logs into a domain controller) should absolutely be used. - When it comes to application security between services within a network, the best advice is to make sure developers really understand what is trying to be accomplished by something like JWT (JSON Web Tokens). Often its the lack of understanding of what they’re actually doing that leads to system vulnerabilities. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2MSIAXG You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2MSIAXG

Mike Milinkovich, Director of the Eclipse Foundation, Discusses the Journey to Jakarta EE 8 Jun 03, 2019

Show notes

Today on the podcast, Wes talks with Mike Milinkovich, Executive Director of the Eclipse Foundation. The Eclipse Foundation was chosen to govern the evolution of Oracle’s Java EE to Jakarta EE. The two discuss the project, the recent news about issues with the javax namespace, the challenges around bundling a Java Runtime with Eclipse, and the path forward for Jakarta EE 9 and beyond. Why listen to this podcast: - Java EE, unlikely Java SE, has always been a multi-vendor ecosystem. It made sense for everyone for Oracle to invite their partners to be involved in the governance of the specification for Java EE for it to continue moving forward. This is the reason for moving Java EE into the Eclipse Foundation as Jakarta EE. - The current plan is for the Eclipse Foundation to get a copyright license to evolve the text of the specification and not a license to the trademarks of Java EE itself. - The javax namespace must remain as is. For it to be evolved, a different namespace must be used. The javax namespace is a trademark of Oracle. Because of this, there are quality controls that Oracle required for its evolution. Ultimately because of those controls, the Eclipse Foundation felt it was better to branch javax into a different namespace and evolve it separately solely under Jakarta EE governance. - Jakarta EE 8 is targeted to be released around Oracle Code ONE. Jakarta EE 8 will be exactly the same as Java EE 8. * The only difference is it will be licensed from Jakarta, not Oracle and only requires membership in the Working Group. - Beyond EE 8, the release cycle, the plan for moving the javax namespace (and keeping compatibility with both the old javax namespace and the new namespace), and new specifications for inclusion into Jakarta EE are still active areas of discussion. - Unrelated to the discussion of Jakarta EE (but discussed in the same board meeting), an attempt to bundle OpenJ9 with the Eclipse IDE failed because of licensing restrictions around a certified Java Runtime. OpenJ9 is certified when acquired through an IBM channel, but not when downloaded directly for us. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2HSfcfM You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2HSfcfM

Piero Molino on Ludwig, a Code-Free Deep Learning Toolbox May 24, 2019

Show notes

Ludwig is a code-free deep learning toolbox originally created and open sourced by UberAI. Today, on the podcast the creator of Ludwig Piero Molino and Wes Reisz discuss the project. The two talk about how the project works, its strengths, it’s roadmap, and how it’s being used by companies inside (and outside) of Uber. They wrap by discussing path ahead for Ludwig and how you can get involved with the project. Why listen to this podcast: • Uber AI is the research and platform team for everything AI at the company with the exception of self-driving cars. Self-driving cars are left to Uber ATG. • Ludwig allows you to specify a Tensorflow model in a declarative format that focuses on your inputs and outputs. Ludwig then builds a model that can deal with those types of inputs and outputs without a developer explicitly specifying how that is done. • Because of Ludwig’s datatype abstraction for inputs and outputs, there is a huge range of applications that can be created. For example, an input could be text and output could be a category. In this case, Ludwig will create a text classifier. An image and text input (such as a question: “Is there a dog in this image”) would output a question answering system. There are many combinations that are possible with Ludwig. • Uber is using Ludwig for text classification for customer support. • Datatypes can be extended easily with Ludwig for custom use cases. • Ludwig would love to have people contribute to the project. There are simple feature requests that are just not prioritized with the current contributor workload. It’s a great place to get involved with machine learning and gain experience with the project. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2JGA5wC You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2JGA5wC From time to time InfoQ publishes trend reports on the key topics we’re following, including a recent one on DevOps and Cloud. So if you are curious about how we see that state of adoption for topics like Kubernetes, Chaos Engineering, or AIOps point a browser to http://infoq.link/devops-trends-2019.

Ben Sigelman, Co-Creator of Dapper & OpenTracing API, on Observability May 05, 2019

Show notes

Ben Sigelman is the CEO of Lightstep and the author of the Dapper paper that spawned distributed tracing discussions in the software industry. On the podcast today, Ben discusses with Wes observability, and his thoughts on logging, metrics, and tracing. The two discuss detection and refinement as the real problem when it comes to diagnosing and troubleshooting incidents with data. The podcast is full of useful tips on building and implementing an effective observability strategy. Why listen to this podcast: - If you’re getting woke up for an alert, it should actually be an emergency. When that happens, things to think about include: when did this happen, how quickly is it changing, how did it change, and what things in my entire system are correlated with that change. - A reality that seems to be happening in our industry is that we’re coupling the move to microservices with a move to allowing teams to fully self-determine technology stacks. This is dangerous because we’re not at the stage where all languages/tools/frameworks are equivalent. - While a service mesh offers a great potential for integrations at layer 7 many people have unrealistic expectations on how much observability will be enabled by a service mesh. The service mesh does a great job of showing you the communication between the services, but often the details get lost in the work that’s being done inside the service. Service owners need to still do much more work to instrument applications. - Too many people focus on the 3 Pillars of Observability. While logs, metrics, and tracing are important, observability strategy ought to be more focused on the core workflows and needs around detection and refinement. - Logging about individual transactions is better done with tracing. It’s unaffordable at scale to do otherwise. - Just like logging, metrics about individual transactions are less valuable. Application level metrics such as how long a queue is are metrics that are truly useful. - The problem with metrics are the only tools you have in a metrics system to explain the variations that you’re seeing is grouping by tags. The tags you want to group by have high cardinality, so you can’t group them. You end up in a catch 22. - Tracing is about taking traces and doing something useful with them. If you look at hundreds or thousands of tracing, you can answer really important questions about what’s changing in terms of workloads and dependencies about a system with evidence. - When it comes to serverless, tracing is more important than ever because everything is so ephemeral. Node is one of the most popular serverless languages/frameworks and, unfortunately, also one of the hardest of all to trace. - The most important thing is to make sure that you choose something portable for the actual instrumentation piece of a distributed tracing system. You don’t want to go back and rip out the instrumentation because you want to switch vendors. This is becoming conventional wisdom. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2PPIdeE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2PPIdeE

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected