BSD Now

Latest Episodes:

297: Dragonfly In The Wild May 09, 2019

Show notes

FreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more.

Headlines

FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS. Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices. FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out. Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.

DragonFlyBSD 5.4.2 is out

Upgrading guide

Here's the tag commit, for what has changed from 5.4.1 to 5.4.2 The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently. If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.

> cd /usr/src
> git pull
> make buildworld.
> make buildkernel.
> make installkernel.
> make installworld
> make upgrade

After your next reboot, you can optionally update your rescue system:

> cd /usr/src
> make initrd

As always, make sure your packages are up to date:

> pkg update
> pkg upgrade

News Roundup

Containing web services with iocell

I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛 If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U

Getting started

There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.

To start, you'll need the following:
- A FreeBSD install (we'll be using 11.0)
- The iocell package (available as a package, also in the ports tree)
- A ZFS pool for hosting the jails

Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.

As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.

See Article for the rest of the walkthrough.

Oracle Solaris 11.4 SRU8

Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.

This SRU introduces the following enhancements:

Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.

UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4

Re-introduction of the service fc-fabric.

ibus has been updated to 1.5.19

The following components have also been updated to address security issues:
- NTP has been updated to 4.2.8p12
- Firefox has been updated to 60.6.0esr
- BIND has been updated to 9.11.6
- OpenSSL has been updated to 1.0.2r
- MySQL has been updated to 5.6.43 & 5.7.25
- libxml2 has been updated to 2.9.9
- libxslt has been updated to 1.1.33
- Wireshark has been updated to 2.6.7
- ncurses has been updated to 6.1.0.20190105
- Apache Web Server has been updated to 2.4.38
- perl 5.22
- pkg.depot

The Problem with SSH Agent Forwarding

After hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].” Here’s what man ssh_config has to say about ForwardAgent: "Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent."" Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine! Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.

[OpenBSD Upgrade Guide: 6.4 to 6.5

Start by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u. Alternatively, you can use the manual upgrade process. You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.

Before rebooting into the install kernel
Configuration and syntax changes
Files to remove
Special packages
Upgrade without the install kernel

Beastie Bits

Feedback/Questions

Casey - Oklahoma City & James
Michael - Question on SAS backplane (camcontrol?)
Ales - OpenBSD, FreeNAS, OpenZFS questions

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

296: It’s Alive: OpenBSD 6.5 May 03, 2019

Show notes

OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode.

Headlines

OpenBSD 6.5 Released

Changelog
Mirrors
6.5 Includes
- OpenSMTPD 6.5.0
- LibreSSL 2.9.1
- OpenSSH 8.0
- Mandoc 1.14.5
- Xenocara
- LLVM/Clang 7.0.1 (+ patches)
- GCC 4.2.1 (+ patches) and 3.3.6 (+ patches)
Many pre-built packages for each architecture:
- aarch64: 9654
- amd64: 10602
- i386: 10535

Mount your ZFS datasets anywhere you want

ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility. When you create zpool maintank, the default mountpoint is /maintank. You might be happy with that, but you don’t have to be content. You can do magical things.

Some highlights are:
- mount point can be inherited
- not all filesystems in a zpool need to be mounted
- each filesystem (directory) can have different ZFS characteristics
- In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently.

News Roundup

Branch for netbsd 9 upcoming, please help and test -current

Folks, once again we are quite late for branching the next NetBSD release (NetBSD 9). Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that. On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later. On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:

please test -current, on all the various machines you have

especially interesting would be test results from uncommon architectures or strange combinations (like the sparc userland on sparc64 kernel issue I ran in yesterday) Please test, report success, and file PRs for failures! We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may. We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go. Our goal is to have an unprecedented short release cycle this time. But.. we always say that upfront.

LibreSSL 2.9.1 Released

We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL directory of your local OpenBSD mirror soon. This is the first stable release from the 2.9 series, which is also included with OpenBSD 6.5

It includes the following changes and improvements from LibreSSL 2.8.x:

API and Documentation Enhancements
- CRYPTO_LOCK is now automatically initialized, with the legacy callbacks stubbed for compatibility.
- Added the SM3 hash function from the Chinese standard GB/T 32905-2016.
- Added the SM4 block cipher from the Chinese standard GB/T 32907-2016.
- Added more OPENSSLNO* macros for compatibility with OpenSSL.
- Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH.
- Implemented further missing OpenSSL 1.1 API.
- Added support for XChaCha20 and XChaCha20-Poly1305.
- Added support for AES key wrap constructions via the EVP interface.
Compatibility Changes
- Added pbkdf2 key derivation support to openssl(1) enc.
- Changed the default digest type of openssl(1) enc to sha256.
- Changed the default digest type of openssl(1) dgst to sha256.
- Changed the default digest type of openssl(1) x509 -fingerprint to sha256.
- Changed the default digest type of openssl(1) crl -fingerprint to sha256.
Testing and Proactive Security
- Added extensive interoperability tests between LibreSSL and OpenSSL 1.0 and 1.1.
- Added additional Wycheproof tests and related bug fixes.
Internal Improvements
- Simplified sigalgs option processing and handshake signing algorithm selection.
- Added the ability to use the RSA PSS algorithm for handshake signatures.
- Added bnrandinterval() and use it in code needing ranges of random bn values.
- Added functionality to derive early, handshake, and application secrets as per RFC8446.
- Added handshake state machine from RFC8446.
- Removed some ASN.1 related code from libcrypto that had not been used since around 2000.
- Unexported internal symbols and internalized more record layer structs.
- Removed SHA224 based handshake signatures from consideration for use in a TLS 1.2 handshake.
Portable Improvements
- Added support for assembly optimizations on 32-bit ARM ELF targets.
- Added support for assembly optimizations on Mingw-w64 targets.
- Improved Android compatibility
Bug Fixes
- Improved protection against timing side channels in ECDSA signature generation.
- Coordinate blinding was added to some elliptic curves. This is the last bit of the work by Brumley et al. to protect against the Portsmash vulnerability.
- Ensure transcript handshake is always freed with TLS 1.2.

The LibreSSL project continues improvement of the codebase to reflect modern, safe programming practices. We welcome feedback and improvements from the broader community. Thanks to all of the contributors who helped make this release possible.

FreeBSD Mastery: Jails – Bail Bond Denied Edition

I had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.) I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly. I downgraded my expectations and tried again. And again. And again. I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition! This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort. But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character. Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile. As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough. And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.

One reason ed(1) was a good editor back in the days of V7 Unix

It is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does. The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented. This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.

Beastie Bits

Feedback/Questions

Brad - iocage
Frank - Video from Level1Tech and a question
Niall - Revision Control

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

295: Fun with funlinkat() Apr 25, 2019

Show notes

Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more.

Headlines

Introducing funlinkat

It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition

One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove. The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:

We will remove an entry in the directory with the filename.

We will decrease a file reference count (in inode).

If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself. Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time. The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.

When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete. We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.

Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it. In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel. This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file. After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor. int funlinkat(int dfd, const char *path, int fd, int flags); The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it. The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat. As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.

Using an OpenBSD Router with AT&T U-Verse

I upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over. Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.

News Roundup

How to use NetBSD on a Raspberry Pi

Do you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered "yes" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi. BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix. Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.

ZFS Encryption is still under development (as of March 2019)

One of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.

CFT for FreeBSD + ZoL

Tutorial On Rump Kernel Servers and Clients

The rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server "mini-operating systems". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand. This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.

Installing Snort on OpenBSD 6.4

As you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!! I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!

Beastie Bits

Feedback/Questions

Malcolm - Laptop Experience : Dell XPS 13
DJ - Feedback
Alex - GhostBSD and Wifi : FIXED

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

294: The SSH Tarpit Apr 18, 2019

Show notes

A PI-powered Plan 9 cluster, an SSH tarpit, rdist for when Ansible is too much, falling in love with OpenBSD again, how I created my first FreeBSD port, the Tilde Institute of OpenBSD education and more.

Headlines

A Pi-Powered Plan 9 Cluster

Plan 9 from Bell Labs comes from the same stable as the UNIX operating system, which of course Linux was designed after, and Apple’s OS X runs on top of a certified UNIX operating system. Just like UNIX, Plan 9 was developed as a research O/S — a vehicle for trying out new concepts — with it building on key UNIX principles and taking the idea of devices are just files even further. In this post, we take a quick look at the Plan 9 O/S and some of the notable features, before moving on to the construction of a self-contained 4-node Raspberry Pi cluster that will provide a compact platform for experimentation.

Endlessh: an SSH Tarpit

I’m a big fan of tarpits: a network service that intentionally inserts delays in its protocol, slowing down clients by forcing them to wait. This arrests the speed at which a bad actor can attack or probe the host system, and it ties up some of the attacker’s resources that might otherwise be spent attacking another host. When done well, a tarpit imposes more cost on the attacker than the defender. The Internet is a very hostile place, and anyone who’s ever stood up an Internet-facing IPv4 host has witnessed the immediate and continuous attacks against their server. I’ve maintained such a server for nearly six years now, and more than 99% of my incoming traffic has ill intent. One part of my defenses has been tarpits in various forms.

News Roundup

rdist(1) – when Ansible is too much

The post written about rdist(1) on johan.huldtgren.com sparked us to write one as well. It's a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it. There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them. We didn't have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn't any interest in building additional logic on top of rsync or repositories. > Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing.

Falling in love with OpenBSD again

I was checking the other day and was appalled at how long it has been since I posted here. I had been working a job during 2018 that had me traveling 3,600 miles by air every week so that is at least a viable excuse. So what is my latest project? I wanted to get something better than the clunky old T500 “freedom laptop” that I could use as my daily driver. Some background here. My first paid gig as a programmer was on SunOS 4 (predecessor to Solaris) and Ultrix (on a DEC MicroVAX). I went from there to a Commodore Amiga (preemptive multitasking in 1985!). I went from there to OS/2 (I know, patron saint of lost causes) and then finally decided to “sell out” and move to Windows as the path of least resistance in the mid 90’s. My wife bought me an iPod literally just as they started working with computers other than Macs and I watched with fascination as Apple made the big gamble and moved away from PowerPC chips to Intel. That was the beginning of the Apple Fan Boi years for me. My gateway drug was a G4 MacMini and I managed somehow to get in on the pre-production, developer build of an Intel-based Mac. I was quite happy on the platform until about three years ago.

How I Created My First FreeBSD Port

I created my first FreeBSD port recently. I found that FreeBSD didn't have a port for GoCD, which is a continuous integration and continuous deployment (CI/CD) system. This was a great opportunity to learn how to build a FreeBSD port while also contributing back to the community

The Tilde Institute of OpenBSD Education

Welcome to tilde.institute! This is an OpenBSD machine whose purpose is to provide a space in the tildeverse for experimentation with and education of the OpenBSD operating system. A variety of editors, shells, and compilers are installed to allow for development in a native OpenBSD environment. OpenBSD's httpd(8) is configured with slowcgi(8) as the fastcgi provider and sqlite3 available. This allows users to experiment with web development using compiled CGI in C, aka the BCHS Stack. In addition to php7.0 and mysql (mariadb) by request, this provides an environment where the development of complex web apps is possible.

Beastie Bits

SoloBSD 19.03-STABLE
WireGuard for NetBSD
[NetBSD - Removing PF](https://mail-index.netbsd.org/tech-kern/2019/03/29/msg024883.html )
What does the N in nmake stand for?
A Map of the Internet from May 1973
NSA-B-Gone : A sketchy hardware security device for your x220

Feedback/Questions

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

293: Booking Jails Apr 11, 2019

Show notes

This week we have a special episode with a Michael W. Lucas interview about his latest jail book that’s been released. We’re talking all things jails, writing, book sponsoring, the upcoming BSDCan 2019 conference, and more.

###Interview - Michael W. Lucas - mwl@mwl.io / @mwlauthor
FreeBSD Mastery: Jails

BR: Welcome back to the show and congratulations on your latest book. How many books did you have to write before you could start on FreeBSD Mastery: Jails?
AJ: How much research did you have to do about jails?
BR: The book talks about something called ‘incomplete’ jails. What do you mean by that?
AJ: There are a lot of jail management frameworks out there. Why did you chose to write about iocage in the book?
BR: How many jails do you run yourself?
AJ: Can you tell us a bit about how you handle book sponsorship these days?
BR: What other books (fiction and non-fiction) are you currently working on?
AJ: Which talks are you looking forward to attend at the upcoming BSDCan conference?
BR: How is the BSD user group going?
AJ: Anything else you’d like to mention before we release you from our interview jail cell?

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

292: AsiaBSDcon 2019 Recap Apr 04, 2019

Show notes

FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more.

##Headlines
###AsiaBSDcon 2019 recap

Both Allan and I attended AsiaBSDcon 2019 in Tokyo in mid march. After a couple of days of Tokyo sightseeing and tasting the local food, the conference started with tutorials.
Benedict gave his tutorial about “BSD-based Systems Monitoring with Icinga2 and OpenSSH”, while Allan ran the FreeBSD developer summit.
On the next day, Benedict attended the tutorial “writing (network) tests for FreeBSD” held by Kristof Provost. I learned a lot about Kyua, where tests live and how they are executed. I took some notes, which will likely become an article or chapter in the developers handbook about writing tests.
On the third day, Hiroki Sato officially opened the paper session and then people went into individual talks.
Benedict attended

Adventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel
Vadot

powerpc64 architecture support in FreeBSD ports by Piotr Kubaj
Managing System Images with ZFS by Allan Jude
FreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz
Security Fantasies and Realities for the BSDs by George V.
Neville-Neil
ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

Allan attended

Adventures in DRMland by Emmanuel Vadot
Intel HAXM by Kamil Rytarowski
BSD Solutions in Australian NGOs
Container Migration on FreeBSD by Yuhei Takagawa
Security Fantasies and Realities for the BSDs by George Neville-Neil

ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

When not in talks, time was spent in the hallway track and conversations would often continue over dinner.
Stay tuned for announcements about where AsiaBSDcon 2020 will be, as the Tokyo Olympics will likely force some changes for next year. Overall, it was nice to see people at the conference again, listen to talks, and enjoy the hospitality of Japan.

###FreeBSD Quarterly Status Report - Fourth Quarter 2018

Since we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered:
i386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves.
Additionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group.
We would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having.

###GhostBSD: A Solid Linux-Like Open Source Alternative

The subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms.
GhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface.
I stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest.
In last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux.
GhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes.
Keep reading to find out what piqued my excitement about Linux-like GhostBSD.

##News Roundup
###SPARCbook 3000ST - The coolest 90s laptop

A few weeks back I managed to pick up an incredibly rare laptop in immaculate condition for $50 on Kijiji: a Tadpole Technologies SPARCbook 3000ST from 1997 (it also came with two other working Pentium laptops from the 1990s).
Sun computers were an expensive desire for many computer geeks in the 1990s, and running UNIX on a SPARC-based laptop was, well, just as cool as it gets. SPARC was an open hardware platform that anyone could make, and Tadpole licensed the Solaris UNIX operating system from Sun for their SPARCbooks. Tadpole essentially made high-end UNIX/VAX workstations on costly, unusual platforms (PowerPC, DEC Alpha, SPARC) but only their SPARCbooks were popular in the high-end UNIX market of the 1990s.

###OpenSSH 8.0 Releasing With Quantum Computing Resistant Keys

OpenSSH 7.9 came out with a host of bug fixes last year with few new features, as is to be expected in minor releases. However, recently, Damien Miller has announced that OpenSSH 8.0 is nearly ready to be released. Currently, it’s undergoing testing to ensure compatibility across supported systems.

https://twitter.com/damienmiller/status/1111416334737244160

Better Security
Copying filenames with scp will be more secure in OpenSSH 8.0 due to the fact that copying filenames from a remote to local directory will prompt scp to check if the files sent from the server match your request. Otherwise, an attack server would theoretically be able to intercept the request by serving malicious files in place of the ones originally requested. Knowing this, you’re probably better off never using scp anyway. OpenSSH advises against it:
“The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead.”

Interesting new features

ssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you.

###Project Trident : 18.12-U8 Available

Thank you all for your patience! Project Trident has finally finished some significant infrastructure updates over the last 2 weeks, and we are pleased to announce that package update 8 for 18.12-RELEASE is now available.
To switch to the new update, you will need to open the “Configuration” tab in the update manager and switch to the new “Trident-release” package repository. You can also perform this transition via the command line by running: sudo sysup --change-train Trident-release

##Beastie Bits

##Feedback/Questions

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

291: Storage Changes Software Mar 28, 2019

Show notes

Storage changing software, what makes Unix special, what you need may be “pipeline +Unix commands”, running a bakery on Emacs and PostgreSQL, the ultimate guide to memorable tech talks, light-weight contexts, and more.

##Headlines

###Tracking a storage issue led to software change

Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.

Problem

While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.

###What Makes Unix Special

Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.

What Makes Unix Special?

It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)

##News Roundup
###What you need may be “pipeline +Unix commands” only

I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
BTW, if your data set can be disposed by an awk script, it should not be called “big data”.

Taco Bell Programming

###Running a bakery on Emacs and PostgreSQL

Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face of it, this is really simple, just work out how much dough you need, then work out what quantities to mix to make that much dough. Easy. You can do it with a pencil and paper. Or, in traditional bakers’ fashion, by scrawling with your finger on a floured work bench.
And that’s how I coped for a few weeks early on. But I kept making mistakes, which makes for an inconsistent product (bread is very forgiving, you have to work quite hard to make something that isn’t bread, but consistency matters). I needed to automate.

###The Ultimate Guide To Memorable Tech Talks

Imagine this. You’re a woman in a male-dominated field. English is not your first language. Even though you’re confident in your engineering work, the thought of public speaking and being recorded for the world to see absolutely terrifies you.
That was me, five years ago. Since then, I’ve moved into a successful career in Developer Advocacy and spoken at dozens of technical events in the U.S. and worldwide.
I think everyone has the ability to deliver stellar conference talks, which is why I took the time to write this post.

The Ultimate Guide
1: Introduction
2: Choosing a Topic
3: Writing a Conference Proposal (or CFP)
4: Tools of the Trade
5: Planning and Time Estimation
6: Writing a Talk
7: Practice and Delivery

###Light-weight Contexts: An OS Abstraction for Safety and Performance (2016)

Abstract: “We introduce a new OS abstraction—light-weight con-texts (lwCs)—that provides independent units of protection, privilege, and execution state within a process. A process may include several lwCs, each with possibly different views of memory, file descriptors, and access capabilities. lwCs can be used to efficiently implement roll-back (process can return to a prior recorded state),isolated address spaces (lwCs within the process may have different views of memory, e.g., isolating sensitive data from network-facing components or isolating different user sessions), and privilege separation (in-process reference monitors can arbitrate and control access).
lwCs can be implemented efficiently: the overhead of a lwC is proportional to the amount of memory exclusive to the lwC; switching lwCs is quicker than switching kernel threads within the same process. We describe the lwC abstraction and API, and an implementation of lwCs within the FreeBSD 11.0 kernel. Finally, we present an evaluation of common usage patterns, including fast roll-back, session isolation, sensitive data isolation, and in-process reference monitoring, using Apache, nginx, PHP,and OpenSSL.”

##Beastie Bits

##Feedback/Questions

Mike - FreeBSD Update and Erased EFI files
Charles - Volunteer work
Jake - Bhyve Front Ends
We’ve hit that point where we are running low on your questions, so if you have any questions rolling around in your head that you’ve not thought of to ask yet… send them in!

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

290: Timestamped Notes Mar 21, 2019

Show notes

FreeBSD on Cavium ThunderX, looking at NetBSD as an OpenBSD user, taking time-stamped notes in vim, OpenBSD 6.5 has been tagged, FreeBSD and NetBSD in GSoC 2019, SecBSD: an UNIX-like OS for Hackers, and more.

##Headlines
###ARM’d and dangerous: FreeBSD on Cavium ThunderX (aarch64)

While I don’t remember for how many years I’ve had an interest in CPU architectures that could be an alternative to AMD64, I know pretty well when I started proposing to test 64-bit ARM at work. It was shortly after the disaster named Spectre / Meltdown that I first dug out server-class ARM hardware and asked whether we should get one such server and run some tests with it.
While the answer wasn’t a clear “no” it also wasn’t exactly “yes”. I tried again a few times over the course of 2018 and each time I presented some more points why I thought it might be a good thing to test this. But still I wasn’t able to get a positive answer. Finally in January 2019 year I got a definitive answer – and it was “yes, go ahead”! The fact that Amazon had just presented their Graviton ARM Processor may have helped the decision.

###Looking at NetBSD from an OpenBSD user perspective

I use to use NetBSD quite a lot. From 2.0 to 6.99. But for some reasons, I stopped using it about 2012, in favor of OpenBSD. Reading on the new 8 release, I wanted to see if all the things I didn’t like on NetBSD were gone. Here is a personal Pros / Cons list. No Troll, hopefully. Just trying to be objective.

What I liked (pros)
Things I didn’t like (cons)
Conclusion

So that was it. I didn’t spend more than 30 minutes of it. But I didn’t want to spend more time on it. I did stop using NetBSD because of the need to compile each and every packages ; it was in the early days of pkgin. I also didn’t like the way system maintenance was to be done. OpenBSD’s 6-months release seemed far more easy to manage. I still think NetBSD is a great OS. But I believe you have to spent more time on it than you would have to do with OpenBSD.
That said, I’ll keep using my Puffy OS.

##News Roundup
###Using Vim to take time-stamped notes

I frequently find myself needing to take time-stamped notes. Specifically, I’ll be in a call, meeting, or interview and need to take notes that show how long it’s been since the meeting started.
My first thought was that there’s be a plugin to add time stamps, but a quick search didn’t turn anything up. However, I little digging did turn up the fact that vim has the built-in ability to tell time.
This means that writing a bit of vimscript to insert a time stamp is pretty easy. After a bit of fiddling, I came up with something that serves my needs, and I decided it might be useful enough to others to be worth sharing.

John Baldwin’s notes on bhyve meetings

###OpenBSD 6.5-beta has been tagged

It’s that time of year again; Theo (deraadt@) has just tagged 6.5-beta. A good reminder for us all run an extra test install and see if your favorite port still works as you expect.

CVSROOT: /cvs
Module name: src
Changes by: deraadt@cvs.openbsd.org 2019/02/26 15:24:41
Modified files:
etc/root : root.mail
share/mk : sys.mk
sys/conf : newvers.sh
sys/sys : ktrace.h param.h
usr.bin/signify: signify.1
sys/arch/macppc/stand/tbxidata: bsd.tbxi
Log message:
crank to 6.5-beta

###The NetBSD Foundation participating in Google Summer of Code 2019

For the 4th year in a row and for the 13th time The NetBSD Foundation will participate in Google Summer of Code 2019!
If you are a student and would like to learn more about Google Summer of Code please go to the Google Summer of Code homepage.
You can find a list of projects in Google Summer of Code project proposals in the wiki.
Do not hesitate to get in touch with us via #netbsd-code IRC channel on Freenode and via NetBSD mailing lists!

###SecBSD: an UNIX-like OS for Hackers

SecBSD is an UNIX-like operating system focused on computer security based on OpenBSD. Designed for security testing, hacking and vulnerability assessment, it uses full disk encryption and ProtonVPN + OpenVPN by default.
A security BSD enviroment for security researchers, penetration testers, bug hunters and cybersecurity experts. Developed by Dark Intelligence Team for private use and will be public release coming soon.

##Beastie Bits

##Feedback/Questions

Russell - BSD Now Question :: ZFS & FreeNAS
Alan - Tutorial, install ARM *BSD with no other BSD box pls
Johnny - New section to add to the show

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

289: Microkernel Failure Mar 14, 2019

Show notes

A kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more.

##Headlines

###A Kernel Of Failure -
How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it.

Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.

###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter

Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.

Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.

Kristof Provost, who maintains the port of pf in FreeBSD added a test for the vulnerability in FreeBSD head.

##News Roundup
###How I’m still not using GUIs in 2019: A guide to the terminal

TL;DR: Here are my dotfiles. Use them and have fun.

GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.

Don’t forget rule number one.

Whenever in doubt, read the manual.

###Using a Yubikey as smartcard for SSH public key authentication

SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
You might have heard of Yubikeys.
These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
In OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.

###The 18 Part FreeBSD Desktop Series by Vermaden

##Beastie Bits

##Feedback/Questions

Sijmen - Hi, and a Sunday afternoon toy project
Clint - Tuning ZFS for NVME
James - Show question

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

288: Turing Complete Sed Mar 07, 2019

Show notes

Software will never fix Spectre-type bugs, a proof that sed is Turing complete, managed jails using Bastille, new version of netdata, using grep with /dev/null, using GMail with mutt, and more.

##Headlines
###Google: Software is never going to be able to fix Spectre-type bugs

Spectre is here to stay: An analysis of side-channels and speculative execution

Researchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution.
The discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations.
Specifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened.

###A proof that Unix utility sed is Turing complete

Many people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete.
I first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete.
Christophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script.

Christophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these:

##News Roundup
###Bastille helps you quickly create and manage FreeBSD Jails.

Bastille helps you quickly create and manage FreeBSD Jails.
Jails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services.
Bastille provides an interface to create, manage and destroy these secure virtualized environments.

Current version: 0.3.20190204-beta.
Shell Script Source here: https://github.com/BastilleBSD/bastille/blob/master/usr/local/bin/bastille

###netdata v1.12 released

Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.

Patch release 1.12.1 contains 22 bug fixes and 8 improvements.

###Using grep with /dev/null, an old Unix trick

Every so often I will find myself writing a grep invocation like this:

find .... -exec grep <something> /dev/null '{}' '+'

The peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep if you’re not certain how many files your wildcard may match.

###USING GMAIL WITH MUTT

I recently switched to using mutt for email and while setting up mutt to use imap is pretty straightforward, this tutorial will also document some advanced concepts such as encrypting your account password and sending emails from a different From address.
This tutorial assumes that you have some familiarity with using mutt and have installed it with sidebar support (sudo apt-get install mutt-patched for the ubuntu folks) and are comfortable with editing your muttrc.
If you would just like to skip to the end, my mutt configuration file can be found here.

##Beastie Bits

##Feedback/Questions

Pablo - Topic suggestion: FreeBSD on a Laptop as daily driver
Ron - ZFS on the fly compression and seek
Dave - two zpool, or not two zpool, that is the question

Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Your browser does not support the HTML5 video tag.

Our TOPPODCAST Picks

Follow Us

Stay Connected

Headlines

News Roundup

[OpenBSD Upgrade Guide: 6.4 to 6.5

Beastie Bits

Feedback/Questions

Headlines

News Roundup

Beastie Bits

Feedback/Questions

Headlines

News Roundup

Beastie Bits

Feedback/Questions

Headlines

News Roundup

Beastie Bits

Feedback/Questions

Related Podcasts

1

2

3

4

5

6