Headlines

EKCD - Encrypted Crash Dumps in FreeBSD

Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.

The main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI.

Time on Unix

Time, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do.

Over millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time.

We’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant.

See the article for more

News Roundup

Improve ZVOL sync write performance by using a taskq

A central log host with syslog-ng on FreeBSD - Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

HEADS UP: NetBSD Entropy Overhaul

This week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html.

Setting Up NetBSD Kernel Dev Environment

I used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters.

Beastie Bits

• You can now use ccache to speed up dsynth even more.
• Improving libossaudio, and the future of OSS in NetBSD
• DragonFlyBSD DHCPCD Import dhcpcd-9.0.2 with the following changes
• Reminder: watch this space for upcoming FreeBSD Office Hours, next is May 13th at 2pm Eastern, 18:00 UTC

Feedback/Questions

• Ghislain - ZFS Question
• Jake - Paypal Donations
• Oswin - Hammer tutorial

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FuryBSD 2020Q2 Images Available for XFCE and KDE

The Q2 2020 images are not a visible leap forward but a functional leap forward. Most effort was spent creating a better out of box experience for automatic Ethernet configuration, working WiFi, webcam, and improved hypervisor support.

Technical reasons to choose FreeBSD over GNU/Linux

Since I wrote my article "Why you should migrate everything from Linux to BSD" I have been wanting to write something about the technical reasons to choose FreeBSD over GNU/Linux and while I cannot possibly cover every single reason, I can write about some of the things that I consider worth noting.

News Roundup

+ Not actually Linux distro review deux: GhostBSD

When I began work on the FreeBSD 12.1-RELEASE review last week, it didn't take long to figure out that the desktop portion wasn't going very smoothly.

I think it's important for BSD-curious users to know of easier, gentler alternatives, so I did a little looking around and settled on GhostBSD for a follow-up review.

GhostBSD is based on TrueOS, which itself derives from FreeBSD Stable. It was originally a Canadian distro, but—like most successful distributions—it has transcended its country of origin and can now be considered worldwide. Significant GhostBSD development takes place now in Canada, Italy, Germany, and the United States.

“TLS Mastery” sponsorships open

My next book will be TLS Mastery, all about Transport Layer Encryption, Let’s Encrypt, OCSP, and so on.

This should be a shorter book, more like my DNSSEC or Tarsnap titles, or the first edition of Sudo Mastery. I would like a break from writing doorstops like the SNMP and jails books.

JT (our producer) shared his Open Source Retail Box Collection on twitter this past weekend and there was a nice response from a few in the BSD Community showing their collections:

• JT's post: https://twitter.com/q5sys/status/1251194823589138432

  • High Resolution Image to see the bottom shelf better: https://photos.smugmug.com/photos/i-9QTs2RR/0/f1742096/O/i-9QTs2RR.jpg
  • Closeup of the BSD Section: https://twitter.com/q5sys/status/1251294290782928897

• Others jumped in with their collections:

  • Deb Goodkin's collection: https://twitter.com/dgoodkin/status/1251294016139743232 & https://twitter.com/dgoodkin/status/1251298125672660992
  • FreeBSD Frau's FreeBSD Collection: https://twitter.com/freebsdfrau/status/1251290430475350018
  • Jason Tubnor's OpenBSD Collection: https://twitter.com/Tubsta/status/1251265902214918144

Do you have a nice collection, take a picture and send it in!

Tale of OpenBSD secure memory allocator internals - malloc(3)

Hi there,

It's been a very long time I haven't written anything after my last OpenBSD blogs, that is,

OpenBSD Kernel Internals — Creation of process from user-space to kernel space.

OpenBSD: Introduction to execpromises in the pledge(2)

pledge(2): OpenBSD's defensive approach to OS Security

So, again I started reading OpenBSD source codes with debugger after reducing my sleep timings and managing to get some time after professional life. This time I have picked one of my favourite item from my wishlist to learn and share, that is, OpenBSD malloc(3), secure allocator

How I learned to stop worrying and love SSDs

my home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...?

So I run three VMs on the SSDs in total. One with Windows 10, two with Ubuntu running Confluence, A wiki essentially, with files for attachments and MySQL as the backend database. Clearly the writes had to stop when the wikis were not used at all, just sitting idle, right?

Well even with a full query log and quite some experience in the operation of web applications I could not figure out what Confluence is doing (productively, no doubt) but trust me, it writes a couple of hundred kbytes to the database each second just sitting idle.

My infrastructure as of 2019

I've wanted to write about my infrastructure for a while, but I kept thinking, "I'll wait until after I've done $next_thing_on_my_todo." Of course this cycle never ends, so I decided to write about its state at the end of 2019. Maybe I'll write an update on it in a couple of moons; who knows?

For something different than our usual Beastie Bits… we bring you…

We're all quarantined so lets install BSD on things! Install BSD on something this week, write it up and let us know about it, and maybe we'll feature you!

• Installation of NetBSD on a Mac Mini

• OpenBSD on the HP Envy 13

• Install NetBSD on a Vintage Computer

• BSDCan Home Lab Panel recording session: May 5th at 18:00 UTC

• Allan started a series of FreeBSD Office Hours

BSDNow is going Independent

• After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.

Feedback/Questions

• Todd - LinusTechTips Claims about ZFS

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Rethinking OpenBSD Security

OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.
I picked a few errata, not all of them, that were interesting and happened to suit my narrative.

FreeBSD 2020 Q1 Quarterly report

Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.

News Roundup

The Notion of Progress and User Interfaces

One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.

How should we think about progress? Both in general and regarding technology?

Thomas E. Dickey on NetBSD curses

I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

Making Unix a little more Plan9-like

I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.

A Warning

The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.

Not-actually Linux distro review: FreeBSD 12.1-RELEASE

This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.

The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.

Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.

FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.

Beastie Bits

• Wifi renewal restarted
• HAMMER2 and a quick start for DragonFly
• Engineering NetBSD 9.0
• Antivirus Protection using OPNsense Plugins
• BSDCan Home Lab Panel recording session: May 5th at 18:00 UTC

BSDNow is going Independent

• After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. LinuxAcademy is now under new leadership, and we understand that cutbacks needed to be made, and that BSD is not their core product. That does not mean your favourite BSD podcast is going away, we will continue and we expect things will not look much different. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.

Feedback/Questions

• Jordyn - ZFS Pool Problem

  • debug - https://github.com/BSDNow/bsdnow.tv/raw/master/episodes/347/feedback/dbg.txt

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Tales From a Core File - Lessons from the Unix stdio ABI: 40 Years Later

On the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages.

Update Lenovo X260 BIOS with OpenBSD

My X260 only runs OpenBSD and has no CD driver. But I still need to upgrade its BIOS from time to time. And this is possible using the ISO BIOS image.

First off all, you need to download the “BIOS Update (Bootable CD)” from the Lenovo Support Website.

News Roundup

The problem of Unix iowait and multi-CPU machines

Various Unixes have had a 'iowait' statistic for a long time now (although I can't find a source for where it originated; it's not in 4.x BSD, so it may have come through System V and sar). The traditional and standard definition of iowait is that it's the amount of time the system was idle but had at least one process waiting on disk IO. Rather than count this time as 'idle' (as you would if you had a three-way division of CPU time between user, system, and idle), some Unixes evolved to count this as a new category, 'iowait'.

My Latest Self Hosted Hugo Workflow using FreeBSD Jails, Caddy, Restic and More

After hosting with Netlify for a few years, I decided to head back to self hosting. Theres a few reasons for that but the main reasoning was that I had more control over how things worked.

In this post, i’ll show you my workflow for deploying my Hugo generated site (www.jaredwolff.com). Instead of using what most people would go for, i’ll be doing all of this using a FreeBSD Jails based server. Plus i’ll show you some tricks i’ve learned over the years on bulk image resizing and more.

Let’s get to it.

Extending support for the NetBSD-7 branch

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

Tale of two hypervisor bugs - Escaping from FreeBSD bhyve

VM escape has become a popular topic of discussion over the last few years. A good amount of research on this topic has been published for various hypervisors like VMware, QEMU, VirtualBox, Xen and Hyper-V. Bhyve is a hypervisor for FreeBSD supporting hardware-assisted virtualization. This paper details the exploitation of two bugs in bhyve - FreeBSD-SA-16:32.bhyve (VGA emulation heap overflow) and CVE-2018-17160 (Firmware Configuration device bss buffer overflow) and some generic techniques which could be used for exploiting other bhyve bugs. Further, the paper also discusses sandbox escapes using PCI device passthrough, and Control-Flow Integrity bypasses in HardenedBSD 12-CURRENT

Beastie Bits

• GhostBSD 20.02 Overview
• FuryBSD 12.1 Overview > Joe Maloney got in touch to say that the issues in the video and other ones found have since been fixed. Now that's community feedback in action, and an example of a developer who does his best to help the community. A great guy indeed.
• OS108-9.0 amd64 MATE released
• FreeBSD hacking: carp panics & test
• Inaugural FreeBSD Office Hours

Feedback/Questions

• Shody - systemd question
• Ben - GELI and GPT
• Stig - DIY NAS

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

NetBSD 8.2 is available!

The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

• Some highlights include:
  • x86: fixed regression in booting old CPUs
  • x86: Hyper-V Gen.2 VM framebuffer support
  • httpd(8): fixed various security issues
  • ixg(4): various fixes / improvements
  • x86 efiboot: add tftp support, fix issues on machines with many memory segments, improve graphics mode logic to work on more machines.
  • Various kernel memory info leaks fixes
  • Update expat to 2.2.8
  • Fix ryzen USB issues and support xHCI version 3.10.
  • Accept root device specification as NAME=label.
  • Add multiboot 2 support to x86 bootloaders.
  • Fix for CVE-2019-9506: 'Key Negotiation of Bluetooth' attack.
  • nouveau: limit the supported devices and fix firmware loading.
  • radeon: fix loading of the TAHITI VCE firmware.
  • named(8): stop using obsolete dnssec-lookaside.

NextCloud on OpenBSD

NextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial.

• Preface

Back when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!).

A rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out.

News Roundup

X11 screen locking: a secure and modular approach

For years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements

NetBSD and RISC OS running parallel

I have been experimenting with running two systems at the same time on the RK3399 SoC.
It all begun when I figured out how to switch to the A72 cpu for RISC OS. When the switch was done, the A53 cpu just continued to execute code.
OK I thought why not give it something to do!
My first step was to run some small programs.
It worked!

• Thanks to Tom Jones for the pointer to this article

Several weeks ago we covered a story about switching from Linux to BSD. Benedict and JT asked for community feedback as to their thoughts on the matter. Allan was out that week, so this will give him an opportunity to chime in with his thoughts as well.

• Jamie - Dumping Linux for BSD
• Matt - BSD Packaging
• Brad - Linux vs BS
• MJ - Linux vs BSD Feedback
• Ben - Feedback for JT
• Henrik - Why you should migrate everything to BSD

Beastie Bits

• ssh-copy-id now included
• OPNsense 20.1.3 released
• A Collection of prebuilt BSD Cloud Images
• Instant terminal sharing

Feedback/Questions

• Ales - Manually verify signature files for pkg package
• Shody - Yubikey
• Mike - Site for hashes from old disks
  • Answer: https://docs.google.com/spreadsheets/d/19FmLs0jXxLkxAr0zwgdrXQd1qhbwvNHH6NvolvXKWTM/edit?usp=sharing

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Text processing in the shell

This article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it!

One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.

When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.

Rebalancing data on ZFS mirrors

One of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?”

If you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool.

Don’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term.

News Roundup

Using OpenBSD relayd to Add Security Headers

I am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443.

How we set up our ZFS filesystem hierarchy in our ZFS pools

Our long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems.

Speeding up ZSH

https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh

I was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results.

In the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy.

How do Unix Pipes work

Pipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel.

What we do to enable us to grow our ZFS pools over time

In my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are.
Our big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space.

Linux maintains bugs: The real reason ifconfig on Linux is deprecated

In my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8).

In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system.

Clear Your Terminal in Style

if you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that.

This post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts.

Feedback/Questions

• Guy - AMD GPU Help
• MLShroyer13 - VLANs and Jails
• Master One - ZFS Suspend/resume

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Fighting the Coronavirus with FreeBSD

Here is a quick HOWTO for those who want to provide some FreeBSD based compute resources to help finding vaccines.

UPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”.

Per default it will now pick up some SARS-CoV‑2 (COVID-19) related folding tasks. There are some more config options (e.g. how much of the system resources are used). Please refer to the official Folding@Home site for more information about that. Be also aware that there is a big rise in compute resources donated to Folding@Home, so the pool of available work units may be empty from time to time, but they are working on adding more work units. Be patient.

How to configure the Wireguard VPN in OPNsense

WireGuard is a modern designed VPN that uses the latest cryptography for stronger security, is very lightweight, and is relatively easy to set up (mostly). I say ‘mostly’ because I found setting up WireGuard in OPNsense to be more difficult than I anticipated. The basic setup of the WireGuard VPN itself was as easy as the authors claim on their website, but I came across a few gotcha's. The gotcha's occur with functionality that is beyond the scope of the WireGuard protocol so I cannot fault them for that. My greatest struggle was configuring WireGuard to function similarly to my OpenVPN server. I want the ability to connect remotely to my home network from my iPhone or iPad, tunnel all traffic through the VPN, have access to certain devices and services on my network, and have the VPN devices use my home's Internet connection.

WireGuard behaves more like a SSH server than a typical VPN server. With WireGuard, devices which have shared their cryptographic keys with each other are able to connect via an encrypted tunnel (like a SSH server configured to use keys instead of passwords). The devices that are connecting to one another are referred to as “peer” devices. When the peer device is an OPNsense router with WireGuard installed, for instance, it can be configured to allow access to various resources on your network. It becomes a tunnel into your network similar to OpenVPN (with the appropriate firewall rules enabled). I will refer to the WireGuard installation on OPNsense as the server rather than a “peer” to make it more clear which device I am configuring unless I am describing the user interface because that is the terminology used interchangeably by WireGuard.

The documentation I found on WireGuard in OPNsense is straightforward and relatively easy to understand, but I had to wrestle with it for a little while to gain a better understanding on how it should be configured. I believe it was partially due to differing end goals – I was trying to achieve something a little different than the authors of other wiki/blog/forum posts. Piecing together various sources of information, I finally ended up with a configuration that met the goals stated above.

News Roundup

NomadBSD 1.3.1

NomadBSD 1.3.1 has recently been made available. NomadBSD is a lightweight and portable FreeBSD distribution, designed to run on live on a USB flash drive, allowing you to plug, test, and play on different hardware. They have also started a forum as of yesterday, where you can ask questions and mingle with the NomadBSD community. Notable changes in 1.3.1 are base system upgraded to FreeBSD 12.1-p2. automatic network interface setup improved, image size increased to over 4GB, Thunderbird, Zeroconf, and some more listed below.

GhostBSD 20.02

Eric Turgeon, main developer of GhostBSD, has announced version 20.02 of the FreeBSD based operating system. Notable changes are ZFS partition into the custom partition editor installer, allowing you to install alongside with Windows, Linux, or macOS. Other changes are force upgrade all packages on system upgrade, improved update station, and powerd by default for laptop battery performance.

New FuryBSD XFCE and KDE images

This new release is now based on FreeBSD 12.1 with the latest FreeBSD quarterly packages. This brings XFCE up to 4.14, and KDE up to 5.17. In addition to updates this new ISO mostly addresses community bugs, community enhancement requests, and community pull requests. Due to the overwhelming amount of reports with GitHub hosting all new releases are now being pushed to SourceForge only for the time being. Previous releases will still be kept for archive purposes.

pf-badhost 0.3 Released

pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet's biggest irritants. Annoyances such as SSH and SMTP bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.

Beastie Bits

• DragonFly i915 drm update
• CShell is punk rock
• The most surprising Unix programs

Feedback/Questions

• Master One - Torn between OpenBSD and FreeBSD
• Brad - Follow up to Linus ZFS story
• Filipe Carvalho - Call for Portuguese BSD User Groups

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

OpenBSD Full Disk Encryption with CoreBoot and Tianocore Payload

It has been a while since I have posted here so I wanted to share something that was surprisingly difficult for me to figure out. I have a Thinkpad T440p that I have flashed with Coreboot 4.11 with some special patches that allow the newer machine to work. When I got the laptop, the default BIOS was UEFI and I installed two operating systems.

Windows 10 with bitlocker full disk encryption on the “normal” drive (I replaced the spinning 2.5″ disk with an SSD)

Ubuntu 19.10 on the m.2 SATA drive that I installed using LUKS full disk encryption

I purchased one of those carriers for the optical bay that allows you to install a third SSD and so I did that with the intent of putting OpenBSD on it. Since my other two operating systems were running full disk encryption, I wanted to do the same on OpenBSD.

• See article for rest of story

FreeBSD 12.0 EOL

Dear FreeBSD community,

As of February 29, 2020, FreeBSD 12.0 will reach end-of-life and will no longer be supported by the FreeBSD Security Team. Users of FreeBSD 12.0 are strongly encouraged to upgrade to a newer release as soon as possible.

• 12.1 Active release
• 12.2 Release Schedule

News Roundup

Some effects of the ZFS DVA format on data layout and growing ZFS pools

One piece of ZFS terminology is DVA and DVAs, which is short for Data Virtual Address. For ZFS, a DVA is the equivalent of a block number in other filesystems; it tells ZFS where to find whatever data we're talking about. The short summary of what fields DVAs have and what they mean is that DVAs tell us how to find blocks by giving us their vdev (by number) and their byte offset into that particular vdev (and then their size). A typical DVA might say that you find what it's talking about on vdev 0 at byte offset 0x53a40ed000. There are some consequences of this that I hadn't really thought about until the other day.

Right away we can see why ZFS has a problem removing a vdev; the vdev's number is burned into every DVA that refers to data on it. If there's no vdev 0 in the pool, ZFS has no idea where to even start looking for data because all addressing is relative to the vdev. ZFS pool shrinking gets around this by adding a translation layer that says where to find the portions of vdev 0 that you care about after it's been removed.

Warning! Active Directory Security Changes Require TrueNAS and FreeNAS Updates.

• Critical Information for Current FreeNAS and TrueNAS Users

Microsoft is changing the security defaults for Active Directory to eliminate some security vulnerabilities in its protocols. Unfortunately, these new security defaults may disrupt existing FreeNAS/TrueNAS deployments once Windows systems are updated. The Windows updates may appear sometime in March 2020; no official date has been announced as of yet.

FreeNAS and TrueNAS users that utilize Active Directory should update to version 11.3 (or 11.2-U8) to avoid potential disruption of their networks when updating to the latest versions of Windows software after March 1, 2020. Version 11.3 has been released and version 11.2-U8 will be available in early March.

Full name of the FreeBSD Root Account

NetBSD now has a users(7) and groups(7) manual. Looking into what entries existed in the passwd and group files I wondered about root’s full name who we now know as Charlie Root in the BSDs....

OpenBSD Go Situation

Over in the fediverse, Pete Zaitcev had a reaction to my entry on OpenBSD versus Prometheus for us:

I don't think the situation is usually that bad. Our situation with Prometheus is basically a worst case scenario for Go on OpenBSD, and most people will have much better results, especially if you stick to supported OpenBSD versions.

If you stick to supported OpenBSD versions, upgrading your machines as older OpenBSD releases fall out of support (as the OpenBSD people want you to do), you should not have any problems with your own Go programs. The latest Go release will support the currently supported OpenBSD versions (as long as OpenBSD remains a supported platform for Go), and the Go 1.0 compatibility guarantee means that you can always rebuild your current Go programs with newer versions of Go. You might have problems with compiled binaries that you don't want to rebuild, but my understanding is that this is the case for OpenBSD in general; it doesn't guarantee a stable ABI even for C programs (cf). If you use OpenBSD, you have to be prepared to rebuild your code after OpenBSD upgrades regardless of what language it's written in.

Beastie Bits

• Test your TOR
• OPNsense 20.1.1 released
• pkg for FreeBSD 1.13

Feedback/Questions

• Bostjan writes in about Wireguard
• Charlie has a followup to wpa_supplicant as lower class citizen
• Lars writes about LibreSSL as a positive example

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FreeBSD on Power

The power and promise of all open source software is freedom. Another way to express freedom is choice — choice of platforms, deployment models, stacks, configurations, etc.

The FreeBSD Foundation is dedicated to supporting and promoting the FreeBSD Project and community worldwide. But, what does this mean, exactly, you may wonder. The truth is it means many different things, but in all cases the Foundation acts to expand freedom and choice so that FreeBSD users have the power to serve their varied compute needs.

This blog tells the story of one specific way the Foundation helps a member of the community provide greater hardware choice for all FreeBSD users.

Dragonfly 5.8

DragonFly version 5.8 brings a new dsynth utility for building your own binary dports packages, plus significant support work to speed up that build - up to and including the entire collection. Additional progress has been made on GPU and signal support.

The details of all commits between the 5.6 and 5.8 branches are available in the associated commit messages for 5.8.0rc1 and 5.8.0. Also see /usr/src/UPDATING for specific file changes in PAM.

• See article for rest of information

2nd HamBUG meeting recap

• The second meeting of the Hamilton BSD Users Group took place last night
• The next meeting is scheduled for the 2nd Tuesday of the month, April 14th 2020

News Roundup

FreeNAS/TrueNAS Brand Unification

FreeNAS and TrueNAS have been separate-but-related members of the #1 Open Source storage software family since 2012. FreeNAS is the free Open Source version with an expert community and has led the pursuit of innovations like Plugins and VMs. TrueNAS is the enterprise version for organizations of all sizes that need additional uptime and performance, as well as the enterprise-grade support necessary for critical data and applications.

From the beginning at iXsystems, we’ve developed, tested, documented, and released both as separate products, even though the vast majority of code is shared. This was a deliberate technical decision in the beginning but over time became less of a necessity and more of “just how we’ve always done it”. Furthermore, to change it was going to require a serious overhaul to how we build and package both products, among other things, so we continued to kick the can down the road. As we made systematic improvements to development and QA efficiency over the past few years, the redundant release process became almost impossible to ignore as our next major efficiency roadblock to overcome. So, we’ve finally rolled up our sleeves.

With the recent 11.3 release, TrueNAS gained parity with FreeNAS on features like VMs and Plugins, further homogenizing the code. Today, we announce the next phase of evolution for FreeNAS and TrueNAS.

OpenBSD versus Prometheus (and Go).

We have a decent number of OpenBSD machines that do important things (and that have sometimes experienced problems like running out of disk space), and we have a Prometheus based metrics and monitoring system. The Prometheus host agent has enough support for OpenBSD to be able to report on critical metrics, including things like local disk space. Despite all of this, after some investigation I've determined that it's not really sensible to even try to deploy the host agent on our OpenBSD machines. This is due to a combination of factors that have at their root OpenBSD's lack of ABI stability

FreeBSD removed gcc from base

As described in Warner's email message[1] to the FreeBSD-arch mailing list we have reached GCC 4.2.1's retirement date. At this time all supported architectures either use in-tree Clang, or rely on external toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later that year, in r171825. GCC has served us well, but version 4.2.1 is obsolete and not used by default on any architecture in FreeBSD. It does not support modern C and does not support arm64 or RISC-V.

Beastie Bits

• New Archive location for Dragonfly 4.x
• A dead simple git cheat sheet
• Xorg 1.20.7 on HardenedBSD Comes with IE/RELRO+BIND_NOW/CFI/SafeStack Protections

Feedback/Questions

• Niclas writes in Regarding the Lenovo E595 user (episode 340)
• Lyubomir writes about GELI and ZFS
• Peter writes in about scaling FreeBSD jails

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Checksumming in filesystems, and why ZFS is doing it right

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:

• Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive.
• Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written.
• Misdirected read - when we miss reading the block when a bit flip occurred.
• Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache.

Checksumming may help us detect errors in a few of those situations.

DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance

It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.

Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."

• https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860

There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.

• https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351

This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.

News Roundup

Why ZFS is not good at growing and reshaping pools (or shrinking them)

recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.

(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)

Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro

I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.

One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)

I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/

A Central Log Host with syslog-ng on FreeBSD

• Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

• Part 2

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.

Beastie Bits

• FreeBSD at Linux Conf 2020 session videos now online
• Unlock your laptop with your phone
• Managing a database of vulnerabilities for a package system: the pkgsrc study
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• CharmBUG Meeting: March 24th 7pm in Severn, MD ***

Feedback/Questions

• Andrew - ZFS feature Flags
• Sam - TwinCat BSD
• Dacian - Freebsd + amdgpu + Lenovo E595

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Meet FuryBSD: A New Desktop BSD Distribution

At its heart, FuryBSD is a very simple beast. According to the site, “FuryBSD is a back to basics lightweight desktop distribution based on stock FreeBSD.” It is basically FreeBSD with a desktop environment pre-configured and several apps preinstalled. The goal is to quickly get a FreeBSD-based system running on your computer.

You might be thinking that this sounds a lot like a couple of other BSDs that are available, such as NomadBSD and GhostBSD. The major difference between those BSDs and FuryBSD is that FuryBSD is much closer to stock FreeBSD. For example, FuryBSD uses the FreeBSD installer, while others have created their own installers and utilities.

As it states on the site, “Although FuryBSD may resemble past graphical BSD projects like PC-BSD and TrueOS, FuryBSD is created by a different team and takes a different approach focusing on tight integration with FreeBSD. This keeps overhead low and maintains compatibility with upstream.” The lead dev also told me that “One key focus for FuryBSD is for it to be a small live media with a few assistive tools to test drivers for hardware.”

Currently, you can go to the FuryBSD homepage and download either an XFCE or KDE LiveCD. A GNOME version is in the works.

NetBSD 9.0

The NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system.

This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes. Here are some highlights of this new release.

News Roundup

OpenBSD Foundation 2019 campaign wrapup

Our target for 2019 was CDN$300K. Our community's continued generosity combined with our corporate donors exceeded that nicely. In addition we received the largest single donation in our history, CDN$380K from Smartisan. The return of Google was another welcome event. Altogether 2019 was our most successful campaign to date, yielding CDN$692K in total.

We thank all our donors, Iridium (Smartisan), Platinum (Yandex, Google), Gold (Microsoft, Facebook) Silver (2Keys) and Bronze (genua, Thinkst Canary). But especially our community of smaller donors whose contributions are the bedrock of our support. Thank you all!

• OpenBSD Foundation 2019 Fundraising Goal Exceeded

A retrospective on our OmniOS ZFS-based NFS fileservers

Our OmniOS fileservers have now been out of service for about six months, which makes it somewhat past time for a retrospective on them. Our OmniOS fileservers followed on our Solaris fileservers, which I wrote a two part retrospective on (part 1, part 2), and have now been replaced by our Linux fileservers. To be honest, I have been sitting on my hands about writing this retrospective because we have mixed feelings about our OmniOS fileservers.

I will put the summary up front. OmniOS worked reasonably well for us over its lifespan here and looking back I think it was almost certainly the right choice for us at the time we made that choice (which was 2013 and 2014). However it was not without issues that marred our experience with it in practice, although not enough to make me regret that we ran it (and ran it for as long as we did). Part of our issues are likely due to a design mistake in making our fileservers too big, although this design mistake was probably magnified when we were unable to use Intel 10G-T networking in OmniOS.

On the one hand, our OmniOS fileservers worked, almost always reliably. Like our Solaris fileservers before them, they ran quietly for years without needing much attention, delivering NFS fileservice to our Ubuntu servers; specifically, we ran them for about five years (2014 through 2019, although we started migrating away at the end of 2018). Over this time we had only minor hardware issues and not all that many disk failures, and we suffered no data loss (with ZFS checksums likely saving us several times, and certainly providing good reassurances). Our overall environment was easy to manage and was pretty much problem free in the face of things like failed disks. I'm pretty sure that our users saw a NFS environment that was solid, reliable, and performed well pretty much all of the time, which is the important thing. So OmniOS basically delivered the fileserver environment we wanted.

NetBSD Fundraising 2020 goal

Is it really more than 10 years since we last had an official fundraising drive?

Looking at old TNF financial reports I noticed that we have been doing quite well financially over the last years, with a steady stream of small and medium donations, and most of the time only moderate expenditures. The last fundraising drive back in 2009 was a giant success, and we have lived off it until now.

OpenSSH 8.2 released February 14, 2020

OpenSSH 8.2 was released on 2020-02-14. It is available from the mirrors listed at https://www.openssh.com/.

OpenSSH is a 100% complete SSH protocol 2.0 implementation and includes sftp client and server support.

Once again, we would like to thank the OpenSSH community for their continued support of the project, especially those who contributed code or patches, reported bugs, tested snapshots or donated to the project. More information on donations may be found at:

• https://www.openssh.com/donations.html

Beastie Bits

• FreeNAS vs. Unraid: GRUDGE MATCH!
• Unix Toolbox
• Rigs of Rods - OpenBSD Physics Game
• NYCBug - Dr Vixie
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• BSD Stockholm - Meetup March 3rd 2020

Feedback/Questions

• Shirkdog - Question
• Master One - ZFS + Suspend/resume
• Micah Roth - ZFS write caching

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Distrowatch Fury BSD Review

FuryBSD is the most recent addition to the DistroWatch database and provides a live desktop operating system based on FreeBSD. FuryBSD is not entirely different in its goals from NomadBSD, which we discussed recently. I wanted to take this FreeBSD-based project for a test drive and see how it compares to NomadBSD and other desktop-oriented projects in the FreeBSD family.

FuryBSD supplies hybrid ISO/USB images which can be used to run a live desktop. There are two desktop editions currently, both for 64-bit (x86_64) machines: Xfce and KDE Plasma. The Xfce edition is 1.4GB in size and is the flavour I downloaded. The KDE Plasma edition is about 3.0GB in size.

My fresh install of FuryBSD booted to a graphical login screen. From there I could sign into my account, which brings up the Xfce desktop. The installed version of Xfce is the same as the live version, with a few minor changes. Most of the desktop icons have been removed with just the file manager launchers remaining. The Getting Started and System Information icons have been removed. Otherwise the experience is virtually identical to the live media.

FuryBSD uses a theme that is mostly grey and white with creamy yellow folder icons. The application menu launchers tend to have neutral icons, neither particularly bright and detailed or minimal.

LLDB now works on i386

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support.

The original NetBSD port of LLDB was focused on amd64 only. In January, I have extended it to support i386 executables. This includes both 32-bit builds of LLDB (running natively on i386 kernel or via compat32) and debugging 32-bit programs from 64-bit LLDB.

News Roundup

wpa_supplicant is definitely a lower-class citizen, sorry

wpa_supplicant is definitely a lower-class citizen, sorry.

I increasingly wonder why this stuff matters; transit costs are so much lower than the period when eduroam was setup, and their reliance on 802.11x is super weird in a world where, for the most part
+ entire cities have open wifi in their downtown core
+ edu vs edu+transit split horizon problems have to be solved anyways
+ many universities have parallel open wifi
+ rate limiting / fare-share approaches for the open-net, on unmetered
+ flat-rate solves the problem
+ LTE hotspot off a phone isn't a rip off anymore
+ other open networks exist

essentially no one else feels compelled to do use 802.11x for a so called "semi-open access network", so I think they've lost the plot on friction vs benefit.

(we've held hackathons at EDU campus that are locked down like that, and in every case we've said no way, gotten a wire with open net, and built our own wifi. we will not subject our developers to that extra complexity).

KDE FreeBSD Updates Feb 2020

Some bits and bobs from the KDE FreeBSD team in february 2020. We met at the FreeBSD devsummit before FOSDEM, along with other FreeBSD people. Plans were made, schemes were forged, and Groff the Goat was introduced to some new people.

• The big ticket things:
  • Frameworks are at 5.66
  • Plasma is at 5.17.5 (the beta 5.18 hasn’t been tried)
  • KDE release service has landed 19.12.2 (same day it was released)
• Developer-centric:
  • KDevelop is at 5.5.0
  • KUserfeedback landed its 1.0.0 release
  • CMake is 3.16.3
• Applications:
  • Musescore is at 3.4.2
  • Elisa now part of the KDE release service updates
• Fuure work:
  • KIO-Fuse probably needs extra real-world testing on FreeBSD. I don’t have that kind of mounts (just NFS in /etc/fstab) so I’m not the target audience.
  • KTextEditor is missing .editorconfig support. That can come in with the next frameworks update, when consumers update anyway. Chasing it in an intermediate release is a bit problematic because it does require some rebuilds of consumers.

Travel Grant Application for BSDCan is now open

Hi everyone,

The Travel Grant Application for BSDCan 2020 is now open. The Foundation can help you attend BSDCan through our travel grant program. Travel grants are available to FreeBSD developers and advocates who need assistance with travel expenses for attending conferences related to FreeBSD development. BSDCan 2020 applications are due April 9, 2020. Find out more and apply at: https://www.freebsdfoundation.org/what-we-do/grants/travel-grants/

Did you know the Foundation also provides grants for technical events not specifically focused on BSD? If you feel that your attendance at one of these events will benefit the FreeBSD Project and Community and you need assistance getting there, please fill out the general travel grant application. Your application must be received 7 weeks prior to the event. The general application can be found here: https://goo.gl/forms/QzsOMR8Jra0vqFYH2

Creating a ZFS dataset for testing iocage within a jail

• Be warned, this failed. I’m stalled and I have not completed this.

I’m going to do jails within a jail. I already do that with poudriere in a jail but here I want to test an older version of iocage before upgrading my current jail hosts to a newer version.

• In this post:
  • FreeBSD 12.1
  • py36-iocage-1.2_3
  • py36-iocage-1.2_4

This post includes my errors and mistakes. Perhaps you should proceed carefully and read it all first.

Beastie Bits

• Reminder: the FreeBSD Journal is free! Check out these great articles
• Serenity GUI desktop running on an OpenBSD kernel
• The Open Source Parts of MacOS
• FOSDEM videos available

Feedback/Questions

• Michael - Install with ZFS
• Mohammad - Server Freeze
• Todd - ZFS Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

The happinesses and stresses of full-time FOSS work

In the past few days, several free software maintainers have come out to discuss the stresses of their work. Though the timing was suggestive, my article last week on the philosophy of project governance was, at best, only tangentially related to this topic - I had been working on that article for a while. I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it.

February will mark one year that I’ve been working on self-directed free software projects full-time. I was planning on writing an optimistic retrospective article around this time, but given the current mood of the ecosystem I think it would be better to be realistic. In this stage of my career, I now feel at once happier, busier, more fulfilled, more engaged, more stressed, and more depressed than I have at any other point in my life.

The good parts are numerous. I’m able to work on my life’s passions, and my projects are in the best shape they’ve ever been thanks to the attention I’m able to pour into them. I’ve also been able to do more thoughtful, careful work; with the extra time I’ve been able to make my software more robust and reliable than it’s ever been. The variety of projects I can invest my time into has also increased substantially, with what was once relegated to minor curiosities now receiving a similar amount of attention as my larger projects were receiving in my spare time before. I can work from anywhere in the world, at any time, not worrying about when to take time off and when to put my head down and crank out a lot of code.

The frustrations are numerous, as well. I often feel like I’ve bit off more than I can chew. This has been the default state of affairs for me for a long time; I’m often neglecting half of my projects in order to obtain progress by leaps and bounds in just a few. Working on FOSS full-time has cast this model’s disadvantages into greater relief, as I focus on a greater breadth of projects and spend more time on them.

Building a FreeBSD File Server

Recently at my job, I was faced with a task to develop a file server explicitly suited for the requirements of the company. Needless to say, any configuration of a kind depends on what the infrastructure needs. So, drawing from my personal experience and numerous materials on the web, I came up with the combination FreeBSD+SAMBA+AD as the most appropriate. It appears to be a perfect choice for this environment, and harmonic addition to the existing network configuration since FreeBSD + SAMBA + AD enables admins with the broad range of possibilities for access control. However, as nothing is perfect, this configuration isn’t the best choice if your priority is data protection because it won’t be able to reach the necessary levels of reliability and fault tolerance without outside improvements.

Now, since we’ve established that, let’s move on to the next point. This article’s describing the process of building a test environment while concentrating primarily on the details of the configuration. As the author, though, I must say I’m in no way suggesting that this is the only way! The following configuration will be presented in its initial stage, with the minimum requirements necessary to get the job done, and its purpose in one specific situation only. Here, look at this as a useful strategy to solve similar tasks. Well, let’s get started!

Report from the first Hamilton BSD Users Group Meeting

February 11th was the first meeting of this new user group, founded by John Young and myself

11 people attended, and a lot of good discussions were had

One of the attendees already owns a domain that fits well for the group, so we will be getting that setup over the next few weeks, as well as the twitter account, and other organization stuff.

Special thanks to the illumos users who drove in from Buffalo to attend, although they may have actually had a shorter drive than a few of the other attendees.

The next meeting is scheduled again for the 2nd Tuesday of the month, March 10th.

We are still discussing if we should meet at a restaurant again, or try to get a space at the local college or innovation hub where we can have a projector etc.

News Roundup

Kubernetes on FreeBSD Bhyve

There are quite a few solutions for container orchestration, but the most popular (or the most famous and highly advertised, is probably, a Kubernetes) Since I plan to conduct many experiments with installing and configuring k8s, I need a laboratory in which I can quickly and easily deploy a cluster in any quantities for myself. In my work and everyday life I use two OS very tightly - Linux and FreeBSD OS. Kubernetes and docker are Linux-centric projects, and at first glance, you should not expect any useful participation and help from FreeBSD here. As the saying goes, an elephant can be made out of a fly, but it will no longer fly. However, two tempting things come to mind - this is very good integration and work in the FreeBSD ZFS file system, from which it would be nice to use the snapshot mechanism, COW and reliability. And the second is the bhyve hypervisor, because we still need the docker and k8s loader in the form of the Linux kernel. Thus, we need to connect a certain number of actions in various ways, most of which are related to starting and pre-configuring virtual machines. This is typical of both a Linux-based server and FreeBSD. What exactly will work under the hood to run virtual machines does not play a big role. And if so - let's take a FreeBSD here!

NetBSD 9 RC1 Available

We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).

• Here are a few highlights of the new release:

  • Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA)
  • Enhanced hardware support for Armv7-A
  • Updated GPU drivers (e.g. support for Intel Kabylake)
  • Enhanced virtualization support
  • Support for hardware-accelerated virtualization (NVMM)
  • Support for Performance Monitoring Counters
  • Support for Kernel ASLR
  • Support several kernel sanitizers (KLEAK, KASAN, KUBSAN)
  • Support for userland sanitizers
  • Audit of the network stack
  • Many improvements in NPF
  • Updated ZFS
  • Reworked error handling and NCQ support in the SATA subsystem
  • Support a common framework for USB Ethernet drivers (usbnet)

• You can download binaries of NetBSD 9.0_RC1 from our Fastly-provided CDN: https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0_RC1/

OPNsense 20.1 Keen Kingfisher released

For over 5 years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.

20.1, nicknamed "Keen Kingfisher", is a subtle improvement on sustainable firewall experience. This release adds VXLAN and additional loopback device support, IPsec public key authentication and elliptic curve TLS certificate creation amongst others. Third party software has been updated to their latest versions. The logging frontend was rewritten for MVC with seamless API support. On the far side the documentation increased in quality as well as quantity and now presents itself in a familiar menu layout.

Idealistic Future for HardenedBSD

Over the past month, we purchased and deployed the new 13-CURRENT/amd64 package building server. We published our first 13-CURRENT/amd64 production package build using that server. We then rebuilt the old package building server to act as the 12-STABLE/amd64 package building server. This post signifies a very important milestone: we have now fully recovered from last year's death of our infrastructure. Our 12-STABLE/amd64 repo, previously out-of-date by many months, is now fully up-to-date!

HardenedBSD is in a very unique position to provide innovative solutions to at-risk and underprivileged populations. As such, we are making human rights endeavors a defining area of focus. Our infrastructure will integrate various privacy and anonymity enhancing technologies and techniques to protect lives. Our operating system's security posture will increase, especially with our focus on exploit mitigations.

Navigating the intersection between human rights and information security directly impacts lives. HardenedBSD's 2020 mission and focus is to deliver an entire hardened ecosystem that is unfriendly towards those who would oppress or censor their people. This includes a subtle shift in priorities to match this new mission and focus. While we implement exploit mitigations and further harden the ecosystem, we will seek out opportunities to contribute a tangible and unique impact on human rights issues. Providing Tor Onion Services for our core infrastructure is the first step in likely many to come towards securely helping those in need.

Beastie Bits

• Warner Losh's FOSDEM talk
• Relational Pipes v0.15
• A reminder for where to find NetBSD ARM images
• New Safe Memory Reclamation feature in UMA
• BSD Users Stockholm Meetup

Feedback/Questions

• ZFS - Rosetta Stone Document?
• Pat - Question
• Sigflup - Wayland on the BSDs

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

OpenBSD has to be a BSD Unix and you couldn't duplicate it with Linux

OpenBSD has a well deserved reputation for putting security and a clean system (for code, documentation, and so on) first, and everything else second. OpenBSD is of course based on BSD (it's right there in the name) and descends from FreeBSD NetBSD (you can read the history here). But one of the questions you could ask about it is whether it had to be that way, and in particular if you could build something like OpenBSD on top of Linux. I believe that the answer is no.

Linux and the *BSDs have a significantly different model of what they are. BSDs have a 'base system' that provides an integrated and fully operational core Unix, covering the kernel, C library and compiler, and the normal Unix user level programs, all maintained and distributed by the particular BSD. Linux is not a single unit this way, and instead all of the component parts are maintained separately and assembled in various ways by various Linux distributions. Both approaches have their advantages, but one big one for the BSD approach is that it enables global changes.

Making global changes is an important part of what makes OpenBSD's approach to improving security, code maintenance, and so on work. Because it directly maintains everything as a unit, OpenBSD is in a position to introduce new C library or kernel APIs (or change them) and then immediately update all sorts of things in user level programs to use the new API. This takes a certain amount of work, of course, but it's possible to do it at all. And because OpenBSD can do this sort of ambitious global change, it does.

This goes further than just the ability to make global changes, because in theory you can patch in global changes on top of a bunch of separate upstream projects. Because OpenBSD is in control of its entire base system, it's not forced to try to reconcile different development priorities or integrate clashing changes. OpenBSD can decide (and has) that only certain sorts of changes will be accepted into its system at all, no matter what people want. If there are features or entire programs that don't fit into what OpenBSD will accept, they just lose out.

FreeBSD Quarterly Status Report 2019Q4

Here is the last quarterly status report for 2019. As you might remember from last report, we changed our timeline: now we collect reports the last month of each quarter and we edit and publish the full document the next month. Thus, we cover here the period October 2019 - December 2019.

If you thought that the FreeBSD community was less active in the Christmas' quarter you will be glad to be proven wrong: a quick glance at the summary will be sufficient to see that much work has been done in the last months.

Have a nice read!

News Roundup

OPNsense 19.7.9 released

As 20.1 nears we will be making adjustments to the scope of the release with an announcement following shortly.

For now, this update brings you a GeoIP database configuration page for aliases which is now required due to upstream database policy changes and a number of prominent third-party software updates we are happy to see included.

Archives are important to retain and pass on knowledge

Archives are important. When they are public and available for searching, it retains and passes on knowledge. It saves vast amounts of time.

HardenedBSD Tor Onion Service v3 Nodes

I've been working today on deploying Tor Onion Service v3 nodes across our build infrastructure. I'm happy to announce that the public portion of this is now completed. Below you will find various onion service hostnames and their match to our infrastructure.

• hardenedbsd.org: lkiw4tmbudbr43hbyhm636sarn73vuow77czzohdbqdpjuq3vdzvenyd.onion
• ci-01.nyi.hardenedbsd.org: qspcqclhifj3tcpojsbwoxgwanlo2wakti2ia4wozxjcldkxmw2yj3yd.onion
• ci-03.md.hardenedbsd.org: eqvnohly4tjrkpwatdhgptftabpesofirnhz5kq7jzn4zd6ernpvnpqd.onion
• ci-04.md.hardenedbsd.org: rfqabq2w65nhdkukeqwf27r7h5xfh53h3uns6n74feeyl7s5fbjxczqd.onion
• git-01.md.hardenedbsd.org: dacxzjk3kq5mmepbdd3ai2ifynlzxsnpl2cnkfhridqfywihrfftapid.onion

Beastie Bits

• The Missing Semester of Your CS Education (MIT Course)
• An old Unix Ad
• OpenBSD syscall call-from verification
• OpenBSD/arm64 on Pinebook
• Reminder: First Southern Ontario BSD user group meeting, February 11th (this coming Tuesday!) 18:30 at Boston Pizza on Upper James st, Hamilton.
• NYCBUG: March meeting will feature Dr. Paul Vixie and his new talk “Operating Systems as Dumb Pipes”
• 8th Meetup of the Stockholm BUG: March 3 at 18:00
• Polish BSD User Group meets on Feb 11, 2020 at 18:15

Feedback/Questions

• Sean - ZFS and Creation Dates
• Christopher - Help on ZFS Disaster Recovery
• Mike - Encrypted ZFS Send

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FreeBSD is an amazing operating System

Update 2020-01-21: Since I wrote this article it got posted on Hacker News, Reddit and Lobster, and a few people have emailed me with comments. I have updated the article with comments where I have found it needed. As an important side note I would like to point out that I am not a FreeBSD developer, there may be things going on in the FreeBSD world that I know absolutely nothing about. I am also not glued to the FreeBSD developer mailing lists. I am not a FreeBSD "fanboy". I have been using GNU/Linux a ton more for the past two decades than FreeBSD, mainly due to hardware incompatibility (lacking or buggy drivers), and I love both Debian GNU/Linux and Arch Linux just as much as FreeBSD. However, I am concerned about the development of GNU/Linux as of late. Also this article is not about me trying to make anyone switch from something else to FreeBSD. It's about why I like FreeBSD and that I recommend you try it out if you're into messing with operating systems.

I think the year was late 1999 or mid 2000 when I one day was browsing computer books at my favorite bookshop and I discovered the book The Complete FreeBSD third edition from 1999 by Greg Lehey. With the book came 4 CD Roms with FreeBSD 3.3.

I had already familiarized myself with GNU/Linux in 1998, and I was in the process of migrating every server and desktop operating system away from Microsoft Windows, both at home and at my company, to GNU/Linux, initially Red Hat Linux and then later Debian GNU/Linux, which eventually became my favorite GNU/Linux distribution for many years.

When I first saw The Complete FreeBSD book by Greg Lehey I remember noticing the text on the front page that said, "The Free Version of Berkeley UNIX" and "Rock Solid Stability", and I was immediately intrigued! What was that all about? A free UNIX operating system! And rock solid stability? That sounded amazing.

Hyperbola Dev Interview

In late December 2019, Hyperbola announced that they would be making major changes to their project. They have decided to drop the Linux kernel in favor of forking the OpenBSD kernel. This announcement only came months after Project Trident announced that they were going in the opposite direction (from BSD to Linux).

Hyperbola also plans to replace all software that is not GPL v3 compliant with new versions that are.

To get more insight into the future of their new project, I interviewed Andre, co-founder of Hyperbola.

News Roundup

Improving the ptrace(2) API and preparing for LLVM-10.0

This month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt.

As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).

The first FreeBSD conference in Australia

FreeBSD has existed as an operating system, project, and foundation for more than twenty years, and its earlier incantations have exited for far longer. The old guard have been developing code, porting software, and writing documentation for longer than I’ve existed. I’ve been using it for more than a decade for personal projects, and professionally for half that time.

While there are many prominent Australian FreeBSD contributors, sysadmins, and users, we’ve always had to venture overseas for conferences. We’re always told Australians are among the most ardent travellers, but I always wondered if we could do a domestic event as well.

And on Tuesday, we did! Deb Goodkin and the FreeBSD Foundation graciously organised and chaired a dedicated FreeBSD miniconf at the long-running linux.conf.au event held each year in a different city in Australia and New Zealand.

A practical guide to containers on FreeNAS for a depraved psychopath

This is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11.

But muh jails?

You know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years…

So jails are dead then?

No, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD.

Why you should migrate everything from Linux to BSD

As an operating system GNU/Linux has become a real mess because of the fragmented nature of the project, the bloatware in the kernel, and because of the jerking around by commercial interests.

• Response Should you migrate from Linux to BSD? It depends.

Beastie Bits

• Using the OpenBSD ports tree with dedicated users
• broot on FreeBSD
• A Trip down Memory Lane
• Running syslog-ng in BastilleBSD
• NASA : Using Software Packages in pkgsrc

Feedback/Questions

• All of our questions this week were pretty technical in nature so I'm going to save those for the next episode so Allan can weigh in on them, since if we cover them now we're basically going to be deferring to Allan anyway.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Upgrading FreeBSD from 11.3 to 12.1

Now here’s something more like what I was originally expecting the content on this blog to look like. I’m in the process of moving all of our FreeBSD servers (about 30 in total) from 11.3 to 12.1. We have our own local build of the OS, and until “packaged base” gets to a state where it’s reliably usable, we’re stuck doing upgrades the old-fashioned way. I created a set of notes for myself while cranking through these upgrades and I wanted to share them since they are not really work-specific and this process isn’t very well documented for people who haven’t been doing this sort of upgrade process for 25 years.

Our source and object trees are read-only exported from the build server over NFS, which causes things to be slow. /etc/make.conf and /etc/src.conf are symbolic links on all of our servers to the master copies in /usr/src so that make installworld can find the configuration parameters the system was built with.

Switching Distrowatch over to BSD

This may be a little off-topic for this board (forgive me if it is, please). However, I wanted to say that I'm one of the people who works on DistroWatch (distrowatch.com) and this past week we had to deal with a server facing hardware failure. We had a discussion about whether to continue running Debian or switch to something else.

The primary "something else" option turned out to be FreeBSD and it is what we eventually went with. It took a while to convert everything over from working with Debian GNU/Linux to FreeBSD 12 (some script incompatibilities, different paths, some changes to web server configuration, networking IPv6 troubles). But in the end we ended up with a good, FreeBSD-based experience.

Since the transition was successful, though certainly not seamless, I thought people might want to do a Q&A on the migration process. Especially for those thinking of making the same switch.

News Roundup

iked(8) automatic IPv6 blocking removed

iked(8) no longer automatically blocks unencrypted outbound IPv6 packets. This feature was intended to avoid accidental leakage, but in practice was found to mostly be a cause of misconfiguration.

If you previously used iked(8)'s -6 flag to disable this feature, it is no longer needed and should be removed from /etc/rc.conf.local if used.

Linus says dont run ZFS

“Don’t use ZFS. It’s that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.”

This is what Linus Torvalds said in a mailing list to once again express his disliking for ZFS filesystem specially over its licensing.

To avoid unnecessary confusion, this is more intended for Linux distributions, kernel developers and maintainers rather than individual Linux users.

GSoC 2019 Final Report: Incorporating the memory-hard Argon2 hashing scheme into NetBSD

We successfully incorporated the Argon2 reference implementation into NetBSD/amd64 for our 2019 Google Summer of Coding project. We introduced our project here and provided some hints on how to select parameters here. For our final report, we will provide an overview of what changes were made to complete the project.

The Argon2 reference implementation, available here, is available under both the Creative Commons CC0 1.0 and the Apache Public License 2.0. To import the reference implementation into src/external, we chose to use the Apache 2.0 license for this project.

Working towards LLDB on i386 NetBSD

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support.

Throughout December I've continued working on our build bot maintenance, in particular enabling compiler-rt tests. I've revived and finished my old patch for extended register state (XState) in core dumps. I've started working on bringing proper i386 support to LLDB.

Beastie Bits

• An open source Civilization V
• BSD Groups in Italy
• Why is Wednesday, November 17, 1858 the base time for OpenVMS?
• Benchmarking shell pipelines and the Unix “tools” philosophy
• LPI and BSD working together

Feedback/Questions

• Pat - March Meeting
• Madhukar - Overheating Laptop
• Warren - R vs S

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Your Impact on FreeBSD in 2019

It’s hard to believe that 2019 is nearly over. It has been an amazing year for supporting the FreeBSD Project and community! Why do I say that? Because as I reflect over the past 12 months, I realize how many events we’ve attended all over the world, and how many lives we’ve touched in so many ways. From advocating for FreeBSD to implementing FreeBSD features, my team has been there to help make FreeBSD the best open source project and operating system out there.

In 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we’ve stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we’re on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams.

Our advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees.

Our travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD user/contributors. We’ve seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund.

Wireguard on OpenBSD Router

wireguard (wg) is a modern vpn protocol, using the latest class of encryption algorithms while at the same time promising speed and a small code base.

modern crypto and lean code are also tenants of openbsd, thus it was a no brainer to migrate my router from openvpn over to wireguard.

my setup : a collection of devices, both wired and wireless, that are nat’d through my router (openbsd 6.6) out via my vpn provider azire* and out to the internet using wg-quick to start wg.

running : doubtless this could be improved on, but currently i start wg manually when my router boots. this, and the nat'ing on the vpn interface mean its impossible for clients to connect to the internet without the vpn being up. as my router is on a ups and only reboots when a kernel patch requires it, it’s a compromise i can live with. run wg-quick (please replace vpn with whatever you named your wg .conf file.) and reload pf rules.

News Roundup

Amazon now has FreeBSD/ARM 12

AWS, the cloud division of Amazon, announced in December the next generation of its ARM processors, the Graviton2. This is a custom chip design with a 7nm architecture. It is based on 64-bit ARM Neoverse cores.

Compared to first-generation Graviton processors (A1), today’s new chips should deliver up to 7x the performance of A1 instances in some cases. Floating point performance is now twice as fast. There are additional memory channels and cache speed memory access should be much faster.

The company is working on three types of Graviton2 EC2 instances that should be available soon. Instances with a “g” suffix are powered by Graviton2 chips. If they have a “d” suffix, it also means that they have NVMe local storage.

• General-purpose instances (M6g and M6gd)

• Compute-optimized instances (C6g and C6gd)

• Memory-optimized instances (R6g and R6gd)

You can choose instances with up to 64 vCPUs, 512 GiB of memory and 25 Gbps networking.

And you can see that ARM-powered servers are not just a fad. AWS already promises a 40% better price/performance ratio with ARM-based instances when you compare them with x86-based instances.

AWS has been working with operating system vendors and independent software vendors to help them release software that runs on ARM. ARM-based EC2 instances support Amazon Linux 2, Ubuntu, Red Hat, SUSE, Fedora, Debian and FreeBSD. It also works with multiple container services (Docker, Amazon ECS, and Amazon Elastic Kubernetes Service).

• Coverage of AWS Announcement

Announcing the pkgsrc-2019Q4 release

The pkgsrc developers are proud to announce the 65th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 20,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/

In total, 190 packages were added, 96 packages were removed, and 1,868 package updates (to 1388 unique packages) were processed since the pkgsrc-2019Q3 release. As usual, a large number of updates and additions were processed for packages for go (14), guile (11), perl (170), php (10), python (426), and ruby (110). This continues pkgsrc's tradition of adding useful packages, updating many packages to more current versions, and pruning unmaintained packages that are believed to have essentially no users.

The Joys of UNIX Keyboards

I fell in love with a dead keyboard layout.

A decade or so ago while helping a friends father clean out an old building, we came across an ancient Sun Microsystems server. We found it curious. Everything about it was different from what we were used to. The command line was black on white, the connectors strange and foreign, and the keyboard layout was bizarre.

We never did much with it; turning it on made all the lights in his home dim, and our joint knowledge of UNIX was nonexistent. It sat in his bedroom for years supporting his television at the foot of his bed.

I never forgot that keyboard though. The thought that there was this alternative layout out there seemed intriguing to me.

OpenBSD on Digital Ocean

Last night I had a need to put together a new OpenBSD machine. Since I already use DigitalOcean for one of my public DNS servers I wanted to use them for this need but sadly like all too many of the cloud providers they don't support OpenBSD. Now they do support FreeBSD and I found a couple writeups that show how to use FreeBSD as a shim to install OpenBSD.

They are both sort of old at this point and with OpenBSD 6.6 out I ran into a bit of a snag. The default these days is to use a GPT partition table to enable EFI booting. This is generally pretty sane but it looks to me like the FreeBSD droplet doesn't support this. After the installer rebooted the VM failed to boot, being unable to find the bootloader.

Thankfully DigitalOcean has a recovery ISO that you can boot by simply switching to it and powering off and then on your Droplet.

Beastie Bits

• FreeBSD defaults to LLVM on PPC
• Theo De Raadt Interview between Ottawa 2019 Hackathon and BSDCAN 2019
• Bastille Poll about what people would like to see in 2020
• Notes on the classic book : The Design of the UNIX Operating System
• Multics History
• First meeting of the Hamilton BSD user group, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St

Feedback/Questions

• Bill - 1.1 CDROM
• Greg - More 50 Year anniversary information
• Dave - Question time for Allan

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

HyperbolaBSD Announcement

Due to the Linux kernel rapidly proceeding down an unstable path, we are planning on implementing a completely new OS derived from several BSD implementations.

This was not an easy decision to make, but we wish to use our time and resources to create a viable alternative to the current operating system trends which are actively seeking to undermine user choice and freedom.

This will not be a "distro", but a hard fork of the OpenBSD kernel and userspace including new code written under GPLv3 and LGPLv3 to replace GPL-incompatible parts and non-free ones.

• Reasons for this include:
  • Linux kernel forcing adaption of DRM, including HDCP.
  • Linux kernel proposed usage of Rust (which contains freedom flaws and a centralized code repository that is more prone to cyber attack and generally requires internet access to use.)
  • Linux kernel being written without security and in mind. (KSPP is basically a dead project and Grsec is no longer free software)
  • Many GNU userspace and core utils are all forcing adaption of features without build time options to disable them. E.g. (PulseAudio / SystemD / Rust / Java as forced dependencies)
  • As such, we will continue to support the Milky Way branch until 2022 when our legacy Linux-libre kernel reaches End of Life.

Future versions of Hyperbola will be using HyperbolaBSD which will have the new kernel, userspace and not be ABI compatible with previous versions.

HyperbolaBSD is intended to be modular and minimalist so other projects will be able to re-use the code under free license.

• Forum Post

A simple IPFW In-Kernel NAT setup on FreeBSD

After graduating college, I am moving from Brooklyn, NY to Redmond, WA (guess where I got a job). I always wanted to re-do my OPNsense firewall (currently a HP T730) with stock FreeBSD and IPFW’s in-kernel NAT.

Why IPFW? Benchmarks have shown IPFW to be faster which is especially good for my Tor relay, and because I can! However, one downside of IPFW is less documentation vs PF, even less without natd (which we’re not using), and this took me time to figure this out.

But since my T730 is already packed, I am testing this on a old PC with two NICs, and my laptop [1] as a client with an USB-to-Ethernet adapter.

News Roundup

HEADS UP: Wayland and WebRTC enabled for NetBSD 9/Linux

This is just a heads up that the Wayland option is now turned on by

default for NetBSD 9 and Linux in cases where it peacefully coexists
with X11.

• Right now, this effects the following packages:
  • graphics/MesaLib
  • devel/SDL2
  • www/webkit-gtk
  • x11/gtk3

The WebRTC option has also been enabled by default on NetBSD 9 for two Firefox versions: www/firefox, www/firefox68

Please keep me informed of any fallout. Hopefully, there will be none.

If you want to try out Wayland-related things on NetBSD 9, wm/velox/MESSAGE may be interesting for you.

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

OpenSSH U2F/FIDO support in base

Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.

You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.

So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.

drm/i915: Update to Linux 4.8.17

• drm/i915: Update to Linux 4.8.17
  • Broxton, Valleyview and Cherryview support improvements
  • Broadwell and Gen9/Skylake support improvements
  • Broadwell brightness fixes from OpenBSD
  • Atomic modesetting improvements
  • Various bug fixes and performance enhancements

Beastie Bits

• Visual Studio Code port for FreeBSD
• OpenBSD syscall call-from verification
• Peertube on OpenBSD
• Fuzzing Filesystems on NetBSD via AFL+KCOV by Maciej Grochowski
• Twitter Bot for Prop65
• Interactive vim tutorial
• First BSD user group meeting in Hamilton, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St ***

Feedback/Questions

• Samir - cgit
• Russell - R
• Wolfgang - Question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Why computers suck and how learning from OpenBSD can make them marginally less horrible

How much better could things actually be if we abandoned the enterprise development model?

Next I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase.

One of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD.

To prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively.

How Unix Works: Become a Better Software Engineer

Unix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system.

Every fancy thing you want done is one google search away.

But understanding why the solution does what you want is not the same.

That’s what gives you real power, the power to not be afraid.

And since it rhymes, it must be true.

News Roundup

FreeBSD 12.1 Runs Refreshingly Well With AMD Ryzen Threadripper 3970X

For those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform.

It was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround.

While the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1.

FreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system.

BSDCan 2020 CFP

BSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu).

NOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June.

We are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

• See http://www.bsdcan.org/2020/

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include:

• How we manage a giant installation with respect to handling spam.
• and/or sysadmin.
• and/or networking.
• Cool new stuff in BSD
• Tell us about your project which runs on BSD
• other topics (see next paragraph)

From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples.

Both users and developers are encouraged to share their experiences.

HardenedBSD Infrastructure Goals

2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion.

The last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity.

My available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them.

My primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub.

As I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor.

Transparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor.

Integrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published.

I hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired.

So, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows.

Beastie Bits

• FuryBSD - KDE plasma flavor now available
• DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud
• LPI is looking for BSD Specialist learning material writers
• ZFS sync/async + ZIL/SLOG, explained
• BSD-Licensed Combinatorics library/utility
• SSL client vs server certificates and bacula-fd
• MaxxDesktop planning to come to FreeBSD Project Page

Feedback/Questions

• Tom - ZFS Mirror with different speeds
• Jeff - Knowledge is power
• Johnny - Episode 324 response to Jacob
• Pat - NYC*BUG meeting Jan Meeting Location

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Authentication vulnerabilities in OpenBSD

• We discovered an authentication-bypass vulnerability in OpenBSD's authentication system: this vulnerability is remotely exploitable in smtpd, ldapd, and radiusd, but its real-world impact should be studied on a case-by-case basis. For example, sshd is not exploitable thanks to its defense-in-depth mechanisms.

• From the manual page of login.conf:

  OpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are:
  > passwd Request a password and check it against the password in the master.passwd file. See login_passwd(8).
  > skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See login_skey(8).
  > yubikey Authenticate using a Yubico YubiKey token. See login_yubikey(8).
  > For any given style, the program /usr/libexec/auth/login_style is used to
  > perform the authentication. The synopsis of this program is:

  > /usr/libexec/auth/login_style [-v name=value] [-s service] username class
  

• This is the first piece of the puzzle: if an attacker specifies a username of the form "-option", they can influence the behavior of the authentication program in unexpected ways.

   login_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program.  The allowed protocols are login, challenge, and response.  (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based).
  

• This is the second piece of the puzzle: if an attacker specifies the username "-schallenge" (or "-schallenge:passwd" to force a passwd-style authentication), then the authentication is automatically successful and therefore bypassed.

• Case study: smtpd

• Case study: ldapd

• Case study: radiusd

• Case study: sshd

• Acknowledgments: We thank Theo de Raadt and the OpenBSD developers for their incredibly quick response: they published patches for these vulnerabilities less than 40 hours after our initial contact. We also thank MITRE's CVE Assignment Team.

First release candidate for NetBSD 9.0 available!

• Since the start of the release process four months ago a lot of improvements went into the branch - more than 500 pullups were processed!
• This includes usbnet (a common framework for usb ethernet drivers), aarch64 stability enhancements and lots of new hardware support, installer/sysinst fixes and changes to the NVMM (hardware virtualization) interface.
• We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).
• Here are a few highlights of the new release: > Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA) > Enhanced hardware support for Armv7-A > Updated GPU drivers (e.g. support for Intel Kabylake) > Enhanced virtualization support > Support for hardware-accelerated virtualization (NVMM) > Support for Performance Monitoring Counters > Support for Kernel ASLR > Support several kernel sanitizers (KLEAK, KASAN, KUBSAN) > Support for userland sanitizers > Audit of the network stack > Many improvements in NPF > Updated ZFS > Reworked error handling and NCQ support in the SATA subsystem > Support a common framework for USB Ethernet drivers (usbnet)
• More information on the RC can be found on the NetBSD 9 release page

News Roundup

Running FreeNAS on a Digitalocean droplet

• ZFS is awesome. FreeBSD even more so. FreeNAS is the battle-tested, enterprise-ready-yet-home-user-friendly software defined storage solution which is cooler then deep space, based on FreeBSD and makes heavy use of ZFS. This is what I (and soooooo many others) use for just about any storage-related task. I can go on and on and on about what makes it great, but if you're here, reading this, you probably know all that already and we can skip ahead.
• I've needed an offsite FreeNAS setup to replicate things to, to run some things, to do some stuff, basically, my privately-owned, tightly-controlled NAS appliance in the cloud, one I control from top to bottom and with support for whatever crazy thing I'm trying to do. Since I'm using DigitalOcean as my main VPS provider, it seemed logical to run FreeNAS there, however, you can't. While DO supports many many distos and pre-setup applications (e.g OpenVPN), FreeNAS isn't a supported feature, at least not in the traditional way :)
• Before we begin, here's the gist of what we're going to do: > Base of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device.
• Part 1: re-image our boot block device to boot FreeNAS install media.
• Part 2: Install FreeNAS on the second block-device
• Part 3: Re-image the boot block device using the FreeNAS-installed block device

NomadBSD 1.3 is now available

• From the release notes:
• > The base system has been changed to FreeBSD 12.1-RELEASE-p1 Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse The GPT layout has been changed to MBR. This prevents problems with Lenovo systems that refuse to boot from GPT if "lenovofix" is not set, and systems that hang on boot if "lenovofix" is set. Support for ZFS installations has been added to the NomadBSD installer. The rc-script for setting up the network interfaces has been fixed and improved. Support for setting the country code for the wlan device has been added. Auto configuration for running in VirtualBox has been added. A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead. NVIDIA driver version 440 has been added. nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the default user and autologin has been added. nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added. Martin Orszulik added Czech translations to the setup and installation wizard. The NomadBSD logo, designed by Ian Grindley, has been changed. Support for localized error messages has been added. Support for localizing the password prompts has been added. Some templates for starting other DEs have been added to ~/.xinitrc. The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved. A script that helps users to configure a multihead systems has been added. The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting". /proc has been added to /etc/fstab A D-Bus session issue has been fixed which prevented thunar from accessing samba shares. DSBBg which allows users to change and manage wallpapers has been added. The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed.

Support for multiple keyboard layouts has been added.
www/palemoon has been removed.
mail/thunderbird has been removed.
audio/audacity has been added.
deskutils/orage has been added.
the password manager fpm2 has been replaced by KeePassXC
mail/sylpheed has been replaced by mail/claws-mail
multimedia/simplescreenrecorder has been added.
DSBMC has been changed to DSBMC-Qt
Many small improvements and bug fixes.

At e2k19 nobody can hear you scream

• After 2 years it was once again time to pack skis and snowshoes, put a satellite dish onto a sledge and hike through the snowy rockies to the Elk Lakes hut.
• I did not really have much of a plan what I wanted to work on but there were a few things I wanted to look into. One of them was rpki-client and the fact that it was so incredibly slow. Since Bob beck@ was around I started to ask him innocent X509 questions ... as if there are innocent X509 questions! Mainly about the abuse of the X509_STORE in rpki-client. Pretty soon it was clear that rpki-client did it all wrong and most of the X509 verification had to be rewritten. Instead of only storing the root certificates in the store and passing the intermediate certs as a chain to the verification function rpki-client threw everything into it. The X509_STORE is just not built for such an abuse and so it was no wonder that this was slow.
• Lucky me I pulled benno@ with me into this dark hole of libcrypto code. He managed to build up an initial diff to pass the chains as a STACK_OF(X509) and together we managed to get it working. A big thanks goes to ingo@ who documented most of the functions we had to use. Have a look at STACK_OF(3) and sk_pop_free(3) to understand why benno@ and I slowly turned crazy.
• Our next challenge was to only load the necessary certificate revocation list into the X509_STORE_CTX. While doing those changes it became obvious that some of the data structures needed better lookup functions. Looking up certificates was done using a linear lookup and so we replaced the internal certificate and CRL tables with RB trees for fast lookups. deraadt@ also joined the rpki-client commit fest and changed the output code to use rename(2) so that files are replaced in an atomic operation. Thanks to this rpki-client can now be safely run from cron (there is an example in the default crontab).
• I did not plan to spend most of my week hacking on rpki-client but in the end I'm happy that I did and the result is fairly impressive. Working with libcrypto code and especially X509 was less than pleasant. Our screams of agony died away in the snowy rocky mountains and made Bob deep dive into UVM with a smile since he knew that benno@ and I had it worse.
• In case you wonder thanks to all changes at e2k19 rpki-client improved from over 20min run time to validate all VRPS to roughly 1min to do the same job. A factor 20 improvement!
• Thanks to Theo, Bob and Howie to make this possible. To all the cooks for the great food and to Xplornet for providing us with Internet at the hut.

Beastie Bits

• FOSDEM 2020 BSD Devroom schedule
• Easy Minecraft Server on FreeBSD Howto
• stats(3) framework in the TCP stack
• 4017 days of uptime
• sysget - A front-end for every package manager
• PlayOnBSD’s Cross-BSD Shopping Guide

Feedback/Questions

• Pat asks about the proper disk drive type for ZFS
• Brad asks about a ZFS rosetta stone

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Special Guest: Mariusz Zaborski.

Interview - Michael Lucas

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Special Guest: Michael W Lucas.

Headlines

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Multiple IPSec VPN tunnels with FreeBSD

The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html)

But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below.

The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE).

Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C).

VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C).

News Roundup

Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance

Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel.

Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made.

For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%.

unwind(8); "happy eyeballs"

In case you are wondering why happy eyeballs: It's a variation on this:
https://en.wikipedia.org/wiki/Happy_Eyeballs

unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers.

This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix.

One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E):
17 files changed, 385 insertions(+), 1683 deletions(-)

Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals.

Amazon now has FreeBSD ARM 12

Product Overview

FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years.

FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems.

OpenSSH U2F/FIDO support in base

I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys.

Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.

You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.

So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.

Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too.

Please test this thoroughly - it's a big change that we want to have stable before the next release.

Beastie Bits

• DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud
• Really fast Markov chains in ~20 lines of sh, grep, cut and awk
• FreeBSD Journal Sept/Oct 2019
• Michael Dexter is raising money for Bhyve development
• syscall call-from verification
• FreeBSD Forums Howto Section

Feedback/Questions

• Jeroen - Feedback
• Savo - pfsense ports
• Tin - I want to learn C

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Sponsored By:

• Linux Academy: Give yourself a year of opportunity and save $150. Get a full year of Hands-On Cloud Training. Limited time Black Friday Offer.

Headlines

FreeBSD third quarterly status report for 2019

This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).

Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.

Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.

Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.

OpenBSD on Sparc64

OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.

First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.

Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.

News Roundup

ZoL repo move to OpenZFS

Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs.

The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020.

Mcclure111 Sun Thread

A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback.

GEOM NOP

Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.

GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks.

Keeping NetBSD up-to-date with pkg_comp 2.0

This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.

Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.

This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.

Beastie Bits

• DragonFly - Radeon Improvements
• NomadBSD review
• Spongebob OpenBSD Security Comic
• Forth : The Early Years
• LCM+L PDP-7 booting and running UNIX Version 0

Feedback/Questions

• Chris - Ctrl-T
  • Improved Ctrl+t that shows kernel backtrace
• Brian - Migrating NexentaStore to FreeBSD/FreeNAS
• Avery - How to get involved

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group

Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018.

G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.”

OpenZFS Trip Report

The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling.

News Roundup

Using FreeBSD with Ports (2/2): Tool-assisted updating

• Part 1 here: https://eerielinux.wordpress.com/2019/08/18/using-freebsd-with-ports-1-2-classic-way-with-tools/

In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient.

In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today.

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Linux VS open source UNIX

Beastie Bits

• Support for Realtek RTL8125 2.5Gb Ethernet controller
• Computer Files Are Going Extinct
• FreeBSD kernel hacking
• Modern BSD Computing for Fun on a VAX! Trying to use a VAX in today's world by Jeff Armstrong
• MidnightBSD 1.2 Released

Feedback/Questions

• Paulo - Zfs snapshots
• Phillip - GCP
• A Listener - Old episodes?

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Sponsored By:

• Linux Academy: Give yourself a year of opportunity and save $150. Get a full year of Hands-On Cloud Training. Limited time Black Friday Offer.

Headlines

FreeBSD 12.1

• Some of the highlights:

  • BearSSL has been imported to the base system.
  • The clang, llvm, lld, lldb, compiler-rt utilities and libc++ have been updated to version 8.0.1.
  • OpenSSL has been updated to version 1.1.1d.
  • Several userland utility updates.

• For a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html

A History of UNIX before Berkeley: UNIX Evolution: 1975-1984.

Nobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today.

Our title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question.

Part One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at Research'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as nowthe Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time.

News Roundup

My FreeBSD Development Setup

I do my FreeBSD development using git, tmux, vim and cscope.

I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd

OPNsense 19.7.6 released

As we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available.

LDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis.

HardenedBSD November 2019 Status Report.

We at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work.

DNSSEC enabled in default unbound(8) configuration.

DNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@)

How to Install Shopware with NGINX and Let's Encrypt on FreeBSD 12

Shopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server.

• Requirements

Make sure your system meets the following minimum requirements:

• Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed.
• PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended.
• MySQL 5.5.0 or higher.
• Possibility to set up cron jobs.
• Minimum 4 GB available hard disk space.
• IonCube Loader version 5.0.0 or higher (optional).

How to Compile RainbowCrack on OpenBSD

Project RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still just his or if it's even been maintained for a while. His page seems to have been last updated in August 2007.

The Project RainbowCrack web page now has just binaries for Windows XP and Linux, both 32-bit and 64-bit versions.

Earlier versions were available as source code. The version 1.2 source code does not compile on OpenBSD, and in my experience it doesn't compile on Linux, either. It seems to date from 2004 at the earliest, and I think it makes some version-2.4 assumptions about Linux kernel headers.

• You might also look at ophcrack, a more modern tool, although it seems to be focused on cracking Windows XP/Vista/7/8/10 password hashes

Feedback/Questions

• Reese - Amature radio info
• Chris - VPN
• Malcolm - NAT

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Migrating drives and the zpool from one host to another.

Today is the day.

Today I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host.

Fortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail.

Now it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up.

• In this post:

  • FreeBSD 12.0
  • Dell R710 (r710-01)
  • Dell R720 (r720-01)
  • drive caddies from eBay and now I know the difference between SATA and SATAu

• PLEASE READ THIS first: Migrating ZFS Storage Pools

OpenBSD in 2019

I’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again.

What triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine.

I don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux.

That and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things.

This is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem.

• Verdict

Ouch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD.

And maybe in a year I’ll have a review of OpenBSD on a laptop.

News Roundup

New zlib, new dhcpcd

zlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed.

• DHCPCD Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719768.html
• ZLIB Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719772.html

Batch renaming images, including image resolution, with awk

The most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code:

$ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done
IMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg
IMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg
IMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg
IMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg
IMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg
IMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg
IMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg
IMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg
IMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg
IMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg
IMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg
IMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg
IMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg
// ... etc etc

The last item on the aforementioned list is “TODO: come up with a shorter title for this list.”

I hate the X11 ICCCM selection system, and you should too - A Rant

d00d, that document is devilspawn. I've recently spent my nights in pain
implementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program.

I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates?

So what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misunderstanding it. If you didn't misunderstand it, you wouldn't be here complaining about it would you?

HAMMER2 emergency space mode

As anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on!

Beastie Bits

• The BastilleBSD community has started work on over 100 automation templates
• PAM perturbed
• OpenBSD T-Shirts now available
• FastoCloud (Opensource Media Service) now available on FreeBSD
• Unix: A History and a Memoir by Brian Kernighan now available
• OpenBSD Moonlight game streaming client from a Windows + Nvidia PC ***

Feedback/Questions

• Tim - Release Notes for Lumina 1.5
  • Answer Here
• Brad - vBSDcon Trip Report
• Jacob - Using terminfo on FreeBSD

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

The Earliest Unix Code: An Anniversary Source Code Release

What is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system.

2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs.

This man sent the first online message 50 years ago

• As many of you have heard in the past, the first online message ever sent between two computers was "lo", just over 50 years ago, on Oct. 29, 1969.

It was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in."

• The CBC Radio show, “The Current” has a half-hour interview with the man who sent that message, Leonard Kleinrock, distinguished professor of computer science at UCLA

"The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there,"

50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it.

Q: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later?

A: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible.

Well, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element.

Q: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another?

A: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it.

So I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting.

Q: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated.

A: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls.

When the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side.

But in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways.

Q: Is there any part of you that regrets giving birth to this?

A: Absolutely not. The greater good is much more important.

News Roundup

How to use blacklistd(8) with NPF as a fail2ban replacement

blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.

The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf

Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.

Unfortunately (dont' ask me why ??) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen.

• FreeBSD’s handbook chapter on blacklistd

OpenBSD crossed 400,000 commits

Sometime in the last week OpenBSD crossed 400,000 commits (*) upon all our repositories since starting at 1995/10/18 08:37:01 Canada/Mountain. That's a lot of commits by a lot of amazing people.

(*) by one measure. Since the repository is so large and old, there are a variety of quirks including ChangeLog missing entries and branches not convertible to other repo forms, so measuring is hard. If you think you've got a great way of measuring, don't be so sure of yourself -- you may have overcounted or undercounted.

• Subject to the notes Theo made about under and over counting, FreeBSD should hit 1 million commits (base + ports + docs) some time in 2020
• NetBSD + pkgsrc are approaching 600,000, but of course pkgsrc covers other operating systems too

How to Install Bolt CMS with Nginx and Let's Encrypt on FreeBSD 12

Bolt is a sophisticated, lightweight and simple CMS built with PHP. It is released under the open-source MIT-license and source code is hosted as a public repository on Github. A bolt is a tool for Content Management, which strives to be as simple and straightforward as possible. It is quick to set up, easy to configure, uses elegant templates. Bolt is created using modern open-source libraries and is best suited to build sites in HTML5 with modern markup. In this tutorial, we will go through the Bolt CMS installation on FreeBSD 12 system by using Nginx as a web server, MySQL as a database server, and optionally you can secure the transport layer by using acme.sh client and Let's Encrypt certificate authority to add SSL support.

• Requirements
• The system requirements for Bolt are modest, and it should run on any fairly modern web server:
  • PHP version 5.5.9 or higher with the following common PHP extensions: pdo, mysqlnd, pgsql, openssl, curl, gd, intl, json, mbstring, opcache, posix, xml, fileinfo, exif, zip.
  • Access to SQLite (which comes bundled with PHP), or MySQL or PostgreSQL.
  • Apache with mod_rewrite enabled (.htaccess files) or Nginx (virtual host configuration covered below).
  • A minimum of 32MB of memory allocated to PHP.

hammer2 - Optimize hammer2 support threads and dispatch

Refactor the XOP groups in order to be able to queue strategy calls, whenever possible, to the same CPU as the issuer. This optimizes several cases and reduces unnecessary IPI traffic between cores. The next best thing to do would be to not queue certain XOPs to an H2 support thread at all, but I would like to keep the threads intact for later clustering work.

The best scaling case for this is when one has a large number of user threads doing I/O. One instance of a single-threaded program on an otherwise idle machine might see a slightly reduction in performance but at the same time we completely avoid unnecessarily spamming all cores in the system on the behalf of a single program, so overhead is also significantly lower.

This will tend to increase the number of H2 support threads since we need a certain degree of multiplication for domain separation.

This should significantly increase I/O performance for multi-threaded workloads.

You know, we might as well just run every network service over HTTPS/2 and build another six layers on top of that to appease the OSI 7-layer burrito guys

I've seen the writing on the wall, and while for now you can configure Firefox not to use DoH, I'm not confident enough to think it will remain that way. To that end, I've finally set up my own DoH server for use at Chez Boca. It only involved setting up my own CA to generate the appropriate certificates, install my CA certificate into Firefox, configure Apache to run over HTTP/2 (THANK YOU SO VERY XXXXX­XX MUCH GOOGLE FOR SHOVING THIS HTTP/2 XXXXX­XXX DOWN OUR THROATS!—no, I'm not bitter) and write a 150 line script that just queries my own local DNS, because, you know, it's more XXXXX­XX secure or some XXXXX­XXX reason like that.

Sigh.

Beastie Bits

• An Oral History of Unix
• NUMA Siloing in the FreeBSD Network Stack [pdf]
• EuroBSDCon 2019 videos available
• Barbie knows best
• For the #OpenBSD #e2k19 attendees. I did a pre visit today.
• Drawer Find
• Slides - Removing ROP Gadgets from OpenBSD - AsiaBSDCon 2019

Feedback/Questions

• Bostjan - Open source doesn't mean secure
• Malcolm - Allan is Correct.

• Michael - FreeNAS inside a Jail

  ---

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Unix is 50

In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing.

Hunting down Ken's PDP-7: video footage found

In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix.

News Roundup

OpenBSD 6.6 Released

• Announce: https://marc.info/?l=openbsd-tech&m=157132024225971&w=2
• Upgrade Guide: https://openbsd.org/faq/upgrade66.html
• Changelog: https://openbsd.org/plus66.html

OPNsense 19.7.5 released

Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version

12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so.

Here are the full patch notes:

• system: show all swap partitions in system information widget
• system: flatten services_get() in preparation for removal
• system: pin Syslog-ng version to specific package name
• system: fix LDAP/StartTLS with user import page
• system: fix a PHP warning on authentication server page
• system: replace most subprocess.call use
• interfaces: fix devd handling of carp devices (contributed by stumbaumr)
• firewall: improve firewall rules inline toggles
• firewall: only allow TCP flags on TCP protocol
• firewall: simplify help text for direction setting
• firewall: make protocol log summary case insensitive
• reporting: ignore malformed flow records
• captive portal: fix type mismatch for timeout read
• dhcp: add note for static lease limitation with lease registration (contributed by Northguy)
• ipsec: add margintime and rekeyfuzz options
• ipsec: clear $dpdline correctly if not set
• ui: fix tokenizer reorder on multiple saves
• plugins: os-acme-client 1.26[1]
• plugins: os-bind will reload bind on record change (contributed by blablup)
• plugins: os-etpro-telemetry minor subprocess.call replacement
• plugins: os-freeradius 1.9.4[2]
• plugins: os-frr 1.12[3]
• plugins: os-haproxy 2.19[4]
• plugins: os-mailtrail 1.2[5]
• plugins: os-postfix 1.11[6]
• plugins: os-rspamd 1.8[7]
• plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks)
• plugins: os-telegraf 1.7.6[8]
• plugins: os-theme-cicada 1.21 (contributed by Team Rebellion)
• plugins: os-theme-tukan 1.21 (contributed by Team Rebellion)
• plugins: os-tinc minor subprocess.call replacement
• plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz)
• plugins: os-virtualbox 1.0 (contributed by andrewhotlab)

Dealing with the misunderstandings of what is GhostBSD

Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE.

Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release.

There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum.

SHUTTLE – VPN over SSH | VPN Alternative

Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy.

• VPN over SSH – sshuttle

sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN.

OpenSSH 8.1 Released

• Security

  • ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program.
  • ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB).

• This release includes a number of changes that may affect existing configurations:

  • ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ...").

• New Features

  • ssh(1): Allow %n to be expanded in ProxyCommand strings
  • ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '^' character, E.g. "HostKeyAlgorithms ^ssh-ed25519"
  • ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email).
  • ssh-keygen(1): print key comment when extracting public key from a private key.
  • ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too.
  • All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's.

Beastie Bits

• Say goodbye to the 32 CPU limit in NetBSD/aarch64
• vBSDcon 2019 videos
• Browse the web in the terminal - W3M
• NetBSD 9 and GSoC
• BSDCan 2019 Videos
• NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders
• FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open
• FOSDEM 2020 - BSD Devroom Call for Participation
• University of Cambridge looking for Research Assistants/Associates

Feedback/Questions

• Trenton - Beeping Thinkpad
• Alex - Per user ZFS Datasets
  • Allan’s old patch from 2015
• Javier - FBSD 12.0 + ZFS + encryption

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Interview - Trenton Schulz - freenas@norwegianrockcat.com

Robot OS on FreeBSD

• BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD?
• AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how?
• BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD?
• AJ: Was EuroBSDcon your first BSD conference? How did you like it?
• BR: Do you have some tips or advice on how to get started with the BSDs?
• AJ: Is there anything else you’d like to tell us before we let you go?

Beastie Bits

• FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open
• Portland BSD Pizza Night: Oct 24th, 19:00 @ Rudy’s Gourmet Pizza
• NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders
• FOSDEM 2020 - BSD Devroom Call for Participation
• University of Cambridge looking for Research Assistants/Associates

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Tags: freebsd, openbsd, netbsd, dragonflybsd, trueos, trident, hardenedbsd, tutorial, howto, guide, bsd, interview, google pixelbook, pixelbook, case study, portability, porting, zfs, zfs performance, performance, quota, quota limits, zfs quota, ka9q, unix, hammer2, fsck, startx
Date: 2019-10-16
*/

Headlines

FreeBSD and custom firmware on the Google Pixelbook

• FreeBSD and custom firmware on the Google Pixelbook

Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something..

• lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);
• with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);
• with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);
• with USB-C ports;
• without a dGPU, especially without an NVIDIA GPU;
• assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);
• supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers);
• how about something with open source firmware, that would be fun.

I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less!

And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge).

So here it is, my new laptop, a Google Pixelbook.

• Conclusion

Pixelbook, FreeBSD, coreboot, EDK2 good.

Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)

Porting NetBSD to the AMD x86-64: a case study in OS portability

• Abstract

NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.

• Portability

Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.

• General

Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.

• Types

Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.

• Conclusions and future work

The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.

News Roundup

ZFS performance really does degrade as you approach quota limits

Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.

• In the September meeting notes, I read a very interesting (to me) agenda item:
  • Relax quota semantics for improved performance (Allan Jude)
  • Problem: As you approach quotas, ZFS performance degrades.
  • Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.

This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).

Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)

With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.

Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.."

I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.

Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.

So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.

But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!

I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.

HAMMER2 and fsck for review

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

[The return of startx(1) for non-root users with some caveats

Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:

CVSROOT:    /cvs
Module name:    src
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:25:41

Modified files:
    etc/etc.amd64  : fbtab 
    etc/etc.arm64  : fbtab 
    etc/etc.hppa   : fbtab 
    etc/etc.i386   : fbtab 
    etc/etc.loongson: fbtab 
    etc/etc.luna88k: fbtab 
    etc/etc.macppc : fbtab 
    etc/etc.octeon : fbtab 
    etc/etc.sgi    : fbtab 
    etc/etc.sparc64: fbtab 

Log message:
Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.

ok jsg@, matthieu@
CVSROOT:    /cvs
Module name:    xenocara
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:31:08

Modified files:
    xserver/hw/xfree86/common: xf86AutoConfig.c 

Log message:
Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4).  In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

ok jsg@, matthieu@

Beastie Bits

• ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal?
• Sourcehut makes BSD software better
• Chaosnet for Unx
• The Vim-Inspired Editor with a Linguistic Twist
• bhyvearm64: CPU and Memory Virtualization on Armv8.0-A
• DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities

Feedback/Questions

• Tim - GSoC project ideas for pf rule syntax translation
• Brad - Steam on FreeBSD
• Ruslan - FreeBSD Quarterly Status Report - Q2 2019

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Causing ZFS corruption for fun and profit

Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios.

• Causing Corruption

Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets.

• Conclusion

At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events.

To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject.

NetBSD Assembly Programming Tutorial

A sparc64 version is also being prepared and will be added when done

This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel.

• Why assembly?

Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs

• Due to the nature of the guide, visit the site for the complete breakdown

News Roundup

The IKEA Lack Rack for Servers

• The LackRack

First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009.

The LackRack will certainly make its appearance again this summer at eth0:2010 Summer.

• Summary

When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks.

The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented!

• Howto

You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat!

OmniOS Community Edition r151030 LTS - Published at May 6, 2019

The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030.

OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details.

This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details.

If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28.

• For full relase notes including upgrade instructions;
• release notes
• upgrade instructions

List Block Devices on FreeBSD lsblk(8) Style

When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world.

Example lsblk(8) output from Linux system:

$ lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE   MOUNTPOINT
sr0                           11:0    1  1024M  0 rom
sda                            8:0    0 931.5G  0 disk
|-sda1                         8:1    0   500M  0 part   /boot
`-sda2                         8:2    0   931G  0 part
  |-vg_local-lv_root (dm-0)  253:0    0    50G  0 lvm    /
  |-vg_local-lv_swap (dm-1)  253:1    0  17.7G  0 lvm    [SWAP]
  `-vg_local-lv_home (dm-2)  253:2    0   1.8T  0 lvm    /home
sdc                            8:32   0 232.9G  0 disk
`-sdc1                         8:33   0 232.9G  0 part
  `-md1                        9:1    0 232.9G  0 raid10 /data
sdd                            8:48   0 232.9G  0 disk
`-sdd1                         8:49   0 232.9G  0 part
  `-md1                        9:1    0 232.9G  0 raid10 /data

What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs.

• See the article for the rest of the guide

Project Trident 19.10 Now Available

This is a general package update to the CURRENT release repository based upon TrueOS 19.10

• PACKAGE CHANGES FROM 19.08
  • New Packages: 601
  • Deleted Packages: 165
  • Updated Packages: 3341

Beastie Bits

• NetBSD building tools
• Sponsorships open for SNMP Mastery
• pkgsrc-2019Q3 release announcement (2019-10-03)
• pfetch - A simple system information tool written in POSIX sh
• Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview)
• Cracking Ken Thomson’s password

Feedback/Questions

• Evilham - Couple Questions
• Rob - APU2 alternatives and GPT partition types
• Tom - FreeBSD journal article by A. Fengler

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X

For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04.

Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs.

We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs.

For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round.

All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test.

JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives

iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete.

Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection.

Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors.

With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes.

• Youtube Video

News Roundup

FreeBSD 12.1-beta available

FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system.

FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes.

For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures.

The FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November.

• Announcement Link

Cool, but obscure X11 tools. More suggestions in the source link

• ASClock
• Free42
• FSV2
• GLXGears
• GMixer
• GVIM
• Micropolis
• Sunclock
• Ted
• TiEmu
• X026
• X48
• XAbacus
• XAntfarm
• XArchiver
• XASCII
• XBiff
• XBill
• XBoard
• XCalc
• XCalendar
• XCHM
• XChomp
• XClipboard
• XClock
• XClock/Cat Clock
• XColorSel
• XConsole
• XDiary
• XEarth
• XEdit
• Xev
• XEyes
• XFontSel
• XGalaga
• XInvaders 3D
• XKill
• XLennart
• XLoad
• XLock
• XLogo
• XMahjongg
• XMan
• XMessage
• XmGrace
• XMixer
• XmMix
• XMore
• XMosaic
• XMOTD
• XMountains
• XNeko
• XOdometer
• XOSView
• Xplore
• XPostIt
• XRoach
• XScreenSaver
• XSnow
• XSpread
• XTerm
• XTide
• Xv
• Xvkbd
• XWPE
• XZoom

vBSDCon 2019 trip report from iXSystems

The fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn.

Project Trident 12-U7 now available

• Package Summary
  • New Packages: 130
  • Deleted Packages: 72
  • Updated Packages: 865
• Stable ISO - https://pkg.project-trident.org/iso/stable/Trident-x64-TOS-12-U7-20190920.iso

A Couple new Unix Artifacts

I fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems.

So I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me:

• by two large organisations
• of great significance to Unix history
• who want me to keep "mum" about them
• as they are going to make announcements about them soon*

and I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-)

Cheers, Warren

* for some definition of "soon"

Beastie Bits

• NetBSD machines at Open Source Conference 2019 Hiroshima
• Hyperbola a GNU/Linux OS is using OpenBSD's Xenocara
• Talos is looking for a FreeBSD Engineer
• GitHub - dylanaraps/pure-sh-bible: A collection of pure POSIX sh alternatives to external processes.
• dsynth: you’re building it
• Percy Ludgate, the missing link between Babbage’s machine and everything else

Feedback/Questions

• Bruce - Down the expect rabbithole
• Bruce - Expect (update)
• David - Netgraph answer
• Mason - Beeps?

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

EuroBSDcon 2019 Recap

We’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials, parallel to the FreeBSD Devsummit, followed by two days of talks. Some speakers uploaded their slides to papers.freebsd.org already with more to come.

The social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant.

• The opening keynote by Patricia Aas was very good. Her talk on embedded ethics, from her perspective as someone trying to defend the sanctity of Norwegian elections, and a former developer for the Opera web browser, provided a great deal of insight into the issues. Her points about how the tech community has unleashed a very complex digital work upon people with barely any technical literacy were well taken. Her stories of trying to explain the problems with involving computers in the election process to journalists and politicians struck a chord with many of us, who have had to deal with legislation written by those who do not truly understand the issues with technology.

Setting up buildbot in FreeBSD jails

In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here.

Setting up a mail server with OpenSMTPD, Dovecot and Rspamd

• Self-hosting and encouraging smaller providers is for the greater good

First of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps.

It doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice.

There is nothing wrong with Random Joes using a service that works.

What is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined.

News Roundup

The HamBSD project aims to bring amateur packet radio to OpenBSD

The HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system.

HamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code.

The amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither).

DragonFlyBSD's HAMMER2 Gets Basic FSCK Support

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

• commit

Add initial fsck support for HAMMER2, although CoW fs doesn't require fsck as a concept. Currently no repairing (no write), just verifying.

Keep this as a separate command for now.
https://i.redd.it/vkdss0mtdpo31.jpg

---

The return of startx for users

Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

Beastie Bits

• Ori Bernstein will be giving the October talk at NYCBUG
• BSD Pizza Night: 2019/09/26, 7–9PM, Portland, Oregon, USA
• Nick Wolff : Home Lab Show & Tell
• Installing the Lumina Desktop in DragonflyBSD
• dhcpcd 8.0.6 added

Feedback/Questions

• Bruce - FOSDEM videos
• Lars - Super Cluster of BSD on Rock64Pr
• Madhukar - Question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

LLVM santizers and GDB regression test suite.

As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.

I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.

• NetBSD distribution changes

I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).

• The following changes were committed to the sources:
  • ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers
  • 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build
  • 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data
  • fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data
  • 40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER
  • b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER
  • c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER
  • 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER
  • fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER
  • 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data
  • 5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot
  • d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk
  • 2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER
  • ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7
  • 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7
  • d8671fba7a78 Set NODEBUG for LLVM sanitizers
  • 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER
  • 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers
  • e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers
  • 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers
  • 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers
  • 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers
  • a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c
  • 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer
  • 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image
  • 64361771c78d Enhance MKSANITIZER support
  • 3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests
  • c27f4619d513 Avoids signedness bit shift in db_get_value()
  • 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)
  • 4ecfbbba2f2a Rework the LLVM compiler_rt build rules
  • 748813da5547 Correct the build rules of LLVM sanitizers
  • 20e223156dee Enhance the support of LLVM sanitizers
  • 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files
  • Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.

Homura - a Windows Games Launcher for FreeBSD

Inspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD.

• Makes it easier to run games on FreeBSD, by providing the tweaks and dependencies for you
• Dependencies
  • curl
  • bash
  • p7zip
  • zenity
  • webfonts
  • alsa-utils (Optional)
  • winetricks
  • vulkan-tools
  • mesa-demos
  • i386-wine-devel on amd64 or wine-devel on i386

News Roundup

Ada—The Language of Cost Savings?

Many myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust.

Chris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solution.”

In general, Ada already makes embedded “programming in the large” much easier by handling issues that aren’t even addressed in other languages. Though these features are often provided by third-party software, it results in inconsistent practices among developers. Ada also supports the gamut of embedded platforms from systems like Arm’s Cortex-M through supercomputers. Learning Ada isn’t as hard as one might think and the benefits can be significant.

FreeBSD core team appoints a WG to explore transitioning from Subversion to Git.

• The FreeBSD Core Team is the governing body of FreeBSD.

Core approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph).

The annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019.

The core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic.

There is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward.

OpenBSD 6.6 Beta tagged

CVSROOT:    /cvs
Module name:    src
Changes by:    deraadt@cvs.openbsd.org    2019/08/09 21:56:02

Modified files:
    etc/root : root.mail
    share/mk : sys.mk
    sys/arch/macppc/stand/tbxidata: bsd.tbxi
    sys/conf : newvers.sh
    sys/sys : param.h
    usr.bin/signify: signify.1

Log message:
move to 6.6-beta

Preliminary release notes

Improved hardware support, including:

• clang(1) is now provided on powerpc.
• IEEE 802.11 wireless stack improvements:
• Generic network stack improvements:
• Installer improvements:
• Security improvements:
• + Routing daemons and other userland network improvements
• + The ntpd(8) daemon now gets and sets the clock in a secure way when booting even when a battery-backed clock is absent.
• + bgdp(8) improvements
• + Assorted improvements:
• + The filesystem buffer cache now more aggressively uses memory outside the DMA region, to improve cache performance on amd64 machines.
• The BER API previously internal to ldap(1), ldapd(8), ypldap(8), and snmpd(8) has been moved into libutil. See ber_read_elements(3).
• Support for specifying boot device in vm.conf(5).
• OpenSMTPD 6.6.0
• LibreSSL 3.0.X
• API and Documentation Enhancements
• Completed the port of RSA_METHOD accessors from the OpenSSL 1.1 API.
• Documented undescribed options and removed unfunctional options description in openssl(1) manual.
• OpenSSH 8.0

Project Trident 12-U5 update now available

This is the fifth general package update to the STABLE release repository based upon TrueOS 12-Stable.

• Package changes from Stable 12-U4

• Package Summary

  • New Packages: 20
  • Deleted Packages: 24
  • Updated Packages: 279

• New Packages (20)

  • artemis (biology/artemis) : 17.0.1.11
  • catesc (games/catesc) : 0.6
  • dmlc-core (devel/dmlc-core) : 0.3.105
  • go-wtf (sysutils/go-wtf) : 0.20.0_1
  • instead (games/instead) : 3.3.0_1
  • lidarr (net-p2p/lidarr) : 0.6.2.883
  • minerbold (games/minerbold) : 1.4
  • onnx (math/onnx) : 1.5.0
  • openzwave-devel (comms/openzwave-devel) : 1.6.897
  • polkit-qt-1 (sysutils/polkit-qt) : 0.113.0_8
  • py36-traitsui (graphics/py-traitsui) : 6.1.2
  • rubygem-aws-sigv2 (devel/rubygem-aws-sigv2) : 1.0.1
  • rubygem-default_value_for32 (devel/rubygem-default_value_for32) : 3.2.0
  • rubygem-ffi110 (devel/rubygem-ffi110) : 1.10.0
  • rubygem-zeitwerk (devel/rubygem-zeitwerk) : 2.1.9
  • sems (net/sems) : 1.7.0.g20190822
  • skypat (devel/skypat) : 3.1.1
  • tvm (math/tvm) : 0.4.1440
  • vavoom (games/vavoom) : 1.33_15
  • vavoom-extras (games/vavoom-extras) : 1.30_4

• Deleted Packages (24)

  • geeqie (graphics/geeqie) : Unknown reason
  • iriverter (multimedia/iriverter) : Unknown reason
  • kde5 (x11/kde5) : Unknown reason
  • kicad-doc (cad/kicad-doc) : Unknown reason
  • os-nozfs-buildworld (os/buildworld) : Unknown reason
  • os-nozfs-userland (os/userland) : Unknown reason
  • os-nozfs-userland-base (os/userland-base) : Unknown reason
  • os-nozfs-userland-base-bootstrap (os/userland-base-bootstrap) : Unknown reason
  • os-nozfs-userland-bin (os/userland-bin) : Unknown reason
  • os-nozfs-userland-boot (os/userland-boot) : Unknown reason
  • os-nozfs-userland-conf (os/userland-conf) : Unknown reason
  • os-nozfs-userland-debug (os/userland-debug) : Unknown reason
  • os-nozfs-userland-devtools (os/userland-devtools) : Unknown reason
  • os-nozfs-userland-docs (os/userland-docs) : Unknown reason
  • os-nozfs-userland-lib (os/userland-lib) : Unknown reason
  • os-nozfs-userland-lib32 (os/userland-lib32) : Unknown reason
  • os-nozfs-userland-lib32-development (os/userland-lib32-development) : Unknown reason
  • os-nozfs-userland-rescue (os/userland-rescue) : Unknown reason
  • os-nozfs-userland-sbin (os/userland-sbin) : Unknown reason
  • os-nozfs-userland-tests (os/userland-tests) : Unknown reason
  • photoprint (print/photoprint) : Unknown reason
  • plasma5-plasma (x11/plasma5-plasma) : Unknown reason
  • polkit-qt5 (sysutils/polkit-qt) : Unknown reason
  • secpanel (security/secpanel) : Unknown reason

Beastie Bits

• DragonFlyBSD - msdosfs updates
• Stand out as a speaker
• Not a review of the 7th Gen X1 Carbon
• FreeBSD Meets Linux At The Open Source Summit
• QEMU VM Escape
• Porting wine to amd64 on NetBSD, third evaluation report.
• OpenBSD disabled DoH by default in Firefox

Feedback/Questions

• Reinis - GELI with UEFI
• Mason - Beeping

[CHVT feedback]
DJ - Feedback
Ben - chvt
Harri - Marc's chvt question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

vBSDcon Recap

Allan and Benedict attended vBSDcon 2019, which ended last week.

It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks.

The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails.

• If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week

John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract and the recent commit we covered in episode 313.

Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs.

David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD.

Shawn Webb followed with his overview talk about the “State of the Hardened Union”.

Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality.

Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale.

People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts.

Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”.

Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads.

“By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms.

Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”.

Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”.

A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night.

We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees!

humungus - an hg server

• Features
  • View changes, files, changesets, etc. Some syntax highlighting.
  • Read only.
  • Serves multiple repositories.
  • Allows cloning via the obvious URL. Supports go get.
  • Serves files for downloads.
  • Online documentation via mandoc.
  • Terminal based admin interface.

News Roundup

OpenBSD on fan-less Tuxedo InfinityBook 14″ v2.

The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.).

I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop.

The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta.

• See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader

Unix at 50: How the OS that powered smartphones started from failure

Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey.

It was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term.

Trying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.”

Within the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for.

Still, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote.

Cancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals.

• Some of Allan’s favourite excerpts:

In the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor.

And so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics.

With the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget.

McIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it.

It was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.”

Eventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document.

Of course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles.

In August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task.

Thompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics.

By the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer.

It wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications.

The computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy.

The rest has quite literally made tech history.

• See the link for the rest of the article

How to configure a network dump in FreeBSD?

A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump.

So, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0.

Now, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client.

• See the link for the rest of the article

Beastie Bits

• Sudo Mastery 2nd edition is not out
• Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development
• soso
• GregKH - OpenBSD was right
• Game of Trees

Feedback/Questions

• BostJan - Another Question
• Tom - PF
• JohnnyK - Changing VT without keys

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

What has to happen with Unix virtual memory when you have no swap space

Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine):

Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...]

I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM.

To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program.

• See link for the rest of the article

Dsynth details on Dragonfly

First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load.

Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes.

• dsynth

DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's).

The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied.

• The intent is to make dsynth compatible with synth's configuration files and directory structure.

• This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts).

• dsynth code

News Roundup

Instant Workstation

Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly.

So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager.

The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think.

In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around.

• Here is the script in my GitHub repository with notes-for-myself.

New Servers, new Tech

Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure.

• New servers in the colo, monster is being retired

We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency.

With SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner.

Monster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be.

But we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot.

I would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly.

Experimenting with streaming setups on NetBSD

Ever since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward.

Capturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK.

My laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen.

NetBSD Made Progress Thanks To GSoC In Its March Towards Steam Support

Ultimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support.

Student developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer.

These interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer.

Those curious about this DRM ioctl GSoC project can learn more from the NetBSD blog. NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux.

Beastie Bits

• FreeBSD in Wellington?
• FreeBSD on GFE
• Clarification
• Distrotest.net now with BSDs
• Lecture: Anykernels meet fuzzing NetBSD
• Sun Microsystems business plan from 1982 [pdf]

Feedback/Questions

• Alan - Questions
• Rodriguez - Feedback and a question
• Jeff - OpenZFS follow-up, FreeBSD Adventures

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

OpenBSD on the Thinkpad X1 Carbon 7th Gen

Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery.
The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life.
Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile.
On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck.

• See link for the rest of the article

How To Install FreeBSD On A MacBook 1,1 or 2,1

• FreeBSD Setup For MacBook 1,1 and 2,1

FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12.

• Installing

FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is:

• A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work.
• A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks.
• An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks.

• Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots.

  • See link for the rest of the guide

News Roundup

Patch for review: Kernel portion of in-kernel TLS (KTLS)

One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later.

• https://reviews.freebsd.org/D21277

DragonFly Boot Enviroments

This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl.

• See link for the rest of the details

Project Trident Updates

• 19.08 Available

This is a general package update to the CURRENT release repository based upon TrueOS 19.08.
Legacy boot ISO functional again
This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems.

• PACKAGE CHANGES FROM 19.07-U1

  • New Packages: 154
  • Deleted Packages: 394
  • Updated Packages: 4926

• 12-U3 Available

This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable.

• PACKAGE CHANGES FROM STABLE 12-U2
  • New Packages: 105
  • Deleted Packages: 386
  • Updated Packages: 1046

vBSDcon

• vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. ***

Beastie Bits

• The next NYCBUG meeting will be Sept 4 @ 18:45

Feedback/Questions

• Tom - Questions
• Michael - dfbeadm
• Bostjan - Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

The UNIX Philosophy in 2019

Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973:

• We write programs that do one thing and do it well
• We write programs to work together
• And we write programs that handle text streams, because that is a universal interface

Why Use Package Managers?

Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case.

Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed.

In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few.

Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this?

The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science.

Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically.

News Roundup

Touchpad, Interrupted

For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it.

It's been a long journey and it's a technical tale, but here it is.

Porting wine to amd64 on NetBSD, second evaluation report

• Summary

Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LD_LIBRARY_PATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity.

Enhancing Syzkaller Support for NetBSD, Part 2

As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period.

You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd

July Update: All about the Pinebook Pro

"So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available."

Killing a process and all of its descendants

Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned:

Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.

Sending signals to all processes in a session is not trivial with syscalls.

Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children.

The answer to the “What happens with orphaned process groups” question is not trivial.

Fast Software, the Best Software

I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness.

Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why.

But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools.

A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.)

Beastie Bits

• Register for vBSDCon 2019, Sept 5-7 in Reston VA
• Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway

Feedback/Questions

• Paulo - FreeNAS Question
• Marc - Changing VT without function keys?
• Caleb - Patch, update, and upgrade management

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

NetBSD 9.0 release process has started

If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on:

• New AArch64 architecture support:
  • Symmetric and asymmetrical multiprocessing support (aka big.LITTLE)
  • Support for running 32-bit binaries
  • UEFI and ACPI support
  • Support for SBSA/SBBR (server-class) hardware.
• The FDT-ization of many ARM boards:
  • the 32-bit GENERIC kernel lists 129 different DTS configurations
  • the 64-bit GENERIC64 kernel lists 74 different DTS configurations
  • All supported by a single kernel, without requiring per-board configuration.
• Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices.
• ZFS has been updated to a modern version and seen many bugfixes.
• New hardware-accelerated virtualization via NVMM.
• NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default.
• NVMe performance improvements
• Optional kernel ASLR support, and partial kernel ASLR for the default configuration.
• Kernel sanitizers:
  • KLEAK, detecting memory leaks
  • KASAN, detecting memory overruns
  • KUBSAN, detecting undefined behaviour
  • These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed.
• The removal of outdated networking components such as ISDN and all of its drivers
• The installer is now capable of performing GPT UEFI installations.
• Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding.
• Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU.

We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find.

• Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/

xargs wtf

xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too.
I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better.
xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else.
It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem:
This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept.

News Roundup

PkgSrc: A Tale of Two Spellcheckers

This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study.
The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole.

Adapting TriforceAFL for NetBSD, Part 2

I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation.
For work done during the first coding period, check out this post.

• Summary > So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation. > Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far!

Exploiting a no-name freebsd kernel vulnerability

• A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions. > A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory.

[Allan and Benedicts Conference Gear Breakdown]

• Benedict’s Gear:

  GlocalMe G3 Mobile Travel HotSpot and Powerbank
  Mogics Power Bagel
  Charby Sense Power Cable

• Allan’s Gear:

  Huawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro
  AOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries
  All my devices charge from USB-C, so that is great
  More USB thumb drives than strictly necessary
  My Lenovo X270 laptop running FreeBSD 13-current
  My 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work

Beastie Bits

• Replacing the Unix tradition (Warning may be rage inducing)
• Installing OpenBSD over remote serial on the AtomicPI
• Zen 2 and DragonFly
• Improve Docking on FreeBSD
• Register for vBSDCon 2019, Sept 5-7 in Reston VA. Early bird ends August 15th.
• Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway

Feedback/Questions

• JT - Congrats

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

OPNsense 19.7.1

We do not wish to keep you from enjoying your summer time, but this
is a recommended security update enriched with reliability fixes for the
new 19.7 series. Of special note are performance improvements as well
as a fix for a longstanding NAT before IPsec limitation.

Full patch notes:

• system: do not create automatic copies of existing gateways
• system: do not translate empty tunables descriptions
• system: remove unwanted form action tags
• system: do not include Syslog-ng in rc.freebsd handler
• system: fix manual system log stop/start/restart
• system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead
• system: allow curl-based downloads to use both trusted and local authorities
• system: fix group privilege print and correctly redirect after edit
• system: use cached address list in referrer check
• system: fix Syslog-ng search stats
• firewall: HTML-escape dynamic entries to display aliases
• firewall: display correct IP version in automatic rules
• firewall: fix a warning while reading empty outbound rules configuration
• firewall: skip illegal log lines in live log
• interfaces: performance improvements for configurations with hundreds of interfaces
• reporting: performance improvements for Python 3 NetFlow aggregator rewrite
• dhcp: move advanced router advertisement options to correct config section
• ipsec: replace global array access with function to ensure side-effect free boot
• ipsec: change DPD action on start to "dpdaction = restart"
• ipsec: remove already default "dpdaction = none" if not set
• ipsec: use interface IP address in local ID when doing NAT before IPsec
• web proxy: fix database reset for Squid 4 by replacing use of ssl_crtd with security_file_certgen
• plugins: os-acme-client 1.24[1]
• plugins: os-bind 1.6[2]
• plugins: os-dnscrypt-proxy 1.5[3]
• plugins: os-frr now restricts characters BGP prefix-list and route-maps[4]
• plugins: os-google-cloud-sdk 1.0[5]
• ports: curl 7.65.3[6]
• ports: monit 5.26.0[7]
• ports: openssh 8.0p1[8]
• ports: php 7.2.20[9]
• ports: python 3.7.4[10]
• ports: sqlite 3.29.0[11]
• ports: squid 4.8[12]

Stay safe and hydrated, Your OPNsense team

ZFS on Linux still has annoying issues with ARC size

One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there.

Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.)

This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case).

The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all).

News Roundup

Hammer2 is now default

commit a49112761c919d42d405ec10252eb0553662c824
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Mon Jun 10 17:53:46 2019 -0700

    installer - Default to HAMMER2

    * Change the installer default from HAMMER1 to HAMMER2.

    * Adjust the nrelease build to print the location of the image files
      when it finishes.

Summary of changes:
 nrelease/Makefile                          |  2 +-
 usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++----------
 2 files changed, 11 insertions(+), 11 deletions(-)

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824

NetBSD audio – an application perspective

NetBSD audio – an application perspective ... or, "doing it natively, because we can"

• audio options for NetBSD in pkgsrc

  • Use NetBSD native audio (sun audio/audioio.h)
  • Or OSS emulation layer: Basically a wrapper around sun audio in the kernel. Incomplete and old version, but works for simple stuff

• Many many abstraction layers available:

  • OpenAL-Soft
  • alsa-lib (config file required)
  • libao, GStreamer (plugins!)
  • PortAudio, SDL
  • PulseAudio, JACK
  • ... lots more!? some obsolete stuff (esd, nas?)

• Advantages of using NetBSD audio directly

  • Low latency, low CPU usage: Abstraction layers differ in latency (SDL2 vs ALSA/OpenAL)
  • Query device information: Is /dev/audio1 a USB microphone or another sound card?
  • Avoid bugs from excessive layering
  • Nice API, well documented: [nia note: I had no idea how to write audio code. I read a man page and now I do.]
  • Your code might work on illumos too

• [nia note: SDL2 seems very sensitive to the blk_ms sysctl being high or low, with other implementations there seems to be a less noticable difference. I don't know why.]

New FreeNAS Mini

Two new FreeNAS Mini systems join the very popular FreeNAS Mini and Mini XL:

FreeNAS Mini XL+: This powerful 10 Bay platform (8x 3.5” and 1x 2.5” hot-swap, 1x 2.5” internal) includes the latest, compact server technology and provides dual 10GbE ports, 8 CPU cores and 32 GB RAM for high performance workgroups. The Mini XL+ scales beyond 100TB and is ideal for very demanding applications, including hosting virtual machines and multimedia editing. Starting at $1499, the Mini XL+ configured with cache SSD and 80 TB capacity is $4299, and consumes about 100 Watts.

FreeNAS Mini E: This cost-effective 4 Bay platform provides the resources required for SOHO use with quad GbE ports and 8 GB of RAM. The Mini E is ideal for file sharing, streaming and transcoding video at 1080p. Starting at $749, the Mini E configured with 8 TB capacity is $999, and consumes about 36 Watts.

Beastie Bits

• Welcome to NetBSD 9.99.1!
• Berkeley smorgasbord — part II
• dtracing postgres
• Project Trident 19.07-U1 now available
• Need a Secure Operating System? Take a Look at OpenBSD

Feedback/Questions

• Jeff - OpenZFS Port Testing Feedback
• Malcolm - Best Practices for Custom Ports
• Michael - Little Correction

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

DragonFlyBSD Project Update - colo upgrade, future trends

For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone.

The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system.

Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while.

Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get.

The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time.

Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going.

Resuming ZFS send

One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot?

For a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off.

In this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice.

News Roundup

Realtime bandwidth terminal graph visualization

If for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal.

The following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5.

fixing telnet fixes

There’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations.

1. The first line is indented with spaces while the others use tabs.

2. The correct type for string length is size_t not unsigned int.

3. sizeof(char) is always one. There’s no need to multiply by it.

4. If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.)

5. Return value of malloc doesn’t need to be cast. In fact, should not be, lest you disguise a warning.

6. Return value of malloc is not checked for NULL.

7. No reason to cast cp to char * when passing to snprintf. It already is that type. And if it weren’t, what are you doing?

8. The whole operation could be simplified by using asprintf.

9. Although unlikely (probably impossible here, but more generally), adding the two source lengths together can overflow, resulting in truncation with an unchecked snprintf call. asprintf avoids this failure case.

A Chapter from the FBI’s History with OpenBSD and an OpenSSH Vuln

Earlier this year I FOIAed the FBI for details on allegations of backdoor installed in the IPSEC stack in 2010, originally discussed by OpenBSD devs (https://marc.info/?l=openbsd-tech&m=129236621626462 …) Today, I got an interesting but unexpected responsive record:

• Freedom of Information Act: FBI: OpenBSD
• GitHub Repo

Beastie Bits

• “Sudo Mastery, 2nd Edition” open for tech review
• FreeBSD Journal: FreeBSD for Makers
• OpenBSD and NetBSD machines at Open Source Conference 2019 Nagoya
• FreeBSD 12.0: WINE Gaming
• Introduction to the Structure and Interpretation of TNF (The NetBSD Foundation)
• vBSDcon speakers announced

Feedback/Questions

• Pat - NYCBug Aug 7th
• Tyler - SSH keys vs password
• Lars - Tor-Talk

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Replacing a (silently) failing disk in a ZFS pool

Maybe I can’t read, but I have the feeling that official documentations explain every single corner case for a given tool, except the one you will actually need. My today’s struggle: replacing a disk within a FreeBSD ZFS pool.
What? there’s a shitton of docs on this topic! Are you stupid?
I don’t know, maybe. Yet none covered the process in a simple, straight and complete manner.

OPNsense 19.7 RC1 released

Hi there,
For four and a half years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
We thank all of you for helping test, shape and contribute to the project! We know it would not be the same without you.
Download links, an installation guide[1] and the checksums for the images can be found below as well.

News Roundup

Implementation of DRM ioctl Support for NetBSD kernel

• What is DRM ioctl ?

Ioctls are input/output control system calls and DRM stands for direct rendering manager The DRM layer provides several services to graphics drivers, many of them driven by the application interfaces it provides through libdrm, the library that wraps most of the DRM ioctls. These include vblank event handling, memory management, output management, framebuffer management, command submission & fencing, suspend/resume support, and DMA services.

• Native DRM ioctl calls

NetBSD was able to make native DRM ioctl calls with hardware rendering once xorg and proper mesa packages where installed. We used the glxinfo and glxgears applications to test this out.

High quality / low latency VOIP server with umurmur/Mumble on OpenBSD

Discord users keep telling about their so called discord server, which is not dedicated to them at all. And Discord has a very bad quality and a lot of voice distorsion.
Why not run your very own mumble server with high voice quality and low latency and privacy respect? This is very easy to setup on OpenBSD!
Mumble is an open source voip client, it has a client named Mumble (available on various operating system) and at least Android, the server part is murmur but there is a lightweight server named umurmur. People authentication is done through certificate generated locally and automatically accepted on a server, and the certificate get associated with a nickname. Nobody can pick the same nickname as another person if it’s not the same certificate.

TMWL June’19 — JS Fetch API, scheduling in Spring, thoughts on Unix

• Unix — going back to the roots

From time to time, I like to review my knowledge in a certain area, even when I feel like I know a lot about it already. I go back to the basics and read tutorials, manuals, books or watch interesting videos.
I’ve been using macOS for a couple of years now, previously being a linux user for some (relatively short) time. Both these operating systems have a common ancestor — Unix. While I’m definitely not an expert, I feel quite comfortable using linux & macOS — I understand the concepts behind the system architecture, know a lot of command line tools & navigate through the shell without a hassle. So-called unix philosophy is also close to my heart. I always feel like there’s more I could squeeze out of it.
Recently, I found that book titled “Unix for dummies, 5th edition” which was published back in… 2004. Feels literally like AGES in the computer-related world. However, it was a great shot — the book starts with the basics, providing some brief history of Unix and how it came to life. It talks a lot about the structure of the system and where certain pieces fit (eg. “standard” set of tools), and how to understand permissions and work with files & directories. There’s even a whole chapter about shell-based text editors like Vi and Emacs! Despite the fact that I am familiar with most of these, I could still find some interesting pieces & tools that I either knew existed (but never had a chance to use), or even haven’t ever heard of. And almost all of these are still valid in the modern “incarnations” of Unix’s descendants: Linux and macOS.
The book also talks about networking, surfing the web & working with email. It’s cute to see pictures of those old browsers rendering “ancient” Internet websites, but hey — this is how it looked like no more than fifteen years ago!
I can really recommend this book to anyone working on modern macOS or Linux — you will certainly find some interesting pieces. Especially if you like to go back to the roots from time to time as I do!

ThePDP-7 Where Unix Began

In preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine.
V0 Unix could run on only one of the PDP-7s. Of the 99 PDP-7s produced, only two had disks. Serial number 14 had an RA01 listed, presumably a disk, though of a different type. In addition to the PDP-7 being obsolete in 1970, no other PDP-7 could run Unix, limiting its appeal outside of Bell Labs. By porting Unix to the PDP-11 in 1970, the group ensured Unix would live on into the future. The PDP-9 and PDP-15 were both upgrades of the PDP-7, so to be fair, PDP-7 Unix did have a natural upgrade path (the PDP-11 out sold the 18 bit systems though ~600,000 to ~1000). Ken Thompson reports in a private email that there were 2 PDP-9s and 1 PDP-15 at Bell Labs that could run a version of the PDP-7 Unix, though those machines were viewed as born obsolete.

LLDB: watchpoints, XSTATE in ptrace() and core dumps

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types and fix compat32 issues. You can read more about that in my May 2019 report.
In June, I have finally finished the remaining ptrace() work for xstate and got it merged both on NetBSD and LLDB end (meaning it's going to make it into NetBSD 9). I have also worked on debug register support in LLDB, effectively fixing watchpoint support. Once again I had to fight some upstream regressions.

Beastie Bits

• Project Trident 19.07 Available
• A list of names from "Cold Blood" -- Any familiar?
• fern: a curses-based mastodon client modeled off usenet news readers & pine, with an emphasis on getting to 'timeline zero'
• OpenBSD Community goes Platinum for 2019!
• tcp keepalive and dports on DragonFly

Feedback/Questions

• Patrick - OpenZFS/ZoL Module from Ports
• Brad - Services not starting
• Simon - Feedback

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FreeBSD 11.3-RELEASE Announcement

The FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch.

• Some of the highlights:
  • The clang, llvm, lld, lldb, and compiler-rt utilities as well as libc++ have been updated to upstream version 8.0.0.
  • The ELF Tool Chain has been updated to version r3614.
  • OpenSSL has been updated to version 1.0.2s.
  • The ZFS filesystem has been updated to implement parallel mounting.
  • The loader(8) has been updated to extend geli(8) support to all architectures.
  • The pkg(8) utility has been updated to version 1.10.5.
  • The KDE desktop environment has been updated to version 5.15.3.
  • The GNOME desktop environment has been updated to version 3.28.
  • The kernel will now log the jail(8) ID when logging a process exit.
  • Several feature additions and updates to userland applications.
  • Several network driver firmware updates.
  • Warnings for features deprecated in future releases will now be printed on all FreeBSD versions.
  • Warnings have been added for IPSec algorithms deprecated in RFC 8221.
  • Deprecation warnings have been added for weaker algorithms when creating geli(8) providers.
  • And more...

OpenBSD Is Now My Workstation

Why OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear).

I will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way.

Disclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while.

• A Bit About Me and OpenBSD

I’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices.

I just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most.

News Roundup

Write your own fuzzer for NetBSD kernel! [Part 1]

• How Fuzzing works? The dummy Fuzzer.

The easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective.

The simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter.

• Coverage and Fuzzing

What can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done.

However, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour.

Additional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags.

For open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution

vBSDcon - CFP - Call for Papers ends July 19th

You can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019

The talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal.

Possible topics include: How we manage a giant installation with respect to handling spam, snd/or sysadmin, and/or networking, Cool new stuff in BSD, Tell us about your project which runs on BSD.

Both users and developers are encouraged to share their experiences.

Exploiting FreeBSD-SA-19:02.fd

In February 2019 the FreeBSD project issued an advisory about a possible vulnerability in the handling of file descriptors. UNIX-like systems such as FreeBSD allow to send file descriptors to other processes via UNIX-domain sockets. This can for example be used to pass file access privileges to the receiving process.

Inside the kernel, file descriptors are used to indirectly reference a C struct which stores the relevant information about the file object. This could for instance include a reference to a vnode which describes the file for the file system, the file type, or the access privileges.

What really happens if a UNIX-domain socket is used to send a file descriptor to another process is that for the receiving process, inside the kernel a reference to this struct is created. As the new file descriptor is a reference to the same file object, all information is inherited. For instance, this can allow to give another process write access to a file on the drive even if the process owner is normally not able to open the file writable.

The advisory describes that FreeBSD 12.0 introduced a bug in this mechanism. As the file descriptor information is sent via a socket, the sender and the receiver have to allocate buffers for the procedure. If the receiving buffer is not large enough, the FreeBSD kernel attempts to close the received file descriptors to prevent a leak of these to the sender. However, while the responsible function closes the file descriptor, it fails to release the reference from the file descriptor to the file object. This could cause the reference counter to wrap.

The advisory further states that the impact of this bug is possibly a local privilege escalation to gain root privileges or a jail escape. However, no proof-of-concept was provided by the advisory authors.

• In the next section, the bug itself is analyzed to make a statement about the bug class and a guess about a possible exploitation primitive.
• After that, the bug trigger is addressed.
• It follows a discussion of three imaginable exploitation strategies - including a discussion of why two of these approaches failed.
• In the section before last, the working exploit primitive is discussed. It introduces a (at least to the author’s knowledge) new exploitation technique for these kind of vulnerabilities in FreeBSD. The stabilization of the exploit is addressed, too.
• The last section wraps everything up in a conclusion and points out further steps and challenges.

The privilege escalation is now a piece of cake thanks to a technique used by kingcope, who published a FreeBSD root exploit in 2005, which writes to the file /etc/libmap.conf. This configuration file can be used to hook the loading of dynamic libraries if a program is started. The exploit therefore creates a dynamic library, which copies /bin/sh to another file and sets the suid-bit for the copy. The hooked library is libutil, which is for instance called by su. Therefore, a call to su by the user will afterwards result in a suid copy of /bin/sh.

Streaming to Twitch using OpenBSD

• Introduction

If you ever wanted to make a twitch stream from your OpenBSD system, this is now possible, thanks to OpenBSD developer thfr@ who made a wrapper named fauxstream using ffmpeg with relevant parameters.

The setup is quite easy, it only requires a few steps and searching on Twitch website two informations, hopefully, to ease the process, I found the links for you.

You will need to make an account on twitch, get your api key (a long string of characters) which should stay secret because it allow anyone having it to stream on your account.

• These same techniques should work for Twitch, YouTube Live, Periscope, Facebook, etc, including the live streaming service ScaleEngine provides free to BSD user groups.
• There is also an open source application called ‘OBS’ or Open Broadcaster Studio. It is in FreeBSD ports and should work on all of the other BSDs as well. It has a GUI and supports compositing and green screening. We use it heavily at ScaleEngine and it is also used at JupiterBroadcasting in place of WireCast, a $1000-per-copy commercial application.

Beastie Bits

• Portland BSD Pizza Night - 2019-07-25 19:00 - Rudy's Gourmet Pizza
• KnoxBUG - Michael W. Lucas : Twenty Years in Jail
• Ohio Linuxfest - CFP - Closes August 17th
• My college (NYU Tandon) is moving their CS department and I saw this on a shelf being moved
• 3 different ways of dumping hex contents of a file

Feedback/Questions

• Sebastian - ZFS setup toward ESXi
• Christopher - Questions
• Ser - Bhyve and Microsoft SQL

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Polprog's Am5x86 based retro UNIX build log

I have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates!

I began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically

There is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled.

Originally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue...

However the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good.

Having used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet.

• See the article for the rest of the writeup

Setting up services in a FreeNAS Jail

This piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands.

This example shows creating a jail, installing an Apache web server, and setting up a simple web page.

NOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment.

News Roundup

First taste of DragonflyBSD

Last week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me.

I mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK.

Furthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully.

Streaming Netflix on NetBSD

Here's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm.

Heads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!!

“Sudo Mastery 2nd Edition” cover art reveal

I’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.)

But the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn.

NetBSD on the last G4 Mac mini

I'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it on a MIPS-based Cobalt RaQ 2 and an HP Jornada 690. I think NetBSD is a better match for smaller or underpowered systems than current-day Linux, and is fairly easy to harden and keep secure even though none of these systems are exposed to the outside world.

Recently I had a need to set up a bridge system that would be fast enough to connect two networks and I happened to have two of the "secret" last-of-the-line 1.5GHz G4 Mac minis sitting on the shelf doing nothing. Yes, they're probably outclassed by later Raspberry Pi models, but I don't have to buy anything and I like putting old hardware to good use.

Hammer vs Hammer2

With the newly released DragonFlyBSD 5.6 there are improvements to its original HAMMER2 file-system to the extent that it's now selected by its installer as the default file-system choice for new installations. Curious how the performance now compares between HAMMER and HAMMER2, here are some initial benchmarks on an NVMe solid-state drive using DragonFlyBSD 5.6.0.

With a 120GB Toshiba NVMe SSD on an Intel Core i7 8700K system, I ran some benchmarks of DragonFlyBSD 5.6.0 freshly installed with HAMMER2 and then again when returning to the original HAMMER file-system that remains available via its installer. No other changes were made to the setup during testing.

And then for the more synthetic workloads it was just a mix. But overall HAMMER2 was performing well during the initial testing and great to see it continuing to offer noticeable leads in real-world workloads compared to the aging HAMMER file-system. HAMMER2 also offers better clustering, online deduplication, snapshots, compression, encryption, and many other modern file-system features.

Beastie Bits

• Unix CLI relational database
• The TTY demystified
• Ranger, a console file manager with VI keybindings
• Some Unix Humor
• OpenBSD -import vulkan-loader for Vulkan API support
• FreeBSD ZFS without drives

Feedback/Questions

• Moritz - ARM Builds
• Dave - Videos
• Chris - Raspberry Pi4

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Website protection with OPNsense

• with nginx plugin OPNsense become a strong full featured Web Application Firewall (WAF)

The OPNsense security platform can help you to protect your network and your webservers with the nginx plugin addition.
In old days, install an open source firewall was a very trick task, but today it can be done with few clicks (or key strokes). In this article I'll not describe the detailed OPNsense installation process, but you can watch this video that was extracted from my OPNsense course available in Udemy. The video is in portuguese language, but with the translation CC Youtube feature you may be able to follow it without problems (if you don't are a portuguese speaker ofcourse) :-)

• See the article for the rest of the writeup

FreeBSD Support Pull Request against the ZFS-on-Linux repo

• This pull request integrates the sysutils/openzfs port’s sources into the upstream ZoL repo > Adding FreeBSD support to ZoL will make it easier to move changes back and forth between FreeBSD and Linux > Refactor tree to separate out Linux and FreeBSD specific code > import FreeBSD's SPL > add ifdefs in common code where it made more sense to do so than duplicate the code in separate files > Adapted ZFS Test Suite to run on FreeBSD and all tests that pass on ZoL passing on ZoF
• The plan to officially rename the common repo from ZFSonLinux to OpenZFS was announced at the ZFS Leadership Meeting on June 25th
• Video of Leadership Meeting
• Meeting Agenda and Notes
• This will allow improvements made on one OS to be made available more easily (and more quickly) to the other platforms
• For example, mav@’s recent work:
• Add wakeup_any(), cheaper version of wakeup_one() for taskqueue(9) > As result, on 72-core Xeon v4 machine sequential ZFS write to 12 ZVOLs with 16KB block size spend 34% less time in wakeup_any() and descendants then it was spending in wakeup_one(), and total write throughput increased by ~10% with the same as before CPU usage.

News Roundup

Episode 5 Notes - How much has UNIX changed?

UNIX-like systems have dominated computing for decades, and with the rise of the internet and mobile devices their reach has become even larger. True, most systems now use more modern OSs like Linux, but how much has the UNIX-like landscape changed since the early days?
So, my question was this: how close is a modern *NIX userland to some of the earliest UNIX releases? To do this I'm going to compare a few key points of a modern Linux system with the earliest UNIX documentation I can get my hands on. The doc I am going to be covering(https://www.tuhs.org/Archive/Distributions/Research/Dennis_v1/UNIX_ProgrammersManual_Nov71.pdf) is from November 1971, predating v1 of the system.
I think the best place to start this comparison is to look at one of the highest-profile parts of the OS, that being the file system. Under the hood modern EXT file systems are completely different from the early UNIX file systems. However, they are still presented in basically the same way, as a heirerarchicat structure of directories with device files. So paths still look identical, and navigating the file system still functions the same. Often used commands like ls, cp, mv, du, and df function the same. So are mount and umount. But, there are some key differences. For instance, cd didn't exist, yet instead chdir filled its place. Also, chmod is somewhat different. Instead of the usual 3-digit octal codes for permissions, this older version only uses 2 digits. Really, that difference is due to the underlying file system using a different permission set than modern systems. For the most part, all the file handling is actually pretty close to a Linux system from 2019.

• See the article for the rest of the writeup

Porting Wine to amd64 on NetBSD

I have been working on porting Wine to amd64 on NetBSD as a GSoC 2019 project. Wine is a compatibility layer which allows running Microsoft Windows applications on POSIX-complaint operating systems. This report provides an overview of the progress of the project during the first coding period.
Initially, when I started working on getting Wine-4.4 to build and run on NetBSD i386 the primary issue that I faced was Wine displaying black windows instead of UI, and this applied to any graphical program I tried running with Wine.
I suspected it , as it is related to graphics, to be an issue with the graphics driver or Xorg. Subsequently, I tried building modular Xorg, and I tried running Wine on it only to realize that Xorg being modular didn't affect it in the least. After having tried a couple of configurations, I realized that trying to hazard out every other probability is going to take an awful lot of time that I didn't have. This motivated me to bisect the repo using git, and find the first version of Wine which failed on NetBSD.

• See the article for the rest of the writeup

FreeBSD Enterprise 1 PB Storage

Today FreeBSD operating system turns 26 years old. 19 June is an International FreeBSD Day. This is why I got something special today :). How about using FreeBSD as an Enterprise Storage solution on real hardware? This where FreeBSD shines with all its storage features ZFS included.
Today I will show you how I have built so called Enterprise Storage based on FreeBSD system along with more then 1 PB (Petabyte) of raw capacity.
This project is different. How much storage space can you squeeze from a single 4U system? It turns out a lot! Definitely more then 1 PB (1024 TB) of raw storage space.

• See the article for the rest of the writeup

The death watch for the X Window System (aka X11) has probably started

Once we are done with this we expect X.org to go into hard maintenance mode fairly quickly. The reality is that X.org is basically maintained by us and thus once we stop paying attention to it there is unlikely to be any major new releases coming out and there might even be some bitrot setting in over time. We will keep an eye on it as we will want to ensure X.org stays supportable until the end of the RHEL8 lifecycle at a minimum, but let this be a friendly notice for everyone who rely the work we do maintaining the Linux graphics stack, get onto Wayland, that is where the future is.
I have no idea how true this is about X.org X server maintenance, either now or in the future, but I definitely think it's a sign that developers have started saying this. If Gnome developers feel that X.org is going to be in hard maintenance mode almost immediately, they're probably pretty likely to also put the Gnome code that deals with X into hard maintenance mode. And public Gnome statements about this (and public action or lack of it) provide implicit support for KDE and any other desktop to move in this direction if they want to (and probably create some pressure to do so). I've known that Wayland was the future for some time, but I would still like it to not arrive any time soon.

Beastie Bits

• Porting NetBSD to Risc-V -- Video
• FreeBSD 11.3RC3 Available
• Open Source Could Be a Casualty of the Trade War
• Celebrate UNIX50 and SDF32
• doas environmental security

Feedback/Questions

• Matt - BSD or Older Hardware
• MJRodriguez - Some Playstation news
• Moritz - bhyve VT-x passthrough

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

DragonflyBSD 5.6 is out

• Version 5.6.0 released 17 June 2019

• Version 5.6.1 released 19 June 2019

• Big-ticket items

• Improved VM

  • Informal test results showing the changes from 5.4 to 5.6 are available.
  • Reduce stalls in the kernel vm_page_alloc() code (vm_page_list_find()).
  • Improve page allocation algorithm to avoid re-iterating the same queues as the search is widened.
  • Add a vm_page_hash*() API that allows the kernel to do heuristical lockless lookups of VM pages.
  • Change vm_hold() and vm_unhold() semantics to not require any spin-locks.
  • Change vm_page_wakeup() to not require any spin-locks.
  • Change wiring vm_page's no longer manipulates the queue the page is on, saving a lot of overhead. Instead, the page will be removed from its queue only if the pageout demon encounters it. This allows pages to enter and leave the buffer cache quickly.
  • Refactor the handling of fictitious pages.
  • Remove m->md.pv_list entirely. VM pages in mappings no longer allocate pv_entry's, saving an enormous amount of memory when multiple processes utilize large shared memory maps (e.g. postgres database cache).
  • Refactor vm_object shadowing, disconnecting the backing linkages from the vm_object itself and instead organizing the linkages in a new structure called vm_map_backing which hangs off the vm_map_entry.
  • pmap operations now iterate vm_map_backing structures (rather than spin-locked page lists based on the vm_page and pv_entry's), and will test/match operations against the PTE found in the pmap at the requisite location. This doubles VM fault performance on shared pages and reduces the locking overhead for fault and pmap operations.
  • Simplify the collapse code, removing most of the original code and replacing it with simpler per-vm_map_entry optimizations to limit the shadow depth.

• DRM

  • Major updates to the radeon and ttm (amd support code) drivers. We have not quite gotten the AMD support up to the more modern cards or Ryzen APUs yet, however.
  • Improve UEFI framebuffer support.
  • A major deadlock has been fixed in the radeon/ttm code.
  • Refactor the startup delay designed to avoid conflicts between the i915 driver initialization and X startup.
  • Add DRM_IOCTL_GET_PCIINFO to improve mesa/libdrm support.
  • Fix excessive wired memory build-ups.
  • Fix Linux/DragonFly PAGE_MASK confusion in the DRM code.
  • Fix idr_*() API bugs.

• HAMMER2

  • The filesystem sync code has been rewritten to significantly improve performance.
  • Sequential write performance also improved.
  • Add simple dependency tracking to prevent directory/file splits during create/rename/remove operations, for better consistency after a crash.
  • Refactor the snapshot code to reduce flush latency and to ensure a consistent snapshot.
  • Attempt to pipeline the flush code against the frontend, improving flush vs frontend write concurrency.
  • Improve umount operation.
  • Fix an allocator race that could lead to corruption.
  • Numerous other bugs fixed.
  • Improve verbosity of CHECK (CRC error) console messages.

OpenBSD Vulkan Support

Somewhat surprisingly, OpenBSD has added the Vulkan library and ICD loader support as their newest port.
This new graphics/vulkan-loader port provides the generic Vulkan library and ICD support that is the common code for Vulkan implementations on the system. This doesn't enable any Vulkan hardware drivers or provide something new not available elsewhere, but is rare seeing Vulkan work among the BSDs. There is also in ports the related components like the SPIR-V headers and tools, glsllang, and the Vulkan tools and validation layers.
This is of limited usefulness, at least for the time being considering OpenBSD like the other BSDs lag behind in their DRM kernel driver support that is ported over from the mainline Linux kernel tree but generally years behind the kernel upstream. Particularly with Vulkan, newer kernel releases are needed for some Vulkan features as well as achieving decent performance. The Vulkan drivers of relevance are the open-source Intel ANV Vulkan driver and Radeon RADV drivers, both of which are in Mesa though we haven't seen any testing results to know how well they would work if at all currently on OpenBSD, but they're at least in Mesa and obviously open-source.

• A note: The BSDs are no longer that far behind.
• FreeBSD 12.0 uses DRM from Linux 4.16 (April 2018), and the drm-devel port is based on Linux 5.0 (March 2019)
• OpenBSD -current as of April 2019 uses DRM from Linux 4.19.34 ***

News Roundup

Bad utmp implementations in glibc and freebsd

I recently released another version – 0.5.0 – of Dinit, the service manager / init system. There were a number of minor improvements, including to the build system (just running “make” or “gmake” should be enough on any of the systems which have a pre-defined configuration, no need to edit mconfig by hand), but the main features of the release were S6-compatible readiness notification, and support for updating the utmp database.
In other words, utmp is a record of who is currently logged in to the system (another file, “wtmp”, records all logins and logouts, as well as, potentially, certain system events such as reboots and time updates). This is a hint at the main motivation for having utmp support in Dinit – I wanted the “who” command to correctly report current logins (and I wanted boot time to be correctly recorded in the wtmp file).
I wondered: If the files consist of fixed-sized records, and are readable by regular users, how is consistency maintained? That is – how can a process ensure that, when it updates the database, it doesn’t conflict with another process also attempting to update the database at the same time? Similarly, how can a process reading an entry from the database be sure that it receives a consistent, full record and not a record which has been partially updated? (after all, POSIX allows that a write(2) call can return without having written all the requested bytes, and I’m not aware of Linux or any of the *BSDs documenting that this cannot happen for regular files). Clearly, some kind of locking is needed; a process that wants to write to or read from the database locks it first, performs its operation, and then unlocks the database. Once again, this happens under the hood, in the implementation of the getutent/pututline functions or their equivalents.
Then I wondered: if a user process is able to lock the utmp file, and this prevents updates, what’s to stop a user process from manually acquiring and then holding such a lock for a long – even practically infinite – duration? This would prevent the database from being updated, and would perhaps even prevent logins/logouts from completing. Unfortunately, the answer is – nothing; and yes, it is possible on different systems to prevent the database from being correctly updated or even to prevent all other users – including root – from logging in to the system.

• A good find
• On FreeBSD, even though write(2) can be asynchronous, once the write syscall returns, the data is in the buffer cache (or ARC), and any future read(2) will see that new data even if it has not yet been written to disk. ***

OpenSSH gets an update to protect against Side Channel attacks

Last week, Damien Miller, a Google security researcher, and one of the popular OpenSSH and OpenBSD developers announced an update to the existing OpenSSH code that can help protect against the side-channel attacks that leak sensitive data from computer’s memory. This protection, Miller says, will protect the private keys residing in the RAM against Spectre, Meltdown, Rowhammer, and the latest RAMBleed attack.
SSH private keys can be used by malicious threat actors to connect to remote servers without the need of a password. According to CSO, “The approach used by OpenSSH could be copied by other software projects to protect their own keys and secrets in memory”.
However, if the attacker is successful in extracting the data from a computer or server’s RAM, they will only obtain an encrypted version of an SSH private key, rather than the cleartext version.
In an email to OpenBSD, Miller writes, “this change encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large ‘prekey’ consisting of random data (currently 16KB).”

ZFS vs OpenZFS

You’ve probably heard us say a mix of “ZFS” and “OpenZFS” and an explanation is long-overdue.
From its inception, “ZFS” has referred to the “Zettabyte File System” developed at Sun Microsystems and published under the CDDL Open Source license in 2005 as part of the OpenSolaris operating system. ZFS was revolutionary for completely decoupling the file system from specialized storage hardware and even a specific computer platform. The portable nature and advanced features of ZFS led FreeBSD, Linux, and even Apple developers to start porting ZFS to their operating systems and by 2008, FreeBSD shipped with ZFS in the 7.0 release. For the first time, ZFS empowered users of any budget with enterprise-class scalability and data integrity and management features like checksumming, compression and snapshotting, and those features remain unrivaled at any price to this day. On any ZFS platform, administrators use the zpool and zfs utilities to configure and manage their storage devices and file systems respectively. Both commands employ a user-friendly syntax such as‘zfs create mypool/mydataset’ and I welcome you to watch the appropriately-titled webinar “Why we love ZFS & you should too” or try a completely-graphical ZFS experience with FreeNAS.
Oracle has steadily continued to develop its own proprietary branch of ZFS and Matt Ahrens points out that over 50% of the original OpenSolaris ZFS code has been replaced in OpenZFS with community contributions. This means that there are, sadly, two politically and technologically-incompatible branches of “ZFS” but fortunately, OpenZFS is orders of magnitude more popular thanks to its open nature. The two projects should be referred to as “Oracle ZFS” and “OpenZFS” to distinguish them as development efforts, but the user still types the ‘zfs’ command, which on FreeBSD relies on the ‘zfs.ko’ kernel module. My impression is that the terms of the CDDL license under which the OpenZFS branch of ZFS is published protects its users from any patent and trademark risks. Hopefully, this all helps you distinguish the OpenZFS project from the ZFS technology.

• There was further discussion of how the ZFSOnLinux repo will become the OpenZFS repo in the future once it also contains the bits to build on FreeBSD as well during the June 25th ZFS Leadership Meeting. The videos for all of the meetings are available here ***

Beastie Bits

• How to safely and portably close a file descriptor in a multithreaded process without running into problems with EINTR
• KnoxBug Meetup June 27th at 6pm
• BSD Pizza Night, June 27th at 7pm, Flying Pie Pizzeria, 3 Monroe Pkwy, Ste S, Lake Oswego, OR
• Difference between $x and ${x}
• Beware of Software Engineering Media Sites
• How Verizon and a BGP optimizer knocked large parts of the internet offline today
• DragonflyBSD - MDS mitigation added a while ago
• Reminder: Register for EuroBSDcon 2019 in Lillehammer, Norway

Feedback/Questions

• Dave - CheriBSD
• Neb - Hello from Norway
• Lars - Ansible tutorial?

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

ZFSonFreeBSD ports renamed OpenZFS

• The ZFS on FreeBSD project has renamed the userland and kernel ports from zol and zol-kmod to openzfs and openzfs-kmod
• The new versions from this week are IOCTL compatible with the command line tools in FreeBSD 12.0, so you can use the old userland with the new kernel module (although obviously not the new features)
• With the renaming it is easier to specify which kernel module you want to load in /boot/loader.conf: > zfs_load=”YES”
• or > openzfs_load=”YES”
• To load traditional or the newer version of ZFS
• The kmod still requires FreeBSD 12-stable or 13-current because it depends on the newer crypto support in the kernel for the ZFS native encryption feature. Allan is looking at ways to work around this, but it may not be practical.
• We would like to do an unofficial poll on how people would the userland to co-exist. Add a suffix to the new commands in /usr/local (zfs.new zpool.new or whatever). One idea i’ve had is to move the zfs and zpool commands to /libexec and make /sbin/zfs and /sbin/zpool a switcher script, that will call the base or ports version based on a config file (or just based on if the port is installed)
• For testing purposes, generally you should be fine as long as you don’t run ‘zpool upgrade’, which will make your pool only importable using the newer ZFS.
• For extra safety, you can create a ‘zpool checkpoint’, which will allow you to undo any changes that are made to the pool during your testing with the new openzfs tools. Note: the checkpoint will undo EVERYTHING. So don’t save new data you want to keep.
• Note: Checkpoints disable all freeing operations, to prevent any data from being overwritten so that you can re-import at the checkpoint and undo any operation (including zfs destroy-ing a dataset), so also be careful you don’t run out of space during testing.
• Please test and provide feedback.

How to use blacklistd(8) with NPF as a fail2ban replacement

• About blacklistd(8)

blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.
The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf
Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.
Unfortunately (dont' ask me why :P) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen

News Roundup

[WIP] raidz expansion, alpha preview 1

• Motivation and Context > This is a alpha-quality preview of RAID-Z expansion. This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn't sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks). > For additional context as well as a design overview, see my short talk from the 2017 OpenZFS Developer Summit: slides video

Rant: running audio VU-meter increases my CO2 footprint

A couple months ago I noticed that the monitor on my workstation never power off anymore. Screensaver would go on, but DPMs (to do the poweroff) never kicked in.
I grovels the output of various tools that display DPMS settings, which as usual in Xorg were useless. Everybody said DPMS is on with a timeout. I even wrote my own C program to use every available Xlib API call and even the xscreensaver library calls. (should make it available) No go, everybody says that DPMs is on, enabled and set on a timeout. Didn’t matter whether I let xscreeensaver do the job or just the X11 server.
After a while I noticed that DPMS actually worked between starting my X11 server and starting all my clients. I have a minimal .xinitrc and start the actual session from a script, that is how I could notice. If I used a regular desktop login I wouldn’t have noticed. A server state bug was much more likely than a client bug.

• See the article for the rest...

XSAVE and compat32 kernel work for LLDB

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types. You can read more about that in my Apr 2019 report.
In May, I was primarily continuing the work on new ptrace interface. Besides that, I've found and fixed a bug in ptrace() compat32 code, pushed LLVM buildbot to ‘green’ status and found some upstream LLVM regressions. More below.

Some things about where icons for modern X applications come from

If you have a traditional window manager like fvwm, one of the things it can do is iconify X windows so that they turn into icons on the root window (which would often be called the 'desktop'). Even modern desktop environments that don't iconify programs to the root window (or their desktop) may have per-program icons for running programs in their dock or taskbar. If your window manager or desktop environment can do this, you might reasonably wonder where those icons come from by default.
Although I don't know how it was done in the early days of X, the modern standard for this is part of the Extended Window Manager Hints. In EWMH, applications give the window manager a number of possible icons, generally in different sizes, as ARGB bitmaps (instead of, say, SVG format). The window manager or desktop environment can then pick whichever icon size it likes best, taking into account things like the display resolution and so on, and display it however it wants to (in its original size or scaled up or down).
How this is communicated in specific is through the only good interprocess communication method that X supplies, namely X properties. In the specific case of icons, the _NET_WM_ICON property is what is used, and xprop can display the size information and an ASCII art summary of what each icon looks like. It's also possible to use some additional magic to read out the raw data from _NET_WM_ICON in a useful format; see, for example, this Stackoverflow question and its answers.

Beastie Bits

• Recent Security Innovations
• Old Unix books + Solaris
• Pro-Desktop - A Tiling Desktop Environment
• The Tar Pipe
• At least one vim trick you might not know

Feedback/Questions

• Johnny - listener feedback
• Brian - Questions
• Mark - ZFS Question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

DragonFlyBSD's Kernel Optimizations Are Paying Off

DragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks.
The work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.)
With Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions.

---

What are the differences between OpenBSD and Linux?

Maybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?"
I've also been there at some point in the past and these are my conclusions.
They also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article.
This list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint.
Please bear with me.

• A terminal is a terminal is a terminal
• Practical differences
• Security and system administration
• Why philosophical differences matter
• So what do I choose?
• How to try OpenBSD ***

News Roundup

NetBSD 2019 Google Summer of Code

We are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects:

• Akul Abhilash Pillai - Adapting TriforceAFL for NetBSD kernel fuzzing
• Manikishan Ghantasala - Add KNF (NetBSD style) clang-format configuration
• Siddharth Muralee - Enhancing Syzkaller support for NetBSD
• Surya P - Implementation of COMPAT_LINUX and COMPAT_NETBSD32 DRM ioctls support for NetBSD kernel
• Jason High - Incorporation of Argon2 Password Hashing Algorithm into NetBSD
• Saurav Prakash - Porting NetBSD to HummingBoard Pulse
• Naveen Narayanan - Porting WINE to amd64 architecture on NetBSD

The communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19.
Please welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code!

Reducing that contention

The opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue!

• State of affairs

Most of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1).
I believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be executed without big lock. On the other hand very few syscalls are already or almost ready to be unlocked, as we incorrectly say. This explains why basic primitives like tsleep(9), csignal() and selwakeup() are only receiving attention now that the top of the Network Stack is running (mostly) without big lock.

• Next steps

In the past years, most of our efforts have been invested into the Network Stack. As I already mentioned it should be ready to be parallelized. However think we should now concentrate on removing the KERNEL_LOCK(), even if the code paths aren't performance critical.

• See the Article for the rest of the post

fnaify 1.3 released - more games are "fnaify & run" now

This release finally addresses some of the problems that prevent simple running of several games.
This happens for example when an old FNA.dll library comes with the games that doesn't match the API of our native libraries like SDL2, OpenAL, or MojoShader anymore. Some of those cases can be fixed by simply dropping in a newer FNA.dll. fnaify now asks if FNA 17.12 should be automatically added if a known incompatible FNA version is found. You simply answer yes or no.

Another blocker happens when the game expects to check the SteamAPI - either from a running Steam process, or a bundled steam_api library. OpenBSD 6.5-current now has steamworks-nosteam in ports, a stub library for Steamworks.NET that prevents games from crashing simply because an API function isn't found. The repo is here. fnaify now finds this library in /usr/local/share/steamstubs and uses it instead of the bundled (full) Steamworks.NET.dll.
This may help with any games that use this layer to interact with the SteamAPI, mostly those that can only be obtained via Steam.

vmctl(8): command line syntax changed

The order of the arguments in the create, start, and stop commands of vmctl(8) has been changed to match a commonly expected style. Manual usage or scripting with vmctl must be adjusted to use the new syntax.
For example, the old syntax looked like this:

# vmctl create disk.qcow2 -s 50G

The new syntax specifies the command options before the argument:

# vmctl create -s 50G disk.qcow2

Something that Linux distributions should not do when packaging things

Right now I am a bit unhappy at Fedora for a specific packaging situation, so let me tell you a little story of what I, as a system administrator, would really like distributions to not do.
For reasons beyond the scope of this blog entry, I run a Prometheus and Grafana setup on both my home and office Fedora Linux machines (among other things, it gives me a place to test out various things involving them). When I set this up, I used the official upstream versions of both, because I needed to match what we are running (or would soon be).
Recently, Fedora decided to package Grafana themselves (as a RPM), and they called this RPM package 'grafana'. Since the two different packages are different versions of the same thing as far as package management tools are concerned, Fedora basically took over the 'grafana' package name from Grafana. This caused my systems to offer to upgrade me from the Grafana.com 'grafana-6.1.5-1' package to the Fedora 'grafana-6.1.6-1.fc29' one, which I actually did after taking reasonable steps to make sure that the Fedora version of 6.1.6 was compatible with the file layouts and so on from the Grafana version of 6.1.5.
Why is this a problem? It's simple. If you're going to take over a package name from the upstream, you should keep up with the upstream releases. If you take over a package name and don't keep up to date or keep up to date only sporadically, you cause all sorts of heartburn for system administrators who use the package. The least annoying future of this situation is that Fedora has abandoned Grafana at 6.1.6 and I am going to 'upgrade' it with the upstream 6.2.1, which will hopefully be a transparent replacement and not blow up in my face. The most annoying future is that Fedora and Grafana keep ping-ponging versions back and forth, which will make 'dnf upgrade' into a minefield (because it will frequently try to give me a 'grafana' upgrade that I don't want and that would be dangerous to accept). And of course this situation turns Fedora version upgrades into their own minefield, since now I risk an upgrade to Fedora 30 actually reverting the 'grafana' package version on me.

---

Beastie Bits

• [talk] ZFS v UFS on APU2 msata SSD with FreeBSD
• NetBSD 8.1 is out
• lazyboi – the laziest possible way to send raw HTTP POST data
• A Keyboard layout that changes by markov frequency
• Open Source Game Clones
• EuroBSDcon program & registration open ***

Feedback/Questions

• John - A segment idea
• Johnny - Audio only format please don't
• Alex - Thanks and some Linux Snaps vs PBI feedback

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

Headlines

GPU Passthrough Reported Working on Bhyve

Normally we cover news focused on KVM and sometimes Xen, but something very special has happened with their younger cousin in the BSD world, Bhyve. For those that don’t know, Bhyve (pronounced bee-hive) is the native hypervisor in FreeBSD. It has many powerful features, but one that’s been a pain point for some years now is VGA passthrough. Consumer GPUs have not been useable until very recently despite limited success with enterprise cards. However, Twitter user Michael Yuji found a workaround that enables passing through a consumer card to any *nix system configured to use X11:

• https://twitter.com/michael_yuji/status/1127136891365658625

All you have to do is add a line pointing the X server to the Bus ID of the passed card and the VM will boot, with acceleration and everything. He theorizes that this may not be possible on windows because of the way it looks for display devices, but it’s a solid start. As soon as development surrounding VGA passthrough matures on Bhyve, it will become a very attractive alternative to more common tools like Hyper-V and Qemu, because it makes many powerful features available in the host system like jails, boot environments, BSD networking, and tight ZFS integration. For example, you could potentially run your Router, NAS, preferred workstation OS and any number of other things in one box, and only have to spin up a single VM because of the flexibility afforded by jails over Linux-based containers. The user who found this workaround also announced they’d be writing it up at some point, so stay tuned for details on the process. It’s been slow going on Bhyve passthrough development for a while, but this new revelation is encouraging. We’ll be closely monitoring the situation and report on any other happenings.

---

Confusion with used/free disk space in ZFS

I use ZFS extensively. ZFS is my favorite file system. I write articles and give lectures about it. I work with it every day. In traditional file systems we use df(1) to determine free space on partitions. We can also use du(1) to count the size of the files in the directory. But it’s different on ZFS and this is the most confusing thing EVER. I always forget which tool reports what disk space usage! Every time somebody asks me, I need to google it. For this reason I decided to document it here - for myself - because if I can’t remember it at least I will not need to google it, as it will be on my blog, but maybe you will also benefit from this blog post if you have the same problem or you are starting your journey with ZFS.

The understanding of how ZFS is uses space and how to determine which value means what is a crucial thing. I hope thanks to this article I will finally remember it!

News Roundup

OmniOS Community Edition

The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28. The OmniOS team and the illumos community have been very active in creating new features and improving existing ones over the last 6 months.

pfSense 2.4.4 Release p3 is available

We are pleased to announce the release of pfSense® software version 2.4.4-p3, now available for new installations and upgrades! pfSense software version 2.4.4-p3 is a maintenance release, bringing a number of security enhancements as well as a handful of fixes for issues present in the 2.4.4-p2 release. pfSense 2.4.4-RELEASE-p3 updates and installation images are available now! To see a complete list of changes and find more detail, see the Release Notes. We had hoped to bring you this release a few days earlier, but given the announcement last Tuesday of the Intel Microarchitectural Data Sampling (MDS) issue, we did not have sufficient time to fully incorporate those corrections and properly test for release on Thursday. We felt that it was worth delaying for a few days, rather than making multiple releases within a week.

• Upgrade Notes

Due to the significant nature of the changes in 2.4.4 and later, warnings and error messages, particularly from PHP and package updates, are likely to occur during the upgrade process. In nearly all cases these errors are a harmless side effect of the changes between FreeBSD 11.1 and 11.2 and between PHP 5.6 and PHP 7.2. Always take a backup of the firewall configuration prior to any major change to the firewall, such as an upgrade. Do not update packages before upgrading pfSense! Either remove all packages or do not update packages before running the upgrade. The upgrade will take several minutes to complete. The exact time varies based on download speed, hardware speed, and other factors such installed packages. Be patient during the upgrade and allow the firewall enough time to complete the entire process. After the update packages finish downloading it could take 10-20 minutes or more until the upgrade process ends. The firewall may reboot several times during the upgrade process. Monitor the upgrade from the firewall console for the most accurate view.

NetBSD 8.1 RC1 is out

The NetBSD Project is pleased to announce NetBSD 8.1, the first update of the NetBSD 8 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.

Some highlights of the 8.1 release are:

• x86: Mitigation for INTEL-SA-00233 (MDS)
• Various local user kernel data leaks fixed.
• x86: new rc.conf(5) setting smtoff to disable Simultaneous Multi-Threading
• Various network driver fixes and improvements.
• Fixes for thread local storage (TLS) in position independent executables (PIE).
• Fixes to reproducible builds.
• Fixed a performance regression in tmpfs.
• DRM/KMS improvements.
• bwfm(4) wireless driver for Broadcom FullMAC PCI and USB devices added.
• Various sh(1) fixes.
• mfii(4) SAS driver added.
• hcpcd(8) updated to 7.2.2
• httpd(8) updated.

FreeNAS as your Server OS

What if you could have a server OS that had built in RAID, NAS and SAN functionality, and could manage packages, containers and VMs in a GUI? What if that server OS was also free to download and install? Wouldn’t that be kind of awesome? Wouldn’t that be FreeNAS? FreeNAS is the world’s number one, open source storage OS, but it also comes equipped with all the jails, plugins, and VMs you need to run additional server-level services for things like email and web site hosting. File, Block, and even Object storage is all built-in and can be enabled with a few clicks. The ZFS file system scales to more drives than you could ever buy, with no limits for dataset sizes, snapshots, and restores. FreeNAS is also 100% FreeBSD. This is the OS used in the Netflix CDN, your PS4, and the basis for iOS. Set up a jail and get started downloading packages like Apache or NGINX for web hosting or Postfix for email service. Just released, our new TrueCommand management platform also streamlines alerts and enables multi-system monitoring.

Beastie Bits

• Keep Crashing Daemons Running on FreeBSD
• Look what I found today... my first set of BSD CDs...
• NetBSD - Intel MDS
• FreeBSD 11.3-BETA2 -- Please test!

Feedback/Questions

• Anthony - Question
• Guntbert - Podcast
• Guillaume - Another suggestion for Ales from Serbia

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FreeBSD 11.3-b1 is out

BSDCan 2019 Recap

• We’re back from BSDCan and it was a packed week as always.
• It started with bhyvecon on Tuesday. Meanwhile, Benedict spent the whole day in productive meetings: annual FreeBSD Foundation board meeting and FreeBSD Journal editorial board meeting.
• On Wednesday, tutorials for BSDCan started as well as the FreeBSD Developer Summit. In the mornings, there were presentations in the big auditorium, while working groups about networking, failsafe bootcode, development web services, swap space management, and testing/CI were held. Friday had a similar format with an update from the FreeBSD core team and the “have, need, want” session for FreeBSD 13. In the afternoon, there were working groups about translation tools, package base, GSoC/Outreachy, or general hacking. Benedict held his Icinga tutorial in the afternoon with about 15 people attending. Devsummit presentation slides can be found on the wiki page and video recordings done by ScaleEngine are available on FreeBSD’s youtube channel.
• The conference program was a good mixture of sysadmin and tech talks across the major BSDs. Benedict saw the following talks: How ZFS snapshots really work by Matt Ahrens, 20 years in Jail by Michael W. Lucas, OpenZFS BOF session, the future of OpenZFS and FreeBSD, MQTT for system administrators by Jan-Piet Mens, and spent the rest of the time in between in the hallway track.
• Photos from the event are available on Ollivier Robert’s talegraph and Diane Bruce’s website for day 1, day 2, conference day 1, and conference day 2.
• Thanks to all the sponsors, supporters, organizers, speakers, and attendees for making this yet another great BSDCan. Next year’s BSDCan will be from June 2 - 6, 2020.

OpenIndiana 2019.04 is out

We have released a new OpenIndiana Hipster snapshot 2019.04. The noticeable changes:

• Firefox was updated to 60.6.3 ESR

• Virtualbox packages were added (including guest additions)

• Mate was updated to 1.22

• IPS has received updates from OmniOS CE and Oracle IPS repos, including automatic boot environment naming

• Some OI-specific applications have been ported from Python 2.7/GTK 2 to Python 3.5/GTK 3

• Quick Demo Video: https://www.youtube.com/watch?v=tQ0-fo3XNrg

News Roundup

Overview of ZFS Pools in FreeNAS

FreeNAS uses the OpenZFS (ZFS) file system, which handles both disk and volume management. ZFS offers RAID options mirror, stripe, and its own parity distribution called RAIDZ that functions like RAID5 on hardware RAID. The file system is extremely flexible and secure, with various drive combinations, checksums, snapshots, and replication all possible. For a deeper dive on ZFS technology, read the ZFS Primer section of the FreeNAS documentation.

SUGGEST LAYOUT attempts to balance usable capacity and redundancy by automatically choosing an ideal vdev layout for the number of available disks.

• The following vdev layout options are available when creating a pool:
  • Stripe data is shared on two drives, similar to RAID0)
  • Mirror copies data on two drives, similar to RAID1 but not limited to 2 disks)
  • RAIDZ1 single parity similar to RAID5
  • RAIDZ2 double parity similar to RAID6
  • RAIDZ3 which uses triple parity and has no RAID equivalent

Why OpenSource Firmware is Important for Security

• Roots of Trust

The goal of the root of trust should be to verify that the software installed in every component of the hardware is the software that was intended. This way you can know without a doubt and verify if hardware has been hacked. Since we have very little to no visibility into the code running in a lot of places in our hardware it is hard to do this. How do we really know that the firmware in a component is not vulnerable or that is doesn’t have any backdoors? Well we can’t. Not unless it was all open source. Every cloud and vendor seems to have their own way of doing a root of trust. Microsoft has Cerberus, Google has Titan, and Amazon has Nitro. These seem to assume an explicit amount of trust in the proprietary code (the code we cannot see). This leaves me with not a great feeling. Wouldn’t it be better to be able to use all open source code? Then we could verify without a doubt that the code you can read and build yourself is the same code running on hardware for all the various places we have firmware. We could then verify that a machine was in a correct state without a doubt of it being vulnerable or with a backdoor. It makes me wonder what the smaller cloud providers like DigitalOcean or Packet have for a root of trust. Often times we only hear of these projects from the big three or five.

OPNsense

This update addresses several privilege escalation issues in the access control implementation and new memory disclosure issues in Intel CPUs. We would like to thank Arnaud Cordier and Bill Marquette for the top-notch reports and coordination.

• Here are the full patch notes:

• system: address CVE-2019-11816 privilege escalation bugs[1] (reported by Arnaud Cordier)

• system: /etc/hosts generation without interfacehasgateway()

• system: show correct timestamp in config restore save message (contributed by nhirokinet)

• system: list the commands for the pluginctl utility when n+ argument is given

• system: introduce and use userIsAdmin() helper function instead of checking for 'page-all' privilege directly

• system: use absolute path in widget ACLs (reported by Netgate)

• system: RRD-related cleanups for less code exposure

• interfaces: add EN DUID Generation using OPNsense PEN (contributed by Team Rebellion)

• interfaces: replace legacygetallinterface_addresses() usage

• firewall: fix port validation in aliases with leading / trailing spaces

• firewall: fix outbound NAT translation display in overview page

• firewall: prevent CARP outgoing packets from using the configured gateway

• firewall: use CARP net.inet.carp.demotion to control current demotion in status page

• firewall: stop live log poller on error result

• dhcpd: change rule priority to 1 to avoid bogon clash

• dnsmasq: only admins may edit custom options field

• firmware: use insecure mode for base and kernel sets when package fingerprints are disabled

• firmware: add optional device support for base and kernel sets

• firmware: add Hostcentral mirror (HTTP, Melbourne, Australia)

• ipsec: always reset rightallowany to default when writing configuration

• lang: say "hola" to Spanish as the newest available GUI language

• lang: updates for Chinese, Czech, Japanese, German, French, Russian and Portuguese

• network time: only admins may edit custom options field

• openvpn: call openvpnrefreshcrls() indirectly via plugin_configure() for less code exposure

• openvpn: only admins may edit custom options field to prevent privilege escalation (reported by Bill Marquette)

• openvpn: remove custom options field from wizard

• unbound: only admins may edit custom options field

• wizard: translate typehint as well

• plugins: os-freeradius 1.9.3 fixes string interpolation in LDAP filters (contributed by theq86)

• plugins: os-nginx 1.12[2]

• plugins: os-theme-cicada 1.17 (contributed by Team Rebellion)

• plugins: os-theme-tukan 1.17 (contributed by Team Rebellion)

• src: timezone database information update[3]

• src: install(1) broken with partially matching relative paths[4]

• src: microarchitectural Data Sampling (MDS) mitigation[5]

• ports: carootnss 3.44

• ports: php 7.2.18[6]

• ports: sqlite 3.28.0[7]

• ports: strongswan custom XAuth generic patch removed

wiregaurd on OpenBSD

Earlier this week I imported a port for WireGuard into the OpenBSD ports tree. At the moment we have the userland daemon and the tools available. The in-kernel implementation is only available for Linux. At the time of writing there are packages available for -current. Jason A. Donenfeld (WireGuard author) has worked to support OpenBSD in WireGuard and as such his post on ports@ last year got me interested in WireGuard, since then others have toyed with WireGuard on OpenBSD before and as such I've used Ted's article as a reference. Note however that some of the options mentioned there are no longer valid. Also, I'll be using two OpenBSD peers here. The setup will be as follows: two OpenBSD peers, of which we'll dub wg1 the server and wg2 the client. The WireGuard service on wg1 is listening on 100.64.4.3:51820.

• Conclusion

WireGuard (cl)aims to be easier to setup and faster than OpenVPN and while I haven't been able to verify the latter, the first is certainly true...once you've figured it out. Most documentation out there is for Linux so I had to figure out the wireguardgo service and the tun parameters. But all in all, sure, it's easier. Especially the client configuration on iOS which I didn't cover here because it's essentially pkgadd libqrencode ; cat client.conf | qrencode -t ansiutf8, scan the code with the WireGuard app and you're good to go. What is particularly neat is that WireGuard on iOS supports Always-on.

Beastie Bits

• Serenity OS
• vkernels vs pmap
• Brian Kernighan interviews Ken Thompson
• Improvements in forking, threading, and signal code
• DragonFly 5.4.3
• NetBSD on the Odroid C2

Feedback/Questions

• Paulo - Laptops
• A Listener - Thanks
• Bostjan - Extend a pool and lower RAM footprint

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Running AiX on QEMU on Linux on Windows

YES it’s real! I’m using the Linux subsystem on Windows, as it’s easier to build this Qemu tree from source. I’m using Debian, but these steps will work on other systems that use Debian as a base. first thing first, you need to get your system with the needed pre-requisites to compile Great with those in place, now clone Artyom Tarasenko’s source repository Since the frame buffer apparently isn’t quite working just yet, I configure for something more like a text mode build. Now for me, GCC 7 didn’t build the source cleanly. I had to make a change to the file config-host.mak and remove all references to -Werror. Also I removed the sound hooks, as we won’t need them. Now you can build Qemu. Okay, all being well you now have a Qemu. Now following the steps from Artyom Tarasenko’s blog post, we can get started on the install!

• See article for rest of walkthrough.

Take Command of Your NAS Fleet with TrueCommand

Hundreds of thousands of FreeNAS and TrueNAS systems are deployed around the world, with many sites having dozens of systems. Managing multiple systems individually can be time-consuming. iXsystems has responded to the challenge by creating a “single pane of glass” application to simplify the scaling of data, drive management, and administration of iXsystems NAS platforms. We are proud to introduce TrueCommand. TrueCommand is a ZFS-aware management application that manages TrueNAS and FreeNAS systems. The public Beta of TrueCommand is available for download now. TrueCommand can be used with small iXsystems NAS fleets for free. Licenses can be purchased for large-scale deployments and enterprise support. TrueCommand expands on the ease of use and power of TrueNAS and FreeNAS systems with multi-system management and reporting.

News Roundup

Unleashed 1.3 Released

This is the fourth release of Unleashed - an operating system fork of illumos. For more information about Unleashed itself and the download links, see our website. As one might expect, this release removes a few things. The most notable being the removal of ksh93 along with all its libs. As far as libc interfaces are concerned, a number of non-standard functions were removed. In general, they have been replaced by the standards-compliant versions. (getgrentr, fgetgrentr, getgrgidr, getgrnamr, ttynamer, getloginr, shmdt, sigwait, gethostname, putmsg, putpmsg, and getaddrinfo) Additionally, wordexp and wordfree have been removed from libc. Even though they are technically required by POSIX, software doesn't seem to use them. Because of the fragile implementation (shelling out), we took the OpenBSD approach and just removed them. The default compilation environment now includes XOPENSOURCE=700 and EXTENSIONS. Additionally, all applications now use 64-bit file offsets, making use of LARGEFILESOURCE, LARGEFILE64SOURCE, and FILEOFFSET_BITS unnecessary. Last but not least, nightly.sh is no more. In short, to build one simply runs 'make'. (See README for detailed build instructions.)

• Why Unleashed

Why did we decide to fork illumos? After all, there are already many illumos distributions available to choose from. We felt we could do better than any of them by taking a more aggressive stance toward compatibility and reducing cruft from code and community interactions alike.

LLDB: extending CPU register inspection support

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and updating NetBSD distribution to LLVM 8 (which is still stalled by unresolved regressions in inline assembly syntax). You can read more about that in my Mar 2019 report. In April, my main focus was on fixing and enhancing the support for reading and writing CPU registers. In this report, I'd like to shortly summarize what I have done, what I have learned in the process and what I still need to do.

• Future plans

My work continues with the two milestones from last month, plus a third that's closely related: Add support for FPU registers support for NetBSD/i386 and NetBSD/amd64. Support XSAVE, XSAVEOPT, ... registers in core(5) files on NetBSD/amd64. Add support for Debug Registers support for NetBSD/i386 and NetBSD/amd64. The most important point right now is deciding on the format for passing the remaining registers, and implementing the missing ptrace interface kernel-side. The support for core files should follow using the same format then. Userland-side, I will work on adding matching ATF tests for ptrace features and implement LLDB side of support for the new ptrace interface and core file notes. Afterwards, I will start working on improving support for the same things on 32-bit (i386) executables.

V7 Unix programs are often not written the way you would expect

Yesterday I wrote that V7 ed read its terminal input in cooked mode a line at a time, which was an efficient, low-CPU design that was important on V7's small and low-power hardware. Then in comments, frankg pointed out that I was wrong about part of that, namely about how ed read its input.

• Sidebar: An interesting undocumented ed feature

Reading this section of the source code for ed taught me that it has an interesting, undocumented, and entirely characteristic little behavior. Officially, ed commands that have you enter new text have that new text terminate by a . on a line by itself:

In other words, it turns a single line with '.' into an EOF. The consequence of this is that if you type a real EOF at the start of a line, you get the same result, thus saving you one character (you use Control-D instead of '.' plus newline). This is very V7 Unix behavior, including the lack of documentation.

This is also a natural behavior in one sense. A proper program has to react to EOF here in some way, and it might as well do so by ending the input mode. It's also natural to go on to try reading from the terminal again for subsequent commands; if this was a real and persistent EOF, for example because the pty closed, you'll just get EOF again and eventually quit. V7 ed is slightly unusual here in that it deliberately converts '.' by itself to EOF, instead of signaling this in a different way, but in a way that's also the simplest approach; if you have to have some signal for each case and you're going to treat them the same, you might as well have the same signal for both cases.

Modern versions of ed appear to faithfully reimplement this convenient behavior, although they don't appear to document it. I haven't checked OpenBSD, but both FreeBSD ed and GNU ed work like this in a quick test. I haven't checked their source code to see if they implement it the same way.

---

Beastie Bits

• CarolinaCon 15: Writing Exploit-Resistant Code With OpenBSD
• CFT: FreeBSD Package Base
• Initial FUSE support in DragonFly
• Two significant bugfixes for 5.4
• Libretto 100ct: 166mhz Pentium, 16gb compactflash, 32mb ram running OpenBSD

Feedback/Questions

• DJ - Feedback
• Fabian - ZFS ARC
• Caleb - Question
• A small programming note: After BSDNow episode 300, the podcast will switch to audio-only, using a new higher quality recording and production system. The live stream will likely still include video.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

36+ year old bug in FFS/UFS discovered and patched

This update eliminates a kernel stack disclosure bug in UFS/FFS directory entries that is caused by uninitialized directory entry padding written to the disk.

• When the directory entry is written to disk, it is written as a full 32bit entry, and the unused bytes were not initialized, so could possibly contain sensitive data from the kernel stack It can be viewed by any user with read access to that directory. Up to 3 bytes of kernel stack are disclosed per file entry, depending on the the amount of padding the kernel needs to pad out the entry to a 32 bit boundary. The offset in the kernel stack that is disclosed is a function of the filename size. Furthermore, if the user can create files in a directory, this 3 byte window can be expanded 3 bytes at a time to a 254 byte window with 75% of the data in that window exposed. The additional exposure is done by removing the entry, creating a new entry with a 4-byte longer name, extracting 3 more bytes by reading the directory, and repeating until a 252 byte name is created. This exploit works in part because the area of the kernel stack that is being disclosed is in an area that typically doesn't change that often (perhaps a few times a second on a lightly loaded system), and these file creates and unlinks themselves don't overwrite the area of kernel stack being disclosed. It appears that this bug originated with the creation of the Fast File System in 4.1b-BSD (Circa 1982, more than 36 years ago!), and is likely present in every Unix or Unix-like system that uses UFS/FFS. Amazingly, nobody noticed until now. This update also adds the -z flag to fsck_ffs to have it scrub the leaked information in the name padding of existing directories. It only needs to be run once on each UFS/FFS filesystem after a patched kernel is installed and running. Submitted by: David G. Lawrence dg@dglawrence.com
• So a patched kernel will no longer leak this data, and running the fsck_ffs -z command will erase any leaked data that may exist on your system
• OpenBSD commit with additional detail on mitigations The impact on OpenBSD is very limited: 1 - such stack bytes can be found in raw-device reads, from group operator. If you can read the raw disks you can undertake other more powerful actions. 2 - read(2) upon directory fd was disabled July 1997 because I didn't like how grep * would display garbage and mess up the tty, and applying vis(3) for just directory reads seemed silly. read(2) was changed to return 0 (EOF). Sep 2016 this was further changed to EISDIR, so you still cannot see the bad bytes. 3 - In 2013 when guenther adapted the getdents(2) directory-reading system call to 64-bit ino_t, the userland data format changed to 8-byte-alignment, making it incompatible with the 4-byte-alignment UFS on-disk format. As a result of code refactoring the bad bytes were not copied to userland. Bad bytes will remain in old directories on old filesystems, but nothing makes those bytes user visible. There will be no errata or syspatch issued. I urge other systems which do expose the information to userland to issue errata quickly, since this is a 254 byte infoleak of the stack which is great for ROP-chain building to attack some other bug. Especially if the kernel has no layout/link-order randomization ...

---

NomadBSD, a BSD for the Road

As regular It’s FOSS readers should know, I like diving into the world of BSDs. Recently, I came across an interesting BSD that is designed to live on a thumb drive. Let’s take a look at NomadBSD. NomadBSD is different than most available BSDs. NomadBSD is a live system based on FreeBSD. It comes with automatic hardware detection and an initial config tool. NomadBSD is designed to “be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD’s hardware compatibility.” This German BSD comes with an OpenBox-based desktop with the Plank application dock. NomadBSD makes use of the DSB project. DSB stands for “Desktop Suite (for) (Free)BSD” and consists of a collection of programs designed to create a simple and working environment without needing a ton of dependencies to use one tool. DSB is created by Marcel Kaiser one of the lead devs of NomadBSD. Just like the original BSD projects, you can contact the NomadBSD developers via a mailing list.

• Version 1.2 Released

NomadBSD recently released version 1.2 on April 21, 2019. This means that NomadBSD is now based on FreeBSD 12.0-p3. TRIM is now enabled by default. One of the biggest changes is that the initial command-line setup was replaced with a Qt graphical interface. They also added a Qt5 tool to install NomadBSD to your hard drive. A number of fixes were included to improve graphics support. They also added support for creating 32-bit images.

• Thoughts on NomadBSD

I first discovered NomadBSD back in January when they released 1.2-RC1. At the time, I had been unable to install Project Trident on my laptop and was very frustrated with BSDs. I downloaded NomadBSD and tried it out. I initially ran into issues reaching the desktop, but RC2 fixed that issue. However, I was unable to get on the internet, even though I had an Ethernet cable plugged in. Luckily, I found the wifi manager in the menu and was able to connect to my wifi. Overall, my experience with NomadBSD was pleasant. Once I figured out a few things, I was good to go. I hope that NomadBSD is the first of a new generation of BSDs that focus on mobility and ease of use. BSD has conquered the server world, it’s about time they figured out how to be more user-friendly.

---

News Roundup

[OpenBSD automatic

upgrade](https://www.tumfatig.net/20190426/openbsd-automatic-upgrade/)

OpenBSD 6.5 advertises for an installer improvement: rdsetroot(8) (a build-time tool) is now available for general use. Used in combination with autoinstall.8, it is now really easy to do automatic upgrades of your OpenBSD instances. I first manually upgraded my OpenBSD sandbox to 6.5. Once that was done, I could use the stock rdsetroot(8) tool. The plan is quite simple: write an unattended installation response file, insert it to a bsd.rd 6.5 installation image and reboot my other OpenBSD instances using that image.

• Extra notes

There must be a way to run onetime commands (in the manner of fw_update) to automatically run sysmerge and packages upgrades. As for now, I’d rather do it manually. This worked like a charm on two Synology KVM instances using a single sd0 disk, on my Thinkpad X260 using Encrypted root with Keydisk and on a Vultr instance using Encrypted root with passphrase. And BTW, the upgrade on the X260 used the (iwn0) wireless connection. I just read that florian@ has released the sysupgrade(8) utility which should be released with OpenBSD 6.6. That will make upgrades even easier! Until then, happy upgrading.

FreeBSD Dtrace ext2fs Support

• Which logs were replaced by dtrace-probes:

  • Misc printf's under DEBUG macro in the blocks allocation path.
  • Different on-disk structures validation errors, now the filesystem will silently return EIO's.
  • Misc checksum errors, same as above.

• The only debug macro, which was leaved is EXT2FSPRINTEXTENTS.

• It is impossible to replace it by dtrace-probes, because the additional logic is required to walk thru file extents.

• The user still be able to see mount errors in the dmesg in case of:

  • Filesystem features incompatibility.
  • Superblock checksum error.

Create a dedicated user for ssh tunneling only

I use ssh tunneling A LOT, for everything. Yesterday, I removed the public access of my IMAP server, it’s now only available through ssh tunneling to access the daemon listening on localhost. I have plenty of daemons listening only on localhost that I can only reach through a ssh tunnel. If you don’t want to bother with ssh and redirect ports you need, you can also make a VPN (using ssh, openvpn, iked, tinc…) between your system and your server. I tend to avoid setting up VPN for the current use case as it requires more work and more maintenance than running ssh server and a ssh client. The last change, for my IMAP server, added an issue. I want my phone to access the IMAP server but I don’t want to connect to my main account from my phone for security reasons. So, I need a dedicated user that will only be allowed to forward ports. This is done very easily on OpenBSD. The steps are: 1. generate ssh keys for the new user 2. add an user with no password 3. allow public key for port forwarding Obviously, you must allow users (or only this one) to make port forwarding in your sshd_config.

---

That was easy. Some info on upgrading VMM VMs to 6.5

We're running dedicated vmm(4)/vmd(8) servers to host opinionated VMs. OpenBSD 6.5 is released! There are two ways you can upgrade your VM. Either do a manual upgrade or leverage autoinstall(8). You can take care of it via the console with vmctl(8).

• Upgrade yourself

To get connected to the console you need to have access to the host your VM is running on. The same username and public SSH key, as provided for the VM, are used to create a local user on the host. When this is done you can use vmctl(8) to manage your VM. The options you have are:

```$ vmctl start id [-c]```

$ vmctl stop id [-fw]```

```-w Wait until the VM has been terminated.```

-c Automatically connect to the VM console.```

• See the Article for the rest of the guide

Beastie Bits

• powerpc64 architecture support in FreeBSD ports
• GhostBSD 19.04 overview
• HardenedBSD will have two user selectable ASLR implementations
• NYCBUG 2016 Talk Shell-Fu Uploaded
• What is ZIL anyway?

Feedback/Questions

• Quentin - Organize an Ada/BSD interview
• DJ - Update
• Patrick - Bhyve frontends
• A small programming note: After BSDNow episode 300, the podcast will switch to audio-only, using a new higher quality recording and production system. The live stream will likely still include video.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS. Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices. FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out. Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.

DragonFlyBSD 5.4.2 is out

Upgrading guide

```The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely.  I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon.  This version includes Matt's fix for the HAMMER2 corruption bug he identified recently.```

If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.```

```> cd /usr/src ```

git pull ```

```And then rebuild: (in /usr/src ) ```

```

```> make buildkernel ```

make installkernel ```

```> make upgrade ```

```

``` ```

(reboot) ```

```> make initrd ```

```

``` ```

pkg update > pkg upgrade```

News Roundup

Containing web services with iocell

I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛 If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U

• Getting started

There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.

• To start, you'll need the following:
  • A FreeBSD install (we'll be using 11.0)
  • The iocell package (available as a package, also in the ports tree)
  • A ZFS pool for hosting the jails

Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.

As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.

• See Article for the rest of the walkthrough.

Oracle Solaris 11.4 SRU8

Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.

• This SRU introduces the following enhancements:
  • Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.
  • UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4
  • Re-introduction of the service fc-fabric.
  • ibus has been updated to 1.5.19

• The following components have also been updated to address security issues:
  • NTP has been updated to 4.2.8p12
  • Firefox has been updated to 60.6.0esr
  • BIND has been updated to 9.11.6
  • OpenSSL has been updated to 1.0.2r
  • MySQL has been updated to 5.6.43 & 5.7.25
  • libxml2 has been updated to 2.9.9
  • libxslt has been updated to 1.1.33
  • Wireshark has been updated to 2.6.7
  • ncurses has been updated to 6.1.0.20190105
  • Apache Web Server has been updated to 2.4.38
  • perl 5.22
  • pkg.depot

The Problem with SSH Agent Forwarding

After hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].” Here’s what man ssh_config has to say about ForwardAgent: "Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent."" Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine! Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.

[OpenBSD Upgrade Guide: 6.4 to 6.5

Start by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u. Alternatively, you can use the manual upgrade process. You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.

• Before rebooting into the install kernel
• Configuration and syntax changes
• Files to remove
• Special packages
• Upgrade without the install kernel

Beastie Bits

• 2019 FreeBSD Community Survey
• Seagate runs Mach.2 demo on FreeBSD
• FreeBSD: Resizing and Growing Disks
• Loading 4.9 on an old Tandy 4025LX - 386, 16MB, 1GB HD. Good old external SCSI CD
• OS108 MATE 20190422 released

Feedback/Questions

• Casey - Oklahoma City & James
• Michael - Question on SAS backplane (camcontrol?)
• Ales - OpenBSD, FreeNAS, OpenZFS questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

OpenBSD 6.5 Released

• Changelog
• Mirrors
• 6.5 Includes
  • OpenSMTPD 6.5.0
  • LibreSSL 2.9.1
  • OpenSSH 8.0
  • Mandoc 1.14.5
  • Xenocara
  • LLVM/Clang 7.0.1 (+ patches)
  • GCC 4.2.1 (+ patches) and 3.3.6 (+ patches)
• Many pre-built packages for each architecture:
  • aarch64: 9654
  • amd64: 10602
  • i386: 10535

Mount your ZFS datasets anywhere you want

ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility. When you create zpool maintank, the default mountpoint is /maintank. You might be happy with that, but you don’t have to be content. You can do magical things.

• Some highlights are:
  • mount point can be inherited
  • not all filesystems in a zpool need to be mounted
  • each filesystem (directory) can have different ZFS characteristics
  • In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently.

News Roundup

Branch for netbsd 9 upcoming, please help and test -current

Folks, once again we are quite late for branching the next NetBSD release (NetBSD 9). Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that. On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later. On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:

• please test -current, on all the various machines you have
• especially interesting would be test results from uncommon architectures or strange combinations (like the sparc userland on sparc64 kernel issue I ran in yesterday) Please test, report success, and file PRs for failures! We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may. We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go. Our goal is to have an unprecedented short release cycle this time. But.. we always say that upfront.

---

LibreSSL 2.9.1 Released

We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL directory of your local OpenBSD mirror soon. This is the first stable release from the 2.9 series, which is also included with OpenBSD 6.5

It includes the following changes and improvements from LibreSSL 2.8.x:

• API and Documentation Enhancements

  • CRYPTO_LOCK is now automatically initialized, with the legacy callbacks stubbed for compatibility.
  • Added the SM3 hash function from the Chinese standard GB/T 32905-2016.
  • Added the SM4 block cipher from the Chinese standard GB/T 32907-2016.
  • Added more OPENSSLNO* macros for compatibility with OpenSSL.
  • Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH.
  • Implemented further missing OpenSSL 1.1 API.
  • Added support for XChaCha20 and XChaCha20-Poly1305.
  • Added support for AES key wrap constructions via the EVP interface.

• Compatibility Changes

  • Added pbkdf2 key derivation support to openssl(1) enc.
  • Changed the default digest type of openssl(1) enc to sha256.
  • Changed the default digest type of openssl(1) dgst to sha256.
  • Changed the default digest type of openssl(1) x509 -fingerprint to sha256.
  • Changed the default digest type of openssl(1) crl -fingerprint to sha256.

• Testing and Proactive Security

  • Added extensive interoperability tests between LibreSSL and OpenSSL 1.0 and 1.1.
  • Added additional Wycheproof tests and related bug fixes.

• Internal Improvements

  • Simplified sigalgs option processing and handshake signing algorithm selection.
  • Added the ability to use the RSA PSS algorithm for handshake signatures.
  • Added bnrandinterval() and use it in code needing ranges of random bn values.
  • Added functionality to derive early, handshake, and application secrets as per RFC8446.
  • Added handshake state machine from RFC8446.
  • Removed some ASN.1 related code from libcrypto that had not been used since around 2000.
  • Unexported internal symbols and internalized more record layer structs.
  • Removed SHA224 based handshake signatures from consideration for use in a TLS 1.2 handshake.

• Portable Improvements

  • Added support for assembly optimizations on 32-bit ARM ELF targets.
  • Added support for assembly optimizations on Mingw-w64 targets.
  • Improved Android compatibility

• Bug Fixes

  • Improved protection against timing side channels in ECDSA signature generation.
  • Coordinate blinding was added to some elliptic curves. This is the last bit of the work by Brumley et al. to protect against the Portsmash vulnerability.
  • Ensure transcript handshake is always freed with TLS 1.2.

The LibreSSL project continues improvement of the codebase to reflect modern, safe programming practices. We welcome feedback and improvements from the broader community. Thanks to all of the contributors who helped make this release possible.

---

FreeBSD Mastery: Jails – Bail Bond Denied Edition

I had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.) I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly. I downgraded my expectations and tried again. And again. And again. I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition! This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort. But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character. Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile. As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough. And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.

One reason ed(1) was a good editor back in the days of V7 Unix

It is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does. The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented. This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.

Beastie Bits

• CFT for FreeBSD ZoL
• Simple DNS Adblock
• AT&T Unix PC in 1985
• OpenBSD-current drm at 4.19, includes new support for Intel GPUs like Coffee Lake
• "What are the differences between Linux and OpenBSD?" - Twitter thread
• Announcing the pkgsrc-2019Q1 release (2019-04-10)

Feedback/Questions

• Brad - iocage
• Frank - Video from Level1Tech and a question
• Niall - Revision Control

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Introducing funlinkat

• It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition

One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove. The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:

• We will remove an entry in the directory with the filename.
• We will decrease a file reference count (in inode).
• If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself. Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time. The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.
  • When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete. We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.
  • Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it. In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel. This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file. After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor. int funlinkat(int dfd, const char *path, int fd, int flags); The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it. The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat. As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.

---

Using an OpenBSD Router with AT&T U-Verse

I upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over. Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.

---

News Roundup

How to use NetBSD on a Raspberry Pi

Do you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered "yes" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi. BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix. Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.

---

ZFS Encryption is still under development (as of March 2019)

One of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.

• CFT for FreeBSD + ZoL

---

Tutorial On Rump Kernel Servers and Clients

The rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server "mini-operating systems". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand. This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.

---

Installing Snort on OpenBSD 6.4

As you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!! I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!

---

Beastie Bits

• os108
• AT&T Archives: The UNIX Operating System
• httpd(8): Adapt to industry wide current best security practices
• Quotes From A Book That Bashes Unix
• OpenBSD QA wiki

Feedback/Questions

• Malcolm - Laptop Experience : Dell XPS 13
• DJ - Feedback
• Alex - GhostBSD and Wifi : FIXED

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

A Pi-Powered Plan 9 Cluster

Plan 9 from Bell Labs comes from the same stable as the UNIX operating system, which of course Linux was designed after, and Apple’s OS X runs on top of a certified UNIX operating system. Just like UNIX, Plan 9 was developed as a research O/S — a vehicle for trying out new concepts — with it building on key UNIX principles and taking the idea of devices are just files even further. In this post, we take a quick look at the Plan 9 O/S and some of the notable features, before moving on to the construction of a self-contained 4-node Raspberry Pi cluster that will provide a compact platform for experimentation.

---

Endlessh: an SSH Tarpit

I’m a big fan of tarpits: a network service that intentionally inserts delays in its protocol, slowing down clients by forcing them to wait. This arrests the speed at which a bad actor can attack or probe the host system, and it ties up some of the attacker’s resources that might otherwise be spent attacking another host. When done well, a tarpit imposes more cost on the attacker than the defender. The Internet is a very hostile place, and anyone who’s ever stood up an Internet-facing IPv4 host has witnessed the immediate and continuous attacks against their server. I’ve maintained such a server for nearly six years now, and more than 99% of my incoming traffic has ill intent. One part of my defenses has been tarpits in various forms.

---

News Roundup

rdist(1) – when Ansible is too much

The post written about rdist(1) on johan.huldtgren.com sparked us to write one as well. It's a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it. There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them. We didn't have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn't any interest in building additional logic on top of rsync or repositories. > Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing.

---

Falling in love with OpenBSD again

I was checking the other day and was appalled at how long it has been since I posted here. I had been working a job during 2018 that had me traveling 3,600 miles by air every week so that is at least a viable excuse. So what is my latest project? I wanted to get something better than the clunky old T500 “freedom laptop” that I could use as my daily driver. Some background here. My first paid gig as a programmer was on SunOS 4 (predecessor to Solaris) and Ultrix (on a DEC MicroVAX). I went from there to a Commodore Amiga (preemptive multitasking in 1985!). I went from there to OS/2 (I know, patron saint of lost causes) and then finally decided to “sell out” and move to Windows as the path of least resistance in the mid 90’s. My wife bought me an iPod literally just as they started working with computers other than Macs and I watched with fascination as Apple made the big gamble and moved away from PowerPC chips to Intel. That was the beginning of the Apple Fan Boi years for me. My gateway drug was a G4 MacMini and I managed somehow to get in on the pre-production, developer build of an Intel-based Mac. I was quite happy on the platform until about three years ago.

---

How I Created My First FreeBSD Port

I created my first FreeBSD port recently. I found that FreeBSD didn't have a port for GoCD, which is a continuous integration and continuous deployment (CI/CD) system. This was a great opportunity to learn how to build a FreeBSD port while also contributing back to the community

---

The Tilde Institute of OpenBSD Education

Welcome to tilde.institute! This is an OpenBSD machine whose purpose is to provide a space in the tildeverse for experimentation with and education of the OpenBSD operating system. A variety of editors, shells, and compilers are installed to allow for development in a native OpenBSD environment. OpenBSD's httpd(8) is configured with slowcgi(8) as the fastcgi provider and sqlite3 available. This allows users to experiment with web development using compiled CGI in C, aka the BCHS Stack. In addition to php7.0 and mysql (mariadb) by request, this provides an environment where the development of complex web apps is possible.

---

Beastie Bits

• SoloBSD 19.03-STABLE
• WireGuard for NetBSD
• [NetBSD - Removing PF](https://mail-index.netbsd.org/tech-kern/2019/03/29/msg024883.html )
• What does the N in nmake stand for?
• A Map of the Internet from May 1973
• NSA-B-Gone : A sketchy hardware security device for your x220

Feedback/Questions

• Jake - A single jail as a VPN client
• Matt - Surprising BSD Features
• cia - Routing and ZFS

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

###Interview - Michael W. Lucas - mwl@mwl.io / @mwlauthor
FreeBSD Mastery: Jails

• BR: Welcome back to the show and congratulations on your latest book. How many books did you have to write before you could start on FreeBSD Mastery: Jails?
• AJ: How much research did you have to do about jails?
• BR: The book talks about something called ‘incomplete’ jails. What do you mean by that?
• AJ: There are a lot of jail management frameworks out there. Why did you chose to write about iocage in the book?
• BR: How many jails do you run yourself?
• AJ: Can you tell us a bit about how you handle book sponsorship these days?
• BR: What other books (fiction and non-fiction) are you currently working on?
• AJ: Which talks are you looking forward to attend at the upcoming BSDCan conference?
• BR: How is the BSD user group going?
• AJ: Anything else you’d like to mention before we release you from our interview jail cell?

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###AsiaBSDcon 2019 recap

• Both Allan and I attended AsiaBSDcon 2019 in Tokyo in mid march. After a couple of days of Tokyo sightseeing and tasting the local food, the conference started with tutorials.
• Benedict gave his tutorial about “BSD-based Systems Monitoring with Icinga2 and OpenSSH”, while Allan ran the FreeBSD developer summit.
• On the next day, Benedict attended the tutorial “writing (network) tests for FreeBSD” held by Kristof Provost. I learned a lot about Kyua, where tests live and how they are executed. I took some notes, which will likely become an article or chapter in the developers handbook about writing tests.
• On the third day, Hiroki Sato officially opened the paper session and then people went into individual talks.
• Benedict attended

  Adventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel
  Vadot

powerpc64 architecture support in FreeBSD ports by Piotr Kubaj
Managing System Images with ZFS by Allan Jude
FreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz
Security Fantasies and Realities for the BSDs by George V.
Neville-Neil
ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

• Allan attended

  Adventures in DRMland by Emmanuel Vadot
  Intel HAXM by Kamil Rytarowski
  BSD Solutions in Australian NGOs
  Container Migration on FreeBSD by Yuhei Takagawa
  Security Fantasies and Realities for the BSDs by George Neville-Neil

ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

• When not in talks, time was spent in the hallway track and conversations would often continue over dinner.
• Stay tuned for announcements about where AsiaBSDcon 2020 will be, as the Tokyo Olympics will likely force some changes for next year. Overall, it was nice to see people at the conference again, listen to talks, and enjoy the hospitality of Japan.

###FreeBSD Quarterly Status Report - Fourth Quarter 2018

Since we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered:
i386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves.
Additionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group.
We would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having.

###GhostBSD: A Solid Linux-Like Open Source Alternative

The subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms.
GhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface.
I stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest.
In last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux.
GhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes.
Keep reading to find out what piqued my excitement about Linux-like GhostBSD.

##News Roundup
###SPARCbook 3000ST - The coolest 90s laptop

A few weeks back I managed to pick up an incredibly rare laptop in immaculate condition for $50 on Kijiji: a Tadpole Technologies SPARCbook 3000ST from 1997 (it also came with two other working Pentium laptops from the 1990s).
Sun computers were an expensive desire for many computer geeks in the 1990s, and running UNIX on a SPARC-based laptop was, well, just as cool as it gets. SPARC was an open hardware platform that anyone could make, and Tadpole licensed the Solaris UNIX operating system from Sun for their SPARCbooks. Tadpole essentially made high-end UNIX/VAX workstations on costly, unusual platforms (PowerPC, DEC Alpha, SPARC) but only their SPARCbooks were popular in the high-end UNIX market of the 1990s.

###OpenSSH 8.0 Releasing With Quantum Computing Resistant Keys

OpenSSH 7.9 came out with a host of bug fixes last year with few new features, as is to be expected in minor releases. However, recently, Damien Miller has announced that OpenSSH 8.0 is nearly ready to be released. Currently, it’s undergoing testing to ensure compatibility across supported systems.

• https://twitter.com/damienmiller/status/1111416334737244160

Better Security
Copying filenames with scp will be more secure in OpenSSH 8.0 due to the fact that copying filenames from a remote to local directory will prompt scp to check if the files sent from the server match your request. Otherwise, an attack server would theoretically be able to intercept the request by serving malicious files in place of the ones originally requested. Knowing this, you’re probably better off never using scp anyway. OpenSSH advises against it:
“The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead.”

• Interesting new features

ssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you.

###Project Trident : 18.12-U8 Available

Thank you all for your patience! Project Trident has finally finished some significant infrastructure updates over the last 2 weeks, and we are pleased to announce that package update 8 for 18.12-RELEASE is now available.
To switch to the new update, you will need to open the “Configuration” tab in the update manager and switch to the new “Trident-release” package repository. You can also perform this transition via the command line by running: sudo sysup --change-train Trident-release

##Beastie Bits

• BSD Router Project - Release 1.92
• EuroBSDcon - New Proposals
• Funny UNIX shirt (René Magritte art parody)
• 51NB’s Thinkpad X210
• DragonFly: No more gcc50
• “FreeBSD Mastery: Jails” ebook escaping!
• FreeBSD talk at the Augsburger Linux Info Days (german)

##Feedback/Questions

• DJ - FuguIta Feedback
• Mike - Another Good Show
• Alex - GhostBSD and wifi

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines

###Tracking a storage issue led to software change

Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.

• Problem

While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.

###What Makes Unix Special

Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.

• What Makes Unix Special?

It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)

##News Roundup
###What you need may be “pipeline +Unix commands” only

I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
BTW, if your data set can be disposed by an awk script, it should not be called “big data”.

• Taco Bell Programming

###Running a bakery on Emacs and PostgreSQL

Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face of it, this is really simple, just work out how much dough you need, then work out what quantities to mix to make that much dough. Easy. You can do it with a pencil and paper. Or, in traditional bakers’ fashion, by scrawling with your finger on a floured work bench.
And that’s how I coped for a few weeks early on. But I kept making mistakes, which makes for an inconsistent product (bread is very forgiving, you have to work quite hard to make something that isn’t bread, but consistency matters). I needed to automate.

###The Ultimate Guide To Memorable Tech Talks

Imagine this. You’re a woman in a male-dominated field. English is not your first language. Even though you’re confident in your engineering work, the thought of public speaking and being recorded for the world to see absolutely terrifies you.
That was me, five years ago. Since then, I’ve moved into a successful career in Developer Advocacy and spoken at dozens of technical events in the U.S. and worldwide.
I think everyone has the ability to deliver stellar conference talks, which is why I took the time to write this post.

• The Ultimate Guide
• 1: Introduction
• 2: Choosing a Topic
• 3: Writing a Conference Proposal (or CFP)
• 4: Tools of the Trade
• 5: Planning and Time Estimation
• 6: Writing a Talk
• 7: Practice and Delivery

###Light-weight Contexts: An OS Abstraction for Safety and Performance (2016)

Abstract: “We introduce a new OS abstraction—light-weight con-texts (lwCs)—that provides independent units of protection, privilege, and execution state within a process. A process may include several lwCs, each with possibly different views of memory, file descriptors, and access capabilities. lwCs can be used to efficiently implement roll-back (process can return to a prior recorded state),isolated address spaces (lwCs within the process may have different views of memory, e.g., isolating sensitive data from network-facing components or isolating different user sessions), and privilege separation (in-process reference monitors can arbitrate and control access).
lwCs can be implemented efficiently: the overhead of a lwC is proportional to the amount of memory exclusive to the lwC; switching lwCs is quicker than switching kernel threads within the same process. We describe the lwC abstraction and API, and an implementation of lwCs within the FreeBSD 11.0 kernel. Finally, we present an evaluation of common usage patterns, including fast roll-back, session isolation, sensitive data isolation, and in-process reference monitoring, using Apache, nginx, PHP,and OpenSSL.”

##Beastie Bits

• May 7th - BSD Users Stockholm Meetup #6
• sysutils/docker-freebsd: Searching for people to help
• Cat Tax - Ever wonder what Midnight the cat was like?
• Fixing Unix/Linux/POSIX Filenames
• Metasploit on OpenBSD
• Run Your @wn Email Server! with NetBSD
• rdist(1)
• Writing a Book with Unix
• 7 Unix Commands Every Data Scientist Should Know
• Explaining Code using ASCII Art
• FreeBSD Aberdeen Hackathon
• FreeBSD Vienna Hackathon

##Feedback/Questions

• Mike - FreeBSD Update and Erased EFI files

• Charles - Volunteer work

• Jake - Bhyve Front Ends

• We’ve hit that point where we are running low on your questions, so if you have any questions rolling around in your head that you’ve not thought of to ask yet… send them in!

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###ARM’d and dangerous: FreeBSD on Cavium ThunderX (aarch64)

While I don’t remember for how many years I’ve had an interest in CPU architectures that could be an alternative to AMD64, I know pretty well when I started proposing to test 64-bit ARM at work. It was shortly after the disaster named Spectre / Meltdown that I first dug out server-class ARM hardware and asked whether we should get one such server and run some tests with it.
While the answer wasn’t a clear “no” it also wasn’t exactly “yes”. I tried again a few times over the course of 2018 and each time I presented some more points why I thought it might be a good thing to test this. But still I wasn’t able to get a positive answer. Finally in January 2019 year I got a definitive answer – and it was “yes, go ahead”! The fact that Amazon had just presented their Graviton ARM Processor may have helped the decision.

###Looking at NetBSD from an OpenBSD user perspective

I use to use NetBSD quite a lot. From 2.0 to 6.99. But for some reasons, I stopped using it about 2012, in favor of OpenBSD. Reading on the new 8 release, I wanted to see if all the things I didn’t like on NetBSD were gone. Here is a personal Pros / Cons list. No Troll, hopefully. Just trying to be objective.

• What I liked (pros)
• Things I didn’t like (cons)
• Conclusion

So that was it. I didn’t spend more than 30 minutes of it. But I didn’t want to spend more time on it. I did stop using NetBSD because of the need to compile each and every packages ; it was in the early days of pkgin. I also didn’t like the way system maintenance was to be done. OpenBSD’s 6-months release seemed far more easy to manage. I still think NetBSD is a great OS. But I believe you have to spent more time on it than you would have to do with OpenBSD.
That said, I’ll keep using my Puffy OS.

##News Roundup
###Using Vim to take time-stamped notes

I frequently find myself needing to take time-stamped notes. Specifically, I’ll be in a call, meeting, or interview and need to take notes that show how long it’s been since the meeting started.
My first thought was that there’s be a plugin to add time stamps, but a quick search didn’t turn anything up. However, I little digging did turn up the fact that vim has the built-in ability to tell time.
This means that writing a bit of vimscript to insert a time stamp is pretty easy. After a bit of fiddling, I came up with something that serves my needs, and I decided it might be useful enough to others to be worth sharing.

• John Baldwin’s notes on bhyve meetings

###OpenBSD 6.5-beta has been tagged

It’s that time of year again; Theo (deraadt@) has just tagged 6.5-beta. A good reminder for us all run an extra test install and see if your favorite port still works as you expect.

CVSROOT: /cvs
Module name: src
Changes by: deraadt@cvs.openbsd.org 2019/02/26 15:24:41

Modified files:
etc/root : root.mail
share/mk : sys.mk
sys/conf : newvers.sh
sys/sys : ktrace.h param.h
usr.bin/signify: signify.1
sys/arch/macppc/stand/tbxidata: bsd.tbxi

Log message:
crank to 6.5-beta

###The NetBSD Foundation participating in Google Summer of Code 2019

For the 4th year in a row and for the 13th time The NetBSD Foundation will participate in Google Summer of Code 2019!
If you are a student and would like to learn more about Google Summer of Code please go to the Google Summer of Code homepage.
You can find a list of projects in Google Summer of Code project proposals in the wiki.
Do not hesitate to get in touch with us via #netbsd-code IRC channel on Freenode and via NetBSD mailing lists!

###SecBSD: an UNIX-like OS for Hackers

SecBSD is an UNIX-like operating system focused on computer security based on OpenBSD. Designed for security testing, hacking and vulnerability assessment, it uses full disk encryption and ProtonVPN + OpenVPN by default.
A security BSD enviroment for security researchers, penetration testers, bug hunters and cybersecurity experts. Developed by Dark Intelligence Team for private use and will be public release coming soon.

##Beastie Bits

• Why OpenBSD Rocks
• Rich’s sh (POSIX shell) tricks
• Drinking coffee with AWK
• Civilisational HTTP Error Codes
• MidnightBSD Roadmap
• NetBSD on Nintendo64
• From Vimperator to Tridactyl

##Feedback/Questions

• Russell - BSD Now Question :: ZFS & FreeNAS
• Alan - Tutorial, install ARM *BSD with no other BSD box pls
• Johnny - New section to add to the show

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines

###A Kernel Of Failure -
How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it.

Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.

###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter

Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.

Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.

• Kristof Provost, who maintains the port of pf in FreeBSD added a test for the vulnerability in FreeBSD head.

##News Roundup
###How I’m still not using GUIs in 2019: A guide to the terminal

TL;DR: Here are my dotfiles. Use them and have fun.

GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.

• Don’t forget rule number one.

Whenever in doubt, read the manual.

###Using a Yubikey as smartcard for SSH public key authentication

SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
You might have heard of Yubikeys.
These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
In OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.

###The 18 Part FreeBSD Desktop Series by Vermaden

• FreeBSD Desktop – Part 1 – Simplified Boot
• FreeBSD Desktop – Part 2 – Install (FreeBSD 11)
• FreeBSD Desktop – Part 2.1 – Install FreeBSD 12
• FreeBSD Desktop – Part 3 – X11 Window System
• FreeBSD Desktop – Part 4 – Key Components – Window Manager
• FreeBSD Desktop – Part 5 – Key Components – Status Bar
• FreeBSD Desktop – Part 6 – Key Components – Task Bar
• FreeBSD Desktop – Part 7 – Key Components – Wallpaper Handling
• FreeBSD Desktop – Part 8 – Key Components – Application Launcher
• FreeBSD Desktop – Part 9 – Key Components – Keyboard/Mouse Shortcuts
• FreeBSD Desktop – Part 10 – Key Components – Locking Solution
• FreeBSD Desktop – Part 11 – Key Components – Blue Light Spectrum Suppress
• FreeBSD Desktop – Part 12 – Configuration – Openbox
• FreeBSD Desktop – Part 13 – Configuration – Dzen2
• FreeBSD Desktop – Part 14 – Configuration – Tint2
• FreeBSD Desktop – Part 15 – Configuration – Fonts & Frameworks
• FreeBSD Desktop – Part 16 – Configuration – Pause Any Application
• FreeBSD Desktop – Part 17 – Automount Removable Media

##Beastie Bits

• Drist with persistent SSH
• ARPANET: Celebrating 50 Years Since “LO”
• Termtris - a tetris game for ANSI/VT220 terminals
• Poor Man’s CI - Hosted CI for BSD with shell scripting and duct tape
• Why I use the IBM Model M keyboard that is older than me?
• A privilege separated and sandboxed IPv6 Stateless Address AutoConfiguration Daemon
• Google-free Android Setup
• BSD Users Stockholm Meetup #6

##Feedback/Questions

• Sijmen - Hi, and a Sunday afternoon toy project
• Clint - Tuning ZFS for NVME
• James - Show question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###Google: Software is never going to be able to fix Spectre-type bugs

• Spectre is here to stay: An analysis of side-channels and speculative execution

Researchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution.
The discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations.
Specifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened.

###A proof that Unix utility sed is Turing complete

Many people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete.
I first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete.
Christophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script.

Christophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these:

• Tetris
• Sokoban (game)
• Calculator

##News Roundup
###Bastille helps you quickly create and manage FreeBSD Jails.

Bastille helps you quickly create and manage FreeBSD Jails.
Jails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services.
Bastille provides an interface to create, manage and destroy these secure virtualized environments.

• Current version: 0.3.20190204-beta.
• Shell Script Source here: https://github.com/BastilleBSD/bastille/blob/master/usr/local/bin/bastille

###netdata v1.12 released

Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.

• Patch release 1.12.1 contains 22 bug fixes and 8 improvements.

###Using grep with /dev/null, an old Unix trick

Every so often I will find myself writing a grep invocation like this:

find .... -exec grep <something> /dev/null '{}' '+'

The peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep if you’re not certain how many files your wildcard may match.

###USING GMAIL WITH MUTT

I recently switched to using mutt for email and while setting up mutt to use imap is pretty straightforward, this tutorial will also document some advanced concepts such as encrypting your account password and sending emails from a different From address.
This tutorial assumes that you have some familiarity with using mutt and have installed it with sidebar support (sudo apt-get install mutt-patched for the ubuntu folks) and are comfortable with editing your muttrc.
If you would just like to skip to the end, my mutt configuration file can be found here.

##Beastie Bits

• An Extensive UNIX Timeline
• Garbage.fm - OEF
• brk() to sbrk()
• Fred models, found again
• Kafe: Can OS Kernels Forward Packets Fast Enough for Software Routers?
• ARPANET: Celebrating 50 Years Since “LO”

##Feedback/Questions

• Pablo - Topic suggestion: FreeBSD on a Laptop as daily driver
• Ron - ZFS on the fly compression and seek
• Dave - two zpool, or not two zpool, that is the question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###The Design and Implementation of the NetBSD rc.d system

• Abstract

In this paper I cover the design and implementation of the rc.d system start-up mechanism in NetBSD 1.5, which replaced the monolithic /etc/rc start-up file inherited from 4.4BSD. Topics covered include a history of various UNIX start-up mechanisms (including NetBSD prior to 1.5), design considerations that evolved over six years of discussions, implementation details, an examination of the human issues that occurred during the design and implementation, as well as future directions for the system.

• Introduction

NetBSD recently converted from the traditional 4.4BSD monolithic /etc/rc start-up script to an /etc/rc.d mechanism, where there is a separate script to manage each service or daemon, and these scripts are executed in a specific order at system boot.
This paper covers the motivation, design and implementation of the rc.d system; from the history of what NetBSD had before to the system that NetBSD 1.5 shipped with in December 2000, as well as future directions.
The changes were contentious and generated some of the liveliest discussions about any feature change ever made in NetBSD. Parts of those discussions will be covered to provide insight into some of the design and implementation decisions.

• History

There is great diversity in the system start-up mechanisms used by various UNIX variants. A few of the more pertinent schemes are detailed below. As NetBSD is derived from 4.4BSD, it follows that a description of the latter’s method is relevant. Solaris’ start-up method is also detailed, as it is the most common System V UNIX variant.

###First impressions of Project Trident 18.12

Project Trident (hereafter referred to as Trident) is a desktop operating system based on TrueOS. Trident takes the rolling base platform of TrueOS, which is in turn based on FreeBSD’s development branch, and combines it with the Lumina desktop environment.

+Installing

The debut release of Trident is available as a 4.1GB download that can be burned to a disc or transferred to a USB thumb drive. Booting from the Trident media brings up a graphical interface and automatically launches the project’s system installer. Down the left side of the display there are buttons we can click to show hardware information and configuration options. These buttons let us know if our wireless card and video card are compatible with Trident and give us a chance to change our preferred language and keyboard layout. At the bottom of the screen we find buttons that will open a terminal or shutdown the computer.

• Early impressions

Trident boots to a graphical login screen where we can sign into the Lumina desktop or a minimal Fluxbox session. Lumina, by default, uses Fluxbox as its window manager. The Lumina desktop places its panel along the bottom of the screen and an application menu sits in the bottom-left corner. On the desktop we find icons for opening the software manager, launching the Falkon web browser, running the VLC media player, opening the Control Panel and adjusting the Lumina theme.
The application menu has an unusual and compact layout. The menu shows just a search box and buttons for browsing applications, opening a file manager, accessing desktop settings and signing out. To see what applications are available we can click the Browse Applications entry, which opens a window in the menu where we can scroll through installed programs. This is a bit awkward since the display window is small and only shows a few items at a time.
Early on I found it is possible to swap out the default “Start menu” with an alternative “Application menu” through the Panels configuration tool. This alternative menu offers a classic tree-style application menu. I found the latter menu easier to navigate as it expands to show all the applications in a selected category.

• Conclusions

I have a lot of mixed feelings and impressions when it comes to Trident. On the one hand, the operating system has some great technology under the hook. It has cutting edge packages from the FreeBSD ecosystem, we have easy access to ZFS, boot environments, and lots of open source packages. Hardware support, at least on my physical workstation, was solid and the Lumina desktop is flexible.

##News Roundup
###PXE booting of a FreeBSD disk image

I had to set up a regression and network performance lab. This lab will be managed by a Jenkins, but the first step is to understand how to boot a FreeBSD disk by PXE. This article explains a simple way of doing it.
For information, all these steps were done using 2 PC Engines APU2 (upgraded with latest BIOS for iPXE support), so it’s a headless (serial port only, this can be IPMI SoL with different hardware) .

• THE BIG PICTURE

Before explaining all steps and command line, here is the full big picture of the final process.

###Why I like middle mouse button paste in xterm so much

In my entry about how touchpads are not mice, I mused that one of the things I should do on my laptop was insure that I had a keyboard binding for paste, since middle mouse button is one of the harder multi-finger gestures to land on a touchpad. Kurt Mosiejczuk recently left a comment there where they said:
Shift-Insert is a keyboard equivalent for paste that is in default xterm (at least OpenBSD xterm, and putty on Windows too). I use that most of the time now as it seems less… trigger-happy than right click paste.
This sparked some thoughts, because I can’t imagine giving up middle mouse paste if I have a real choice. I had earlier seen shift-insert mentioned in other commentary on my entry and so have tried a bit to use it on my laptop, and it hasn’t really felt great even there; on my desktops, it’s even less appealing (I tried shift-insert out there to confirm that it did work in my set of wacky X resources).
In thinking about why this is, I came to the obvious realization about why all of this is so. I like middle mouse button paste in normal usage because it’s so convenient, because almost all of the time my hand is already on the mouse. And the reason my hand is already on the mouse is because I’ve just used the mouse to shift focus to the window I want to paste into. Even on my laptop, my right hand is usually away from the keyboard as I move the mouse pointer on the touchpad, making shift-Insert at least somewhat awkward.

###NetBSD Gains Hardware Accelerated Virtualization

• NetBSD Virtual Machine Monitor

NVMM provides hardware-accelerated virtualization support for NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is shipped via libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.

##Beastie Bits

• SoloBSD 19.02-STABLE
• Project Trident 18.12-U5 available
• “Sudo Mastery, Second Edition” and Cover Art
• MKSANITIZER - bug detector software integration with the NetBSD userland
• Darn kids nowadays… back in my day we drew rude symbols like normal people. {{top two comments}}
• ShellCheck
  finds bugs in your shell scripts.
• Old School Sean - A history of UNIX

##Feedback/Questions

• Ales - OpenBSD, FreeNAS, OpenZFS questions
• Malcolm - Thoughts on Pgsql + ZFS thread?
• Brad - Boot Environments in FreeBSD

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###Adding Glue To a Desktop Environment

In this article we will put some light on a lot of tools used in the world of Unix desktop environment customization, particularly regarding wmctrl, wmutils, xev, xtruss, xwininfo, xprop, xdotools, xdo, sxhkd, xbindkeys, speckeysd, xchainkeys, alttab, triggerhappy, gTile, gidmgr, keynav, and more. If those don’t make sense then this article will help. Let’s hope this can open your mind to new possibilities.
With that in mind we can wonder if what’s actually needed from a window manager, presentation and operation, can be split up and complemented with other tools. We can also start thinking laterally, the communication and interaction between the different components of the environment. We have the freedom to do so because the X protocol is transparent and components usually implement many standards for interfacing between windows. It’s like gluing parts together to create a desktop environment.

• The tools we’ll talk about fall into one of those categories:
• Debugging
• Window manipulation
• Simulation of interaction
• Extended manipulation
• Hotkey daemon
• Layout manager

###Flashing the BIOS on the PC Engines APU4c4

I absolutely love the PC Engines APU devices. I use them for testing HardenedBSD experimental features in more constrained 64-bit environments and firewalls. Their USB and mSATA ports have a few quirks, and I bumped up against a major quirk that required flashing a different BIOS as a workaround. This article details the hacky way in which I went about doing that.
What prompted this article is that something in either the CAM or GEOM layer in FreeBSD 11.2 caused the mSATA to hang, preventing file writes. OPNsense 18.7 uses FreeBSD 11.1 whereas the recently-released OPNsense 19.1 uses HardenedBSD 11.2 (based on FreeBSD 11.2). I reached out to PC Engines directly, and they let me know that the issue is a known BIOS issue. Flashing the “legacy” BIOS series would provide me with a working system.
It also just so happens that a new “legacy” BIOS version was just released which turns on ECC mode for the RAM. So, I get a working OPNsense install AND ECC RAM! I’ll have one bird for dinner, the other for dessert.
Though I’m using an APU4, these instructions should work for the other APU devices. The BIOS ROM download URLs should be changed to reflect the device you’re targeting along with the BIOS version you wish to deploy.
SPECIAL NOTE: There be dragons! I’m primarily writing this article to document the procedure for my own purposes. My memory tends to be pretty faulty these days. So, if something goes wrong, please do not hold me responsible. You’re the one at the keyboard. ;)
VERY SPECIAL NOTE: We’ll use the mSATA drive for swap space, just in case. Should the swap space be used, it will destroy whatever is on the disk.

##News Roundup
###Revive a Cisco IDS into a capable OpenBSD computer!

Even though Cisco equipment is very capable, it tends to become End-of-Life before you can say “planned obsolescence”. Websites become bigger, bandwidths increase, and as a side effect of those “improvements”, routers, firewalls, and in this case, intrusion prevention systems get old quicker and quicker.
Apparently, this was also the case for the Cisco IDS-4215 Intrusion Detection Sensor that I was given a few months ago.
I’m not too proud to admit that at first, I didn’t care about the machine itself, but rather about the add-on PCI network card with 4 Fast Ethernet interfaces. The sensor has obviously seen better days, as it had a broken front panel and needed some cleaning, but upon a closer inspection under the hood (which is held closed by the 4 screws on top), this IDS consists of an embedded Celeron PC with two onboard Ethernet cards, a 2.5″ IDE hard disk, a CF card, and 2 PCI expansion slots (more on them later). Oh, and don’t forget the nasty server-grade fan, which pushed very little air for the noise it was making.

###An OpenBSD desktop using WindowMaker

Since I started using *N?X, I’ve regularly used WindowMaker. I’ve always liked the look and feel, the dock system and the dockapps. It may look a bit oldish nowadays. And that’s enough to try to change this. So here it is, a 2019 flavored WindowMaker Desktop, running on OpenBSD 6.4/amd64.
This configuration uses the Nord color-scheme, the Adapta-Nokto-Eta GTK theme and the Moblin Unofficial Icons icon set. I did remove applications icons. I just don’t need them on the bottom of the screen as I heavily use “F11” to pop-up the windows list. To be able to do that and keep the dockapps, I tweaked my ~/GNUstep/Defaults/WMWindowAttributes and created a ~/GNUstep/Library/WindowMaker/Themes/Nord.themed/style.
And here it is, the NeXT OpenBSD Desktop!

###RealTime Data Compression

In a previous episode, we’ve seen that it is possible to create opaque types. However, creation and destruction of such type must be delegated to some dedicated functions, which themselves rely on dynamic allocation mechanisms.
Sometimes, it can be convenient to bypass the heap, and all its malloc() / free() shenanigans. Pushing a structure onto the stack, or within thread-local storage, are natural capabilities offered by a normal struct. It can be desirable at times.
The previously described opaque type is so secret that it has no size, hence is not suitable for such scenario.
Fortunately, static opaque types are possible.
The main idea is to create a “shell type”, with a known size and an alignment, able to host the target (private) structure.
For safer maintenance, the shell type and the target structure must be kept in sync, by using typically a static assert. It will ensure that the shell type is always large enough to host the target structure. This check is important to automatically detect future evolution of the target structure.

###For the Love of Pipes

My top used shell command is |. This is called a pipe.
In brief, the | allows for the output of one program (on the left) to become the input of another program (on the right). It is a way of connecting two commands together.
According to doc.cat-v.org/unix/pipes/, the origin of pipes came long before Unix. Pipes can be traced back to this note from Doug McIlroy in 1964

##Beastie Bits

• Installation Notes for NetBSD/i386 0.9
• Porting Zig to NetBSD - a fun, speedy port
• NNN - Tiny, lightning fast, feature-packed file manager Release v2.3
• eta - A tool for monitoring progress and ETA of an arbitrary process
  
• A FreeBSD User Tries Out…NetBSD 8.0
• Faster vlan(4) forwarding?
• FuguIta - OpenBSD 6.4 Live System
• Adding Name-based hosting To Nginx on OpenBSD with Acme-Client
• HOWTO set up QEMU with HAXM acceleration on NetBSD
• README: gcc 7 switch coming to a port near you!

##BUG Calendar

• ChiBUG, Chicago, USA: Tuesday, February 26th 18:00 at the Oak Park Library
• CharmBUG, Baltimore, USA: Wednesday, February 27, 2019
  19:30 at Columbia Ale House
• NYC*BUG, New York, USA: Wednesday, March 6, 2019 18:45 at Suspenders
• KnoxBUG, Knoxville, USA: Monday, February 25, 2019 - 18:00 at iX Systems offices
• BSDPL, Warsaw, Poland: February 28, 2019 18:15 - 21:00 at Wheel Systems Office

##Feedback/Questions

• Sam - Customizing OpenBSD ports source code
• Frank - Rivalry Linux & BSD
• Zach - mysql/mariadb tuning

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant

Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.

###Reflecting on The Soul of a New Machine

Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.
Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.
Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.

• See Article for rest…

##News Roundup

###Out-Of-The-Box 10GbE Network Benchmarks On Nine Linux Distributions Plus FreeBSD 12

Last week I started running some fresh 10GbE Linux networking performance benchmarks across a few different Linux distributions. That testing has now been extended to cover nine Linux distributions plus FreeBSD 12.0 to compare the out-of-the-box networking performance.
Tested this round alongside FreeBSD 12.0 was Antergos 19.1, CentOS 7, Clear Linux, Debian 9.6, Fedora Server 29, openSUSE Leap 15.0, openSUSE Tumbleweed, Ubuntu 18.04.1 LTS, and Ubuntu 18.10.
All of the tests were done with a Tyan S7106 1U server featuring two Intel Xeon Gold 6138 CPUs, 96GB of DDR4 system memory, and Samsung 970 EVO SSD. For the 10GbE connectivity on this server was an add-in HP NC523SFP PCIe adapter providing two 10Gb SPF+ ports using a QLogic 8214 controller.
Originally the plan as well was to include Windows Server 2016/2019. Unfortunately the QLogic driver download site was malfunctioning since Cavium’s acquisition of the company and the other Windows Server 2016 driver options not panning out and there not being a Windows Server 2019 option. So sadly that Windows testing was thwarted so I since started testing over with a Mellanox Connectx-2 10GbE NIC, which is well supported on Windows Server and so that testing is ongoing for the next article of Windows vs. Linux 10 Gigabit network performance plus some “tuned” Linux networking results too.

###Integration of the LLVM sanitizers with the NetBSD base system

Over the past month I’ve merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I’ve also managed to get a functional set of Makefile rules to build all of them, namely:
ASan, UBSan, TSan, MSan, libFuzzer, SafeStack, XRay.
In all supported variations and modes that are supported by the original LLVM compiler-rt package.

###Distrowatch FreeNAS 11.2 review

The project’s latest release is FreeNAS 11.2 and, at first, I nearly overlooked the new version because it appeared to be a minor point release. However, a lot of work went into the new version and 11.2 offers a lot of changes when compared next to 11.1, “including a major revamp of the web interface, support for self-encrypting drives, and new, backwards-compatible REST and WebSocket APIs. This update also introduces iocage for improved plugins and jails management and simplified plugin development.”

##Beastie Bits

• Instructions for installing rEFInd to dual boot a computer with FreeBSD and windows (and possibly other OSes as well).
• NetBSD desktop pt.6: “vi(1) editor, tmux and unicode $TERM”
• Unix flowers
• FreeBSD upgrade procedure using GPT
• Pull-based Backups using OpenBSD base*
• Developing WireGuard for NetBSD
• OpenZFS User Conference, April 18-19, Norwalk CT
• KnoxBug Feb 25th

##Feedback/Questions

• Jake - C Programming
• Farhan - Explanation of rtadvd
• Nelson - Bug Bounties on Open-Source Software

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines

###FOSDEM 2019 Recap

• Allan and I were at FOSDEM 2019 in Brussels, Belgium over the weekend.
• On the Friday before, we held a FreeBSD Devsummit in a hotel conference room, with 25 people attending. We talked about various topics of interest to the project. You can find the notes on the wiki page.
• Saturday was the first day of FOSDEM. The FreeBSD Project had a table next to the Illumos Project again. A lot of people visited our table, asked questions, or just said “Hi, I watch BSDNow.tv every week”. We handed out a lot of stickers, pens, swag, and flyers. There was also a full day BSD devroom, with a variety of talks that were well attended.
• In the main conference track, Allan held a talk explaining how the ZFS ARC works. A lot of people attended the talk and had more questions afterwards. Another well attended talk was by Jonathan Looney about Netflix and FreeBSD.
• Sunday was another day in the same format, but no bsd devroom. A lot of people visited our table, developers and users alike. A lot of meeting and greeting went on.
• Overall, FOSDEM was a great success with FreeBSD showing a lot of presence. Thanks to all the people who attended and talked to us. Special thanks to the people who helped out at the FreeBSD table and Rodrigo Osorio for running the BSD devroom again.

###FreeBSD Foundation Update, January 2019

Dear FreeBSD Community Member,
Happy New Year! It’s always exciting starting the new year with ambitious plans to support FreeBSD in new and existing areas. We achieved our fundraising goal for 2018, so we plan on funding a lot of work this year! Though it’s the new year, this newsletter highlights some of the work we accomplished in December. We also put together a list of technologies and features we are considering supporting, and are looking for feedback on what users want to help inform our 2019 development plans. Our advocacy and education efforts are in full swing as we prepare for upcoming conferences including FOSDEM, SANOG33, and SCaLE.
Finally, we created a year-end video to talk about the work we did in 2018. That in itself was an endeavor, so please take a few minutes to watch it! We’re working on improving the methods we use to inform the community on the work we are doing to support the Project, and are always open to feedback. Now, sit back, grab a refreshing beverage, and enjoy our newsletter!
Happy reading!!
Deb

###OPNsense 19.1 released

For more than four years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
The 19.1 release, nicknamed “Inspiring Iguana”, consists of a total of 620 individual changes since 18.7 came out 6 months ago, spread out over 12 intermediate releases including the recent release candidates. That is the average of 2 stable releases per month, security updates and important bug fixes included! If we had to pick a few highlights it would be: The firewall alias API is finally in place. The migration to HardenedBSD 11.2 has been completed. 2FA now works with a remote LDAP / local TOTP combination. And the OpenVPN client export was rewritten for full API support as well.

• These are the most prominent changes since version 18.7:

• fully functional firewall alias API

• PIE firewall shaper support

• firewall NAT rule logging support

• 2FA via LDAP-TOTP combination

• WPAD / PAC and parent proxy support in the web proxy

• P12 certificate export with custom passwords

• Dpinger is now the default gateway monitor

• ET Pro Telemetry edition plugin[2]

• extended IPv6 DUID support

• Dnsmasq DNSSEC support

• OpenVPN client export API

• Realtek NIC driver version 1.95

• HardenedBSD 11.2, LibreSSL 2.7

• Unbound 1.8, Suricata 4.1

• Phalcon 3.4, Perl 5.28

• firmware health check extended to cover all OS files, HTTPS mirror default

• updates are browser cache-safe regarding CSS and JavaScript assets

• collapsible side bar menu in the default theme

• language updates for Chinese, Czech, French, German, Japanese, Portuguese and Russian

• API backup export, Bind, Hardware widget, Nginx, Ntopng, VnStat and Dnscrypt-proxy plugins

• Here are the full changes against version 19.1-RC2:

• ipsec: add firewall interface as soon as phase 1 is enabled

• ipsec: phase 1 selection GUI JavaScript compatibility fix

• monit: widget improvements and bug fix (contributed by Frank Brendel)

• ui: fix regression in single host or network subnet select in static pages

• plugins: os-frr 1.7 updates OSFP outbound rules (contributed by Fabian Franz)

• plugins: os-telegraf 1.7.4 fixes packet filter input

• plugins: os-theme-rebellion 1.8.2 adds image colour invert

• plugins: os-vnstat 1.1[3]

• plugins: os-zabbix-agent now uses Zabbix version 4.0

• src: revert mmc_calculate_clock() as HS200/HS400 support breaks legacy support

• src: update sqlite3-3.20.0 to sqlite3-3.26.0[4]

• src: import tzdata 2018h, 2018i[5]

• src: avoid unsynchronized updates to kn_status[6]

• ports: ca_root_nss 3.42

• ports: dhcp6c 20190128 prevent rawops double-free (contributed by Team Rebellion)

• ports: sudo patch to fix listpw=never[7]

##News Roundup
###The hardware-assisted virtualization challenge

Over two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux’ KVM, or Hyper-V).
Today, I am here to report: Mission Accomplished!
It’s been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.

###ZFS and GPL terror: How much freedom is there in Linux?

• ZFS – the undesirable guest

ZFS is todays most advanced filesystem. It originated on the Solaris operating system and thanks to Sun’s decision to open it up, we have it available on quite a number of Unix-like operating systems. That’s just great! Great for everyone.
For everyone? Nope. There are people out there who don’t like ZFS. Which is totally fine, they don’t need to use it after all. But worse: There are people who actively hate ZFS and think that others should not use it. Ok, it’s nothing new that some random guys on the net are acting like assholes, trying to tell you what you must not do, right? Whoever has been online for more than a couple of days probably already got used to it. Unfortunately its still worse: One such spoilsport is Greg Kroah-Hartman, Linux guru and informal second-in-command after Linus Torvalds.
There have been some attempts to defend the stance of this kernel developer. One was to point at the fact that the “ZFS on Linux” (ZoL) port uses two kernel functions, __kernel_fpu_begin() and __kernel_fpu_end(), which have been deprecated for a very long time and that it makes sense to finally get rid of them since nothing in-kernel uses it anymore. Nobody is going to argue against that. The problem becomes clear by looking at the bigger picture, though:
The need for functions doing just what the old ones did has of course not vanished. The functions have been replaced with other ones. And those ones are deliberately made GPL-only. Yes, that’s right: There’s no technical reason whatsoever! It’s purely ideology – and it’s a terrible one.

###ClonOS 19.01-RELEASE

ClonOS is a turnkey Open Source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for easily controlling, deploying and managing FreeBSD jails containers and Bhyve/Xen hyperviser virtual environments.
ClonOS is currently the only platform available which allow both Xen and Bhyve hypervisor to coexist on the same host. Being a FreeBSD base platform, ClonOS ability to create and manage jails allows you to run FreeBSD applications without losing performance.

• Features:

• easy management via web UI interface

• live Bhyve migration [coming soon, roadmap]

• Bhyve management (create, delete VM)

• Xen management (create, delete VM) [coming soon, roadmap]

• connection to the “physical” guest console via VNC from the browser or directly

• Real time system monitoring

• access to load statistics through SQLite3 and beanstalkd

• support for ZFS features (cloning, snapshots)

• import/export of virtual environments

• public repository with virtual machine templates

• puppet-based helpers for configuring popular services

• ClonOS is a free open-source FreeBSD-based platform for virtual environments creation and management. In the core:

• FreeBSD OS as hoster platform

• bhyve(8) as hypervisor engine

• Xen as hypervisor engine

• vale(4) as Virtual Ethernet Switch

• jail(8) as container engine

• CBSD Project as management tools

• Puppet as configuration management

##Beastie Bits

• Florian Obser on unwind(8)
• A low tech SMS gateway for fun and no profit
• Netflix and FreeBSD : Using Open Source to Deliver Streaming Video
• powerd++ 0.4.0 release
• Is it time to rewrite the operating system in Rust?
• Small change, big effect
• Swedish BSD Meetup, Feb 19, 2019
• Polish BSD User Group Meetup, Feb 21, 2019

##Feedback/Questions

• Casey - Cool new Digital Ocean Feature
• Morgan - Jail w/differnet version of FreeBSD
• Brad - FreeBSD Installer

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Interview - Niclas Zeising - zeising@FreeBSD.org / @niclaszeising
Interview topic: FreeBSD Graphics Stack

• BR: Welcome Niclas. Since this is your first time on BSDNow, can you tell us a bit about yourself and how you started with Unix/BSD?
• AJ: What made you start working in the FreeBSD graphics stack?
• BR: What is the current status with the FreeBSD graphics stack?
• AJ: What challenges do you face in the FreeBSD graphics stack?
• BR: How many people are working in the graphics team and what kind of help do you need there?
• AJ: You’re also involved in FreeBSD ports and held a poudriere tutorial at last years EuroBSDcon. What kind of feedback did you get and will you give that tutorial again?
• BR: You’ve been organizing the Stockholm BSD user group meeting. Can you tell us a bit about that, what’s involved, how is it structured?
• AJ: What conferences do you go to where people could talk to you?
• BR: Is there anything else you’d like to mention before we let you go?

##Feedback/Questions

• Casey - TrueOS
• Troels - zfs send vs zfs send -R
• matclarke - Orphaned packages