Red Hat, Fedora, and containers

LWN.net needs you! Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing

DevConf.cz is an annual conference in Brno, the second-largest city of the Czech Republic. While the conference is open to the public, both as attendees and as speakers, the majority of the 1600 people at the conference were Red Hat staff. As a first-time attendee and newly minted Red Hat employee, it felt like DevConf was a Red Hat internal technical conference that the public was invited to.

Fortunately, the company supports and contributes to enough different open-source projects to make for an interesting and varied program. Topics covered included Fedora, Project Atomic, OpenShift, CentOS, OpenStack, JBoss, and many smaller projects. And, thanks to the Red Hat engineering center in Brno, presentations tended to be technically detailed.

If there was a theme to this year's conference, though, it was Linux containers. Session after session either discussed technology and techniques for containers, or demonstrated other tools in conjunction with container infrastructures. This included sessions on immutable infrastructure, containerizing distributions, and the future of Fedora.

Red Hat containers with Project Atomic

Pretty much every company in the world of software today has a container story and even container products; Red Hat is no exception. Most of Red Hat's container efforts are funneled through Project Atomic, which is an umbrella project for multiple open-source tools and platforms. Among these are Atomic Host, the Atomic Developer Bundle, RPM-OSTree, as well as Atomic.app and the Nulecule specification. Red Hat staff also contribute to Docker, Kubernetes, and other third-party container tools, and the company is a member of the Open Container Initiative.

Atomic Host is probably the most visible of these efforts. Not really a project in itself, Atomic Host is a re-distribution, or "spin", of each of the three Red Hat platforms: Fedora, CentOS, and Red Hat Enterprise Linux (RHEL). Atomic Host is meant as a "thin OS" that only has the tools required to be a container-management platform, with the idea that all applications are installed as containers instead of packages. Its own software is managed through RPM-OSTree (see below), and it includes distributions of popular container tools such as Kubernetes, Docker, Flannel, and etcd. Atomic Host can be considered an alternative to CoreOS's Tectonic.

The Atomic Developer Bundle (ADB) is a virtual machine full of tools for building application container images. Packaged as a Vagrant virtual machine, it's intended to make it easy for developers on Windows and Mac laptops to get into container-based development. The ADB is less important on the Linux desktop, where all of these tools are available natively. The project serves the same purpose as the Docker Toolbox.

Another problem Red Hat engineers wanted to solve was to provide the ability to bundle important metadata and even multi-container applications without attaching them to a specific orchestration system. The Nulecule specification, and its sole implementation, Atomic.app, is meant as an answer to this. Packaging containers with structured metadata allows the Atomic.app software to generate code for any of several orchestration systems, such as Kubernetes or Mesos.

While many of the products based on these projects are still under heavy development, Red Hat has released an all-new version of their Platform-as-a-Service cloud, OpenShift, which is built on top of Project Atomic components. The new containerized infrastructure was praised at the conference as being more scalable, manageable, and faster than the previous design. OpenShift is available as an open-source stack in the OpenShift Origin project.

While it was already apparent that Red Hat was involved in Linux containers, DevConf.cz made it clear how central containers are to its plans for the OS in the future. Various presentations explained how the staff and executives plan to make use of container technologies and ideas to change how Linux is distributed and deployed. One of these ideas is "immutable infrastructure."

Immutable infrastructure and OSTree

Two presenters, Adam Miller of Fedora Engineering and Colin Walters of Platform Engineering, each explained in their presentations how the move to containerization, coupled with other new ideas, will change things.

In "The Magical Future", Adam Miller explained the idea of "immutable infrastructure". This has two principles, he said: full automation and the immutability of each component, which doesn't change from development through testing to deployment. The idea is that everything you deploy should be a "build artifact" that doesn't get configured at runtime. Instead, all configuration management happens at build time, the only exception being differences between development and production environments. Even those should be handled by read-only configurations; one per environment.

"The idea is that you don't configure services in the environment, you configure and re-deploy," explained Miller.

This principle of immutability is familiar in the Linux container world. Miller introduced the idea of an "immutable OS", based on RPM-OSTree on top of Atomic Host. OSTree, introduced in 2011, manages a root filesystem in a way that is similar to Git commits, where all files in a commit change at once. RPM-OSTree, created in 2014, takes this concept and applies it to RPMs and sets of RPMs. This makes updates to the Linux server "atomic" in nature, meaning they update or roll back as a unit.

The Atomic Host versions of Fedora and CentOS work on this principle of immutability by using RPM-OSTree. Instead of updating individual packages on each host, each server is copied from a "parent" image in the same way that Docker containers are copied from a filesystem image. No local changes to installed software are possible, only updates to synchronize with changes in the parent image.

Walters expanded on this in his session about "Containerizing the Distribution." Walters was recently involved in building OpenShift version 3, which required a total overhaul of the platform to use containerization and immutability. According to him, while containers have given us "new boxes", they haven't really changed how we build software.

Walters then demonstrated building a container using RPM-OSTree instead of "docker pull". Each "branch" of the OSTree represents a package, and can be treated like a "layer" of a container image. He said that this approach is superior because it makes it much easier to maintain common library packages across a container infrastructure. Right now, this approach is working only using a hacked version of the Mock build-and-chroot tool.

Walters also did a demonstration of container image creation starting from RPMs. Again using OSTree tools, he showed creating a Docker image directly from RPMs and then pushing it to a shared registry. Ideally, he explained, the registry would be on shared storage so container images could simply be mounted by the individual servers.

Containerization and the future of Fedora

While the presentation by Denise Dumas, VP of Platform Engineering at Red Hat, and Mathew Miller, Fedora Project Leader, wasn't marked as a keynote on the program, it might as well have been. Dumas's half of the talk was full of grand ideas and ended with a call to action. In it, she presented her vision of a new direction for the Fedora project, which unsurprisingly involves containers.

Part of the Fedora project values are the "four Fs": Freedom, Friends, Features, and First. To that, Dumas would like to add one more: "Faster". Increasingly, she explained, users are looking to rapid delivery of applications from their platform. She wants Fedora to help Red Hat find ways to deliver an OS that is faster, smaller, and more modular, while still being more secure.

"Two releases a year used to be fast," she said. "We want to be moving continuously, we want to be DevOps." From her perspective, Red Hat does a lot to support Fedora. In return, she wants Fedora to be the place where it can experiment with what it means to be an OS in 2016, or in 2020. Dumas claimed that "disruption is coming," and that Fedora should be part of it instead of being disrupted by it.

A lot of her vision for Fedora centers around containerization and immutability. This includes both Fedora Atomic Host and the new Fedora Atomic Workstation. The latter uses OSTree and xdg-app, a tool for sandboxing GUI applications, to deploy images and updates to developer workstations. Specific applications could be installed as containers, as they are on Atomic Host. Right now Atomic Workstation is still in early alpha stages, with plans for a prototype of Fedora 24.

Miller agreed with Dumas, saying "Fedora Atomic is the future of the operating system [...] Look at RancherOS. We need to be in the forefront of that." Most of his portion of the presentation reviewed the last year of Fedora's progress. Fedora had contributions from around 2000 people last year, of whom about 35% are Red Hat staff and the rest are from outside. Fedora Workstation, a variant of the distribution aimed at software developers, has been popular. So has the network install option for new machines.

He also showed the statistics from the various "spins" of Fedora, such as KDE, Security Lab, and Robotics. Fedora KDE has been popular since it was the first version of the OS to include Plasma 5. Other spins aren't used by as many people, but are the primary OS for their audiences. For example, Fedora Robotics has been the OS of choice for the DARPA-sponsored robot soccer matches.

The distribution is also replacing the venerable X server with the new Wayland graphics server. That will happen in either Fedora 24 or 25, depending on readiness.

Miller also mentioned the Fedora Layered Docker Image Build Service, which was explained in more detail in a later presentation by Fedora Release Engineer Dennis Gilmore. The new service will provide standard builds for images to be distributed as Fedora applications. In Fedora 24, the service will be simply available, but by Fedora 25 it is expected that some applications will be distributed as containers instead of as traditional packages. The project will also be creating its own container-image registry to support this.

More to come

Of course, not everything on the road to Red Hat containers has been a smooth ride. Dan Walsh detailed some of the technical conflicts with Docker, in his presentation "Systemd vs. Docker", which will be covered in a later article.

DevConf.cz was full of all sorts of other interesting projects thanks to presentations by what seemed like half of Red Hat's engineering staff. This also included the Cockpit server management GUI, the FreeIPA identity management server, a new configuration management tool, the CentOS build service, and more. Look for further coverage in the coming weeks.

Index entries for this article
GuestArticles	Berkus, Josh
Conference	DevConf.cz/2016

---

(Log in to post comments)