CoreOS: A different kind of Linux distribution

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

By Jake Edge
April 9, 2014

Truly new Linux distributions don't spring up that often any more. New derivatives from a (typically) well-known parent distribution are common enough, but purpose-built distributions meant to scratch a particular itch (or set of itches) don't appear all that frequently. CoreOS is one of the few. While CoreOS is originally based on Chrome OS (another of the few), it has a much different target than that mobile-focused distribution; CoreOS calls itself: "Linux for Massive Server Deployments".

The design of the system is meant to avoid some of the pain from one of the more difficult parts of server maintenance: updates. By using a double-buffering technique on each full-system update, any problems can be fixed by simply rolling back the update. In addition, all of the applications are self-contained in their own Docker container, which decouples application and operating system updates.

While CoreOS runs just fine on a single machine, it is clearly targeted at clusters. It comes with a number of features, such as service discovery, a distributed key-value store for shared configuration (etcd), distributed locking, and master election, that make it well-suited for clusters. The fleet orchestration tool teams up with systemd and etcd to deploy applications (i.e. Docker containers) throughout a cluster. It can maintain a certain number of the applications running at any given time, starting or restarting the applications as required.

It is also open-source software (Apache v2 for the CoreOS-developed pieces), available on GitHub. The latest version as of this writing is CoreOS 273.0.0, which uses kernel 3.13.6 and Docker 0.9.1. Unlike many other enterprise-targeted distributions, it runs on a fairly recent kernel.

Getting started with CoreOS is pretty easy. It comes packaged for a handful of cloud providers and virtualization solutions, with QEMU/KVM being an obvious, easy choice for those running another Linux distribution. For that, it only requires downloading a disk image of the operating system and a shell script to start it up with QEMU.

Starting up the system using the script was straightforward and did not require root privileges on my Fedora 20 box (relieving my concern about running a shell script as root that I had just snagged off the internet "somewhere"). It brings up a CoreOS system on the "console" (the shell where it was started) but, even though it gives a login prompt, you cannot log into it that way. There is only one user that can log into the system, "core", and that user has no password. The wrapper script will pick up public SSH keys from the account where the system is started (or the key location can be specified on the command line), so connecting is done via a command like:

    ssh -l core -p 2222 localhost

Obviously the SSH daemon runs on port 2222. Setting things up using SSH keys neatly avoids the hassles of a default password.

As one would guess, the CoreOS system looks much like a regular Linux system once you log in, but there are differences. For one thing, CoreOS is engineered to be much smaller than "standard" Linux installations—roughly 50% of the RAM usage at boot, for example, according to the CoreOS home page. It doesn't come with a package manager like Yum or APT and relies, instead, on Docker for installing fully self-contained applications.

The configuration for those applications is stored in the etcd distributed key-value store. It provides an HTTP interface, so keys can be easily set and retrieved using simple tools like curl. Things like database names and credentials could be stored that way and retrieved by any other host in the CoreOS cluster. So, when a new database-using service starts up (either for scaling up the capacity of the application or to restart a failed instance), it can query for that information. Other types of configuration information can similarly be distributed to multiple hosts.

Access to etcd is simple, all CoreOS systems will find it at http://127.0.0.1:4001, so the Docker containers only need to "hardcode" that value and will be able to access the store from any member of the cluster. For obvious reasons, etcd access should not be exposed outside of the cluster. Beyond using a firewall (which is, of course, recommended), client certificates can be used to restrict etcd traffic. Access control lists are on the drawing board as well.

In addition to just changing and querying values from etcd, interested services can also listen to it to find out about changes to particular values. That allows for actions like service discovery or global configuration changes of various sorts. It is perhaps interesting to note that etcd is written in the Go language.

Docker provides isolation in the form of Linux containers. Each Docker container runs as its own virtual system, but all of them on a single host run under the same kernel. That is one of the main distinctions between container virtualization and hardware virtualization. For the latter, there is a separate kernel (or other operating system) running for each virtual system. Beyond just packaging these containers, Docker provides other features that will be useful to those building large, distributed systems using CoreOS. For example, Docker provides Git-like facilities for committing changes to running containers. Those changes can be stored locally or pushed to a public or private repository for others to use.

CoreOS uses systemd as part of its distributed init system. It is deeply integrated into the distribution for a number of reasons, including boot speed, socket activation, and the systemd journal. Systemd is used to start and stop the Docker containers which hold various pieces of the distributed system, much like systemd handles managing regular services on other distributions.

For those who are putting together large, distributed systems—web applications being a prime example—CoreOS would appear to have a lot of interesting functionality. It should allow applications of that type to grow and shrink as needed with demand, as well as provide a stable platform where upgrades are not a constant headache. For "massive server deployments", CoreOS, or something with many of the same characteristics, looks like the future.

(Log in to post comments)