Gentoo Ought to be About Choice
“Gentoo is about choice.” We’ve said it so often that it seems like we just don’t bother to say it any more. However, with some of the recent conflicts on the lists (which I’ve contributed to) and indeed across the FOSS community at large, I think this is a message that is worth repeating… Read the rest of this entry »
Running cron jobs as units automatically
I just added sys-process/systemd-cron to the Gentoo repository. Until now I’ve been running it from my overlay and getting it into the tree was overdue. I’ve found it to be an incredibly useful tool.
All it does is install a set of unit files and a crontab generator. The unit files (best used by starting/enabling cron.target) will run jobs from /etc/cron.* at the appropriate times. The generator can parse /etc/crontab and create timer units for every line dynamically.
Note that the default Gentoo install runs the /etc/cron.* jobs from /etc/crontab, so if you aren’t careful you might end up running them twice. The simplest solutions this are to either remove those lines from /etc/crontab, or install systemd-cron using USE=etc-crontab-systemd which will have the generator ignore /etc/crontab and instead look for /etc/crontab-systemd where you can install jobs you’d like to run using systemd.
The generator works like you’d expect it to – if you edit the crontab file the units will automatically be created/destroyed dynamically.
One warning about timer units compared to cron jobs is that the jobs are run as services, which means that when the main process dies all its children will be killed. If you have anything in /etc/cron.* which forks you’ll need to have the main script wait at the end.
On the topic of race conditions, each cron.* directory and each /etc/crontab line will create a separate unit. Those units will all run in parallel (to the extent that one is still running when the next starts), but within a cron.* directory the scripts will run in series. That may be a bit different from some cron implementations which may limit the number of simultaneous jobs globally.
All the usual timer unit logic applies. stdout goes to the journal, systemctl list-timers shows what is scheduled, etc.
Quick systemd-nspawn guide
I switched to using systemd-nspawn in place of chroot and wanted to give a quick guide to using it. The short version is that I’d strongly recommend that anybody running systemd that uses chroot switch over – there really are no downsides as long as your kernel is properly configured.
Quick EC2 Backups with Duplicity
I’ve been doing online EC2 backups on my Gentoo box for a while, but switched to Duplicity a few months ago and have been very happy with the results. Setting this up took some trial and error, so I figured I’d share my config in case others find it useful. But first, here’s why I switched… Read the rest of this entry »
btrfs and snapper with portage on Gentoo
This is just a quick share-a-recipe post to introduce snapper to anybody who hasn’t heard of it, and explain how to use it.
Snapper is a utility that manages btrfs snapshots. One of the nice features of btrfs is that snapshots are cheap (virtually instant, and consume space only as changes accumulate), and easy to access. Snapper allows you to automatically create and manage them based on time, events, manual action, etc.
Once snapper is set up you can display a list of snapshots. I have 10 hourly snapshots, 10 daily snapshots, and snapshots from before/after each emerge. I can diff them, browse them, etc. Btrfs snapshots can be browsed right from the filesystem, so if I nuke /etc/passwd I can always do a cp /.snapshots/1875/snapshot/etc/passwd /etc/passwd to restore one from a few hours before (though I do also have /etc in a git repo).
Snapper is currently available in the sunrise overlay – I won’t spend time on how to set that up/etc. Also, I’ve had time-based snapshots running for a while now and my memory is hazy as to whether I had to do anything to get those working – it just requires sticking some scripts in /etc/cron.*/ and creating a config file containing your policies.
What I did want to post is a recipe for getting pre/post-emerge snapshots working. All you need to do is add some lines to /etc/portage/bashrc:
case "${EBUILD_PHASE}" in
preinst)
DESC="${CATEGORY}/${PF}"
NUMBER=`snapper create -t pre -p -d "${DESC}"`
;;
postinst)
snapper create -t post --pre-number $NUMBER -d "${DESC}"
;;
esac
One of these days I might clean this up a bit and move it to portage.
The Balance of Power in Gentoo
The recent concerns with the request to re-populate QA have re-opened a debate that is a few years old now. I’ve already made some specific recommendations on the lists, but I wanted to step back and explain why I feel the way I do.
Gentoo’s system of governance has some internal ironies – ones which occasionally even lead to calls to establish a benevolent dictator position. I think the mistake that Gentoo makes is that the problem is perceived as being democracy, when in reality the problem is with competing governance bodies with differing constituencies…
MythTV 0.26 In Portage
Well, all of MythTV 0.26 is now in portage, masked for testing for a few days.
If anyone is interested now is a good time to give it a try and report any issues you find. If all is quiet the masks will come off and we’ll be up-to-date (including all patches up to a few days ago).
Thanks to all who have contributed to the 0.26 bug. I can also happily report that I’m running Gentoo on my mythtv front-end, which should help me with maintaining things. MiniMyth is a great distro, but it has made it difficult to keep the front- and back-ends in sync.
Gentoo and Copyright Assignments
A topic that has been fairly quiet for years has roared into life on a few separate occasions in the last month within the Gentoo community: copyright assignments. The goal of this post is to talk a little about the issues around these as I see them. I’ll state upfront that I’m not married to any particular approach.
But first, I think it is helpful to consider why this topic is flaring up. The two situations I’m aware of where this has come up in the last month or so both concern contributions (willing or not) from outside of Gentoo. One concerns a desire to be able to borrow eclass code from downstream distros like Exherbo, and the other is the eudev fork. In both cases the issue is with the general Gentoo policy that all Gentoo code have a statement at the top to the effect of “Copyright 2012 Gentoo Foundation.”
Now, Diego has already blogged about some of the issues created by this policy already, and I want to set that aside for the moment. Regardless of whether the Foundation can lay claim to ownership of copyright on past contributions, the question remains, should Gentoo aim to have copyright ownership (or something similar) for all Gentoo work be owned by the Foundation?
Right now I’m reaching out to other free software organizations to understand their own policies in this area. Regardless of whether we want to have Gentoo own our copyrights or not there are still legal questions around what to put on that copyright line, especially when a file is an amalgamation of code originated both inside and outside of Gentoo, perhaps even by parties who are hostile to the effort. I can’t speak for the Trustees as a whole, but I suspect that after gathering info we’ll try to have some open discussion on the lists, and perhaps even have a community-wide vote before making new policy. I don’t want to promise that – in fact I’d recommend that any community-wide vote be advisory only unless a requirement for supermajority were set, as I don’t want half the community up in arms because a 50.1% majority passed some highly unpopular policy.
So, what are some of the directions in which Gentoo might go? Why might we choose to go in these directions? Below I outline some of the options I’m aware of:
Maintain the status quo
We could just leave the issue of copyright assignment somewhat ambiguous as has been done. If Gentoo were forced to litigate over copyright ownership right now an argument could be made that because contributors willingly allowed us to stick that copyright notice on our files and made their contribution with the knowledge of our policies, that they have given implicit consent to our doing so.
I’m not a big fan of this approach – it has the virtue of requiring less work, but really has no benefits one way or the other (and as you’ll read below their are benefits from declaring a position one way or the other).
This requires us to come up with a policy around what goes on the copyright notice line. I suspect that there won’t be much controversy for Gentoo-originated work like most ebuilds, as there isn’t much controversy over them now. However, for stuff like eudev or code borrowed from other projects this could get quite messy. With no one organization owning much of the code in any file the copyright line could become quite a mess.
Do not require copyright assignment
We could just make it a policy that Gentoo would aim to own the name Gentoo, but not the actual code we distribute. This would mean that we could freely accept any code we wished (assuming it was GPL or CC BY-SA compatible per our social contract). This would also mean that Gentoo as an organization would find it difficult to pursue license violations, and future relicensing would be rather difficult.
From an ability to merge outside code this is clearly the preferred solution. This approach still carries all the difficulties of managing the copyright notice, since again no one organization is likely to hold the majority of copyright ownership of our files. Also, if we were to go this route we should strongly consider requiring that all contributions be licensed under GPL v2+, and not just GPL v2. Since Gentoo would not own the copyright if we ever wanted to move to a newer GPL version we would not have the option to do so unless this were done.
Gentoo would still own the name Gentoo, so from a branding/community standpoint we’d have a clear identity. If somebody else copied our code wholesale the Foundation couldn’t do much to prevent this unless we retroactively asked a bunch of devs to sign agreements allowing us to do so, but we could keep an outside group from using the name Gentoo, or any of our other trademarks.
Require copyright assignment
We could make it a policy that all contributions to Gentoo be made in conjunction with some form of copyright assignment, or contributor licensing agreement. I’ll set aside for now the question of how exactly this would be implemented.
In this model Gentoo would have full legal standing to pursue license violations, and to re-license our code. In practice I’m not sure how likely we’d actually be to do either. The copyright notice line would be easy to manage, even if we made the occasional exception to the policy, since the exceptions could of course be managed as exceptions as well. Most likely the majority of the code in any file would only be owned by a few entities at most.
The downside to this approach is that it basically requires turning away code, or making exceptions. Want to fork udev? Good luck getting them to assign copyright to Gentoo.
There could probably be blanket exceptions for small contributions which aren’t likely to create questions of copyright ownership. And we could of course have a transition policy where we accept outside code but all modifications must be Gentoo-owned. Again, I don’t see that as a good fit for something like eudev if the goal is to keep it aligned with upstream.
I think the end result of this would be that work that is outside of Gentoo would tend to stay outside of Gentoo. The eudev project could do its thing, but not as a Gentoo project. This isn’t necessarily a horrible thing – OpenRC wasn’t really a “Gentoo project” for much of its life (I’m not quite sure where it stands at the moment).
Alternatives
There are in-between options as well, such as encouraging the voluntary assignment/licensing of copyright (which is what KDE does), or dividing Gentoo up into projects we aim to own or not. So, we might aim to own our ebuilds and the essential eclasses and portage, but maybe there is the odd eclass or side project like eudev that we don’t care about owning. Maybe we aim to own new contributions (either all or most).
There are good things to be said for a KDE-like approach. It gives us some of the benefits of attribution, and all of the benefits of not requiring attribution. We could probably pursue license violations vigorously as we’d likely hold control of copyright over the majority of our work (aside from things like eudev – which obviously aren’t our work to begin with). Relicensing would be a bit of a pain – for anything we have control over we could of course relicense it, but for anything else we’d have to at least make some kind of effort to get approval. Legally that all becomes a murky area. If we were to go with this route again I’d probably suggest that we require all code to be licensed GPL v2+ or similar just to give us a little bit of automatic flexibility.
I’m certainly interested in feedback from the Gentoo community around these options, things I hadn’t thought of, etc. Feel free to comment here or on gentoo-nfp.
The Dark Side of Quality
Voltaire once said that the best is the enemy of the good. I think that there are few places where one can see as many abuses of quality as you’ll find in many FOSS projects, including Gentoo.
Often FOSS errs on the side of insufficient quality. Developers who are scratching itches don’t always have incentive to polish their work, and as a result many FOSS projects result in a sub-optimal user experience. In these cases “good enough” is standing in the way of “the best.”
However, I’d like to briefly comment on an opposite situation, where “the best” stands in the way of “good enough.” As an illustrative example, consider the excellent practice of removing bundled libraries from upstream projects. I won’t go on about why this is a good thing – others have already done so more extensively. And make no mistake – I agree that this is a good thing, the following notwithstanding.
The problem comes when things like bundled libraries become a reason to not package software at all. Two examples I’m aware of where this has happened recently are media-sound/logitechmediaserver-bin and media-gfx/darktable. In the former there is a push to remove the package due to the inclusion of bundled libraries. In the latter the current version is lagging somewhat because while upstream actually created an ebuild, it bundles libraries. Another example is www-client/chromium, which still bundles libraries despite a very impressive campaign by the chromium team to remove them.
The usual argument for banning packages containing bundled libraries is that they can contain security problems. However, I think this is misleading at best. If upstream bundles zlib in their package we cry about potential security bugs (and rightly so), however, if upstream simply writes their own compression functions and includes them in the code, we don’t bat an eyelash, even though this is more likely to cause security problems. The only reason we can complain about zlib is BECAUSE it is extensively audited, making it easy to spot the security problems. We’re not reacting to the severity of problems, but only to the detectablity of them.
Security is a very important aspect of quality, but any reasonable treatment of security has to consider the threat model. While software that bundles a library is rightfully considered “lower” in quality than one that does not, what matters more is whether this is a quality difference that is meaningful to end users, and what their alternatives are. If the alternative for the user is to just install the same software with the same issues, but from an even lower quality source with no commitment to security updates, then removing a package from Gentoo actually increases the risks to our users. This is not unlike the situation that exists with SSL, where an unencrypted connection is presented to the user as being more secure than an SSL connection with a self-signed certificate, when this is not true at all. If somebody uses darktable to process photos that they take, then they’re probably not concerned with a potential buffer overflow in a bundled version of dcraw. If the another user operated a service that accepted files from strangers on the internet, then they might be more concerned.
What is the solution?: A policy that gives users reasonably secure software from a reputable source, with clear disclosure. We should encourage devs to unbundle libraries, consider bugs pointing out bundled libraries valid, accept patches to unbundle libraries when they are available, and add an elog notice to packages containing bundled libraries in the interest of disclosure. Packages with known security vulnerabilities would be subject to the existing security policy. However, developers would still be free to place packages in the tree that contain bundled libraries, unmasked, and they could be stabilized. Good enough for upstream should be good enough for Gentoo (again, baring specific known vulnerabilities), but that won’t stop us from improving further.
Gentoo EC2 Tutorial / Bootstrapping
I want to accomplish a few things with this post.
First, I’d like to give more attention to the work recently done by edowd on Bootstrapping Gentoo in EC2.
Second, I’d like to introduce a few enhancements I’ve made on these (some being merged upstream already).
Third, I’d like to turn this into a bit of a tutorial into getting started with EC2 as well since these scripts make it brain-dead simple.
Rich0's Gentoo Blog 