AdamW on Linux and more (Posts about Mandriva)

Site and blog migration

2020-11-24T00:36:54Z

So I've been having an adventurous week here at HA Towers: I decided, after something more than a decade, I'm going to get out of the self-hosting game, as far as I can. It makes me a bit sad, because it's been kinda cool to do and I think it's worked pretty well, but I'm getting to a point where it seems silly that a small part of me has to constantly be concerned with making sure my web and mail servers and all the rest of it keep working, when the services exist to do it much more efficiently. It's cool that it's still possible to do it, but I don't think I need to actually do it any more.

So, if you're reading this...and I didn't do something really weird...it's not being served to you by a Fedora system three feet from my desk any more. It's being served to you by a server owned by a commodity web hoster...somewhere in North America...running Lightspeed (boo) on who knows what OS. I pre-paid for four years of hosting before realizing they were running proprietary software, and I figured what the hell, it's just a web serving serving static files. If it starts to really bug me I'll move it, and hopefully you'll never notice.

All the redirects for old Wordpress URLs should still be in place, and also all URLs for software projects I used to host here (fedfind etc) should redirect to appropriate places in Pagure and/or Pypi. Please yell if you see something that seems to be wrong. I moved nightlies and testcase_stats to the Fedora openQA server for now; that's still a slightly odd place for them to be, but at least it's in the Fedora domain not on my personal domain, and it was easiest to do since I have all the necessary permissions, putting them anywhere else would be more work and require other people to do stuff, so this is good enough for now. Redirects are in place for those too.

I've been working on all the other stuff I self-host, too. Today I set up all the IRC channels I regularly read in my Matrix account and I'm going to try using that setup for IRC instead of my own proxy (which ran bip). It seems to work okay so far. I'm using the Quaternion client for now, as it seems to have the most efficient UI layout and isn't a big heavy wrapper around a web client. Matrix is a really cool thing, and it'd be great to see more F/OSS projects adopting it to lower barriers to entry without compromising F/OSS principles; IRC really is getting pretty creaky these days, folks. There's some talk about both Fedora and GNOME adopting Matrix officially, and I really hope that happens.

I also set up a Kolab Now account and switched my contacts and calendar to it, which was nice and easy to do (download the ICS files from Radicale, upload them to Kolab, switch my accounts on my laptops and phone, shut down the Radicale server, done). I also plan to have it serve my mail, but that migration is going to be the longest and most complicated as I'll have to move several gigs of mail and re-do all my filters. Fun!

I also refreshed my "desktop" setup; after (again) something more than a decade having a dedicated desktop PC I'm trying to roll without one again. Back when I last did this, I got to resenting the clunky nature of docking at the time, and also I still ran quite a lot of local code compiles and laptops aren't ideal for that. These days, though, docking is getting pretty slick, and I don't recall the last time I built anything really chunky locally. My current laptop (a 2017 XPS 13) should have enough power anyhow, for the occasional case. So I got me a fancy Thunderbolt dock - yes, from the Apple store, because apparently no-one else has it in stock in Canada - and a 32" 4K monitor and plugged the things into the things and waited a whole night while all sorts of gigantic things I forgot I had lying around my home directory synced over to the laptop and...hey, it works. Probably in two months I'll run into something weird that's only set up on the old desktop box, but hey.

So once I have all this wrapped up I'm aiming to have substantially fewer computers lying around here and fewer Sysadmin Things taking up space in my brain. At the cost of being able to say I run an entire domain out of a $20 TV stand in my home office. Ah, well.

Oh, I also bought a new domain as part of this whole thing, as a sort of backup / staging area for transitions and also possibly as an alternative vanity domain. Because it is sometimes awkward telling people yes, my email address is happyassassin.net, no, I'm not an assassin, don't worry, it's a name based on a throwaway joke from university which I probably wouldn't have picked if I knew I'd be signing up for bank accounts with it fifteen years later. So if I do start using it for stuff, here is your advance notice that yeah, it's me. This name I just picked to be vaguely memorable and hopefully to be entirely inoffensive, vaguely professional-sounding, and composed of sounds that are unambiguous when read over an international phone line to a call centre in India. It doesn't mean anything at all.

On inclusive language: an extended metaphor involving parties because why not

2020-06-17T00:57:49Z

So there's been some discussion within Red Hat about inclusive language lately, obviously related to current events and the worldwide protests against racism, especially anti-Black racism. I don't want to get into any internal details, but in one case we got into some general debate about the validity of efforts to use more inclusive language. I thought up this florid party metaphor, and I figured instead of throwing it at an internal list, I'd put it up here instead. If you have constructive thoughts on it, go ahead and mail me or start a twitter thread or something. If you have non-constructive thoughts on it, keep 'em to yourself!

Before we get into my pontificating, though, here's some useful practical resources if you just want to read up on how you can make the language in your projects and docs more inclusive:

To provide a bit of context: I was thinking about a suggestion that people promoting the use of more inclusive language are "trying to be offended". And here's where my mind went!

Imagine you are throwing a party. You send out the invites, order in some hors d'ouevres (did I spell that right? I never spell that right), queue up some Billie Eilish (everyone loves Billie Eilish, it's a scientific fact), set out the drinks, and wait for folks to arrive. In they all come, the room's buzzing, everyone seems to be having a good time, it's going great!

But then you notice (or maybe someone else notices, and tells you) that most of the people at your party seem to be straight white dudes and their wives and girlfriends. That's weird, you think, I'm an open minded modern guy, I'd be happy to see some Black folks and maybe a cute gay couple or something! What gives? I don't want people to think I'm some kind of racist or sexist or homophobe or something!

So you go and ask some non-white folks and some non-straight folks and some non-male folks what's going on. What is it? Is it me? What did I do wrong?

Well, they say, look, it's a hugely complex issue, I mean, we could be here all night talking about it. And yes, fine, that broken pipeline outside your house might have something to do with it (IN-JOKE ALERT). But since you ask, look, let us break this one part of it down for you.

You know how you've got a bouncer outside, and every time someone rolls up to the party he looks them up and down and says "well hi there! What's your name? Is it on the BLACKLIST or the WHITELIST?" Well...I mean...that might put some folks off a bit. And you know how you made the theme of the party "masters and slaves"? You know, that might have something to do with it too. And, yeah, you see how you sent all the invites to men and wrote "if your wife wants to come too, just put her name in your reply"? I mean, you know, that might speak to some people more than others, you hear what I'm saying?

Now...this could go one of two ways. On the Good Ending, you might say "hey, you know what? I didn't think about that. Thanks for letting me know. I guess next time I'll maybe change those things up a bit and maybe it'll help. Hey thanks! I appreciate it!"

and that would be great. But unfortunately, you might instead opt for the Bad Ending. In the Bad Ending, you say something like this:

"Wow. I mean, just wow. I feel so attacked here. It's not like I called it a 'blacklist' because I'm racist or something. I don't have a racist bone in my body, why do you have to read it that way? You know blacklist doesn't even MEAN that, right? And jeez, look, the whole 'masters and slaves' thing was just a bit of fun, it's not like we made all the Black people the slaves or something! And besides that whole thing was so long ago! And I mean look, most people are straight, right? It's just easier to go with what's accurate for most people. It's so inconvenient to have to think about EVERYBODY all the time. It's not like I'm homophobic or anything. If gay people would just write back and say 'actually I have a husband' or whatever they'd be TOTALLY welcome, I'm all cool with that. God, why do you have to be so EASILY OFFENDED? Why do you want to make me feel so guilty?"

So, I mean. Out of Bad Ending Person and Good Ending Person...whose next party do we think is gonna be more inclusive?

So obviously, in this metaphor, Party Throwing Person is Red Hat, or Google, or Microsoft, or pretty much any company that says "hey, we accept this industry has a problem with inclusion and we're trying to do better", and the party is our software and communities and events and so on. If you are looking at your communities and wondering why they seem to be pretty white and male and straight, and you ask folks for ideas on how to improve that, and they give you some ideas...just listen. And try to take them on board. You asked. They're trying to help. They are not saying you are a BAD PERSON who has done BAD THINGS and OFFENDED them and you must feel GUILTY for that. They're just trying to help you make a positive change that will help more folks feel more welcome in your communities.

You know, in a weird way, if our Party Throwing Person wasn't quite Good Ending Person or Bad Ending person but instead said "hey, you know what, I don't care about women or Black people or gays or whatever, this is a STRAIGHT WHITE GUY PARTY! WOOOOO! SOMEONE TAP THAT KEG!"...that's almost not as bad. At least you know where you stand with that. You don't feel like you're getting gaslit. You can just write that idiot and their party off and try and find another. The kind of Bad Ending Person who keeps insisting they're not racist or sexist or homophobic and they totally want more minorities to show up at their party but they just can't figure out why they all seem to be so awkward and easily offended and why they want to make poor Bad Ending Person feel so guilty...you know...that gets pretty tiring to deal with sometimes.

Wifi display (Miracast) support on Linux!

2019-02-21T13:04:11Z

I was really pleased to see this blog post today, because I was looking for exactly this a few weeks ago and was sad to find it didn't exist. Now it does!

Displaying on an external screen via wifi is a really handy thing that is increasingly commonly available. Lately I'm finding most TVs I run across - including hotel room TVs! - have the feature available. For whatever reason I don't really see many people using or talking about it, but it's great - you can watch videos from your phone or laptop without having to carry an HDMI cable around everywhere and hope the inputs aren't blocked or full on the TV.

So I'm looking forward to testing that out, and hope all the bits land in official distribution packages soon :) Thanks Benjamin!

QA protip of the day: make sure your test runner fails properly

2016-12-31T17:44:33Z

Just when you thought you were safe...it's time for a blog post!

For the last few days I've been working on fixing Rawhide packages that failed to build as part of the Python 3.6 mass rebuild. In the course of this, I've been enabling test suites for packages where there is one, we can plausibly run it, and we weren't doing so before, because tests are great and running them during package builds is great. (And it's in the guidelines).

I've now come across two projects which have a unittest-based test script which does something like this:

#!/usr/bin/python3

class SomeTests(unittest.TestCase):
    [tests here]

def main():
    suite = unittest.TestLoader().loadTestsFromTestCase(SomeTests)
    unittest.TextTestRunner(verbosity=3).run(suite)

if __name__ == '__main__':
    main()

Now if you just run this script manually all the time and inspect its output, you'll be fine, because it'll tell you whether the tests passed or not. However, if you try and use it in any kind of automated way you're going to have trouble, because this script will always exit 0, even if some or all the tests fail. This, of course, makes it rather useless for running during a package build, because the build will never fail even if all the tests do.

If you're going to write your own test script like this (which...seriously consider if you should just rely on unittest's 'gathering' stuff instead, or use nose(2), or use pytest...), then it's really a good idea to make sure your test script actually fails if any of the tests fail. Thus:

#!/usr/bin/python3

import sys

class SomeTests(unittest.TestCase):
    [tests here]

def main():
    suite = unittest.TestLoader().loadTestsFromTestCase(SomeTests)
    ret = unittest.TextTestRunner(verbosity=3).run(suite)
    if ret.wasSuccessful():
        sys.exit()
    else:
        sys.exit("Test(s) failed!")

if __name__ == '__main__':
    main()

(note: just doing sys.exit() will exit 0; doing sys.exit('any string') prints the string and exits 1).

Packagers, look out for this kind of bear trap when packaging...if the package doesn't use a common test pattern or system but has a custom script like this, check it and make sure it behaves sanely.

On Snappy and Flatpak: business as usual in the Canonical propaganda department

2016-06-16T16:39:21Z

NOTE: this post is entirely personal. The opinions are my own and do not represent Fedora or Red Hat. The facts, however, are all 100% truthy. ;) Just to make it 100% clear for any visiting journalists etc. who don't know me: I work for Red Hat, on Fedora. I am not unbiased and am not claiming to be, but I am claiming that the concrete stuff I say below is true.

You may have read some stuff this week about an application delivery mechanism called Snappy and how it's going to unite all distributions and kill apt and rpm!

This is, to put it diplomatically, a heaping pile of steaming bullshit. You may not be surprised to learn that said pile has been served by the Canonical press department.

The source of all this press was this gem of a press release, which has been widely covered in a fairly...uncritical way by several outlets. Even Ars Technica, which is usually fairly good at doing actual journalism rather than just unquestioningly paraphrasing press releases, gave it a pretty anodyne write-up.

The press release and the stories together give you the strong impression that this thing called Snappy is going to be the cross-distribution future of application delivery, and it's all ready for use today and lots of major distributions are buying into it. In the press release you'll find stuff like this:

"Developers from multiple Linux distributions and companies today announced collaboration on the “snap” universal Linux package format, enabling a single binary package to work perfectly and securely on any Linux desktop, server, cloud or device."

The stories have headlines like "Adios apt and yum? Ubuntu’s snap apps are coming to distros everywhere" and "Snap Packages Become Universal Binary Format for All GNU/Linux Distributions" (jeez, I particularly love that one).

So what are the problems with this happy-clappy story? Several of them!

First let's be clear: Snappy is a Canonical project. The press release was issued, I think, sort of as if it came from some sort of independent or cross-vendor project, and there's the snapcraft.io site to back up that impression, but every Snappy committer is a Canonical employee, and contributions to Snappy require signing the notorious Canonical CLA:

"Contributions are always welcome! Please make sure that you sign the Canonical contributor licence agreement at http://www.ubuntu.com/legal/contributors"

Now, does Snappy actually have the cross-distribution buy-in that the press release claims (but never outright states) that it has? No. The press release sure sounds superficially impressive:

"Developers from multiple Linux distributions...Snaps now work natively on Arch, Debian, Fedora, Kubuntu, Lubuntu, Ubuntu GNOME, Ubuntu Kylin, Ubuntu MATE, Ubuntu Unity, and Xubuntu...Together, these distributions represent the vast majority of common Linux usage on the desktop, server and cloud."

but it's a pretty big mis-representation. The other distributions cited have not actually declared their support for Snappy and said 'yes, this is how we want applications to be distributed in future'. Canonical employees have independently built and released Snappy packages for those distributions. This is the basis of all the press release's claims. For instance, the Snappy packages for Fedora are in a COPR - COPR is a system like PPA that lets anyone build packages - maintained by a Canonical employee. The sum total of communication between Canonical and Fedora before the release of this press release was that they mailed us asking about the process of packaging snappy for Fedora, and we told them about the main packaging process and COPR. They certainly did not in any way inform Fedora that they were going to send out a press release strongly implying that Fedora, along with every other distro in the world, was now a happy traveler on the Snappy bandwagon.

There is in fact another system with very similar goals, which is now called Flatpak and was previously called xdg-app. To be as fair as I can, I'll say that Flatpak is quite heavily Red Hat influenced: the main author of Flatpak is Alex Larsson, a Red Hat employee. It is not, however, a "Red Hat project" to anything like the extent Snappy is a Canonical project. There are more than 20 other committers to Flatpak, most of whom are not RH employees (and including contributors to several other distributions). Flatpak is not under any corporate CLA. Insofar as Fedora is supporting one of these systems, it's behind Flatpak. No other distribution besides Ubuntu is particularly committed to either system, so far as I can tell. Flatpak and Snappy both began, so far as I could find from internet research, in December 2014. Canonical's press release, of course, doesn't even acknowledge Flatpak's existence...which is kind of par for the PR course, but you'd think at least some journalists might go out and do a bit of independent research.

UPDATE: since writing this post I've also been made aware of another system, AppImage, which has been around somewhat longer than Flatpak or Snappy (though not necessarily their forerunners). I know little about it so I will say little, but one thing to note is that - so I've heard - it does not attempt to do sandboxing like Snappy and Flatpak do, which is a major feature of those two implementations. It's purely an app bundle format. But hey, it's a choice! And it's been around a while!

Neither Snappy nor Flatpak is at all close to being 'done', in the sense of being a credible system for cross-platform app distribution with buy-in from software publishers and distributions. The PR's claim that Snappy enables "a single binary package to work perfectly and securely on any Linux desktop, server, cloud or device" sounds lovely, doesn't it? Let's take a look at the truth. Taking Fedora as an example, the Snappy install instructions for Fedora - go to the Snappy site and click the Fedora logo - say:

# SELinux support is in beta, so on Fedora 24 you currently have to:
sudo setenforce 0

well, that doesn't seem terribly 'secure' or 'perfect' now, does it? Along the same lines, the Fedora packages are actually compiled with Snappy's confinement disabled. Confinement being the entirety of what's supposed to be secure about this form of app distribution. If confinement isn't turned on, you've basically just got a big blob with uncontrolled access to the system. Good luck with that.

AIUI, the builds for other distributions are in similar states.

Note that neither Snappy nor Flatpak can possibly provide meaningful confinement of apps running under X11, as mjg59 illustrated. Flatpak will only provide meaningful confinement with Wayland. Snappy, of course, is designed to work with Mir, though they claim it can/could (not sure which) also work with Wayland. But the point here is that neither Wayland nor Mir is out there in real widespread use by Linux users at present, yet here's Canonical happily glossing over that point while they talk about how Snappy right now allows "a single binary package to work perfectly and securely on any Linux desktop".

At the time this Panglossian PR was sent out, there were exactly two actual useful applications available as snaps: LibreOffice and Krita. Phoronix quickly found that the LibreOffice snap was huge (over 1GB in size) and buggy. The size issue was quite quickly resolved, but this just goes to show that reality is vastly different from Canonical's claims. This stuff is in early Alpha or proof-of-concept state. It is not remotely 'done' and ready for practical use in the real world outside of the very limited contexts where Canonical was already using it.

Neither is Flatpak, of course. But this is why Flatpak's developers have been communicating with technical conference presentations and blog posts and trying to build a dialog with application developers and distributors, rather than issuing press releases trumpeting how great Flatpak is and how it's ready to kill apt RIGHT NOW.

Here's another interesting thing about Snaps: the server end (the 'app store' bit of the equation) is closed source, and Canonical have been refusing to tell anyone how to run their own 'app store' - see the comment from Cassidy James Blaede, of Elementary. If you want to distribute your snaps, your choices are 1) publish it through the Canonical store, entirely under Canonical's control, 2) upload it as a file and tell people to use the CLI to install it, or 3) try to figure out how to reconfigure the snap client to use a different server by reading the source code, then write your own server end from scratch, and tell your users to do that. Hmm.

So: Snappy is, like Flatpak, a heavily-under-development, interesting attempt to provide an app store-like app provision mechanism for Linux. It is not finished, it is not close to finished. It is not independent or cross-distribution, it is entirely controlled by Canonical. It does not have, so far as I can tell, meaningful buy-in from a single major distribution outside of Ubuntu. It does not work properly on other distributions yet and it likely will not do so in the near future.

But apart from that, sure, it's all ready to kill apt and dnf tomorrow!

sigh

Now I'm sure I will get criticized for being mean and nasty and cynical and attacking Canonical instead of being constructive and all they want to do is make things better for everyone, Adam why are you such an ass?

Well, if Canonical actually wanted to work constructively with others, the way to do so would be to talk to them. We have forums for cross-distribution and cross-project collaboration. Lots of them. We have tech conferences where you can go and talk about your project and try to get buy-in for it. Canonical could have come to other distributions and said, hey, how about we all try to get together behind a single format and a common delivery mechanism for this kind of a confined app bundle?

But they didn't. They just decided to send out a wildly misleading press release and actively encourage the specialist press to report that Snappy was all set to take over the world and everyone was super happy with that.

That's not being constructive or working together with others. That's being a bunch of asshats and trying to present the rest of the community with a fait accompli - and notably, a fait accompli in which Canonical holds all the strings (by means of the Canonical CLA controlling contributions to the client end, and the closed source, closed shop server end that is owned entirely by Canonical).

Monospace fonts and freetype emboldening: a follow-up

2015-02-14T16:24:27Z

Last week I wrote about some font stuff, mentioning along the way that it was prompted by an issue in freetype. If a font has no native bold version, Freetype can produce one algorithmically when an app asks for bold text. This mechanism makes the glyphs wider. This is a bit of a problem for monospace (fixed-width) fonts, because the whole point is that their character widths should always be consistent, so blocks of characters will align correctly.

I did actually try and figure out how to make freetype do this - which it calls 'emboldening' - without widening glyphs, but wasn't able to figure it out, so instead I suggested a change to fontconfig which simply disables the emboldening entirely for fixed-width fonts. I sent this upstream. In response, someone very kindly stepped up and figured out how to do it properly: that someone is Raimund Steger, and here's his blog post on the topic.

So if you've been annoyed by this issue, you can build freetype with the patch from his post, and - joy of joys - you'll still get bold characters in Droid Sans Mono or Inconsolata, but they'll no longer be wider than regular ones!

I then worked out that the infamous Infinality had actually come up with something functionally identical in his patch set four years ago, but then changed it to simply disable widening for all fonts, as he apparently prefers it that way. That's obviously a much more 'subjective' change than disabling it for fixed-width fonts only, and it seems like no-one ever quite joined the dots and suggested the more objective improvement upstream. Details on that here, if you're interested.

I suggested to Raimund that he submit the change to freetype, so here's hoping it'll get picked up by upstream and save anyone else from encountering this in future!

EDIT: A further follow-up: Raimund submitted his patch to freetype, but it was rejected by Werner Lemberg on the grounds that it is actually most appropriately fixed 'one level higher than FreeType', i.e. in the platform/toolkit layer, i.e. in Cairo and Qt (to cover the major cases). So I've filed a Cairo bug. I tested KDE's behaviour, and it seems...interesting: I've filed a bug there as well.

FreeIPA: setting polkit / PolicyKit rules for users (make your user a polkit administrator on your clients)

2014-09-09T12:49:59Z

Yup, it's another FreeIPA post!

One thing you might notice if you move your main user account to FreeIPA is that your client systems don't consider the user to be an 'administrator' for polkit (formerly known as PolicyKit) purposes. That is, when you run anything that uses PolicyKit for privilege escalation, you are prompted for the root password, not your user's password.

By default, PolicyKit considers all members of the 'wheel' user group to be administrators. We can see this in /etc/polkit-1/rules.d/50-default.rules:

polkit.addAdminRule(function(action, subject) {
    return ["unix-group:wheel"];
});

If your user is managed exclusively in FreeIPA, they're unlikely to be considered a member of the local system 'wheel' group. My first attempt to fix this involved creating a sort of clone of the 'wheel' group on my FreeIPA server and adding my account to that. This exposed a couple of interesting bugs in polkit but also caused the FreeIPA and polkit devs to break out words like 'completely wrong' and 'idiot', so I don't do that any more. The 'right' way to do it is to create a new group on the FreeIPA server which isn't trying to shadow/copy/override a group on the clients. I called mine 'sysadmins' and let FreeIPA generate one of its extremely high GIDs for it. Add all the user accounts you want to have admin privileges on your client machines to this group.

Then you add a file to /etc/polkit-1/rules.d on all your clients which will be evaluated before 50-default.rules (this is very important - polkit's behaviour is that it evaluates addAdminRule() functions in order until one returns a valid result and then ignores the others, so if your rules file comes after 50-default.rules, it will never take effect) which tells polkit to treat users of that group as administrators instead:

polkit.addAdminRule(function(action, subject) {
    return ["unix-group:sysadmins"];
});

I called mine /etc/polkit-1/rules.d/40-happyassassin.rules, following my own naming policy, but so long as it comes before 50-default.rules you'll be fine. Now any user in the FreeIPA-managed 'sysadmins' group will be considered an admin user by polkit on all systems with that rules file. You can of course use the same group for FreeIPA sudo rules, and then sudo and polkit admin privileges will be managed together.

You could tweak the rules file as you like, of course, and do other stuff with the mechanism. If you have a hybrid setup and you also want users in the local system's wheel group to be considered admins, you'd want the rule to return ["unix-group:sysadmins", "unix-group:wheel"] (I think that's the syntax, anyway). You can set up PolicyKit rules files to grant all sorts of privileges in different cases to different FreeIPA-managed user or group accounts; just remember that it's a bad idea to have collisions between group names and GIDs in FreeIPA and on your local systems, keep them separate.

Of course in a properly-managed domain you'd want to use ansible or Puppet or whatever to distribute the rules file to clients when they're set up.

FreeIPA for amateurs: why?

2014-09-07T23:53:19Z

You may have noticed I've posted a bit about FreeIPA recently, and also when I first set it up.

Other smart folks have blogged about using FreeIPA before (note some of those links are old and refer to bugs that have long since been fixed, and workflows that have been long since improved - the FreeIPA guide for Fedora 18 is probably the best entry point at present, though it misses improvements between 3.1.x from F18 and 3.3.x in F20). But one thing I didn't explain very well yet, and I'm not sure any of the other posts entirely covers, is why you'd want to run FreeIPA in the first place.

I didn't have much of an answer at the time I first installed it - I was trying to fix a single specific problem, and didn't really have a conception of what else it would give me. It's something that's particular unclear if you're just an amateur hobbyist, not a sysadmin in a large environment. Why would you care about FreeIPA if all you have is <10 boxes and one real user?

Believe it or not, there are a ton of good reasons. I've really enjoyed having FreeIPA set up since I got it working, and wouldn't go back to living without it.

Let's recap briefly what FreeIPA is for. Very approximately, it's a centralized identity store and access control system for your network. Instead of creating user accounts and configuring authentication and access control system by system, using passwd files and ssh keys and various other mechanisms, you set up a single server which knows who exists, manages their passwords, manages authentication and decides who gets to connect to what. (If you know what Active Directory is, it's basically that, only it's on Linux and written by Red Hat so of course it's much better).

You might think, like I did, that this isn't much of a big deal. You know your account name, you know your password. You set up your password every time you set up a new box, copy some ssh keys around, and you're fine, right?

Well...kinda, yeah. When's the last time you changed your password? It's kind of a pain in the ass, right? You have to remember to change it everywhere. Probably not worth bothering with, anyway. It's a pretty strong password.

Well, yeah, it's pretty strong, but then you probably had to type it into that Windows terminal in a hotel that one time. Did you check all the USB ports? Want to bet much on the hotel 'sysadmin''s security procedures?

So the most obvious thing a centrally managed system gets you is, change your password once, it's changed. On every system enrolled in your FreeIPA domain. That makes it a lot easier to change passwords, so you're going to do it more often, so you're not going to get hacked by the bellhop of that hotel.

That's just the start of it, though. FreeIPA also does access control. You can set up admin users of various types and split up admin permissions between them on a network-wide basis - maybe your personal account needs to be root on your client systems, but doesn't really need to be an IPA admin, or be the admin user for your server boxes? Split that stuff up, improve security. You can trivially set up guest accounts, for instance, that can get a login on all or some of your client boxes but can't get in to any others or do various other things, like sudo.

Ah, sudo. You can manage your sudo policy centrally in FreeIPA. That's really damn useful. It requires a bit of one-time manual configuration when you enrol a new client still in 3.x, unfortunately, but I hope they make that go away soon. And of course the policy can be fine-grained and complex, like your account can sudo anything without a password on (unimportant hosts) but can only sudo these commands with a password on (important hosts).

We're still just scratching the surface, though!

One of the really nice things FreeIPA does is give you a Kerberos setup that works out of the box.

(waits for screaming to die down)

Yes, that's about how I feel about Kerberos too. No, don't run away. That's why FreeIPA is great - you don't really have to completely understand Kerberos to use it. Heck, I still don't really understand Kerberos and I'm using it. You just need to follow a few monkey guides here and there.

The reason people are always telling you to use Kerberos is it does single sign-on, and yes, that's a great reason and really cool the first time you see it working. There's another reason which gets slightly buried in the detail, though, and that is that well-configured Kerberos means the box running the service you're accessing never has to see your password. Not in the clear, not hashed, not in a TLS session, not at all. Clients talk to the Kerberos server (so, the FreeIPA server in this case), and so do the app host boxes - the two don't talk directly, and the client never sends its password directly to the server. There are Kerberos methods which involve the client sending a password to the host and the host verifying it with the FreeIPA server, but that's not the best way.

The next plan I have for my setup is to make the Kerberos server accessible from the public internet, and enable 2FA. The result of that will be that whether I'm on my local network or not, I can auth to my server and then get SSO to all my enabled services (including email), using 2FA. That's pretty neat!

FreeIPA also provides all sorts of other services, almost as a coincidence. It kinda needs to handle TLS certificates as a consequence of its intended capabilities, for example - so as a byproduct, you can very easily issue a certificate for any of your FreeIPA-enrolled machines, signed by the FreeIPA domain CA, and you have nice centralized control over them - issue them, revoke them, renew them, track them, and all your enrolled systems will trust them (so long as you follow the instructions in my last post, until FreeIPA 4.x arrives). Obviously you'll need to get your certificate issued by a public CA for services you intend to expose to the general public, but for ones that will only be accessed by your own machines that are enrolled in your FreeIPA domain, this is easier, cheaper and probably better (as you control the whole chain).

It really is an awesome system, both in capability and implementation - I've had very few problems with it since I deployed it, it rapidly promoted itself to the mental category of 'things that I almost unconsciously expect to just keep working'. I'd keep a local root password on all your systems - I use randomly-generated passwords stored in a password manager - for emergency recovery purposes, but you'll likely never or very rarely have to use it. In case you're worried about offline operation, don't - SSSD caches credentials for use locally until you can connect to the server again (so, for instance, you can always log in to your laptop when you can't access your network or the internet at all).

If you have more than a couple of machines and/or you host a few services for yourself, I'd really recommend looking into it.

Yak shaving, again: ownCloud 7 for Fedora 20, and TLS/SSL certificate generation in Fedora and Red Hat

2014-08-30T06:19:34Z

Introduction

I seem to have been setting yak shaving world records all over the place this week. This last one has been particularly fun, though.

I've been working on and off all week on my ownCloud testing deployment kickstarts. These are kickstarts that can be used to deploy completely configured and working (though highly insecure) ownCloud instances with no interaction needed: you can install Fedora 20, 21 or 22 with inst.ks=https://www.happyassassin.net/ks/oc/oc7-httpd-mysql.ks, for instance, and out pops a completely configured ownCloud 7 instance running on Apache with MariaDB as the database. You browse to http://hostname/owncloud and up pops the ownCloud interface (even the OC first time wizard is automated).

I'm kinda proud of these anyway as it's a neat thing, but the actual point was to make it easy for me to run ownCloud smoke tests. Now I actually have the kickstarts for all three databases our packages support (SQLite, MariaDB / MySQL, PostgreSQL) working, I was able to build ownCloud 7.0.2 packages and test both clean deployment and upgrade from 6.x for all three databases in about half an hour. That made me confident enough to submit 7.0.2 as an update for Fedora 20 as well as sending to Fedora 21 and Rawhide. Please do test it out and submit feedback! Take backups first, of course.

That's where the yak shave of the last day and a half kicked off. I thought it'd be nice to ask upstream to promote or at least mention the native Fedora packages instead of or alongside their own OBS packages in the upstream documentation. Of course, that works best if there's a nice attractive page they can link to. So I went to see if we had a Fedora wiki page for ownCloud, and lo and behold, there it is!

It was a bit outdated, though, and missing some useful information I figured it should have, so I sat down to rewrite it. (I've done that now - it should be all nice and updated and comprehensive, do let me know if you see any problems). Now, wouldn't you know it, part of the page dealt with configuring TLS/SSL for Apache. "Huh," I thought, "surely we have a generic page explaining how to do that?"

Guess what? Here's the Fedora wiki Apache page as it existed before I started poking it. I bet you didn't know about that, huh? I actually at first assumed that it was an ancient thing no-one had touched for years and so I hacked into it with wild abandon, but now I see it was only written in May of this year - I hope I didn't hurt any feelings by editing it.

So I spent some time improving (I hope) and extending the Apache page. When it came to the TLS/SSL stuff, somehow - I'm not exactly sure how - I got the idea that the genkey command the previous version of the page recommended using was boring old stuff, and /etc/pki/tls/Makefile was shiny and new. I think it's because the page recommended using openssl directly if you needed to bump the certificate serial number, and the Makefile has a parameter for that, so I just figured, hey, the Makefile must be better, and switched to it.

So last night I finished tweaking the ownCloud and Apache wiki pages, and also the PostgreSQL one, and was feeling pretty pleased with myself. Then this morning I woke up to an innocuous little discussion between Stephen Gallagher and Dan Mossor about the TLS/SSL certificate generation stuff, in relation to the Apache web page (I'd asked a few people to check my changes to it). Dan found make testcert hadn't worked for him (though I think it was just that he had a pre-existing certificate), and there was some discussion of other methods. This caused me to wonder exactly what the deal was with all these different SSL/TLS certificate generation thingies we have anyway, and, well, now it's 4am the next morning.

Fedora / Red Hat TLS/SSL certificate generation tools

First I found out that both /etc/pki/tls/Makefile - shipped as part of the openssl package - and /usr/bin/genkey - shipped as part of the crypto-utils package - are damn ancient, dating back to at least March 2000 (when it was moved into openssl) and November 2000 respectively. /etc/pki/tls/make-dummy-cert, also part of the openssl package, is nearly as venerable, dating back to June 2001. It's not clear from the changelog why it was invented at all.

So we've had (at least, I'm sure there are more) three different Red Hat/Fedora-ish tools for producing TLS/SSL certificates in RHL/Fedora for, oh, a little over 13 years now. Yay duplication of effort!

`make-dummy-cert`

make-dummy-cert is the simplest: it takes a filename as an argument, and (using openssl) writes a self-signed certificate and the associated private key together into a single file named whatever you told it to call it. That's all it does. In Fedora, since June 2009 (openssl-0.9.8k-6) it produces a 2048-bit key; prior to that it produced a 1024-bit key. The hash algorithm is mostly irrelevant for self-signed certificates, but it uses openssl's default, which was changed from MD5 to SHA-1 in Fedora in 2005 (and is changed to SHA-256 in Fedora 21). Its companion command renew-dummy-cert simply renews the certificate, because hey, it'll expire some time and you'll want to do that. make-dummy-cert doesn't tell OpenSSL what to do about the certificate's serial number; since 0.9.8 in 2004, if not instructed otherwise, OpenSSL has used a large random number. Prior to that, it used 0, so all certificates generated with make-dummy-cert would have had 0 as their serial number.

`/etc/pki/tls/Makefile`

The Makefile is slightly more complex/powerful. It can be called in different ways to write a private key only, a key and a CSR, or a key and a self-signed certificate - the latter either as separate files or as a single combined file, the way make-dummy-cert does it. You can specify the filenames. Apart from the 'single combined file' case, the key will be saved to /etc/pki/tls/private and the certificate or CSR (if any) to /etc/pki/tls/certs. This is the layout commonly used with the mod_ssl Apache extension, which is powered by OpenSSL (mod_ssl can also use the 'single combined file' format output by make-dummy-cert).

By default it always uses a serial number of 0, which can cause issues if you generate a new certificate for the same site: you can pass it SERIAL=1 (or any other integer) to change the serial number. As with make-dummy-cert, its default key length was changed to 2048 in June 2009 and it uses openssl's default hash function, whatever it is in the version of openssl you have installed when you run it.

`/usr/bin/genkey`

genkey is more powerful again than the Makefile, and rather different to the other two. It's not a simple non-interactive script, but an interactive (Newt-based) console application - basically a step-through wizard. It has a few different modes you can call with parameters. It can generate a self-signed server certificate, a self-signed CA certificate, or a CSR (but, ironically given its name, not a key alone). And it can generate these either in mod_ssl style - as described above - or in NSS style. NSS stores keys and certificates in separate flat-file databases. When using the mod_nss Apache extension (which is powered, obviously, by NSS) these are typically named /etc/httpd/alias/keyX.db and /etc/httpd/alias/certY.db, where X and Y are numbers (in current Fedora releases, it's usually key3.db and cert8.db, and these are the filenames genkey will use unless instructed otherwise).

Also unlike the other two tools (which use openssl), genkey uses NSS for generating keys and certificates (even when generating 'OpenSSL-compatible' certificates). A program called keyutil is included alongside genkey in the crypto-utils package. It provides (on top of NSS) some functions genkey needs to produce OpenSSL-style certificates which are not available in NSS itself.

genkey explicitly sets the key length (which you choose interactively) on both NSS and OpenSSL paths. On the OpenSSL path, keyutil (not genkey itself) explicitly specifies the hashing function to use for certificate signing, it does not rely on NSS' default. On the NSS path, the default NSS hashing function is used (though one of the patches I've worked on today might change that). Since January 2014 the default key length (the value pre-selected when you enter the screen) in genkey has been 2048-bit, and the hashing function used when signing certificates and CSRs on the OpenSSL path has been SHA-1. Prior to that, the default key length was 1024-bit and the OpenSSL path hashing function was MD5.

Things I did, and thoughts on future improvements

Early as I was figuring all this out, I came across a bug report requesting genkey use SHA-2 hashing functions instead of SHA-1. I'm now reasonably sure it's not that big a deal, but that's only after working through a lot of this stuff, and writing a patch to fix that problem.

I also noticed an odd place in NSS where an MD5 hashing function is specified but never used - I spent quite a long time making sure it really isn't used, and eventually found that someone else had been there before, and confirmed their bug report.

I confirmed a bug report from Kai Engert, who ran into some of this stuff recently too, that the parameter that lets you specify a hashing function for the NSS certutil command is undocumented, which is sure a hurdle when you're trying to patch this kind of thing.

I submitted another documentation bug report to OpenSSL - hasn't shown up anywhere linkable yet - to report that the manpage for the 'req' command claims the default serial number for new self-signed certificates is 0, when it has not been for ten years.

I wrote this damn blog post. ;) I hope it's useful to someone.

It seems silly to me to have two fairly simple scripts for doing certificate generation (in the same package, no less). Makefile is a superset of make-dummy-cert; I think it would probably make sense to turn /etc/pki/tls/make-dummy-cert into a tiny thing that simply calls the Makefile with the correct parameters. Or, you know, we could put a binary script or wrapper around the Makefile in an actual binary directory, I know it's radical, just a thought. We ought to be able to get down to one non-interactive script-y thing and one interactive app, at least.

It does rather amaze me that there's no simple light tool for doing this sort of thing shared across distributions.

I should ask Stephen exactly what he meant about his new method for self-signed certificate generation that reduces the possibility of MITM attacks using stolen self-signed certs.

I now actually more or less understand how x509 public key cryptography works (that is, the 'who signs what and encrypts what and why' of it, I'm not claiming to understand the damn maths). And I know for damn sure I didn't know that this morning. Both of those things feel like worthwhile upgrades.

Tomorrow I'll edit the Apache wiki page back to recommending the use of genkey, and now I'm thoroughly prepared to explain it comprehensively and correctly :) And that should bring this whole shave to an end...

Fedora 21 GNOME Test Day today!

2014-08-28T02:22:30Z

Hey folks! Apologies for the late notice, but this crept up on me while I fiddled with mail server configurations...today (that is, 2014-08-28) is GNOME Test Day! We'll be testing GNOME 3.14 pre-release on Fedora 21 pre-Alpha, so I expect everything will work absolutely perfectly and no-one will find any bugs at all ;)

But seriously! Come on down and test, there are lots of fun new features in GNOME 3.14 to try and both GNOME and Fedora teams could definitely use some exercise on the new bits that'll be included. We have brand-spanking-new nightly live images for i686 and x86_64, and even an ARM image if you're that one person with hardware capable of running GNOME Shell on ARM. Wayland is actually in more-or-less usable shape for Intel video adapters now, so if you have an Intel-based system you can even give that a try!

The wiki page has full instructions, and we're using the handy test day results reporting system so you don't have to hand-edit results into wiki pages, that's so 2012. Friendly folks - both QA folks and GNOME developers - will be standing by all day European and North American time in #fedora-test-day on Freenode IRC to help you out with testing and discuss any bugs you come across. If you don’t know how to use IRC, you can read these instructions, or just use WebIRC. Please come by and help out if you have a few spare minutes! Thanks.