Oh hi! Come in. I didn't see you there. Don't mind me, I'm just rewriting the entire damn wiki while pretending I know how to code stuff and doing stupid tricks with Wiki templates. I'll try and get to a summary of my interesting week tomorrow.

In the mean time! Today is Virtualization Test Day! You're gonna wanna be there, because virtualization is awesome, Fedora virtualization is doubly awesome, and Fedora 21 virtualization basically oozes awesome so bad the city's writing it a ticket right now. And on the off-chance it isn't, you can file some bugs and have eager developers fix them right away(*), so it's basically a win win.

So! Come join us (I say 'us', but after I get done with this monumental wiki slash-n-burn I'm going to sleep for a week) in #fedora-test-day on Freenode IRC to help you out with testing and discuss any bugs you come across. If you don’t know how to use IRC, you can read these instructions, or just use WebIRC.

(*) fix not guaranteed

Hi folks! Well, your humble monkey has been spending more time than he strictly ought to pretending he's a web developer this week, but Fedora 21 QA rolls on regardless. We have had some major headaches with getting Fedora 21 Alpha built, but as of this morning we finally have a release candidate. Network installs still don't pick up the mirrors properly by default, but the images are now doing the right thing, and the remaining problem is in MirrorManager. This means that once release engineering get MirrorManager pointed in the right direction, the RC1 netinst images will (should...) 'magically' start working out of the box.

With that wrinkle noted, we need to run all the usual validation testing on this compose (and any future RCs), so please do help out if you can! It'd be nice to finally sign off on the Alpha release this week.

Aside from Alpha release validation, we have two Test Days this week. Today is Cockpit Test Day. Cockpit is a 'server user interface' - it gives you an overview and control panel for a set of server machines, essentially. It's a key part of the Fedora Server Product that is one of the main components of the Fedora 21 release. It's a very new and shiny technology, so it needs a lot of testing - it'll be fun to play with, and we really need to get it into shape. All good reasons to join #fedora-test-day on Freenode IRC and join in. If you don’t know how to use IRC, you can read these instructions, or just use WebIRC.

Thursday (2014-09-18) will be ARM Test Day. Fedora 21 is shaping up to be another awesome release for ARM, with more support for more devices than ever. By now half the world must have one or more ARM devices lying around cluttering up the place, so if you have one that Fedora targets, please do come out and help us check that everything is in order for Fedora 21! Again, you can join #fedora-test-day on Freenode IRC to take part, and again, if you don’t know how to use IRC, you can read these instructions, or just use WebIRC.

Yup, I finally got around to putting together a new build of Fedlet, my Fedora remix for Intel Bay Trail tablets like the Dell Venue 8 Pro, Asus T100, Lenovo Miix 2 and others.

The new build has a current Fedora 21 userland and a 3.16 kernel, which brings some substantial hardware support improvements. Check out all the details over at the Fedlet page and grab the new release if you like.

The github repo has been updated to match the new build.

Yup, it's another FreeIPA post!

One thing you might notice if you move your main user account to FreeIPA is that your client systems don't consider the user to be an 'administrator' for polkit (formerly known as PolicyKit) purposes. That is, when you run anything that uses PolicyKit for privilege escalation, you are prompted for the root password, not your user's password.

By default, PolicyKit considers all members of the 'wheel' user group to be administrators. We can see this in /etc/polkit-1/rules.d/50-default.rules:

polkit.addAdminRule(function(action, subject) {
    return ["unix-group:wheel"];
});

If your user is managed exclusively in FreeIPA, they're unlikely to be considered a member of the local system 'wheel' group. My first attempt to fix this involved creating a sort of clone of the 'wheel' group on my FreeIPA server and adding my account to that. This exposed a couple of interesting bugs in polkit but also caused the FreeIPA and polkit devs to break out words like 'completely wrong' and 'idiot', so I don't do that any more. The 'right' way to do it is to create a new group on the FreeIPA server which isn't trying to shadow/copy/override a group on the clients. I called mine 'sysadmins' and let FreeIPA generate one of its extremely high GIDs for it. Add all the user accounts you want to have admin privileges on your client machines to this group.

Then you add a file to /etc/polkit-1/rules.d on all your clients which will be evaluated before 50-default.rules (this is very important - polkit's behaviour is that it evaluates addAdminRule() functions in order until one returns a valid result and then ignores the others, so if your rules file comes after 50-default.rules, it will never take effect) which tells polkit to treat users of that group as administrators instead:

polkit.addAdminRule(function(action, subject) {
    return ["unix-group:sysadmins"];
});

I called mine /etc/polkit-1/rules.d/40-happyassassin.rules, following my own naming policy, but so long as it comes before 50-default.rules you'll be fine. Now any user in the FreeIPA-managed 'sysadmins' group will be considered an admin user by polkit on all systems with that rules file. You can of course use the same group for FreeIPA sudo rules, and then sudo and polkit admin privileges will be managed together.

You could tweak the rules file as you like, of course, and do other stuff with the mechanism. If you have a hybrid setup and you also want users in the local system's wheel group to be considered admins, you'd want the rule to return ["unix-group:sysadmins", "unix-group:wheel"] (I think that's the syntax, anyway). You can set up PolicyKit rules files to grant all sorts of privileges in different cases to different FreeIPA-managed user or group accounts; just remember that it's a bad idea to have collisions between group names and GIDs in FreeIPA and on your local systems, keep them separate.

Of course in a properly-managed domain you'd want to use ansible or Puppet or whatever to distribute the rules file to clients when they're set up.

You may have noticed I've posted a bit about FreeIPA recently, and also when I first set it up.

Other smart folks have blogged about using FreeIPA before (note some of those links are old and refer to bugs that have long since been fixed, and workflows that have been long since improved - the FreeIPA guide for Fedora 18 is probably the best entry point at present, though it misses improvements between 3.1.x from F18 and 3.3.x in F20). But one thing I didn't explain very well yet, and I'm not sure any of the other posts entirely covers, is why you'd want to run FreeIPA in the first place.

I didn't have much of an answer at the time I first installed it - I was trying to fix a single specific problem, and didn't really have a conception of what else it would give me. It's something that's particular unclear if you're just an amateur hobbyist, not a sysadmin in a large environment. Why would you care about FreeIPA if all you have is <10 boxes and one real user?

Believe it or not, there are a ton of good reasons. I've really enjoyed having FreeIPA set up since I got it working, and wouldn't go back to living without it.

Let's recap briefly what FreeIPA is for. Very approximately, it's a centralized identity store and access control system for your network. Instead of creating user accounts and configuring authentication and access control system by system, using passwd files and ssh keys and various other mechanisms, you set up a single server which knows who exists, manages their passwords, manages authentication and decides who gets to connect to what. (If you know what Active Directory is, it's basically that, only it's on Linux and written by Red Hat so of course it's much better).

You might think, like I did, that this isn't much of a big deal. You know your account name, you know your password. You set up your password every time you set up a new box, copy some ssh keys around, and you're fine, right?

Well...kinda, yeah. When's the last time you changed your password? It's kind of a pain in the ass, right? You have to remember to change it everywhere. Probably not worth bothering with, anyway. It's a pretty strong password.

Well, yeah, it's pretty strong, but then you probably had to type it into that Windows terminal in a hotel that one time. Did you check all the USB ports? Want to bet much on the hotel 'sysadmin''s security procedures?

So the most obvious thing a centrally managed system gets you is, change your password once, it's changed. On every system enrolled in your FreeIPA domain. That makes it a lot easier to change passwords, so you're going to do it more often, so you're not going to get hacked by the bellhop of that hotel.

That's just the start of it, though. FreeIPA also does access control. You can set up admin users of various types and split up admin permissions between them on a network-wide basis - maybe your personal account needs to be root on your client systems, but doesn't really need to be an IPA admin, or be the admin user for your server boxes? Split that stuff up, improve security. You can trivially set up guest accounts, for instance, that can get a login on all or some of your client boxes but can't get in to any others or do various other things, like sudo.

Ah, sudo. You can manage your sudo policy centrally in FreeIPA. That's really damn useful. It requires a bit of one-time manual configuration when you enrol a new client still in 3.x, unfortunately, but I hope they make that go away soon. And of course the policy can be fine-grained and complex, like your account can sudo anything without a password on (unimportant hosts) but can only sudo these commands with a password on (important hosts).

We're still just scratching the surface, though!

One of the really nice things FreeIPA does is give you a Kerberos setup that works out of the box.

(waits for screaming to die down)

Yes, that's about how I feel about Kerberos too. No, don't run away. That's why FreeIPA is great - you don't really have to completely understand Kerberos to use it. Heck, I still don't really understand Kerberos and I'm using it. You just need to follow a few monkey guides here and there.

The reason people are always telling you to use Kerberos is it does single sign-on, and yes, that's a great reason and really cool the first time you see it working. There's another reason which gets slightly buried in the detail, though, and that is that well-configured Kerberos means the box running the service you're accessing never has to see your password. Not in the clear, not hashed, not in a TLS session, not at all. Clients talk to the Kerberos server (so, the FreeIPA server in this case), and so do the app host boxes - the two don't talk directly, and the client never sends its password directly to the server. There are Kerberos methods which involve the client sending a password to the host and the host verifying it with the FreeIPA server, but that's not the best way.

The next plan I have for my setup is to make the Kerberos server accessible from the public internet, and enable 2FA. The result of that will be that whether I'm on my local network or not, I can auth to my server and then get SSO to all my enabled services (including email), using 2FA. That's pretty neat!

FreeIPA also provides all sorts of other services, almost as a coincidence. It kinda needs to handle TLS certificates as a consequence of its intended capabilities, for example - so as a byproduct, you can very easily issue a certificate for any of your FreeIPA-enrolled machines, signed by the FreeIPA domain CA, and you have nice centralized control over them - issue them, revoke them, renew them, track them, and all your enrolled systems will trust them (so long as you follow the instructions in my last post, until FreeIPA 4.x arrives). Obviously you'll need to get your certificate issued by a public CA for services you intend to expose to the general public, but for ones that will only be accessed by your own machines that are enrolled in your FreeIPA domain, this is easier, cheaper and probably better (as you control the whole chain).

It really is an awesome system, both in capability and implementation - I've had very few problems with it since I deployed it, it rapidly promoted itself to the mental category of 'things that I almost unconsciously expect to just keep working'. I'd keep a local root password on all your systems - I use randomly-generated passwords stored in a password manager - for emergency recovery purposes, but you'll likely never or very rarely have to use it. In case you're worried about offline operation, don't - SSSD caches credentials for use locally until you can connect to the server again (so, for instance, you can always log in to your laptop when you can't access your network or the internet at all).

If you have more than a couple of machines and/or you host a few services for yourself, I'd really recommend looking into it.

I noticed today that when I accessed my FreeIPA server's web interface with Firefox it was considered to have a valid TLS certificate, but when I accessed it with Epiphany (GNOME's "Web" browser) it was not.

After some poking around I developed a suspicion that they were using different CA trust lists. This is where I will handwave madly to avoid an extremely long digression:

HANDWAVE HANDWAVE nss HANDWAVE HANDWAVE PKCS#11 HANDWAVE HANDWAVE p11-kit HANDWAVE HANDWAVE GIO TLS

Very simple executive summary: neither Firefox nor Epiphany nor, probably, any other browser will trust your FreeIPA server's certificate right after you do ipa-client-install, most likely.

If you have a FreeIPA setup, you probably want all your clients to trust the server's CA certificate as hard as possible. ipa-client-install has only recently been modified to add the CA certificate to the Fedora system-wide certificate store. This change is not in Fedora 19 or Fedora 20, to my knowledge. It's also not in RHEL 7. There's a bug report to have it backported to RHEL 7.1.

Prior to that change, it added the server's CA certificate to the NSS store in /etc/pki/nssdb. Now there's something irking me here, which is that so far as I can tell Firefox ought to trust things in this certificate store, but it doesn't actually seem to do so. Epiphany certainly doesn't. I'll update this post if I clear up the Firefox case.

It's relatively easy to verify that your IPA server CA cert is in the NSS store:

[root@adam adamw]# certutil -L -d /etc/pki/nssdb
Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

IPA Machine Certificate - adam.happyassassin.net             u,u,u
IPA CA                                                       CT,C,C
[root@adam adamw]# certutil -L -n "IPA CA" -d /etc/pki/nssdb
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=Certificate Authority,O=HAPPYASSASSIN.NET"
        Validity:
            Not Before: Mon Sep 23 22:17:56 2013
            Not After : Fri Sep 23 22:17:56 2033
        Subject: "CN=Certificate Authority,O=HAPPYASSASSIN.NET"

To check if it's in the system-wide store, you can open the file /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt with Seahorse ("Passwords and Keys").

Adding it to the system-wide store is very easy. ipa-client-install keeps a copy of it at /etc/ipa/ca.crt. So just do something like:

# cp /etc/ipa/ca.crt /etc/pki/ca-trust/source/anchors/ipa-happyassassin.crt
# update-ca-trust

If you have Fedora 20, you can simplify this to:

# trust anchor /etc/ipa/ca.crt

You can pick any filename you like, so long as it's not likely anything else will use it later (and overwrite the file). As long as it's in the right directory, update-ca-trust will pick it up.

After you add your CA cert to the store, Firefox and Epiphany will no longer complain that the certificate is untrusted when you browse to the web interface. Most other Fedora apps should also benefit from this fix.

Hi folks! It's a busy day today in Fedora QA.

First off, the Fedora 21 Alpha release validation process 'officially' kicks off with the release of Alpha TC5. As you can probably tell from the number, we've generated a number of test composes already, but there were major issues with each one so they weren't 'officially' declared (and we didn't create test result pages for them). There are some serious issues with TC5 as well - we know the network install is broken! - but we decided it was time to go ahead and at least announce and run a full validation test on one of the builds. We have made adjustments to the test process to reflect the Fedora.next changes where appropriate, so please double-check test case texts and release page layouts before filing results, and be aware of the new Server results page!

Second - and apologies for the late notice, it was a long weekend here - today is Fedora 21 i18n Test Day. I'm on planes all day, but the great folks from the Fedora i18n/l10n teams will be testing Fedora 21's i18n support over in #fedora-test-day on Freenode IRC. If you don’t know how to use IRC, you can read these instructions, or just use WebIRC. Please come by and help out if you have a few spare minutes! Thanks.

Introduction

I seem to have been setting yak shaving world records all over the place this week. This last one has been particularly fun, though.

I've been working on and off all week on my ownCloud testing deployment kickstarts. These are kickstarts that can be used to deploy completely configured and working (though highly insecure) ownCloud instances with no interaction needed: you can install Fedora 20, 21 or 22 with inst.ks=https://www.happyassassin.net/ks/oc/oc7-httpd-mysql.ks, for instance, and out pops a completely configured ownCloud 7 instance running on Apache with MariaDB as the database. You browse to http://hostname/owncloud and up pops the ownCloud interface (even the OC first time wizard is automated).

I'm kinda proud of these anyway as it's a neat thing, but the actual point was to make it easy for me to run ownCloud smoke tests. Now I actually have the kickstarts for all three databases our packages support (SQLite, MariaDB / MySQL, PostgreSQL) working, I was able to build ownCloud 7.0.2 packages and test both clean deployment and upgrade from 6.x for all three databases in about half an hour. That made me confident enough to submit 7.0.2 as an update for Fedora 20 as well as sending to Fedora 21 and Rawhide. Please do test it out and submit feedback! Take backups first, of course.

That's where the yak shave of the last day and a half kicked off. I thought it'd be nice to ask upstream to promote or at least mention the native Fedora packages instead of or alongside their own OBS packages in the upstream documentation. Of course, that works best if there's a nice attractive page they can link to. So I went to see if we had a Fedora wiki page for ownCloud, and lo and behold, there it is!

It was a bit outdated, though, and missing some useful information I figured it should have, so I sat down to rewrite it. (I've done that now - it should be all nice and updated and comprehensive, do let me know if you see any problems). Now, wouldn't you know it, part of the page dealt with configuring TLS/SSL for Apache. "Huh," I thought, "surely we have a generic page explaining how to do that?"

Guess what? Here's the Fedora wiki Apache page as it existed before I started poking it. I bet you didn't know about that, huh? I actually at first assumed that it was an ancient thing no-one had touched for years and so I hacked into it with wild abandon, but now I see it was only written in May of this year - I hope I didn't hurt any feelings by editing it.

So I spent some time improving (I hope) and extending the Apache page. When it came to the TLS/SSL stuff, somehow - I'm not exactly sure how - I got the idea that the genkey command the previous version of the page recommended using was boring old stuff, and /etc/pki/tls/Makefile was shiny and new. I think it's because the page recommended using openssl directly if you needed to bump the certificate serial number, and the Makefile has a parameter for that, so I just figured, hey, the Makefile must be better, and switched to it.

So last night I finished tweaking the ownCloud and Apache wiki pages, and also the PostgreSQL one, and was feeling pretty pleased with myself. Then this morning I woke up to an innocuous little discussion between Stephen Gallagher and Dan Mossor about the TLS/SSL certificate generation stuff, in relation to the Apache web page (I'd asked a few people to check my changes to it). Dan found make testcert hadn't worked for him (though I think it was just that he had a pre-existing certificate), and there was some discussion of other methods. This caused me to wonder exactly what the deal was with all these different SSL/TLS certificate generation thingies we have anyway, and, well, now it's 4am the next morning.

Fedora / Red Hat TLS/SSL certificate generation tools

First I found out that both /etc/pki/tls/Makefile - shipped as part of the openssl package - and /usr/bin/genkey - shipped as part of the crypto-utils package - are damn ancient, dating back to at least March 2000 (when it was moved into openssl) and November 2000 respectively. /etc/pki/tls/make-dummy-cert, also part of the openssl package, is nearly as venerable, dating back to June 2001. It's not clear from the changelog why it was invented at all.

So we've had (at least, I'm sure there are more) three different Red Hat/Fedora-ish tools for producing TLS/SSL certificates in RHL/Fedora for, oh, a little over 13 years now. Yay duplication of effort!

make-dummy-cert

make-dummy-cert is the simplest: it takes a filename as an argument, and (using openssl) writes a self-signed certificate and the associated private key together into a single file named whatever you told it to call it. That's all it does. In Fedora, since June 2009 (openssl-0.9.8k-6) it produces a 2048-bit key; prior to that it produced a 1024-bit key. The hash algorithm is mostly irrelevant for self-signed certificates, but it uses openssl's default, which was changed from MD5 to SHA-1 in Fedora in 2005 (and is changed to SHA-256 in Fedora 21). Its companion command renew-dummy-cert simply renews the certificate, because hey, it'll expire some time and you'll want to do that. make-dummy-cert doesn't tell OpenSSL what to do about the certificate's serial number; since 0.9.8 in 2004, if not instructed otherwise, OpenSSL has used a large random number. Prior to that, it used 0, so all certificates generated with make-dummy-cert would have had 0 as their serial number.

/etc/pki/tls/Makefile

The Makefile is slightly more complex/powerful. It can be called in different ways to write a private key only, a key and a CSR, or a key and a self-signed certificate - the latter either as separate files or as a single combined file, the way make-dummy-cert does it. You can specify the filenames. Apart from the 'single combined file' case, the key will be saved to /etc/pki/tls/private and the certificate or CSR (if any) to /etc/pki/tls/certs. This is the layout commonly used with the mod_ssl Apache extension, which is powered by OpenSSL (mod_ssl can also use the 'single combined file' format output by make-dummy-cert).

By default it always uses a serial number of 0, which can cause issues if you generate a new certificate for the same site: you can pass it SERIAL=1 (or any other integer) to change the serial number. As with make-dummy-cert, its default key length was changed to 2048 in June 2009 and it uses openssl's default hash function, whatever it is in the version of openssl you have installed when you run it.

/usr/bin/genkey

genkey is more powerful again than the Makefile, and rather different to the other two. It's not a simple non-interactive script, but an interactive (Newt-based) console application - basically a step-through wizard. It has a few different modes you can call with parameters. It can generate a self-signed server certificate, a self-signed CA certificate, or a CSR (but, ironically given its name, not a key alone). And it can generate these either in mod_ssl style - as described above - or in NSS style. NSS stores keys and certificates in separate flat-file databases. When using the mod_nss Apache extension (which is powered, obviously, by NSS) these are typically named /etc/httpd/alias/keyX.db and /etc/httpd/alias/certY.db, where X and Y are numbers (in current Fedora releases, it's usually key3.db and cert8.db, and these are the filenames genkey will use unless instructed otherwise).

Also unlike the other two tools (which use openssl), genkey uses NSS for generating keys and certificates (even when generating 'OpenSSL-compatible' certificates). A program called keyutil is included alongside genkey in the crypto-utils package. It provides (on top of NSS) some functions genkey needs to produce OpenSSL-style certificates which are not available in NSS itself.

genkey explicitly sets the key length (which you choose interactively) on both NSS and OpenSSL paths. On the OpenSSL path, keyutil (not genkey itself) explicitly specifies the hashing function to use for certificate signing, it does not rely on NSS' default. On the NSS path, the default NSS hashing function is used (though one of the patches I've worked on today might change that). Since January 2014 the default key length (the value pre-selected when you enter the screen) in genkey has been 2048-bit, and the hashing function used when signing certificates and CSRs on the OpenSSL path has been SHA-1. Prior to that, the default key length was 1024-bit and the OpenSSL path hashing function was MD5.

Things I did, and thoughts on future improvements

Early as I was figuring all this out, I came across a bug report requesting genkey use SHA-2 hashing functions instead of SHA-1. I'm now reasonably sure it's not that big a deal, but that's only after working through a lot of this stuff, and writing a patch to fix that problem.

I also noticed an odd place in NSS where an MD5 hashing function is specified but never used - I spent quite a long time making sure it really isn't used, and eventually found that someone else had been there before, and confirmed their bug report.

I confirmed a bug report from Kai Engert, who ran into some of this stuff recently too, that the parameter that lets you specify a hashing function for the NSS certutil command is undocumented, which is sure a hurdle when you're trying to patch this kind of thing.

I submitted another documentation bug report to OpenSSL - hasn't shown up anywhere linkable yet - to report that the manpage for the 'req' command claims the default serial number for new self-signed certificates is 0, when it has not been for ten years.

I wrote this damn blog post. ;) I hope it's useful to someone.

It seems silly to me to have two fairly simple scripts for doing certificate generation (in the same package, no less). Makefile is a superset of make-dummy-cert; I think it would probably make sense to turn /etc/pki/tls/make-dummy-cert into a tiny thing that simply calls the Makefile with the correct parameters. Or, you know, we could put a binary script or wrapper around the Makefile in an actual binary directory, I know it's radical, just a thought. We ought to be able to get down to one non-interactive script-y thing and one interactive app, at least.

It does rather amaze me that there's no simple light tool for doing this sort of thing shared across distributions.

I should ask Stephen exactly what he meant about his new method for self-signed certificate generation that reduces the possibility of MITM attacks using stolen self-signed certs.

I now actually more or less understand how x509 public key cryptography works (that is, the 'who signs what and encrypts what and why' of it, I'm not claiming to understand the damn maths). And I know for damn sure I didn't know that this morning. Both of those things feel like worthwhile upgrades.

Tomorrow I'll edit the Apache wiki page back to recommending the use of genkey, and now I'm thoroughly prepared to explain it comprehensively and correctly :) And that should bring this whole shave to an end...

Hey folks! Apologies for the late notice, but this crept up on me while I fiddled with mail server configurations...today (that is, 2014-08-28) is GNOME Test Day! We'll be testing GNOME 3.14 pre-release on Fedora 21 pre-Alpha, so I expect everything will work absolutely perfectly and no-one will find any bugs at all ;)

But seriously! Come on down and test, there are lots of fun new features in GNOME 3.14 to try and both GNOME and Fedora teams could definitely use some exercise on the new bits that'll be included. We have brand-spanking-new nightly live images for i686 and x86_64, and even an ARM image if you're that one person with hardware capable of running GNOME Shell on ARM. Wayland is actually in more-or-less usable shape for Intel video adapters now, so if you have an Intel-based system you can even give that a try!

The wiki page has full instructions, and we're using the handy test day results reporting system so you don't have to hand-edit results into wiki pages, that's so 2012. Friendly folks - both QA folks and GNOME developers - will be standing by all day European and North American time in #fedora-test-day on Freenode IRC to help you out with testing and discuss any bugs you come across. If you don’t know how to use IRC, you can read these instructions, or just use WebIRC. Please come by and help out if you have a few spare minutes! Thanks.

I was looking to add application-specific passwords to my Dovecot-based mail server today. That is, allow login to my mail account using multiple passwords, and create new passwords that would be used only by a particular client. The idea is that you configure, say, each of your laptops and phones to use a different password for logging into your mail account. That way if a particular system is lost or stolen, or you worry the credentials stored on it might have been compromised in any other way, you can revoke just that system's password, without having to change the password used by all your other systems or for interactive login (e.g. via a webmail app). It also means you can change the password you use for interactive login without having to change all the stored passwords on your various devices.

There is a fairly well-known guide to doing this already, written by Daniel Siegel, but it had a couple of problems for me: it's based on virtual users, and it assumes you'll set all the passwords you want to use for a given account in the passwd file that contains the app-specific passwords.

This doesn't really work for my setup. I use real user accounts, and use FreeIPA for authentication / identity management. Basically, I have user foo configured in FreeIPA, my mail server is enrolled as a FreeIPA client, and Dovecot is configured to use the PAM password database and the passwd user database (this is what you get if you use the auth-system.conf.ext file packaged with dovecot).

If you're in a similar situation and want to add application-specific passwords to your existing configuration so you can log in using both the password configured in the local user database or FreeIPA and the application-specific password trick Daniel uses, it turns out it's very simple - all you have to do is add Daniel's passdb directive and the password file. Don't do anything else - don't add any userdb directive. So, just add this stanza to your config:

passdb {
  driver = passwd-file
  args = /etc/dovecot/app_specific_passwd
}

(or whatever filename you want to use, obviously) and populate the file as Daniel recommends but without the first line that sets the main password. e.g.:

mobile:{SHA512-CRYPT}123456789...::::::user=foo
webmail:{SHA512-CRYPT}0123abcd...::::::user=foo
tablet:{SHA512-CRYPT}987654321...::::::user=foo

make sure you get the number of colons in a row right - it's six. Don't literally include the ..., Daniel just used those to indicate the rest of the hashed password (not that anyone would make THAT mistake, cough cough). Generate the hashed passwords with doveadm pw -s SHA512-CRYPT (or whatever crypt scheme you want to use, of course). As long as foo is a valid user so far as your userdb is concerned - that is, when using the passwd userdb, so far as your NSS configuration is concerned, so just /etc/passwd in a simple configuration, but it could also pull from your FreeIPA server or whatever - you'll be able to login as 'mobile', 'webmail' or 'tablet' with the password specified, and you'll wind up authenticated as 'foo'.

I did this by creating a new file /etc/dovecot/conf.d/auth-appspecificpasswd.conf.ext and editing /etc/dovecot/conf.d/10-auth.conf to include it before auth-system.conf.ext:

!include auth-appspecificpasswd.conf.ext
!include auth-system.conf.ext

the result is that the app-specific password file will be tried before PAM - I like it that way around because the file check is much faster and also doesn't generate error messages when it fails. If you do it the other way around, app-specific password logins take longer and spam your logs with failure messages from the PAM check failing before the passwd-file check kicks in.