SELinux RPM bug, QA tasks during pre-F21 quiet time

So, as you may well have read by now, over the weekend a rather unfortunate bug in Fedora 20 cropped up. A stray change in an selinux-policy update, intended for Rawhide, mistakenly wound up in F20, and caused package transactions to fail once it had been installed, if SELinux was in enforcing mode.

This was particularly unfortunate as SELinux has been vastly improved in the last half dozen or so releases compared to many people's memories of it, and I hope this doesn't lead to a revival of the old 'everyone disable SELinux!' meme. It really was an unfortunate accident - the SELinux maintainers have a strict policy that stable release updates should only loosen policy, never tighten it, and this was purely an erroneous change.

From a QA perspective, obviously it's bad that we didn't catch this. Unfortunately this kind of 'delayed action' bug is rather tricky to catch in testing, because everything appears to be fine when you install the update, and it's not at all 'obvious' to anyone that an selinux-policy update might break package installs. So a few people install the update, everything appears to work fine, it fixes their bugs, they +1 the update, it gets queued for release...and then we realize we're boned.

If you're affected by the bug, the commonbugs note will help you out in fixing it up, and we're really very sorry for the inconvenience.

Longer term, automated testing should help us resolve this. We already have the proposed test case filed in Taskotron's tracker. But right now we're still full steam ahead on getting Taskotron itself up and running with the old AutoQA tests re-implemented in a sufficiently complete state to be useful for the F21 cycle, which has been our goal for a while. Once we're there, this will be one of the highest priority new test cases to write, as it will catch serious bugs that are difficult to catch with manual testing.

Aside from that...

I wrote up a post to test@ suggesting some things that QA folks can be working on during the current 'quiet' period, while Fedora 20 is released but the plans for Fedora 21 are not yet finalized (aside from the Taskotron work, of course). There are a lot of things that can usefully be done, from updates-testing work on F20 and F19 through Rawhide testing to creating new test cases and improving the release criteria and validation test cases ahead of F21. So if you're looking for something to do, dive in!

I also transferred a bunch of AutoQA test case suggestions from autoqa trac over to Taskotron phab, so they're ready once the team has time for working on new tests.

Also had a bit of time to play with my new hobby - as I mentioned on G+, I've been getting into OpenStreetMap mapping. I'm working on putting together a mass import of the City of Vancouver's open data street address dataset, which would give OSM street addresses for the whole of the city. It's been made rather easier by the previous work done by a local OSM contributor, Paul Norman, who's done some previous mass imports in the area and provided some useful tooling and figured out some of the legal issues (and has been kindly helping me out, so big thanks to him). I'm using his fork of the ogr2osm tool and basing my translation script loosely on the one he wrote for Surrey's street name data, and I've contacted the city to ask for a clarification that it's OK to use the dataset for OSM purposes - I'm hoping to be able to put the project together quite quickly after that. It's been a fun little side project, and I managed to do some stuff with regexs in Python, so...yay me?

Comments

Anon wrote on 2014-01-23 16:51:
I tried to say something but it was too late ( https://admin.fedoraproject.org/updates/FEDORA-2014-0806/selinux-policy-3.12.1-116.fc20 - yes you have Anonymous Testers). There were seventeen hours between seeing a comment in bugzilla saying that package could be tested and my leaving feedback (I'm in Europe too)... Seeing how that vote turned out I just don't think you can expect regular (non Red Hat) folk to have QA turnarounds that fast. Next time I'm leaving it to the experts...
adamw wrote on 2014-01-23 17:25:
Actually, none of the people who left the +3 feedback is a Red Hatter. cicku is Remi Collet, wadenton is W. Andrew Denton the reporter of https://bugzilla.redhat.com/show_bug.cgi?id=1050210 (not a RH email address), and arnom is Arno Mayrhofer from https://bugzilla.redhat.com/show_bug.cgi?id=1049801#c8 (also not an RH email address). correction: someone pointed out Remi does work for RH, I'd somehow forgotten. However, testing updates is not his job. What tends to happen with SELinux updates is that they fix a lot of bugs - and remember, when a bugfix update is sent out, a message is automatically posted to each bug it's listed as fixing. So anyone who's keen on one of those bugs and happens to be online may well go out and grab the update direct from Koji, so it gets a lot of very fast testing...
Kevin Kofler wrote on 2014-01-24 00:04:
I don't agree at all with the conclusions you are drawing. I think that quite the opposite, this regression (and also the login-related one in Rawhide that's still unfixed!) are definite reasons to disable SELinux once and for all: https://lists.fedoraproject.org/pipermail/devel/2014-January/194384.html (As for the QA issue, we need direct stable pushes back to get regression fixes out as quickly as possible. But in this case, the best measure to ensure quality is to prevent the whole class of issues from happening in the first place by disabling SELinux.)
ktdreyer wrote on 2014-01-30 00:33:
cicku is not actually Remi. It's Christopher Meng. https://fedoraproject.org/wiki/User:Cicku