I shall be off work for the next week (plus weekends), so you'll probably see things slow down in this blog too. But before I go away, I thought I'd post an update on some of the exciting stuff that's been happening in QA and BugZappers lately.

The long-running project to use the severity and priority fields in Bugzilla is finally at an end. I've sent an announcement to -devel-announce which has not yet been approved, but the meat is that the severity field will be set by triagers during triage, according to the policy in the bug workflow page, to indicate how significant the bug is. The priority field will be reserved for maintainers to use, if they so choose, to organize their workload as they choose.

We have been working, lately, to improve and create pages dedicated to explaining how to debug issues in particular components, and how to provide appropriate information when filing bugs on those components. This grew out of a proposal made by Jóhann Guðmundsson to improve bug reporting. These pages are centrally linked to from the main page on proper bug reporting procedure and can also be found in the 'Debugging' category. One example of a page that's been substantially revised is the page on X.org.

Finally, Matej Cepl and the rest of the Desktop team have been working on a project called Fit and Finish to work on the small details that make a polished desktop experience. This will be based around a series of Test Days focussing on the user experience rather than specific developer-driven 'features'. I've been helping to set up the test days and it's also acting as a good test of the SOP I wrote for running test days, which we intended to help with projects like this where a different group takes ownership of its own track of Test Days. Their first event on display configuration is happening this coming Tuesday (in #fedora-fitandfinish on IRC), so please come out to support these events!

It's good to see all these exciting projects going on, and I hope they'll reflect in the quality and innovation in future Fedora releases, starting with Fedora 12. See you all in a week or so.

I come at the Poulsbo problem from, pretty much, a plain old 'frustrated user with the time to try and build random things' perspective. The redoubtable Matthew Garrett comes at it from the 'equally frustrated, very smart kernel developer' perspective. His post on the mess is well worth reading for a different angle on the whole deal.

An update on the situation, BTW: I haven't worked on cleaning up the packages for the driver for F11, as I said I would, because of some later experience with it. For both me and a couple of others who've successfully built and got it working, it causes the system to hang solid, quite reliably, about half an hour after you turn it on. Which is a bit of a big problem. With benchmarking, it also seems to be slow even for basic 2D operations, almost as slow as the vesa driver; so it's not providing much of a benefit besides the RandR 1.2 support for external monitors.

An email I got from within Intel hints that some of the proprietary special sauce is required even for basic 2D acceleration to work right. I can't find any indication of this in the logs, but it may be that - even though I sucked as much of the proprietary stuff as I could into the packages - I missed some, or it somehow doesn't quite play right, and hence the 2D acceleration isn't working. I'm not sure.

So for now I'm just using the vesa driver and sucking it up. Sigh. BTW, through this whole three week UK trip I'll be doing all my work from the P. So far this is going pretty well, despite the slow graphics!

I may be missing something here (be great if I am), but it seems to me that the content of the Ubuntu Wiki - which contains some great stuff - is not licensed under one of the common 'shareable' licenses, like CC, GFDL or OPL. Neither the front page nor any of the several random pages of content I checked has a license declaration that I could find, and the "Legal information" link in the footer takes you to the general ubuntu.com legal info page. So as far as I can tell, the license on that page - which is basically "for anything other than personal non-commercial use, apply to Canonical" - applies. That's a bit unfortunate, and against the open source spirit of collaboration, if it's true. I had a couple of people check my sanity on this one, and asked in #ubuntu-doc, and no-one could find anything to the contrary.

I got onto this by looking at the Ubuntu debugging procedures page, which is great. We're looking at improving the Fedora wiki pages on what information to include when reporting bugs on particular components, and it would make sense to just re-use the Ubuntu community's work here rather than spend time re-do it all ourselves which could more usefully be spent elsewhere. But if I'm right, we can't.

If anyone knows that I'm wrong here (or can explain why there isn't a less restrictive license, if I'm right), please do comment. Thanks!

So, this being the day we're leaving for England - and a Sunday to boot - the fan in my server machine naturally figured today would be just about the perfect day to drop dead. And, of course, my server machine is a Shuttle ST62K, which uses a heatpipe / rear heatsink / fan / fan shroud arrangement. That can only accommodate a 80mm fan that's 15mm thick.

You want to know how easy it is to buy a 80x80x15mm fan in Vancouver on a Sunday afternoon? I'll tell you - not freaking easy.

After running around town like a headless chicken for an hour or two, I gave up, came home, stole an 80x80x25mm case fan from my partner's PC, opened up the server, yanked the whole fan shroud arrangement, zip tied the 'new' fan to the heatsink and called it a day.

So far, it's holding up. If it falls apart and cooks itself while I'm in England, this web site will be down until I get back. Nothing much I can do about that. So if you come here tomorrow and it ain't working, at least you'll know why!

Well, it's hard to kick the habit of a lifetime, and I still watch the happily meaningless charts on the Distrowatch front page.

Fedora has just gone past Mint into third place. As I recall, that's the first time in really rather a long time it's been up there.

Mandriva's in sixth, now, which is around where it's been (6th or 7th) for a while.

I've been keeping busy working on our Grand Plans for the Fedora 12 cycle, and also working on congruity and navit. I'm now chasing up a bug with navit/freetype which is preventing recent builds of Navit from SVN from working on Fedora 11. It's not yet entirely clear if it's freetype or GCC at fault. Read all about it here if you're morbidly curious, or happen to be a code / compiler / optimization / valgrind whiz and feel a strange urge to help.

I shall be in England for three weeks or so from this Sunday. Probably won't be much difference observed, but I just thought I'd note the fact.

I'm planning to throw a Rawhide survey up on the forums soon, as part of our ongoing 'get more people running Rawhide' plan. Other plans in this area are afoot. Oh, yes, we have plans a-plenty. Muahahah!

This morning I learned some new stuff. That's always fun.

I have a Logitech Harmony remote control. This is one of those smart ones which can be programmed to control just about anything, and control multiple things to do a single 'operation' (e.g. I can press the 'watch TV' button, and it switches on my cable box, my TV and my amp, and switches the amp and the TV to the appropriate input settings). Neat stuff. It's programmed via a web interface, which sends out little files containing firmware updates or instruction sets or whatever, which then have to get uploaded to the remote. Of course, Logitech only supply a Windows program to do this.

There is Linux support, in the form of the fine Concordance project, which provides libconcord (that does all the heavy lifting), concordance (a console UI) and congruity (a GUI). So we're close to remote control nirvana, defined as the point where you just run the web interface as you would on Windows, and when it sends you one of the little files to get uploaded to the remote, congruity kicks in and handles it.

However, there were two problems. One, the stuff was only actually working as root - concordance and congruity couldn't see the remote when run as normal user. I filed a bug report on that and it got fixed in short order - thanks to the maintainer (Douglas Warner) for that. Two, there was no handling of the file types involved - they're XML files with the extensions .EZHex, .EZTut and .EZUp - so you had to make sure Firefox was set to manually ask you what to do with every file (or else it'd just figure they were XML and try and open or save them), and then manually associate these types with congruity. Which sucked.

So, what I learnt is the freedesktop.org shared mime information policy, and a highly fascinating policy it is! How it helps us in this case is it tells us how to define a MIME type for a file based on its extension. Basically you stick a little XML file in /usr/share/mime/packages which tells it that any file with the extension .EZHex, .EZUp or .EZTut is of MIME type application/x-libconcord . So I wrote up a patch for upstream libconcord to make it include and install such a file, and submitted it to our Bugzilla and to upstream concordance. Once that's done, we can just ship a congruity package (it's been stuck in review hell for a year, I'll try and get that sorted out) with a .desktop file saying it handles the application/x-libconcord MIME type, and remote control nirvana will be ours - all you'll need to do is install the congruity package and you'll be able to use the Harmony web programming interface with no messing about. Ahh, I like it when I can do useful stuff.

So, time for a Poulsbo update!

I have updated my P to Fedora 11, and I have psb running on it. I'm using all the stuff from my F10 post, except I'm using the Mandriva dkms-psb package for the kernel module instead of my psb-kmod. I haven't actually tried using my psb-kmod package instead, yet, to see if that would make any difference - I'll try that tomorrow.

There was one gotcha; this is one that caught at least one other person who contacted me out (hadess), and may have been what others who said 'it don't work!' were seeing. If X fails to start and the logs seem to show an awful lot of stuff about it trying to find an SDVO display, but precious little about LVDS, add this line to /etc/X11/xorg.conf:

Option "IgnoreACPI"

and that should shift it. (It goes in the driver section). The X logs actually give you this hint, it's just a bit buried. Alternatively, you can hook up an external display and not use your internal panel. That'd make it happy too. :)

It also seems that if I boot to runlevel 3 and then do startx it hangs after displaying a pointer for a bit, which frustrated me for quite a while. Eventually I just thought 'what the hell, I'll just boot straight to runlevel 5 and see what happens' and it worked.

I still intend to tidy the packages up a bit to make this easier, it's just I never seem to find the time for it!

I am also indebted to Pierre-Henri Beguin for an intriguing hint. He found a repository - it's here - which is part of Moblin 2's source, it seems. Split across the psb-headers, xf86-video-psb and kernel packages is what appears to be a complete, freely-licensed Poulsbo driver. Looks can be deceptive, though - it might still require the Xpsb and psb-firmware stuff. I did try to build it, but couldn't get far; I transferred the kernel patches to a Fedora kernel .src.rpm and tried to build it, but it failed. Then I tried just running that X driver with the other psb kernel module, and that doesn't work either (falls over on DRM stuff). So that one's stuck for now. Might be interesting to those with more chops than myself, though.

Mandriva users should note that Mandriva now has packages for the kernel module (dkms-psb), custom libdrm (libdrm-psb2 and libdrm-psb-devel) and the X driver (x11-driver-video-psb). I'm not sure whether these actually work, though, or if the Xpsb and firmware stuff is packaged (it may be in non-free, I didn't look yet). Those packages are available for 2009, 2009 Spring and current Cooker, according to the changelog mailing list. Olivier Blin's been building them. I'm sure he'd just love you to get in touch if you can't get them to work. ;)

I have been, as I mentioned, following up on Fedora 11 release stuff. Sadly, there've been a few very negative reviews and comments, based on problems with the partitioning stage of installation.

So, here's the deal on that. I've been explaining it in comments and so on, but I thought it would be worth a recap blog post. The reason there's an unusually high number of bugs in the storage code in anaconda is simple: it was entirely rewritten for Fedora 11. Here's the feature page explaining this.

As you can see from the feature page, this is a pretty complex area: there are multiple filesystems, things like RAID and LVM, hardware issues, and desired and pre-existing partition layouts to consider. There's a ton of variables, in other words, many of the combinations of which we are unlikely ever to hit in internal testing. We (the QA group) could test installs forever and a day and probably not hit some of the situations that get hit very rapidly once code gets out to the real world.

Some questions arise. Why was a rewrite needed in the first place? This is explained on the feature page. Basically, the existing code is very old and was not written to be easily extensible; it makes it very hard to add support for interesting new things like LUKS and iSCSI. We wanted a more modular code base so these and future desirable storage-related innovations could be handled properly. The longer you delay a necessary rewrite, the more pain is involved, so it made sense to do it as soon as the need was identified.

Why was the rewrite put into the main Fedora release so soon? It would have been possible to keep the 'old' and 'new' tracks separate, maintain both, and ship Fedora 11 with the 'old' code, and maybe only bring the 'new' code into Fedora 12. There's a couple of reasons not to do this.

First, as noted above, many situations just aren't tested until the code gets out. Pretty much all the situations we can actually test internally actually work in F11's anaconda. Our test matrix is full of green check marks. So even if we'd delayed the new storage code until F12, it probably wouldn't have had many more problems fixed than it does at present. We have to find out about the problems before we can fix them.

Second, this would have used up (or, in many ways, wasted) rather a lot of developer resources. We would have had to split the anaconda team and had some of them work on maintaining the old code for the F11 release. This work would have been essentially wasted as that code was destined for the trash, and it would have been that many man hours diverted from work on the new code. So we decided to push the new branch into the main codebase relatively quickly and have it released in F11.

Final question is more of a rant I'm seeing a lot of: why do you guys suck so much? Why are there so many bugs? Surely the coders must just be lazy / incompetent if they couldn't fix $MY_ISSUE before release?

Short answer - well, no, they're not. First, here's a solid number for you: from January 1st to now, a total of 332 bugs were filed on anaconda in Rawhide (so F11 at the time) and fixed. Here's the list. Just as a quick ballpark comparison, the number for the kernel over the same period is 122. So it's certainly not the case that the anaconda team are a bunch of lazy asses; they've been working their behinds off on bug fixing throughout the F11 cycle.

Are they incompetent? No, they're not that either. As I mentioned at the top, storage during installation is an innately tricky area, and Fedora has to support a lot of different scenarios here (especially as more or less the same code is used in RHEL, which has a lot of fairly robust requirements in this area). A lot of the variables have a huge range of possible values (previously existing partition layout, for example) - so what looks like just a 'perfectly standard installation' could actually have several thousand variations for different sets of data that previously exist on the disk and different hardware. When you're working from scratch to write code to handle a situation this icky, it's just inevitable that bugs happen. No set of coders could likely have produced a significantly different result.

In conclusion - we knew this was going to happen, and we went into it with our eyes open. We knew there'd be regressions. That's regrettable, and it's fine to criticize this, mark F11 down for it in reviews, and warn readers that the installer's storage code is a bit problematic and they may hit issues here. That's all perfectly true and valid and fair. What I wanted to address with this are just the questions of why this is the case, why it's not because we just suck at what we do, and why we went ahead and did it even though we knew it would cause some level of pain. Hope it's been useful.

And of course: FILE BUGS! When Anaconda fails, it usually gives you a dialog box with a traceback of the issue, and lets you save a copy. Please do so, and file a bug explaining where it failed, what choices you made during installation, and ideally the previous partition layout of your system. Include the traceback as an attachment. If you don't report the problems, they don't get fixed. Thanks!

So, I had some grand plans for this week, which I haven't yet managed to work on as I've been busy following up the F11 release around the news sites, IRC and forums, as I used to do for Mandriva. This probably isn't strictly part of my job description, but it does 'feel' right. I want to have a good idea of what the heck is broken. :)

Some neat developments this week: well, yes, we released Fedora 11. You may have heard about this. Go download it, it's awesome. I've been busy warping everything into the shape I'm most comfortable with, so here's the good old pair of documentation links I like to have for releases:

Release Notes Common Bugs (what we called Errata, at Mandriva)

Aside from that - there was a Fedora Activity Day to discuss potential changes to the Fedora 12 development cycle. It seems to have gone off really well. Unfortunately I couldn't really attend as I had Ekiga issues (something strange between Ekiga and my headset...the Ekiga guy said "I don't like ALSA. It makes my head ache. I can't help you."), which was probably a blessing in disguise as otherwise I'd be even more behind than I already am.

Fedora Community was released. It looks, well, incredibly awesome. It's a sort of centralised front end to a lot of other tools, intended to provide a useful 'launch control' for Fedora contributors - seemingly mostly for packagers at the moment, but hopefully it'll become more generalized over time. It's sort-of-but-not-really a bit like Launchpad, only it's 100% open source. There's a great introductory blog post on it here, by Máirín Duffy. Go read it.

Carl Worth wrote a really great post on Intel driver development. It explains very clearly why the Intel driver seems to have gone backwards in some ways lately, why that's not exactly the case, why you shouldn't worry about it too much, and how to help get things fixed quickly if you're still having problems with it. If you're a suffering Intel user, go read it.

Myself and others have continued to develop the Common Bugs page for F11, which is reaching a pleasing length now. Of course, the length of Errata-type pages should never be considered a measure of the quality of a release, only the commitment level of those contributing to it. I'm pretty happy with it now. My experience with Mandriva showed that this kind of page is a valuable resource.

We had a great discussion in the QA group about how those in QA and those doing support in IRC or forums could help contribute to the documentation process, and why we should try to keep reference material centralized in one or two well-known and properly maintained locations. On the back of that discussion, I've joined the documentation team, and I'm hoping to provide some more input (and earlier input) into the release notes process for Fedora 12. Hopefully we can get to the point where we don't have little FAQs and references split across many different sites, often with duplicate content, being used in different contexts (mailing lists, IRC support, forums etc).

And this morning I wrote a little Wiki page on how to create an xorg.conf file if you don't have one already. This is unfortunately still required in several situations, and it's bugged me that there was no clear reference on how to create one - which has annoyed a fair share of people over time. I hope this page fills the gap. I'll be linking to it anywhere in the Wiki stuff I write which suggests the creation of an xorg.conf file.

Phew! Busy week. There's been other stuff going on too, but this is too long already!

Just a quick note that, despite the refereeing^H^H^H^H^H^H^H^H^HBlackhawks beating the Canucks earlier in the playoffs, I've still been watching, of course. Naturally I'm pulling for the Penguins in the finals, because they're exciting and fun, and the Wings are dull, annoying, and Josh won our little contest. :) It was great to see them pull back from 0-2 and 2-3, but I can't honestly see them winning game 7, just because the series so far has gone so totally to the home team in every game. It's been strange, actually - of course home ice is an advantage, but it's rare for it to be so...clear-cut. The games played in Detroit and the games played in Pittsburgh feel like two completely different series, it's like both teams are completely different depending on the rink they're playing at. Really bizarre. So unless that somehow turns around completely, it's rather hard to pick anyone but Detroit to win out in the end, sadly.

My highlight of the series so far - Jordan Staal's shorthanded goal. That was just brilliant.