The crazy world of ebooks, again

Ah, the car crash world that is the ebook industry strikes once more.

Most of the Flashman books showed up as ebooks just recently, and I'm off on vacation on Wednesday, so I figured I'd buy the ones I hadn't read - perfect spot of light holiday reading.

First I look in the native epub stores, as I have an epub reader (a Sony). Kobo has a couple of the books, at least. So I go through the convenient, easy process of buying a <1MB file full of text and transferring it to a USB storage device...

...by which I mean I fire up a Windows VM, open Firefox in the VM, go to Kobo, buy the book, and download it into the Sony Reader software in the VM. Then I open up a console, locate the downloaded, encrypted, epub file, download a Windows scp client, re-configure sshd on my real system to accept password login temporarily as I really can't be arsed figuring out where to put my key files in the Windows VM, and scp the epub file across to the host. Then I shut down password login on sshd again, and run ineptepub on the downloaded epub file to strip the DRM. (This is still legal here in Canada, for now at least.) Then I import it into Calibre, plug in my reader, and finally transfer it across.

...then I open it on my reader and find that, for some inexplicable reason, all the text renders ridiculously large and the font size settings have no effect.

So I say screw it, go to a torrent site, and find an unencrypted copy with zero text size issues in three seconds. Download it straight into Calibre and transfer it onto the reader.

And publishers wonder why people pirate stuff.

For further comedy value, I had to buy the other books in the series from Amazon, as they don't seem to be available through any of the epub-native stores. So for those I had to install Kindle for Windows into the VM, buy all the books and download them, install Calibre into the VM, install the plugin that allows Calibre to strip the DRM from Kindle books (which AFAIK only works in Windows, not Linux), find the raw copies of the book files, import them into Calibre (which strips the DRM using the plugin), convert to ePub format, then scp the ePub files from Calibre's library across to the host for transfer to the Reader. (If you're wondering why I don't just do the transfer-to-reader step inside the VM, qemu/KVM's USB passthrough support isn't good enough for Calibre on the Sony software to pick up the Reader, it seems). Despite the format conversion step in this process, those books display fine on my reader - no text size issues. Only the one which was a native ePub file in the first place, and which I only really 'manipulated' to the extent of shuttling it around a lot and stripping the DRM, doesn't render correctly.

You may be shocked - shocked! - to find that I greatly prefer to buy stuff from Baen, who offer all their eBooks in every format known to man, un-DRMed, and allow you to download them as many times as you damn well please, once you've paid the (usually very small) asking price. No faffing around with VMs or tools to strip the DRM, no ridiculous geographical restrictions. If only they published everything I wanted to read. Everyone should go buy stuff from Baen to support their model, and help them show the rest of the industry how it's done.

Fedora 17 Test Days

Many thanks to John Dulaney, who's taken on the task of Test Day co-ordinator for Fedora 17. He's put up a post announcing the F17 Test Day cycle and explaining how to propose a Test Day (it hasn't changed at all, so if you're used to the process, go ahead and do it the usual way). So if you're interested in proposing a Test Day for Fedora 17, please see John's post and follow the instructions therein! Thanks.

What's going on in Fedora QA

I apologize for not updating this blog for a bit - there's always one more thing that needs doing!

However, just because I've been quiet doesn't mean Fedora QA has been inactive - far from it.

We managed to crawl across the Fedora 16 finishing line back in November, and we all took a few days to get our breath back before jumping right into Fedora 16 damage control (I'm sure there must be a more diplomatic phrase for that) and Fedora 17 planning.

Helping to write the Fedora 16 Common Bugs page has been an interesting experience, and quite educational - it's interesting how wide the gap can be between understanding an issue well enough to deal with it in terms of release validation, and understanding it well enough to document it for a notional audience that knows nothing about it, particularly in the cases of problems booting from GPT-labelled disks and issues with the size of grub2's core image on older partition layouts.

The F16 common bugs page is longer than we'd ideally want it to be for any release, but we knew going in that F16 would be a challenging release with the grub2 and GPT migrations. It certainly points up some issues we'll be trying to fix as soon as possible for Fedora 17.

I've also been hanging out in the Fedora forums and trying to help out with F16 issues where I can; the other forum denizens are doing a great job of getting on top of the F16 changes and passing on the news (and the workarounds...) to others, though, so kudos to them.

Now Fedora 16 work is mostly out of the way and I've caught up with all the todo items that piled up while F16 validation was taking up all of our time, I recently managed to work through all the feedback that had been added to the Fedora 16 QA retrospective page on the Wiki and come up with an action plan. The retrospective is a great initiative of James Laska's - it's a single Wiki page where we can throw thoughts that grow out of the QA process on a specific release. If we notice something that's not working great, or think of a way we could improve the process, we can quickly throw it in the retrospective and get back to work, knowing it'll be developed into a proper action plan later.

So I went through each of the thoughts contributed by a wide range of people in the QA team, took direct action on several of them, and turned the rest into trac tickets for the QA and release engineering teams. That gives us a nice clear view of the work we can do in the 'quiet time' before Fedora 17 Alpha lands to improve our processes ahead of that validation cycle.

So what can you look forward to in terms of new and improved QA procedures (as I know that's just the highlight of your life)? Well...

And probably a few more changes besides.

Of course, the AutoQA train keeps on a-rollin'. The team recently completed deployment of AutoQA 0.7 and have already started AutoQA 0.8 planning. Some of the goals for the Fedora 17 cycle are to complete work on the ResultsDB system for managing AutoQA test results, and to have complete daily automated testing of Fedora 17 installation after branching.

We've also been talking to the anaconda team to try and ensure we work together to start testing the currently in-development anaconda UI rewrite as soon as possible, in order to try and make it as robust as possible and avoid the last-minute crises during the validation process we had so much trouble with in relation to Fedora 16's major installation changes.

It's going to be another exciting release cycle - but then, it always is, in Fedora land!

Myself and Tim Flink will be at FUDCon Blacksburg in January, together with several members of the QA community. Sandro Mathys has been working on planning a hackfest to bring the anaconda and QA groups together to work on new anaconda UI testing, we will probably be giving a few presentations, and Tim will be available to talk about AutoQA and any other QA-related tooling stuff. See you there!

Google Maps on N9 and N900: help, please, Google

So for months I've used http://www.google.com/maps/m to get a mostly-usable Google Maps interface for my N900. This seems to have been the standard way to do it for N9 users as well. (Ovi Maps is useless: its public transit routing is awful and it just doesn't have the 'interesting places' database that Google Maps has, it doesn't really know where, well, anything is).

However, in the last few days, Google seems to have killed this service. Instead of giving me a pretty nice 'mobile' version of Google Maps, visiting the URL on my N900 instead just gives me the full desktop Maps page, which is very slow to load and use on an N900, and much more of a pain to navigate with all the space that's basically 'wasted' on a small cellphone screen.

Google, can you please restore the old behaviour here, and give N9 and N900 users our workable Google Maps back?

PSA: bad nss update for F16 messing up yum

As Fabian Deutsch noted recently, some F16 users might be seeing yum somewhat inexplicably failing to download repomd.xml for the Fedora repositories, which will stop you being able to install updated packages. Seth Vidal mentioned in the comments that this was caused by a bad nss update.

To complete the information, the offending build of nss is nss-3.13.1-2.fc16. It was submitted as an update but then retracted when the bug became apparent, which means it doesn't show up in Bodhi's interface any more. If you have this build of nss, you should manually downgrade all installed nss subpackages - nss, nss-devel, nss-pkcs11-devel, nss-sysinit, nss-tools - to nss-3.12.10-7.fc16, which should resolve the issue and allow you to use yum as normal from there on. You will only have been affected by this issue if you had updates-testing enabled and updated while the bad nss update was available, it never made it to the stable update repo.

Stupid Fedora tricks

I decided to make the F17 jump early here Chez AdamW, and just for laughs, I hereby present the following Stupid Fedora Trick:

Neverball in Shell in a VM, oh my

That's Neverball, running inside GNOME Shell, running in a Fedora 17 VM, on a Fedora 17 host. Unstable enough for ya?!

Impressively, it's just about playable, though the graphics are bit messed up, there's some flickering that shouldn't happen. Looks like about 8fps. This is using the current F17 kernel rebuilt with debugging disabled on both guest and host - it's a lot slower with the 'stock' F17 kernel, as that has debugging enabled.

F17 more or less works at the moment, probably because no-one's changed much vs. F16, though PolicyKit seems to be broken; I'm looking into that.

do...what?

listening to the end of Dancing with the Stars before Castle came on:

"The impressive part [of tomorrow's show]...Andrea Bocelli and Flo Rida!"

blink blink

Fedora 16 is gold, but more importantly...

EDIT: A previous version of this post listed the release as 2011-11-10, it's actually 2011-11-08, my error! We did not delay two days or anything.

So we just got done signing off on the Fedora 16 release. It'll be going out according to the (post-Beta) schedule, on 2011-11-08. Mark your calendars! It'll be a fun release. Huge thanks to the anaconda team, ajax, releng, and all the awesome people on the QA team - Andre, Jayson, Sandro, Athmane, John, Konrad from Oracle, and RH's Kamil (and his merry band of interns), Tao, Hongqing, Tim, and everyone else - for the incredible effort you all put in to get this thing done, it's utterly absurd that we made the date for this thing with all the fixing and testing that was necessary.

But never mind that! That's not the coolest news of the day, not by a long shot. The coolest news of the day is:

GNOME Shell running in a KVM

As well as contributing one of the vital F16 fixes at one of the earlier 'last minutes' in the process, ajax has only been and gone and got GNOME Shell running in a KVM. Oh, hell yes.

That's Shell running on software OpenGL, llvmpipe specifically. Still very early, but should be committed to Rawhide soon and therefore available in Fedora 17. Amazing work. As well as making it work in VMs, this should allow Shell to work on just about any system with a reasonable amount of CPU power, even if it doesn't have hardware accelerated 3D. Almost no need for fallback mode any more!

Random pontification: The end of work?

There's a very interesting (if entirely wrong-headed) article over on Slashdot right now, and an even more interesting comment thread again.

It's essentially the classic Luddite argument brought back yet again: increasingly smart automation is going to make all human employment obsolete. We'll have robots not just to grow food and make cars but "flipping burgers, cleaning your house, approving your loan, handling your IT questions, and doing your job faster, better, and more cheaply."

The implied questions are "will this happen?" and "will it matter?", and the answers are "it's been happening for centuries" and "not really, no".

An interesting place to start with this is America in the early-to-mid-20th century, in which the Luddite argument was frequently re-framed in a positive way. Industrial automation, it was expected, would essentially kill 'work'. People wouldn't have to work 40 hours a week. They'd barely have to work at all. Robots would do all the work, leaving people to laze about and have fun.

Now, an interesting question: has this happened? You might, at first, be tempted to say "no". Most people still have full-time jobs, after all.

I'd argue, however, that on a deeper level the answer is "yes", it's just not immediately obvious. Furthermore, it's something that's been happening for much longer than is often recognized; really, throughout human history.

We can just abstract the question to one of the basic attributes of humanity, one of the things we consider to define an 'intelligent' species: tool use. We use tools. Why? To increase the efficiency of our labor.

Let's take the most basic, cliched example imaginable: knives. A knife's a tool. What does it let us do? A human with a stone knife can kill an animal more easily. They can eat a dead animal, or most other types of food, more easily. They can construct shelter more easily.

So: it makes their labor more efficient.

Let's throw 20th century political thinking back to the stone age: did the introduction of stone knives render stone age humans unemployed and lead to massive social problems? Well, no. It meant stone age humans had to spend less time and effort killing animals and cutting up food and constructing shelter. So it meant rather more of them didn't die in inconvenient ways, and gave them a bit more time to do things like get around to inventing more tools.

And so we fast-forward, extremely damn quickly, to, let's say, the middle of the last millennium or so.

Somewhere around there - it'd be interesting to pick an example society and try and pinpoint roughly where - humanity passed a kind of tipping point where the effect of our ability to increase the efficiency of our own labor changed.

Up until then, humans were still pretty much in a struggle for bare survival. As a species we had very little in the way of luxury. Inventing new tools was downright essential to enable more people to actually live full lives without dying of starvation or disease or exposure or whatever. A significant majority of the labor that most people did could be characterized as essential. Of course, no picture is perfect, and some forms of non-essential labor have been around a long time. You could argue we passed the tipping point as soon as we were able to make our labor efficient enough that human social groupings could support people who'd passed the age of practical reproductive ability. Or singers. Or, hell, prostitutes. All of which has been true for a long time.

But still, I think it's reasonable to say this effect became far far more obvious and significant in the last few hundred years. We became so efficient at the work of growing food and building shelter and protecting ourselves from predators and so forth that it took substantially less than the total amount of potential work that all human beings could do in order to support the basic necessities of life for all human beings. We didn't need all 500 million people on Earth, or whatever it was at that time - or, if you want to simplify to a single country, let's say the 5 million people in England, or whatever it was - to actually support the bare necessities of life for those 5 million people.

We had, in effect, a labor surplus - we had more potential labor than we actually, strictly speaking, needed.

Not coincidentally, this is the point in history at which the Luddite argument first emerged. When you're a stone age hunter you're probably not really going to be worried about the consequences of some bright spark inventing the wheel; it's not as if you're going to wonder what you're going to do now your 'job' of transporting things around agonizingly slowly and inefficiently has disappeared. It's still pretty painfully obvious on any scale that there's going to be Other Stuff For You To Do. Finding Stuff To Do is not going to be a problem.

It's when we hit this situation of a labor surplus that it can start to look like a problem. Because you can look around and think, hey, there isn't actually anything else that really needs doing, so...what now? The steam hammer came along and took my job! And on a micro scale, you're often actually right. New inventions do take away people's jobs.

On a macro scale, though, you're wrong, in general. Why? Because of what human societies seem to do when they hit the labor surplus case.

It's very interesting, and it's where the early-20th-century American framing of the Luddite argument - no-one will have to work so hard any more! - goes slightly wrong. Because instead of adjusting things so that each person has less labor to do, it seems like what human societies end up doing is inventing more labor.

We come up with utterly non-essential things to do, and have people do those instead. And we call them 'work'. But really, they aren't.

Most jobs aren't really jobs. They're occupations. Pastimes. Professions, perhaps. So really, the future that the American robot utopians predicted has come to pass, but by stealth. Most of us do things that anyone from the past would consider recreational pursuits, but call them 'jobs'.

For an entirely random and not-at-all-pointed example, let's look at advertising, or TV. If you look at it in a certain way, people who 'work' in those fields aren't really working at all. No-one in the entire world needs the Big Bang Theory to exist. There is no fundamental reason why hundreds of people should be paid to write, act out, film, produce, edit, promote and transmit that television show. It's entirely non-essential labor, and what really is the difference, at a fundamental level, between 'non-essential labor' and 'recreational activity'? It's a tricky hair to split.

In case it's not clear, I don't really think this is a bad thing. I think it's an interesting thing - the way people don't seem wired to accept a world in which they 'don't work', so instead of 'not working', we play little mind games with ourselves in order to call utterly frivolous pursuits 'work', so we can have people spend their whole time doing stuff that isn't really necessary but not get mad at them for 'not working'. Really, we're all living in the future and have been for decades, centuries: the robots came and took almost all the really essential labor over years ago. It takes a microscopic percentage of humanity to grow our food and build our houses. All the stuff the rest of us do is just fun and games, and we've built a wonderfully intricate Heath Robinson device of a society to misrepresent it as 'work' so we don't feel like we're slacking off. Aren't people amazing?

Getting It Rite

Reading this neat guide to writing git commit messages, a random thought occurred to me which I hadn't actually seen written down anywhere before...

I wonder how many guides to getting started in open source (whatever) include the single most important instruction I've ever come across, which applies to just about any action you can take in a F/OSS project?

0: Look and see how other people do it

Want to learn to write a git commit message? Well, you can read Danni's awesome post, but what did you do before it was there? Go find some git commit messages by established developers - ideally, the other contributors to the project you want to commit to - and see what they look like. Figure out the structure, and copy it.

This applies to everything. It's almost always easier, and usually will give you a better result, to copy what everyone else does instead of trying to figure it out from first principles. Don't stand out from the crowd! Don't innovate! Don't think outside the box! Don't blaze your own trail! Copy shamelessly!

I exaggerate, of course, but remember, even the most incredible innovators are coming up with amazing new stuff 0.01% of the time, and writing git commit messages 99.99% of the time. And you really don't want to invite an amazing new way of writing git commit messages every time you write one.