As is often the case, this event isn’t really that Fedora specific: the work Fedora does on webcam support in the kernel and libv4l of course goes upstream where other distributions use it, so even if you don’t use Fedora, come along to the Test Day, do the test with a Fedora nightly live CD so you don’t need to install any Fedora stuff, and send in your results; if you report a problem and get it fixed, the fixed will wind up in your distribution too. Isn’t open source great?

This Thursday, 2010-02-18, will be Color Management Test Day. There’s some exciting new color management features in Fedora 13. That is to say, there are color management features in Fedora 13! We’ve never had any real color management in Fedora before, so this is great news for photographers and designers in particular – but everybody, really.

The good news is testing for this Test Day is very easy – anyone with a monitor can do most of the testing. A scanner, printer, webcam or colorimeter let you do some other tests. You don’t need a Rawhide installation to do the testing, just a nightly live image will do fine, and the testing process is fully documented and won’t take more than a few minutes. So come out and help us test! Even if you don’t use Fedora, you can test very easily with a live image, and this is one of those features that’s popping up in Fedora first but will soon appear in all distributions (as it’ll be part of GNOME itself), so you’ll be helping your own distribution by testing.

The Test Day will run all day in Freenode IRC #fedora-test-day. See this page if you’re not sure how to use IRC – or you can use WebIRC just by clicking this link! Ah, the wonders of science.

That’s Spring. What’s Spring? Well, it’s a fairly neat open source RTS framework that started out as a Total Annihilation clone, but more to the point, it’s a pretty complex 3D game. What’s cool about the picture? Well, it’s in the renderer info in the console (which you probably can’t quite make out, never mind). Yup, that’s Spring…running on Nouveau!

Huge kudos to the whole nouveau team for this. Nouveau’s 3D support recently became available in Rawhide / Fedora 13 with an update to the mesa-dri-drivers-experimental package. I was impressed when it ran Compiz, really impressed when it ran Neverball and Foobillard, but running Spring is nuts. Doesn’t seem to manage gnome-shell yet, but I’m sure they can fix that. I admit I never expected to see things working like that in such a time frame. Obviously it’s heavily hardware dependent, it doesn’t run super-fast (looked like about 20-25fps, just by eyeballing it, on my 9400 GT) and I haven’t stressed it much yet so it may be crashy, but still, that’s incredible stuff.

edit: it also runs Quake 3. Quake fricking 3!

It’s fine. I have it running a third party ROM (EnergyROM) with Windows Mobile 6.5.5 and the latest version of Manila (HTC Sense 2.5). It does everything I need it to, pretty much, and Manila’s a nice UI. But still, Windows Mobile ain’t that cool these days, y’know.

So I’m tinkering with the Android port that’s available for it. It’s impressive stuff; most basic things work now – calls, SMS, 3G data, most bits of Android itself including the Marketplace. Sound outside of calls, WiFi, GPS, Bluetooth aren’t working and power management is dicey, so it’s nowhere near done, but it’s certainly tinker-with-able.

Most of my frustrations with it are nothing to do with the highly bleeding edge nature of the port, but with Android itself being bloody fucking stupid in places. No-one would take this shit from Microsoft, but since it’s Google we’re apparently supposed to not care and just feel the love.

Stupid stupid frustration #1: if you want to synchronize with a Google account, that Google account has to have Gmail enabled. Never mind that I have absolutely no use in the world for Gmail and just want to sync my contacts and calendar with an account that exists solely for that purpose. Nope, I need to have Gmail. I can’t even set up a dummy Gmail as the primary account and then synchronize contacts and calendar from my real account as a second account; Android lets me add an account with no Gmail as a secondary account, but refuses to sync anything with it.

Google – *why*? Just why? Why would you consider this remotely not evil? I do not want Gmail. I do not need it. By doing this you are not convincing me to use Gmail, you are just hugely fucking pissing me off. There is no justification in the world for this. If you claim you need an email address for me for some bizarre reason, like to email me about the Marketplace or whatever – fine. Ask me for an email address and verify that it’s mine. There is no reason in the world it needs to be a Gmail account. Just quit this stupidity, pronto.

Stupid stupid frustration #2: Google’s email client is totally broken. It cannot parse perfectly standard IMAP folder hierarchies. As someone on the bug has pointed out, every other IMAP client in the known universe – including Microsoft’s, for God’s sake – manages this perfectly well. This has been broken since Android first showed up a year and a half ago and the bug has received not a single word of response from Google. All they’ve done is reclassify it as a feature request (duh, what?)

Even the IMAP prefix option seems broken for me. Whether I set it when creating an account or after having created one, whether I set it to INBOX or INBOX. (note the period), it just doesn’t seem to work.

Grah. Google. Stop being a bunch of doofuses and fix this crap already. Much of Android is nice, but this sort of idiocy just leaves a really icky taste in my mouth. Someone, please, do me a Maemo port…

MythTV has, as I mentioned, a neat web frontend called MythWeb. Obviously since I have a webserver I wanted to run MythWeb on that; doesn’t make any sense to have two of my local machines exposed to the internet. To my surprise, however, these seems an obscure configuration in the Myth world; I could find only two references to it, one from 2004 and one from 2005, both mentioning in passing that it was possible, but no details. So here’s a quick highlight reel about how to do it.

First, stick MythWeb on your webserver. It is a very good idea at this point to secure access to it, especially (of course) if this is a public-facing server; there’s some example directives in the default config file for an htdigest setup. There’s nothing specific to MythWeb about restricting access, so just look it up in Apache docs if you aren’t sure. I found I also had to adjust AllowOverride to None to enforce the access restrictions, since I have a .htaccess at a lower level which would otherwise have granted access to the MythWeb directory, so keep that in mind.

Now you’re going to need to tweak MythWeb’s config a bit. In its config file, look for the setenv db_server parameter and change it to the hostname or IP address for the server. Also make sure the password is correct, of course.

Finally, the trickiest bit: you need to adjust your MySQL database configuration to allow access from the webserver machine…without stopping access from the local machine. This is surprisingly non-trivial. MySQL has a very strict permissions model.

On the backend machine, edit the MySQL config file – probably /etc/my.cnf. Comment out the line ’skip-networking’ and add a line:

bind-address=0.0.0.0

unfortunately it doesn’t seem possible to bind to two specific IP addresses with mysql; you can only do one specific address, or a wildcard. If your machine only has the loopback interface and a single network interface, this line will be fine, as it will bind to just those two, which is what you need. If it has more than that and you only want to allow access on the loopback interface and one of the real interfaces and you can’t do it with a more restricted wildcard, you’ll have to use firewalling to block off the ones you don’t want to have access. Which sucks. If anyone knows different, let me know, I’m no MySQL expert.

The trickiest bit is the MySQL privileges. It’s not actually that hard, but there are guides on the Google which hate you and want to eat your configuration. Do not follow things like this, which you can find all over the Google results, which tell you to use things like ‘update user set Host blahblah…’ commands. These will sort of work to allow remote access, but they will also stop access from localhost, which is a pain, and I’m damned if I can figure out how to reverse them properly. Nightmare. No. What you want to do is this:

grant all privileges on mythconverg.* to ‘mythtv’@'192.168.1.26′ IDENTIFIED by ‘your_password_here’;
grant all privileges on mythconverg.* to ‘mythtv’@'localhost’ IDENTIFIED by ‘your_password_here’;

where 192.168.1.26 is the IP of your webserver and your_password_here is, obviously, the password you want to use. Even if you’ve already created the user, these will do the right thing. You’ll be able to access the database from the webserver and the backend machine, but nowhere else, with the appropriate password – and that’s what we want. Yay. Obviously, if the backend machine has a firewall, you’ll need to adjust it as appropriate. Now you should be able to visit http://www.yourwebserver.com/mythweb , enter the username and password if you set up restricted access, and access a working mythweb interface. Success!

For me, this is enough to make it all work. If you stumble across this page via Google, do let me know if it helps or if I messed it up somewhere.

Frustratingly, just like many many devices these days, my box (a Motorola 6200, remember) doesn’t have separate Power On and Power Off functions (either in hardware or IR codes). It just has a power toggle button, which turns it on if it’s off, and off if it’s on. This becomes a bit tricky; I want to make sure it’s always on and off at the right times, particularly when you just want to sit down and press one button and use it, and when MythTV wants to record stuff.

The MythTV side of the equation turned out to be simple: there’s a neat channel changer script for MythTV called ‘mythchanger’ which supports switching the box on before changing channels – it doesn’t toggle the power, it is able to detect whether it’s on or off, so it turns it on if it’s off and leaves it on if it’s on. The Firewire interface must support that. Neat. That covers MythTV, as MythTV always runs the ‘channel changer’ script before watching or recording TV. (This one also doesn’t need you to specify the Firewire node of the box – it goes off the box’s UUID, or just picks the first box it finds. So it works without reconfiguration when my box decides to change nodes. Yay!)

That left the Harmony side of the equation. If you just use the wizard setup, it depends on the remote tracking whether the box is on or off, which isn’t going to be reliable now MythTV can potentially change it. The Harmony setup is capable of setting up discrete Power On and Power Off commands and then using them appropriately, but my box doesn’t have any! What now?

I luckily found a somewhat obscure forum thread where one Hailey Williamson (no relation) is credited with a great little logic hack. She noticed that, if the box is off and you hit Menu, it will be turned on (and sent straight into the menu system). If the box is turned on and you hit Menu, the menu system comes up. So you can now derive a simple little logic hack: the command string MENU POWER will always turn the device off, whether it’s on or off to start with (if it’s off, it turns it on (and into the menu) then immediately off again; if it’s on, it goes to the menu then turns it off). It follows of course that the command string MENU POWER POWER will always turn the device *on*. So you just define those command strings as the Power On and Power Off buttons in the Harmony configuration tool (luckily it supports that kind of hackery). Incredibly simple, but I doubt I’d ever have thought of that on my own. Cool trick! I set mine up that way and it works perfectly.

Another little tweak I set up today is to monitor the mythbackend process (that’s the MythTV ’server’). It’s known to not be really super-reliable; every so often it does fall over, for one reason or another. Obviously you don’t want that to happen to a PVR setup. So I found a reference to using a neat little tool called Monit to monitor it. Monit is a fairly powerful generic monitoring tool for *nix systems; it can monitor all sorts of things in different ways and perform actions depending on what it sees. It can be run as a one-time check or as a daemon which checks all the things it’s set to monitor at regular intervals. So with monit running as a daemon and the following configuration:

check process mythbackend with pidfile /var/run/mythbackend.pid
start program = “/etc/init.d/mythbackend restart”
stop program = “/etc/init.d/mythbackend stop”
if failed host localhost port 6544 then restart
if 5 restarts within 5 cycles then timeout

(thanks Thomas Mashos!), every time a monit scan happens, it checks if mythbackend is currently supposed to be running (by checking for the pidfile), and if it is, checks if it really is (by trying to poke the port it should be listening on). If it’s not there, it restarts it. If it fails the check five consecutive times, it figures there’s something really wrong – it’s not just randomly falling over – and gives up until you poke it manually. monit even does email notification, so you’ll know when it’s falling over and when it hits timeout. Really neat little tool.

Lately I bought a second HD box, as I think I mentioned, and it has a fairly neat feature – if you connect an eSATA hard disk to it, it’ll work as a PVR. But of course it’s a typical cable network PVR, the files are locked up and there’s no commercial skipping or anything. (It is dual-tuner, though). And I don’t have any appropriate disks lying around – only an IDE one, and I can’t find any enclosure that lets an IDE drive connect via eSATA. So I’d have to buy either Shaw’s ‘own’ disk (apparently it’s a rebadged Western Digital), another eSATA disk, or an eSATA enclosure and SATA disk. All those options are over $100, and…meh. I also realized I’m likely going to be in the UK during the NHL playoffs, which is clearly terrible.

So I decided I’d get my HTPC working properly as a PVR again, and see if I can’t do something SlingBox-y with it too. (The SlingBox streams live video over the internet, basically, if you’ve never heard of it). Luckily for me, my older HD box has its own neat feature: it’s a Motorola 6200. That box has a Firewire output, and if you hook that Firewire output up to a PC, what you get is the raw, full-quality high definition MPEG-2 stream, complete with audio. Neat, yes? I’ve known about this for a while but just never bothered hooking it all up. Once or twice Shaw have turned on copy protection on some or all channels, as far as I can gather, but people write angry letters and they generally turn it off again after a bit.

(Apparently, there’s a law in the States which says that if you have a box from your TV service which has a Firewire port, they have to enable it and allow you to stream video out of it, if you ask them to. It’s been diluted a bit but it more or less still stands. Google it if you’re interested.)

This feature is sufficiently cool that people have been using it for years now, and MythTV has fairly mature support for this kind of Firewire input. It even comes with a neat script which changes channels via the Firewire connection (no, I didn’t know you could do that either!) The motherboard in my HTPC has no Firewire ports (it’s a damn cheap motherboard) so I went out and bought the absolute cheapest yum-cha Firewire adapter I could find – a princely $10, including tax. I followed the MythTV instructions, and damned if…nothing. Nada. Zip. Not a fricking dicky bird. Upon further investigation, it seemed as if the kernel hadn’t done anything with the card at all – lspcidrake (the HTPC runs Mandriva) was showing the module as ‘unknown’. Crap, I thought. I nearly gave it up as a bad job and got a more expensive card, but happened across a post from a guy who’d had to switch PCI slots to make it work. So I experimented with different PCI card/slot permutations until I got one which got the ohci1394 module loaded and a /dev/raw1394 device showing up. Then started a cycle of twiddling with configuration commands and re-running the connectivity tests, with mixed results. In the end, I think the combination of the super-cheapo card and cheapo motherboard does affect the reliability a bit. I’ve entirely hung the system by nudging the cable while it’s busy streaming TV, and it seems to have an odd habit of switching from node 0 to node 1 (some Firewire thing), which requires a reconfiguration of MythTV’s channel-switching script, which is a pain. But basically I’ve got it working reasonably reliably, now.

Setting up MythTV was actually pretty easy – I just glued together the Firewire instructions and the general MythTV setup instructions from the official docs and pretty much got it working. It has its little idiosyncracies, of course. In a fairly typical way for Linux apps, it’s designed to scale to ridiculous levels – it has a client/server architecture and can actually have multiple ’servers’, each with multiple video sources, all connected together in a ridiculous giant agglomeration so a single MythTV setup can be recording seventeen shows from ten different cable boxes spread across five machines while simultaneously playing back video on forty different frontends, or something silly like that. Which of course means that various bits of the configuration are way more complex than they need to be, for a simple one-box setup. The options for transcoding recordings down to a reasonable size (the raw HD streams come to 5GB/hour, with surround sound) are pretty bizarre at first sight; only if you imagine configuring one of the ridiculous setups I described above do you suddenly realize ‘oh, yeah, that’s why it was designed that way’. There’s the usual media center knobs to twiddle – getting your remote control set up, setting up the video and music plugins, and so on. You have to set up a third-party TV listings service (it’s $20/year, not that terrible). But really it’s not bad, certainly less painful than Freevo (which requires you to hand edit its configuration files, which are written in pure Python…) or the MythTV I first tried to set up in 2004 (though you could get listings for free back then).

So right now I’m watching Andy Murray against Mario Cilic in the Australian Open semi-finals; the stream comes into my HD box, gets piped to my PC, and rendered via VDPAU. It looks flawless, just as if I were watching direct via the box (which you can still do even with the Firewire up and running – useful if it all goes pear-shaped). I can do the whole ‘pause live TV’ thing, and the recording will be available for me to watch tomorrow if I happened to want to, or I can store it for the future. It’s really pretty neat and works very well. I also have MythTV configured to let me watch videos and listen to music (though honestly I prefer Freevo’s interface for those).

The SlingBox bit may be trickier. There are avenues to explore. There’s a neat web interface for MythTV you can set up – it’s called MythWeb, it’s basically a MythTV frontend that is a web application, you connect to the webserver and can configure recordings and stuff, most of the things you can do from the ‘normal’ frontend. It also has some neat remote video watching tricks: you can get a direct download or ASX stream of any stored recording. I can actually log in to the web interface from another system on my network and stream a recorded show in full HD quality – pretty cool. Obviously, though, 5GB/hr is far too much for the system to stream over the Internet.

I could transcode a recording down to a reasonable size and stream that, of course, but then you can’t use that to stream live TV. What may be the most promising avenue is MythWeb’s neatest trick: it actually has an inbuilt Flash streaming server system, so if you click on any recording, besides the ASX and direct download links, you see a Flash streaming player, much like Youtube. There’s a configuration widget where you can tell it what resolution and what bitrate to use for that, and MythWeb has the backend transparently transcode whatever it is you want to stream before serving it up through the Flash widget, so it’s much more suitable to Internet bandwidth. *Some* people have reported that they’ve been able to successfully use this on a currently-recording show – so they can set their box to ‘record’ something at 9:00pm, then log in to MythWeb at 9:01 and successfully access the Flash stream of that recording, while it carries on being recorded. This is obviously a fairly dodgy chain, but there’s no absolute reason it can’t work. I tried it briefly and it didn’t quite get there, but I’ll tinker a bit more and see if I can get it to fly.

So, the final chapter of this little odyssey…when it has fallen over (see above) I suspect it has something to do with resources. My HTPC box is nothing particularly oomphy – it’s a Pentium dual-core E2180, which is not a high-end chip (that particular line of ‘Pentiums’ is essentially the cut-down, ‘Celeron’ version of the Core Duo), and it’s only had 512MB of RAM for years. It never really needed more than that to be a media player, especially since I got VDPAU support working so all the video decoding work happened on the graphics card. But it’s pretty borderline for a Myth PVR setup. I noticed it was nudging up against the top end of the RAM.

Then I realized I actually have a couple of gigabytes of RAM lying around the place. Only DDR-1, but the motherboard – being a cheapo one meant for cheapskate upgraders like me – actually does support both DDR-1 and DDR-2 (it’s got two pairs of RAM slots…it also has PCI-E *and* AGP video slots, and can support regular old PCI video cards too. You can have one each of all three if you really like. Crazy.) So I yanked out the 512MB stick and threw the 2GB in there. Thought ‘what the hell’, cranked the FSB up to 266MHz – many people have the E2180 overclocking easy to 2.66GHz, from 2GHz – and hit the big red button. Well, those successful people are obviously using better motherboards than mine, cos mine doesn’t even POST at 266MHz FSB. Did the CMOS jumper reset thing, and the board came back up fine. Reconfigured my BIOS settings, and…

the fucking thing wouldn’t boot any more. Just sat there with the Blinking Cursor Of Death at the point it’s supposed to load up the bootloader. Now, it’s 1:30am at this point and I’m getting cranky. I start swearing under my breath and doing all the usual crap. Reset the BIOS again. Twiddle with the FSB some more. Pull one of the RAM sticks. Pull the other. Go back to the 512MB one. Mess with the RAM speed and timings and voltages. Twiddle all the SATA interface settings I can find. All that crap. Nothing doing. I’m halfway through trying to reinstall the bootloader from a rescue CD (not a simple operation on a system with four hard disks, none of which has the same device node in a rescue environment as it does for realz) when I figure that’s a bad idea and it’s probably something else. Finally I hit on disconnecting the Silicon Image PCI SATA controller card to which my RAID array disks are all connected. Success! the bloody thing boots fine. Of course, the system’s not a hell of a lot of use without the RAID array. Finally I discover that setting the motherboard’s SATA controller to RAID mode – even though it’s controlling one whole disk, which is not a part of *any* RAID array – inexplicably makes it work. Even though I never had to do that before, the whole time I’ve had the freaking setup.

Hardware. I hate hardware.

So I put the 2GB of RAM back in, double-cross my fingers and kick the FSB to 233, and – fricking hallelujah – up it comes. Jebus, that was an hour I’ll never get back. Still, it’s up now, with four times the memory and a modest 1/6th CPU speed bump. Hopefully that’ll help keep it stable and make it do the more demanding PVR-y bits a little quicker.

Despite that last icky hardware crappiness footnote, I’m pretty happy with this project – got the whole thing up and running really quite nicely at minimal expense ($20 in total: $10 for the firewire controller, $10 for the cable) in an afternoon. The SlingBox trick may be harder and I may have to give up and just live with watching the games the morning after they happen, but I’ll do my best to hack something up. Never surrender!

(Yes, I did some work today too. Honest!)

I was still sore about having to rely on Google for my calendar / contact synchronization, so I figured I’d sort that out. Cue eGroupWare. This is a rather nice SOHO/small business-targeted groupware suite; it does contacts, calendars, tasks and a few other bits and pieces, with a decent web interface and synchronization via GroupDAV / WebDAV and SyncML. I spent most of the evening updating the Mandriva packages for eGroupWare, which existed but were extremely old. I got those updated nicely, and set up the software onto my web server. Making sure to follow all the instructions (and make the package do the same, as much as I could) I was able to set up a calendar and contact list and synchronize them to Evolution on my laptop and desktop via WebDAV – very slick! I haven’t yet tested syncing with my phone via SyncML / Funambol, but I expect it to work fine.

This may in fact be a good candidate for something I’ve been working with the infrastructure group to try and sort out for a long time – a Fedora project calendaring (and possibly other groupware functions) system. Everything else we’ve come across is either broken, obsolete, has a bad web interface / no interface, no CalDAV support, only works with Sun Java, is crazily coded, or any or all of the above. This doesn’t seem particularly crazy, is actively maintained, has a nice web interface and CalDAV support, and is written in PHP – so it looks good. We’ll see where that goes. The ultimate goal would be to have a Fedora project groupware server where Fedora projects and SIGs could do scheduling and stuff in a collaborative way.

I’m uploading the updated eGroupWare packages for Mandriva to all supported repos.