Intel GMA 500 (Poulsbo) graphics on Linux: a precise and comprehensive summary as to why you're screwed

So, poking around idly on Google this morning, I came across the interesting fact that there is, in fact, a native Linux driver for Intel GMA 500 - Poulsbo - graphics on Linux.

Don't get excited, though. It's not like it's actually any use.

That was the summary. Here's the gory details...

As I wrote in my previous post, the GMA 500 is not really a follow up to all the previous Intel chips. It's a bit of Intel's platform with a PowerVR (those of you who've been breaking crap as long as I have may remember PowerVR as an early contender to 3DFX and NVIDIA; they've since scurried off into niche markets) chip slapped on top of it. So it's basically a whole new architecture that needs a whole new driver.

Aside from the Vaio P and a couple of other systems, there's one significant machine with the GMA 500 chip in it: the Dell Mini 12. Why's it significant? Because Dell ship it with Ubuntu.

So, yes, there's a native driver. Intel, Dell or some combination of the two contracted out to Tungsten Graphics to write a driver for it. This is presumably because Tungsten have been supporting PowerVR chips for a long time, so they know their way around the hardware. So, the driver was written, and included in the special-sauce version of Ubuntu that Dell uses, and also in Ubuntu Netbook Remix. It was even developed under the auspices of the Moblin project at first, and you can find the code in Moblin git - X.org driver, kernel module. Where it hasn't been touched for eight months.

This is the first clue that all is not well. Up to that point it all looks fairly straightforward, but after that it rapidly goes to the dogs.

Trying to get the thing to work on a modern distribution, the first thing you notice is that it's messy code. The kernel driver consists of a copy of about half of drm, with all sorts of stuff included and built that doesn't really need to be. And it's not remotely in sync with current drm. All it really ought to be is a few files which should have rapidly been merged into Mesa upstream. There's even a comment in the code that shows the authors are aware of this, saying that one fairly trivial change should be done before the code is contributed upstream. But the change hasn't been done to any Tungsten-, Dell-, Intel- or Ubuntu-sourced version of the code I can find, so it's still a mess languishing in its own repository.

The second thing you notice is it doesn't build with any kernel post-2.6.24, due to that issue and a couple of others.

The third thing you notice, if you're lucky, is that it's not even really the latest version of the code, and this is where things get really odd. As I mentioned, the copies of both the X driver and kernel module in moblin.org git have not been touched for eight months. However, squirrelled away in dank corners of the Ubuntu empire, you'll find rather later versions of the code. It has eventually been figured out that the latest version of the X driver code is this one, if you combine the 'upstream' tarball (which is not, in fact, available anywhere 'upstream' that anyone I know of can find) and Ubuntu patch in that directory. The latest version of the kernel module code is stuffed in Ubuntu's Miscellaneous Kernel Modules tarball - this one.

The further you dive down the rabbit hole, the odder things get. The X driver seems to depend on a proprietary firmware file, called msvdx_fw.bin, for...something. Apparently it's needed for 3D acceleration and video playback acceleration to work. You can download this file from a post in an extensive and frequently messy and/or downright ill-informed Ubuntu forum thread on the topic. And, as far as I can tell, nowhere else. It also seems to depend on a supplementary X driver, Xpsb, for the same things - without both the firmware and Xpsb, 2D video playback acceleration and 3D acceleration don't work. I don't know where the heck you find that. The source trees definitely don't build it.

Some of this is documented - in passing, seemingly more as an example case than anything else - here. A page that's just sort of lying in the Ubuntu Wiki. Again, I only found it through the giant forum thread. The associated Launchpad project is listed as 'Approved', since May 2007, but 'Not started'. With no other information.

So, anyway. I picked my way through this bizarre minefield and then tried to get the fricking thing to work.

First I hacked up a patch for the kernel module myself to make it build, not really knowing what I was doing. But it seems to more or less work. When it loads, it gives me a native resolution framebuffer. Out of interest, trying to start X with the fbdev driver at this point gets started and shows the grey dithered "X is starting up now" screen, but then hangs the system with caps lock and scroll lock lights flashing. Interesting result, but not what I was really looking at anyway.

Later on, while trying in vain to find some combination that would work, I found that Olivier Blin of Mandriva has been poking at this stuff, as he has a cleaner patch to make it build against 2.6.27 in his personal scratch directory: here. So I tried building with that patch instead of my own hack. They seem to work about the same.

Then I poked about at the X driver. It expects to build against various headers from the version of drm in the kernel module source, not the system copies (or at least not the system copies on Fedora 10). It also expects a particular xf86mm.h, which I pulled out of the relevant Ubuntu libdrm-dev package - so already having to hack stuff up here. With that and a few other minor tweaks, I got it to build.

But...yeah, it doesn't work. With the newer version of the X driver from Ubuntu, it fails with a "could not mmap framebuffer (invalid argument)" error, almost immediately on trying to start X. With the older version from Moblin git, it just freezes on X startup. The system's still running - I can power it off clean with the power button, it shuts down properly - but X doesn't start and I can't see a console either. There's just a text cursor frozen at the top left of the screen. The last lines in Xorg.0.log are:

(II) Primary Device is: PCI 00@00:02:0 (WW) Falling back to old probe method for psb (II) Debug: psbProbe

Okay, so after a whole day spent bashing around at this crap, I can very confidently and conclusively say, it's utterly broken. By the looks of it, you can get it up and running, with a bit of futzing around, either by running Ubuntu Netbook Remix or running Ubuntu Hardy and forcing the Netbook Remix packages onto it - see the forum thread for details on that. But with any modern distro, it just doesn't look like it's going to work.

This is really silly, because the code is obviously 99% basically there and working. It just needs the people who actually know the code and the hardware to put it in the right places and not random obscure undocumented repositories, submit it upstream, and keep it up to date. But because, for whatever reason, the Intel, Tungsten, Dell and Ubuntu folks can't get their fingers out of their asses and actually provide the code in a sensible, sane and open way, few people even know it exists, no-one's seemingly working to keep it going with newer kernels, X.org and libdrm, and no-one has a freaking clue how to use it. And every Linux user with this chipset is getting screwed.

It's not good enough, guys. Just get together, fix the freaking code, and get it in the kernel and X.org so distributions can make it just work and everyone will be happy. Thanks.

Vaio P: suspend works

Quick update - after a full update of Fedora 10, suspend magically starts working on the Vaio P. Guess something got fixed in the kernel or suspend scripts. Either way, it works. Yay for that.

Oh, and - hello Planet Fedora, since I seem to have managed to get myself on there now. :) I'm Adam Williamson. I will be working for Red Hat (in community QA) from next month onwards. I'll do my best not to break anything!

Vaio P Linux update: Fedora 10

So, an update on the Vaio P and Linux.

Basically, it works, with one big caveat. I had some fun getting it to install Fedora 10 - the live CD ISO to USB converter script just didn't seem to work right, whatever I generated would boot to a flashing cursor (on any system, not just the Vaio). Finally I found something that worked - using Unetbootin to set up a Fedora 10 network install. (Using it to set up a boot of the live CD image didn't seem to work). Once I found that trick, install proceeded perfectly normally, but in text mode - a shade of things to come.

The system booted to text mode. /var/log/Xorg.0.log showed it trying the intel driver, failing, then trying fbdev and vesa, and falling over on fbdev. So I stuck in a basic /etc/X11/xorg.conf to force it to go straight to vesa rather than try fbdev, and that works. Bottom line: this system uses the new Intel GMA 500 graphics chipset, which is not actually a successor to previous Intel chips at all, it's a completely foreign architecture. It's a PowerVR chip, more or less, and there is not yet any native driver for it. I'm fairly sure this is being worked on and there will be one reasonably soon, but for now, VESA is all you're going to get. Which means it's slow and won't drive the display's (ridiculously high) native resolution. I can live with that, but it's nowhere near optimal. I hope the native driver shows up soon. Note that if I'd run the installation with the 'vesa' kernel parameter, it would probably have worked in graphical mode and set up the VESA driver out of the box too, so if you're installing on a P, probably best do that.

Mandriva will likely do 'the right thing' out of the box - graphical install will work, and it'll use the VESA driver.

Wireless is an Atheros chipset, which appears to more or less work out of the box with the very new 'ath9k' driver. I haven't tested whether the sound actually works yet, but it's Intel HDA and the driver is loaded, so even if it doesn't it'll just be a case of a quirk to be added upstream, so either it works or it will soon.

Haven't tested the Bluetooth yet but it shows up without errors, so I expect it'll work.

Suspend basically works, but the display part doesn't. The system itself suspends and wakes up again, but the screen backlight doesn't wake up. If I tilt the display so I'm looking at it from the bottom at about 120 degrees, I can just about make out the console. I suspect that may well be fixed with the native display driver, when it shows up. The basic kernel support, though, is clearly there, so I expect this to work in future.

The webcam works out of the box with the uvcvideo module. Cheese works just fine. Bit slow with VESA as the display driver, though.

So my overall take is the Linux support for this system isn't quite all there yet, but it likely will be quite soon. Overall, my first impressions with the unit are very positive - the weight is ridiculous, it's just a massive gulf to the weight of a regular netbook. The screen's amazing, the build quality of the unit feels very solid in good-Sony style, the keyboard's great, the 'mouse' is decently implemented. We'll see how it fares over time.

New toy: Vaio P arrived

So, I ordered my Vaio P from the local Sony Store rather than directly from Sony. Good choice - I got to pick it up from the store today, I don't think any of the direct orders have arrived yet.

It's amazing. A lot of people consider it overpriced compared to 'normal' netbooks, but it really is a different story - the size, weight, build quality and the screen are on a different level. Here's some fairly crappy pictures (my years old camera doesn't like focussing properly without flash). The comparison shots are with my partner's Aspire One - as you can see, the Vaio is a lot smaller and a lot thinner. It also weighs a lot less.

vaiop_1 vaiop_2 vaiop_3 vaiop_4 vaiop_5 vaiop_6 vaiop_7

Hopefully I'll have a picture of it up and running Linux soon!

Switcharoonie

So tonight I've basically switched my main desktop to Fedora. I have dual Cooker and Fedora 10 installs, now, but I plan to stick with Fedora as much as I can. It was pretty easy, really - only took about two hours to make it pretty much identical to my usual desktop. Linux is Linux, after all. It's a bit odd needing a third party repository to get my graphics card and wireless working, but it works perfectly well. I suppose the oddest thing, to me, is that it seems more or less impossible to run Rawhide as a main system - it's not a constant rolling distro exactly (it gets updated in a big clump, one time a day) and it isn't really developed in such a way as to be usually usable, it seems. I dunno if I'm wrong here, but that's the impression I'm getting. I get the idea that even Fedora developers mostly only run Rawhide once a release goes into its alpha / beta / RC cycle. Maybe I'll try to get a working install of it going in a VirtualBox or something. It's also a bit strange not having the Mandriva tools available for things like remote shares and bootloader configuration...

I was planning to do some testing on issues with nv / nouveau on my particular card for Red Hat's new NVIDIA guy, but didn't get around to it in the end. Hope I will tomorrow.

Oh, and I helped Jerome Soyer out with a little gotcha in updating slrn to the latest release, in Cooker.

Job announcement

So, now I've checked I'm allowed to announce it. I'm happy to say that, from February 2nd, I'll be working for Red Hat, as a Senior Quality Assurance Engineer (sadly, I can't have 'monkey' in my job title any more). I'll be working under Jay Turner in the QE department. Initially I'm going to be working on a new community QA system they're developing. If I'm not misremembering entirely from my interview last month and horribly butchering the concept, it's basically a system for automated submission and testing of fixes by external contributors. I'm going to be helping to build a developer community around the system. It sounds like a really fun area to work on, and I'm really grateful to the folks at Red Hat for taking me on.

Mandriva was always a natural fit for me: it was the first and only distribution I ever used, and I got on well (I hope!) with the developer and user communities. It will be strange to move into a much bigger company with much more of a traditional corporate structure and organization - I'm used to being able to just break the heck out of whatever I like, at Mandriva anyone who's been around for a couple of years has commit access to just about everything! However, Red Hat has always been the open source company I've most admired: they're by far the biggest example of a company which has taken free software ideals to heart and built a big and extremely successful business around them, without ever losing sight of those ideals. RH definitely 'gets it' - the employment contract and non-compete agreements even have language specifically encouraging upstream contributions in them. RH has a long and awesome history of contributing to every area of the free software environment and releasing really high-quality products. Plus everyone I've talked to there so far has been great. So I'm really excited to be joining them. I'll be going down to Raleigh on Feb 2nd for orientation and starting work as soon as I get back - I'll still be working from home here in Vancouver, so no need to worry about a lack of moaning about the Canucks' latest performance! - so I expect I'll be blogging about the new project as soon as I get cracking on it.

Just wanted to say thanks again to everyone who sent messages of support when I was laid off at Mandriva, and especially to Tom Callaway, Alexander Kurtakov and the others at Red Hat who helped me get my foot in the door, you guys all rule. For now I'm going to enjoy the rest of my little vacation. See you on the other side!

Active day: Android on my Titan

Boy, it sure is fun to be irritatingly right, huh? On the topic of yesterday's post on security: Exhibit A. Yes, I know that's OS X, not Linux, but it illustrates my point exactly. Install code from random third parties, have your system completely compromised. This could have happened in exactly the same way on Linux. Believe me yet?

Had a pretty fun day today. I installed the latest Android image for my phone, courtesy of the great work done by Martin Johnson on Android for the HTC Vogue (otherwise known as the HTC Touch), and the modifications of his work by Magister for the HTC Titan / Mogul, which is the phone I have. It's really drop-dead simple - you download a zip file from Magister's site, unzip it onto the root directory of the phone's micro SD card, run a .exe from said zip file from within Windows, and it boots up Android. The latest version is really pretty functional - pretty much everything important works, the only bugbears for me are outgoing SMS doesn't seem to work and external headphones (via the mini-USB port) don't work yet. Aside from that it's pretty much all there for me - I may use it exclusively for a while, if I can work around the SMS issue somehow. Although I've written before about the fact that I really like the Titan as a device and I don't mind Windows Mobile as an OS, Android definitely feels nicer, and this edition of it isn't as heavily tied in to Google as the one you get with the T1 (i.e. it'll actually run without you feeding it a Google login). The dialer, the contact system, the email app - they're all pretty well laid out and thought through, the general flow of using the phone is pretty smooth. The browser's terrible on the Titan - I think it has less memory than it really needs - but there's an easy solution to that - use Opera Mini for Android instead. It's a great browser.

Other than doing that I took a bunch of old clothes to the local thrift store, picked up my tennis racket from stringing, went for a 3k swim and went out for dinner. Still enjoying my holidays!

On Linux security

Well, I was about to go to bed, but I can't sleep till I get this written.

This was prompted by yet another comment thread in which people seem to believe Linux has some kind of special sauce that keeps them super-safe from malware.

The problem is, no, no it doesn't. In practical respects, from the point of view of the individual end-user, a typical modern Linux system is only marginally more 'safe' from malware 'attacks' than a typical modern Windows system.

There's some things many people misunderstand, that it's important to understand. One: most 'malware' on Windows, these days, is pretty simple. The majority of it falls into the category known as a trojan, or trojan horse. What this means it is doesn't use some sort of clever exploitation of a security vulnerability in the operating system in order to propagate itself without human intervention. No, it's much more dull. Most malware just works by getting the victims to infect themselves. Why bother going to all that trouble to exploit a security vulnerability that'll probably be patched in a month when you can just write your nasty pop-up ad spawning application, say it's a porn video or a poker game or a bleeding fart noise generator, and slap a few Google ads on some dodgy porn sites? This is how most modern adware works. It gets run because naive users run it, not by exploiting intrinsic vulnerabilities in the operating system.

Two, most of the much-vaunted Linux / Unix 'security measures' have very little relevance to your average desktop Linux user. The whole Linux security model is designed on the basis of a multi-user system. The intent is essentially that every user be allowed to do exactly what the hell s/he wants in his own environment - and only in his/her own environment. And this is done, in general, very well (though there's usually at least a few unpatched privilege escalation vulnerabilities in the wild - still, never mind). But you have to realize the implications of this.

If you're in a proper multi-user environment - a single system (however the hardware is set up) with dozens of unprivileged user accounts, and a system administrator - it's a godsend. For the administrator, and all the smart users. But it does nothing for the dumb users. Why? Well, think about it. The point is that nasty code can only screw up an individual user's own stuff - basically, his/her home directory. So no other users are affected, and the system administrator can easily deal with the situation. This is what most Linux / Unix security is about.

So, think through the implications. If you're Joe Dumb User and you just ran a trojan, none of this helps you a jot. Why? Because all the stuff you care about lives...in your user environment. Sure, it's great to know that Jane Smart User and the BOFH can still truck along hunky dory, but there's nothing at all to stop the trojan you just ran from popping up several zillion ads on your desktop. Or opening a browser to a selection of the finest in erotic entertainment sites, just as your boss walks in. Or wiping out all your precious files. Because they're all stored in your flipping home directory, where any process running with your user privileges can do whatever the hell it likes.

Think a little further. If you run Linux on your desktop, you probably don't run a true multi-user environment. Again, the whole concept of multi-user security does stuff all for you. If you run a trojan, it can annoy you as much as it likes, and kill all the data you care about. Yes, multi-user security prevents it doing anything to any system-wide files. But, so what? There's very little you actually care about stored in /usr or /etc, is there? All your contacts, presentations and the fifth draft of your Next Great American Novel all live in /home/yourusername, where the trojan can do whatever the heck it likes to them.

Three, and this is the big one, 99% of desktop Linux users run completely arbitrary code as root, on at least a semi-regular basis.

Yes, really, they (you) do. Why? Well, think about what happens when you install a package. The installation process runs as root. And it runs utterly arbitrary code. When you install an RPM package, something called the post script is run with root privileges. That script can be, well, anything at all. There's something equivalent for DEB packages, and ebuilds, and every other form of package management. It's not possible to build a package management system without this.

And, shout out if you only ever install properly signed packages from your distribution's official repositories.

...quiet room, huh?

Just about everyone has installed a package for Some Kewl Application from some random third party website somewhere. Heck, this is a third-party website and I throw up packages for people all the time. When you go to www.mycoollinuxsite.com and install a package for a wireless card driver or a poker game or whatever the crap it is - congratulations, you just ran completely arbitrary code as root. You just did exactly the same thing any Windows-using clown who installs some poker game from www.mycoolwindowssite.com did. You don't have a single iota more "protection" or "security" than the Windows-using clown. If whoever made that package - and you don't have a clue who it was - wanted to own your system, they just did. Good job, sport.

I'm not saying this to be negative about the importance of security or how well security is managed on Linux. Security is vital, and Linux distributions generally achieve their security goals rather well. The point is a general one. As long as an operating system lets the user run arbitrary code, it's always going to be possible for a trojan to do whatever it likes to all the data that user cares about. As long as an operating system wants to make it relatively easy to install applications, it's always going to be possible for a trojan to do whatever the hell it likes to any system, if someone chooses to install it. There is no security measure against that. The only operating system that can be 'secure' against this type of attack is one which is very locked down, and does not allow you to install or run arbitrary code. Like a restricted cellphone operating system. That's fine, but it's not what people want on their computers.

It's really important that everyone who owns a computer understands this. Your computer is not magic. It cannot protect you from malicious code if you choose to run that code, no matter whether your operating system is Windows, Linux, OS X, FreeBSD or bloody BeOS. Dan says that data you keep in just one place is obviously data you don't care about. I'd add that data you keep on a system on which you run arbitrary code, and nowhere else, is also obviously data you don't care about.

Most people are not willing to accept the inconvenience that comes with only ever installing code that's either a) 100% provably provided by a person or company you trust implicitly, or b) that you've examined every line of yourself. And that's fine. But if you're in this majority, you have to accept that the danger of trojan-type malware is one that can never be designed out by an operating system, and don't be lulled into a false sense of security by the 'Linux is invincible' crowd. Yes, Linux is a properly multi-user operating system and secure from that perspective, in most implementations. Yes, Linux is generally quite well-secured against true 'viruses' - malicious code that is run without explicit user intervention. No, it is not magically safe against malicious code unwittingly run by the operator. It can never be.

The bottom line - I could publish a package along with this post which, after you double-clicked on it, entered your root password, and said "yes" to "the package is not signed, install it anyway?", would zero out your hard disk. I could, by writing this post differently, probably convince quite a lot of people to install said package. Just because it (to my knowledge) hasn't happened yet, doesn't mean it can't. Be careful what you run.

Ow. Ow ow owie.

...so, I just finished off Rock Band 2's main challenges on drums. I was trying to get through the whole game on first tries without dying, but the last couple of songs took me three or four tries each to get through the intro. foo!

Now my right arm and leg hurt like hell. :)

Job announcement coming soon!

Mandriva Cooker (2009 Spring) boot times

Fred looked into bootchart and discovered the Cooker package is just broken, and suggested I use the 2009.0 one for now, while he fixes Cooker. So I did, and here's the results. What Fred calls 'perceived boot time' - time to GDM up and running - is 0:17: bootchart. What Fred calls 'real boot time' - time to console login being available, i.e. all startup processes have finished - is 0:21: bootchart. Interestingly, I had to disable ntpd to get that reading. With ntpd enabled, real boot time is 0:27. I think it's because this system uses wireless, so ntpd doesn't successfully come up until the network connection is done (takes five seconds or so).

The installer guys claim to have fixed the ext4 issue, so I should be able to do an install to ext4 now and test with that - should be interesting. Obviously, I'll be doing a parallel install this time!