On Snappy and Flatpak: business as usual in the Canonical propaganda department
NOTE: this post is entirely personal. The opinions are my own and do not represent Fedora or Red Hat. The facts, however, are all 100% truthy. ;) Just to make it 100% clear for any visiting journalists etc. who don't know me: I work for Red Hat, on Fedora. I am not unbiased and am not claiming to be, but I am claiming that the concrete stuff I say below is true.
You may have read some stuff this week about an application delivery mechanism called Snappy and how it's going to unite all distributions and kill apt and rpm!
This is, to put it diplomatically, a heaping pile of steaming bullshit. You may not be surprised to learn that said pile has been served by the Canonical press department.
The source of all this press was this gem of a press release, which has been widely covered in a fairly...uncritical way by several outlets. Even Ars Technica, which is usually fairly good at doing actual journalism rather than just unquestioningly paraphrasing press releases, gave it a pretty anodyne write-up.
The press release and the stories together give you the strong impression that this thing called Snappy is going to be the cross-distribution future of application delivery, and it's all ready for use today and lots of major distributions are buying into it. In the press release you'll find stuff like this:
"Developers from multiple Linux distributions and companies today announced collaboration on the “snap” universal Linux package format, enabling a single binary package to work perfectly and securely on any Linux desktop, server, cloud or device."
The stories have headlines like "Adios apt and yum? Ubuntu’s snap apps are coming to distros everywhere" and "Snap Packages Become Universal Binary Format for All GNU/Linux Distributions" (jeez, I particularly love that one).
So what are the problems with this happy-clappy story? Several of them!
First let's be clear: Snappy is a Canonical project. The press release was issued, I think, sort of as if it came from some sort of independent or cross-vendor project, and there's the snapcraft.io site to back up that impression, but every Snappy committer is a Canonical employee, and contributions to Snappy require signing the notorious Canonical CLA:
"Contributions are always welcome! Please make sure that you sign the Canonical contributor licence agreement at http://www.ubuntu.com/legal/contributors"
Now, does Snappy actually have the cross-distribution buy-in that the press release claims (but never outright states) that it has? No. The press release sure sounds superficially impressive:
"Developers from multiple Linux distributions...Snaps now work natively on Arch, Debian, Fedora, Kubuntu, Lubuntu, Ubuntu GNOME, Ubuntu Kylin, Ubuntu MATE, Ubuntu Unity, and Xubuntu...Together, these distributions represent the vast majority of common Linux usage on the desktop, server and cloud."
but it's a pretty big mis-representation. The other distributions cited have not actually declared their support for Snappy and said 'yes, this is how we want applications to be distributed in future'. Canonical employees have independently built and released Snappy packages for those distributions. This is the basis of all the press release's claims. For instance, the Snappy packages for Fedora are in a COPR - COPR is a system like PPA that lets anyone build packages - maintained by a Canonical employee. The sum total of communication between Canonical and Fedora before the release of this press release was that they mailed us asking about the process of packaging snappy for Fedora, and we told them about the main packaging process and COPR. They certainly did not in any way inform Fedora that they were going to send out a press release strongly implying that Fedora, along with every other distro in the world, was now a happy traveler on the Snappy bandwagon.
There is in fact another system with very similar goals, which is now called Flatpak and was previously called xdg-app. To be as fair as I can, I'll say that Flatpak is quite heavily Red Hat influenced: the main author of Flatpak is Alex Larsson, a Red Hat employee. It is not, however, a "Red Hat project" to anything like the extent Snappy is a Canonical project. There are more than 20 other committers to Flatpak, most of whom are not RH employees (and including contributors to several other distributions). Flatpak is not under any corporate CLA. Insofar as Fedora is supporting one of these systems, it's behind Flatpak. No other distribution besides Ubuntu is particularly committed to either system, so far as I can tell. Flatpak and Snappy both began, so far as I could find from internet research, in December 2014. Canonical's press release, of course, doesn't even acknowledge Flatpak's existence...which is kind of par for the PR course, but you'd think at least some journalists might go out and do a bit of independent research.
UPDATE: since writing this post I've also been made aware of another system, AppImage, which has been around somewhat longer than Flatpak or Snappy (though not necessarily their forerunners). I know little about it so I will say little, but one thing to note is that - so I've heard - it does not attempt to do sandboxing like Snappy and Flatpak do, which is a major feature of those two implementations. It's purely an app bundle format. But hey, it's a choice! And it's been around a while!
Neither Snappy nor Flatpak is at all close to being 'done', in the sense of being a credible system for cross-platform app distribution with buy-in from software publishers and distributions. The PR's claim that Snappy enables "a single binary package to work perfectly and securely on any Linux desktop, server, cloud or device" sounds lovely, doesn't it? Let's take a look at the truth. Taking Fedora as an example, the Snappy install instructions for Fedora - go to the Snappy site and click the Fedora logo - say:
# SELinux support is in beta, so on Fedora 24 you currently have to: sudo setenforce 0
well, that doesn't seem terribly 'secure' or 'perfect' now, does it? Along the same lines, the Fedora packages are actually compiled with Snappy's confinement disabled. Confinement being the entirety of what's supposed to be secure about this form of app distribution. If confinement isn't turned on, you've basically just got a big blob with uncontrolled access to the system. Good luck with that.
AIUI, the builds for other distributions are in similar states.
Note that neither Snappy nor Flatpak can possibly provide meaningful confinement of apps running under X11, as mjg59 illustrated. Flatpak will only provide meaningful confinement with Wayland. Snappy, of course, is designed to work with Mir, though they claim it can/could (not sure which) also work with Wayland. But the point here is that neither Wayland nor Mir is out there in real widespread use by Linux users at present, yet here's Canonical happily glossing over that point while they talk about how Snappy right now allows "a single binary package to work perfectly and securely on any Linux desktop".
At the time this Panglossian PR was sent out, there were exactly two actual useful applications available as snaps: LibreOffice and Krita. Phoronix quickly found that the LibreOffice snap was huge (over 1GB in size) and buggy. The size issue was quite quickly resolved, but this just goes to show that reality is vastly different from Canonical's claims. This stuff is in early Alpha or proof-of-concept state. It is not remotely 'done' and ready for practical use in the real world outside of the very limited contexts where Canonical was already using it.
Neither is Flatpak, of course. But this is why Flatpak's developers have been communicating with technical conference presentations and blog posts and trying to build a dialog with application developers and distributors, rather than issuing press releases trumpeting how great Flatpak is and how it's ready to kill apt RIGHT NOW.
Here's another interesting thing about Snaps: the server end (the 'app store' bit of the equation) is closed source, and Canonical have been refusing to tell anyone how to run their own 'app store' - see the comment from Cassidy James Blaede, of Elementary. If you want to distribute your snaps, your choices are 1) publish it through the Canonical store, entirely under Canonical's control, 2) upload it as a file and tell people to use the CLI to install it, or 3) try to figure out how to reconfigure the snap client to use a different server by reading the source code, then write your own server end from scratch, and tell your users to do that. Hmm.
So: Snappy is, like Flatpak, a heavily-under-development, interesting attempt to provide an app store-like app provision mechanism for Linux. It is not finished, it is not close to finished. It is not independent or cross-distribution, it is entirely controlled by Canonical. It does not have, so far as I can tell, meaningful buy-in from a single major distribution outside of Ubuntu. It does not work properly on other distributions yet and it likely will not do so in the near future.
But apart from that, sure, it's all ready to kill apt and dnf tomorrow!
Now I'm sure I will get criticized for being mean and nasty and cynical and attacking Canonical instead of being constructive and all they want to do is make things better for everyone, Adam why are you such an ass?
Well, if Canonical actually wanted to work constructively with others, the way to do so would be to talk to them. We have forums for cross-distribution and cross-project collaboration. Lots of them. We have tech conferences where you can go and talk about your project and try to get buy-in for it. Canonical could have come to other distributions and said, hey, how about we all try to get together behind a single format and a common delivery mechanism for this kind of a confined app bundle?
But they didn't. They just decided to send out a wildly misleading press release and actively encourage the specialist press to report that Snappy was all set to take over the world and everyone was super happy with that.
That's not being constructive or working together with others. That's being a bunch of asshats and trying to present the rest of the community with a fait accompli - and notably, a fait accompli in which Canonical holds all the strings (by means of the Canonical CLA controlling contributions to the client end, and the closed source, closed shop server end that is owned entirely by Canonical).
well, I don't wanna make too many claims about Flatpak. It is pretty RH-ish. Just not to the extent Snappy is Canonical-ish.
they don't have to talk about it with all the 'self-important grey-beards', just all the people they heavily imply they're speaking for in the press release. you can't really claim to be building a cross-distribution app delivery system without actually getting any buy-in from any significant distributions at all.
Please read the last bit of the post, as it's specifically intended as a reply to this objection. I have no problem at all with Snappy itself or the people building it. I don't see it as a Wayland / Mir situation at all. This is why I've never said anything bad about it before now. The only thing I have a problem with is this ridiculously misleading press release which claims or implies that it's highly mature and ready for widespread use, and has support from many other distributions.
In a word...no, that's not how the GPL works. I can go into more detail if you like, but the GPL does not mean you have to GPL everything you ever loaded into emacs or something.
I already specifically mentioned that in the post - "The size issue was quite quickly resolved". I was quite clear about why I was mentioning it: the point is that at the time Canonical was issuing a press release saying how wonderful and mature and ready-for-use snap was, of the two useful things you could actually install with it, one was clearly a complete mess. That's not a great look.
Sure, and that's a valid context for it. But this isn't a press release saying "it's ready for use for specific purposes on Ubuntu phones!", it's a press release saying "it's ready for use on all Linux systems everywhere!" Which it clearly is not.
Sorry, I actually didn't know about AppImage when I wrote the post. From what I've read since, though, AppImage doesn't do the most novel / interesting thing that Snappy and Flatpak are trying to do, which is sandboxing. It's purely a package blob format, which is honestly not a terribly difficult thing to do.
I'd say all the technologies I'm aware of still have a lot of big problems to solve. One of the most obvious is that there is still, AFAIK, no standard or consensus on the underlying interfaces for the app bundles. Both Snappy and Flatpak have technology for this, but no-one's actually brought distributions or desktops together and made an effort to say "OK, we're going to define some standard interfaces that app bundles can expect to sit on top of, and we're going to collaborate to make sure those interfaces are available as Flatpak 'runtimes' and/or Snappy 'interfaces' across the major distributions". So even if you want to release your app as a snap or flatpak today, how do you decide what to put in the bundle and what to expect the underlying distribution to provide? You've got no framework to make that decision, there's nothing you can point to and say "OK, I can at least rely on the OS providing X, Y and Z". So you either make your bundle huge (and probably make it not integrate very well with the system, if it's bundling its own builds of all sorts of libraries), or you try to make it small and find it doesn't work consistently or even work at all across distributions, which was the point of the exercise in the first place.
For Flatpak there is at least a GNOME 'runtime' which is I think the primary target for Flatpak packages for now, but that's it, a single runtime that exists and that you can target, no more than that, and no written standards (AFAIK) that define what interfaces that runtime actually provides and that you can rely on it providing in the future. There is actually also a KDE runtime, but its description says "Do NOT use this in production, it's work in progress and it doesn't enable important safety features. It might also eat your pet or first-born."
I haven't checked this myself, but I think someone said the LibreOffice snap has no menu bar on non-Ubuntu distributions, because of the way Unity merges the menubar into the window header. That would be an example of the problem right there.
I didn't criticize any developers at all. I criticized a PR department. I don't think there's anything wrong with Snappy or its developers. I think there were an awful lot of things wrong with the press release Canonical's PR department issued about it.
And why do you think I wrote this blog post, if not to deal with 'perception'?
I don't care which one is best either. Have I ever written anything bad about Snappy before this press release came out? Nope. That's not because I didn't know about it, but because I didn't have any problem with it. I am personally all in favour of working together on things; you may note I've spent most of the last year working on Fedora's deployment of openQA, which is a system created by SUSE. Heck, I gave a joint talk with Richard Brown of SUSE at LFNW this year.
The whole reason this press release pissed me off so much is that it's exactly how you don't go about working well with others: by issuing press releases making grandiose and completely misleading claims about how great your system is, in an effort to effectively pressure everyone else to go along with your system - instead of actually working together with everyone else to come to a technical and social consensus. Do you know how we (Fedora) came to start using openQA? Because Richard spent one of SUSE's hack weeks setting up a proof-of-concept openQA setup for Fedora then came and talked to us (Fedora QA folks) in a friendly way to suggest we could work with it. That's awesome collaboration, and it worked out great for everyone. He didn't do his little proof of concept then go and issue a goddamn press release saying "OpenQA Comes To Every Other Distribution In The World", or something.
Have you noticed that there never seems to be any 'bad blood' between RH and SUSE? Or between RH and Arch, or SUSE and Arch, or Elementary and Arch or Elementary and RH or Debian and Arch or just about any other combination you can think of? No, whenever the 'handbags' come out, there's just one common factor: Canonical. Always Canonical. Ever wonder why that is? Because Canonical as a company seems to be congenitally incapable of actually trying to work together with others in good faith. I don't think this is the fault of Canonical's engineers, many of whom seem to be perfectly good people. I think it's a problem that comes from the top - from Mark - and has been built into the culture of their management and their public and community relations. They're always doing stuff like this.
I wouldn't say I've highlighted some 'shady secrets' of Snap. Nothing I mentioned is particularly secret - the technical people who actually built the packages and stuff didn't try to hide it, it's all right out there for anyone capable of understanding it to read. It's only 'secret' insofar as Canonical's PR department figured they could basically lie about it and no-one who read their press release would bother checking if any of its claims were actually true (by, e.g., asking any other distribution if they were buying into Snaps, or actually taking half an hour to look at the non-Ubuntu packages for snap and see what state they were in). And it seems they were mostly right.
Note to anyone wondering where their comment went: I reserve the right to nuke any comment for excessive sweariness (I know I swear: my blog, my rules) and no-one including me gets to use unpleasant gendered swear words on my blog.
It's still not acceptable to flat out lie in them, as Canonical did in this one.
I only just found this comment, but no, for the record, I don't "know" any of those things. Please note, I work on Fedora. Not anything else. Red Hat is big and I don't know what everyone in it is doing. It's possible some other team in RH talked to you about snaps (though based on your documented history of flat out lying about things, which I have documented here many times and which you've never actually refuted, only yelled at me about, just like you are here I'm disinclined to take your word for it).
But I did check with all relevant Fedora people before writing this post, and no-one said they had received or initiated any contact with you about snaps beyond an inquiry about whether it could be packaged for Fedora, to which we gave a pretty standard reply.
It's kind of funny how your response to this kind of criticism is always to suggest that it's the other person who wants to talk politics not technology. No, it isn't; it's always you who wants to do that, Mark. But you do it in such a passive-aggressive way - by making wildly controversial claims as if they were undisputed truths, then acting hurt, shocked, saddened and amazed when anyone has the temerity to challenge you - that you seem to have convinced yourself you're always being wronged...
Well, saying "oh hey, you can wrap some external sandbox system around our packages if you like" seems somewhat different to the Flatpak and Snappy designs where it's one of the core features of the whole system.
"Can you please post a link to the source code of Fedora’s store server? (I cannot find it)"
There isn't a store in the Flatpak system, exactly, and Fedora has not yet adopted Flatpak officially in any way. There has been code recently going into GNOME Software to support deployment of Flatpaks, and you can of course find that in the GNOME Software repo: https://git.gnome.org/browse/gnome-software - just check the log for 'flatpak' and, earlier, 'xdg-app'.
"Can you please provide proof that those people which are not marked on github as working for Canonical, that they really are Caonical employees?"
The only committers I can find to snap-confine and snapd are zyga - profile says "Canonical", jdstrand - no profile but almost certainly https://launchpad.net/~jdstrand , which lists an @canonical.com email, teknoraver - facebook (https://www.facebook.com/teknoraver ) and linkedin (https://it.linkedin.com/in/teknoraver ) list him as working for Canonical, mvo5 - profile says Canonical, Alberto Milone - https://wiki.ubuntu.com/AlbertoMilone says "I'm an employee of Canonical OEM Services Custom Engineering Group", fgimenez - profiles says "@CanonicalLtd", kyrofa - profile says "Canonical", niemeyer - profile links to http://niemeyer.net/ which says "I'm part of the Canonical team since September of 2005", chipaca - profile says "Canonical", pedronis - profile links to http://www.lucediurna.net/ , which says "Working on the backends for services behind Ubuntu like Push Notifications & U1DB @Canonical", pete-woods - profile says "Canonical", caldav - David Callé, his posts in the snappy archives (http://comments.gmane.org/gmane.linux.ubuntu.devel.snappy/1706 ) list a canonical.com address. How'm I doing so far?
"How is this post constructive?"
That's an odd question. Do we have to be 'constructive' in response to bullshit chicanery? When Oracle pulls some bullshit, does everyone have to be 'constructive' in their response? When Microsoft pulls some bullshit, do I have to be 'constructive'?
Being constructive requires good will on all parts, and generally doesn't make the news. If you read my archives, you'll find acres of 'constructive' posts which didn't make the news and thus don't have 30+ comments. But yeah, I reserve the right sometimes to point out bullshit instead of being 'constructive'. It's my blog.
"Do you know that Fedora had similar CLA? How does it makes it better now that they changed it?"
No, it did not. We had something called a CLA, but it was not at all similar to Canonical's. This has all been written about extensively by others; https://mjg59.dreamwidth.org/29160.html is a good concise explanation.
"How does Canonical’s CLA makes it worse than other big names are using (Apache, Qt, Python, Fedora)?"
Again, see https://mjg59.dreamwidth.org/29160.html .
"A last point: xdg-app first commits where in 14 december 2014 (first test release as seen from mailing list was in may 2015), snappy’s initial release was on 9 december 2014 and from my knowledge it evolved from click packaging system.(this is what I found from my search)."
Sure, that's approximately what I found. It's more complex than that, though, as both can claim earlier projects going back to 2009 or so as ancestors in some sense or other.
Well, that's nice. I don't write my blog for "guys at Data Centers", though, and I'm pretty sure I didn't say that was who my blog is for, so...maybe don't read it?
One of the two major problems with the press release is that 1) does not actually work properly yet. On non-Debian-ish distros, confinement is entirely disabled at build time. Even when it's built, it is literally impossible to effectively confine an app running on X11 (only Mir or Wayland), and almost everyone still runs on X11 - see https://mjg59.dreamwidth.org/42320.html . So this is exactly the problem I'm talking about: Canonical is happy for you to run around talking about how Snappy "does security isolation", when in fact it doesn't at all, yet.
"It looks like this web page is not really up to date."
Well of course it isn't. It's a blog. Blog posts aren't supposed to be up to date. My god, I'm imagining a world where I have to wake up every day and update every blog post I ever wrote, and it's terrifying. ;)
Because of the press release, lots of people started playing with snappy, and built more snaps. So of course there are more now then when I wrote the article. But also, I think there's a difference about where we're looking for snaps, I think I was looking somewhere different than you; there definitely are LO and Krita snaps, they were referenced in the PR and articles.
Anyhow, my general point is that when the press release was issued not a lot of stuff was available as a snap, and those that were available were not exactly polished, end-user-ready stuff.
You're replying to a comment that's more than a year old, on an article that is too. I'm very likely the only one reading it, and that's only cos I happened to notice it in the admin interface.
AdamW: Proudly Being An Awkward Bastard On The Internet Since 1993
OK, comments are closed, folks. I think we're done here.
This wasn't about the technologies, but the PR. FWIW, Flatpak is more of a desktop-focused thing than Snappy. I think the people involved with Flatpak generally think there are already enough tools for container-ish server deployment.