The continuing state of contact + calendar synchronization suck

I've written about this before and now I'm doing it again! Fun for all the family.

Since that post I've continued trying to find a reliable way to share my contacts and calendar between my phone and multiple computers, and still haven't found one yet. Mostly this is down to my phone being an N900, sadly enough.

It's kind of ironic given that everyone has a hard-on for the cloud these days, but it seems weirdly hard to get many people to understand the concept that you might want your contacts to be the same no matter whether you're using your phone or your computer at the moment.

Microsoft gets this. Being Microsoft they got it and then implemented it in the most terrible, terrible way possible; anyone who's actually used ActiveSync (or tried to write a third-party implementation of it - pity the poor SyncE developers) knows what I mean, but they at least got the basic idea. They have a mechanism via which you can make your Windows PC and your Windows Mobile phone (and your Exchange-based company, if you run one) sync up your contacts and calendar, and it more or less works, no matter how crappily it's architected. So they get a C.

Google gets this. Being Google they have an internal identity crisis, so they more or less use existing open protocols but try as hard as they can to pretend they don't. Google Calendar seems to support CalDAV. Kinda. More or less. As far as I can tell, it's kind of believed that Android syncs with Google Calendar using CalDAV. Kinda. More or less. (As far as I can tell, there's no CardDAV support for contacts; there's apparently some kind of mechanism besides ActiveSync, as Evolution can use a Google contact list, but I don't know the protocol it uses). But what you get on Android is a contact / calendar stack that can sync with Google Calendar but not with any other CalDAV-supporting server / service; there's no provision to just enter server information. (There are third-party implementations of generic CalDAV / GroupDAV support available). So if you run your systems or business on Google and use Android phones, you can keep your stuff in sync easily; but only if you're happy for Google to be looking after all that confidential information for you. (Android also has a pretty good ActiveSync implementation, so bonus marks for interoperability there). Google gets a B: marks for a reasonably well-designed and working system, marks knocked off for wanting to own all your data and seemingly intentionally pushing the open protocol on which their system is based way into the background. The third-party clients on Android seemed to have issues when I tried them, but I haven't tried for a bit.

Apple gets this, and actually has generic CalDAV and CardDAV support (CalDAV as of iOS 3, and CardDAV as of iOS 4). They also support ActiveSync. Their server side also has CalDAV and CardDAV support. So, uh...wow. Through gritted teeth, an A for Apple.

Maemo and Meego...really don't get this. Maemo's whole synchronization story is based around SyncEvolution, which is a phrase that's probably enough to strike fear into the hearts of most Maemo users already. From what I can tell, sadly, so is Meego's.

What's the problem with this? The problem with this is two-fold.

One, SyncEvolution is the wrong approach. It's a synchronization tool.

Two, it doesn't fucking work.

To take point 1 first: synchronization is the 2001 way of looking at this whole issue. It's back in the stone age. It's how Microsoft's approach works, and that's always a good sign you're Doing It Rong. ActiveSync is better in that it does actually work, though. Synchronization means that every system in the loop (the 'synchronization group', or whatever) has its own private, somewhat-independent store of the data in question, and every so often you get them all to compare notes and try to resolve their differences so everyone's private, somewhat-independent store is temporarily in the same state.

This is not Web 2.0. This is not The Cloud. This is not any buzzword you care to name. In this case, the buzzwords actually happen to be on to something. By far the better way to do this is the good ol' client-server approach, or what we currently like to refer to as 'storing data in the cloud'. Each system in the loop should only have local copies of the data as a cache so they're not entirely useless when offline. The proper approach is the Google approach: there's one 'true copy' of the data and it lives on a central server. Clients retrieve that data, and send changes to the server immediately (or as soon as you get online). It's hardly something new.

Meego, in a crowning moment of WTF, seems to be going out of its way to screw this up: they have implemented support for the right protocols - CalDAV and CardDAV...as backends for SyncEvolution.

Excuse me while I go and bash my head against my desk for a while. It's just the wrong design. It makes no damn sense to crowbar support for a client/server protocol into a synchronization app...especially when Meego's messaging stack is based on evolution-data-server in the first place. Which, in case you didn't know, has CalDAV and CardDAV (WebDAV) support already. By far the better design would simply be to allow you to create client accounts for remote calendars and address books in Meego's calendar / address book UI, just as you can in Evolution on the desktop.

I mean, let's really hammer this point home: imagine if Meego didn't have a UI for setting up an IMAP email account. Instead, it just had an email app which accessed an unchangeable mbox account on your phone, and you had to run some separate application which you had to fire up every so often to 'synchronize' the mbox file with an IMAP account; it'd go out and take a look at the IMAP account, figure out which mails had shown up since the last time the local mbox file was synchronized, check if read/unread/replied states had changed, pull down all the data, and update the local mbox file...push local changes (mails sent, statuses changed and so on) off to the IMAP server...then log out again.

Would anyone think that was a remotely sane system? I rather doubt it. So why does it seem at all sane to expect everyone to do it for their calendar and address book?

Here's the bug report where this seems to have been decided, but it doesn't look like it was really looked at from an overall perspective: the reporter asks whether this should be done in Dates, but pohly doesn't seem to consider it, and no other Meego developer replies.

Whew. So, Point 2. That one's a lot simpler...and it's just what it says on the tin. I haven't managed to make SyncEvolution sync reliably with anything. Its flakiness in trying to sync with Google is notorious; it seems like it will do an initial sync, but pretty much stop working the instant you actually change any data and then try to sync. I've tried syncing it with various open source groupware server suites using SyncML, notably including egroupware and Horde, and can't get it to fly properly with any of them. Notably, with egroupware, it will happily run a sync with good-looking logs as often as you want, and then no data will actually show up in egroupware. With Horde, it wants to do a slow-sync every time, which results in tasks and contacts being duplicated every time - rendering it useless.

I've spent hours trying to fix these problems and got nowhere. I don't know if the ultimate bug is in the client or the server in each case. I just know it doesn't damn well work, and it makes me tired and pissy - no-one's paying me to do this, I just want my damn data to be consistent and available. I am un-motivated to spend too much time trying to fix it, too, for the reasons given in Point 1: synchronization is fundamentally Doing It Rong. It's a concept from the days of Palm Pilots which had no radios and which you plugged into a computer twice a day to pick up the latest information, it makes absolutely no sense in the days of cellphones with wifi radios and always-on mobile broadband data connections.

Can't we just get proper damn support for CalDAV and CardDAV and/or GroupDAV into everything, but most especially into Maemo / Meego (I'd love to say Android as well, but you know Google. Here's a thread where Yahoo shows up with a CalDAV and CardDAV implementation for Android, all ready to go. Last word? From the Yahoo submitter..."Any update here? Haven't heard anything yet. "), in a sensible way? It'd make life so much easier, and it's the Right Way to do things.

Comments

[...] now I am reading Adam’s blogpost on his suffering with the state of synchronization and I feel at least a bit of consolation that I [...]
m20554443 wrote on 2011-04-15 09:13:
Hi, I am using SyncEvolution v0.9 (GUI) on my N900 to sync Google contacts. This works like a charm. SyncEvolution provides a template configuration for Google's SyncML. Unfortunately Google isn't providing SyncML for calendars :(
adamw wrote on 2011-04-15 15:04:
I know it does, but it's never 'worked like a charm' for me. For me, any time I change any data on the phone after the initial sync, future syncs will fail to work. Which makes it rather useless.
pohly wrote on 2011-04-18 10:09:
Adam kindly contacted me regarding his post. We had a private discussion via email. A public response from me can be found here: http://syncevolution.org/blogs/pohly/2011/state-syncing-open-source In a nutshell, I think that SyncEvolution and the things I am trying to do with it are sound. Its scope is limited by a lack of external contributors. What is missing is a open source groupware to complement it, for those who don't want to depend on commercial or advertisement sponsored services.
adamw wrote on 2011-04-18 15:09:
We're not short on open source groupware suites, but I can get behind the idea they need to work better with syncevolution.
[...] it should show that synchronisation [of PIM data] can work reliably and as such it opposes a wide spread [...]
matej.ceplovi.cz wrote on 2011-09-17 16:05:
@adamw we are short on *working* groupware projects ;). IMHO, there are two ... Zimbra (with 400+MB of the installation packages ;)) and Zarafa. Trying the second one and it seems like doable.
[...] of the SyncEvolution commented on mine and Adam Williamson´s blogpost about problems with synchronization. Patrick asks in his post: “I’m not sure what kind of [...]
adamw wrote on 2011-09-17 18:52:
mcepl: last time I looked, zarafa could only do ActiveSync, and it only did that through an add-on (z-push) which is not f/oss.