Akonadi goes Web2.0

Lots of people are posting about implementing functionality in web browsers to do what we do today in rich clients.

I don’t know what’s going on, but I didn’t want Akonadi to miss the party. At the recent Akonadi sprint, I decided to spend some time putting together a proof of concept for a web client for Akonadi. Here’s a screencast of the result:

Ogg link.

The video shows web pages in multiple web browsers showing the same data as KMail, KAddressBook and the proof of concept gtk applications I wrote about before, with everything kept in sync. They share the same data. I could edit the same data in the web pages too if I had taken the time to write the javascript code to handle it.

One of the advantages Akonadi can bring is to free us from the web browser and put our data and user experience into rich clients. Why the perversion of putting it all back in to a web browser? Initially, I started writing it just to try out the rich web client technology webby people talk about, and to see if Akonadi has what it takes to go Web2.0. First requirement is a catchy name. Check.

The zeroth requirement for any application is use cases. Where could an Akonadi web application be used? One idea would be to put a finished version of this technology onto a public web server and allow the public to create accounts and connect to multiple different data resources like email providers and unify them all in a single interface like many people do with KMail and other rich desktop client apps. However, that would not be practical as a service to provide to the public. Akonadi is designed to handle lots of resources for one single user. Using it to handle lots of resources for lots of different users probably wouldn’t scale in its current form.

I can think of two valid use cases though. Consider for example gOS, or Google Chrome OS, which encourage the user to access everything through a web browser and seem to be targetted at particular segments of netbook users. To access different email services on a gOS system, users would have to access different web applications each with different interfaces, and different usability designers deciding how to interact with the system. Using Akonadi in a web client, vendors like gOS and others could design one consistent interface which allows users to access all of their data of any type. Because the server and the client are on the same machine, the vendors would have complete control over the appearance and interactions with the system. Users who want to try out a rich client application can just install KMail and pick up where they left off. No need to import the data from elsewhere (and creating diverging copies). The data is already shared.

Another use case is for developers who find it easier to develop application UIs in html and javascript instead of C++ or want to incorporate web slices. They could simply put their html into a QWebFrame, and they’re done. A new application without a recompile. Wacky ideas and platforms to host them. Check.

I knew that a web page showing Akonadi data would need to change asynchronously, and not cause a page to reload when selecting a mail folder. jQuery is a javascript library with facilities for exactly that and with plugin support. I used the jstree and dataTable plugins to put the page together. Check out the demos here and here of what else those plugins can do. Asynchronous user interaction. Check.

The web browser needed a way to get data from Akonadi into the browser when requested. That means implementing a server capable of responding to requests from the web browser, and capable of communicating with Akonadi on both parts of its interface, D-Bus and IMAP. Finally, I needed a way to deliver change notifications to the web browser without the browser requesting anything. This is variably called HTTP pull, long polling and Comet. I first saw it described here last month, which covers the concept pretty well. Rather than needing to keep events synchronized though I only needed to keep sending notifications from Akonadi (which may be rare) to the browser.

Rather than learn the details of a Comet implementation though, I wanted a ready made solution to abstract the details away from me. Orbited is a Comet framework written with twisted which can be used with the comet server protocol and js.io, a javascript library simplifying network communication in javascript environments. I already had an implementation of a twisted akonadi client library to get the data to orbited, and Orbited would take care of the rest of the transport. In the browser html, js.io provides a listening socket to recieve all notifications and asynchronous data responses. A big thanks goes to desmaj on #orbited who got me started with a small contained comet server and client. All I had to do was add Akonadi bits to it, and a bit of jQuery glue later the result is in the screencast above. Akonadi in a real-time web application. Asynchronous two-way data transfer. Check.

I noticed yesterday that Mozilla announced Raindrop. They’ve clearly spent more time on it than the two days did. I haven’t looked into their implementation, but it’s interesting to note both of us trying out these concepts at the same time. Multiple different projects with similar but distinct goals. Check. (Akonadi is a cooler name anyway 😛 )

The implementation of this is messy because my C++/python ideas of how to write code and classes get thrown out the window when using javascript, mainly because the ‘this’ object in javascript doesn’t always resolve to what I expect it to when thinking in C++/python design mode. To get it done I did some ‘code sharing’ with copy and paste. It’s just a proof of concept anyway, so that doesn’t really matter. I’ve put the code in the same git repo with python-twisted-akonadi (Sorry V 🙂 ).

Advertisement

Cross platform Akonadi video

Here’s my very first screen cast showing the synchronisation between Akonadi applications using different APIs:

OGG link

In the video, I change addresses in KAddressBook, and the entries in gtkAddressBook are immediately updated too. It would work the other way around too if the code was more complete, but python-twisted-akonadi is just getting started, so most features are not implemented yet.

This is a follow up to my previous blog about a Gtk Akonadi library.

Model/View design

The reason the synchronisation works is all down to the design of Akonadi. There is an architecture diagram here showing the basic principle of the design. The main feature is that all communication with Akonadi is done in a platform neutral way using D-Bus for notifications and an IMAP-like protocol for data transfer. One of the principle benefits of that is that most modern platforms already have libraries for dealing with D-Bus and IMAP, so there is a natural starting point for any Akonadi client implementation. The KDE API and the python API hide those implementation details, and provide ways for applications to fetch, delete, change and move items, and to receive notifications when another process or application performs one of those actions. That means that any application written in any language on any platform can access and modify the data in Akonadi, and the changes would be synchronised everywhere else.

The applications are not communicating with each other via plugins. That’s KDE 3 design and it doesn’t scale very well and leads to a lot of duplication. For example, a fully working Kontact has the complete address book in memory 6 times! The applications aren’t even aware that other applications exist. They’re all part of a macro scale Model/View pattern where each application is a View, and Akonadi is the Model.

This could be useful if for example you want to use gtkAddressBook instead of KAddressBook (hypothetically 🙂 ), but you still want auto-completion of your contacts in KMail. Because the contacts are accessed through Akonadi, not the address book application, that would work just fine. For the user, it seems that ‘My contacts are in KAddressBook’, but because the technical reality is different, any other applications can use the same data in a versatile way.

Low level implementation

So how does a motivated developer implement Akonadi support for a new platform? A good starting point is finding existing frameworks for IMAP parsing, asynchronous communication, Unix socket communication (or named pipes on Windows), flexible event loops, and D-Bus support. As I chose to write an Akonadi library in python, I could easily find existing code for all of those pieces. python-dbus provides the D-Bus support, and I was quickly directed to Twisted python which provides the rest.

Twisted has some central components like reactors which provide an event loop and reading/writing to network sockets, deferreds, which fulfil a role similar to KJob in that they make asynchronous programming easy, and implementations of clients and servers in many different protocols. The only one needed for Akonadi is twisted-mail, which provides an IMAP parser and lots of convenience to hide the implementation details of the actual communication of the data.

I’ve written several times that the protocol Akonadi uses is Almost IMAP. The protocol mostly follows the IMAP standard, but it has some non-standard extensions which bring the flexibility and performance needed in a PIM system. One of the benefits of the python language is that all methods on classes behave as if they were virtual methods in C++. They can be overridden in subclasses and the overridden version will always be called. By implementing an overridden method in the twisted.imap4.IMAP4Client, I could implement support for the non-standard extensions and let the base class handle the rest (all the difficult stuff).

The next step was to enable serialisation to and from standard PIM types, like RFC822 messages, vCards, iCal entries etc. Again, python already has several high quality libraries for parsing and generating these types, and thanks to dynamic typing, handling the types can be done in a generic way, leading to even more type independence in the library.

The bit about models

The high level part of the KDE Akonadi libraries is the EntityTreeModel which simplifies data in Akonadi into a tree structure for use in a GUI. High level API will always be toolkit dependent, so I needed to choose a GUI toolkit to get the high level stuff started. PyGtk has a TreeStore class which provide an API somewhat similar to QStandardItemModel in that it allows creation of a tree of items on a conceptual level (the model), a TreeView class, somewhat simliar to QTreeView, and a CellRenderer somewhat similar to a QItemDelegate. Gtk also provides proxy models for sorting and filtering.

The gtkAkonadi.EntityTreeStore is the conceptual equivalent to the EntityTreeModel in the python API. It is a generic model for storing the data from Akonadi. There are also two existing subclasses, the MailTreeStore and ContactsTreeStore, which make a tree of Akonadi.Items appear as emails or contacts respectively. The rest is pure PyGtk code to make a real application out of those classes.

Fin

Akonadi with interfaces in multiple toolkits

PIM could be shared across the whole free desktop, with cross platform Akonadi at the center of it all. Getting this far with the python API was remarkably simple, considering that I only started writing the code a week and a half ago, and didn’t have a lot of time to spend on it. There’s a long way to go, but if you want to help out on the python API or another client for any other toolkit, we can get you started on kde-pim@kde.org or #akonadi on freenode.

Email rendering with Django

I’ve just pushed some changes to give the gtkEmailReader theming support.

Do those themes look familiar? They should. They look the same as the themes I made for the KDE themable email reader. In fact, the theme files themselves are almost identical.

The KDE email reader is themed using the Grantlee template system, which is itself a port to C++ of a python technology, the Django template system. The templates themes themselves are designed to be mostly source compatible between Grantlee and Django, so that made the choice of theming system for the gtkEmailReader an obvious one. The only changes I needed to make to use the Grantlee themes in gtkEmailReader was to change some variable names, and account for some differences in html rendering between QTextDocument and gtkhtml2.View.

In theory it would be possible to make some simple changes to the themes and really use the exact same templates for both applications, but it would be a pain to maintain as the applications and themes themselves change. This is still a good demo of the source template compatibility of Grantlee with Django though.

Holy Grail? No thanks, we’ve already got one!

KAddressBook and gtkAddressBook with same data

On the left is KAddressBook from KDE4.4, written in C++ with the KDE/Qt platform retrieving data from Akonadi. On the right is gtkAddressBook written in python with the PyGtk platform retrieving data from Akonadi.

KMail and gtkMailReader

On the left is KMail 4.5 (not ready to be released with KDE4.4), written in C++ with the KDE/Qt platform retrieving data from Akonadi. On the right is gtkMailReader written in python with the PyGtk platform retrieving data from Akonadi.

In both cases, the actual data – the emails and contacts – come from a kolab server, but none of the applications have any code that it aware of what Kolab, IMAP, maildir, POP etc are. Akonadi handles all of that. Both applications of the same type on different platforms show the exact same data and Akonadi keeps it synchronized. If I edit a contact in KAddressBook, the gtkAddressBook is updated automatically and immediately.

The technical details of this will be in a later, longer blog post, but the basic message is that I’ve started working on a proof of concept Akonadi client library written in python using the twisted framework. I’ve also started a more high level proof-of-concept PyGtk Akonadi library, and two concept applications to use it. I could of course have used PyQt or PySide for the Gui, but I wanted to make a point that Akonadi is for GTK too. It is truly cross platform. An Akonadi client library is a library that communicates with the Akonadi server, where all PIM data can be stored. We already have a maturing library written for the KDE platform in kdepimlibs, which KAddressBook, KOrganizer, KMail etc are currently being ported to. There is already a proof of concept Java client library for Akonadi, and now there’s a proof of concept for python too.

What next?

Python already provides useful facilities for PIM data. There are existing classes for handling different types of data, like emails, vCards and iCalendar. This adds possibilities for writing Akonadi resources not just in C++, but also in python. You could write a new Akonadi resource for the next social network everyone must join, and feed its data into Akonadi, and all the applications would not have to be changed, and nothing would need to be recompiled.

Another possibility is using scripting to modify or handle the data in some way by writing Akonadi agents in python.

Also, because the Akonadi server is written in pure Qt (no KDE dependency), and python-Akonadi does not depend on KDE either, this could be used to create a new PIM framework on platforms where python and Qt are available, but KDE is not. Such a framework could be attractive to third party developers to create PIM applications without considering the transport mechanism or details of storage. This would be a bit more work, but given that there are already frameworks like twisted to handle transport and other libraries to handle storage, it would not be excessive.

I know the gtk applications don’t look good or useful. They’re not supposed to. I had never written anything with the Gtk API before yesterday afternoon, so the only purpose is to demonstrate that the data is available and comes from Akonadi. If you want to improve any of this the code is on gitorious here.

Cache invalidation in Akonadi models

Somewhere hidden among the mass of words in my last blog entry I described an important drawback of using a single EntityTreeModel to lazily fetch and cache the Items in a tree, instead of fetching them anew each time.

The problem is knowing when to invalidate the cache. If no application is currently showing the contents of a Collection, there is no need for the Items in that Collection to be fetched, cached and kept up to date in the model. The effect we would like to achieve is to purge the Items in a Collection when those items are no longer shown anywhere in the application. Generally, that will mean that the Collection is not selected.

In Qt Model-View, the application data is stored in a model, and there may be one or more views attached to it displaying its contents. The model doesn’t have any knowledge of the views, and so it can’t know whether any particular Collection is selected, and purge its Iitems.

To solve this, I turned to the KSelectionProxyModel, which already has a lot of code for managing the selection a user makes in a view. By subclassing it, I was able to implement a reference counting system which increments the refcount of a Collection when it is selected, and decrements it when deselected. If the reference count of a Collection goes down to zero, it is put in a queue to be purged. It is not purged immediately, but queued because if the user is clicking around several Collection in a short time, we don’t want to purge the Collections each click or we’d lose the benefit of the caching. Like other similar optimisation techniques, this violates the object-oriented principle of modularity, but it is worth it for the benefit it brings. The effect can be seen in a customised version of our development tool akonadiconsole.

In the screenshots below I removed the filtering out of Items in the tree so that the fetching/purging can be seen. In real applications, the Items in the tree on the left would not be visible.

When a Collection is clicked, its Items are put into the model. The rest of the Collections have no items

The Inbox Collection is selected, so its items are fetched. Personal Contacts is no longer selected, so it is put into a queue to be purged.

Select another Collection and its items are fetched too.

Another Collection is selected, pushing the Personal contacts out of the queue and purging them

If the Collection is selected again, its Items are refetched

For this example, I used a queue length of just two Collections, so that if a Collection was deselected two clicks ago, it will be purged. In real applications, a longer queue length will be used, but it’s harder to illustrate in screenshots. Another unrealistic part of this demo is that this feature will like be used in applications like KMail where Collections can contain tens of thousands of Items and fetching them is an expensive operation.

This feature should be totally invisible to users and even developers using Akonadi, but it should offset the main disadvantage of using a cache of Items in the EntityTreeModel.

Akonadi: It’s all Sausage

Preface

The Germans have a great sense of humour.

I’ve been telling people that for years, but I’m battling against established stereotypes. It always gives me a good laugh when I hear a new expression or turn of phrase that they use. Back in April sebas introduced me to a new one which still gets a chuckle every time I hear it: “Mir ist alles Wurst!”, or sometimes simply “Wurst!”. It is used to express indifference and translates as “It’s all just sausage to me”. The equivalent English phrase would be “It’s all the same to me.”

Introduction

I always think of “Mir ist alles Wurst” when I think of how Akonadi works. Akonadi is the next generation PIM system being developed to fill the needs of the KDE4 platform. There is work ongoing to port existing KDE PIM applications like KMail, KAddressBook etc to Akonadi, as well as bring it to new platforms like Plasma, because the data it stores is completely independent from the applications used to access and modify it. That is a design feature which makes it possible to access your emails in Mailody, KMail and plasma, and have it all in sync. Always. By using Akonadi, KJots notes can be synced between the KJots application and a plasmoid in plasma. Updates to the notes in either user context are automatically and immediately transferred to the other.

Of course, this is not limited to emails, contacts and notes. In fact it’s not limited at all. Throughout most of the Akonadi stack, all items are treated the same way from an API point of view through the Akonadi::Item class and are serialized and deserialized as needed when retrieved from the Akonadi server. The Akonadi server itself doesn’t care what type a particular Item is. Whether it’s an email, a contact, a note, or anything else, it is simply stored in serialized form, and can be fetched, replaced, removed, moved etc without knowing its type. It’s all sausage.

The way that a particular item is serialized depends on its type. Emails are all serialized and stored the same way. Contacts are serialized and stored the same way (But of course, differently to emails). Rather than using types of sausage to name the different types of data stored in Akonadi (opportunity missed IMO), the Akonadi designers decided to use mimetypes. Emails have the mimetype “message/rfc822”, contacts have the mimetype “text/directory”, etc. These types are not chosen by the Akonadi team, but are specified in several international standards and are core to how email and address systems function. The mimetype describe the structure of the data, and standard compliant libraries can be used to convert it into something that’s easy to use for applications. By serializing an email in “message/rfc822” format, not only KDE applications, but any application capable of deserializing that type can access and manipulate the data. In KDE the data gets transformed to a KMime::Message. In a python world, the same message could potentially be deserialized into an email.Message object and handled in a python only (read: Non-KDE) application. Again, all changes would be immediately and automatically synchronized.

Caches All The Way Down

It makes sense to keep this type independence in as much of the Akonadi stack as possible to allow reusing the same code for different types, including types we haven’t even thought about storing in Akonadi yet. The highest level API we have in Akonadi for dealing with Items is the EntityTreeModel. This class is an implementation of a QAbstractItemModel, which means that the data represented in an application is completely separate to the way it is interacted with. By using this class, any type of Item stored in Akonadi can be stored in a tree which mirrors the structure in Akonadi. In its vanilla form, the EntityTreeModel represents all items in a generic way- just the same as the Akonadi server, they have a mimetype, which is used to determine how to deserialize the Item, and several identifiers. It can also be configured to contain only a certain type, eg only emails, and processing only the subset, kind of like a sharding system.

EntityTreeModel showing all Items

The numbers in the tree represent Akonadi::Items. Each number is a unique identifier for an item. In its vanilla form, EntityTreeModel has 3 columns.

Akonadi is an asynchronous system. All interaction is done via Job objects which get created to fetch a bunch of items for example. Rather than waiting on the Job to finish and freezing the application UI until the items are returned, the job is “forgotten” until it reports that it has retrieved the items, or has falied to retrieve the items. In the meantime, the user can continue to interact with the application. This has many advantages, but it doesn’t suit the Qt Model-View API, which requires synchronous access, although the model can be filled asynchronously. For that reason, the EntityTreeModel maintains a cache of data from Akonadi, which is itself a cache of data on a remote server, or somewhere in the filesystem. Data can be retrieved and manipulated in the model in a way that seems synchronous to the application by using the QAbstractItemModel API.

Swings and Roundabouts

Having type independence in EntityTreeModel is useful from a library point of view, but for applications, it is necessary to know exactly what type the model is storing. That allows for example to show Subject, Sender and Date if the model contains emails, and Given Name, Last Name and Email Address if it contains contacts. The way this is achieved is by subclassing. EntityTreeModel forms part of a Template Pattern (Template Pattern, not to be confused with C++ Templates as in template class Foo;.). It generically handles fetching, inserting, removing, moving, drag and drop, lazy population and more. It also defines some extra API beyond the Qt Model API to make applications function as required for different types, such as representing different Items differently if they are emails or contacts for example. For those two cases we have subclasses of EntityTreeModel like ContactsModel and MailModel, but a subclass could be implemented for any type known to Akonadi.

MailModel showing only emails with contextual data

ContactsModel showing only contacts with contextual data

Notice that the Items in the tree, the emails and contacts, are not shown as numbers anymore, but show the contact details in the case of the ContactsModel and the Email subject etc in the case of the MailModel. Part of the template interface to implement when subclassing EntityTreeModel is to specify how many columns should be in the model, and what data each column should show. Notice also that there is a resource of tags. Each folder below that, eg “Something” is a Tag in Nepmouk, and the emails in that folder are emails tagged with the Nepmouk tag “Something”. That’s just a teaser for a future post.

Those are typically processed a little bit using proxy models to separate mail folders from emails so they can be represented in different views, for example. That’s a typical pattern in KDEPIM applications, but one not followed by the KJotsModel, which represents Collections of note pages in a tree together.

MailModel split by proxy models into separate views

The mail model split into two views using a series of proxy models. Notice that each view now has different header data. The headers read “Folder” in one view and “Subject”, “From”, “Date” in the other. Defining the header data to be shown is another part of the EntityTreeModel interface that subclasses should implement to customize its appearance. It’s not obvious how to do that using the Qt Model View API, but eventually I did find a way to do it relatively cleanly. To see the result of the split in the case of contacts, see Tobias Koenigs excellent blog post.

One of the more interesting implementations of EntityTreeModel is the AkonadiBrowserModel. We have developed several tools to make Akonadi development possible, including akonadiconsole, which allows browsing the contents of Akonadi, debugging the low level jobs and the notification system and more. The AkonadiBrowserModel is a browsable tree of Collections and Items in Akonadi without any filtering. It can represent the data differently whether the contents of the Item is an email, a contact, an event etc.

AkonadiBrowserModel showing different representations of data

Akonadi browser model showing items in their generic form, represented as emails, and represented as contacts. This is implemented internally using a State Pattern. There is an individual State object for each way the data is to be represented.

If Collections and items are usually separated, this begs the question of why keep them together in the EntityTreeModel in the first place? The answer is to make better synchronous caching possible, and to minimize the number of classes to maintain. The previous design used separate Models for Collections and Items. Whenever a Collection was selected, like a mail folder for example, the items contained in it would have to be retrieved from Akonadi. If a different Collection was then selected, the old items would be discarded, and might have to be fetched again later. Additionally, Having separate classes for these things meant implementing and maintaining in two places all the logic to store and keep up-to-date the synchronous cache of Akonadi data, handle drops etc. It’s almost twelve months since I started writing EntityTreeModel, and I can confirm that that’s not trivial. There’s a lot to consider such as handling drops, insertions, removals and moves of items and collections etc. I did toy with having a separate class for actual storage, and making models use that internally, but that turned out to be the wrong approach.

Finally, KJots required that it was possible to represent them all in one tree, so that’s what’s important. You’ve gotta look out for number 1 :).

Putting Collections and Items together in one model also has some disadvantages. Because items are not removed when a mail folder is un-selected, the amount of data stored in the model can get large and must be invalidated. Also, because the entire data structure of Akonadi is represented by default in EntityTreeModel, application writers have to be a little bit more specific about what they want their instance of it to include. Finally, it provides a single point of failure, and I’ve been breaking it a lot recently to add new features. It’s swings and roundabouts, but this is the working solution today.

Just because you have a model doesn’t mean it must be put in a view. EntityTreeModel can perfectly well be used without a view. That is how the Model for retrieving Grantlee Templates works. Nowhere in the UI is the tree of template data shown, but it does exist in the AkonadiTemplateLoader. It is essentially a headless model. The match() method is used to locate wanted data, and all notifications from Akonadi about templates being changed, inserted or removed are handled internally by the model. By using the lower level Akonadi API like Jobs and Monitor directly, application writers would need to ensure that they connect to Monitor signals related to the items they retrieve using Jobs. I have doubts that application developers would remember to do that each time, so hopefully this “Headless model” approach to accessing data from Akonadi can be used to full effect in the future. EntityTreeModel is missing some pieces to make it a general solution for that currently, but it should be in place in a few months.

Obligatory Conclusion

Akonadi deals with any type of data you would care to use it for. Serializers exist to serialize data to emails, contact, events, templates, notes, images etc, and writing new serializers can be trivial. Throughout the Akonadi API we have tried to keep functionality of dealing with items separate to knowledge about the type of data in those items, and allow a lot of versatility in how that data can represented to the user. This means that many Akonadi using applications will have a similar code design, and share a lot of code. The application design is of course a separate issue, and limited only by creativity in writing QAbstractItemView implementations. It is too late to rename Akonadi::Item to Akonadi::Sausage because that would be binary incompatible and source incompatible. EntityTreeModel is @since 4.4 though, so I cold potentially rename it as SausageModel. I’m not sure it would pass API review though…