The Turtle^W Indexes move

Last week a patch I submitted to Qt was merged into master, which will eventually be Qt4.6.

The patch is interesting, I think, as it takes a lot of workload away from model and proxy model implementors. The TL;DR version is: It’s now easier to move rows and columns in your QAbstractItemModel and QAbstractProxyModel implementations. Instead of figuring out how each persistent index should be updated between layout{,AboutToBe}Changed signals, you just call beginMoveRows(…) and endMoveRows(). This causes rowsAboutToBeMoved and rowsMoved signals to be emitted by the model.

Since January or so I’ve been implementing a complex generic model for representing a tree of pim data from Akonadi, and several proxy models which when used together make it possible to implement efficent, portable applications using the Model/View framework. In the Model/View pattern, a single model may have several views observing or changing it. The model has no mechanism for knowing which views are attached to it, so it must broadcast to any observing views any time some state in the model changes. That’s why there are protected methods in in QAbstractItemModel like beginInsertRows and endInsertRows. The model notifies observers that it is about to change, and that it has completed the change. That way, consistency is always maintained, and the views can update their visualization of the model.

There are also existing methods for notifing about rowsAboutToBeRemoved and rowsRemoved etc, but there are no methods for signalling that rows are to be *moved* in the model. One way to solve that problem would be to remove the rows from one position, and add them to another position, however, that would mean that any information about which items are expanded or selected in a view would be invalidated. Also, this is a slow method of moving things. Compare using the mv command to move a directory containing gigabytes of data, versus using the cp command followed by rm. The mv command is far faster because it’s essentially a metadata change on the filesystem. Similarly, when implementing models, if an index is removed, all of its child indexes must also be removed, and then reinserted (possibly re-fetched if the model works asynchronously). Remove + Insert is obviously not an acceptable situation.

To solve this issue, the model emits layoutAboutToBeChanged and layoutChanged signals before and after moving, and in between, the model implementer must assure that all QPersistentModelIndexes are updated to their new positions. This ensures that selection is not broken, and that child indexes of moved indexes are not affected by the move. Implementing model moves can be tricky because you have to make sure that moving an item to one of its own descendants can not happen for example. In addition, the layout{,AboutToBe}Changed signals don’t convey any information about which rows or columns are being moved. That means that any proxy models in the application would have to take persistent model indexes of every single item in its source model when layoutAboutToBeChanged is emitted, and then when layoutChanged is emitted, compare the stored persistent indexes against the indexes currently in the model, and then update its own persistent indexes so observing proxies and views stay up to date. In applications a with a very large model and several proxies it would be a large performance hit for each proxy to take persistent indexes like that each time an item is moved, particularly in applications such as KMail, where it is not unusual to have mail folders containing tens of thousands of mails, and on platforms with less powerful hardware such as mobile devices.

So we have development complexity in implementing moves in the root model, and a performance hit in proxies. To solve this for Akonadi, I initially created an AbstractItemModel class and an AbstractProxyModel class containing new API for emitting move notifications. Then, instead of calling layout{,AboutToBe}Changed, and futzing around with QPersistentModelIndexes, I’d just call beginMoveRows(source, startRow, endRow, destination, destinationRow) and endMoveRows(). I was able to unit test it so I knew it worked, and I didn’t have to worry about writing buggy implementation code in the multiple proxies I was writing.

Eventually, though I decided that having those extra abstract classes was the wrong approach, and could easily be broken by accident in the future (it uses a mysterious and ugly reinterpret_cast hack). So after meeting Trolls Olivier and Marius at GCDS, I instead rewrote a better patch and better unit tests for QAbstractItemModel itself and sent it upstream to Qt for everyone to use.

The system works.

Email rendering with Grantlee

The GCDS videos have been online for a while now, but I thought I’d draw some attention to my talk about Grantlee for those that missed it.

Video — Slides

The demo which gets most peoples attention is the example_mail application. example_mail is a small demo application which accesses emails through Akonadi. It’s not intended to be fully functional, but currently it can read email.

I ported it to use Grantlee for the rendering of emails, which allows for theming support. That in itself is not really special, but it gets more
interesting when you consider that the Grantlee themes and images themselves are coming from Akonadi too. There is a Grantlee loader for retrieving templates from Akonadi, and an Akonadi resource for storing and handling updates Grantlee themes. Go orthongonality!

You can imagine a rollout of 10,000 workstations, with a customized theme in the KMail reader with the corporate identity of a customer, and using the right colours and logo.

If one day that company gets a new identity or is bought and rebranded, each of those workstations will need to be updated with a new theme. How do you update all of the 10,000 workstations? With Grantlee and Akonadi, you simply update the theme in one place, your groupware server, which pushes the update, and every workstation is updated instantly. I mean really instantly. No need to push a refresh button, or to select another email for it to be re-rendered with the updated theme.

The way the theming works is quite interesting (I can write about how the instant updates work another time). There is a base.html file, which describes a skeleton html file with some css style. It also has a little bit of extra markup, such as {{ messageContent|safe }}, which tells Grantlee where to put the actual text of the message, and some {% block %} tags. The {% block %} tags don’t have any effect in the base.html file except to indicate that the text between the blocks can be over-ridden. There is one block defined with some css inside it, one block above the email content called "header", and one block below called "footer".

The file template.html contains a tag at the top with the content {% extends "base.html" %}. That indicates to the Grantlee system that this file will inherit from the base.html file and override some blocks in it. Each theme file can override as many or as few blocks as they want. template.html only overrides the header block from base.html, and inserts the table you see in the first screenshot above. Images in the are represented by <img src="{% media_finder "qtlogo-2007.png" %}" /> for example. The media_finder tag is a tag which inserts a url into the rendered theme. In this case, I’m retrieving media using Akonadi, so it will be replaced by an Akonadi url, something like akonadi:item=6. There is already a plugin for retrieving themes and media from filesystem, which could be configured to use KStandardDirs. By next week, there should be a Grantlee plugin to load themes and media from the Qt Resource System. There are also some more variables in the theme file- {{ subject }} is replaced with the subject of the email, and {{ from }} is replaced with the name of the person who sent it.

The way the date is represented shows another feature of Grantlee. {{ date }} in the theme is a QDateTime object. The object can be passed through a filter which processes it before rendering. For example, the date and time filters allow the template writer to use QDateTime string formatting to render the date in different ways. The timesince filter shows how much time has passed between that date and now.

The rendering of the second screenshot is done with this theme. It is mostly similar to the first one, except that it also overrides the style block from base.html to define some more css. It also shows the use of a special {{ super }} variable. When used inside a block, {{ super }} contains the content of the block it overrides. In this case, the style from base.html is added to the rendered theme, and then some more css is defined inside that block.

In Grantlee terminology, the stuff inside {%  %} are called tags, and pipes("|") separate filters from the variables they operate on. There are many more tags and filters than just media_finder and date. See here for a list of all the tags and filters available by default. Note that downstreams can also define their own tags and filters using c++ or javascript/QtScript.

The code for this is in playground/pim/akonadi if you want to take it for a test run yourself. You can update the theme in real time using the akonadiconsole tool. The usual warnings about putting the kids safely in another room apply here though. This example requires KDE trunk to run, but Grantlee itself requires only QtCore4.5.

Lastly, don’t worry if you think my theming skills suck. The reason I wrote Grantlee was so that other people would be able to do the theming instead of me. If you want to get some cool themes into KMail 4.4 (or 4.5- not certain yet) and KJots you can start sending mockups now 🙂