Flight of the Pechvogel

January 29, 2010 by steveire

On Wednesday afternoon I boarded a Greyhound bus in Las Vegas, and by Thursday morning I was at the airport in San Diego to fly home to Berlin by Friday evening. It’s 8 am by my frame of reference, and I didn’t sleep on the plane because of the old guy beside me who looked so uncomfortable in his chair I thought he might collapse.

Of course, just like last time, my luggage didn’t get as far as I did. There must be some way to measure the hysteresis of my luggage–self movements. Or something, I’m very tired and I have to stay up until nighttime. But at least I learned a new German word.

There was a lot of interesting content at CampKDE which will be shared with the world in due course. Aside from the talks, it was very cool to meet KDE people I’d never met before and some that I’ve met a few times now. This side of the pond had a good turnout, but it was mostly North Americans attending. The trainings in CMake, Qt and QtEmbedded were very useful to attend too.

I should be back to the usual interesting blog posts about kde pim, more ModelView stuff and Grantlee in the next week now that I’m out of the land of distraction.

Getting (mostly) to CampKDE

January 15, 2010 by steveire

A guy walks to the checkin desk of his airline with a lot of luggage:

Flyer: Hi there. I’d like you to send my suitcase to Beijing, my rucksack to Washington, my surfboard to Berlin, and I’m flying to San Diego.
Airline: Uh, I’m sorry sir I don’t think we can do that.
Flyer: Well you did it before without me even asking you to…

In fairness I had only one checked in bag on my flights, but it got lost somewhere through my cancelled and delayed connections. Hopefully it will turn up in the next day or two, or it could shaft my plans for after the event.

Right now though, they’re playing Moondance by Van Morrison in the lobby. The day is looking up. :) Time to debug my presentation methinks.

Testing proxy models gets easier. Again.

January 6, 2010 by steveire

I just pushed a few commits to the tests section of kdelibs which make it easier to test proxy models.

I have written before about testing of proxy models, but things have progressed greatly since then. Aside from creating a unit test suite, I also created a small application for testing proxy models, imaginatively called proxymodeltestapp.

The idea was to have a visual indication of whether the proxy models used in Akonadi were working properly. Those proxies are all implemented generically so that they can be used with any model, in this case DynamicTreeModel which I wrote about before. Having full control over the contents of the DynamicTreeModel by using various commands meant that I could tackle various bugs in the proxies as they came up by recreating models and scenarios in the proxymodeltestapp, such as “starting with a complex tree, we move these items from here to there. Then the application crashes.”. Having a way to reproduce those scenarios in a Short, Self Contained, Correct, Example, away from the additional Akonadi and application code has proved very useful. There is even a widget to drive the DynamicTreeModel in the same way as it is driven in unit tests. Again, having a visual representation of what is going on and how the proxy should react takes some of the pain out of writing and fixing unit tests for the proxies.

It’s all in the interpretation

The first of todays interesting commits adds an interpreter to DynamicTreeModel. The two most annoying parts of writing tests for proxy models are setting up an initial condition, and inserting some new rows, some of which have child items themselves. Both issues have the same root cause: The DynamicTreeModel did not have any easy to use command to insert a tree or sub-tree into the model. The old solution was to define some fragments in a ModelInsertWithDescendantsCommand (catchy name as always), and when the command was executed, the fragments would be inserted at once, creating a new subtree. This kind of worked, but the code was difficult to read and implement, both counter-productive to unit testing.

The new interpretation method looks something like this:

insertCommand->interpret(
  "- 1"
  "- 2"
  "- - 3"
  "- 4"
  "- 5"
  "- - 6"
  "- - - 7"
  "- 8"
);
insertCommand->doCommand();

which will of insert 5 new rows in the model, some of which have children. Much better for writing unit tests. The parsing logic is very simple, and each of the items in the tree does not need to be a sequential number. They can all be the same number, letter or series of gibberish if it makes it easier. There’s even a #define to dump a nice numerical tree to stdout if you want to create a readable version from a gibberish version.

Simultaneous interpretation

Of course the interpret feature is not limited to writing unit tests. Through the use of the new DynamicTreeWidget, proxymodeltestapp can be used to quickly create a model for testing purposes without recompliling or even restarting the application.

DynamicTreeWidget View and Edit tabs

On the left is the “View” tab of the DynamicTreeWidget, which contains the DynamicTreeModel in a QTreeView. By clicking on the “Edit” tab, we get an editable textual representation of the tree. We can move, add and remove items in the text view, and when we switch back, the “View” tab is already updated with the new edited version. We can also use separate controls in the DynamicTreeWidget to insert or remove items in the model, or move them with drag and drop, and the text version is instantly updated. This is a very simple way to create and manipulate the model for proxy (or view) testing purposes and allowed the removal of a lot of code in the test app.

Scriptable descendants

The last interesting commit for today (there will be plenty more days) combines a new proxy model with QtScript. I haven’t written about the KReparentingProxyModel before because it will hopefully be part of kdelibs 4.5, but not 4.4. The idea is to define in a proxied model a completely different structure than is present in the source model. It could be used for example to turn a list of emails into a threaded tree of emails in KMail, or a list of todo tasks into a tree of tasks and subtasks in KTimeTracker. It could be used to flatten a tree into a list for use in a combobox or similar widget, or to turn a particular tree into another tree of a different structure for a reason I can’t imagine. On the whole, it should be useful for Akonadi when fully working.

There is only one method in the API:

virtual bool isDescendantOf(const QModelIndex &ancestor, const QModelIndex &descendant);

Consumers of the class reimplement that method to return whether descendant should be a descendant of ascendant in the proxy model in much the same way that you might implement QSortFilterProxyModel::lessThan. By using a little bit of QtScript magic, I can define the structure in a proxy at runtime in the proxymodeltestapp.

Scripted reparenting proxy model 1

Scripted reparenting proxy model 2

Scripted reparenting proxy model 3

Nice, no?

Award for the most gratuitous use of the word “QAbstractItemModel” in a serious presentation

December 23, 2009 by steveire

It seems I’m one of the recipients of the first annual NQDF award for outstanding achievement in the field of excellence. It seems the phone will be delivered in a few weeks, but they haven’t yet revealed when the big award ceremony is, or when I will get money, power, fame and influence as a result. I’ll be patient, but until then, thanks for the phone!

The schedule for CampKDE was published last week. I’ll be giving a talk on the challenges the Akonadi team faced in using the Qt Model View framework to implement the posted KDE PIM applications and how it results in reusable components and allows a lot of scope for innovation. I have been blogging on the subject for some time now, but there will be a lot of new content in the presentation that I haven’t covered here yet which is relevant to all users of Qt Model/View – not just Akonadi and not just KDE. Registration is still open if you want to come to attend.

By the way, I didn’t know this until just now. Very funny.

Community, Identity, Stability

December 13, 2009 by steveire

The second beta release of KDE Software Compilation 4.4 is coming up in about a week and a half. It’s a long name for a release, but it carries with it additional meanings and allows emphasis in important areas which were not as strongly emphasized before.

KDE is a community, not a software release. Not a desktop environment.

KDE is a brand, part of an Identity. Something to be proud of.

This communications shift also marks a shift in the maturity and stability of the 4.x series of the KDE Software Compilation.

The thing is, ending is better than mending sometimes. People holding put for a return to the days of KDE 3.5 can take note that the time is once again coming to try out the 4 series. History is bunk and, well, progress is lovely isn’t it? Everyone’s happier nowadays. We haven’t any use for old things here. Particularly if they’re beautiful. Beauty is attractive and we don’t want people to be attracted by old things. We want them to like the new ones.

The upcoming 4.4 features subtle animations and visual feedback which is wonderfully pneumatic, a host of new features (I do love new features), elaborate games and more to help increase consumption through your computer.

The wheels of KDE must turn steadily, but cannot turn untended. There must be people to tend them, people as steady as the wheels themselves to help test new releases, and ensure that all is going well. The whole 900 meters (or more?) or more of conditioning the new release. It’s great being a Beta tester. You get to wear blue t-shirts as a real contributor, a real part of KDE. Happy and good.

You get to try out all the latest features and toys in the upcoming release. Never put off til tomorrow the fun you can have today. Nothing can be achieved without perseverance, and help is always welcome to achieve the best and a Fordly level of quality with the new release.

(As some of you can no doubt guess, I finished my book today, so I thought I’d action Stuarts suggestion. This has nothing to do with official KDE communications :) )

Shall we play a game?

December 7, 2009 by steveire

My solution to yesterdays tic tac toe problem is here.

The trick is explained in the comment at the top, but essentially I am storing the positions of markers for ‘X’ and ‘O’ only from the first 8 positions in the grid so that it takes up only 8 bits for each player. The state of the 9th box in the game is discovered by using a population count on both players game states and determining which if any is missing a mark. This requires the extra information of which player was last to make a move. You could call that cheating, but it does work, and the simulation is there to prove it.

The simulation runs 1 million games of tic tac toe in 14.905 seconds on my laptop. Not bad really. It’s good to get into the spirit of optimization now that we’re in bug fixing mode for 4.4. There’s lots of good tips in Jon Bentley’s Programming Pearls. I highly recommend it to anyone interested in this stuff.

As an exercise for the reader, modify the example to pick the best available position each time instead of a random box to place a mark.

Anyway, onto my next project, Global Thermonuclear War…

Tip top tic tac toe task

December 6, 2009 by steveire

Yesterday I got linked to an article with interview questions for engineers and problem solvers at google including this one about tic-tac-toe.

The problem is to write a function which takes as arguments a Game of tic-tac-toe and a player, and returns whether the player has won the game. As an extension I wanted to do it in the most efficient way I could.

My first thought on reading it was that an sequence of bits could be used to store the state of the game for each player. Then the is_winner(Game, player) function would only have to test the bits for that player to see if the Xs or Os are in the right place to declare a win. As that would mean nine bits for each player, you would either need two 8 bit words each, or one each with a third shared between them to represent the extra bit.

Eventually I managed to squeeze the game into two 8-bit words which contain the state of play, but before I post and describe mine, I’ll give others the opportunity to do the same. To prevent cheating and so you know I didn’t steal your idea, make sure to save this somewhere I can’t tamper with it:


$ sha1sum tic_tac_toe.cpp
70ef68844d3173dd4363ff544e24aa57c28a7b52 tic_tac_toe.cpp

Akonadi porting explained

December 3, 2009 by steveire

KDEPIM oldspeak doubleplusungood. Unbellyfeel Ingo.
Akonadi correctful newspeak.
Crimespeak is slavery.
MINIPIM make rectify fullwise.
Duckspeak Till.
Party members make KDEPIM unwith oldspeak.
Remake fullwise with newspeak.
Party members fulluse Akonadi. Newmake plenty KDEPIM.
Unapplication knotes.
Goodthinkers speakwrite kde prolefeed.
Goodmake prolethinkers newspeak pluswise.
Good prolethinker comrades make inner party members.
Newspeak centerward make easy newapplications.
Telescreen #akonadi wisewatch. Speakwrite helpful party members.

Akonadi applications sharing notes

November 20, 2009 by steveire

There has been movement lately on getting notes into and out of Akonadi and sharing them between several applications. Currently in progress is porting of KJots to Akonadi, and creating a desktop notes solution based on Akonadi.

Sounds like an excuse for a screencast.

Ogg Link

Here again I can demonstrate that data is shared between the desktop notes and KJots, as both are using Akonadi. When I update a note in either KJots or on the desktop, the other views on the note is updated automatically and instantly.

The data is shared, not copied.

On display is actually two different Plasma widgets. One of the widgets shows a single note at a time. There are several such notes in the screencast. The other widget is the notes listing widget. This widget shows all of the Notes in a particular Collection, or seen another way, pages in a KJots Book.

The drag and drop of pages or notes onto the desktop works because of the recent work by sebas in that area. Individual plasmoids can advertise that they can handle particular types of URLs, and when a drop occurs, an instance of that plasmoid can be created. All Items in Akonadi have a unique URL for these kinds of use cases.

Also on show is a KJots plasmoid, which is a simple mini KJots stuffed into a plasmoid. It too can be configured to show the books and pages below a chosen book. It would be possible to have a different KJots book on each of your activities with different scraps of information in each organized however you like it.

This is part of a wider effort to improve notes in the KDE ecosystem and make use of some of our pillars, Akonadi, Plasma and Nepomuk. It’s all at prototyping and concept stage at the moment, so if you have any ideas about how you should be able to access your notes, now is the time to speak up.

There is also some discussion started to attempt to unify the the storage format we use for notes, and hopefully make it easier to share notes between more different applications and devices. Possibly even Tomboy notes could be shared if they are stored in a shared location and someone steps up to write an Akonadi resource for it. :)

Akonadi goes Web2.0

October 24, 2009 by steveire

Lots of people are posting about implementing functionality in web browsers to do what we do today in rich clients.

I don’t know what’s going on, but I didn’t want Akonadi to miss the party. At the recent Akonadi sprint, I decided to spend some time putting together a proof of concept for a web client for Akonadi. Here’s a screencast of the result:

Ogg link.

The video shows web pages in multiple web browsers showing the same data as KMail, KAddressBook and the proof of concept gtk applications I wrote about before, with everything kept in sync. They share the same data. I could edit the same data in the web pages too if I had taken the time to write the javascript code to handle it.

One of the advantages Akonadi can bring is to free us from the web browser and put our data and user experience into rich clients. Why the perversion of putting it all back in to a web browser? Initially, I started writing it just to try out the rich web client technology webby people talk about, and to see if Akonadi has what it takes to go Web2.0. First requirement is a catchy name. Check.

The zeroth requirement for any application is use cases. Where could an Akonadi web application be used? One idea would be to put a finished version of this technology onto a public web server and allow the public to create accounts and connect to multiple different data resources like email providers and unify them all in a single interface like many people do with KMail and other rich desktop client apps. However, that would not be practical as a service to provide to the public. Akonadi is designed to handle lots of resources for one single user. Using it to handle lots of resources for lots of different users probably wouldn’t scale in its current form.

I can think of two valid use cases though. Consider for example gOS, or Google Chrome OS, which encourage the user to access everything through a web browser and seem to be targetted at particular segments of netbook users. To access different email services on a gOS system, users would have to access different web applications each with different interfaces, and different usability designers deciding how to interact with the system. Using Akonadi in a web client, vendors like gOS and others could design one consistent interface which allows users to access all of their data of any type. Because the server and the client are on the same machine, the vendors would have complete control over the appearance and interactions with the system. Users who want to try out a rich client application can just install KMail and pick up where they left off. No need to import the data from elsewhere (and creating diverging copies). The data is already shared.

Another use case is for developers who find it easier to develop application UIs in html and javascript instead of C++ or want to incorporate web slices. They could simply put their html into a QWebFrame, and they’re done. A new application without a recompile. Wacky ideas and platforms to host them. Check.

I knew that a web page showing Akonadi data would need to change asynchronously, and not cause a page to reload when selecting a mail folder. jQuery is a javascript library with facilities for exactly that and with plugin support. I used the jstree and dataTable plugins to put the page together. Check out the demos here and here of what else those plugins can do. Asynchronous user interaction. Check.

The web browser needed a way to get data from Akonadi into the browser when requested. That means implementing a server capable of responding to requests from the web browser, and capable of communicating with Akonadi on both parts of its interface, D-Bus and IMAP. Finally, I needed a way to deliver change notifications to the web browser without the browser requesting anything. This is variably called HTTP pull, long polling and Comet. I first saw it described here last month, which covers the concept pretty well. Rather than needing to keep events synchronized though I only needed to keep sending notifications from Akonadi (which may be rare) to the browser.

Rather than learn the details of a Comet implementation though, I wanted a ready made solution to abstract the details away from me. Orbited is a Comet framework written with twisted which can be used with the comet server protocol and js.io, a javascript library simplifying network communication in javascript environments. I already had an implementation of a twisted akonadi client library to get the data to orbited, and Orbited would take care of the rest of the transport. In the browser html, js.io provides a listening socket to recieve all notifications and asynchronous data responses. A big thanks goes to desmaj on #orbited who got me started with a small contained comet server and client. All I had to do was add Akonadi bits to it, and a bit of jQuery glue later the result is in the screencast above. Akonadi in a real-time web application. Asynchronous two-way data transfer. Check.

I noticed yesterday that Mozilla announced Raindrop. They’ve clearly spent more time on it than the two days did. I haven’t looked into their implementation, but it’s interesting to note both of us trying out these concepts at the same time. Multiple different projects with similar but distinct goals. Check. (Akonadi is a cooler name anyway :P )

The implementation of this is messy because my C++/python ideas of how to write code and classes get thrown out the window when using javascript, mainly because the ‘this’ object in javascript doesn’t always resolve to what I expect it to when thinking in C++/python design mode. To get it done I did some ‘code sharing’ with copy and paste. It’s just a proof of concept anyway, so that doesn’t really matter. I’ve put the code in the same git repo with python-twisted-akonadi (Sorry V :) ).