Archive for the ‘Uncategorized’ Category

Grantlee version 5.3 now available

September 10, 2022

I previously announced the end of new Qt5-based Grantlee releases. The Grantlee template system is to find new life as part of KDE Frameworks 6 in the form of KTextTemplate. The Grantlee textdocument library will probably become part of another KDE library with similar scope.

Meanwhile, some changes have accumulated since the last Grantlee release, so I’ve made a new release to make them available to users. Many of the changes are small, but with a few new features which were cherry-picked from the Cutelee repo.

The other significant change is that Grantlee 5.3.0 can be built with Qt 6. This is not a change of plan regarding migration to KDE Frameworks, but is intended to assist with porting existing code to Qt 6.

Speaking of new releases, we welcomed our baby into the world almost a year ago. Years ago it was a common refrain within the KDE community to new parents to remind them, tongue in cheek, to never shake the baby. I was amused to find that the advise given as a printed book to all new Irish parents reminds the same :).

CMake Daemon for user tools

January 24, 2016

I’ve been working for quite some time on a daemon mode for CMake in order to make it easier to build advanced tooling for CMake. I made a video about this today:

The general idea is that CMake is started as a long-running process, and can then be communicated with via a JSON protocol.

So, for example, a client sends a request like

{
  "type": "code_completion_at",
  "line": 50,
  "path": "/home/stephen/dev/src/cmake-browser/CMakeLists.txt",
  "column": 7
}

and the daemon responds with

{  
   "completion":{  
      "commands":[  
         "target_compile_definitions",
         "target_compile_features",
         "target_compile_options",
         "target_include_directories",
         "target_link_libraries",
         "target_sources"
      ],
      "matcher":"target_"
   }
}
Many more features are implemented such as semantic annotation, variable introspection, contextual help etc, all without the client having to implement it themselves.
Aside from the daemon, I implemented a Qt client making use of all of the features, and a Kate plugin to use the debugging features in that editor. This is the subject of my talk at FOSDEM, which I previewed in Berlin last week.
Come to my talk there to learn more!

Grantlee 5.0.0 (codename Umstraßen) now available

September 24, 2014

The Grantlee community is pleased to announce the release of Grantlee version 5.0 (Mirror). Grantlee contains an implementation of the Django template system in Qt.

I invented the word ‘umstraßen’ about 5 years ago while walking to Mauerpark with a friend. We needed to cross the road, so I said ‘wollen wir umstraßen?’, because, well ‘umsteigen’ can be a word. Of course it means ‘die Straßenseite wechseln’ in common German, but one word is better than three, right? This one is generally popular with German native speakers, so let’s see if we can get it into the Duden :).

This is a source and binary compatibility break since the 0.x.y series of Grantlee releases. The major version number has been bumped to 5 in order to match the Qt major version requirement, and to reflect the maturity of the Grantlee libraries. The compatibility breaks are all minor, with the biggest impact being in the buildsystem, which now follows patterns of modern cmake.

The biggest change to the C++ code was removal of a lot of code which became obsolete in Qt 5 because of the QSequentialIterable as part of the type-erased iteration features.

Grantlee 5.0.0 release candidate available for testing

September 22, 2014

I have tagged, pushed and tarball‘d a release candidate for Grantlee 5.0.0, based on Qt 5. Please go ahead and test it, and port your software to it, because some things have changed.

Also, following from the release last week, I’ve made a new 0.5.1 release with some build fixes for examples with Qt 5 and a change to how plugins are handled. Grantlee will no longer attempt to unload plugins at any point.

Grantlee 0.5.0 (codename Auf 2 Hochzeiten tanzen) now available

September 19, 2014

The Grantlee community is pleased to announce the release of Grantlee version 0.5 (Mirror). Source and binary compatibility are maintained as with all previous releases. Grantlee is an implementation of the Django template system in Qt.

This release builds with both Qt 5 and Qt 4. The Qt 5 build is only for transitional purposes so that a downstream can get their own code built and working with Qt 5 without being first blocked by Grantlee backward incompatible changes. The Qt 5 based version of Grantlee 0.5.0 should not be relied upon as a stable interface. It is only there to assist porting. There won’t be any more Qt 4 based releases, except to fix build issues if needed.

The next release of Grantlee will happen next week and will be exclusively Qt 5 based. It will have a small number of backward incompatible changes, such as adding missing const and dropping some deprecated stuff.

The minimum CMake required has also been increased to version 2.8.11. This release contains most of the API for usage requirements and so allows cleaning up a lot of older CMake code.

Also in this release is a small number of new bug fixes and memory leak plugs etc.

I Did it My Way

September 10, 2014

I flew to Bilbao on the 11th of August and took a train to Burgos. From there I walked 500km over 21 days to arrive in Santiago. Although technically a Catholic pilgrimage, I approached it secularly as a walking holiday, a different environment while in-between jobs and some time for myself. Everyone walks their own Camino.

First Insight: There is suffering

On arrival in Burgos, I needed to first get a ‘Credential’, which allows sleeping in the albergues for Peregrinos along the route, and records qualification to recieve a Compostela at the end of the journey. A credential may be obtained from the municipal albergue in Burgos so that was my first stop. It was almost 8pm when I arrived and the Hospitalero had been telling people since 4pm that the albergue was full. However, he kept a spare bed for such late arrivals and luckily he gave it to me. It made for a good start.

Many sunflowers on the Meseta

Many sunflowers on the Meseta

Stated in a deliberately impersonal way, the nature of suffering on the Camino was immediately apparent when I arrived in Burgos. For most people (with sufficient time), The Camino starts in St. Jean Pied de Port in France from where pilgrims walk over the Pyrenees, and through Pamplona before arriving in Burgos two weeks later. Colloqially, this first of three stages of the walk is known as the ‘Camino of Suffering’, where the pilgrim starts the challenge and experience. I spent the first evening with a few people who had developed the blisters and the tendonitis, people who were bidding farewell to the journey already (some people do it over multiple years), and people who were bidding farewell to companions.

In the morning I got up at 6am and started walking in the dark out of Burgos, following the centuries-old route along the second stage – the ‘Camino of Death’, so called because of the barren, flat landscape surroundings until reaching Astorga.

Initially I walked with one Italian guy and we soon caught up with two more Italians. Most of the time walking the Camino is spent walking alone though, so that little group quickly dissolved as people broke away or fell behind (ahem!). A social shock I experienced is that when someone in a group stops or slows (even one more-familiar than a few hours/days), the others simply continue on – they’re sure to be re-united at a break-point or end point later along the way. It’s something to get used to.

My shadow's the only one that walks beside me...

My shadow’s the only one that walks beside me… I have tonnes of photos like this. Depending on whether I like you, I might sit you down to show you all of them!

Second Insight: Suffering should be understood

One of the things I learned on the Camino is that ‘normal’ is partly an environmental concept. It became ‘normal’ for me to get up at 6am, walk 20-30km per day, eat with some strangers and some familiar faces and go to sleep at 10pm. It did take about a week for that to become ‘normal’ (and even enjoyable!), but it is certainly not similar to my Berlin experience.

Dawn

Dawn

I had blisters since the first day of walking, but by the time I reached Bercianos I was no longer able to stand, let alone walk. I had a day off followed by a 7km walk to the next town where I happened to meet a foot doctor from Berlin who gave me all the help I needed, including her sandals. My footwear was the problem causing my blisters (I was walking in running shoes), so I bought myself a good, expensive pair of sandals when I got to Leon, gave the good doctor hers back and had no new problems caused by footwear for the next two weeks. What are the odds of a foot doctor having the same size feet as me?

Chica the dog

Chica the dog

Most of the people I encountered on the Camino were Italians, Spanish a close second, and plenty of Germans. I fell into a good rhythm with a group of Germans for the second two weeks which was nice. We spoke German as our common language.

camino-5

Third Insight: Suffering (of my Camino) has been understood

Astorga is a beautiful town and it marks the transition of the peregrino from the ‘Camino of Death’ into the ‘Camino of Life’. The route through Galicia is much more steep than the previous stages and full of the sights, sounds and smells of dairy farms. Most of the milk produced in Spain is produced here. The sunflowers filling the landscape and the trail are long since gone.

Although the blisters did not return, the steep climbs and descents brought with them some tendonitis for the final two days of walking. Not much to complain about, timing wise.

It’s sad that the experience of walking the Camino can’t be captured by any camera or prose, but must be experienced to be understood. That’s just part of the nature of suffering :).

camino-6

Like many, I continued on to Finisterre for a few days of sleeping on the beach and unlike most I spent the weekend after in Barcelona, for a very different experience of parks and beaches.

Grantlee v0.2 (codename Doppelt-gemoppelt) now available

November 3, 2011

The Grantlee community is pleased to announce the release of Grantlee version 0.2 (Mirror).

This release is source and binary compatible with the 0.1 series. Initially I thought I’d be able to make the 0.2 release in August with few changes other than the version number bump. Linux 3.0 was just out at the time and I thought it would be fashionable, but I kept finding new things to get done. Besides I have trouble counting past nine (zero-indexed) without taking my socks off.

For the benefit of the uninitiated, Grantlee is a set of Qt based libraries including an advanced string template system in the style of the Django template system.

  {# This is a simple template #}
  {% for item in list %}
    {% ifequal item.quantity 0 %}
      We're out of {{ item.name }}!
    {% endifequal %}
  {% endfor %}

String template systems are very common in the world of web frameworks. In such web frameworks it is common common requirement to decouple data retrieved from a database and processed in some way from the presentation of that data. Typically it is presented in the form of html output, so most string template systems are optimised for html output including special handling of html entities and protection from cross site scripting exploits.

In the context of Qt, we already have several ways to separate data from its presentation in user interfaces. Apart from the more low level model view framework we have the more recent addition of QML for flexible QMetaObject and property based relations.

Grantlee is also in an abstract sense a QMetaObject based language, though it can be extended beyond the qmetaobject apis too. It is not suitable for user interfaces comparable to the way QML is though. Grantlee is more suited to creation of static output suitable for theming, and reports iterating over already processed data, which is recreated on demand and non autonomously, unlike QML. This of course is suitable for markup based user interface with stateless data connections. I wonder if there is a usecase for creation of locally processed static data based on varying inputs which is automatically regenerated when those inputs change which is not already satisfied by QML. Something to investigate in the future perhaps.

At about the same time though i discovered testcocoon and have been working with testcocoon developer Sébastien Fricker on coverage instrumentation for Grantlee. The motivation is more generally about learning how to use testcocoon with Qt and CMake projects. This is in anticipation of making it possible to gather coverage data for KDE productions in the future. The effort has led to a good deal of work by Sébastien on both testcocoon itself and on Grantlee to ensure useful coverage data collection. Some of the changes required relate to specifics of loading plugins, some crash fixes, and some issues associated with buildsystem integration and peculiarities of unit testing private api.

The result is some clear indications of what parts of Grantlee are adequately unit tested, and which are not. There are some false positives still produced by testcocoon such as in unit tests but I’m confident that will continue to improve. The good news is that most hard parts of Grantlee are unit tested. Ideally there should be no such thing as an edge case when considering what should be unit tested though. Increasing the coverage stats of Grantlee will be a focus for the next while. The coverage report generated by gcov and lcov is also available for this release.

Coverage result for part of Grantlee

Another set of changes since the 0.1.9 release is concerned with the buildsystem. I’ve learned a lot about CMake since starting with Grantlee; and I have been able to apply that knowledge to increase the 0-knowledge portability of Grantlee. 0-knowledge portability refer to the portability of Grantlee to platforms I have no access to. Features are enabled as much as possible by testing their availability rather than testing for platforms known to support certain features. For example some of the Grantlee unit tests require some third-party containers for testing. I’m able to test for both tr1 and boost to use as third parties when running those tests.

Enjoy!

Grantlee v0.1.9 (Codename affengeil) now available

June 24, 2011

The Grantlee community is pleased to announce the release of Grantlee version 0.1.9 (Mirror).

This release contains the usual round of bugfixes, features, documentation updates and a new example application. The use of funny german vernacular for codenames continues.

Notably absent from this release is the work on Grantlee::Tubes. The QIODevice API needs a bit more research, so those classes will appear in a future release instead while I figure out what parts of the QIODevice API can be relied upon.

Much of the directed effort in this release has been on the build system. With the help of Alex Neundorf, the entire CMake system was updated to make Grantlee easier for consumers to consume. This includes installing EXPORTS from Grantlee which can be imported with find_package, proper RPATH handling, and reducing hardcoded paths in convenience properties.

A follow-on from the CMake work was to add some CPack configuration. Now it is possible to create binary tarballs for Grantlee as well as a Windows install wizard. CPack makes the process very easy once everything is set up.

The Grantlee component installer for windows.

In an effort to make generation of content where whitespace is not insignificant, I’ve added a smartTrim feature to Grantlee (based on an idea in the Django bug tracker which I’ve been pushing forward simultaneously). The idea is to discard insignificant whitespace while processing the input content. Whitespace can be considered insignificant if there is only one piece of template syntax on a line.

This feature should make it easier to write templates to generate C++ code for example. For html output, whitespace is rarely significant.

The efforts around introspection of QObject derived types in QVariants did not lead to improvements in Qt4. It is possible that improvements will be made to the introspection system in Qt5, but the features could not be added in a binary compatible way in Qt4. The Qt4 solution is more documentation.

I also added some more documentation on the examples shipped with Grantlee along with links to the brilliant prose in these blog entries.

GCov based coverage data is now also going to be produced for each release. The existing coverage data shows that Grantlee has excellent test coverage in most parts – 90% of functions are covered and 85% of lines in the libraries and plugins. Most of the non-covered code indicates missing tests for failure cases, but the ‘happy path’ is generally well covered. There might be some advantage to using CTest and CDash for coverage information. Something Grantlee coverage scripts can do that I don’t think CTest can though is to break down the coverage data by the unit test that contributed to it. This might help identify where a completely new test is needed to purposefully cover a class.

Coverage of Grantlee::Context. Break down by unit test

Because much of the effort in this release went into improving the buildsystem, it should be much easier now to add new libraries to Grantlee, so I expect that to happen in the next release with Grantlee::Tubes.

Grantlee::Thermodynamics and Refrigeration

April 1, 2011

With some new classes in Grantlee::Tubes starting to take shape, I started making use of them in the new Grantlee::Thermodynamics library with the Grantlee::Refridgeration system.

Grantlee::Refrigeration makes use of components from Grantlee::Tubes, like Grantlee::Pump, QtIOCompressor and Grantlee::Cat to create an ideal heat transfer system. The intention is to create a stream of “cold bytes” taking heat out of hot paths in your code, and disposing of the heat through your cooling fan.

Coolness. You can't get enough

Thermodynamics teaches us that while the quantity of energy in a closed system is constant, the quality of the energy (it’s entropy) is not. Entropy in a closed system is always increasing (the quality or useful energy in the universe is always going down), but we can locally decrease the entropy in a body of mass if we transfer it to another body of mass. This is what refrigeration is about. The decrease in entropy is made visible in the cold cavity of a fridge by the state change that water undergoes from fluid (higher entropy) to solid (lower entropy).

We can take heat (enthalpy and entropy) away from somewhere, but we have to dump that heat somewhere else. It takes work and a refrigerant to transfer heat between thermodynamic bodies. Heat won’t move spontaneously by itself (beyond the limits of equilibrium) so typically the work of heat transfer is done by a pump. Heat transfer in a refrigerator works in a cycle of four stages.

Grantlee already provides a Pump which we can use in our thermodynamic system, and we can use any refrigerant which is plentiful and which has a high capacity for entropy and a lot of hot air, such as a twitter feed.

We start by seeding the system with our refrigerant.

  QNetworkReply *reply = QNetworkAccessManager::get("http://www.twitter.com/feed");
  Grantlee::Cat *cat = new Grantlee::Cat;
  cat->appendSource(reply);

Cat will read the data from the reply object until it closes, at which point it will start to read from another device to close the cycle (shown later). At this point the data is already saturated; It can’t contain any more entropy at this temperature and pressure.

1 – 2: Reversible Adiabatic (Isentropic) Compression

Typically the first step described in a refrigeration cycle is Isentropic compression – that is, compressing the refrigerant without changing its entropy. The compression causes the data to become super-saturated. We compress the data by tubing the refrigerant through a QtIOCompressor.

  // (condenser shown later)
  Condenser *condenser = new Condenser;
  QtIOCompressor *compressor = new QtIOCompressor(condenser);
  cat->setTarget(compressor);

2 – 3: Constant Pressure Heat Rejection

After compression comes constant pressure heat rejection. As all developers know, constraints of constness can be expressed in C++ with the const keyword. So we require a class for rejecting the heat which will enforce that constraint:

  class Condenser : public QIODevice
  {
    ...  
  protected:
    void writeData( const char* data, qint64 len );
  };

Fortunately, the virtual writeData method of QIODevice already takes the data (our refrigerant) as const, so that constraint is already enforced for us. The condenser causes a change of state of the data, thereby decreasing its entropy. The data is once again saturated, but now in its low entropy state and at a lower temperature.

3 – 4: Adiabatic Throttling

We now have to connect up a throttle to perform isentropic expansion, so the entropy and the temperature is maintained, but the refrigerant changes state and becomes unsaturated.

A throttle is trivially implemented by using a QtIOCompressor in reverse, so we omit that for brevity.

At this point, we have our stream of cold bytes at a low temperature and unsaturated, so with a capacity to absorb some heat, so let’s do that with an evaporator.

4 – 1: Constant Pressure Heat Addition

We require that heat absorption occurs at constant pressure, and once again the const keyword ensures that.

  class Evaporator : public QIODevice
  {
    ...  
  protected:
    void writeData( const char* data, qint64 len );
  };

(The implementation of the Evaporator is left as an exercise for the reader)

We can then use the cold bytes in the method that defines our hot code path where we use an evaporator to facilitate the heat transfer to the refrigerant:

  void MyClass::hotPath(..., QIODevice *source, QIODevice *target)
  {

   // ...

    Evaporator *evaporator = new Evaporator;
    evaporator->setTarget(target);
    evaporator->setSource(source);
  }

The very presence of the evaporator in the hot path of our code is enough to cause heat transfer to the cold bytes, increasing their entropy by causing them to change state.

Of course this means that we need to call our hot path with the refrigerant tubing included:

  Grantlee::Channel *channel1 = new Grantlee::Channel;

  myInstance->hotPath(..., throttle, channel1);

  Grantlee::Pump *pump = new Grantlee::Pump;
  pump->setSource(channel1);

  Grantlee::Channel *channel2 = new Grantlee::Channel;
  pump->setTarget(channel2);

  cat->setSource(channel2);

As a result of the state change in the evaporator the data also becomes saturated at high entropy. Recall that this is the same state the refrigerant we originally introduced from twitter was in.

We route the refrigerant from the hot path and into a Grantlee::Pump, which provides the work required to satisfy the Second Law, and then forwarding the result on to cat, thereby closing the cycle.

Results

I ran some tests using the Grantlee Thermodynamics Toolkit on various algorithms with various parameters of compression and throughput, with results indicating a universal increase in performance when refrigeration was used.

Templating HTML Applications with Grantlee

February 6, 2011

A lot of people are talking about using web technologies with Qt again.

Very well, I’ll throw my oar in too :).

The video shows a new example application I just added to Grantlee, which uses html/css/javascript/jQuery for its UI. The main window is simply a webkit widget, and the rest of the visuals are HTML. The video capture is not the best, but jQuery makes the slides in the flickr page very smooth.

The HTML is generated of course with Grantlee Templates, so the base template sets out a skeleton html page, which is inherited by the application html pages. The application html pages are also very simple. They just fill in some blanks in the base template in the same way that concrete classes in a c++ template pattern might implement virtual methods. They only need to specify what differentiates them from the base template, so there is almost no HTML repetition.

The applications just consume rss feeds so I wrote a simple Grantlee tag to process a generic rss feed, and it is used many times in the application. The Grantlee rss feed tag is actually modeled on the QML XmlListModel element in terms of API and flexibility, in that a delegate is implemented to lay out the data, and the data is made available through XQuery roles.

The actual rendering of the Grantlee templates and serving the result is done by a custom QNetworkAccessManager and QNetworkReply. Working in tandem, they make it very easy to serve content directly into a webkit powered QWebView. New applications are added by adding a new HTML page, and style and structure of the main application can be customized by using alternative HTML files.

I wrote this application to show how a template system like Grantlee could fit into a larger runtime for creating HTML based applications. The example of course doesn’t provide everything such a runtime would require such as an application authentication and distribution mechanism etc. Those things are provided by other html application frameworks already. I’m just showing how to achieve a consistent styling and structure without needing to repeat fragments of HTML markup.