Grantlee pairs up with the “other” template system

For the first time in a while a post about Grantlee which is not a release announcement. 🙂

QObject introspection

Since I started working on Grantlee it has been possible to put QObject derived types into the template system as context, as well as QLists or QHash containers of QObject derived types in a QVariant. Properties on the instances are then available through the Q_PROPERTY system and can be used when rendering a template.

  QVariantList peopleList;
  // Inconvenient!
  peopleList.append(QVariant::fromValue(static_cast<QObject*>(new Person("Alice"))));
  peopleList.append(QVariant::fromValue(static_cast<QObject*>(new Person("Bob"))));

  context.insert("people", peopleList);
  template->render(&context);

  ...

  <ul>
  {% for person in peopleList %}
    <li>{{ person.name }}, {{ person.age }}</li>
  {% endfor %}
  </ul>

The restriction that the type be QObject derived was because the only way to introspect them was through the Qt Property system. Only QVariantList and QVariantHash were supported because that provides sequential and associative containers, and both already have built in support in QVariant.

It is not an ideal situation however, because existing applications already have a data API which might not use QObject derived types, so wrappers would have to be written. That is a common situation in KDEPIM where we have KMime::Message, KABC::Addressee and KCalCore::Incidence and related classes which do not inherit QObject. Additionally, QList is not often the best container to use as a sequential container. Commonly people use QVector or std::vector instead.

Generic introspection

Near the end of July Michael Jansen did some work on generic type support in Grantlee. It took me until the middle of September to finally get around to reviewing the patch, and it showed me something I didn’t know was possible. There are several aspects to the work on the mjansen-experimental branch, but the core concept is being able to use C++ templates (very different to Grantlee templates) and function pointers to extract custom types from QVariants from within the Grantlee processing code. This post is about how it works, not how to use it. For the documentation about how it is used, see here.

Because it uses C++ templates, Grantlee doesn’t need specific code to support the custom types, but can expose a static interface which applications can implement for their own custom types. The real magic happens by using function pointers to create a registry of available types. It works something similar to this (some details and magic omitted for clarity):

  // static interface
  template<typename T>
  struct TypeAccessor
  {
    static QVariant lookUp(const T &object, const QString property);
  };

  // Converts the object parameter from QVariant to T for use with the TypeAccessor
  template<typename T>
  struct LookupTrait
  {
    static QVariant doLookUp( const QVariant &object, const QString &property )
    {
      typedef typename Grantlee::TypeAccessor<T> Accessor;
      return Accessor::lookUp( object.value<T>() ), property );
    }
  };

  struct MetaType
  {
    typedef QVariant (*LookupFunction)(const QVariant &, const QString &);

    QVariant doLookup(const QVariant &object, const QString &property);

    static void registerLookUpOperator( int id, LookupFunction f );
  };

  template<typename T>
  void registerMetaType()
  {
    QVariant ( *lf )(const QVariant&, const QString&) = LookupTrait<T>::doLookUp;

    const int id = qMetaTypeId<T>();

    MetaType::registerLookUpOperator( id, lf );
  }

  QVariant MetaType::doLookup(const QVariant &object, const QString &property)
  {
    LookupFunction func = registry.value(object.userType());

    // Calls LookUpTrait<T>::doLookUp(QVariant, QString);
    return func( object, property);
  }

There are some things to note there:

• The signature for looking up a property on an object is available as a typedef on the MetaType class.
• The same signature is available as a static method on the TypeAccessor class
• Custom types can be registered using a template function. That retrieves a function pointer to the TypeAccessor template implementation and stores in in the MetaType using the qMetaTypeId of the type as an identifier.
• The MetaType can later use the type in the QVariant to call the correct function to return a property
• The custom type must have already been declared as a metatype with Q_DECLARE_METATYPE, but that’s necessary anyway to put it in a QVariant.
• In practice the details of the TypeAccessor are hidden behind the GRANTLEE_BEGIN_LOOKUP macro.

With the above infrastructure, the rest is just the bread and butter of full and partial template specialization in applications to supply the correct hooks to Grantlee.

  // Application code:

  class Person // Not a QObject
  {
    QString name() const;
    int age() const;
  };

  Q_DECLARE_METATYPE(Person)

  template<>
  struct TypeAccessor<Person>
  {
    static QVariant lookUp(const Person &object, const QString &property)
    {
      // can only call const methods here
      if (property == "name")
        return object.name(); // Returns a QString
      if (property == "int")
        return object.age(); // Returns an int
      else
        return QVariant();
    }
  };

  void init() {
    Grantlee::registerMetaType();
  }

Any type that can be put in a QVariant can be returned from such a function, including other custom types, and including containers. All of this is not necessary for QObject derived types of course, as it is provided automatically by moc. This method is safer however as constness provides compile-time safety which is not provided by Q_PROPERTY. You can not define a clear() method on Person and then call it in the TypeAccessor specialization, whereas with a Q_PROPERTY, the READ method is not enforced to be const.

Template containment

Aside from support for generic types in Grantlee, I wanted generic container support too. Grantlee provides tags to perform {% for %} loops as above, as well as index based lookup for sequential containers and key lookup in associative containers. Rather than just allowing that with QVariantList and QVariantHash, I wanted to be able to loop over or access any Qt container, or any other user specified container. That means that if you already have an API which needs to stay as it is (for binary compatiblity reasons for example), you can use it even without changing the types of any containers in the API to QVariantList or providing conversion wrappers.

The following containers have automatic, built in support in Granltee:

• QVector<T>, QList<T>, QStack<T>, QQueue<T>, QLinkedList<T>, QSet<T>
• std::vector<T>, std::deque<T>, std::list<T>
• QHash<Key, T>, QMap<Key, T>, std::map<Key, T> where Key is a QString or integral

and where T is one of:

• bool
• a number
• QString
• QVariant
• QDateTime
• QObject*
• Any custom type registered with Grantlee
• A container of any type on this list.

The container must also be registered as a QMetaType with Q_DECLARE_METATYPE.

Did you see the recursion by the way? It is also possible to use nested containers like QList<std::vector<QSet<Person> > > or StackMapListVectorInt, though possible doesn’t mean should.

Additional containers can easily be supported with a macro:

  // Add support to Grantlee for boost::circular_buffer<T>
  GRANTLEE_REGISTER_SEQUENTIAL_CONTAINER(boost::circular_buffer)

  // Make it possible to put a boost::circular_buffer<Person> into a QVariant
  Q_DECLARE_METATYPE(boost::circular_buffer<Person>)

The unit tests use std::tr1::array and std::tr1::unordered_map containers to demonstrate how to support ‘third party’ containers.

Recursive Autoconditional Container registration

The best part about the container support is that it is completely automatically conditional and determined at build time. That is, if you declare QSet<Person>, but not QList<Person> with Q_DECLARE_METATYPE, Grantlee will support only QSet<Person>, saving the build-time (building template code takes a long time) and the start-up time of registering the other container. The unit test which tests all possible builtin containers (non-nested) takes 5 minutes alone to build. The same is true of the numeric and Qt types. Support for them is determined at build time using sufficient magic

  enum
  {
    Magic,
    MoreMagic
  };

  template<typename T, int n>
  struct RegisterTypeContainer
  {
    static void reg()
    {
    }
  };

  namespace Grantlee {
  template<typename T>
  struct RegisterTypeContainer<QList<T>, MoreMagic>
  {
    static int reg()
    {
      // registerSequentialContainer is like registerMetaType
      return registerSequentialContainer<QList<T> >();
    }
  };
  }

  template<typename T>
  void registerMetaType()
  {
    // This registers the container if QList<T> has been declared as a metatype, and does nothing otherwise.
    Grantlee::RegisterTypeContainer<QList<T>, QMetaTypeId2<QList<T>>::Defined>::reg();
  }

Do you see how the magic works?

Smart pointers

There’s a memory leak on this page. In the first example I created Person QObjects on the heap and then never called delete for them. They also don’t have a parent in the example, so they leak. There are several reasons not to give a QObject a parent. Some short-lived objects have an unclear end-point, but shouldn’t be around as long as their parent. In some threading situations there are also reasons not to give a QObject a parent.

In those cases we can use QSharedPointer to manage the memory of the QObject. Grantlee now also has special (automatic condtional) support for QSharedPointers which manage a QObject derived type. QSharedPointer can now be put into a QVariant and into a Grantlee Context just like any other object, and its Q_PROPERTIES are automatically available in the Grantlee::Template.

  Q_DECLARE_METATYPE(QSharedPointer<Person>)
  Q_DECLARE_METATYPE(QVector<QSharedPointer<Person> >)

...

  QVector<QSharedPointer<Person> > peopleList;
  QSharedPointer<Person> alice(new Person("Alice"));
  peopleList.append(alice);
  QSharedPointer<Person> bob(new Person("Bob"));
  peopleList.append(bob);

  context.insert("people", QVariant::fromValue(peopleList));
  template->render(&context);

Of course it is also possible to use other smart pointers, and smart pointers to non QObject derived types. To support an additional smart pointer, a simple macro is used:

  GRANTLEE_SMART_PTR_ACCESSOR(boost::shared_ptr)
  Q_DECLARE_METATYPE(boost::shared_ptr<Person>)

In KMail, we use objects like boost::shared_ptr<KMime::Message>, so these kinds of features may prove quite useful for advanced theming in the message viewer.

TTP without TTP

There were several times while implementing the support for templated types that TTP could have been useful. Apparently support for TTP is not widespread in some compilers yet, so I avoided using that language feature. What it allows is constructs like this:

  template<typename <typename> class Container, typename T>
  struct ContainerCleaner
  {
    QVariant clean(Container<T> &container)
    {
      // Do some cleaning on Container<T> 
    }
  };

That struct would work with QList<T>, QVector<T> etc automatically as long as the symbolic API used is the same. I wanted something like that in unit tests for clean-up code. After using a Container<QObject*> I wanted to call qDeleteAll() on the container. That would be possible with the above template by partially specializing it for T=T*. Doing the same thing without TTP is almost as easy

  template<typename Container, typename T = typename Container::value_type>
  struct ContainerCleaner
  {
    void clean(Container &container)
    {
      // Do nothing in the common case.
    }
  };

  // Partial specialization for Container<QObject*>
  template<typename Container>
  struct ContainerCleaner<Container, QObject*>
  {
    void clean( Container &container)
    {
      qDeleteAll(container);
    }
  };

  template<typename Container>
  void cleanupContainer c)
  {
    CleanupContainer<Container>::clean(c);
  }

  ...
  // After test:
  cleanup(container);

The only disadvantage is that I need two clean up methods, one for sequential containers, and one for associative containers, because they are not statically polymorphic (value_type vs mapped_type). I also need another specialization for std::map types because qDeleteAll does not work for those.

Conclusion

If you have read this far I can only assume a strong interest in template programming. I’d encourage you to read the commits in the branch. They are sequential and easy to read, and if you spot anything that could be done better, do please tell me.