Grantlee pairs up with the “other” template system

For the first time in a while a post about Grantlee which is not a release announcement. :)

QObject introspection

Since I started working on Grantlee it has been possible to put QObject derived types into the template system as context, as well as QLists or QHash containers of QObject derived types in a QVariant. Properties on the instances are then available through the Q_PROPERTY system and can be used when rendering a template.

  QVariantList peopleList;
  // Inconvenient!
  peopleList.append(QVariant::fromValue(static_cast<QObject*>(new Person("Alice"))));
  peopleList.append(QVariant::fromValue(static_cast<QObject*>(new Person("Bob"))));

  context.insert("people", peopleList);
  template->render(&context);

  ...

  <ul>
  {% for person in peopleList %}
    <li>{{ person.name }}, {{ person.age }}</li>
  {% endfor %}
  </ul>

The restriction that the type be QObject derived was because the only way to introspect them was through the Qt Property system. Only QVariantList and QVariantHash were supported because that provides sequential and associative containers, and both already have built in support in QVariant.

It is not an ideal situation however, because existing applications already have a data API which might not use QObject derived types, so wrappers would have to be written. That is a common situation in KDEPIM where we have KMime::Message, KABC::Addressee and KCalCore::Incidence and related classes which do not inherit QObject. Additionally, QList is not often the best container to use as a sequential container. Commonly people use QVector or std::vector instead.

Generic introspection

Near the end of July Michael Jansen did some work on generic type support in Grantlee. It took me until the middle of September to finally get around to reviewing the patch, and it showed me something I didn’t know was possible. There are several aspects to the work on the mjansen-experimental branch, but the core concept is being able to use C++ templates (very different to Grantlee templates) and function pointers to extract custom types from QVariants from within the Grantlee processing code. This post is about how it works, not how to use it. For the documentation about how it is used, see here.

Because it uses C++ templates, Grantlee doesn’t need specific code to support the custom types, but can expose a static interface which applications can implement for their own custom types. The real magic happens by using function pointers to create a registry of available types. It works something similar to this (some details and magic omitted for clarity):

  // static interface
  template<typename T>
  struct TypeAccessor
  {
    static QVariant lookUp(const T &object, const QString property);
  };

  // Converts the object parameter from QVariant to T for use with the TypeAccessor
  template<typename T>
  struct LookupTrait
  {
    static QVariant doLookUp( const QVariant &object, const QString &property )
    {
      typedef typename Grantlee::TypeAccessor<T> Accessor;
      return Accessor::lookUp( object.value<T>() ), property );
    }
  };

  struct MetaType
  {
    typedef QVariant (*LookupFunction)(const QVariant &, const QString &);

    QVariant doLookup(const QVariant &object, const QString &property);

    static void registerLookUpOperator( int id, LookupFunction f );
  };

  template<typename T>
  void registerMetaType()
  {
    QVariant ( *lf )(const QVariant&, const QString&) = LookupTrait<T>::doLookUp;

    const int id = qMetaTypeId<T>();

    MetaType::registerLookUpOperator( id, lf );
  }

  QVariant MetaType::doLookup(const QVariant &object, const QString &property)
  {
    LookupFunction func = registry.value(object.userType());

    // Calls LookUpTrait<T>::doLookUp(QVariant, QString);
    return func( object, property);
  }

There are some things to note there:

  • The signature for looking up a property on an object is available as a typedef on the MetaType class.
  • The same signature is available as a static method on the TypeAccessor class
  • Custom types can be registered using a template function. That retrieves a function pointer to the TypeAccessor template implementation and stores in in the MetaType using the qMetaTypeId of the type as an identifier.
  • The MetaType can later use the type in the QVariant to call the correct function to return a property
  • The custom type must have already been declared as a metatype with Q_DECLARE_METATYPE, but that’s necessary anyway to put it in a QVariant.
  • In practice the details of the TypeAccessor are hidden behind the GRANTLEE_BEGIN_LOOKUP macro.

With the above infrastructure, the rest is just the bread and butter of full and partial template specialization in applications to supply the correct hooks to Grantlee.

  // Application code:

  class Person // Not a QObject
  {
    QString name() const;
    int age() const;
  };

  Q_DECLARE_METATYPE(Person)

  template<>
  struct TypeAccessor<Person>
  {
    static QVariant lookUp(const Person &object, const QString &property)
    {
      // can only call const methods here
      if (property == "name")
        return object.name(); // Returns a QString
      if (property == "int")
        return object.age(); // Returns an int
      else
        return QVariant();
    }
  };

  void init() {
    Grantlee::registerMetaType();
  }

Any type that can be put in a QVariant can be returned from such a function, including other custom types, and including containers. All of this is not necessary for QObject derived types of course, as it is provided automatically by moc. This method is safer however as constness provides compile-time safety which is not provided by Q_PROPERTY. You can not define a clear() method on Person and then call it in the TypeAccessor specialization, whereas with a Q_PROPERTY, the READ method is not enforced to be const.

Template containment

Aside from support for generic types in Grantlee, I wanted generic container support too. Grantlee provides tags to perform {% for %} loops as above, as well as index based lookup for sequential containers and key lookup in associative containers. Rather than just allowing that with QVariantList and QVariantHash, I wanted to be able to loop over or access any Qt container, or any other user specified container. That means that if you already have an API which needs to stay as it is (for binary compatiblity reasons for example), you can use it even without changing the types of any containers in the API to QVariantList or providing conversion wrappers.

The following containers have automatic, built in support in Granltee:

  • QVector<T>, QList<T>, QStack<T>, QQueue<T>, QLinkedList<T>, QSet<T>
  • std::vector<T>, std::deque<T>, std::list<T>
  • QHash<Key, T>, QMap<Key, T>, std::map<Key, T> where Key is a QString or integral

and where T is one of:

  • bool
  • a number
  • QString
  • QVariant
  • QDateTime
  • QObject*
  • Any custom type registered with Grantlee
  • A container of any type on this list.

The container must also be registered as a QMetaType with Q_DECLARE_METATYPE.

Did you see the recursion by the way? It is also possible to use nested containers like QList<std::vector<QSet<Person> > > or StackMapListVectorInt, though possible doesn’t mean should.

Additional containers can easily be supported with a macro:

  // Add support to Grantlee for boost::circular_buffer<T>
  GRANTLEE_REGISTER_SEQUENTIAL_CONTAINER(boost::circular_buffer)

  // Make it possible to put a boost::circular_buffer<Person> into a QVariant
  Q_DECLARE_METATYPE(boost::circular_buffer<Person>)

The unit tests use std::tr1::array and std::tr1::unordered_map containers to demonstrate how to support ‘third party’ containers.

Recursive Autoconditional Container registration

The best part about the container support is that it is completely automatically conditional and determined at build time. That is, if you declare QSet<Person>, but not QList<Person> with Q_DECLARE_METATYPE, Grantlee will support only QSet<Person>, saving the build-time (building template code takes a long time) and the start-up time of registering the other container. The unit test which tests all possible builtin containers (non-nested) takes 5 minutes alone to build. The same is true of the numeric and Qt types. Support for them is determined at build time using sufficient magic

  enum
  {
    Magic,
    MoreMagic
  };

  template<typename T, int n>
  struct RegisterTypeContainer
  {
    static void reg()
    {
    }
  };

  namespace Grantlee {
  template<typename T>
  struct RegisterTypeContainer<QList<T>, MoreMagic>
  {
    static int reg()
    {
      // registerSequentialContainer is like registerMetaType
      return registerSequentialContainer<QList<T> >();
    }
  };
  }

  template<typename T>
  void registerMetaType()
  {
    // This registers the container if QList<T> has been declared as a metatype, and does nothing otherwise.
    Grantlee::RegisterTypeContainer<QList<T>, QMetaTypeId2<QList<T>>::Defined>::reg();
  }

Do you see how the magic works?

Smart pointers

There’s a memory leak on this page. In the first example I created Person QObjects on the heap and then never called delete for them. They also don’t have a parent in the example, so they leak. There are several reasons not to give a QObject a parent. Some short-lived objects have an unclear end-point, but shouldn’t be around as long as their parent. In some threading situations there are also reasons not to give a QObject a parent.

In those cases we can use QSharedPointer to manage the memory of the QObject. Grantlee now also has special (automatic condtional) support for QSharedPointers which manage a QObject derived type. QSharedPointer can now be put into a QVariant and into a Grantlee Context just like any other object, and its Q_PROPERTIES are automatically available in the Grantlee::Template.

  Q_DECLARE_METATYPE(QSharedPointer<Person>)
  Q_DECLARE_METATYPE(QVector<QSharedPointer<Person> >)

...

  QVector<QSharedPointer<Person> > peopleList;
  QSharedPointer<Person> alice(new Person("Alice"));
  peopleList.append(alice);
  QSharedPointer<Person> bob(new Person("Bob"));
  peopleList.append(bob);

  context.insert("people", QVariant::fromValue(peopleList));
  template->render(&context);

Of course it is also possible to use other smart pointers, and smart pointers to non QObject derived types. To support an additional smart pointer, a simple macro is used:

  GRANTLEE_SMART_PTR_ACCESSOR(boost::shared_ptr)
  Q_DECLARE_METATYPE(boost::shared_ptr<Person>)

In KMail, we use objects like boost::shared_ptr<KMime::Message>, so these kinds of features may prove quite useful for advanced theming in the message viewer.

TTP without TTP

There were several times while implementing the support for templated types that TTP could have been useful. Apparently support for TTP is not widespread in some compilers yet, so I avoided using that language feature. What it allows is constructs like this:

  template<typename <typename> class Container, typename T>
  struct ContainerCleaner
  {
    QVariant clean(Container<T> &container)
    {
      // Do some cleaning on Container<T> 
    }
  };

That struct would work with QList<T>, QVector<T> etc automatically as long as the symbolic API used is the same. I wanted something like that in unit tests for clean-up code. After using a Container<QObject*> I wanted to call qDeleteAll() on the container. That would be possible with the above template by partially specializing it for T=T*. Doing the same thing without TTP is almost as easy

  template<typename Container, typename T = typename Container::value_type>
  struct ContainerCleaner
  {
    void clean(Container &container)
    {
      // Do nothing in the common case.
    }
  };

  // Partial specialization for Container<QObject*>
  template<typename Container>
  struct ContainerCleaner<Container, QObject*>
  {
    void clean( Container &container)
    {
      qDeleteAll(container);
    }
  };

  template<typename Container>
  void cleanupContainer c)
  {
    CleanupContainer<Container>::clean(c);
  }

  ...
  // After test:
  cleanup(container);

The only disadvantage is that I need two clean up methods, one for sequential containers, and one for associative containers, because they are not statically polymorphic (value_type vs mapped_type). I also need another specialization for std::map types because qDeleteAll does not work for those.

Conclusion

If you have read this far I can only assume a strong interest in template programming. I’d encourage you to read the commits in the branch. They are sequential and easy to read, and if you spot anything that could be done better, do please tell me.

About these ads

11 Responses to “Grantlee pairs up with the “other” template system”

  1. Viet Says:

    Hi! Thanks for the informative article. It’s indeed lovely to use C++ templates to ease the work with data.

    May I ask why std::string is not supported, only QString? In case I need to use std::string or any custom string classes for member data, how should I go about that? For example:

    class Person {
    custom_string_t name;
    int age;
    };

    Please advise. Thank you!

  2. steveire Says:

    std::string can indeed be used. It is used in the unit test too.

    It needs to be converted to a QString though.

    http://www.gitorious.org/grantlee/grantlee/commit/90d95b58375092cf9811493f7590541546b7095e

    if ( property == QLatin1String( “name” ) )
    return QString::fromStdString( object.name );

  3. Aurélien Gâteau Says:

    Interesting article, thanks!

    One question though: why do you need the static_cast in your first example? If Person inherits from QObject it should work without cast. Am I missing something?

  4. steveire Says:

    Person does inherit QObject, yes, but the way to get the instance into a QVariant is by using fromValue<T>(). With T = Person* (without the static_cast), you would need to use Q_DECLARE_METATYPE(Person*) to get it into the QVariant. Even then though Grantlee wouldn’t be able to take it out of the QVariant automatically:

    QVariant v = QVariant::fromValue( new Person );
    v.value<QObject*>(); // Returns QObject(0x0)

    You can write a main.cpp to try that out. It might be possible to special case that in QVariant actually.

    You would need to tell Grantlee to call v.value<Person*>() in the LookupTrait by defining a trivial TypeAcessor for it:

    GRANTLEE_BEGIN_LOOKUP(Person*)
    return doQobjectLookup(object, property);
    GRANTLEE_END_LOOKUP

    You might think sfinae is an option, but you would still need to register the derived type with Grantlee. A macro like GRANTLEE_REGISTER_QOBJECT_DERIVED(Person) could make that easier, but actually I think the static_cast is easier and less template-related work for the compiler.

    The same issue exists in QML by the way. In kontact-mobile you see a lot of

    QAbstractItemModel *model = getModel();
    qmlContext->addContextProperty(“myModel”, QVariant::fromValue( static_cast<QObject*>( model ) ) );

    • Ben Says:

      It seems to me that the “QVariant::fromValue(static_cast(obj))” pattern is crying out for an inline helper function, i.e. “QVariant variantFromObject(QObject *)” and let C++ automate the upcasting…

  5. Viet Says:

    @steveire: I think that there should be natural support for std::string too. This is useful for making use of initial code base without converting to QString. Would you consider that for the next release?

  6. Aurélien Gâteau Says:

    @steveire: oh I see, thanks for the explanation.

  7. steveire Says:

    @Viet: Not at this time I’m afraid.

    The reason that std::string and QByteArray (which is a wrapper around const char *) are not supported at this time is due to encoding issues. std::string and const char * do not have an encoding. They’re just an array of bytes. The encoding of those bytes has to be known externally.

    QString is different in that it stores data in a particular format (UTF-16). If concatenating a QString and a std::string I would have to guess the encoding. Qt guesses the encoding in QString::fromAscii by using QTextCodec::codecForCStrings, but I think the choice to use that should be explicit in the applications, not built into Grantlee (the encoding of the std::string might not be the encoding returned by QTextCodec::codecForCStrings). That’s why Grantlee standardizes on QString and UTF-16 as the operating encoding. As string operations are the core of what Grantlee does, it needs to be as explicit as possible on encoding related issues.

    This makes Grantlee robust in mixed localization environments.

  8. Viet Says:

    Thanks steveire, now I’m clear about that.

  9. Grantlee version 0.1.7 now available « Steveire's Blog Says:

    [...] release introduces the significant feature of generic types, but it also has many other smaller improvements. Even that was not quite right, and needed some [...]

  10. Defensive Patent Publication for Qt - KDAB Says:

    […] features in QVariant come from my Grantlee library, based on Qt 4. I described part of it in a blog post in September 2010 as “Recursive Autoconditional Container registration”, and made another relevant commit […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: