Gory technical details

In a previous post I wrote some details about how SFINAE works and provides type introspection, and how it can be used to make QVariant more effective at providing access to QObject derived classes. I submitted the patches to Qt where it got thoroughly reviewed.

One of the issues raised in review was that if the SFINAE template is used with a T which is only forward declared it does not work as expected.

  class MyObject;

  void fwdDeclared() {
    qDebug() << "fwdDeclared" << QTypeInfo<MyObject*>::isQObjectPointer;
  }

  class MyObject : public QObject 
  {
    Q_OBJECT
    // ...
  }

  void fullyDefined() {
    qDebug() << "fullyDefined" << QTypeInfo<MyObject*>::isQObjectPointer;
  }

  int main() {
    fwdDeclared();
    fullyDefined();
  }

...

$ ./test
fwdDeclared 0
fullyDefined 0

Why do we get 0 instead of one in both cases? MyClass clearly inherits QObject. Let’s try commenting out the body of fwdDeclared.

  void fwdDeclared() {
//    qDebug() << "fwdDeclared" << QTypeInfo<MyObject*>::isQObjectPointer;
  }
  
$ ./test
fullyDefined 1

Now it works. We get the expected 1 resulting from the isQObjectPointer being assigned a true. What’s going on?

It is often wise to avoid false negatives

At the point where the fwdDeclared function is defined, MyClass has only been forward declared – it is an incomplete type.

Recall that our SFINAE template depends on function overloading depending on the type of the argument to the template:

template<typename T>
struct QTypeInfo<T*>
{  
  yes_type check(QObject*);
  no_type check(...);
  enum { isQObjectPointer = sizeof(check(static_cast<T*>(0)) == sizeof(yes_type) };
};

The first time QTypeInfo<FirstType*>::isQObjectPointer is encountered is in fwdDeclared. At that point in the file, any FirstType* (such as the one in the SFINAE template) will be treated as a void pointer. That means that the catch-all check method which is not the QObject* overload will be used which returns a no_type, and the enum will be resolved to false.

This result as it is first defined will be used everywhere in the translation unit. QTypeInfo<FirstType*>::isQObjectPointer will be treated as an alias to false even after FirstType is fully defined. When we comment out the body of the fwdDeclared function, then the first time QTypeInfo<FirstType*>::isQObjectPointer is encountered is in the fullyDefined function so the base type can be seen by the compiler to be a QObject and the enum evaluates to true.

So what is a translation unit? Can’t we just consider not using them if they’re a problem?

Lost in translation

Translation is what a compiler does to turn source code into object code. A translation unit or compilation unit, is effectively the output of the preprocessor after resolving all the #if, #ifdef and #include etc directives. A forward declaration can appear multiple times in a translation unit, but a definition can only appear zero or one times. This is the one definition rule.

In the example above, the first time QTypeInfo<FirstType*>::isQObjectPointer is encountered it is evaluated to false, therefore if ODR is to be enforced within the translation unit, it must must be false throughout the entire translation unit.

QTypeInfo<FirstType*>::isQObjectPointer might even have a different value in different translation units depending on whether FirstType was defined or forward declared in each one, or the compiler might just spit it out if you try it, though GCC isn’t there yet.

The ODR is not necessarily enforced by the compiler. Causing the template trait to be evaluated differently would also conceivably be an ODR violation and it is undefined whether the compiler needs to do anything about that because compilers don’t necessarily have all the information to make that detection.

Relying on undefined behaviour would be dangerous.

You just have to deliberately the whole thing!

To bring this back to the QVariant/QMetaType context, the SFINAE template as it was before did function if the type T was forward declared, although it gave the wrong answer. Any of the QVariant functions that make use of the template require T to be a complete type anyway, but that still leaves the issues of maintainability (Qt developer in the future might use the trait in a way that doesn’t require the full type) and C++ header voyeurism (third party sees something in internal API that looks useful and uses it, with undefined, undebuggable results).

Both are valid issues of course, but we already know that using the trait with a forward declared T always gives the wrong answer simply because it doesn’t always give the right answer and might cause ODR violations. So we want to enforce that the type is fully defined.

The way to do that is to simply do use the type in a way that requires the full type to be known. One option would be to try to call a static method like connect() on the QObject that T is supposed to inherit. If a call to connect() can be made the type is fully defined. This fails to compile of course if T happens to not have a static connect() method in it’s interface. That option goes out the window as we do still need to compile if it is not a QObject.

Size *does* matter

Another language feature that requires the full type to be defined is the sizeof operator. sizeof(T*) == sizeof(void*) in most cases and that works even if T is forward declared. sizeof(T) however requires that T be fully defined. So all we need to do is invoke sizeof(T) somewhere in our SFINAE template and we’ll get a compiler error if it isn’t.

template<typename T>
struct QTypeInfo<T*>
{  
  yes_type check(QObject*);
  no_type check(...);
  enum { isQObjectPointer = sizeof(check(static_cast<T*>(0)) == sizeof(yes_type) + (0 * sizeof(T)) };
};

Consequence

Because we now require that the full type is known when evaluating the SFINAE template, we compromise the feature of automatic Q_PROPERTY handling. The translation unit that the .moc file is in does not necessarily contain a full definition for T so QTypeInfo<T*>::isQObjectPointer won’t necessarily compile and we can’t put it in .moc files.

The final twist in the tale is that we can’t even use QMetaType to store the information about whether a type is a QObject subclass anymore. In the previous patch that information was stored in the QMetaType data structures as a result of the qRegisterMetaType() call. However, it turns out that qRegisterMetaType also does not require the T to be a complete type. Using our SFINAE class inside that method imposes that as a new restriction which is source incompatible.

So instead of storing that information once per metatype in QMetaType, we have to store it in the QVariant as part of the data type stored in it. This turned out to be a better solution in the end anyway because it eliminates calls to QMetaType which require locking and unlocking a mutex.

It’s still not in Qt yet though, we’ll have to see if it makes it.

[Aside: A picture is worth a thousand words, and this post is now just over a thousand. I guess the picture is only worth the words if you know the words…]

2 Responses to “Gory technical details”

  1. Naproxeno Says:

    Thanks for taking the time to write this.

  2. Grantlee v0.1.9 (Codename affengeil) now available « Steveire's Blog Says:

    […] efforts around introspection of QObject derived types in QVariants did not lead to improvements in Qt4. It is possible that improvements will be made to the […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: