Python Bindings
Python provides a C API to define libraries which can be imported into a python environment. That python C interface is often used to provide bindings to existing libraries written in C++ and there are multiple existing technologies for that such as PyQt, PySide, Boost.Python and pybind.
pybind seems to be a more modern implementation of what Boost.Python provides. To create a binding, C++ code is written describing how the target API is to be exposed. That gives the flexibility of defining precisely which methods get exposed to the python environment and which do not.
PyQt and PySide are also similar in that they require the maintenance of a binding specification. In the case of PySide, that is an XML file, and in the case of PyQt that is a DSL which is similar to a C++ header. The advantage both of these systems have for Qt-based code is that they ship with bindings for Qt, relieving the binding author of the requirement to create those bindings.
PyKF5 application using KItemModels and KWidgetsAddons
Generated Python Bindings
For libraries which are large library collections, such as Qt5 and KDE Frameworks 5, manual binding specification soon becomes a task which is not suitable for manual creation by humans, so it is common to have the bindings generated from the C++ headers themselves.
The way that KDE libraries and bindings were organized in the time of KDE4 led to PyQt-based bindings being generated in a semi-automated process and then checked into the SCM repository and maintained there. The C++ header parsing tools used to do that were written before standardization of C++11 and have not kept pace with compilers adding new language features, and C++ headers using them.
Automatically Generated Python Bindings
It came as a surprise to me that no bindings had been completed for the KDE Frameworks 5 libraries. An email from Shaheed Hague about a fresh approach to generating bindings using clang looked promising, but he was hitting problems with linking binding code to the correct shared libraries and generally figuring out what the end-goal looks like. Having used clang APIs before, and having some experience with CMake, I decided to see what I could do to help.
Since then I’ve been helping get the bindings generator into something of a final form for the purposes of KDE Frameworks, and any other Qt-based or even non-Qt-based libraries. The binding generator uses the clang python cindex API to parse the headers of each library and generate a set of sip files, which are then processed to create the bindings. As the core concept of the generator is simply ‘use clang to parse the headers’ it can be adapted to other binding technologies in the future (such as PySide). PyQt-based bindings are the current focus because that fills a gap between what was provided with KDE4 and what is provided by KDE Frameworks 5.
All of that is internal though and doesn’t appear in the buildsystem of any framework. As far as each buildsystem is concerned, a single CMake macro is used to enable the build of python (2 and 3) bindings for a KDE Frameworks library:
ecm_generate_python_binding(
TARGET KF5::ItemModels
PYTHONNAMESPACE PyKF5
MODULENAME KItemModels
SIP_DEPENDS
QtCore/QtCoremod.sip
HEADERS
${KItemModels_HEADERS}
)
Each of the headers in the library are parsed to create the bindings, meaning we can then write this code:
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import sys
sys.path.append(sys.argv[1])
from PyQt5 import QtCore
from PyQt5 import QtWidgets
from PyKF5 import KItemModels
app = QtWidgets.QApplication(sys.argv)
stringListModel = QtCore.QStringListModel(
["Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday", "Sunday"]);
selectionProxy = KItemModels.KSelectionProxyModel()
selectionProxy.setSourceModel(stringListModel)
w = QtWidgets.QWidget()
l = QtWidgets.QHBoxLayout(w)
stringsView = QtWidgets.QTreeView()
stringsView.setModel(stringListModel)
stringsView.setSelectionMode(
QtWidgets.QTreeView.ExtendedSelection)
l.addWidget(stringsView)
selectionProxy.setSelectionModel(
stringsView.selectionModel())
selectionView = QtWidgets.QTreeView()
selectionView.setModel(selectionProxy)
l.addWidget(selectionView)
w.show()
app.exec_()
and it just works with python 2 and 3.
Other libraries’ headers are more complex than KItemModels, so they have an extra rules file to maintain. The rules file is central to the design of this system in that it defines what to do when visiting each declaration in a C++ header file. It contains several small databases for handling declarations of containers, typedefs, methods and parameters, each of which may require special handling. The rules file for KCoreAddons is here.
The rules file contains entries to discard some method which can’t be called from python (in the case of heavily templated code for example) or it might replace the default implementation of the binding code with something else, in order to implement memory management correctly or for better integration with python built-in types.
Testing Automatically Generated Python Bindings
Each of the KDE Frameworks I’ve so far added bindings for gets a simple test file to verify that the binding can be loaded in the python interpreter (2 and 3). The TODO application in the screenshot is in the umbrella repo.
The binding generator itself also has tests to ensure that changes to it do not break generation for any framework. Actually extracting the important information using the cindex API is quite difficult and encounters many edge cases, like QStringLiteral (actually includes a lambda) in a parameter initializer.
Help Testing Automatically Generated Python Bindings
There is a call to action for anyone who wishes to help on the kde-bindings mailing list!
Steveire's Blog 