The Future of AST-Matching refactoring tools (EuroLLVM and ACCU)

I recently made a trip to LLVM in Brussels and ACCU in Bristol. It was a busy week. I gave a talk at both conferences on the topic of the future of AST Matchers-based refactoring.

As usual, the ‘hallway track’ also proved useful at both conferences, leading to round-table discussions at the LLVM conference with other interested contributors and getting to talk to other developers interested in refactoring tooling at ACCU.

Presentations

The learning curve for AST-Matcher-based refactoring is currently too steep. Most C++ developers who are not already familiar with the internal Clang APIs need to invest a lot in order to learn how to make such bespoke tooling to improve and maintain their codebase.

The presentations both include demos of steps I’ve been taking to try to address these problems.

The first demo is of clang-query discovery features which aim to reduce the need to infer AST Matcher code by examining the Clang AST itself. I also showed the debugging features I am preparing to upstream to clang-query. Finally – in terms of demo content – I showed a Qt tool which can eliminate some of the additional difficulties and problems of long-developer-iteration-time.

The debugging features and the Qt tool were world exclusives at the LLVM conference (and at the ACCU conference because most people didn’t attend both 🙂 ). I hadn’t shown them to anyone else before, so I was quite happy the demos went well.

Videos

My 25 minute presentation to the LLVM developers tried to show that these changes can make mechanical refactoring more easily available to C++ developers.

The aim was to show the features to the LLVM community to

  1. illustrate the issues as I see them
  2. get some feedback about whether this is a good direction
  3. introduce myself for the sake of further code reviews (and collaborators). As this was my first LLVM conference, I am not already familiar with most of the attendees.

My 1.5 hour ACCU presentation is a far-less-rushed presentation of the same tools and a repetition of some of the content at code::dive 2018. In the ACCU presentation, the new demo content starts about 30 minutes in. This talk is the one to watch if you are interested in using mechanical refactoring on your own code.

Feedback was very positive from both talks, so I’m happy with that.

Qt Tooling

Earlier this year I refactored the clang AST dump functionality. It was previously implemented in one class, ASTDumper, which had the dual responsibilities of traversing the clang AST and creating a textual representation of it. I separated the textual output from the generic output independent traversal, which introduces the possibility of alternative output formats such as JSON.

Of course, given my KDE and Qt contribution history, I would only create a generic tree traversal class in order to implement QAbstractItemModel for it.

The demos show all of the features you would expect from a point-and-click refactoring tool including exploring, feature discovery, debugging with in-source feedback, live source updates, experimental refactoring etc.

Of course, all of this requires changes to Clang upstream (for example to add the debugging interface) which was the point of my visit to EuroLLVM. Hopefully, once enough of the changes are upstream, I’ll be able to open source the tool.

The idea as always is to hopefully have enough functionality in Clang itself that IDEs such as Qt-Creator, KDevelop and Visual Studio would be able to integrate these features using their own GUI APIs, making the simple tool I made obsolete anyway. I only made it for demo purposes.

This will take the mechanical refactoring workflow which is currently

and turn it into

You will still do the same things, but with much faster development iteration to achieve the same result.

There is even more that can be done to make the process of mechanical refactoring with clang easier and faster. We discussed some of that at EuroLLVM, and hopefully all the pieces can come together soon. Meanwhile I’ll be upstreaming some of this work, talking at NDC Oslo, and at my local meetup group on this topic.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: