Archive for April, 2011

An elaborate joke?

April 6, 2011

I started writing Grantlee::Tubes some time in December 2010. In the course of writing it I’ve mostly been researching what the dependable API of QIODevice is. I don’t know when Tubes will get into a release of the Grantlee libraries, but it probably won’t be the next release. All of the classes don’t do error handling that they could do yet, and the QIODevice research is still ongoing.

Nevertheless I decided to start publishing blogs about Tubes in mid-March to give context to the April fools post about it.

Talk about building up a joke.

The library and concepts are real and useful though, so I’ll push on with publishing these experimental devices to the repo.

Consider a use case where you want to read from a QIODevice that is being written to by another class. For example, QTextDocumentWriter writes stream of valid ODF XML to a QIODevice and QXMLStreamReader reads a stream of XML data from a QIODevice.

How can we connect them together?

One way might be to use a QBuffer.

  QBuffer buffer;
  QTextDocumentWriter writer(&buffer);
  writer.write(document);

  buffer.seek(0);

  QXMLStreamReader reader(buffer);

  // ... Do something with the XML.

This works, but it’s not a generic solution. If we wanted to do asynchronous writing of data to the device and asynchronous line based reading from it, we would have to make the buffer a member of a class, and when reading from it we would have to do something like this:

  void MyClass::onReadyRead()
  {
    if (!m_buffer->canReadLine())
      return;
    m_buffer->seek(0);
    const QByteArray line = m_buffer->readLine();
    m_buffer->buffer.chop(line.length());
    m_buffer->seek(m_buffer->size());
    useLine(line);
  }

Reading from a buffer does not invalidate the data it holds. We have to use a method returning the QByteArray to chop off the part we read ourselves. We also have to remember to seek() a few times on the buffer. I didn’t even try the code out for off-by-ones.

Enter Grantlee::Channel.

Grantlee::Channel already made an appearance in my last post. The idea is to solve a connection problem with QIODevices. While Pump can transfer data from a device that should be read to a device that should be written to, Grantlee::Channel is an intermediary providing both a device that consumes data and one that produces data.

The difference between Grantlee::Pump and Grantlee::Channel

There are several significant differences between it and QBuffer. QBuffer is not a sequential device, but Channel is. That means that The pos() and seek() methods are relevant API when working with a QBuffer, but irrelevant and meaningless when working with a Channel. As an implementor of the QIODevice API that means I don’t have to implement any meaning into those virtual methods and can ignore them. Instead I can implement the readData and writeData methods to implement a FIFO semantic. The Channel can be written to at any time, and read from whenever it has data. There is no need for seek()ing, and it automatically discards data that has been read, meaning no manual memory conservation responsibility for the caller.

    QTextDocument *document = getDocument();

    Grantlee::Channel *channel = new Grantlee::Channel(this);
    channel->open(QIODevice::ReadWrite);

    // Write to the channel
    QTextDocumentWriter *writer = new QTextDocumentWriter(channel, this);

    // Read from the channel
    QXmlStreamReader reader(channel);

    // Use the reader.

Easy.

Advertisements

Another one bytes the dust

April 5, 2011

I’ve just pushed a change to kdepim 4.4 which removes this annoying dialog in a few annoying cases.

For users, this dialog was coming up, and not seeming to give any useful information, and when dismissed, the application was usable.

Showing the dialog was actually a bug that was fixed some time in February 2010 with improvements to the kdepim libraries, but because there was no KDEPIM applications 4.5 release, didn’t make it to users.

The fix was to make the applications not call dangerous API with sub-eventloops.

Making KDEPIM less annoying

April 4, 2011

I’ve started looking into KDEPIM 4.6 on Kubuntu Natty to see if it can be made less annoying to use. There are two unpopular dialogs which appear when using KDEPIM. Both are telling the user that essential PIM services are not fully operational.

The essential PIM services are Akonadi and Nepomuk. Akonadi provides access to all PIM data (emails, contacts, events etc) the user has. It is started automatically if using a PIM application like KMail2, KAddressBook, KOrganizer, KJots and more. There is no configuration option to turn Akonadi off. Akonadi is a cache server which uses a database like MySQL or SQLite to cache data.

Nepomuk provides indexing and searching capabilities to the PIM infrastructure. If you want to search your email, or use autocompletion when typing in email addresses, you need Nepomuk. These are currently considered essential features for a useful PIM stack, so Akonadi depends on Nepomuk being operational. Unfortunately it can be turned off, and when it is off, that’s when the user gets the two unpopular dialogs.

There may be a case for coming up with a unified framework for how services can depend on each other and give the user the opportunities to start essential dependent services. It might be something to discuss at the Platform 11 sprint.

However, there are things we can change in the short-term that can benefit the user. For one, I’ve turned one of the annoying dialogs into a passive notification using KNotification.

A notification is less annoying than a dialog

Next I’ll have to consider how to show the other annoying dialog only when attempting to search or autocomplete email addresses…

Grantlee::Thermodynamics and Refrigeration

April 1, 2011

With some new classes in Grantlee::Tubes starting to take shape, I started making use of them in the new Grantlee::Thermodynamics library with the Grantlee::Refridgeration system.

Grantlee::Refrigeration makes use of components from Grantlee::Tubes, like Grantlee::Pump, QtIOCompressor and Grantlee::Cat to create an ideal heat transfer system. The intention is to create a stream of “cold bytes” taking heat out of hot paths in your code, and disposing of the heat through your cooling fan.

Coolness. You can't get enough

Thermodynamics teaches us that while the quantity of energy in a closed system is constant, the quality of the energy (it’s entropy) is not. Entropy in a closed system is always increasing (the quality or useful energy in the universe is always going down), but we can locally decrease the entropy in a body of mass if we transfer it to another body of mass. This is what refrigeration is about. The decrease in entropy is made visible in the cold cavity of a fridge by the state change that water undergoes from fluid (higher entropy) to solid (lower entropy).

We can take heat (enthalpy and entropy) away from somewhere, but we have to dump that heat somewhere else. It takes work and a refrigerant to transfer heat between thermodynamic bodies. Heat won’t move spontaneously by itself (beyond the limits of equilibrium) so typically the work of heat transfer is done by a pump. Heat transfer in a refrigerator works in a cycle of four stages.

Grantlee already provides a Pump which we can use in our thermodynamic system, and we can use any refrigerant which is plentiful and which has a high capacity for entropy and a lot of hot air, such as a twitter feed.

We start by seeding the system with our refrigerant.

  QNetworkReply *reply = QNetworkAccessManager::get("http://www.twitter.com/feed");
  Grantlee::Cat *cat = new Grantlee::Cat;
  cat->appendSource(reply);

Cat will read the data from the reply object until it closes, at which point it will start to read from another device to close the cycle (shown later). At this point the data is already saturated; It can’t contain any more entropy at this temperature and pressure.

1 – 2: Reversible Adiabatic (Isentropic) Compression

Typically the first step described in a refrigeration cycle is Isentropic compression – that is, compressing the refrigerant without changing its entropy. The compression causes the data to become super-saturated. We compress the data by tubing the refrigerant through a QtIOCompressor.

  // (condenser shown later)
  Condenser *condenser = new Condenser;
  QtIOCompressor *compressor = new QtIOCompressor(condenser);
  cat->setTarget(compressor);

2 – 3: Constant Pressure Heat Rejection

After compression comes constant pressure heat rejection. As all developers know, constraints of constness can be expressed in C++ with the const keyword. So we require a class for rejecting the heat which will enforce that constraint:

  class Condenser : public QIODevice
  {
    ...  
  protected:
    void writeData( const char* data, qint64 len );
  };

Fortunately, the virtual writeData method of QIODevice already takes the data (our refrigerant) as const, so that constraint is already enforced for us. The condenser causes a change of state of the data, thereby decreasing its entropy. The data is once again saturated, but now in its low entropy state and at a lower temperature.

3 – 4: Adiabatic Throttling

We now have to connect up a throttle to perform isentropic expansion, so the entropy and the temperature is maintained, but the refrigerant changes state and becomes unsaturated.

A throttle is trivially implemented by using a QtIOCompressor in reverse, so we omit that for brevity.

At this point, we have our stream of cold bytes at a low temperature and unsaturated, so with a capacity to absorb some heat, so let’s do that with an evaporator.

4 – 1: Constant Pressure Heat Addition

We require that heat absorption occurs at constant pressure, and once again the const keyword ensures that.

  class Evaporator : public QIODevice
  {
    ...  
  protected:
    void writeData( const char* data, qint64 len );
  };

(The implementation of the Evaporator is left as an exercise for the reader)

We can then use the cold bytes in the method that defines our hot code path where we use an evaporator to facilitate the heat transfer to the refrigerant:

  void MyClass::hotPath(..., QIODevice *source, QIODevice *target)
  {

   // ...

    Evaporator *evaporator = new Evaporator;
    evaporator->setTarget(target);
    evaporator->setSource(source);
  }

The very presence of the evaporator in the hot path of our code is enough to cause heat transfer to the cold bytes, increasing their entropy by causing them to change state.

Of course this means that we need to call our hot path with the refrigerant tubing included:

  Grantlee::Channel *channel1 = new Grantlee::Channel;

  myInstance->hotPath(..., throttle, channel1);

  Grantlee::Pump *pump = new Grantlee::Pump;
  pump->setSource(channel1);

  Grantlee::Channel *channel2 = new Grantlee::Channel;
  pump->setTarget(channel2);

  cat->setSource(channel2);

As a result of the state change in the evaporator the data also becomes saturated at high entropy. Recall that this is the same state the refrigerant we originally introduced from twitter was in.

We route the refrigerant from the hot path and into a Grantlee::Pump, which provides the work required to satisfy the Second Law, and then forwarding the result on to cat, thereby closing the cycle.

Results

I ran some tests using the Grantlee Thermodynamics Toolkit on various algorithms with various parameters of compression and throughput, with results indicating a universal increase in performance when refrigeration was used.