27 Posts

Google offers an REST API for most of its services. The user has to allow the app access through OAuth authorization and ask for certain permissions, and the fact that an app has been granted those permissions is done by checking the access token on every request to the Google APIs. Access tokens are just a series of characters that you pass on every request.

In total, there are three distinct requests involved in the authorization process for devices: Retrieving a user code and authorization link, retrieving an access token intitially and refreshing your access token once it has expired. See the following diagram for reference:


  • The first step is to retrieve a generated user code and authorization link. The user is expected to be shown the user code, then he must follow the authorization link (may that be on another device), log in with his Google account and type in the user code. This is when Google knowns the application has been granted permissions to actually make calls to the Google APIs.
  • On the first time the application tries to obtain an access token, you will need to call retrieveAccessToken. You will get an access token that will expire after some time. Along with this first access token, you will be supplied a refresh token. The refresh is used to renew you access token, once it has expired.
  • If your access token expires (Google says you’ll get a 401 then), you need to call refreshAccessToken to obtain a new one.

For all these things I have written a tiny authorization library that you’ll find in github: OAuth2 on github

You probably noticed that in order to even make calls to the authorization servers you need to provide a “client id”, a “client secret” and a “scope”. You obtain these by registering your application the Google Developer Console.

I have struggled a lot to simply obtain a continous video stream from QCamera without QtGui as single QImages. After a lot of searching, I finally got to a very short and elegant solution. The Qt documentation is really lacking about this. The obvious (and imo correct) solution would be to use QVideoProbe and connect to the videoFrameProbed signal, but this does only work on a couple of platforms and in very exotic constellations (for example, as of now QCamera only works with with QVideoProbe on Android).

The solution is to subclass QAbstractVideoSurface and override its present-method. This is how it looks like:

#include "cameraframegrabber.h"

CameraFrameGrabber::CameraFrameGrabber(QObject *parent) :

QList<QVideoFrame::PixelFormat> CameraFrameGrabber::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
    return QList<QVideoFrame::PixelFormat>()
        << QVideoFrame::Format_ARGB32
        << QVideoFrame::Format_ARGB32_Premultiplied
        << QVideoFrame::Format_RGB32
        << QVideoFrame::Format_RGB24
        << QVideoFrame::Format_RGB565
        << QVideoFrame::Format_RGB555
        << QVideoFrame::Format_ARGB8565_Premultiplied
        << QVideoFrame::Format_BGRA32
        << QVideoFrame::Format_BGRA32_Premultiplied
        << QVideoFrame::Format_BGR32
        << QVideoFrame::Format_BGR24
        << QVideoFrame::Format_BGR565
        << QVideoFrame::Format_BGR555
        << QVideoFrame::Format_BGRA5658_Premultiplied
        << QVideoFrame::Format_AYUV444
        << QVideoFrame::Format_AYUV444_Premultiplied
        << QVideoFrame::Format_YUV444
        << QVideoFrame::Format_YUV420P
        << QVideoFrame::Format_YV12
        << QVideoFrame::Format_UYVY
        << QVideoFrame::Format_YUYV
        << QVideoFrame::Format_NV12
        << QVideoFrame::Format_NV21
        << QVideoFrame::Format_IMC1
        << QVideoFrame::Format_IMC2
        << QVideoFrame::Format_IMC3
        << QVideoFrame::Format_IMC4
        << QVideoFrame::Format_Y8
        << QVideoFrame::Format_Y16
        << QVideoFrame::Format_Jpeg
        << QVideoFrame::Format_CameraRaw
        << QVideoFrame::Format_AdobeDng;

bool CameraFrameGrabber::present(const QVideoFrame &frame)
    if (frame.isValid()) {
        QVideoFrame cloneFrame(frame);
        const QImage image(cloneFrame.bits(),
                           QVideoFrame::imageFormatFromPixelFormat(cloneFrame .pixelFormat()));
        emit frameAvailable(image);
        return true;
    return false;

// Qt includes
#include <QAbstractVideoSurface>
#include <QList>

class CameraFrameGrabber : public QAbstractVideoSurface
    explicit CameraFrameGrabber(QObject *parent = 0);

    QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const;

    bool present(const QVideoFrame &frame);

    void frameAvailable(QImage frame);

public slots:



This is how you would use it in your Qt application:

_camera = new QCamera();
_cameraFrameGrabber = new CameraFrameGrabber();
connect(_cameraFrameGrabber, SIGNAL(frameAvailable(QImage)), this, SLOT(handleFrame(QImage)));

You can download both files here:

Within this blogpost, I’d like to write about QJackAudio, which is actually the result of a personal itch I scratched. I know that others had the same problem as we had, since there have been libraries developed such as QJack, which are abandonend now. But first things first, let me explain what JACK is and why you might need a library like QJackAudio.

JACK is a low-latency sound server that runs on top of the the operating system’s sound architecture. Actually, this is not a new idea, a lot of professional audio software manufacturers introduced such a system in order to make it possible to route audio signals in near-realtime. A practical use-case would be recording audio and being able to monitor your recording via software without a significant delay in your playing and the monitored signal. JACK has evolved to what one might call a standard in the professional GNU/Linux audio world. Practically all DAWs, effects processors and analysis software running on GNU/Linux support JACK.

Due to the fact that all software is able to communicate with the JACK audio server, you can connect any application to another using virtual cables, for example with qjackctl, which is a control application for the JACK server and allows you to draw connections (audio and MIDI) visually. If you launch an application and that application registers at the JACK server, qjackctl will display that unit and all its in- and outputs that you can drag on those of other JACK applications.

Applications that want to use JACK can use libjack, a C-library that you can link against (it’s LGPL). I you are aiming for clean code, you will most probably wrap that C code in C++/Qt classes – and that is where QJackAudio kicks in. I have done that once, and rewritten everything a second time to provide a wrapper in a library that you can make use of. As a plus, I have included a few common digital filters that are common in the audio world, such an EQ, an amplifier, compressor and the the list goes on (and there is more to come). Just create a new project and write your own effects, virtual guitar amps, mixers or anything you might think of, without worrying about hardcore DSP stuff.

In order to demonstrate its capabilities, I have written a demo application called mx2482 that is build on top of QJackAudio. The project is hosted on github, for more information see here: https://github.com/cybercatalyst/qjackaudio

It has been a few months since I have been working on octave and when I launched GNU Octave after that time I was surprised to see how much progress it made compared to the state when it was considered to be working. Unlike in the usual industry, where being successful means to meat your deadlines by any means, what mostly results in confronting the users with buggy software, these rules do not apply to the octave developers team – and I am really glad about that. Things are done when they’re done.

Nevertheless I found the GNU Octave to be quite stable, I did not experience any bad failures like crashes or hangups. There were some annoyances here and there, but I was able to use it for my bachelor thesis months ago without problems, probably the first time someone ever tried to use the GUI in a light form of “production”. I have not experienced critical failures like data loss or the like.

As I was rediscovering the application I used to be writing myself, I immediately noticed slight, but powerful changes applied by the developer community. For example, every subwindow can now be handled as an independent top level window, as it adds a system bar to the dock widget. Quite a cool idea, I thought, because I intuitively did the right thing when grabbing the subwindow: When I tried to pull it onto another screen, I grabbed the system title bar and afterwards, in order to put it back into the tiled view, I grabbed the dock bar. Whoever is responsible for this idea, great job!

Anyways, I decided to make a few snapshots of the current GNU Octave, just to give you an impression of what the GNU Octave developers already achieved. First off, there is the overview over the whole application, leaned towards a well-known proprietary software solution that you might be aware of. On the top left side you can see the workspace, all variables sorted for categories with their respective values and their type. Unfortunately, there is still no way to edit these directly in the workspace view, but it helps beginners a lot to be able to track what’s actually going on inside. Even if it is only readable at that time, it is a vast improvement compared to the terminal only version of octave.


On the bottom left side, there is the history view. Unlike you might be used to, it is not organized in a tree structure, but rather a plain list, which is inline-searchable. If you are looking for certain command, it is highly unlikely that you might remember the exact date and time you used it in the past, so this didn’t make much sense to us. On the other hand, in most cases you will remember parts of it. The natural way of finding a command is to use the inline search then to decrease the number of possible hits and then help you mind by looking at the few proposals left over after that search. Now we’re heading over to the right side: The file browser. It allows you to directly navigate your files. By clicking twice on a file, you can open it up in the built-in editor (I will refer to that later on). Also, by changing the current directory, you change the current working directory for the interpreter.


Last, there is the documentation subwindow, which is able to parse the texinfo documentation of octave and reinterpret it as hyperlinked, tabbed documents. Before, this was part of QtOctave, but was taken out and slightly improved for GNU Octave. If you wish, you are still able to use the doc command in the terminal.


The terminal is a true terminal emulation taken from the KDE Konsole project and heavily modified for GNU Octave. Whereas a terminal emulation has to do session handling and communicate to a foreign process, this terminal is something special: It runs in the same process it is connected to, but different threads. A “short-circuit”, so to say. The reason it does this is that the GUI lives in the same process as the interpreter and this is able to access all octave data that resides in the same process space. Well, this is the basic idea, in practice it is a bit more complicated because GNU Octave is not designed to be threadsafe, but this is not important to know unless you really need to hack the code. It was very important to us to preserve the capabilities of a true terminal emulation, as most users grew accustomed to using a terminal and readline features. The mediocre terminal solutions provided by previous IDEs would not even have a slight chance to convince resident terminal users to switch over to an IDE-based solution. The fact that you can operate octave in a real terminal emulation gives the terminal user the power he needs. You can even run other terminal-based programs right within the IDE, if you need to, like the screenshot demonstrates.

Still there are some headaches with this, for example a very basic feature does not work yet: Ctrl+C does not cancel the current action, an absolute show-stopper. Also, the d esign of “short-circuiting” the process to itself leads to the undesired fact, that even debug messages on std::err that usually are supposed to be ignored by a GUI application (to be exact: printed on the invisible terminal) will be printed into the terminal. These were already identified bugs and were put into the GNU Octave bugtracker. Ripping out the Konsole frontend and make it work that way was an adventerous quest and it would have been a miracle if it had worked out of the box. I guess it is a question of time until we have fixed these weird bugs, at least many have been fixed already.



The editor is a key component in GNU Octave. It comes with everything a modern editor should have today, including code completion, syntax highlighting, search/replace facilities, line numbering and so on. A truly novel feature is the possiblity of graphical debugging: by clicking on the left margin you can set break points and steer the GNU Octave debugger with buttons or shortcuts directly in the editor, tracking the current state in the workspace window.

If you can’t wait for the GNU Octave IDE version, please consider supporting that project. You can help directly in the development, pay someone to develop a specific feature or resolve a bug, test it with your own workflow and make suggestions on improvements or – what’s also very important – spread the word about it. Spreading the word is very underrated, but it helps to draw the attention of that people who actually make decisions. Instead of buying licenses for proprietary software, even if these people decide to use GNU Octave and donate a small fraction of these license fees, this immensely helps GNU Octave to gain momentum.

Qt is an applicationd development framework originally developed by Trolltech, then Nokia and finally Digia bought the rights to distribute Qt under a proprietary license. Besides the proprietary license, Qt is also available under the terms of the LGPL and as far as I was reading somewhere on the internet Nokia agreed to release it as free software entirely when support for the proprietary license will run out. That means that it’s safe to develop free software that relies on Qt. As always, there are different on opinions on Qt, but a remarkable number of developers would agree that Qt is one of the best application frameworks so far in terms of consistency and documentation.

Unnecessarily, Qt introduced its own build system called qmake, but the standard in the free software world undoubtly is GNU Autotools, which is the build system you might know for the three steps: ./configure, make and make install. I am pretty sure that if you’re reading this blog you have seen these commands at least once in your life. When we’re translating a Qt project to make it work with GNU Autotools, we first have to understand how the Qt tools moc (Meta-Object-Compiler), uic (User-Interface-Compiler) and rcc (Resource-Compiler) work together to compile a Qt program. The following diagram illustrates what qmake does:

What we need to do is to replace four things:

  1. Write a rule that specifies how a *.ui-file shall be handled.
  2. Write a rule that specifies how a *.cc-file containing a Q_OBJECT macro shall be handled.
  3. Add generated files to the Makefile.
  4. Write a rule that specifies how a *.qrc-file shall be handled (not in the diagram).

Also, in order to satisfy GNU Autoconf, we should add checks in the configure.ac. The following implementation is not from me, I have taken it from the GNU Octave project. Originally, this has been written by John W. Eaton who integrated GNU Octave’s Qt GUI into the GNU Octave build. Let’s start with GNU Autoconf, these are the lines you have to insert into your configure.ac:


# ...

# Check for Qt libraries
PKG_CHECK_MODULES(QT, [QtCore, QtGui, QtNetwork], [], [AC_MSG_ERROR([Qt libraries are required.])])

# Retrieve Qt compilation and linker flags
CPPFLAGS="`$PKG_CONFIG --cflags-only-I QtCore QtGui QtNetwork` $CPPFLAGS"
LDFLAGS="`$PKG_CONFIG --libs-only-L QtCore QtGui QtNetwork` $LDFLAGS"
LIBS="`$PKG_CONFIG --libs-only-l QtCore QtGui QtNetwork` $LIBS"

if ! `$PKG_CONFIG --atleast-version=4.6.0 QtCore`; then
   AC_MSG_ERROR([Qt >= 4.6.0 is required.])

AC_CHECK_PROGS(MOC, [moc-qt5 moc-qt4 moc])
AC_CHECK_PROGS(UIC, [uic-qt5 uic-qt4 uic])
 if test -z "$MOC" || test -z "$UIC" || test -z "$RCC"; then
   AC_MSG_ERROR([Qt utility programs moc, uic, and rcc are required.])

For GNU Automake we now have to specify build rules to mimic the code generation via the various Qt tools (moc, uic, rcc). Add this to the bottom to your Makefile.am:

moc-%.cc: %.h

ui-%.h: %.ui
	@UIC@ -o $@ $<

qrc-%.cc: %.qrc
	@RCC@ -o $@ $<

Please not that the C++ source files now follow the GNU Style using a *.cc-extension instead of *.cpp. Now that we have set up the rules for processing various files with the Qt tools, we can incorporate our source files as follows:

bin_PROGRAMS = myProgram


myProgram_SOURCES = 

The BUILT_SOURCES-variable defines targets that have to be build first. If you take a look on the diagram explaining the Qt build process, you will note that the uic tool will actually create header files. This is the reason why we specify the ui-* targets to reside in the BUILT_SOURCES. The targets generating object code reside in the program_SOURCES-variable, as these do not depend on a specific build order.

Of course, I have boiled this down to a small example. If you are interested in how the solution developed by John W. Eaton looks like, I recommend to look into the GNU Octave sources under the libgui-folder.

Within our (B.Sc. O. Ritters and mine) bachelor thesis, which I am about to publish as free software on my website soon, we have been developing an audio application as a plugin for the JACK Audio Connection Kit. This article should give you a brief overview of how JACK works and how you access it to develop your own high-performance audio applications.

The JACK Server is a free software audio server running on all major platforms allowing for very low latencies (less than 10ms) due to bypassing the heavy-weight audio processing of the operating system. By the fact it’s written in C and optimized for speed, you can expect it to work with as little overhead as possible, yet it abstracts the underlying audio hardware very well. For audio recording and processing, low latency is absolutely crucial. Playing the guitar with latencies more than about 20 ms in the effect chain (think of it as the time that passes by from when you hit the string until the sound reaches your ear) is barely possible anymore, because you completely lose the feeling for the instrument. From a practical point of view, the technologies chosen to develop JACK are just the right pick. It can hold well against – or is even better than – its proprietary counterpart called ASIO (Audio Stream Input/Output), which has been developed by Steinberg Media Technologies GmbH. There is plenty of professional software available that uses JACK, for example Rackarrack, which is a complete guitar effects processor, Ardour 2, a high-end digital audio workstation for music production, a lot of synthesizers, generators and even more is being developed in many projects while you are reading this article.

JACK comes with the concept of virtual audio clients that can provide input and output channels, just as real audio hardware would do, too. The underlying system will be integrated as such a virtual device, called the “system”-device, ready for connecting to other devices. There is a widely used, graphical tool called qjackctl (Qt JACK Control) that nicely resembles the idea of having a virtual patchpanel. It looks like this:

When writing an application for JACK, it would appear next to the system device and you would be able to connect by simply dragging one input or output on another. And that’s it. If you connect an audio source (an input such as a microphone) to multiple sinks (for example multiple programs, speakers etc.) all sinks will receive the exact same signal. Also, if you connect multiple sources to a sink, their signals will add up, just as you would expect from ideal inputs/outputs. Unlike with true audio hardware, you do not have to worry about input/output impedance, cable properties (length, capacitance, resistance), broken connectors or even induced interference.

So how do we make our application work with JACK? The basic idea is very simple: All you need to do in your program is to supply a callback function to be called by JACK whenever the need arises to process data. Within that callback you can access input buffers and write into output buffers, ie. there is a certain amount of memory reserved by JACK for that purpose and all you get is a pointer to these parts of memory and an information on how many samples you are expected to process. All input values are normalized and your output will clip to -1.0 .. +1.0 (“clipping” = hard truncation of a signal. You really should avoid doing that!). That’s pretty easy to straightforward.

I will introduce you to the API by showing you excerpts from the program that we have developed during the bachelor thesis. Since we were developing a C++ program, but JACK offers a C API, we used a singelton style wrapper class with static methods. In C++, methods do not have a fixed address and therefore cannot be used for callbacks, but static methods do. So here is what it looks like:

bool JackAdapter::connectToServer(QString name) {
    if((m_jackClient = jack_client_open(name.toStdString().c_str(), JackNullOption, NULL)) == 0){
        return false;
        qDebug() << "Can't connect to JACK Server.";

    jack_set_process_callback(m_jackClient, JackAdapter::process, 0);
    jack_set_buffer_size_callback(m_jackClient, JackAdapter::bufferSizeCallback, 0);
    jack_set_sample_rate_callback(m_jackClient, JackAdapter::sampleRateCallback, 0);
    return true;

void JackAdapter::startAudioProcessing() {
    if(jack_activate(m_jackClient) != 0) {
        emitError("Activation of JACK client failed.");

void JackAdapter::stopAudioProcessing() {

That code is self-explanatory and it’s all you need to do for initialization. Note the call to jack_set_process_callback, which will register the callback triggered by the JACK Server. Now, we have talked about registering inputs and outputs to the JACK Server. For that purpose we have also used a struct to better organize the code of our application, but basically it’s a simple call to jack_port_register to make the magic happen.

void JackAdapter::registerStereoInputPort(QString label) {
    StereoPort stereoPort;
    stereoPort.m_leftPort = jack_port_register(m_jackClient,
        (label + "_1").toStdString().c_str(), JACK_DEFAULT_AUDIO_TYPE,
            JackPortIsInput, 0);
    stereoPort.m_rightPort = jack_port_register(m_jackClient,
        (label + "_2").toStdString().c_str(), JACK_DEFAULT_AUDIO_TYPE,
            JackPortIsInput, 0);
    m_stereoInputPorts[label] = stereoPort;

void JackAdapter::registerStereoOutputPort(QString label) {
    StereoPort stereoPort;
    stereoPort.m_leftPort = jack_port_register(m_jackClient,
        (label + "_1").toStdString().c_str(), JACK_DEFAULT_AUDIO_TYPE,
            JackPortIsOutput, 0);
    stereoPort.m_rightPort = jack_port_register(m_jackClient,
        (label + "_2").toStdString().c_str(), JACK_DEFAULT_AUDIO_TYPE,
            JackPortIsOutput, 0);
    m_stereoOutputPorts[label] = stereoPort;

As soon as we have obtained a handle to a port, we’re able to access buffers, which has been encapsulated in the StereoPort struct:

/** Defines a stereo port. */
typedef struct StereoPort {
    friend class JackAdapter;
 * Provides the address to the sample buffer for the left channel.
 * @param samples Number of samples.
 * @return Pointer to samples buffer.
    jack_default_audio_sample_t *leftChannelBuffer(int samples)
        return (jack_default_audio_sample_t*)jack_port_get_buffer(m_leftPort, samples);

 * Provides the address to the sample buffer for the right channel.
 * @param samples Number of samples.
 * @return Pointer to samples buffer.
    jack_default_audio_sample_t *rightChannelBuffer(int samples)
        return (jack_default_audio_sample_t*)jack_port_get_buffer(m_rightPort, samples);
    /** Left JACK port associated with this stereo port. */
    jack_port_t *m_leftPort;
    /** Right JACK port associated with this stereo port. */
    jack_port_t *m_rightPort;
} StereoPort;

There is a fixed time span that depends on the sampling rate and amount of buffers that is available for writing to the output buffers. It is your responsibility to get all your calculations done within this time, but on a decent hardware this is usually not a problem. However, if your calculations take too long, they will just disappear into nirvana. The good news: We have easily transferred signals from the time into the frequency domain with FFTW and back again, recalculated FIR filter coefficients and much more without any dedicated hardware, so is not as much as problem as it sounds; the lower the latency you’re aiming at, the more you should be aware of this. There are a couple of websites on the internet that explain why this happens and how to fix it, so I won’t go into detail here.

Well, now you should have a basic overview of how easy it is to write applications interfacing JACK. If this article raised your interest, you may have a further read on the official JACK website.


People keep writing me mails containing the same questions over and over again regarding my work on GSoC 2012, and I decided just to do a blogpost to clarify things. So here we go:

What is the Octave GUI called?

We decided not to give the GUI a separate name. It is just called “GNU Octave” and it will be the default view of using GNU Octave as soon as the developers think it is stable enough. That means that after a certain amount of time has passed (of which I can’t say how long this will be), you will someday install or update GNU Octave on your system and suddenly have a GUI. The command line version will still be available as octave-cli. If you want to test it today, you will have to build it from the sources at the official code repositories. The development version already launches the GUI per default. However, note that it is not recommended for production use; we collectively decided to rather take the time it needs to have a stable release than let the user discover bugs.

Where can I download the Octave GUI?

Well, if you read the above answer carefully, you will already have the answer: From the official code repositories.

Is there Octave GUI for Windows?

We have had Windows builds in the past and as far as I know, John is currently being paid to do a MinGW build for Windows. There have always been problems with building for Windows. Partly, because there weren’t a lot of Windows developers among us which meant almost noone has had the experience or enthiusasm to make a Windows build happen. Michael, for example, was one of the few being able to do it. Unfortunately, he preferred to use Visual Studio to do a Windows build. There are some legal issues with redistributing these builds and also idealistic ones. We would have a hard time to explain why we’re using a proprietary tool to build free software, when there is a free development environment readily available (MinGW). Of course, MinGW is generating native applications just like Visual Studio does.

So that’s why there is not build for Windows yet, although it’s proven that it’s possible. If you are an experienced developer with MinGW and want to have an Octave GUI on Windows, then you can help us making this real. Either visit us on #octave on FreeNode or drop a mail to the GNU Octave mailing lists.