QT GUI Widgets with Message Passing

 Just today, I issued a Pull Request for a new feature in GNU Radio. This adds a new form of what we already called the QT GUI Entry widget. That widget provides a simple QLineEdit box where we type in values, they get stored as a Python variable, and we pass these variables around like normal GRC variables. When running, updates to the entry box are propagated to anything using that variable in the flowgraph.

We're trying to move beyond this world where everything is in a Python file that's assumed to be completely local. Instead, we want to move to a world where we can use the message passing architecture to manage data and settings. So when we have a message interface to a block, we might want to post data or update a value through that message interface from a UI somehow. This leads us to the possibility that the UI is a separate window, application, or even machine from the running flowgraph. We have already made strides in this direction by adding the postMessage interface to ControlPort, which I spoke about in the last post on New Way of Using ContorlPort. However, ControlPort can take some time to set up the application, craft the UI, and make a working product. Aside from that method, we wanted to have easy ways within GRC to build applications that allow us to pass and manage messages easily. Hence the new QTGUI Message Edit Box (qtgui_edit_box_msg).


This is an flowgraph example that now comes with GNU Radio's gr-qtgui (test_qtgui_msg.grc). This shows the use of three of the new message edit boxes. In the upper part of the graph, we have the edit box controlling the center frequency of a Waterfall and Frequency Sink. These sinks can take in a message that's a PMT pair in the form ( "freq" <float frequency> ). So the edit box has a setting called Pair that sets this up to handle the PMT pair messages. It's actually the default since we'll be using the key:value pair concept a lot to manage message control interfaces. When the edit box is updated and we press enter, it publishes a message and the two GUI sinks are subscribed to them, so they get the message, parse it, and update their center frequency values accordingly.

Now the flip side of this is that the edit boxes also have an input message port. This allows us to programmatically set the value inside the box. The two GUI sinks have the ability to have a user adjust their center frequency. When a user double-clicks on a frequency in the plot, the block sets that value as the center frequency and then publishes a message (in the same key:value pair where key="freq"). This means that not only is the widget we just double-clicked updated, anything listening for that message is updated. So the edit box is kept in sync with the change in the GUI. Now, when the new data received is different than what was in that edit box to begin with, the box re-emits that message. So now, say we double-clicked on the Frequency Sink near the 10 kHz frequency line. That message is propagated not only to the edit box, but the message also gets sent from that box through to the waterfall sink. Now all of the widgets are kept in sync. And because the re-posting of the message only happens when a change occurs, we prevent continuous loops. Here's the output after the double-clicking:

Both GUI display widgets have the same center frequency, and that value is also shown in the Frequency edit box above. Because it's using the pair concept, we have two boxes, the left for the key and right for the value.

This example has two other edit boxes that show up at the bottom of the GUI. This is to allow us to easily test other properties and data types. They are connected in a loop, so when one is changed, the other immediately follows. Note here that we are not using the pair concept but that the data type is complex. To make it easy on us, we use a Boost lexical_cast, which means we use the format "(a,b)" for the real and imaginary parts. All of this, by the way, is explained in the blocks'd documentation.

Now, we have the ability to pass messages over ZMQ interfaces. Which means we can create flowgraphs on one side of the ZMQ link that just handle GUI stuff. On the other side, we use ZMQ sources to pass messages around the remotely running flowgraph. Pretty cool and now very easy to do in GRC.

This is just one way of handling QT widgets as message passing blocks. Pretty easy and universal for sending messages with data we're used to. But it's just a start and a good template for other future widgets to have similar capabilities, like with range/sliders, check boxes, combo boxes, etc. All of these should be able to pretty easily use this as a template with different GUI widgets for manipulating the actual data.

QTGUI Tools and Tips

It's been made apparent to me that not everyone knows about all of the capabilities in the QTGUI plotting tools we have in GNU Radio. We've also recently added a number of features that I think people will find really useful and wanted to expose here. I'll focus on just the time and frequency plots, but you'll also find a constellation plot (real and imaginary), a waterfall plot, a histogram, a raster, a generic vector plot, and a number sink.

Drop-down Menus

Each QT sink has a set of control that are specific to controlling each graph, though they share many common attributes. To access the drop-down menu, use the middle mouse button, which in cases of two-button mice might be a ctrl+click or clicking both mouse buttons together. The mouse wheel often acts as the middle mouse button. There's been some call for changing this to the right mouse button, which I'm sympathetic to, but I think we'll want to review all mice interactions to see what makes sense for everything. But for now and through v3.7.7 at least, it's the middle mouse button.

This shows the menu options for the time plot (if you can read it; my resolution is pretty high on my screen). You can set the line properties such as colors, width, markers, and style. Set the number of points to plot, change to log scale for either or both axes, turn it to a stem plot, or set the trigger options.

Some of the common properties to the graphs is the ability to start and stop the plotting, autoscale the y-axis, and save the image to file. The time, frequency, and constellation plots also have the ability to trigger off of the incoming signal. You can set the trigger to free (no triggering), auto (trigger off an event but update the plot after a time anyways), or normal (only update when the trigger hits). For the constellation and time plots, there is even the option to trigger off a specific stream tag. You can then set the level of the trigger event, the channel to trigger off of, and a few other standard triggering attributes. For the time and frequency plot, we now (as of v3.7.7) display a dashed red line to show where the triggering will occur.

There are plenty of options here to look into and play around with once you know that drop-down menu exists.

QSS Files

If you've used the QT GUI plotters before, the above graph might look a bit odd to you. You're probably used to seeing it as a white background with blue and red lines drawn on it. Well, the plotters can actually have most of their attributes controlled by a QT Style Sheet (QSS file). You install them by simply editing your GNU Radio config file. Either edit or create a new $HOME/.gnuradio/config.conf file and add the following:

qss = $prefix/share/gnuradio/themes/alt.qss

Where $prefix is the installation prefix you installed GNU Radio into (often /usr or /usr/local). Below is a look at the normal, non-qss frequency plot versus the alt.qss plot that's installed with GNU Radio.

QTGUI Frequency Plot without a QSS file used.

QTGUI Frequency Plot without a QSS file used.

QTGUI Frequency plot with alt.qss file used.

QTGUI Frequency plot with alt.qss file used.


Control Panel

We just recently added control panels to the time and frequency sinks. We will be continuing to roll out this concept to the other plots as well, but these were the first two to get the attention. The control panel is a set of widgets on the right-hand side of the graph to provide very quick and easy access to manipulating the look of many of the plot properties. You can see in the image below that we can adjust things like the x and y axis limits, do an autoscale, toggle things like min/max hold or the grid, and adjust the triggers. For the time plots, the autoscal check box turns on continuous autoscaling of the plot based on the latest set of samples while the button just does a one-shot autoscale and then holds the y-axis from there.

We can also toggle this on and off. One of the reasons I did not have this in the first place was that it takes up a lot of plotting real estate on the screen. However, when manipulating the plots, it is definitely much easier to use these tools than the drop-down menu for many of the purposes -- setting the trigger is a good example. Still, we can actually enable and disable the control panels as we need them. We can do this in GRC by going into the properties box for the plotters and setting the Control Panel option in the Config tab to Yes or No. At runtime, the drop-down menu has the option of enabling or disabling the control panel as well, so you can use it and hide as you need.


Another brand new feature is the ability to turn the plotter legends off. Tim O'Shea requested this feature, and it really seemed like a good idea. The lines in a plot are labeled, and those labels show up in a legend on the right-hand side. When plotting a lot of lines together, this can be really useful to distinguish them and present the results to others. See my recent post on the new LDPC and TPC codes and the BER plot. There, we use the legend to show which line is for which FEC technique. However, often we just plot a single line, in which case the legend just takes up an unfortunate amount of space. So now, in the GRC properties box under the Config tab for each QTGUI plotter, we can toggle the legend on or off. This cannot be done at runtime, however. Below is an image of two QTGUI time plotters showing the same signals. The one on the right has the legend on and the other has it off. This shows how much less of the display is wasted by a legend that, in this instance, doesn't tell us much about what's being plotted.

The only plotter that works differently than the rest The waterfall plot just removes the intensity legend on the right-hand side. You won't be able to relate the colors to specific power levels, but often we just need to get a glimpse of the signals and have an understanding of the relative power levels, not the exact numbers.

New Forward Error Correction Codes

Thanks to the work of the last GSoC and the researchers at Virginia Tech, we now have an implementation of LDPC and turbo product code (TPC). These classes of codes have a number of possibilities for their setup and runtime behavior. LDPC in particular has a pretty good selection of techniques to compute them, and we're working on adding at least a second way of doing things -- again, coming from last year's GSoC projects.

GNU Radio contains a BER curve generator called ber_curve_gen.grc and installed as a gr-fec example. We can easily add new FEC codes to this test, the output of which is plotted below, truncated after some time so the curves haven't all been smoothed out, yet.

First, we're plotting BER vs Es/N0 and not Eb/N0, so this is showing performances at different code rates. The TPC and LDPC codes are using the default parameters, which could be improved upon. In fact, there is still probably lots to learn about all of this. In this case, the LDPC was using the default alist that comes with GNU Radio (found in gr-fec examples directory) and the only change from the default TPC encoder and decoder was to use the MAX LOG-MAP decoder.

To see the differences in just the TPC decoder modes available, I plotted all five against each other. These are the same codes and rates with just different approaches to decoding them. What was not analyzed here, however, is the computational cost of each, and I would suspect we'd see a direct trade-off between BER performance and required compute power.

And if you haven't seen the new FEC-API, this is a way to easily use encoders and decoders of varying styles and requirements in both streaming and bursty communications. It makes using and reusing FEC blocks very easy between GNU Radio applications as well as allows us to easily use and replace the FEC being used in any given application. GNU Radio installs many examples for how to use the encoders and decoders in streaming, tagged stream, and PDU message modes into the digital examples directory.

Finally, these codes are not greatly optimized for speed. We felt it was important to provide access to the codes now so they can start being utilized in real systems. Hopefully, this is intriguing enough that one or more serious coders takes this as an opportunity to improve things.

Working with GRC Busports

Busports are a fairly new addition to the GNU Radio Companion (GRC). They allow us to group block ports together to make connecting many ports together much easier. While most flowgraphs we work with don't require this, we are moving to more complex structures that require handling many channels of data, which can get graphically tricky in GRC. Enter busports.

This post walks through two setups using busports to help explain how to work with them and a few things to look out for. Also, some of these concepts are brand new to GNU Radio and will not be made available until version 3.7.3.

Connecting Null Sources and Sinks

Many of the cases we'll come to involve the need to sink a number of channels or outputs to null sinks so that we can ignore those while focusing on the important ones. Previously, we would have to have dropped many null sinks into the flowgraph and connect each line individually. Well, I have now outfitted the null sinks with the ability to sink multiple data streams and to be able to control the bus ports connections in GRC.

By default, if we have a block with multiple ports on one side or the other, we can toggle busports that group all ports into a single bus (right-click on the block and use either "Toggle Source Bus" or "Toggle Sink Bus" for whichever version is right for the block). For example, if our null sink has three sink ports, we toggle the sink bus on, which looks like this:

However, for the null_sink and null_source blocks, I have instrumented the ability to selectively break up the bus ports to arbitrary busses. Let's take the example of a source block that has 10 source ports with 4 output groupings: Ports 0-2, 3-5, 6-7, and 8-9. We handle these groupings by specifying the "Bus Connection" parameter of the null source block.

The Bus Connections parameter is a vector of vectors. Underneath, it is translated using the XML tag "<bus_structure_source> that I put into the block's XML file. Again, it is a list of lists. Each element of the outer list allows us to specify which ports are connected to that one source port. The internal lists are the list of ports to connect. Given our specification above for our 4 groupings of the 10 ports, we would use:

Bus Connections: [[0, 1, 2], [3, 4, 5], [6, 7], [8, 9]]

Now, when we toggle Toggle Source Bus on for this block, it will have 4 bus ports.

Let's now connect three null sinks to these four ports. The first two sinks will each connect to one bus port and the third null sink will sink the last two bus ports. For the first two null sinks, we only have to specify the number of input ports and the bus connections is simply "[[0,1,2],]" or alternatively "[range(3),]". The third null sink takes in 4 connections  in 2 busports, so the bus connections parameter is slightly more complicated as "[[0,1], [2,3]]". This creates two input ports that each take 2 connections. We then toggle the sink bus for each of the null sinks and create a graph that looks like this:

Obviously, this flowgraph doesn't do anything really interesting, but I think it useful to understand how to work with busport connections. Notice the numbering tells us which bus port it is and how many internal connections each bus has. When connecting busports together, GRC will check the cardinality and only connect ports that have the same number of connections. So we couldn't, for instance, connect Bus0 of the source to bus0 of the third null sink.

WARNING: There is a bug in GRC that can really screw up a flowgraph if we try and change the bus connections parameter when a connection already exists. Until we fix this, I would highly recommend that you disconnect the bus connection before making any modifications to the number of ports or the busport connections. If you forget and your canvas all of a sudden goes blank, DO NOT SAVE and instead just close GRC and reopen it.

Grab the flowgraph example here (remember, this will require GNU Radio 3.7.3 or later to run).

Using Busports with a Channelizer

The null sinks and sources are instructive but don't actually do anything. So I've made a more complex example that channelizes five sine waves of different frequencies. The flowgraph looks like this:

The signal generators from top to bottom generate sine waves with frequencies:

  • 1 kHz
  • 22 kHz
  • 44 kHz
  • -23 kHz
  • -45 kHz

These are added together with a sample rate of 100 kHz (so we have a spectrum from -50 to +50 kHz). Since we're splitting this spectrum into 5 equal sections, we have the following channels: 

  • Channel 0: -10 to 10 kHz
  • Channel 1: 10 to 30 kHz
  • Channel 2: 30 to 50 kHz
  • Channel 3: -50 to -30 kHz
  • Channel 4: -30 to -10 kHz

What that means is that when we channelize them, the signals in these bandwidths are moved to baseband. So we get output signals at 1, 2, 4, -5, and -3 kHz on the output channels.

The flowgraph shows us using two channelizers. The first one on top sends all five channels to a single frequency sink to display all the baseband channels together. We use busports to keep the connection in the GRC canvas clean. The second channelizers says we only care about 3 of the 5 channels, so we'll split the busports output into 2 and send channels 0, 2, and 4 to the plotter and channels 1 and 3 to a null sink to ignore them. The busports connection for this channelizer looks like:

Bus Connections: [[0,2,4], [1,3]]

So in the output of the first channelizers, we'll see a single sine wave at the already specified frequencies on each channel. The output of the second channelizer will only show three signals with frequencies 1, 4, and -3 kHz. In the following figure showing the output, the top display is the input to the channelizer, the bottom left is the first channelizer with all 5 channels connected, and the bottom right is the second channelizer with just the 3 connections.

You can get the example script here.


I hope this gives some better ideas how to work with the new busports features in GRC. I didn't really want to go overboard with a huge number of connections just to show them off, but these examples should give you some understanding about where we would want to use busports in the future. And again, be careful updating the bus connections or number of ports when the connections already exist.

Volk Integration to GNU Radio

Getting Volk into GNU Radio

We've been talking about integrating Volk into GNU Radio for what seems like forever. So what took us so damn long? Well, it's coming, very shortly, and I wanted to take a moment to discuss both the issues of Volk in GNU Radio and how to make use of it with some brand-new additions.

The main problem with using Volk in GNU Radio is the alignment requirements of most SIMD systems. In many SIMD architectures, Intel most notably (and we'll stick with them in these examples as it's what I'm most familiar with), we have a byte-alignment requirement when loading data into the SIMD-specific registers. When moving data in and out, there is a concept of aligned and unaligned loads and stores. You take a hit when using the unaligned versions, though, and they are not desirable. In SSE code, we need to by 16-byte aligned while the newer AVX architecture wants a 32-byte alignment.

But we have the dynamic scheduler in GNU Radio that moves data between blocks in chunks of items (where an item is whatever you want: floats, complex floats, samples, etc.). The scheduler tries to maximize system throughput by moving as large a chunk as possible to give the work function lots of data to crunch at once. Larger chunks minimize the overhead of going into the scheduler to get more data. But because we are never sure how much data any one block has ready for the next in the chain of GNU Radio blocks, we cannot always guarantee the number of items available, and so we cannot guarantee a specific byte alignment of our data streams.

We have one thing going for us, though: all buffers are page-aligned at the start. This is great since a page alignment is good enough for any current or foreseeable SIMD alignment requirement (16 or 32 bytes right now, and when we get to the problem of requiring more than 4k alignments, I'll be happy enough to readdress the problem then). So the first call to work on a buffer is always aligned. 

But what if the work function is called with a number of items that breaks the alignment? What are we supposed to do then?

The first attempt at a solution was to use the concept of setting a set_output_multiple value for the block. This call tells the scheduler that the block can only handle chunks of data that contain a number of items that is a multiple of this value. So if we have floats in SSE chips, we need a multiple of 4 floats per call to the work function. It will never be called with less than 4 or some odd number that will ruin our alignment. 

But there's a problem with that approach. The scheduler doesn't really function well when given that restriction. Specifically, there are two issues. First, if the data stream being processed is finite and that number is not a multiple of what's required by the alignment, then the last number of items won't ever be processed. That's not the biggest deal in the world as GNU Radio is typically meant to stream data, but it could be a problem for many applications of processing data from a file.

The second problem, though, is latency. When processing packetized data, we cannot produce a packet until we have enough samples to make the packet. But at some point, we have the last few samples sitting in the buffer waiting to be processed. Because of our output multiple restriction, we leave those sitting behind until more samples are available so that the scheduler can pass them on. That would mean a fairly large amount of added latency to handle a packet, and that's unacceptable.

No, what we need is a solution that keeps the data flowing as best as it can while still working towards keeping the buffers aligned.

Branch Location

This post discusses issues that will hopefully be merged into the main source code of GNU Radio soon. However, I would like it to undergo significant testing, first, and so have only published a branch at:

as the branch safe_align.

Scheduler Alignment

Instead of using the set_output_multiple approach, we need a solution that meets the following goals:

  • Minimize effect to latency; maximize throughput.
  • Try to maintain alignment of buffers whenever possible.
  • When not possible to keep alignment, pass on data quickly.
    • minimize latency accrued by holding data.
  • Re-establish alignment but not at the expense of extra calls.
    • pass on the largest buffer possible that re-establishes alignment.
    • don't pass the minimum required. The extra overhead of calling a purposefully-truncated work function is greater than the benefit of realigning quickly.

In it's implementation, we want to minimize any added computation to the scheduler and slow down our code.

In the approach that we came up with, the scheduler looks at the number of items it has available for the block. If there are enough items to keep the buffers aligned, it passes on the largest number of samples possible that maintains the alignment. If there aren't enough, then it sends them along anyway, but it sets a flag that tells the block of the alignment problem.

When the buffers are misaligned, the scheduler must try to correct the alignment. There are two ways of doing this. The easiest way is just to pass on the minimum number of items possible that re-establishes alignment. The problem with this approach is that the number is really small, so you are asking the work function to handle 1, 2, or 3 items, say. Then it has to go back to the scheduler and ask for more. This kind of behavior incurs a tremendous amount of overhead in that it deals more with moving the data than processing it.

The second way of handling the unalignment is to take the amount of data currently available and pass on the largest possible chunk of items that will re-establish the alignment. This forces us to handle another call to work with unaligned data, but the penalty for doing that is much less than the overhead of purposefully handling small buffers. In my experiments and analysis, most of the data comes across aligned, anyway, so these calls are minimal.

To accomplish these new rules, the GNU Radio gr_block class (which is a parent class to all blocks) has these new functions:

  void set_alignment (int multiple);

  int  alignment () const { return d_output_multiple; }

  void set_unaligned (int na);

  int unaligned () const { return d_unaligned; }

  void set_is_unaligned (bool u);

  bool is_unaligned () const { return d_is_unaligned; }

A block's work function can check it's alignment and make the proper decision on what to do based on that information. The block can test the is_unaligned() and call. If it indicates that the buffers are aligned, than the aligned Volk kernel can be called. Otherwise, it can either process the data directly or call an unaligned kernel.

In order not to make this blog post longer than it already is, I will post a separate blog post discussing the method and results of benchmarking all of this work. In it, just to tease, I'll be showing a few surprising results. First, I'll show that the use of Volk can give us dramatic improvements for a lot of simple blocks (ok, that's not surprising). Second, on the tested processors, I see almost no penalty for making unaligned loads and stores. And third, lest you think that last claim makes all of this work unnecessary, my test show that the efforts to keep the alignment going in the new scheduler actually improves the processing speed even without using Volk. So there is a two-fold benefit to this work: one from the scheduler itself and then a second effect of Volk. 

Making Unaligned Kernels

Because we will be processing unaligned buffers in this approach, we need to either handle these cases with generic implementations or use an unaligned kernel. The generic version of the code would be like what is already in a block now that we would like to transition to using Volk. This would be the standard C/C++ for-loop math.

A useful approach, though, is to make use of unaligned Volk kernel. Even though an unaligned load is a bit more costly than an aligned call, we try to maximize the size of the buffers to process and the overall affect is still faster than a generic for loop. So it behooves us to call the unaligned version in these cases, which might mean making a new kernel specifically for this. 

Luckily, in most cases, the only difference between an aligned Volk kernel and an unaligned one is the use of loadu instead of load and storeu instead of store. These two simple differences makes it really easy to create an unaligned kernel.

With this approach, a GNU Radio block can look really simple. Let's use the gr_multiply_cc block as an example. Here's the old version of the call:

gr_multiply_cc::work (int noutput_items,
  gr_vector_const_void_star &input_items,
  gr_vector_void_star &output_items)
  gr_complex *optr = (gr_complex *) output_items[0];
  int ninputs = input_items.size ();
  for (size_t i = 0; i < noutput_items*d_vlen; i++){
    gr_complex acc = ((gr_complex *) input_items[0])[i];
    for (int j = 1; j < ninputs; j++)
      acc *= ((gr_complex *) input_items[j])[i];
    *optr++ = (gr_complex) acc;
  return noutput_items;

That version uses a for-loop over both he number of inputs and number of items. Here's what it looks like when we call Volk.

gr_multiply_cc::work (int noutput_items,
     gr_vector_const_void_star &input_items,
     gr_vector_void_star &output_items)
  gr_complex *out = (gr_complex *) output_items[0];
  int noi = d_vlen*noutput_items;
  memcpy(out, input_items[0], noi*sizeof(gr_complex));
  if(is_unaligned()) {
    for(size_t i = 1; i < input_items.size(); i++)
      volk_32fc_x2_multiply_32fc_u(out, out, (gr_complex*)input_items[i], noi);
  else {
    for(size_t i = 1; i < input_items.size(); i++)
      volk_32fc_x2_multiply_32fc_a(out, out, (gr_complex*)input_items[i], noi);

  return noutput_items;

Here, we only loop over each input, but the calls themselves are to the Volk multiply complex kernel. We test the unaligned flag first. If the buffers are flagged as unaligned, we use the volk_32fc_x2_multiply_32fc_u kernel where the final "u" indicates that this is an unaligned kernel. So for each input stream, we process the data this way. In particular, this kernel only takes in two streams at once to multiply together, so we take the output and multiply it by the next input stream after having first pre-loaded the output buffer with the first input stream.

Now, if the block's buffers are aligned, the flag will indicate as much and the aligned version of the kernel is called. Notice that the only difference between the kernels is the "a" at the end instead of the "u" to indicate that this is an aligned kernel.

If we didn't have an unaligned kernel available, we could either create one or just call the old version of the gr_multiply_cc's work function in this case.

Blocks Converted so Far

These next few sections are starting to get really low-level and specific, so feel free to stop reading unless you are really interested in the development work. I include this as much for the historical reference as anything.

Most of these blocks that I have so far moved over to using Volk fall into the category of the "low-hanging fruit." That means that, mostly, the Volk kernels existed or were easy to create from existing Volk kernels (such as making unaligned versions of them), that the block only needed a single Volk kernel to perform the activity required, and that had very straight-forward input to output relationships.

On occasion, I went and added a few things that I thought were useful. The char->short and short->char type conversions did not exist, but they were already a Volk kernel, so making them a GNU Radio block was easy and, hopefully, useful.

I also added a gr_multiply_conjugate_cc block. This one made a lot of sense to me. First, it was really easy to add the two lines it took to convert the Volk kernel that did a complex multiply into the conjugate and multiply kernel that's there now. Since this is such an often-used function in DSP, it just seemed to make sense to have a block that did it. My benchmarking shows a notable improvement in speed by combining this operation into a single block, too. Just to note, this block takes in two (and only two) inputs where the second stream is the one that gets conjugated.

What follows is a list of blocks o different types convered to using Volk

Type conversion blocks

  • gnuradio-core/src/lib/general/gr_char_to_float
  • gnuradio-core/src/lib/general/gr_char_to_short
  • gnuradio-core/src/lib/general/gr_complex_to_xxx
  • gnuradio-core/src/lib/general/gr_float_to_char
  • gnuradio-core/src/lib/general/gr_float_to_int
  • gnuradio-core/src/lib/general/gr_float_to_short
  • gnuradio-core/src/lib/general/gr_int_to_float
  • gnuradio-core/src/lib/general/gr_short_to_char
  • gnuradio-core/src/lib/general/gr_short_to_float

Filtering blocks

  • gnuradio-core/src/lib/filter/gr_fft_filter_ccc
  • gnuradio-core/src/lib/filter/gr_fft_filter_fff
  • gnuradio-core/src/lib/filter/gri_fft_filter_ccc_generic
  • gnuradio-core/src/lib/filter/gri_fft_filter_fff_generic

General math blocks

  • gnuradio-core/src/lib/general/gr_add_ff
  • gnuradio-core/src/lib/general/gr_conjugate_cc
  • gnuradio-core/src/lib/general/gr_multiply_cc
  • gnuradio-core/src/lib/general/gr_multiply_conjugate_cc
  • gnuradio-core/src/lib/general/gr_multiply_const_cc
  • gnuradio-core/src/lib/general/gr_multiply_conjugate_cc
  • gnuradio-core/src/lib/general/gr_multiply_const_cc
  • gnuradio-core/src/lib/general/gr_multiply_const_ff
  • gnuradio-core/src/lib/general/gr_multiply_ff

Gengen to General

One thing that might confuse people who have previously developed in the guts of GNU Radio is how I moved some of the blocks from gengen to general. Many GNU Radio blocks perform some function, like basic mathematical operations on two or more streams, that behave identically from a code standpoint but which use different data types. These have been put into the gengen directory as templated files where a Python script is used to autogenerate the type-specific class. This was before Swig would properly handle actual C++ templates, so we were left doing it this way.

Well, with Volk, we don't really have the option to template classessince the Volk call is highly specific to the data type used. So when moving certain math block for a specific type out of gengen, we went with the simple solution of removing that data type from the autogeneration scripts and placing it into general as a stand-alone block that can call the right Volk function. Good examples are the gr_multiply_cc and gr_multiply_ff blocks to see what I mean.

This really seems like the simplest possible answer to the problem. It maintains our block structure that we've been using for almost a decade now and keeps things clean and simple for both developers and users. The downside is some duplication of code, but with the Volk C functions, that is somewhat inevitable and not a huge issue to deal with.

SNR Estimators

In GNU Radio, we have been slowly evolving our digital communications capabilities. One thing that becomes quickly and painfully obvious to anyone doing real over-the-air communications with digital modulations is that the modulators and demodulators are the easy part. It's the synchronization that's the hard part and where most of your work as a designer goes.

Generally speaking, synchronization means three (maybe four) things: frequency, timing, and phase. The fourth is automatic gain control (AGC). While AGC isn't really "synchronization," it follows similar principles of an adaptive loop. Different modulation schemes have different methods for AGC and synchronization. These tend to fall into categories of narrowband, wideband (and then ultrawideband), and OFDM, but we can easily dispense with these categories and go in depth into differences within them, too. For instance, narrowband PSK and FSK systems have pretty widely different requirements out of a receiver for demodulation and the appropriate synchronization algorithms reflect this.

But this post is about SNR estimation. The reason to talk about synchronization here it twofold. First, like synchronizers, SNR estimation techniques can vary widely depending on the modulation scheme being used. Second, most synchronization schemes like to be reactive to changes in SNR. You can often find different algorithms that work well in low SNR cases but not high, or they are too costly to perform in high SNR where the signal quality allows you to use something simpler and/or more accurate. Take for example an equalizer. The constant modulus equalizer is great for PSK signals, but we know that an LMS decision-directed equalizer works better. But a decision-directed equalizer only works in cases where the SNR is large enough that the majority of samples are correct. So we often start with a blind CMA for acquisition purposes and then move to a decision-directed approach once we've properly locked on to the signal.

We've been meaning to add SNR estimators to GNU Radio for some time, now, and I finally developed enough of an itch to start doing it. But as I said, each modulation could use a different equalizer, and there are various equalizers designed for this modulation in that channel or that modulation in such and such channel. If you have access to IEEExplore, a simple search for "snr estimator" produces 1,279 results. Now, I know that's nothing to a Google search, but twelve hundred scholarly (we hope) articles on a topic is a lot to get through, and you will quickly see that there are finely-tuned estimators for different modulations and channels.

What I did was give us a start into this area. I took a handful of the most generic, computationally realistic estimators that I could find for PSK signals and implemented them. I'll give a shout out here to Normon Beaulieu, who has written a lot in this field. I've found that a lot of his work in SNR estimators to be accessible and useful, and he presents his work in ways that can be easily translated into actual, working code.

I also took the tack of looking for estimators that worked in AWGN channels. Now, if you've ever heard me speak on the subject of communications education or research, you've probably heard me scoff at anyone who develops a system under simulated AWGN conditions. In the case of an SNR estimator, though, I thought about this and had to come to the conclusion that the only way to handle this is to have an estimator that you can plug in variables for your channel model, which of course assumes that you have or can estimate these parameters. So in the end, I followed Beaulieu's lead in at least one of this papers and took algorithms that could be both simplified and tested by assuming AWGN conditions. I did, however, provide one of these algorithms (what I refer to as the M2M4 algorithm) in a way that allows a user to specify parameters to better fit a non-AWGN channel and non-PSK signals. Using the AWGN-based algorithms with this other version of the M2M4 seemed like a good compromise for being computable without more information but at least providing a tip of my hat to the issue of non-AWGN channels. If nothing else, these estimators should give us a ballpark estimate.

I also specifically developed these estimators based on a parent class that would easily allow us to add more estimators as they are developed. Right now, the parent class is specifically for MPSK, but we can add other estimator parent classes for other modulations; maybe have them all inherit from a single class in the end -- this is fine, since the inheritance would really be hidden from the end user. The class itself is in the gr-digital component and called digital_impl_mpsk_snr_est. It's constructor is simply:

digital_impl_mpsk_snr_est(double alpha);

Where the parameter alpha is the value used in a running average as every estimator I've seen is based on expected values of a time series, which we estimate with the running average. This value defaults to 0.001 and should be kept small.

I have created four estimators that inherit from this block. These are named the "simple," "skew," "M2M4," and "SVR" estimators. The last two come from [1]. The "skew" estimator uses a skewness measure that was developed in conversation with fred harris. The simple estimator is probably written up and documented somewhere, but it's the typical measurement based on the mean and variance of the constellation cloud.  I've tried to document these as best as possible in the header files themselves, so I'll refer you to the Doxygen documentation for details (note that as of the writing of this blog post, these estimators are only in the Git repository but will be available starting in the 3.5.1 release). The "SNR estimators"  group in the Doxygen manual can be used to find all of the available estimators and details about how to use them.

In particular, the M2M4 and SVR methods were developed around fading channels and both use the kurtosis of the modulation signal (k_a) and kurtosis of the channel (k_w) in their calculations. This is great if these values are know or can be estimated. In the case of the M2M4 algorithm, I provide a version of it, called the digital_impl_snr_est_m2m4, as an example of a non-PSK and non-AWGN method; right now, this block is unavailable through any actual GNU Radio block. It's untested and unverified, but I wanted it there for reference and hopefully to use later.


SNR Use in Demodulation

The main intent of having an SNR estimator block is to enable the use of SNR information by other blocks. As such, there are two GNU Radio blocks that are defined for doing this in different ways. First off, let's say that the SNR estimation is done after timing recovery (see harris' paper "Let’s Assume the System is Synchronized", which is unfortunately fairly costly but worth it if you can get a copy). So in GNU Radio terms, this means that the SNR is estimated down stream of most of the synchronization blocks. While I would like to just pass a tag along with the SNR information, that does won't work for every block, like the frequency recovery, AGC, and timing recovery loops that come before. Instead, we will have to pass them a message with this information. However, some blocks exist downstream that want this info, too, like the channel equalizer.

To accommodate both possible uses, I created two blocks. The digital_mpsk_snr_est_cc block is a flowgraph with a single input and single output port, so it's meant to go inline in a flowgraph. This block produces tags with the SNR every N samples, where N is set by the user (the second arg in the constructor and through the set_tag_nsamples(int N) function). The downstream blocks can then look for the tag with the key "snr" and pull out this information when they need it. The value of N defaults to 10,000 as an arbitrary, fairly large number. You'll want to set this depending on the speed you expect the SNR conditions to change.

The second block is a probe, which is GR speak for a sink. It's called digital_probe_mpsk_snr_est_c and only takes a single complex input stream. Right now, it just acts as a sink where the application running the flow graph can query the SNR by calling the "snr()" function on this block (the same is true for the digital_mpsk_snr_est_cc block, too). However, this block uses a similar constructor in that you set a value N for the number of samples between messages. In this case, instead of sending a tag down stream, it will send a message every N samples. The problem with this is that our message passing system isn't really advanced or easy enough to use to set this up properly. Recent work by Josh Blum might fix this, though.

Eventually, though, we hope to be able to create flow graphs where the SNR estimation is passed around to other blocks to allow them to adjust their behavior. In the case I'm interested in right now, I'd like to pass this info to the frequency lock loop to stop it if the SNR falls below a certain level so that it doesn't walk away when there is no signal to acquire.


[1]  D. R. Pauluzzi and N. C. Beaulieu, "A comparison of SNR estimation techniques for the AWGN channel," IEEE Trans. Communications, Vol. 48, No. 10, pp. 1681-1691, 2000.