Major Milestone in v3.7 Progress

Earlier this week, we finished converting all of the main blocks over to the new 3.7 style and top-level components. We still haven't fully addressed gr-atsc and gr-shd, but I'm holding off on those two right now on purpose.

The main win here is that you will only see minor changes in which modules you pull blocks in from. Any changes now are bug fixes, but they will be minor and easily corrected. So you can now rebuild your Python and GRC flowgraphs around the 'next' branch, and you should be pretty much good to go.

Note that we've removed an 'blocks' from gnuradio-core. So now, the only things you pull in from the Python gr. module are data structure sizes, the top block, message queue stuff, and similar. If it's a block, it's now located in one of the other blocks. We'll try to post our spreadsheet of where each block is now, but hopefully we've done it so it's pretty intuitive. And if you don't know where something should be, it's probable in gr-blocks (from gnuradio import blocks).

We're really starting to close in on the release of 3.7, so the major churn from our users' perspective should be slowing down.

Configuring ControlPort

We've just pushed some new code to the 'next' branch for better control over ControlPort. Part of this was to make using configuration/preference files more easily and widely used in GNU Radio. While we've had a bunch of configuration files installed, we only really used them for some audio and wxgui parameters. I think we might start using them more heavily in the future for better control over how GNU Radio works, especially now that we're adding a lot of new features that need a finer touch to handle.

First, a word on the preference files. These get installed by default into ${prefix}/etc/gnuradio/conf.d. Each component has its own, but we actually treat it as a flattened list of sections and options (and so we have to be careful that we don't duplicate section and option names). These config files would be system-wide defaults. However, any option can be overwritten for a user by making a [HOME]/.gnuradio/config.conf file. Just duplicate the section name and the option name in this file, and it will take precedence over the system file.

For example, gnuradio-core.conf has a section [ControlPort] and an option "on". This toggles ControlPort on/off. Say by default we want to keep ControlPort off (which is how GNU Radio is installed be default now). So our /opt/etc/gnuradio/conf.d/gnuradio-core.conf file has the "on" option set to "Off" (or False, or 0). But a particular user wants to use ControlPort. So he would create a file /home/me/.gnuradio/config.conf and put in the following:

[ControlPort]
on = True

And then next time GNU Radio is launched, this preference is read and ControlPort is turned on.

As a side note, we can also control any preference with the use of environmental variables. Just take the section name and option name and convert them into all caps with underscores between them. Then append "GR_CONF_" to it and you have your variable name. Set this to whatever you'd like. So our user me above could use "export GR_CONF_CONTROLPORT_ON=True" to manipulate that setting.

But back to the main point here. ControlPort can now be easily configured based on any configuration parameters available through ICE. We have a section:option in gnuradio-core.conf called "ControlPort:config" that points to the ICE configuration file location. I've made a really simple example that you can find installed at ${prefix}/etc/gnuradio/ctrlport.conf.example. This just shows you the format of creating a specific endpoint, like:

ControlPort.Endpoints = tcp -t 300 -h 127.0.0.1 -p 23456

Why would this be important? Well, by default ControlPort doesn't know which interface or ports you want to use in your setup, so it defaults to opening a random port on all possible interfaces. This is good because it makes it easy and is the right behavior when no one knows any better, but it is also bad for two reasons. First, it's a security risk to open a random port on all of your machines network interfaces. Second, you probably want to know which port you are using so that you can connect remotely, possibly through a firewall. Setting specific endpoints allows us to control where ControlPort is exposed and on what port. And, frankly, it's about time.

By the way, most of what I've discussed here is also available in the Doxygen manual that you can build off the code in master/next (and will be made available online with the next release).

Block Core Affinity

I just pushed an update to master/next of GNU Radio that allows us to specifically set the affinity of a block to a particular core. We can do this pretty easily because each block is its own thread in the Thread-Per-Block scheduler, so really what we are doing is setting the thread affinity.

The reason to introduce this behavior is to help us start developing an understanding of GNU Radio on many-core machines. For example, we've been contacted by the Parallella project, and this kind of functionality might be crucial to working well on that platform. On most current systems, we can talk about multi-core, which are a small (up to a dozen or two) number of the same types of processors with shared memory and local cache. The OS scheduler probably does a fantastic job with scheduling the threads in this situations, and kernels like Linux have alternative scheduling strategies we can use for specific purposes. But in a many-core system where we have heterogeneous cores and maybe different access to memory transfers or DMA to IO devices, the ability to selectively choose which cores to run on may be more important. Think about a situation where you want your UHD source block to run on the core that has the closest/fastest access to the GigE system. I'm not saying that this will actually work or help in these scenarios, but it's worth it to make this kind of research and experimentation available.

The API for this is pretty straight-forward, and block affinity can be set before and during the flowgraph run. On a given block, you just call the function 'set_processor_affinity()'.

Notice that the function takes a list (a Python list or a C++ vector of ints), so you can set the affinity to a group of cores if you want. There's an example of how to use this in gnuradio-core/src/examples/mp-sched/affinity_set.py. When you run it, watch your CPU usage monitor(s). This runs two gigantic filters that are designed to take up most of a core, one is set to core 0 and the other is set to either core 0 or 1. As long as at least 1 of the cores specified is present, the system will just use that, so this example will run even on single-core machines (but I have not tested this behavior in Windows). On systems with 2 or more cores, core 0 will be largely taken up by the first filter. The second filter could use core 0, but will likely be scheduled to use core 1.

One thing to note: this is not fully cross-platform. Setting thread affinity seems to be a delicate subject and not fully supported. In fact, the pthread library call pthread_setaffinity_np has that suffix 'np' to specify that it is implicitly non-portable. The GNU Radio Utilities and Etc. Library (GRUEL) has been updated to provide abstracted calls for the different OSes that we support for this. Right now, we have Linux and Windows support. I tried to get OSX support, but their API isn't exactly what we are looking for (they are interested in sharing cache lines, not specifically tying a thread to a core). So until things change, our GRUEL thread affinity calls in OSX are nops.

In the Doxygen manual, look for the Related Pages page named 'Block Thread Affinity' for more details.

Project Updates

As always, we've been hard at work on extending and improving the features in GNU Radio. This is just a quick update on a few of the things we've been working on recently.

First, I have been adding specific pages to the Doxygen Manual that catalogue and describe certain features and functions of GNU Radio. You can find these in the "Related Pages" section. They feature discussions and examples for how to use some of the features in GNU Radio that might not otherwise be obvious. I like this model since it keeps the documentation of the code directly integrated into the code and the build system. You can always have a local copy of the manual for your version. We're also not trying, then, to keep multiple versions of code documentation all over the place.

Today, to make things easier, I expanded our Documentation section on the main GNU Radio webpage to directly link to some of these pages. I'm not sure how many others have found them or are using them already, but I wanted to make sure it was easy to what was available and where to locate the information.

The other thing that I wanted to mention today is the work that's been done on the QTGUI blocks. These are visualization tools for signals in GNU Radio. We've been significantly improving them over the past few months.

The first major improvement has been to add direct user control over the parameters while operating live. The figure below shows a screenshot of the gr-qtgui example *pyqt_freq_c.py*. First, notice the overall color scheme. This is based of the ability to use QT style sheets that can define the look of a QTGUI app. In this running example, I right-clicked outside of the curve canvas itself (where right-click means to unzoom) to pull up the menu. As you can see, the menu allows us to adjust a number of parameters. Some of these are available in all plotters, but others are specific to the type of plot (the FFT size or averaging would be meaningless in a time domain or constellation plot). In this figure, I have adjusted the color of line 'Data 0' to red, increased the width of the other two lines, changed the FFT size to 512, and turned on averaging. I then stopped the graph and tool a screen shot to make this image (notice, though, that there is a 'Save' option in the drop-down menu; this saves the current plot, but it would not have included the menu information).

I would encourage anyone using the QTGUI plotters to play around in these menus to understand the kind of flexibility we now have while looking at signals.

Finally, before this post gets too much longer, I wanted to also note that we have just added a new QTGUI plotter to do time raster plots. These types of plots show samples in time, line by line, so both x and y axes are in time. This is often used for looking at data and packet structure. The GNU Radio plotters can accept multiple signals in that are then plotted as an overlay. The intensity is then shown as the sum of the samples, so it will be higher when then overlap and lower when they are different.

The figure below shows the output of the example pyqt_time_raster_b.py that is shipped with GNU Radio. It has one signal that has 10 zeros, then 80 bits of alternating ones and zeros, and then ended with another 10 zeros. Another signal is then overlaid on top that just has a pattern of 3 one part-way into the vector. These vectors are then repeated. We can see the bright white stripe is where we get the 3 ones from signal 1 on top of the samples in signal 0.

 

Also note the diagonal. We draw these figures on a grid of a number of rows and number of columns. If we had 100 columns in this example, the samples would be plotted in straight columns. But in many situations where these types of plots are useful, we may have some fractional offset we want to handle. This example is showing us that we can set the number of columns to 100.25, so there's a quarter bit overlap between rows that causes this diagonal. Using the drop-down menu, we can adjust the number of columns to 100 to get what we see below.

 

Update on Filter Design Tool

Some quick work between myself and (mostly) Sreeraj helped me realize something that I instantly wanted to do with his new API for using gr_filter_design. I just wanted to publicize it here.

The gr_filter_design tool nicely ties into Python and GNU Radio projects. We launch the filter design tool from the program and can have it return the resulting filter to us in the code to use directly with our filters. Sreeraj also adds a callback function, so that every time the filter design is updated, this callback function is called and it is passed the filter object.

So what did I want to do? I wanted to do a live update of the filter on a live signal. So we built up an example (found in gr-filter/examples/gr_filtdes_live_upd.py) that constructs a simple flowgraph. Two sinusoids are created at different frequencies, some channel noise is added to them, then they are passed through a filter and finally displayed on a QT GUI. The filter starts of with just a single tap of 1, so it's really not doing any filtering.

We then have a function called 'update_filter' that is used as the callback function with the filter design tool. So when you run this program, both the filter design tool and the QT GUI window showing the signal pop up. Whenever we design a new filter, we can immediately see the effects on the real signal that's running through GNU Radio. I've put a screen shot of both windows below.

This image shows the original signal (we've just been storing the maximum value at all FFT points since the start, so we turn on that plot curve) and the new signal through our filter. In this static case, I just designed a band-reject filter around the higher frequency.

Here's another fun experiment that I did. I first created a simple low-pass filter, so the higher frequency signal was rejected but the lower frequency wasn't. I then used the pole-zero plot to move around a couple of the poles onto the unit circle. Setting two of them at the right phase orientation allowed me to put zeros at the frequency of the lower signal. We can see that we've almost cancelled that one out now, too. I used two because the accuracy of getting just a single zero on the correct frequency was too great, so putting two around it allowed me to hit it pretty good with a null in the filter.

Really good fun (if signal processing is within your definition of fun). Also, really instructive and educational. Seeing how these filters behave as we update their parameters and poles and zeros can really help you learn what's going on with the signal processing.