Monday, August 31, 2015

A PIC-based audio source for locking a VHF oscillator

Several years ago I decided to build a weather satellite receiver from scratch.  It is described here - link.

I didn't really need a weather satellite receiver, and I could have easily bought a kit somewhere else - or bought a second-hand one via EvilBay, but I just wanted to go through the exercise of throwing everything together and making it work using parts on hand - and I wanted to try out some ideas.

Locking a VCO to an audio DDS reference:
Figure 1:
The front panel of the VHF weather satellite receiver.
This receiver has been in continuous operation for several years, working
flawlessly in that time.
Click on the image for a larger version.

One of these ideas was to use a PIC to lock the VHF local oscillator.  On the face of it, this isn't unique - except that the PIC was to be the sole source of the precise frequency to which the PLL (Phase Locked Loop) for the local oscillator:  No divide-by-N chips here!

For this receiver the local oscillator operated 10.7 MHz below the receive frequency nominally at about 126 MHz.  Since I was already using a 100 MHz oscillator (a VCXO) that I'd pulled from some scrapped commercial satellite gear, I used a simple 3-transistor mixer/amplifier circuit to convert this to about 26 MHz (137 MHz - 10.7 MHz - 100 MHz) and this allowed me to use a 74HC4040 12-stage binary ripple counter to bring a representation of the local oscillator down to the audio range - about 6.3 kHz.

As it so-happened, I'd chosen the 100 MHz oscillator on purpose - mostly because it was free, but it also provided a nice, stable 20 MHz clock for the PIC by dividing its output by 5 using a 74F191 so both the down-conversion and the PIC's clock were referenced from the same source.

The goal was to provide a minimum tuning step size of at least 1 kHz, and because I'd already divided-by-4096, this meant that my audio-frequency step size was on the order of 1/4 of one Hz - but that was no problem since I was going to use DDS techniques in the PIC.

The DDS:

A DDS (Direct Digital Synthesis - see the Wikipedia article about DDS techniques here - link) circuit is fairly simple in operation:  Typically, one takes a register (called an "accumulator") and on every clock cycle you add to it a constant number (we'll call it a "frequency word") allowing it to "wrap around" once the accumulator's capacity is exceeded or, in other words, you do unsigned binary addition without overflow.

If you were to keep track of how often the accumulator overflows you'd notice that if you added a smaller number to it, it would overflow less often which makes sense since it would take more clock cycles to overflow!  What you might notice is that one can easily predict the rate at which it will overflow:

( (frequency word) / (maximum accumulator value) ) * clock frequency

Typically, the "maximum accumulator value" is the maximum number (plus one) that can be represented by the number of bits used by the accumulator (e.g. 8 bits = 256, 16 bits = 65536, 32 bits = 4294967296).

In my case, it was easy to make the PIC do 32 bit unsigned addition.

Figure 2:
The PIC controller board that generates the precise audio frequency
based on a PIC16F88 and driven from a 20 MHz clock source.  This PIC
also drives the LCD and does the serial data communications,
receiving frequency tuning commands from the host computer.
Click on the image for a larger version.

The PIC that I used (a PIC16F88) can be clocked to 20 MHz and among other things it contains a PWM generator that can operate as a simple D/A (Digital-to-Analog) converter with as much as 10 bits of resolution.  As such, it has a 10-bit timer and with the PWM operating at (up to) 10 bits of resolution it will sample at up to 1/1024th of the clock frequency, or:

20 MHz / 1024 = 19.53125 kHz

Since we have 232 (4 billion+) counts in our 32 bit accumulator, and we clock it at as high as 19.53125 kHz, that means that our frequency resolution is about one four-billionth of 19.53125, or:

19.53125 kHz / (232) = 0.000004547 Hz - or about one five-millionths of one Hertz resolution!

There's one more step in generating a useful frequency output.  If one watches the MSB (most-significant bit) of the accumulator, we can see that it flips between 0 and 1 at the desired frequency, but we don't want a digital output:  Even if we did take the MSB which is, on average, at the desired frequency, it typically has a lot of phase jitter that makes it unsuitable for most frequency control purposes.

If, instead, we take the top several bits of the accumulator and feed them to a lookup table that has a sine wave and then outputting that value to a D/A converter, we get a more analog-looking signal with much less phase jitter:  The more bits we have, the better job we can do in representing a sine wave.

Now, remember that we divided our mixed-down local oscillator by 4096, so this means that our effective resolution would be reduced by that much, but if you do the math, that still means that we have - when multiplied by 4096 - a step size of 0.0186 Hz or so!

If you've been following along, you might noticed that I skipped several steps, so let me explain:

The idea was to divide down a representation of the 126 MHz local oscillator to audio and we did this by first subtracting 100 from it and then dividing-down the 26 MHz by 4096 to audio.  We would then generate a precise audio frequency at one-4096th of that 26 MHz frequency and using a PLL, lock our local oscillator to it!

Simple - almost.

The DDS technique is imperfect when implemented using hardware that doesn't have infinite resolution - and the PIC's hardware and software capabilities are rather limited - in my case, I managed to implement the equivalent of a 1 ksample sine wave with 10 bits of resolution.  (Actually, it was just 1/4th of a sine wave - which is enough if you flip the pieces upside-down and/or play it backwards in the right order as needed!)

So now I had precision audio generator that could output a reasonable facsimile of a sine wave at any frequency from about 5 milliHertz (including DC, if you want to be pedantic) to something less than 1/2 of the sample rate - about 9 kHz!  The PWM output from the PIC is really a bunch of samples of a 19 kHz variable duty-cycle digital waveform and it needed to be filtered a bit so I ran it through a simple op-amp bandpass filter - and then converted it back into a square wave - before passing it on to the a 4046 chip and into the edge-triggered phase detector.  In the 4046 this was compared with the converted/divided signal from my local oscillator and with the magic of the PLL, my VHF oscillator was nicely locked to the precise audio frequency from the PIC!


At this point, the imperfection of the DDS became apparent.

One of the satellite frequencies is 137.62 MHz with a local oscillator frequency of 26.92 MHz.  When this was divided by 4096, this yielded a frequency of 6.5723 kHz approximately.

If one takes a close look at the spectrum produced by any DDS-type synthesizer, a myriad of low-level (and some not-so-low-level) spurious signals will be generated because of rounding-off errors related to the finite resolution of the D/A converter, the size of the sine table, and the relationship between the desired frequency and the clock frequency.  As one approaches frequencies that are related to an integer sub-multiple of the higher order bits (e.g. multiples of 1/2, n/4, n/8, n/16, n/32, n/64, etc. of the clock frequency) these low-level spurs get closer and closer to those multiples mentioned above.  As these sub-multiples get "smaller", the amplitude of these spurious components decrease as well.

In the case of the 6.5723 kHz signal required to synthesize 137.62 MHz frequency, this was very close to 43/128ths of the clock frequency - or about 10.986 Hz off.  What this caused was a very low-level 11-ish Hz modulation of the generated frequency which, when effectively multiplied upwards by the 4096 division - which increased the apparent loop gain - appeared as a very obvious tone (more of a buzz, actually!) at the local oscillator frequency.

Normally, loop filtering would take care of this, but this rather low frequency (just 11 Hz!) could get through the filter too well - and further-slowing of the loop filter wasn't particularly attractive - but this is software and we can do sleight-of-hand to fix this!  What I did was to pick a slightly different clock frequency - 20 MHz / 896 = 22.32142857... kHz instead and this moved the spurious signals from the DDS far enough away that they were effectively removed by the loop filtering.

The end result was a VCO that would tune anywhere within the designed range in less than a second and have very low-level spurious signal content from the DDS!

Locking a VCXO to an audio DDS reference:

As it turns out, locking a VCO - essentially a free-running oscillator with an implied, wide tuning range - is a comparatively "worst-case" scenario when it comes to the minimization of things like "reference sidebands" owing to the very high loop gain involved which can arise from both the "tuning sensitivity" of the VCO itself and if high divisor ratios are used.  If one starts out with an oscillator with a comparatively narrow tuning range - such as a VCXO (Voltage Controlled Crystal Oscillator) - in which the tuning sensitivity can be orders of magnitude smaller - and the lock time may be longer, particularly if high divisor ratios are used since it may take some time for the phase fo the comparison signal to "slide" into alignment - it is much easier to keep those already low-level spurious signals down to levels that may be ignored in typical applications.

A practical implementation of this technique has been employed in the W7SP Synchronous/Voting repeater system operated by the Utah Amateur Radio Club (described here - link) in which the transmit frequency is referenced from 10 MHz OCXOs (Oven Controlled Crystal Oscillators) and held within 1-2 Hz of the intended frequency.  Using DDS techniques with 32 bit accumulators operating at approximately 3.2 kHz, the transmit frequency can be controlled - via the audio frequency - to a resolution of 0.0023 Hz, an accuracy that far exceeds the accuracy and stability of the reference oscillators themselves!

Producing exact frequencies:

The difficulty with using DDS techniques arises when an exact frequency is desired, such as for a frequency standard - at least unless one is willing to crunch a few numbers and/or make a few compromises.  For example, since the typical DDS algorithm is based on binary counters and thus has denominators of 2n power, one will likely not end up with the precise frequency desired.  In our example, above, with a 32 bit counter, we can likely get the frequency to with a fraction of a Hertz, but not exactly where it should be.

There are several ways around this, including one or more of the following:
  • Extending the resolution of the binary counter used in the DDS with even more bits to get ridiculous resolution so that the resulting frequency is "good enough."  If enough processor time is available and 32 bits of resolution is not enough, 48 or even 64 bits of addition may be implemented to do unsigned math.
  • The careful selection of a clock frequency such that the divisors result in the exact frequency desired.  The difficulty here is that if it is an "exact" frequency that is desired, many reference frequencies - such as 10 MHz - are not "binary friendly", requiring a bit of clever math to come up with exact relationships with the target frequency.
  • Designing the DDS counter to use something other than a binary (2n) counter.  If, say, a 10 MHz clock is used, the software DDS may be implemented using counters that will roll over at 10n instead of 2n, driven by a hardware divisor set to a base-10 relatable value to yield exact frequencies.
  • Implementing "dithering" of the DDS count to achieve fractional tuning.  This involves switching between two or more frequencies at a specific rate to achieve a third, averaged frequency.
The last method, dithering, must be used with care as it will, by its nature, introduce spectral components that are necessarily lower in frequency than that of the reference being generated by the DDS - possibly very much lower if the fraction being represented by the dithering is complex - and these lower frequencies can greatly complicate effective loop filtering!  In most cases it would be more beneficial to simply extend the resolution of the software DDS (e.g. more bits) rather than implement dithering making this technique most useful if one us using a hardware-based DDS.

Final thoughts:

While there are definite limitations in using a DDS reference to lock a high frequency oscillator, namely the need to suppress the inevitable reference sidebands that result from the DDS synthesis itself by filtering, careful selection of reference frequencies and/or choice of the type of oscillator, but appropriate application of these methods can produce a reliable, versatile - even simple - frequency source.

Wednesday, July 29, 2015

A PIC-based audio comb filter to remove AC mains hum

If you have read the page - and realized that I had something to do with its content - you will know that one of my interests is Free-Space, through-the-air optical communications.

In our many experiments one thing that we have often run across is the presence of mains-related hum from the spillover and "glow" of city lights that can invade the audio.  In the tests that in which I have personally been involved this has not necessarily been a huge issue as our location (in the relatively sparsely western U.S.) is not completely saturated with such illumination.  Even when we have run tests across town we have been able to avoid paths that are "terribly" affected by such energy.
Figure 1:
The prototype comb filter.
Click on the image for a larger version.

Not so for some of the folks doing similar experiments in the U.K., one of the most densely-populated countries in Europe.  There, it can be difficult - particularly near populated areas - to find a location that does not have a "glow" of city lights on the horizon.

To be sure, there are at least two energy components within such glows that can cause issues:
  • The modulation of the lights themselves, typically at twice the AC mains frequency.  Because of the non-sinusoidal nature of the waveforms this "hum" contains many harmonics, and because the lighting overall may be from all three phases of the electrical grid, some of these harmonics can be quite strong!
  • A "hiss" caused by the thermal noise of such lighting.
While the former may be removed electronically owing to its being confined to discrete, narrowband frequencies, the latter (the hiss)  is entirely random and cannot be "notched" out in the same way as the hum or buzz of the lights:  Some sort of "noise reduction" software such as that used in modern HF transceivers could be implemented, but that is beyond the scope of this discussion - and the processors discussed here do not have the "horsepower" to implement such filtering.

Fortunately, the thermal noise generally follows the "1/F" profile - which is to say that it is more intense at lower frequencies and the energy rapidly drops off, being much less of a problem at the frequencies that we use for voice (300-3000 Hz) than at lower frequencies.  Because of the nature of human speech, our brains can typically do some "mental DSP" to reduce the deleterious effects of the noise as background noise is a fact of modern life, but the raucous noise from the buzz and hum of the harmonics is far more difficult to deal with!
Figure 2:
A typical spectra that contains "buzz" from urban lighting
superimposed atop speech from audio on an optical path in the U.K.
As can be seen, there is less energy at the mains
frequency (50 Hz) and harmonics with most of signal
power in"spikes" occurring at 100 Hz intervals.
Click on the image for a larger version.

The Comb filter:

As it turns out what we need to filter the hum/buzz and its harmonics is a "comb filter" which, as its name implies, filters out a base frequency and the harmonics.  What is also fortuitous is the fact that such filters are very easy to implement in a simple processor using fixed-point arithmetic.

The type of filter that is implemented in this case is an "IIR" (Infinite Impulse Response).  To "construct" a notch filter in software we need only do the following steps:

  • Set up a buffer that will delay the "audio" data put into it by the period of the base frequency of the comb filter.  If you wanted to filter out a signal that consisted of 100 Hz and its harmonics, this delay would be 1/100th of a second, or 10 milliseconds. 
  • Take the output of the delay and multiply it by a factor of n, where "n" <1.  If "n" is 0.75 for "75% feedback", you would reduce it to 75% of its original amplitude.
  • Take a copy of the original signal and reduce it by "1-n".  Taking the 75% example above, this original signal would be reduce to 25% of its original amplitude.
  • Invert either signal (it does not matter which) and sum the two together and put this result into the delay.
  • The output is taken from the delayed signal.
In the example above we have a filter that feeds back onto itself 75% of the delayed signal with a contribution of just 25% of the original signal - with one of the two signals "inverted" to cancel out their contribution and the result is a filter that has notches at 100 Hz, 200 Hz, 300 Hz, etc.  By varying the ratio - for example, less "original" signal and more feedback - one can make the notches increasingly narrow at the expense of the filter being able to respond quickly to changes.

Practically speaking the useful values for the above range from comb filters that have as little as 50% feedback (fairly wide notches - which impart a somewhat "hollow" sound on the audio and fast response) to as much as 93.75% feedback which has quite narrow notches but is quite slow to respond to changes:  Values outside this range have been empirically tested and found to be less useful in this particular application - particularly higher values of feedback which tend to excessively slow the response.  (Note that the relative response time is proportional to the "base" frequency of the filter, so a 1000 Hz comb filter would respond more quickly than a 100 Hz comb filter with the same amount of feedback.)

This filtering is all done in "C" using the CCS compiler.  While not as streamlined as straight assembly, I've worked with this compiler for many years and have learned how to tweak it to produce "reasonably efficient" code that isn't terribly less efficient in execution speed than assembly - plus it takes far less time for me to write and debug C than assembly and the additional bonus is that the "meat" of the algorithms are portable!

For this sort of filtering all that is required in the maths are shifts, adds and subtracts:  You may note that the ratios, above (50%, 75%, 93.75%) all consist of discrete inverse powers-of-two (e.g. 93.75% = 100 - (50% + 25+ 12.5% + 6.25%) - all being numbers that can be derived by doing simple right shifts of the data!

The circuit:

Figure 3, below, depicts the schematic diagram of the prototype comb filter:

Figure 3:
The schematic of the comb filter board.
Click on the image for a larger version.
In the circuit above the input signal is first low-pass filtered by U101A and associated components - this, to reduce the energy at and above 16 kHz, the Nyquist frequency.  The filtering in the above circuit isn't extremely "strong", but because the majority of energy is typically around the speech range of 300-3000 Hz - and the fact that the overall energy contained in the speech and noise tends to decrease with increasing frequency - it is adequate for such use.

The heart of the circuit is U102, a PIC16F1847 processor internally clocked at 32 MHz which yields a sample rate of approximately 32 kHz with full 10 bit resolution of the PWM generator, used for D/A conversion.  The processed audio output, in PWM form, is then integrated and low-pass filtered back to "baseband" audio by U101B which is also configured as a low-pass filter.  This filtering is not as "strong" as the input filtering since, in most applications, it is less important that the generated energy above the Nyquist frequency be removed to the same degree as the input filtering.

As can be seen in the diagram the input pins are used to select the various filter modes as follows:
  • 50/60 Hz - This selects the mains frequency to be filtered.  60 Hz is used in the U.S. and its possessions, parts of North and South America and a few other countries while 50 Hz is used in the rest of the world.
  • 1x/2x - This selects whether the "base" frequency of the comb filter is 50/60 Hz ("1x") or twice the frequency (100/120 Hz for "2x").
  • Bypass - When left open (high) the filter is bypassed, but pulling this line low will enable the filter.
  • Sel1, Sel2 - This selects the various types of IIR filters, each with different amounts of "feedback" as noted in Figure 3.
Also present is the "Clip" LED and related circuitry (D102, Q101, etc.) that will illuminate if the input audio is more than "half scale" (e.g. 6 dB below clipping).  If this LED flashes more than occasionally it is recommended that the input audio level be reduced, particularly if audible distortion is present.  The occasional flashing of this LED on brief signal peaks and/or noise pulses is acceptable and does not usually result in audible distortion.

So there you go:  Using a low-end PIC for DSP!

Sunday, May 31, 2015

Adding a waterfall display to the mcHF transceiver

Figure 1:
The "Grey" waterfall mode of the mcHF self-contained SDR transceiver
in "Magnify" mode which shows only 24 kHz total bandwidth.
Three - possibly four SSB QSOs are visible in this picture.  This picture
was taken before I added windowing to the FFT, improving the "look".
One of the things that the mcHF SDR transceiver has been missing from the beginning was a type of spectral display often referred to as a "Waterfall" display.

It seems that all "good" SDR rigs have them these days - so why not this one?  When I first started working with the mcHF and contemplated this question, "there's plenty of processor horsepower for this... I think..."

Actually, it was more complicated than that.

Two types of LCDs and interfaces:

There are two "flavors" of the mcHF:  The early units used LCDs with "SPI" (high speed Serial) interfaces while the current crop (and by far the majority) use a parallel interface that physically maps the display's RAM to the processor's I/O controller so that it sees it as RAM - plus, it can access the display's normal commands with 16 bit-wide parallel data - either method being orders of magnitude faster than the SPI.

Now, as new features are developed for the mcHF, it is strongly desired that they all be compatible with both the SPI and Parallel version.  To that end Chris, M0NKA kindly send me an older "Rev 0.2" UI board with an SPI interface-type LCD so that I could spend some time improving the SPI-based routines that work with the display - plus it gave an opportunity to test both types prior to releasing software to make sure that I'd not "broken" anything on one or the other version of hardware.

Fortunately, after a bit of studying of the data sheets for the LCD unit I was able to discern that there was a hitherto un-utilized hardware feature that allowed bulk-writing to a defined area of display RAM and with this - and a few other tweaks - was able to speed the "user experience" of those with SPI interface mcHF transceivers by a factor of 4-10 or so, depending on what they were doing.  To be sure, the display updates on SPI-interface radios are still significantly slower than the Parallel-interface radios, but the former are very usable.

Rewriting the "Spectrum Scope":

Figure 2:
Rewritten spectrum Scope showing multiple SSB signals on
40 meters.  Signal-noise conditions were rather poor, but
at least 3 QSOs (and possibly 5) are visible in this shot.
The 'scope is set to display the full 48 kHz mode and 10dB/division.
Note also that the transceiver is set to "LO Low Frequency Translate
mode which moves its demodulation +6 kHz from "zero", eliminating
the majority of issues that are encountered near "DC" with
typical "audio" SDR implementations that use
direct-conversion and/or sound cards.
This brought back another thing that I'd wanted to do from the beginning:  A complete rewrite of the Spectrum Scope (e.g. "Spectrum Analyzer").  I'd reworked it early on, adding some IIR low-pass filtering to "smooth" the display and adding a bit of non-linear display function so that its dynamics were improved, by the vertical scale meant absolutely nothing at all - in fact, its dB/division scaling varied depending on what else was on in the display's passband:  It looked "OK" aesthetically if viewed from a sufficiently long distance, but it did not stand up to close, technical scrutiny.

So, I completely scrapped the Spectrum Scope function save for the original RFFT written by Chris, the somewhat hairy and efficient display functions originally written by Chris and hacked on by me (this is open-source!) and I kept my IIR low-pass filter which seemed to be just fine.

When I was done I had a "proper" spectrum scope with the vertical scale adjustable from 5dB/division to 20dB/division and even 1, 2 and 3 S-Units/Division (because I could...).  There was even an AGC function so that the strongest signal was adjusted to be at the top of the scope, but at the same time it would detect the peak/average of the signals on the scope and prevent an "empty" span from rising up and simply filling the display full of noise!

Not wanting to stop there, I put some thought into the waterfall display.

Adding a "Waterfall" display:

The waterfall display is the same as the spectrum scope, except that it adds the dimension of time:  While the frequency is represented horizontally along the "X" axis as with the spectrum scope, with a waterfall display the strength of the signal is represented by the color or intensity of what is on the display rather than the height.  The newest signal is "painted" along the bottom of the display and when the next scan is complete, the "old" information is scrolled upwards and the new information takes its place.  In this way, one has a brief history of signals - both on and nearby the center frequency.

There was a problem:  How do I do this without taking up massive amounts of processor power and/or memory?

As it turns out, all I really needed to store at any given time was the magnitudes and signals in the passband.  In "normal size" mode, the waterfall display is 70 pixels high and since it used a 512 point FFT (sort of...) I had 256 frequency bins, so I represented 48 kHz of bandwidth with 256 pixels horizontally with each pixel representing 187.5 Hz, nominally.  (Note:  It really doesn't work that way - read about FFT and windowing if you are interested...)   If all I wanted to do was store that information, I needed only 17920 bytes of storage since all I was actually storing was the same sort of data that the Spectrum Scope produced:  A signal that went from 0 for the weakest signal to 63 for the strongest - which would fit in a byte!  Making this a circular buffer with a pointer for each line, I could easily keep track of where I was in "time".
Figure 3:
Several CW signals (at least 5) shown with the Waterfall display
in "Rainbow" mode.  In this mode weak signals start at the "blue"
and of the rainbow and progress to red for the strongest.
(Pardon the garish colors and poor contrast - it's difficult to take a good
picture of an LCD screen with a cell-phone camera!)

For the color palette I did the obvious:  A 65 entry RGB palette that matched that 0-63 amplitude range:  That extra color?  I'll get to that in a moment...

The LCD actually uses a 16 bit RGB (5,6,5) color scheme, so it was easy to make another array into which I could simply load my color palette of choice.

Updating the display:

Updating the display was the slightly tricky bit.

Remembering that I'd implemented some "block write" functions when optimizing the SPI interface functions, I wrote some additional, minimalist functions to perform the needed tasks:  One to set up the area on the screen for the block write, another to accept data and another to "close" the block write and return the LCD to "normal" operation.

To display the waterfall, one simply defines the area of the screen defining the extent of the waterfall, enables the "write" command and then blindly throws the exact number of bytes of 16 bit RGB data at it constituting the size of the waterfall display.  It was a simple matter to use the stored amplitude data from the 17920 byte circular buffer to index the color palette and send that data to the LCD - I just had to make absolutely sure that I sent exactly the amount of data that conformed to the expected column and row sizes.

Since this was a "blind" write to the LCD it took an absolute minimum of I/O and even on an SPI interface LCD - while significantly slower than on a Parallel-interface LCD - it is still very usable!  Better yet, it doesn't take oodles, gobs and tons of processor power.

It does take more processor power to update the display than does the spectrum scope, particularly if one has it set to its highest speed, but in actual use one would have it set so that it would take 10-20 seconds for a signal to disappear from the display, and that really doesn't load the processor too much.  (Audio processing always takes precedence, so if one turns on all of the DSP modes with the waterfall set to its fastest mode, you will see it slow down significantly!)

About that "65th" color?  If you look carefully at Figure 3, above, you'll note a blue (cyan, actually) vertical line aligned with the "7035" graticule on the bottom of the screen to indicate the "tuned" frequency:  It's used for that!

Other modes:

In Figures 1 and 3 the waterfall is shown in the "Magnify" mode in which only 24 kHz is actually being displayed instead of the possible 48 kHz (I need to add a picture to show this...) and there is also a mode of the waterfall display that takes up more of the screen, covering the word "Waterfall Display".  There is also a "HotCold" palette in which weak signals are blue and strong are red - and it would be an easy matter to add even more palettes - if someone wants to go through and derive all 64 colors.
Note:  As of version, eight types of FFT windowing are available, significantly improving the "look" of both the spectrum scope and waterfall display! - For more information about this read the article on Wikipedia about Window Functions.  If one chooses the appropriate window function the waterfall display and spectrum scope's response to signals is much "tighter" as spillover energy from one "bin" on the screen can be minimized.

Monday, April 20, 2015

It should have not been BPL that worried us! (And using a separate receive antenna at HF)

About 10-15 years ago - more or less - the ham magazines and press were abuzz with concerns about the impending doom that was "BPL" - Broadband Over Powerlines.

Ostensibly, there was good reason to be concerned.  These devices utilized much of the same frequency spectrum as HF and low VHF communications and were bound to cause some degree of interference.

It did not help that, despite optimistic predictions by advocates of the technology, power lines were not very good RF transmission lines in the sense that they were lossy and DID radiate rather badly - not to mention being rather noisy in their own right!  These were all factors that contributed to emissions from BPL systems radiating much more broadly than initially "predicted" by its proponents and the subsequent need to use far more RF power to convey data with the integrity necessary to present to the consumer what appeared, on the surface, to be a viable product.

Interestingly, despite the attempts to assuage the amateur community that BPL would not cause significant interference, some independent "field readings" were taken indicating that the radiated signal strengths were far higher than expected and that the utilized power levels were far higher than predicted - apparently in an effort to overcome the inherent shortcomings of using the power utility's conductors as a conduit for data.  It got to the point that, allegedly, some attributes of the laws of physics themselves were conveniently "revised" to favor the BPL proponents without regard to potential interference issues - a fact touched upon in the video linked below:

The video above:
"BPL Unredacted":  ARRL's Dave Sumner reviews
redacted FCC info on BPL interference.
Total Run time:  10:59
(From the ARRL Forum, 2009 Dayton Hamvention)

Not too unpredictably, BPL as a technology has all but died:  It was just too "slow", even compared to the competing technologies at the time, and it had the inconvenience of not being "wireless" in any sense of the word!  Inevitably, even the "non-wireless" technologies such as DSL and "Cable Internet" quickly trumped the available capacity of the proposed BPL systems with their more practical "scalability" - that is, the ability to practically construct a data network in which many people can simultaneously use bandwidth by being able to sub-divide the resources to the "local" area.

Nevertheless, many hams attempting to use HF have reported an ever-rising tide of background noise on HF - not from BPL or even the plethora of "wireless" electronic devices that are designed to emit radio signals, but from devices that the uninitiated may not even consider as a potential RF radiator.

I am talking about power supplies and the devices that contain them.

True, there are a few devices such as Plasma TVs that are known to cause all sorts of "racket" but these, too, are a dying technology - literally:  Very few of these TVs are sold these days and even fewer are repaired as they quickly die of old age and failing electrolytic capacitors!

The type of power supply to which I'm referring is a switching power supply.  For the very same reason that BPL tended to radiate more more widely than "predicted" (it was not much of a prediction, actually:  As the video states, the FCC's own engineers expected problems from the beginning!) a badly-designed (or failing) switching power supply can radiate its "crud" quite effectively.

These devices are ubiquitous these days, being lighter, cheaper, and more efficient than their older, iron and copper counterparts.  These devices are essentially an oscillator powered by the AC mains voltage that feeds a transform that is made small, a feat possible by the high frequency (typically 30-60 kHz) of the oscillator, with the output of the transformer rectified back to DC again with some feedback used to regulate the voltage.  These devices may take the form of the small, external "wall wart" or they may be built into the appliance or product - but if it has been built since the early 2000's and it doesn't feel as though it has a big chunk of iron somewhere in it, it probably has a switching supply!

Being that these devices are oscillators at their core it is no surprise that some of this energy can "escape" and appear on HF frequencies in the form of harmonics.  While it is well within practicality to design these devices such that they would radiate a negligible amount of energy in the HF - or even VHF - frequency range, it may not surprise you to know that many of these devices actually do radiate significant amounts of energy.

Any electronic device sold in the U.S should be subject to certain levels of certification and one of these certifications is related to FCC part "15" which dictates limits of how much stray energy a given device may emit at various frequencies.  Now, it should be made clear that this does NOT mean that a device that complies with these rules DOES NOT emit stray signals at all, but rather it means only that they comply with a certain limit.  In other words, if a certain device meets these rules, it can still radiate - and even cause interference.

This brings us to another portion of FCC part 15 that says, in a nutshell, that if the device causes interference, it is incumbent on the user of that device to take care of the interference - even if it means that they must stop using that device.

This is all well and good, but if such a device belongs to an uncooperative neighbor or, perhaps worse yet, it is a device that is in your house - perhaps the "wall wart" (plug-in wall adapter) that runs something that you or someone in your house finds "essential" or is inconvenient,, expensive or impractical to replace or discard, what can be done?

Resources on identifying and fixing the problem:

Followers of this blog will have noticed that there are a number of entries devoted entirely to the quashing of "grunge" emitting from these very devices - some of which appear to be well-designed pieces of equipment.  A few such blog entries include:

In the above articles may be found some descriptions of the inner-workings of such power supplies along with how and why they may radiate.  A bit of searching on the internet will yield many hundreds of similar articles related to taming noise problems with such power supplies.

While it may not be just one power supply that causes a problem - or even "too much" of a problem, the fact that one may have a house that contains dozens of these things, each emitting a small amount of "grunge" across HF, can cause a cumulative degradation of the reception of HF.  Unlike the "old-fashioned" type of interference caused by a fluorescent lamp or noisy power line that one may recall where there is an obvious "buzz" at twice the line frequency (120 or 100 Hz), the noise from these supplies often takes the form of a "hiss" - often with a bit of AC mains-frequency modulation on it - but sometimes not.

What's worse is that if there are a lot of these devices all interleaving and summing their energy together, it may not be immediately obvious that there is an interference issue at first, particularly when one goes around the house and starts to unplug devices one-at-a-time:  If it is a multitude of devices, unless all of the devices are powered down at once by turning off the main breaker to the house, the magnitude of the problem - if the source is confined to your house - may not be readily apparent!

There is hope:

For the most part, even if you live in an urban area it is most likely that the worst offenders will be in your house, on your electrical system.  If you live in an apartment/condo/townhouse where electrical utilities are shared and you live in much closer quarters, there will be more of a challenge in reducing noise sources, but identifying and fixing (or removing) the offending devices in your possession will certainly help!

Not mentioned previously is the fact that in HF operation one can "split" antennas effectively between receive and transmit.  While it is well-known that a "small" HF antenna such as a mobile whip or a loop typically has a disadvantage when it comes to effectively radiating a signal, there is usually less of a problem when it comes to receiving signals.  An antenna such as a loop that is both directive in its ability to cast "nulls" in specific directions and it is somewhat resistant to local noise "E" field noise pickup which makes it a good candidate for receive-only while while a larger antenna may be used for transmitting.  If it is used for receive-only purposes, it need not be constructed using the same large components that might be required for an antenna that would need to handle 10's or 100's of watts, greatly simplifying its construction and lowering costs.

The caveat with the use of "split" antennas is that most radios cannot be easily configured to transmit with one antenna and receive with another, but there are some T/R (Transmit/Receive) relays available that may be keyed by the output on the radio that would be used to key an external HF power amplifier:  A bit of quality time with a search engine should locate a bit of information regarding such devices - including ones that you can build!

Sources of external TR switches - Buying or building:

One such device, commercially available, are these offerings from MFJ:

Another commercially available device is this:

I have not personally used either of the above units so I cannot personally vouch for their suitability, but they look as though they suitable for the purpose.

If you want to build one of your own, consider this article:

While designed original for an Icom radio, it can easily be used with almost any transceiver, as the article describes:  All it needs is a "keying" line from the transceiver - the very same one that would be used to key an external power amplifier.


While some of the commercially-made devices can automatically switch upon sensing transmitted RF, it is strongly recommended that one "hard wire" the T/R relay to the keying line of the transceiver - the same line that would be used to key an external RF amplifier.  This is recommended to prevent any possibility of "hot switching" the relay or sending any RF at all to the receive antenna - even for an instant!

Because an RF-sensing switch requires, by necessity, that there be at least an instant where there is RF being sent to the receive antenna, this could result in damage to a receive-only antenna that uses an internal RF amplifier or components that cannot handle transmitting power/voltages - even for an instant!

Additional resources related to the use of separate receive antennas:

A review on a receive-only loop antenna that describes how it may be used to reduce local noise sources:

Video above:
VK3YE demonstrates the efficacy
of a separate receive antenna.
The video is of rather low resolution, but you get the idea!

Comment about other wireless devices around the home:
  • There are a lot of devices that are designed to transmit, such as wireless modems, access points, etc.  By themselves, these will NOT cause HF interference because they do not operate anywhere near that spectrum, but rather in an amateur bands approximately 100 times above the HF range.  What can cause interference with HF operation is the power supply that operates these devices as they are likely of the "switching" type!

Tuesday, March 10, 2015

Update on the mcHF - Adding more features (Part 2) - Implementing audio filters and fractional I/Q phase adjustments

The front panel display of the mcHF SDR transceiver, an entirely
self-contained all-mode HF transceiver based on open-source software.
This is shown not in its 3D-printed case - which is open-source, too...
(No, you don't need a computer for this transceiver to work!)
Last time (See the January 27, 2015 entry - LINK) the addition of AGC and a gain control to maximize receiver dynamic range was discussed.

This time:  A discussion on the addition of adding audio filtering, decimation/interpolation and fractional I/Q phase adjustments for both receive and transmit.

Doing audio processing with limited resources:

Being that the mcHF is entirely self-contained and has a reasonably powerful, yet modest processor (the STM32F405 or '407 processor running at 168 MHz - a device with an ARM Cortex M4 core that includes hardware floating-point support) there are practical restrictions on how much number-crunching one can get away with when performing various tasks.  As you might expect, the most time-critical task is the "real time" processing of the receive data from the dual A/D converters as I/Q channels into demodulated audio.

If one goes through the literature on typical SDRs in the amateur world it quickly becomes apparent that limited processing power is often not a prime consideration.  This is not too surprising considering the fact that multi-GHz, multimedia processors in desktop computers are ubiquitous, so processing power is considered to be "cheap" and very often a lot of processing is thrown at a problem simply because it is available!

For example, one might implement an FIR (Finite Impulse Response) audio filter with 512 or even as many as 2048 taps without batting an eye on a PC and still have resources to burn, but try that on the processor in the mcHF and you'll have suddenly used up the vast majority of processor power (if not all of it - and more!) on that one task!

So, one starts to ask various questions like:
  • How can I do something while using less processor horsepower?
  • How "good" is "good enough"?
  • If I cut a corner somewhere, how detrimental will it really be in practical, real-world situations?

Receive signal processing:

The mcHF uses a standard, inexpensive "sound card" type codec (a Wolfson WM8731 or the compatible TLV320AIC23) for all A/D and D/A.  This is a 16 bit device capable of a number of sample rates from 8 to 96 ksps (KiloSamples Per Second), but in the mcHF it is typically operated at 48 ksps, this to allow the visualization of the spectrum +/- 24 kHz from the center as can be seen in the picture at the top of the page.

Being a typical SDR one of the first steps in signal processing is the same as that of any typical "phasing" rig - analog or digital - and that is to impart a differential 90 degree phase shift on the audio channels, a task that is, in this case, accomplished with a "Hilbert" transform.

Now, at the beginning the mcHF's audio path was "baseband" based, which is a confusing way to say that all demodulation was done around "zero Hz" or "near DC":  A demodulated CW tone tuned to yield 700 Hz was, in fact, 700 Hz away from the local oscillator frequency.  This is not an ideal situation for a number of reasons (which will be covered in a later installment!) but it was, from the beginning, the easiest thing to do.

Because the demodulation was done "near DC" this meant that the Hilbert transformer had to be of the "0 degree/90 degree" type rather than the more typical "-45 and +45" degree type:  While the former can be made to behave fairly well at low audio frequencies with a reasonable number of FIR taps, the latter cannot!  An 81-tap FIR-based Hibert transformer is used to provide the audio phase shift, providing reasonable performance down to a couple hundred Hz - as low as we need to go for SSB!

Once the two audio channels are set 0/90 degrees from each other one can then do the math to convert the two channels into USB and LSB - which is just addition or subtraction of these channels:  Which does which depends on which phase happens to have been assigned where in the hardware.  This demodulation converts the separate I/Q channels into just ONE audio channel, but it is not yet bandpass-filtered!

Working with limited resources:

IIR instead of FIR audio filtering:

Originally the code used a combination of FIR low-pass and bandpass filters with 48 taps to provide the receive audio filtering but because the receiver sample rate was 48 ksps this meant that with so few taps that it was not practical to define a very "sharp" audio filter.  To get a "properly sharp" FIR audio filter at 48 ksps would require, perhaps, 3-4 times as many taps as that and commensurately larger audio buffers and an increased amount of processor overhead, so I decided to take a different approach.

Most of my embedded programming has been using fairly low-end PIC microcontrollers (the PIC16 and PIC18 families) and I have used both FIR and IIR filters on these devices of rather low complexity since the resource of these processors are extremely limited.  Having cut my teeth on these types of filters I was comfortable enough with IIR filters that I was not scared of their (somewhat undeserved!) reputation of being "inherently unstable" and I also knew from experience that IIR filters could, when properly finessed, offer superior performance to FIR filters in situations where one needed to severely limit the amount of processor overhead, particularly when one has floating point math available.

Having access to MatLab and its filter design/simulation tools I soon had working some fairly low-complexity IIR filters thanks to the built-in support of the CMSIS DSP library (link) that offered performance that was far superior to the original FIR filters with reasonable "Shape Factors" (e.g. the ratio between the passband and attenuation response).

Decimation to reduce processor loading:

So it remained like this for several versions of code:  All of the audio processing was being done at 48 ksps but I had not yet taken advantage of another trick available to reduce processor overhead:  Decimation.

At this point I was still crunching numbers at 48 ksps - this, to produce audio that had no components higher than 4 kHz or so!  Nyquist tells us that to process such signals we need not sample at any higher than 8 kHz, so running at many times that sample rate was simply wasting CPU power!

The term "decimation" in DSP terms simply means keeping 1 out of N samples and throwing the rest away, and if you have fewer samples, there are fewer numbers to crunch.  For practical reasons - sometimes those dictated by available library functions and/or the sizes of buffers - it is usually best to pick "N" as a power-of-two (e.g. 2, 4, 8, 16, etc.) For the second of these reasons (e.g. the audio "chunk" size was 64 samples - a number that was NOT evenly divisible by 6!) my best choice was to decimate-by-four to reduce my sample rate from 48 ksps to 12 ksps.

If you throw away samples you must still do something about the audio spectral content that would be above the Nyquist limit at the new sample frequency, so one must first low-pass filter and remove those signals and that meant that I had to get rid of those signals that would be at 6 kHz and above!

  • In theory, a decimation-by-8 to yield a sample rate of 6 ksps would have worked since, for normal SSB, my maximum audio frequency could be less than 3 kHz.  The problem is that this would have placed my Nyquist limit very near my desired maximum frequency of 2700 Hz or so - and in the audible range - requiring pretty tight filtering.  The added complexity of such filtering may have countered any gains afforded by the reduction in sample rate, plus it would have precluded the use of "wide" audio filters (such as the 3.6 kHz filter) for SSB and AM unless I were to have added yet another decimation rate!  (For CW or Digital-mode filters this would probably be fine...)

The CMSIS library for the ARM M4 processor contains a handy decimation function with a built-in FIR low-pass filter, but number-crunching (with the aid of MatLab) showed that I'd need quite a few FIR taps - and additional processor overhead - to get the needed 60dB or so low-pass filtering that was required!

Fortunately, I already had a low-pass filter at hand:  The Hilbert transformer!

Using the free "Iowa Hills" filter design tools I designed a new Hilbert Transformer that was identical to what was already in there, except that it had a low-pass cut off starting at about 3.6 kHz or so - just about right for the 3.6 kHz audio filter in the radio - and with 81 taps it provided reasonable (>=55dB) worst-case low-pass filtering to prevent aliasing at and above 3.6-ish kHz, the highest audio frequency that my audio filters would pass.

At this point I'll say a few words about "appropriate" audio filtering.  I really didn't care too much if the filtering above this frequency was insufficient to prevent "audible" aliasing since it was going to be removed by the SSB filters anyway.

For example, if one picked, say, 4 kHz has the highest-frequency signal that was going to get through the 3.6 kHz audio filter (e.g. a strong CW signal down on the "skirts" of that filter) we know via math that its alias would be at 12 - 4 = 8 kHz.  What that means is that worst case, our strongest alias signal - and the "closest" to our audio passband - would be at 8 kHz, so our filter must attenuate adequately - by some 60dB or so - between the 3.6 kHz representing the "top" of our widest filter and 8 kHz.

On this point I actually "fudged" a bit:  The filtering was adequate to knock it down only by 50 dB or so, but since the 3.6 kHz filter was not going to be used very often I looked, instead, at the numbers for the "2.3 kHz" filter - which actually passes audio between about 300Hz and 2600 Hz, and its steep skirt is around 3100 Hz or so.  Taking 3100 Hz as our "new" highest frequency the alias would be at 12 - 3.1 = 8.9 kHz and this extra (almost) 1 kHz of roll-off provided another 10 dB or so on our low-pass filter.

Because the CMSIS decimation function required that I put some FIR low-pass filtering in place - that is, I could not use the function without it - I used as few as FIR taps as practical, finessing them with MatLab so that they, too, did as much filtering as they could with those few taps and achieved something in the 10-20dB area (depending on frequency) on top that achieved with the Hilbert transform, so we now had our target of at least 60 dB!

At the output of the decimator I now had to work at 12 ksps rather than 48 ksps which meant that I had to redesign all of my audio filters, but by keeping them as complex as they had been before - even though the sample rate was now lower - meant that they could now be made to be made "sharper" than before and thus offer higher-performance!

Since I had one forth as much audio data to crunch, more processor horsepower was now available to do other things, allowing to add additional features in the future!

Interpolating back to 48 ksps:

At the end of the audio filtering and AGC processing I had another problem:  The D/A converter still operated at 48 ksps so I had to do an "interpolation" step to up-convert from 12ksps.  Here, too, one must do a bit of low-pass filtering, but for a different reason:  The raw 12 ksps audio would contain aliased audio if upconverted directly to 48 ksps and this can be heard (if your ears are good enough) and could be extremely annoying!

As with the decimation, there is a CMSIS library interpolation function and it has a built-in FIR low-pass function, but there is no real need to make this filtering particularly strong.

In the worst case scenario with the 3.6 kHz audio filter selected, there will be audio content at (12kHz - 3.6 kHz = ) 8.4 kHz - but nothing below that, so a fairly weak low-pass filter with relatively few FIR taps (to minimize processor loading!) was designed in MatLab.  This filter was designed to start rolling off above 3.6 kHz and by the time it got to 8.4 kHz it was attenuating by 25dB or so and was down by 30-40 dB the time it got to 9-10 kHz, where the vast majority of the aliasing energy would be when one operated the most-used 2.3 kHz audio filter:  Because of the way that the human ear works, the "clutter" at the high audio frequencies, knocked down by 25+ dB would probably not even be noticed by someone with even the most acute hearing!

Upon getting this code operational, I fed the LINE OUT from the receiver into the Spectrum Lab program with a sound card running at 192 kHz and verified that the low-pass filtering did, in fact, reduce the aliased signals by the predicted amount.  I then routed the audio into full-range speakers and, with a graphic equalizer, intentionally boosted the highs by 10-15dB in an effort to accentuate the aliasing signal but even in a worst-case scenario with single tones the aliasing was pretty much inaudible.

  • For the 10 kHz audio filter mode, decimation/interpolation-by-2 was used and similar tricks were used with the Hilbert transformer to minimize processor loading.  In this case the interpolation's low-pass filtering was far less effective since the Nyquist frequency is 12 kHz so the output contains aliasing components that are only 6-15dB down, worst-case.  Even in this case, with a 9.8 kHz tone - with the alias at 24 - 9.8 = 14.2 kHz - the psychoacoustical properties of human hearing make it somewhat difficult to hear this tone.
  • The codec chip is also capable of operating at 8 ksps, both for A/D and D/A operations.  While this would greatly reduce processor overhead, it would limit the view on the spectrum scope to just +/- 8 kHz!  If there had not been enough processing power to do what was needed to be done this may have been a necessary strategy to take, but with voice modes, it turned out not to be needed.  In the future when digital modes are contemplated and additional processing power may be needed - but a wide "spectrum scope" is not - the use of an 8 ksps rate will be considered.

Fractional I/Q phase adjustments:

One of the problems with real-world hardware is that there will inevitably be variations in the I/Q phasing and amplitude of the audio as it is processed by the two audio A/D converters.  This phase difference could be from a slight shift in the local oscillator signal or, more likely, it could be due to component variations in the analog circuitry comprising the mixer and filtering that precedes the A/D converter.  Whatever causes this problem, it must be addressed to maximize the opposite-sideband rejection.

Addressing the amplitude imbalance is pretty easy:  One simply multiplies the amplitude of the I/Q channels by a small, fractional number made adjustable by the user.   Typically both channels are multiplied by equal and opposite amounts so that the total amplitude remains generally constant.

Phase adjustment, on the other hand, is trickier!

In my researching how this is done on PC-based SDR implementations I saw that there appeared to be handy, on-the-fly calculations phase adjustments that could be done using various library functions that would transform the I/Q signals.  While such functions may have existed somewhere in the bowels of the libraries available to me, real-time calculation of the phase for each sample that came in would represent a prohibitive cost in terms of processor power!

So, how would one take care of this problem with a minimum of overhead, preferably with a one-time calculation?

It struck me that if I could modify the Hilbert transformer in a "fractional" manner I might be able to effect a fractional phase adjustment.  Since calculating the 81 coefficients internally was out of the question (I didn't want to figure out how to do that from scratch, plus it would have been a pain if I needed to modify the Hibert transformer in the future if I wanted to modify its parameters!) I wondered if I could "tweak" a fixed set of coefficients and not "break" the transformation to a significant degree?

Using the Iowa Hills program I calculated four sets of Hilbert coefficients:  0 degrees, 90 degrees, 89.5 degrees and 90.5 degrees.  In the mcHF, the 0 degree set (the "I" channel) would remain constant, but if the I/Q phase needed to be adjusted, the data from the nearest "alternate" set of coefficients for the "Q" channel would be proportionally blended.  For example, if 89.95 degrees was needed, it would use 10% of the 89.5 degree set and 90% of the 90.0 degree coefficients and this new data would be input to the Hilbert transformer.

Putting this new code into the receiver, using a signal generator, and making very careful measurements at the "worst case" settings of 89.75 and 90.25 degrees (and a few points in between) I measured only very minor amplitude (for which I could compensate!) and phase degradation in the performance of the Hilbert transformer due to these "straight line" approximations and called it good!

This method of phase adjustment is applied for both receive and transmit, but in practice it has been observed that far more amplitude adjustment is typically required than phase adjustment to effect optimal opposite-sideband rejection, at least if good-tolerance components - especially capacitors - are used in the receive and transmit paths!

  • On my mcHF transceiver I have typically require well under 1/10th of a degree of phase adjustment, so the available range of +/- 0.5 degrees is probably overkill!

What about transmit?

All of the above tricks could be applied to transmit, but thusfar, there has been no need to do any decimation/interpolation - just the Hilbert transformations, amplitude and phase adjustments, some audio processing and filtering - topics for later discussion.

For transmitting, since fewer operations need to be accomplished than in receive, everything still operates at 48 ksps with processing power to spare.  If it does become necessary in the future, I could decimate/interpolate within the transmit function and "regain" a significant number of CPU cycles!

How does the receiver sound "on the air"?

In tuning around with this receiver, the result of this processing - which is all done with floating-point arithmetic - is at least comparable to any of my other all-analog radios in terms of filter performance, adjacent channel rejection and opposite-sideband rejection:  Even under "contest" conditions with a very strong signal "next door" the receiver seems to be perfectly capable of holding its own!

While a few shortcuts had to be taken to reduce the amount of processing to an amount that could be handled by this radio, it does not seem to have demonstrably compromised its receive performance in actual, real-world conditions!

Next time:

Adding DSP noise reduction and automatic notch filtering.

Saturday, March 7, 2015

Using the "RTS.SDR" dongle (with HF upconverter) as found on Ebay

Some months ago I was perusing the wares of a seller on EvilBay and having decided to purchase one of those all-purpose component testers, I was tempting myself with other items on sale.  One of the items that I could not resist was the "RTL.SDR" that was billed as having an onboard frequency upconverter to allow HF operation, plus a more stable reference oscillator.

Figure 1: 
The EvilBay "RTL.SDR" with built-in upconverter
for HF reception.
According to the pictures on EvilBay, the units
currently sold host different picture, with
 a graphical representation of the schematic
diagram showing the mixer.
Click on the image for a larger version.
Stepping back for a moment, the "RTL" SDR dongles have been around for a few years, so-called because they typically contain as one of their active devices the Realtek RTL2832U.  These devices - costing roughly $10 (more or less) also contain a device (a Rafael Micro R820T in the case of the one pictured) with an onboard frequency synthesizer, I/Q mixer and (8 bit) A/D converter that allows one to receive signals from 50 MHz (or lower) to well above 700 MHz - sometimes way above 700 MHz, just using a computer with reasonable horsepower and FREE software!

While the thought of an 8 bit A/D converter might make it sound like these devices may be useless when connected to an antenna, the fact that they convert only a limited range of frequencies (approximately +/- 2.4 MHz at most) around the tuned frequency - plus the "oversampling" nature of the raw A/D sampling rate of the 8 bit converter means that relatively narrowband signals can have quite good ultimate quantization range.  What can suffer is when, within the sample-rate passband there are signals that range widely in signal strength:  Under such conditions it is recommended that one "finesse" the gain controls for optimum input signal levels to the A/D converter.

Perhaps the most popular software to do this is "SDR#" (a.k.a. "SDR-'Sharp'").  Easily found via a web search, this software is compatible with many software defined radio packages and it supports many modes and features, also offering many optional plug-ins.

These low-cost USB dongles were intended to allow the reception of digital TV and FM/Digital Audio broadcast (but not the sort of TV or digital audio broadcast that is used in North America) on a computer so they were designed to tune from something around the low-medium VHF range - say, 20's to 50's of MHz and up, almost completely leaving out the HF frequency bands.

Several schemes have been devised to allow the use of these handy devices to receive HF and these typically involve the use of some sort of upconverter to translate the HF frequencies from as low as the AM broadcast band up to a range that these devices will tune.

So, I soon found myself with the device pictured in Figure 1 in my hands.  Getting busy with life and other things, it was a few months before I got around to playing with it and while I quite easily got it working on the "UV" input which, on this particular unit tunes from just below 25 MHz to over 1 GHz (although I'm not sure exactly how far above 1 GHz...) but I was puzzled as to how to configure it operate at HF.

How to make it work on HF:

Back then, a web search didn't help as there was a plethora of hits on "RTL SDR Upconverter" so I removed the unit from its case and found the callsign "BA5SBA" on the board that you see in Figure 2 and got a hit with Google on a German page (this one - LINK - you may need to run it through a translator, so try THIS LINK which may or may not work...)  While this web page didn't describe exactly what I needed to do, it did provide enough clues to get it working!

  • More recently, the some of the sellers on EvilBay are including in their postings some basic instructions on how to get these devices to work on their "HF" ports, but what follows below contains more information - and is hopefully easier to find and more permanent - than an EvilBay posting!

Figure 2: 
Inside the BA5SBA "RTL.SDR" with the
built-in upconverter - the singly-balanced diode mixer
being located under the white blob in the foreground.
The jumper in the upper-left corner marked "A" and
"P" appears to be intended for putting 5 volts on
the "UV" antenna connector to power an external
RF amplifier.  ("A" = active, "P" = passive).
Click on the image for a larger version.
As I surmised from what can be seen in Figure 2 - and was verified by the German page listed above - the "HF" converter was simply a diode-ring mixer driven by the already-present 28.3 MHz reference oscillator on board the converter.  The output of the diode ring mixer was then connected to one of the inputs of the chip, so all I had to do was to figure out the offset frequency and how to access that input.

Using a signal generator set to 5 MHz I connected it to the "HF" input of the device and, just for kicks, tuned SDR# to 5.000 MHz as well.  After a bit of fiddling about, I finally found how to configure the unit to receive at HF.

  • STOP the SDR receiver first by hitting the square "stop" button at the top.
  • Bring up "RTL-SDR Controller" window (accessible by clicking on the "gear" symbol)
  • Under "Sampling Mode" set it to "Direct Sampling (Q Branch)".  Note:  You cannot change the sampling mode without first STOPPING the SDR.
  • Restart the SDR by pressing the "start" button (the right-pointing triangle resembling a "PLAY" button)

Once I had done this I saw the 5 MHz signal on the display!

  • By default, SDR# will set the "RF Gain" setting in this same "RTL-SDR Controller" window (see Figure 6) to minimum gain (all the way to the left.)  If you leave it there, the SDR dongle will be EXTREMELY DEAF when it is in the normal "Quadrature Sampling" mode.  When you have it in either "Direct Sampling" mode this control is disabled.
  • I would suggest checking the box that says "RTL AGC" and moving the RF Gain control in either "Quadrature Sampling" or "Direct Sampling" modes:  If you do not do this your receiver may be very deaf!
  • I experimented with using this device with the "HDSDR" program and found that at HF, its sensitivity was noticeably worse than that obtained when using it with SDR#.  I'm not sure why this might be, but it seems to have something to do with the way the "ExtIO" driver - or HDSDR itself - handled the internal gain settings of the dongle.
Figure 3: 
A screenshot of SDR# showing the configured "Shift" for the mixer.
Click on the image for a larger version.

Now I am very surprised that I can tune SDR# to "5.0" MHz and see the 5 MHz signal that I am inputting as the chip on this dongle should not function at that frequency!  What I expected to have to do was as follows as shown in Figure 3:
  • Knowing that the onboard mixer uses the 28.8 reference oscillator, check the "Shift" button on SDR#.
  • Enter a frequency of "-28800000" (yes, negative 28.8 million) into the box as shown in Figure 3.
  • Note:  There is a bug in some versions of SDR# that, if you are already tuned to a frequency lower than the "shift" frequency, the tuning may not work properly until one sets a much higher frequency with the 100 MHz digit and tunes back down.
  • Having done this, one need not mentally add the desired HF receive frequency to 28.8 MHz:  The receive frequency is simply input to the main frequency tuning as one would any other frequency!
  • If the 28.8 MHz frequency reference oscillator is somewhat off frequency one must first set the frequency using the "normal" VHF/UHF mode and calibrate it in the "RTL-SDR Controller" window and THEN, after switching to HF mode as described above, one can input a frequency that is slightly different from 28.8 MHz as needed to correct the offset.  I did not have to make any correction at all!
  • The 28.8 MHz reference oscillator supplied with this unit appears to be quite a good, stable TCXO and is only off a few PPM!
  • If you want to use the VHF/UHF input again, remember to reset the "Sampling Mode" back to "Quadrature Sampling" and UN-CHECK the "Shift" box that is visible in Figure 3.
Does it matter whether one uses the "0 MHz" method or the "-28.8 MHz" method with the offset?  Not really:  I could see no difference in the sensitivity of the SDR - but it is good to be aware of both methods just in case the "0 MHz" method doesn't work!

Comments on the design of the HF mixer and its limitations:

Sensitivity (or the lack of...)

Figure 4:
The low-pass filter in front of the mixer.
Click on the image for a larger version.
The very simple mixer on this SDR Dongle has its limitations, the first of which is that being entirely passive, the ultimate sensitivity at HF is not particularly good:  Using SSB (2400 Hz) bandwidth, signals down to around -90dBm were audible, but that was about it.  While this represents a signal of around 7 microvolts and may seem horrible (and it is, compared to modern HF receivers!) it should be remembered that if this is connected to anything resembling a full-sized HF antenna, it "hears" just fine since the background noise on HF bands - and thus the signals - will typically be above this level - at least on the "lower" bands of 10 MHz and below.

Because it does use a passive diode mixer it can take more signal and this means that if an amplifier is placed in front of it, it is not as likely to be overloaded as other upconverter designs for SDR dongles that use, say, an NE602!  This is probably a good thing since the converter chips in these devices are fairly easily overloaded, anyway!

Nyquist rules!

The other problem is that the local oscillator frequency is 28.8 MHz.  What this means is that there is a Nyquist limit of 14.4 MHz - and that means that any signal above 14.4 MHz will reappear again as an "image" at a lower frequency, approaching Zero Hz as it approached 28.8 MHz.  In other words, if you were tuned to 7.1 MHz in the 40 meter band, you would also hear a signal at 7.1 MHz below 28.8 MHz, or 21.7 MHz!  This also means that if you were trying to listen on 21.1 MHz in the 15 meter band, you would hear a signal at its 7.7 MHz image!  The situation is worse on 10 meters since the 28.8 MHz local oscillator is nearly in the middle of this band:  An FM signal on 29.5 MHz would also appear on 28.1 MHz!
Figure 5:
The response of the low-pass filter depicted in Figure 4.
The bandpass response is the solid line.
Click on the image for a larger version.

If you look at the circuit board in Figure 2 you'll notice several components in the foreground, namely a number of inductors and surface-mount capacitors, all with the values marked on the silkscreen.  Presuming that the values of these components are as-marked, I drew the schematic in LT Spice that is shown in Figure 4 and simulated it with the result in Figure 5.  As can be seen, this low-pass filter effectively blocks signals much above 40 MHz or so, offering more than 40 dB of attenuation above 60 MHz.

Other than keeping signals from low-band TV and FM broadcast out of the "HF" input, this filter does not help us at all in our image problem.  Because this filter does not block signals immediately above 28.8 MHz, we have another set of images that occur which means that a signal that is at 33.8 MHz - 5 MHz above 28.8 MHz - will also show up at 5 MHz! 

If there are no signals (or noise!) on these "image" frequencies, there should be no problem, but this is likely not to be the case during the daytime on a large wire antenna - which means that if you really want to use this dongle for serious HF reception, you need some sort of filter in front of it, either a separate Low Pass (below 14.4 MHz) and High Pass (above 14.4 MHz) to be used individually, or you could use a tunable preselector to pass only the frequencies of interest.  (There would be a problem with 10 and 20 meters as these bands are close to or include the Nyquist frequency!)

Possible modifications:

If you look at Figure 2 you'll note that there are several small "prototype" areas on the circuit board.  These are large enough to allow the construction of some simple circuits, namely a bit of additional gain (amplifiers, filters) to improve the sensitivity of the receiver.

As mentioned before another useful modification - one external to this box - would be the addition of a preselector to filter a narrower range of frequencies.

Is this upconverter useful as-is?

In general, yes, with a good antenna.  Despite its flaws, when one casually tunes around the HF bands using the built-in upconverter its problems do not immediately jump out, at least on the lower bands.

The biggest advantage of this particular SDR Dongle is that it seems to have quite a stable oscillator on board - much better than the $8 USB "stick" that I have experimented with previously:  This one seems to move only a few 100 Hz from cold start even at UHF frequencies whereas the very cheap ones seemed to move all over the place with temperature!

To be sure, one could probably get better HF performance by building (or buying) an upconverter for this sort of SDR that uses a much higher local oscillator frequency - say, 100 MHz -  but if one upconverts all of the HF frequencies en-masse, there are a few things that one must remember:
  • If you use a chip like the NE602, it may simply overload when connected to a full sized HF antenna unless you are prepared to add switchable attenuation to it and/or a preselector!
  • On a full-sized antenna, intercepting and converting the entire HF spectrum can represent quite a bit of signal energy.  If the HF upconverter that you are using does not, itself, overload, there is a good change the the converter chip in the RTL-SDR itself will overload with so much energy from such a wide variety of frequencies!  Remember:  The receiver chips in these dongles were intended to work with the relatively weak and sparse broadcast signals intercepted on small, indoor antennas, not the entirety of the HF spectrum when the bands are open!
Figure 6: 
The screen for configuring the dongle's
Click on the image for a larger version.

A few brief comments on using it at VHF/UHF:

Remembering to set it to the "Quadrature Sampling" mode when using the "U/V" input, there are a few things to know when using this device - and possibly others like it - with SDR#.

Clicking on the "gear" symbol will open a screen such as that depicted in Figure 6 that allows modification of settings related to the dongle's hardware.  In particular, note the check in the box for RTL AGC and the lack of a check in the Tuner AGC box:  The RTL AGC box is necessary in both the Quadrature Sampling mode to get the most gain out of the device.

The Tuner AGC box - which is only available when Quadrature Sampling mode is active - if checked, does not seem to work properly in the the version of driver and/or SDR# at the time of initial posting of this blog entry (March, 2015) as it seems to increase the gain too much, causing overload of the A/D converter and actually reducing performance - particularly if a decent antenna is connected!
Figure 7: 
 Top:  Properly adjusted RF gain setting
Bottom:  The degradation from too much RF gain!
Click on the image for a larger version.

Figure 7 demonstrates the problem.  The top half of the image shows an optimally adjusted amount of gain - a setting of 12.5 dB for this particular antenna (yours will vary!) while the bottom shows a gain setting of 37.0 dB.

Even though more signal is reaching the A/D converter in the bottom example, it is way too much and the converter is being badly overloaded, causing the noise floor to rise from below, submerging weaker signals!  If the RF gain is sett too low, degradation can also occur, but this time from too little signal.

The idea is to adjust the RF gain so that there is there is the maximum difference between the highest signal and the noise floor seen between stations - even if the absolute level of the signals may be lower, overall!