Adding FM transmit to the mcHF transceiver

In previous postings I wrote about how FM reception - including squelch and subaudible tone detection - was accomplished on the mcHF.  As is often the case, it is usually more difficult to receive a signal than to generate one and this is arguably the case with FM as well.

In writing the code to generate FM I found the above to be true:  It was comparatively trivial to produce an FM signal, particularly applying "tricks" that I'd already done in the demodulation.

Producing the FM carrier:

One of the features that I added to the mcHF many code releases back was that of "Frequency Translation" in which the local oscillator was "off-tuned" from the receive frequency, + or - 6 kHz for the mcHF with the "baseband" signals for receive and transmit being shifted by 6 kHz (in the opposite direction) in software.

One reason that this was done was to improve the performance of the transceiver by removing its receive and transmit passbands from the immediate vicinity of the "zero Hertz hole" which not only improves frequency response, but it reduces other issues related to thinks like "1/F" noise and, perhaps most importantly, greatly reduces the likelihood that other receiver audio (e.g. audio amplifier energy) will find its way back into the microvolt-level audio paths via power supply and ground loops and cause feedback!

Applying this to transmit, we soon realize that if wished to produce a signal with a constant carrier, such as AM or FM, we would have to remove ourselves from this "zero Hertz hole" as it would be, by definition, impossible to produce a carrier in that hole as a the carrier is, in fact, represented by DC.  (For SSB, which purposely has its carrier removed, this "hole" is nearly irrelevant...)

This explains another reason why this feature was added:  The eventual addition of AM transmission (and reception) several revisions ago, but this same feature used once again for FM, but in a different way:  Via the use of DDS (Direct Digital Synthesis) techniques.

The use of DDS techniques has been discussed on this blog before - see the article "Generating low-distortion audio sine waves using a PIC and DDS techniques." - link.

Using DDS techniques to generate an FM carrier:

In this technique one generates a sine wave (or any other arbitrary signal, for that matter) by jumping through a "lookup table".  In the case of the mcHF, with its 48 kHz sample rate, if we wanted to generate a 6 kHz sine wave this implies that we would need to step through this sine wave table once every 8 samples.  This sounds easy enough - but how would one do this?

Take a look at this bit of code:

loop:
   accumulator = accumulator + frequency_word
   table_index = accumulator > (size of accumulator in bits - size of table index in bits)
   amplitude = sine_table[table_index]

To explain the above:

- The variables "accumulator" and "frequency_word" are both integers.  Let us presume 16 bits, unsigned, each, which means that each value would range from 0-65535.  Incremented past 65535, it would return to zero.
- "sine table" is a lookup table containing values mapped to a sine wave.  Let us presume that our sine table contains 1024 entries - a number that may be represented by precisely 10 bits.
- "table_index" is used to index the table.  It must be able to index all of the sine table (10 bits) so we will use a 16 bit value for this.
- "amplitude" is the result from the sine table.  This could be an integer or floating point value - whatever format your system ultimately requires.

To calculate the value of "table index" we need to take the top 10 bits of the "accumulator", which means that we can obtain by taking the accumulator value and shifting it to the right by 6 bits (e.g. size of accumulator in bits, minus the size of table index in bits, which are 16 and 10, respectively.)  By doing this we can see that as the value of "accumulator" increases, it also points farther along the sine table.  When the value of accumulator "rolls over" back to zero, the pointer into the sine table also resets back to the beginning.

To understand how the frequency is generated, let us now assume that "frequency word" is set to 1.  We can see that every 65536 times through the loop we will "roll" through an entire sine wave, but since our sample rate is 48 kHz, we know that the produced frequency will be:

48000 / 65536 = 0.732 Hz (approx.)

If we want to generate an arbitrary frequency, we would take the above ratio and use it to calculate the "frequency word" as in:

desired frequency / (48000/65536)

or, rewriting a bit:

(desired frequency * 65536) / 48000

or, reducing the fraction even more:

(desired frequency * 512) / 375

If we wanted to generate a frequency of precisely 6 kHz, the above equation would yield a "frequency word" value of 8192 - which just happens to be exactly 1/8^th of 65536, which makes sense since we have already figured out that since 6 kHz is 1/8^th of our sample rate of 48 kHz, it would therefore take 8 samples to produce such a sine wave!

Modulating our carrier:

We now know how to generate a carrier, but how to modulate it?

We know that FM is simply "Frequency Modulation", and we also know that by varying the value of "frequency_word" above, we can change the frequency, does this mean that if we superimpose audio on our value of "frequency_word" that we can modulate our signal?

Yes, it does.

Let us rewrite the above code a bit:

loop:
   accumulator = accumulator + frequency_word + audio[audio_index++]
   table_index = accumulator > (size of accumulator in bits - size of table index in bits)
   amplitude_I = sine_table[table_index]
   table_index = table_index + 256
   if(table_index >= 1024)
      table_index = table_index - 1024
   amplitude_Q = sine_table[table_index]

(Let us assume that "audio_index" is updated each time through the loop and represents one sample of the audio, also sampled at 48 kHz, to be modulated onto the carrier.)

Let us first take a look at the first line after the start of the loop where we added the term "audio".  Because our audio is already represented digitally as a numerical value that goes above zero for a positive voltage and below zero for a negative voltage, it would make sense that we could simply add this to our "frequency_word" value.

In other words (pun intended!) when the audio voltage increased above zero, our frequency would increase, but as it went below zero, our frequency would decrease - just as FM would.  What's more, because this is an exact numerical representation, our frequency change would be proportional to the audio applied - which is just want we want to occur for low-distortion, faithful representations of our audio.

Figure 1:
A demonstration of a typical FM signal modulated with a tone as
displayed on the mcHF's waterfall display.
(This picture doesn't have much to do with transmitting, but
I wanted to include some color in this posting!) 

There is another modification to the above code as well.  If you look, you will see that we do two look-ups in the "sine_table".  The first one is our original value, now called "amplitude_I" (In-phase) but we now see that we have taken our table index and added 256 to it which is precisely 1/4^th the size of our sine table:  One quarter of a sine wave is, of course, represented by 90 degrees.  After "fixing" that value so that it is always lower than 1024, we look up into the sine table again and call this value "amplitude_Q".

What we have done here is generated two sine waves exactly 90 degrees apart from the same frequency synthesis operation.  As you will recall from your understanding of the "phasing" required to generate an SSB signal, you need both an "I" and "Q" signal for transmit and unlike the generation of quadrature audio for SSB which requires some fairly "hairy" math, we have handily done this for FM with almost no math at all!

Comment:  In reality one would employ modulus operators rather than "greater-than and subtract" or even logically "AND" the table index value with 1023 (decimal) after adding 256 to it, either being a much quicker operation for a computer than in the example shown above.

Additional audio processing:

As was mentioned in the discussion about the demodulator, for amateur radio purposes we don't actually want to transmit an "FM" signal, but really a "PM" (Phase Modulated) signal.  For our purposes, a PM signal is really an FM signal in which the audio is pre-emphasized at a rate of 6dB per octave - which is a fancy way of saying that a signal voltage that causes +/- 1 kHz of deviation with a modulation frequency of 1 kHz would cause +/- 2 kHz of deviation with a modulation frequency of 2 kHz.  As noted in previous postings, this is done to improve the overall signal-noise performance of the system as it boosts the "highs" in the audio at about the same rate as the noise increases on weak signals.

There are some practical issues with this pre-emphasis that must be considered.  If you were to start your pre-emphasis at 1 Hz, by the time you get to 2048 Hz would have pre-emphasized your audio by 66 dB or so (if I've done my math right) - a ridiculous amount, and any audio content that might be present at higher frequencies would be boosted even more!  Such high frequency content would also cause high amounts of deviation which, in turn, would greatly expand the occupied bandwidth of the transmitted signal - something that is neither necessary or neighborly!

Clearly, this implies that we must do two things:

• Limit the frequency range over which we do our pre-emphasis
• Filter the the audio to the desired range for speech communications

If we start our pre-emphasis at around 200 Hz, instead, we can see that by the time we get to 3200 Hz we need to boost only by 36 dB - a far more reasonable value than the 66 dB mentioned above!

For speech we need only reproduce audio from around 250 Hz to something in the area of 2500-2700 Hz.  Our low-frequency limit is imposed by our desire to include the encoding of "subaudible" tones on our transmitted signal and it is important that we remove a reasonable amount of energy in that frequency range so that spectral content of the human voice - particularly that of the adult male - does not encroach in that area and cause reliability problems with decoding on the receiving end.

Fortunately, such tools are already at hand!

Pre-emphasis:

We already met the differentiator algorithm in our receiver as it was used to reduce the low-frequency, subaudible tones from received audio.  This algorithm reproduced below.

loop:

  filtered = α * (old_input + input - old_filtered)

  old_filtered = filtered

  old_input = input

Where:
  "α" is the the equivalent of the time constant in that a "small" α implies an R/C circuit with a fast time-constant strongly affecting "low" frequencies.
  "input" is the new audio sample.
  "filtered" is the high-pass filtered (differentiated) audio

In the case of the receiver we used it as a high-pass filter with a cut-off below the speech range, but for transmit we can adjust the "knee" of this differentiator such that it is just above the speech range, instead.  As it turns out, an "α" value of 0.05 is suitable for our purposes.

Filtering:

Having done pre-emphasis, we still need to do filtering, but I'd already implemented a "transmit" filter on the mcHF for both SSB and AM and all I needed to do was redesign the filter to suit the FM audio characteristics.  I used MatLab and the filter designing plug in for this, but the "Iowa Hills" filter designer suite (free and easily found via a web search) could be used to produce suitable sets of coefficients.  As are most of the filters used on the mcHF, these filters were IIR since one can get a lot of "bang for the buck" in terms of good, "sharp" filtering with relatively few computation cycles.

With a fairly compact filter with fairly low computational overhead I was able to achieve >20dB of voice rejection in the upper frequencies used for subaudible tones and well over 50 dB of attenuation above 3200 Hz - much better than that achieved in a typical, analog FM transmitter.  At the low end, audio below 250 Hz was attenuated by at least 20dB with over 40 dB reduction for audio content below 200 Hz - this, to prevent "pollution" of the frequencies occupied by subaudible tones.

Comment:  Because I was using floating-point math, the order in which pre-emphasis or filtering is done is unimportant.  If fixed-point/integer math was used, instead, you would need to carefully analyze the signal path and the resulting values to assure that nothing "blew up" (e.g. exceeded the integer range or, at the other extreme, was so "small" that resolution was compromised and distortion/noise introduced) at the expected audio levels at all frequencies!

Limiting:

One necessary function employed in typical amateur FM transmitters is that of the limiter to "clip" the audio to an absolute maximum level.  This device improves overall intelligibility by allowing the designer to set the microphone gain to a somewhat excessive level, but the clipper forcing a maximum loudness.  The result of this is that somewhat low audio from soft-spoken users is boosted to promote intelligibility while those who have "hot" microphones and/or speak loudly do not cause excess amounts of deviation of the transmitted signal.

The mcHF does not have a clipper, per se, but it does have an audio compressor that was implemented many versions ago to improve usability on SSB.  Like a limiter, this device prevents the audio from exceeding an absolute maximum level and it also adjusts the gain upwards during quiet portions to reduce the "peak-to-average" ratio of the audio, thereby improving intelligibility.

I did experiment with both a "hard" and a "soft" limiter (or clipper) in software.  A "hard" limiter is one that sets an absolute ceiling on the amplitude of the signal present while a "soft" limiter, as the name implies, is less abrupt, more like the "knee" of a diode with some sort of logarithmic-like action.  Because they alter the waveforms, both of these methods generate harmonics and intermodulation products - the "soft" limiter being a bit less aggressive - which require that filtering be done.  Since we are low-pass filtering the audio, the higher-frequency harmonics outside the speech range will not contribute to the occupied bandwidth of the signal but the increased energy in the upper speech frequencies from harmonics of the lower-frequency audio components coupled with the pre-emphasis can somewhat broaden the signal.  Finally, because the signal is distorted by the clipping action, high audio levels that result in a lot of clipping are likely to result in audio that "sounds" degraded.

In comparing the sounds of the limiter/clipper to that of the audio compressor, I decided to use the latter as it was more "pleasing" to the ear and more versatile, overall since there are a number of available adjustments (e.g. the amount of audio into the variable gain stage and the decay rate of the variable gain stage.) As noted, I eventually decided not to use a clipper and used the already-existing compressor, instead.

Without either this compressor or a limiter, an FM transmitter would have the problem of their signal being "too wide" for loud-speaking operators and "too quiet" for those that were soft spoken - neither condition being desirable for communications!

Subaudible tone:

A desirable feature of a modern FM transmitter is that of the Subaudible Tone, discussed previously.  This signalling method consists of the generation of a low-level sine wave in the range of approximately 67 to 250 Hz that is superimposed on the transmitted audio which is used by the receiver to validate the presence of signal.  While this was traditionally used in commercial radio to allow several groups of users to "share" the same frequency, amateurs have typically used it as interference mitigation techniques to prevent the receiver - that of the repeater or the user - from responding to noise or signals for other sources.

For additional information about subaudible tone signalling, read the Wikipedia article - link.

Since it is just a sine wave, it is very easy to generate - and we already know how!

Re-using the DDS algorithm, above, we need only generate a single tone, unmodulated this time, and sum it with our transmitted audio.  We would of course, do this after we have done our pre-emphasis, filtering and limiting/compressing, placing this tone generator just before the DDS that produced our FM signal as the code snippet below illustrates.


[Filtering and limiting/clipping of audio already done]

if(tone_generator=TRUE) {
      tone_accumulator = accumulator + tone_frequency_word
      table_index = tone_accumulator > (size of tone accumulator in bits - size of table index in bits)
      tone = sine_table[table_index]
      audio = audio + (tone * amplitude)
}

[The code that follows is the DDS that generates the FM signal as shown above]

As we can see, only if the tone generator is turned on do we go through the loop - something that we'd do to save processing power.  Included in the above code snipped is an additional parameter, "amplitude" which would be used to scale the value from the sine lookup table such that it yielded the proper amount of deviation on the transmitted signal - typically in the area of 15-20% of peak deviation.

In the case of generating the audio tone we'd need to make certain that we had enough frequency resolution to accurately produce the tone, and as we already calculated we know that with a 16 bit counter at a 48 kHz sample rate our resolution is approximately 0.732 Hz. Assuming no sample rate errors, this would imply that we could generate the desired frequency to within half that resolution worst-case, or approximately 0.366 Hz.

This frequency resolution is adequate for generation of these tones, again assuming that there are no additional error sources related to sample rate, but if you were not satisfied with that amount of resolution it would be a fairly simple matter to increase the number of bits used by the accumulator and frequency word to improve the resolution, just as was suggested for the frequency modulation.

For the calculation of the frequency words, all that was required was that the code include a table containing the frequency, in Hertz, of each of the subaudible tones:  Since we already know the sample rate and the number of bits - and therefore the maximum counts for our accumulator - we can calculate, on the fly, the needed "frequency word".

Tone burst:

There is one more tone signalling scheme occasionally encountered on FM repeater systems, and that is the "tone burst", sometimes called "Whistle-up".  Although it has largely disappeared from use, it is reportedly used in some areas in Europe.

In this system a short burst of a specific tone, typically 1750 or 2135 Hz, is transmitted to "wake up" a repeater for use, and once this is done, it may be used normally.  Once the repeater has again become dormant, a timer expires and it will no longer respond to signals until it, again, receives a burst.

For a Wikipedia article that includes a section about single-tone signalling, look here:  link

This is generated in exactly the same way as a subaudible tone, namely with a bit of DDS code that looks just like the above!  From a purely practical standpoint, unless one absolutely needed to generate both a subaudible tone and a tone burst at the same time, one could actually use the same bit of code - provided that the amplitudes of the different tones (subaudible, burst) were taken into account.

Unlike a subaudible tone, a tone burst is typically transmitted only at the beginning of a transmission and for a fairly short period - perhaps one second.  While one could rely on the user to time the duration of the tone burst, on the mcHF the duration of the burst was timed by counting the number of interrupt cycles called to process the audio, making the process semi-automatic:  The user needed only activate push-to-talk and then press-and-hold the button that produced the tone and the rest would be completed automatically.

"DCS" codes:

Not mentioned previously there is one additional signalling scheme sometimes found on amateur frequencies, and that is "DCS" (Digital Coded Squelch) which consists of a binary signal with a base frequency of 134.4 Hz modulated with a specific bit pattern.  This signalling scheme is quite rare in the amateur radio community - and even rarer on HF (10 meter) repeaters where this radio is likely to be used - so there has been no serious consideration in its support


How well does it work?

Generating a sine wave with a low-distortion audio generator and feeding the modulated signal into a communications test set (a.k.a. "Service Monitor") - a device specially designed to analyze the quality of communications gear - the modulation was tested at several audio frequencies and found that the distortion was at approximately the level of detection of the instrument to at least +/- 5 kHz deviation.

Testing was also done using speech at various levels, including attempts to overdrive the audio input and on a spectrum analyzer the occupied bandwidth was observed to be contained within the expected bandwidth mask for both the "narrow" (+/- 2.5 kHz) and "wide" (+/- 5 kHz) deviation settings with no audible distortion present nor were there any unexpected spectral components outside the frequency range typical of such an FM signal - even though the "accumulator" of the frequency-modulating DDS is only 16 bits and the audio represented by it will have even lower resolution (e.g. on the order of 12 bits, maximum.)

At the present time I can't think of any additional features that would need to be added to the FM mode so it is, for now, "good to go."


[End]

This page stolen from "ka7oei.blogspot.com".

A PIC-based audio comb filter to remove AC mains hum

If you have read the Modulatedlight.org page - and realized that I had something to do with its content - you will know that one of my interests is Free-Space, through-the-air optical communications.

In our many experiments one thing that we have often run across is the presence of mains-related hum from the spillover and "glow" of city lights that can invade the audio.  In the tests that in which I have personally been involved this has not necessarily been a huge issue as our location (in the relatively sparsely western U.S.) is not completely saturated with such illumination.  Even when we have run tests across town we have been able to avoid paths that are "terribly" affected by such energy.

Figure 1:
The prototype comb filter.
Click on the image for a larger version.

Not so for some of the folks doing similar experiments in the U.K., one of the most densely-populated countries in Europe.  There, it can be difficult - particularly near populated areas - to find a location that does not have a "glow" of city lights on the horizon.

To be sure, there are at least two energy components within such glows that can cause issues:

• The modulation of the lights themselves, typically at twice the AC mains frequency.  Because of the non-sinusoidal nature of the waveforms this "hum" contains many harmonics, and because the lighting overall may be from all three phases of the electrical grid, some of these harmonics can be quite strong!
• A "hiss" caused by the thermal noise of such lighting.

While the former may be removed electronically owing to its being confined to discrete, narrowband frequencies, the latter (the hiss)  is entirely random and cannot be "notched" out in the same way as the hum or buzz of the lights:  Some sort of "noise reduction" software such as that used in modern HF transceivers could be implemented, but that is beyond the scope of this discussion - and the processors discussed here do not have the "horsepower" to implement such filtering.

Fortunately, the thermal noise generally follows the "1/F" profile - which is to say that it is more intense at lower frequencies and the energy rapidly drops off, being much less of a problem at the frequencies that we use for voice (300-3000 Hz) than at lower frequencies.  Because of the nature of human speech, our brains can typically do some "mental DSP" to reduce the deleterious effects of the noise as background noise is a fact of modern life, but the raucous noise from the buzz and hum of the harmonics is far more difficult to deal with!

Figure 2:
A typical spectra that contains "buzz" from urban lighting
superimposed atop speech from audio on an optical path in the U.K.
As can be seen, there is less energy at the mains
frequency (50 Hz) and harmonics with most of signal
power in"spikes" occurring at 100 Hz intervals.
Click on the image for a larger version.

The Comb filter:

As it turns out what we need to filter the hum/buzz and its harmonics is a "comb filter" which, as its name implies, filters out a base frequency and the harmonics.  What is also fortuitous is the fact that such filters are very easy to implement in a simple processor using fixed-point arithmetic.

The type of filter that is implemented in this case is an "IIR" (Infinite Impulse Response).  To "construct" a notch filter in software we need only do the following steps:

• Set up a buffer that will delay the "audio" data put into it by the period of the base frequency of the comb filter.  If you wanted to filter out a signal that consisted of 100 Hz and its harmonics, this delay would be 1/100th of a second, or 10 milliseconds. 
• Take the output of the delay and multiply it by a factor of n, where "n" <1.  If "n" is 0.75 for "75% feedback", you would reduce it to 75% of its original amplitude.
• Take a copy of the original signal and reduce it by "1-n".  Taking the 75% example above, this original signal would be reduce to 25% of its original amplitude.
• Invert either signal (it does not matter which) and sum the two together and put this result into the delay.
• The output is taken from the delayed signal.

In the example above we have a filter that feeds back onto itself 75% of the delayed signal with a contribution of just 25% of the original signal - with one of the two signals "inverted" to cancel out their contribution and the result is a filter that has notches at 100 Hz, 200 Hz, 300 Hz, etc.  By varying the ratio - for example, less "original" signal and more feedback - one can make the notches increasingly narrow at the expense of the filter being able to respond quickly to changes.

Practically speaking the useful values for the above range from comb filters that have as little as 50% feedback (fairly wide notches - which impart a somewhat "hollow" sound on the audio and fast response) to as much as 93.75% feedback which has quite narrow notches but is quite slow to respond to changes:  Values outside this range have been empirically tested and found to be less useful in this particular application - particularly higher values of feedback which tend to excessively slow the response.  (Note that the relative response time is proportional to the "base" frequency of the filter, so a 1000 Hz comb filter would respond more quickly than a 100 Hz comb filter with the same amount of feedback.)

This filtering is all done in "C" using the CCS compiler.  While not as streamlined as straight assembly, I've worked with this compiler for many years and have learned how to tweak it to produce "reasonably efficient" code that isn't terribly less efficient in execution speed than assembly - plus it takes far less time for me to write and debug C than assembly and the additional bonus is that the "meat" of the algorithms are portable!

For this sort of filtering all that is required in the maths are shifts, adds and subtracts:  You may note that the ratios, above (50%, 75%, 93.75%) all consist of discrete inverse powers-of-two (e.g. 93.75% = 100 - (50% + 25+ 12.5% + 6.25%) - all being numbers that can be derived by doing simple right shifts of the data!

The circuit:

Figure 3, below, depicts the schematic diagram of the prototype comb filter:

Figure 3:
The schematic of the comb filter board.
Click on the image for a larger version.

 In the circuit above the input signal is first low-pass filtered by U101A and associated components - this, to reduce the energy at and above 16 kHz, the Nyquist frequency.  The filtering in the above circuit isn't extremely "strong", but because the majority of energy is typically around the speech range of 300-3000 Hz - and the fact that the overall energy contained in the speech and noise tends to decrease with increasing frequency - it is adequate for such use.

The heart of the circuit is U102, a PIC16F1847 processor internally clocked at 32 MHz which yields a sample rate of approximately 32 kHz with full 10 bit resolution of the PWM generator, used for D/A conversion.  The processed audio output, in PWM form, is then integrated and low-pass filtered back to "baseband" audio by U101B which is also configured as a low-pass filter.  This filtering is not as "strong" as the input filtering since, in most applications, it is less important that the generated energy above the Nyquist frequency be removed to the same degree as the input filtering.

As can be seen in the diagram the input pins are used to select the various filter modes as follows:

• 50/60 Hz - This selects the mains frequency to be filtered.  60 Hz is used in the U.S. and its possessions, parts of North and South America and a few other countries while 50 Hz is used in the rest of the world.
• 1x/2x - This selects whether the "base" frequency of the comb filter is 50/60 Hz ("1x") or twice the frequency (100/120 Hz for "2x").
• Bypass - When left open (high) the filter is bypassed, but pulling this line low will enable the filter.
• Sel1, Sel2 - This selects the various types of IIR filters, each with different amounts of "feedback" as noted in Figure 3.

Also present is the "Clip" LED and related circuitry (D102, Q101, etc.) that will illuminate if the input audio is more than "half scale" (e.g. 6 dB below clipping).  If this LED flashes more than occasionally it is recommended that the input audio level be reduced, particularly if audible distortion is present.  The occasional flashing of this LED on brief signal peaks and/or noise pulses is acceptable and does not usually result in audible distortion.

So there you go:  Using a low-end PIC for DSP!

[End]

This page stolen from "ka7oei.blogspot.com".

Update on the mcHF - Adding more features (Part 2) - Implementing audio filters and fractional I/Q phase adjustments

The front panel display of the mcHF SDR transceiver, an entirely
self-contained all-mode HF transceiver based on open-source software.
This is shown not in its 3D-printed case - which is open-source, too...
(No, you don't need a computer for this transceiver to work!)

Last time (See the January 27, 2015 entry - LINK) the addition of AGC and a gain control to maximize receiver dynamic range was discussed.

This time:  A discussion on the addition of adding audio filtering, decimation/interpolation and fractional I/Q phase adjustments for both receive and transmit.


Doing audio processing with limited resources:

Being that the mcHF is entirely self-contained and has a reasonably powerful, yet modest processor (the STM32F405 or '407 processor running at 168 MHz - a device with an ARM Cortex M4 core that includes hardware floating-point support) there are practical restrictions on how much number-crunching one can get away with when performing various tasks.  As you might expect, the most time-critical task is the "real time" processing of the receive data from the dual A/D converters as I/Q channels into demodulated audio.

If one goes through the literature on typical SDRs in the amateur world it quickly becomes apparent that limited processing power is often not a prime consideration.  This is not too surprising considering the fact that multi-GHz, multimedia processors in desktop computers are ubiquitous, so processing power is considered to be "cheap" and very often a lot of processing is thrown at a problem simply because it is available!

For example, one might implement an FIR (Finite Impulse Response) audio filter with 512 or even as many as 2048 taps without batting an eye on a PC and still have resources to burn, but try that on the processor in the mcHF and you'll have suddenly used up the vast majority of processor power (if not all of it - and more!) on that one task!

So, one starts to ask various questions like:

• How can I do something while using less processor horsepower?
• How "good" is "good enough"?
• If I cut a corner somewhere, how detrimental will it really be in practical, real-world situations?

Receive signal processing:

The mcHF uses a standard, inexpensive "sound card" type codec (a Wolfson WM8731 or the compatible TLV320AIC23) for all A/D and D/A.  This is a 16 bit device capable of a number of sample rates from 8 to 96 ksps (KiloSamples Per Second), but in the mcHF it is typically operated at 48 ksps, this to allow the visualization of the spectrum +/- 24 kHz from the center as can be seen in the picture at the top of the page.

Being a typical SDR one of the first steps in signal processing is the same as that of any typical "phasing" rig - analog or digital - and that is to impart a differential 90 degree phase shift on the audio channels, a task that is, in this case, accomplished with a "Hilbert" transform.

Now, at the beginning the mcHF's audio path was "baseband" based, which is a confusing way to say that all demodulation was done around "zero Hz" or "near DC":  A demodulated CW tone tuned to yield 700 Hz was, in fact, 700 Hz away from the local oscillator frequency.  This is not an ideal situation for a number of reasons (which will be covered in a later installment!) but it was, from the beginning, the easiest thing to do.

Because the demodulation was done "near DC" this meant that the Hilbert transformer had to be of the "0 degree/90 degree" types rather than the more typical "-45 and +45" degree type:  While the former can be made to behave fairly well at low audio frequencies with a reasonable number of FIR taps, the latter cannot!  An 81-tap FIR-based Hibert transformer is used to provide the audio phase shift, providing reasonable performance down to a couple hundred Hz - as low as we need to go for SSB!

Once the two audio channels are set 0/90 degrees from each other one can then do the math to convert the two channels into USB and LSB - which is just addition or subtraction of these channels:  Which does which depends on which phase happens to have been assigned where in the hardware.  This demodulation converts the separate I/Q channels into just ONE audio channel, but it is not yet bandpass-filtered!

Working with limited resources:

IIR instead of FIR audio filtering:


Originally the code used a combination of FIR low-pass and bandpass filters with 48 taps to provide the receive audio filtering but because the receiver sample rate was 48 ksps this meant that with so few taps that it was not practical to define a very "sharp" audio filter.  To get a "properly sharp" FIR audio filter at 48 ksps would require, perhaps, 3-4 times as many taps as that and commensurately larger audio buffers and an increased amount of processor overhead, so I decided to take a different approach.

Most of my embedded programming has been using fairly low-end PIC microcontrollers (the PIC16 and PIC18 families) and I have used both FIR and IIR filters on these devices of rather low complexity since the resource of these processors are extremely limited.  Having cut my teeth on these types of filters I was comfortable enough with IIR filters that I was not scared of their (somewhat undeserved!) reputation of being "inherently unstable" and I also knew from experience that IIR filters could, when properly finessed, offer superior performance to FIR filters in situations where one needed to severely limit the amount of processor overhead, particularly when one has floating point math available.

Having access to MatLab and its filter design/simulation tools I soon had working some fairly low-complexity IIR filters thanks to the built-in support of the CMSIS DSP library (link) that offered performance that was far superior to the original FIR filters with reasonable "Shape Factors" (e.g. the ratio between the passband and attenuation response).

Decimation to reduce processor loading:

So it remained like this for several versions of code:  All of the audio processing was being done at 48 ksps but I had not yet taken advantage of another trick available to reduce processor overhead:  Decimation.

At this point I was still crunching numbers at 48 ksps - this, to produce audio that had no components higher than 4 kHz or so!  Nyquist tells us that to process such signals we need not sample at any higher than 8 kHz, so running at many times that sample rate was simply wasting CPU power!

The term "decimation" in DSP terms simply means keeping 1 out of N samples and throwing the rest away, and if you have fewer samples, there are fewer numbers to crunch.  For practical reasons - sometimes those dictated by available library functions and/or the sizes of buffers - it is usually best to pick "N" as a power-of-two (e.g. 2, 4, 8, 16, etc.) For the second of these reasons (e.g. the audio "chunk" size was 64 samples - a number that was NOT evenly divisible by 6!) my best choice was to decimate-by-four to reduce my sample rate from 48 ksps to 12 ksps.

If you throw away samples you must still do something about the audio spectral content that would be above the Nyquist limit at the new sample frequency, so one must first low-pass filter and remove those signals and that meant that I had to get rid of those signals that would be at 6 kHz and above!

Comments:

• In theory, a decimation-by-8 to yield a sample rate of 6 ksps would have worked since, for normal SSB, my maximum audio frequency could be less than 3 kHz.  The problem is that this would have placed my Nyquist limit very near my desired maximum frequency of 2700 Hz or so - and in the audible range - requiring pretty tight filtering.  The added complexity of such filtering may have countered any gains afforded by the reduction in sample rate, plus it would have precluded the use of "wide" audio filters (such as the 3.6 kHz filter) for SSB and AM unless I were to have added yet another decimation rate!  (For CW or Digital-mode filters this would probably be fine...)

The CMSIS library for the ARM M4 processor contains a handy decimation function with a built-in FIR low-pass filter, but number-crunching (with the aid of MatLab) showed that I'd need quite a few FIR taps - and additional processor overhead - to get the needed 60dB or so low-pass filtering that was required!

Fortunately, I already had a low-pass filter at hand:  The Hilbert transformer!

Using the free "Iowa Hills" filter design tools I designed a new Hilbert Transformer that was identical to what was already in there, except that it had a low-pass cut off starting at about 3.6 kHz or so - just about right for the 3.6 kHz audio filter in the radio - and with 81 taps it provided reasonable (>=55dB) worst-case low-pass filtering to prevent aliasing at and above 3.6-ish kHz, the highest audio frequency that my audio filters would pass.

At this point I'll say a few words about "appropriate" audio filtering.  I really didn't care too much if the filtering above this frequency was insufficient to prevent "audible" aliasing since it was going to be removed by the SSB filters anyway.

For example, if one picked, say, 4 kHz has the highest-frequency signal that was going to get through the 3.6 kHz audio filter (e.g. a strong CW signal down on the "skirts" of that filter) we know via math that its alias would be at 12 - 4 = 8 kHz.  What that means is that worst case, our strongest alias signal - and the "closest" to our audio passband - would be at 8 kHz, so our filter must attenuate adequately - by some 60dB or so - between the 3.6 kHz representing the "top" of our widest filter and 8 kHz.

On this point I actually "fudged" a bit:  The filtering was adequate to knock it down only by 50 dB or so, but since the 3.6 kHz filter was not going to be used very often I looked, instead, at the numbers for the "2.3 kHz" filter - which actually passes audio between about 300Hz and 2600 Hz, and its steep skirt is around 3100 Hz or so.  Taking 3100 Hz as our "new" highest frequency the alias would be at 12 - 3.1 = 8.9 kHz and this extra (almost) 1 kHz of roll-off provided another 10 dB or so on our low-pass filter.

Because the CMSIS decimation function required that I put some FIR low-pass filtering in place - that is, I could not use the function without it - I used as few as FIR taps as practical, finessing them with MatLab so that they, too, did as much filtering as they could with those few taps and achieved something in the 10-20dB area (depending on frequency) on top that achieved with the Hilbert transform, so we now had our target of at least 60 dB!

At the output of the decimator I now had to work at 12 ksps rather than 48 ksps which meant that I had to redesign all of my audio filters, but by keeping them as complex as they had been before - even though the sample rate was now lower - meant that they could now be made to be made "sharper" than before and thus offer higher-performance!

Since I had one forth as much audio data to crunch, more processor horsepower was now available to do other things, allowing to add additional features in the future!

Interpolating back to 48 ksps:

At the end of the audio filtering and AGC processing I had another problem:  The D/A converter still operated at 48 ksps so I had to do an "interpolation" step to up-convert from 12ksps.  Here, too, one must do a bit of low-pass filtering, but for a different reason:  The raw 12 ksps audio would contain aliased audio if upconverted directly to 48 ksps and this can be heard (if your ears are good enough) and could be extremely annoying!

As with the decimation, there is a CMSIS library interpolation function and it has a built-in FIR low-pass function, but there is no real need to make this filtering particularly strong.

In the worst case scenario with the 3.6 kHz audio filter selected, there will be audio content at (12kHz - 3.6 kHz = ) 8.4 kHz - but nothing below that, so a fairly weak low-pass filter with relatively few FIR taps (to minimize processor loading!) was designed in MatLab.  This filter was designed to start rolling off above 3.6 kHz and by the time it got to 8.4 kHz it was attenuating by 25dB or so and was down by 30-40 dB the time it got to 9-10 kHz, where the vast majority of the aliasing energy would be when one operated the most-used 2.3 kHz audio filter:  Because of the way that the human ear works, the "clutter" at the high audio frequencies, knocked down by 25+ dB would probably not even be noticed by someone with even the most acute hearing!

Upon getting this code operational, I fed the LINE OUT from the receiver into the Spectrum Lab program with a sound card running at 192 kHz and verified that the low-pass filtering did, in fact, reduce the aliased signals by the predicted amount.  I then routed the audio into full-range speakers and, with a graphic equalizer, intentionally boosted the highs by 10-15dB in an effort to accentuate the aliasing signal but even in a worst-case scenario with single tones the aliasing was pretty much inaudible.

Comments:

• For the 10 kHz audio filter mode, decimation/interpolation-by-2 was used and similar tricks were used with the Hilbert transformer to minimize processor loading.  In this case the interpolation's low-pass filtering was far less effective since the Nyquist frequency is 12 kHz so the output contains aliasing components that are only 6-15dB down, worst-case.  Even in this case, with a 9.8 kHz tone - with the alias at 24 - 9.8 = 14.2 kHz - the psychoacoustical properties of human hearing make it somewhat difficult to hear this tone.
• The codec chip is also capable of operating at 8 ksps, both for A/D and D/A operations.  While this would greatly reduce processor overhead, it would limit the view on the spectrum scope to just +/- 8 kHz!  If there had not been enough processing power to do what was needed to be done this may have been a necessary strategy to take, but with voice modes, it turned out not to be needed.  In the future when digital modes are contemplated and additional processing power may be needed - but a wide "spectrum scope" is not - the use of an 8 ksps rate will be considered.

Fractional I/Q phase adjustments:

One of the problems with real-world hardware is that there will inevitably be variations in the I/Q phasing and amplitude of the audio as it is processed by the two audio A/D converters.  This phase difference could be from a slight shift in the local oscillator signal or, more likely, it could be due to component variations in the analog circuitry comprising the mixer and filtering that precedes the A/D converter.  Whatever causes this problem, it must be addressed to maximize the opposite-sideband rejection.

Addressing the amplitude imbalance is pretty easy:  One simply multiplies the amplitude of the I/Q channels by a small, fractional number made adjustable by the user.   Typically both channels are multiplied by equal and opposite amounts so that the total amplitude remains generally constant.

Phase adjustment, on the other hand, is trickier!

In my researching how this is done on PC-based SDR implementations I saw that there appeared to be handy, on-the-fly calculations phase adjustments that could be done using various library functions that would transform the I/Q signals.  While such functions may have existed somewhere in the bowels of the libraries available to me, real-time calculation of the phase for each sample that came in would represent a prohibitive cost in terms of processor power!

So, how would one take care of this problem with a minimum of overhead, preferably with a one-time calculation?

It struck me that if I could modify the Hilbert transformer in a "fractional" manner I might be able to effect a fractional phase adjustment.  Since calculating the 81 coefficients internally was out of the question (I didn't want to figure out how to do that from scratch, plus it would have been a pain if I needed to modify the Hibert transformer in the future if I wanted to modify its parameters!) I wondered if I could "tweak" a fixed set of coefficients and not "break" the transformation to a significant degree?

Using the Iowa Hills program I calculated four sets of Hilbert coefficients:  0 degrees, 90 degrees, 89.5 degrees and 90.5 degrees.  In the mcHF, the 0 degree set (the "I" channel) would remain constant, but if the I/Q phase needed to be adjusted, the data from the nearest "alternate" set of coefficients for the "Q" channel would be proportionally blended.  For example, if 89.95 degrees was needed, it would use 10% of the 89.5 degree set and 90% of the 90.0 degree coefficients and this new data would be input to the Hilbert transformer.

Putting this new code into the receiver, using a signal generator, and making very careful measurements at the "worst case" settings of 89.75 and 90.25 degrees (and a few points in between) I measured only very minor amplitude (for which I could compensate!) and phase degradation in the performance of the Hilbert transformer due to these "straight line" approximations and called it good!

This method of phase adjustment is applied for both receive and transmit, but in practice it has been observed that far more amplitude adjustment is typically required than phase adjustment to effect optimal opposite-sideband rejection, at least if good-tolerance components - especially capacitors - are used in the receive and transmit paths!

Comment:

• On my mcHF transceiver I have typically required well under 1/10th of a degree of phase adjustment, so the available range of +/- 0.5 degrees is probably overkill!

What about transmit?

All of the above tricks could be applied to transmit, but thusfar, there has been no need to do any decimation/interpolation - just the Hilbert transformations, amplitude and phase adjustments, some audio processing and filtering - topics for later discussion.

For transmitting, since fewer operations need to be accomplished than in receive, everything still operates at 48 ksps with processing power to spare.  If it does become necessary in the future, I could decimate/interpolate within the transmit function and "regain" a significant number of CPU cycles!


How does the receiver sound "on the air"?

In tuning around with this receiver, the result of this processing - which is all done with floating-point arithmetic - is at least comparable to any of my other all-analog radios in terms of filter performance, adjacent channel rejection and opposite-sideband rejection:  Even under "contest" conditions with a very strong signal "next door" the receiver seems to be perfectly capable of holding its own!

While a few shortcuts had to be taken to reduce the amount of processing to an amount that could be handled by this radio, it does not seem to have demonstrably compromised its receive performance in actual, real-world conditions!



[End]

This page stolen from "ka7oei.blogspot.com".

Update on the mcHF transceiver - Adding features to the original code

For an follow-up article, see this link.

It has been a few months since I have posted anything on the mcHF transceiver  (my previous post may be found here - link) so unless you have been following the mcHF Yahoo group (link - membership required) you would be forgiven for thinking that I had abandoned work on it.

Figure 1:
The mcHF transceiver in operation, showing signals on 10 meters.
Click on the image for a larger version

Quite the contrary, a lot of features have been added to the code, starting with the fine foundation written by Chris, M0NKA based on version 0.0.181.  Considering that he pulled the disparate pieces together pretty much single-handedly, it is quite amazing what was there when I started!

Of course, this is open-source and the whole idea behind it is to allow multiple contributors:  It would be the height of hubris to think that just one person had a monopoly on good ideas, let alone the time to implement a fraction of them, so a group effort is key here.  Even if the majority of the group does not actually contribute directly to writing code, getting user-feedback is absolutely vital to knowing if one is going down the proper path and the detection and fixing of the inevitable bugs that creep in when work is done!

What is the mcHF?

I've had a few people ask me about this transceiver, wondering what it is, some apparently under the mistaken impression that it connects to a computer somehow like many SDR products.

This is, in fact, a completely stand-alone SDR transceiver capable of operating on all bands from 80 through 10 meters. *


RF goes in/out through a BNC connector, DC is applied, one listens to audio on a speaker or headphone, and "talks" with a microphone or an attached CW key/paddle/bug.  If so-desired, you can even connect it to a computer (or smart phone) and run some "sound card" digital modes like PSK31, RTTY, Olivia, WSPR, DV2 or SSTV if you like via its Line in/Line out jacks. **

The "heart" of this transceiver is the ARM Cortex M4 processor (made by ST) running at something under 200 MHz, a reasonably-powerful device, but not super powerful, which makes for some interesting challenges when one is trying to squeeze in various features - more on that in later posts.

* 160 meters is possible too, with just a bit of "hacking" - maybe just as simple as adding an outboard 160 meter low-pass filter.
** Because it is an SDR it should, in theory, be possible for the mcHF to run some digital modes stand-alone as well, but there is the problem of "too many desired features and too little time"!

Why did I do this?

You might think that I started this project knowing all about SDR and DSP.

You would be wrong.

The reason that I tackled this project was that I DID NOT  know all that much about SDR and DSP.  To be honest, I do have a programming and electronics background (but almost entirely self-taught) and I "knew" how SDR radios and DSP worked at the high level, but I'd really never gotten my hands dirty - so this was my chance to get myself drenched, not just my feet!

Not having worked with the Cortex M4 processor before or the CooCox programming environment, but being familiar with the "C" programming language in general (I've used it for many years in programming PICs and other things) I found Chris' source code to be fairly easy to follow, fairly well documented, and the programming software pretty easy to use and I was soon able compile my own code successfully - so I started adding features.

In the next few posts I'll describe the "behind the scenes" of a few of the features that I added.

* * * * *

Adding features - AGC:

As it happened, just as I got started with this project, Chris had other obligations that diverted his attention for a while, but he was happy to see progress being made:  He did make it open source, after all!

The first major change that I made was the implementation of an AGC in the receiver - something that had been missing from the beginning.

Admittedly, I had no pre-conceived notion as to how to do this in code and I purposely did not look at other open-source implementations, but since I knew exactly how an AGC circuit worked, so I simply wrote some code that emulated the rapid charging of a capacitor in the presence of a signal and the slow discharge in its absence and adjusted the "gain" of the audio path in the process simply by multiplying the "live" audio by the floating point number on-the-fly.

In a nutshell, it works like this.

In a loop, operating on each audio sample:

• Take the absolute value of the pre-AGC, post-filter audio signal.
• Multiply it by the current AGC value.
• If the resulting value is above the AGC "knee" value, reduce the AGC value quickly (e.g. the AGC "attack") by an amount proportional to the current AGC value, but if it is below the AGC "knee" (e.g. the AGC "decay") increase the AGC value comparatively slowly by a proportional amount.   The rate of the "decay" is the AGC "hang" time.
• Enforce limits to minimum and maximum AGC values.
• Multiply the audio signal by the AGC value.
• Use the linear AGC value to calculate the logarithmic S-Meter reading.

To my amazement, it actually worked the first time!

Not mentioned above is the fact that a bit of delay is added to the audio path, allowing the AGC correction to be applied "before" the signal was too high:  Doing this "look ahead" properly can completely prevent AGC overshoot when there is a (suddenly) strong signal in the passband.

Signal path gain control:

As often happens, tweaking the software leads to tweaking the hardware - which leads back to more tweaking the software.  Originally there had been an N-channel JFET across the RF signal path ("Q2" on the RF board) to provide a manual attenuation control.  The problem was that, due to an oversight it was not possible to completely bias this JFET to an OFF state as there was no way to apply a negative gate bias, so it was always causing a bit of signal attenuation (typically 6 dB) even when the attenuation control (and gate voltage) was set to "zero."

I'd also noticed that, while tuning around the bands, particularly 40 and 80, that some strong signals would cause terrible distortion so I put the transceiver on my service monitor and observed that its receiver would overload badly, with clipping of the A/D converter occurring at around -55dBm!

Ouch!

This transceiver uses a Wolfson WP8371 (which is very similar to the pin-compatible TI TLV320AIC23) which contains both a 16 stereo A/D and D/A converters.  In receive mode, the A/D converter takes the quadrature I and Q channels from the receive mixers and digitizes them, sends them to the MCU (processor) and then the D/A converter produces the receive audio.  In transmit, the A/D converter takes the transmit audio which is processed by the MCU and the I and Q channels are sent out via the D/A converter to the transmit mixers.

Upon inspecting the code I noted that the internal gain control of the codec was set to maximum which made me wonder if I could extend the dynamic range of the receiver with the codec's own gain control.

A bit of quick experimentation showed that by setting the codec gain to minimum the signal overload point of the receiver was raised from about -55dBm to around -19dBm - a significant improvement!

Note:  This "overload" level applied to signals within the A/D passband, +/- 24 kHz of center.  The "overload" level of signals beyond this range is approximately -5 dBm or so, the level at which the mixer, RF amplifiers and operational amplifiers started to compress/clip.  This is possible due to hardware-based "brick-wall" filtering within the codec chip that removes out-of-band signals beyond this +/- 24 kHz range plus the intrinsic low-pass filtering of the signal chain that attenuates signals that are much farther removed than that.

A bit more "hacking" at the code yielded the final result - an additional "AGC" loop wrapped around the main AGC loop:

• The output of the A/D converter is monitored.  If the output exceeds 1/8th full-scale, the codec gain is reduced by one "step" ("several" dB) immediately.  At this same instant other places in the receive signal chain (e.g. audio, S-meter) are rescaled by the same amount to compensate.
• If the output of the A/D converter never exceeds 1/16th full scale for "a little while" the gain is increased by one step.  In other words, when the strong signal(s) disappear, the gain is increased relatively slowly.  A lower threshold is used for the gain increase to add hysteresis and reduce the likelihood of constant "hunting".
• If the output of the A/D converter is higher than 1/4 full scale, the bottom portion of the S-meter scale - which is normally white - is displayed in red to indicate a possible overload condition.  This does not indicate a malfunction, but is mostly for informational purposes and it could be useful if one decides to override the "Automatic" codec gain control and use the "manual" codec gain control instead.
• Internally, the "front end" gain of the codec is tracked and this value is used to compensate, on the fly, those other values that require the input level to always be properly scaled with respect to the RF signal's input level.  Such parameters include the S-meter, spectrum scope/waterfall and the digitized audio from the A/D converter itself.  Where this not done the aforementioned items would "jump" as the codec's gain was adjusted, affecting their operation!

The upshot of the above is that on a "busy" band with strong signals one will occasionally see the bottom portion of the S-meter flash red, but it is extremely rare for the receiver to overload.  Even though reducing the gain of the A/D converter does reduce the ultimate sensitivity of the receiver, in practice this is almost never noticed since, on a very busy band where there are very strong signals, one would probably not "miss" an S-unit or two of sensitivity at the bottom end of sensitivity, anyway!  In other words, we are more prudently using our limited, available dynamic range!

This also meant that by changing the code and taking advantage of the hardware gain control built into the codec, the "iffy" JFET attenuator circuit could be completely eliminated as this new method of "attenuation" was much more effective, anyway!



The next time I write about the mcHF:  Implementing audio filters and fractional I/Q phase adjustments with limited processor horsepower - follow this LINK.

[End]

This page stolen from ka7oei.blogspot.com