While this discussion is focused on methods and measurements exclusively using the KiwiSDR as that is the hardware with which I am most familiar, where similar capabilities exist, these same methods can be applied to other receive platforms.
Similarly, the methods discussed on this page need not be specific to the software mentioned and may be applied any time receiver signal level measurement is needed.
Care has been taken such that at the Northern Utah WebSDR, all of the WebSDR's receivers' S-meters are calibrated to within a dB or so in the center of their target band using the methods described below.HF noise floor measurements:
In addition to facilitating the decoding and reporting of WSPR signals, the WSPRDaemon script can also divine the apparent noise floor within the receive bandwidth on some of these platforms. In the case of the KiwiSDR - where an "S-Meter" reading is available via the remote interface - this reading can be used to derive absolute measurements and such data, taken from multiple locations over disparate geographical locations, can be used to glean information about the noise on the HF bands over time.
In practice, accurate, absolute signal level readings are quite difficult to obtain owing to the issue of meaningful calibration - and the more complicated the antenna system is, the more difficult it may be to get such measurements as one must consider things like:
- Cabling losses
- Splitter losses
- Filter losses
- Calibration of the receiver
- Gain of the antenna (e.g. dBi, dBd)
The "short" version:
The ultimate goal of this exercise is quite simple:
Input a signal of known amplitude into your antenna system - say, -60 dBm - and have the noise logger report that same signal level, +/- your known antenna gain.The steps below are included in the event you may not be familiar with the methods/techniques involved.
An example receive system:
As an example, the KiwiSDR-based portion of the system in use at the Northern Utah WebSDR (a browser-enabled remote HF receiver found at sdrutah.org) will be discussed and a simplified block diagram of the signal path may be seen below:
The components of the receive system depicted above in detail:
As with any antenna system, it begins with an antenna - but for the moment, we will ignore any intrinsic gain - or loss - that the antenna may have.
- Directional coupler: At the Northern Utah WebSDR, the Mini-Circuit Labs ZFDC-20-3 is used at the main antenna input. This device has minimal insertion loss (<0.25dB) on the "through" line and there is a separate port (labeled "CPL") that is coupled at 20dB from the through line. This sort of device is typically used to extract a small sample present on the line while minimally affecting it and "in" and "out" ports are labeled for such purposes, but here it is being used in reverse to inject a signal into the receive signal path.
This device is convenient in that it has the advantage that it may be left in line all of the time with negligible effects at HF and it can put test signals into the receive signal path without affecting off-air performance.
This device is optional in that the same measurements may be obtained without it by connecting the signal source directly to the antenna port - the obvious disadvantage being that the signal path is interrupted - something that we try to avoid on a busy, multi-user system such as a WebSDR.
- Calibrated signal source: It should be no surprise that having a known-accurate signal source is an absolute necessity when it comes to calibrating the signal level readings with available levels in the range of -60 to -30dBm being the most useful. Having access to such devices (e.g. Communications Test Set or RF Service Monitor) is helpful - but other means of obtaining a usefully-accurate signal include:
- An "inexpensive" calibrated signal source like the Elecraft XG-3 - the >$300 price tag (when fully equipped) being the reason for the quotes.
- A "transfer" measurement from a signal source with a consistent level. Some antenna analyzers - including the popular "NanoVNA" - can output a level that is fairly consistent. Coupled with outboard attenuators (a switchable step attenuator and/or an assortment of fixed attenuators) one can use known-accurate test equipment (an RF power meter, signal level meter, spectrum analyzer) to obtain a table of signal power readings for each of the HF amateur bands.
- Because of the harmonic content of the NanoVNA, it is preferred that a selective means of measuring RF power like a spectrum analyzer be used rather than radiometric detector like a power meter - but the difference is unlikely to be significant.
- Important note: The amplitude of the NanoVNA's output port is too high to be directly used as a calibration signal - especially if amplification is used in the signal path. Expect to need attenuation in the range of 20 to 50 dB to provide a usable signal level: It's recommended that one use a switchable step attenuator and/or a series of fixed attenuators to get reasonable signal levels.
- A transfer measurement from a piece of equipment that is likely to be "pretty close" - such as the KiwiSDR itself: With the default "factory" S-meter calibration level of -13dB the KiwiSDR to which I have access have typically been within a couple of dB when comparisons were made.
- For calibrating, a signal source (e.g. NanoVNA) would be connected directly to the input of the receiver - via the attenuator(s) - and the S-meter levels noted for different settings on each of the amateur bands.
- AM BCB Filtering: In many cases, strong AM broadcast band (mediumwave) signals will be present on the antenna port. Even if these signals are not strong enough to overload the RF amplifier that follows, such signals can overload the receiving device downstream. As mentioned later, there are advantages to strategically attenuating strong signals to minimize the total power impinging on the A/D converter of any direct-sampling receiver.
The AM broadcast-band filter used at the Northern Utah WebSDR is a rather complicated affair (and is described in this article: Managing HF Signal Dyanmics and preventing overload... (link)). Its main purpose is to reduce the amplitude of several very strong (50kW) local transmitters to prevent the "wasting of amplifier power and A/D bits" on very few signals.
- Amplification (post AM/BCB filtering): In general, a 15 dB system noise figure is sufficient to allow the detection of a 10 meter signal above even the lowest expected level of background noise found at this frequency. For this reason it is suggested that an RF amplifier be placed after the AM BCB filter (to minimize the probability of overload and intermodulation/mixing products). Remember: Once loss appears in a receive system due to loss/attenuation, no amount of amplification can restore the system noise figure beyond that point.
Such an amplifier should also be placed prior to any RF splitting as much as is practical as any loss - which can be from the feedline, in filters or in a splitter - will directly contribute to the the system noise figure. In cases where very strong signals may be present on the antenna system (local AM BCB, strong Shortwave Broadcast signals - when the bands open) it is preferable to have an amplifier at this point that has only modest gain - in the area of 12-15 dB: A high-gain amplifier (>20dB) may be more prone to overload in such conditions and the typical means of preventing this (addition of an attenuator in front of the amplifier) is not recommended as this will increase system noise figure.
- Limited attenuation high-pass filter: Any receive system is capable of handling only a certain amount of total signal input power - and the direct-sampling receiver of the KiwiSDR is no exception. The overload level of the KiwiSDR's 14 bit A/D converter is approximately -13dBm, so the total power arriving at its antenna port should be kept below that level and strategic, selective attenuation of local, strong carriers - often those of AM/mediumwave broadcast transmitters - should be considered - but there is another problem:
The sensitivity of a KiwiSDR is approximately -155dBm (in a 1 Hz bandwidth) over its frequency range which means that in a unity-gain HF antenna system that at 10 meters, the KiwiSDR will be "short" by at least 6-8dB from hearing the textbook "rural quiet" noise floor on that band, requiring 10-15 dB of overall amplification to place the 10 meter noise floor comfortably above that of the receiver's in that environment. This amplifications has a cost: With the addition of a 12 dB RF amplifier, the KiwiSDR will now overload at -25dBm. It is not uncommon for the absolute power level of a single high-power shortwave broadcast station to exceed this level under good band conditions - and there will be many such signals on the bands - not including the wide-band energy from lightning static - that can combine in overall signal power and exceed this overload level.
What system noise figure is required for HF reception?
This is a tricky question to answer because conditions on the HF band vary so much - most strongly on a per-band basis.
Assuming quiet (e.g. "dead"), bands the lack of storm static. the absence of solar noise and the receiver being located in an area completely absent of man-made noise, noise figures equal to or lower should be attained for the receive system to have a "natural noise" limit:
Were the KiwiSDR being used over a very narrow range of frequencies - such as the case for most commercial direct-sampling receivers like the Icom IC-7300 and IC-7610 - this could be managed by gain adjustment in the signal path, but if the KiwiSDR is to be used over the entire HF spectrum, this cannot readily be done. Instead, one can selectively insert attenuation where the signals+noise are likely to be the highest, namely below approximately 10 MHz and adjust calibration accordingly.
The use of this "limited attenuation" high-pass filter leaves signals above 12 MHz alone but offers significant attenuation (about 12 dB - around 2 "S" units) to signals below 8 MHz. Because of the relatively high noise levels in the lower HF bands, additional attenuation - and the commensurate increase in system noise figure - can be tolerated without the loss of useful sensitivity.
Note: This same issue of overload will occur with similar devices - such as the Red Pitaya - if operated as wideband HF receivers, without input band-pass filtering. Simple math will indicate that even if the KiwiSDR had a 16 bit A/D converter - which would theoretically yield another 12 dB of dynamic range - one would still need to take these same precautions. It is for this reason why all well-designed direct-sampling receive gear has narrowband filtering that precedes its A/D converter.
Unfortunately, the "limited attenuation" filter fits a niche requirement as is not a commercially-available device, but one may be built without too much difficulty: Two versions are described here:
- A limited-attenuation high-pass filter for the KiwiSDR - link. This filter starts to attenuate below 12 MHz, offering about 12 dB below 8 MHz.
- Revisiting the limited-attenuation high-pass filter - link. This version is similar to that above, but its attenuation pass range is limited to the range of approximately 1700 kHz to 12 MHz, allowing the passage of signals in the AM broadcast band and below for the reasons discussed in the article.
- Because of the simplicity of these filters they exhibit a low return loss (high reflectance) at lower frequencies and it is recommended that they be preceded with an amplifier to present the input feedline with a more consistent "50 ohm" impedance.
- Amplification (post high-pass filter): Where high signal levels are likely to exist it is prudent to distribute the signal gain across several blocks rather than have a single high-gain amplifier near the "front" of the signal path. An amplifier placed after the "limited attenuation" high-pass filter will see lower signal levels from the very strong lower HF-band signals and be less subject to overload.
- Splitter: A 4-way splitter is depicted in figure 1. A real-world 4-way splitter will have 6.5-7.5 dB loss between the input and any of its output ports which means that for the higher-frequency bands it is particularly important to have established the system gain and noise figure prior to the splitter. Like any other block of gain or loss, the impact of such a device must be considered when evaluating the system overall.
Again, we are characterizing the signal path beyond the antenna and ignoring any gain/loss of the antenna itself for the moment.
A procedure for obtaining the calibration of the receive signal path is as follows:
- Input a known signal level at the input of the receive signal path. This signal would be input at the antenna port - or via the "coupling" port of the directional coupler. This signal should be input at a frequency in the band of interest as calibration values should be obtained for every amateur band to be used. For WSPR operation it is desirable that one generates test signals within the WSPR passband.
- Tune in the signal and note the signal level on the receiver. A known signal level of at least 20dB (30dB is better) above the receiver noise floor and other signals in the receiver passband is suggested to minimize effects of the noise and signals - but the level should be no higher than necessary to avoid overload/compression of any amplifiers - or the receiver itself.
- If using a directional coupler as suggested, make sure that it is oriented correctly by comparing the signal levels when the generator output is fed directly into the receive signal path and again with signals fed via the coupler: If its "in/out" is reversed the coupling level will be inaccurate. (Again, for a coupler used for extracting a signal from a coax, the connection will be reversed.)
- In the case of the 20 dB coupler suggested above, the signal levels reaching the receiver will be 20 dB lower due to the coupling loss than that if the signal generator coupled directly and this should be remembered when taking measurements.
- If the levels of the test signal are low (less than 20dB S/N) and cannot be increased, narrowing of the receive passband can minimize the effects of noise - just observe the S-meter while making such adjustments to verify that the test signal remains within the passband.
- Because we are measuring the amplitude of a coherent signal within a finite bandwidth rather than trying to measure the noise power in a specific bandwidth, we must make sure our test signal is strong enough that the power of the background noise is a small percentage of the total.
- While it is necessary to compensate for detection bandwidth when measuring the noise within that bandwidth, one does not do so for coherent signals as all of the power of that signal is (theoretically) all on a single frequency. It is for this reason that the measurement obtained of the coherent signal should match the actual power of that signal.
- Knowing the signal level being input to the signal path, observe the S-meter reading on the KiwiSDR on a per-band basis. It is not important that the S-meter read correctly - except for the benefit of users connected to the KiwiSDR and looking at the meter - as the levels reported by wsprdaemon will be adjusted in its calibration file.
- If you are running wsprdaemon and are producing a signal within the WSPR passband, the signal will show up on the graphs after several minutes as a constant level - and this level can be used to verify calibration.
- The level displayed on the graph will be that of the KiwiSDR plus the amount of attenuation/amplification specified in the wsprdaemon.conf file for that band. Use either raw data or the interactive Grafana data to read the signal levels precisely.
- At the Northern Utah WebSDR, the KiwiSDR S-meter calibration setting has been adjusted so that for bands above the roll-off of the high-pass filter (e.g. 20-10 meters) the S-meter read the same as the input signal level. At lower HF bands, the S-meter reads low by the amount of attenuation of the filter. Since there is currently only one S-meter calibration point that applies to all frequencies, there is nothing to be done to correct this.
- Record, for each band, the amount of RF you are applying at the antenna port (taking into account the coupler, if used) and the S-meter reading on the KiwiSDR.
- The difference between these two readings will be the amount of gain - or loss - in your signal path. Again, we are not concerned about the absolute gain of the antenna itself at this point.
- If the reading on the KiwiSDR is above that of the absolute input level, signal gain is indicated at that frequency.
- If the reading on the KiwiSDR is below that of the absolute input level, signal loss is indicated at that frequency.
- We have now characterized the gain/loss of the signal path. If our antenna had unity (0 dB) gain we would use the numbers that we'd just obtained as an offset in the wsprdaemon.conf file. For example:
- If -50dBm were present at the input of the signal path and the KiwiSDR read -38dBm on 10 meters, we would know that we had 12 dB gain at that frequency. To compensate we would put -12 db in the entry for 10 meters in the wsprdaemon.conf file.
- If -50dBm were present at the input of the signal path and the KiwiSDR read -52 dBm on 80 meters we would know that we had 2dB attenuation at that frequency. To compensate we would put 2 dB in the entry for 80 meters in the wsprdaemon.conf file.
- Again, to verify our calculations we can place a carrier of known signal level within the WSPR passband for the band being tested. If the measurements have been done correctly the levels on the graph should match those of the level of the RF at the input of the signal path.
The next step would be to include the losses of any coaxial cable in the system before the point in the signal path where we injected our test signal. This can be directly measured with a wattmeter and dummy load being used to measure the power lost over its length or - presuming a fairly low VSWR - can be estimated using the manufacturer's loss values on a per-band basis.
These loss values would be added to the per-band numbers in the wsprdaemon.conf file.
Compensating for antenna gain:
With simple equipment, calibrating the signal path for an absolute level is pretty easy - but the gain of the receive antenna at a given frequency is more difficult to determine as very few people have access to a known-accurate antenna range - so we will have to make an educated guess. Because many HF antennas have definite patterns of of lobes and nulls one will have to decide if the peak gain of the pattern will be used, or some sort of averaged gain value.
For simple wire antennas, the gain can be estimated with reasonable (+/- a few dB) accuracy. For example, a half-wave dipole above typical ground will likely be in the area of 1-2 dBi gain peak while the gain of a commercially made vertical or Yagi antenna might be available from the manufacturer's data sheet.
For multi-wavelength wire antennas such as long wires or rhombics, antenna simulation programs are suggested and may be the only reasonable means of estimating gain - but again, one will have to make a decision on what gain value to apply in light of the likely existence of lobes and nulls.
Active HF antennas:
If the feed from a conventional antenna (beam, dipole) includes an amplifier, the steps mentioned above may be used to estimate its gain and one need only add the loss of the feedline and gain of an amplifier to obtain the desired offset for the wsprdaemon.conf file.
Electrically-short active antennas such as the PA0RDT mini-whip pose a particular problem. They are essentially a capacitive plate (or wire) that couples RF from the "aether" - but the amount of coupling (which correlates with the apparent gain of the antenna) can vary widely, the determining factors including:
- Height above "ground". In general, the higher above the ground - earth or metal - the more signal such an antenna will intercept.
- Type of "ground". The ground above which the antenna is mounted could be earth (dirt. soil) which is lossy or it could be a metal roof (much preferred!) to which the feedline's ground is bonded.
- Size of the "antenna". The traditional PA0RDT antenna uses a PC board plate for coupling - but there are many variations on what size of coupling conductor - wire or plate - is used, all of which can affect the apparent gain.
- Gain response of the amplifier with frequency. In most - if not all - implementations of an E-field whip are subject to gain drop-off with higher frequency and this is not easy to measure directly and determining a suitable number is almost entirely guesswork.
- Unfortunately, such drop-off conspires with the high RF levels at low-medium HF frequencies. With a broadband, direct-sampling receiver like the KiwiSDR, overload will surely occur if enough gain is added to the system to attain the natural noise floor at the higher HF bands (e.g. 10 meters) unless strong compensation (e.g. something like the high-pass filter described above) is used.
What to expect:
In theory, the noise floor one sees should generally reflect that predicted in the ITU-R data, reproduced in figure 2, below:
Cross-checking noise level measurements with the receiver:
It is possible to check the noise level measurement provided by wsprdaemon using the receiver in question, but to do this two conditions must be satisfied:
- You must know the receive bandwidth. For most SDRs this is pretty easy: The "bandwidth" setting - which is typically at the -3dB or -6dB points - is "fairly close".
- The passband must contain ONLY noise and no other signals. On the WSPR frequencies this may not be possible, but measuring on a "nearby" frequency (up to a few 10s of kHz away, provided that such measurements aren't affected by narrow band-pass filters that might be present).
- Wider (SSB) bandwidths are fine, but it may be necessary to user narrow bandwidths to avoid signals within the passband.
- Note the average signal level reading (in dBm).
- Since we are looking for what amounts to a rolling average over time, the readings of occasional noise bursts must be considered which means that a "fast" AGC should be used and one should not be looking at any "peak" readings.
- Since we are likely doing a "sanity check" simply staring at the signal meter and guesstimating the average.
- It is best to do this during "quiet" conditions on a given band - that is, when it is dead: Trying it on 80 meters during the night where there may be a lot of summer static is not recommended.
- Note the receive bandwidth.
- On an SDR, this is likely to be the "bandwidth" figure which is probably taken from either the -3dB or -6dB points - which will be "close enough.
- Calculate the bandwidth noise power ratio.
- Because wsprdaemon's noise power measurements are scaled for 1 Hz detection bandwidth, the reading obtained on any receiver with a wider noise bandwidth than that will be much higher and the reading (in dB) must be offset by that amount.
- As an example: If you are using a 500 Hz receiver bandwidth (e.g. a CW filter) you need to account for a 500:1 power ratio which is:
- 10 * log10(500) = 26.99 = 27 dB - that is, one would subtract 27 dB from the average level that you observed.
- In other words, to scale to the 1 Hz wsprdaemon bandwidth:
- Your signal level - (10 * log10(RX bandwidth in Hz))
While it is unlikely that the typical amateur will be able to be absolutely confident of the accuracy of their absolute HF noise floor measurements, the steps above will greatly limit the uncertainty of the system's measurement to the the properties of the antenna itself.
This page stolen from ka7oei.blogspot.com
Than you for a thorough, insightful, and hence very valuable, guide to what is a really thorny problem. I've posted a link to this blog from the brand new wsprdaemon.org website to point those wishing to use the calibration offsets facility in wsprdaemon.conf to this post for detailed guidance.
1. You've convinced me that a narrow-band calibrated signal source has useful advantages over the calibrated noise source I use here (antenna to relay switching unit, so the Kiwi gets the antenna, 50 ohms or the 22dB ENR noise source). I have Python code that does the switching and measurements; the 50 ohm load and noise source are each on for 350ms in the WSPR end-gap. Advantage of noise source is 300microamp consumption so battery powered to minimise additional noise.
2. While you mention height above ground and ground conductivity for Active Antennas for a passive dipole your 1-2dBi (with no comment on ground or height) may need expanding? My own experience with an active balanced dipole is that, when vertical, the 'gain' variation with height is much smaller when vertical than when horizontal - in the lower 0.25 wavelength above ground.
3. Thank you for the details on implementing these methods at the Northern Utah SDR site. Your notes have helped me understand better your distribution system and how you configure offsets in wsprdaemon.conf. But I am still not sure I have a full grasp. Specifically, you wrote:
that for bands above the roll-off of the high-pass filter (e.g. 20-10 meters) the S-meter read the same as the input signal level. At lower HF bands, the S-meter reads low by the amount of attenuation of the filter.
Now I do see a 'breakpoint' between 30m and 20m in the noise level differences between KPH and NUT, from NUT 17-20dB above KPH at 160m-30m and 6-13dB above KPH at 20m and shorter. It is the direction of the offset that concerns me - if the S meter is to read low for 160m-30m due to the filter loss, then why is the noise level offset greater at those wavelengths? I'd have expected it to reduce.
Thank you again for treating a topic that needs very careful attention is such a comprehensive and clear manner.
Regarding #3 - and presuming that I have understood your question:
Envisioning the calibration as a two-step process, one would:
- Input a known CW signal at the antenna input (in the middle of the WSPR passband) and adjust the KiwiSDR S-meter offset and the offsets in wsprdaemon.conf so that the signal reading from the wsprdaemon graphs was precisely that of the input signal.
- Having established a baseline, one would then adjust the values on a per-band basis in wsprdaemon.conf so that they would offset for the known gain of the antenna being used.
* * *
In the case of the Northern Utah WebSDR Kiwis, since the "limited attenuation high pass filter" has practically no attenuation (<1dB) at, say, 10 meters, the S-meter on the Kiwi has been adjusted so that it reads precisely that as the known input signal level. Because the high-pass filter mentioned above has, say, 12 dB of attenuation at 40 meters, the KiwiSDR S-meter would read low by precisely the amount of attenuation of that filter: Because the KiwiSDR has no provisions for multiple calibration points on the S-meter for various frequencies, this is unavoidable
This offset is, of course, irrelevant as that would be compesated in wsprdaemon.conf.
* * *
At the Northern Utah WebSDR, the peak gain of the antenna (TCI-530) is around 6 dB at the strongest lobes, but closer to unity gain across most of the pattern (and at lower angles) so an antenna gain of unity is assumed for the frequencies covered by the antenna (3-30 MHz). For the lower bands on that antenna (160, 630 meters) one has to estimate the gain and the numbers that we have are less certain. It just so-happens that when the band is quiet, the estimated gain is pretty close to the predicted ITU curves, so that gives us a least a small bit of confidence that the numbers aren't totally bogus.
Practically speaking, we've made a few (minor) changes to the RF distribution system at Northern Utah and I need to run the calibrations again - but it's not terribly high on the priority list during the site visits, and since it's 130km each way and a 75 minute drive) it hasn't been done recently. I do have a remote signal generator on site, but I have yet to make a calibration chart for it to allow calibration to be checked at a distance.
Thanks for writing down and detailing some of these questions around system calibration. Certainly if we are to have a multitude of systems, widely separated and disparate, getting to a common understanding is crucial to useful data.
It seems to me that the antenna problem is the thorniest. That is that essentially getting useful and accurate antenna factors such that we can compare absolute noise and signal levels at a site is truly difficult. In addition to the issues you touch on, I think that the multitude of incoming polarizations and arrival angles and the effects of local and far field grounds are daunting. Even ITU doesn't account for this in their noise data and it is certainly relevant.
In the (very little) experimentation I've done trying to identify and measure galactic noise at upper HF I find myself cautious about accepting as high a noise floor as ITU suggests - at least at some locations and seasons.
In the past it's been possible to watch the noise floor rise as the MUF was rising through 20 MHz. I've seen the propagated noise vary with QSB at the same rate as WWV's signal from Fort Collins, one hop away.
There's also some reason to think that we may be able to detect a diurnal change in noise that is correlated to Sagittarius rising though this needs more examination. It's been my thought that perhaps the only common, calibrated far-field noise source we all have is galactic noise but that is certainly a challenge to achieve across the large number of potential receive sites.
Thanks again for putting this all down and perhaps we'll be able to mull over ways of improving all of our systems' ability to make useful measurements.
I've mulled over the antenna gain problem. We are fortunate that with the TCI-530 we have reasonable knowledge of the gain and pattern characteristics and we have to go on faith that they are correct - at least over the 3-30 MHz range.
As you infer, when it comes to any practical antenna that has lobes and nulls, I suppose that knowing the noise power becomes even more difficult for obvious reason: Is the noise constant in all directions and elevations? Of course not, but what was one do about that? What about a Yagi where one is ignoring the cumulative noise power arriving from azimuths/elevation other than the peak of the pattern?
All of this shouldn't divert attention from what we *can* control - which is the gain of "everything else" besides the antenna, of course.