Log checking for several recent contests that used the FT4 and FT8 modes has shown undesirably large numbers of claimed QSOs that receive not-in-log ("NIL") status from the other station. The WSJT development team has worked together with contest sponsors and log checkers to analyze the probable causes of these NILs. Our findings and some operating advice for future contests:
FT4 and FT8 Contesting Since December 2018 there have been three major HF contests explicitly for the FT4 and FT8 protocols: the 2018 FT8 Roundup, the 2019 WW-Digi contest, and the 2019 FT Roundup. In addition, FT modes have been widely used in multi-mode VHF contests, especially the ARRL January and June events, and in the 2019 and 2020 ARRL RTTY Roundups. At this time it seems worthwhile to take stock of the state of FT4 and FT8 contesting and to address one particular issue: the undesirably large rate of NIL (“not in log”) QSOs found during the logchecking procedures. To this end, the WSJT development team has worked together with contest sponsors and log checkers for the three HF contests mentioned in the first sentence. This brief write-up is a summary of our findings, and includes some advice that could help to reduce your NIL rate in future contests. For input data we used the scoring results for all submitted logs and examples of the WSJT-X journal file ALL.TXT for a few participating stations. This file contains a complete record of all messages decoded or transmitted by the operating station. Let us first summarize the basic statistics. The number of logs, claimed QSOs, reported NILs, and average NIL rate for the three contests are listed in the summary table below. The final column shows median NIL rates for logs with scores greater than 10,000 points. The rates of “busted” (incorrectly copied) calls and exchanges are essentially zero, as expected for protocols like FT4 and FT8 with strong forward error correction. Avg Median Contest Logs QSOs NILs NIL% >10,000 ----------------------------------------------------- 2018 FT8 Roundup 1253 127,340 5803 4.6% 3.6% 2019 FT Roundup 880 134,038 6110 4.6% 3.5% 2019 WW-Digi 1328 178,906 9669 5.4% 3.1% For comparison, we note that average NIL rates in recent major RTTY contests have been around 2%. As specific examples, the 2019 CQ WW RTTY contest averaged 1.9% NILs, 1.0% busted calls, and 0.5% busted exchanges, for a 3.4% total error rate; preliminary results for the 2020 CQ WPX RTTY contest show 1.8% NILs, 0.9% busted calls, and 1.4% busted exchanges, for a 4.1% total error rate. Median NIL rates for these two events were 1.6% and 1.4%, respectively. In nearly all cases we examined, FT4 and FT8 QSOs logged by one station but not the other were legitimate partial QSOs, at least. Callsigns were always exchanged correctly, and in most cases the required exchanges were copied as well. The most common cause of NIL reports is that one station transmitted RR73 and logged the QSO without adequate confidence that the QSO partner will copy that transmission. In traditional modes like CW, SSB, and RTTY operators use hints involving the timing of transmissions, signal strengths, QRM conditions, and narrow-band filtering to gain useful information on successful QSO completion. A timed-sequence mode with structured messages and all-or-nothing decoding may require additional information of a different sort for full confidence that a QSO is complete and will be logged at both ends. Most important is careful operator attention to what is sent and received, and the context of these messages. For example: Did the QSO involve repeat requests? Was anything copied from my QSO partner after I sent RR73? Is his signal weak, perhaps close to the decoding threshold? Is my signal strength likely similar to his, or stronger, or weaker? Is his frequency clear? Many NILs are created by operators trusting too much in the optional, partially automated message sequencing and logging features provided by some software. An operator may reasonably decide to log a contest QSO with station “X” when all of the following are true: 1. X has called me (in other words, I received a message containing my callsign followed by X) 2. I have called X (I sent a message with X followed by my callsign) 3. I have received contest exchange from X 4. I have received Roger from X, so I know X has copied my exchange 5. I have sent Roger to X Note, however, that no fixed set of logical rules can ensure that the last transmitted Roger has been received. A final message with 73 following reception of RR73 is sometimes sent in the hope that it will add confidence, but this can slow down a run of QSOs and is certainly not a requirement. A human decision to log a QSO — or to allow a software controlled logging step to proceed, based on the overall context — is always best. In these first-run contests many operators were still gaining experience with the FT modes, especially under contest conditions. With increased skills there is good reason to expect lower NIL rates in future contests, especially now that some attention has been focused on the matter. Already in the 2019 WW-Digi contest there were three “Golden Logs” with more than 100 QSOs, scores over 10,000, and zero NILs, and 15 such logs with fewer than 1% NILs. Our examination of ALL.TXT files shows that avoidable human error has likely been responsible for the majority of recorded NILs. Blind reliance on a software package’s automatic logging features is never a good strategy. As a general set of guidelines for logging in FT4/FT8 contests we recommend the following: • In WSJT-X, activate and learn to use the Alternate F1-F6 bindings selectable on the Settings | General tab. • Always log a QSO when you have received RRR, RR73, or 73 from a station you are working. • Log a QSO when you send RR73 if you are reasonably confident it will be copied. But be sure to watch for any indication that it was not copied, and then take appropriate action. For example, if you receive the Tx3 message (R plus contest exchange) again, hit F4 to re-send your RR73. Steve Franke, K9AN, Don Hill, AA5AU, Ed Muns, W0YK, Iztok Saje, S52D, and Joe Taylor, K1JT February 25, 2020
will also appear in the May-June 2020 issue of NCJ, the National Contest Journal.
73 from the authors of the study:
Steve Franke, K9AN
Don Hill, AA5AU
Ed Muns, W0YK
Iztok Saje S52D
Joe Taylor, K1JT
FT4 and FT8 Contesting Since December 2018 there have been three major HF contests explicitly for the FT4 and FT8 protocols: the 2018 FT8 Roundup, the 2019 WW-Digi contest, and the 2019 FT Roundup. In addition, FT modes have been widely used in multi-mode VHF contests, especially the ARRL January and June events, and in the 2019 and 2020 ARRL RTTY Roundups. At this time it seems worthwhile to take stock of the state of FT4 and FT8 contesting and to address one particular issue: the undesirably large rate of NIL (“not in log”) QSOs found during the logchecking procedures. To this end, the WSJT development team has worked together with contest sponsors and log checkers for the three HF contests mentioned in the first sentence. This brief write-up is a summary of our findings, and includes some advice that could help to reduce your NIL rate in future contests. For input data we used the scoring results for all submitted logs and examples of the WSJT-X journal file ALL.TXT for a few participating stations. This file contains a complete record of all messages decoded or transmitted by the operating station. Let us first summarize the basic statistics. The number of logs, claimed QSOs, reported NILs, and average NIL rate for the three contests are listed in the summary table below. The final column shows median NIL rates for logs with scores greater than 10,000 points. The rates of “busted” (incorrectly copied) calls and exchanges are essentially zero, as expected for protocols like FT4 and FT8 with strong forward error correction. Avg Median Contest Logs QSOs NILs NIL% >10,000 ----------------------------------------------------- 2018 FT8 Roundup 1253 127,340 5803 4.6% 3.6% 2019 FT Roundup 880 134,038 6110 4.6% 3.5% 2019 WW-Digi 1328 178,906 9669 5.4% 3.1% For comparison, we note that average NIL rates in recent major RTTY contests have been around 2%. As specific examples, the 2019 CQ WW RTTY contest averaged 1.9% NILs, 1.0% busted calls, and 0.5% busted exchanges, for a 3.4% total error rate; preliminary results for the 2020 CQ WPX RTTY contest show 1.8% NILs, 0.9% busted calls, and 1.4% busted exchanges, for a 4.1% total error rate. Median NIL rates for these two events were 1.6% and 1.4%, respectively. In nearly all cases we examined, FT4 and FT8 QSOs logged by one station but not the other were legitimate partial QSOs, at least. Callsigns were always exchanged correctly, and in most cases the required exchanges were copied as well. The most common cause of NIL reports is that one station transmitted RR73 and logged the QSO without adequate confidence that the QSO partner will copy that transmission. In traditional modes like CW, SSB, and RTTY operators use hints involving the timing of transmissions, signal strengths, QRM conditions, and narrow-band filtering to gain useful information on successful QSO completion. A timed-sequence mode with structured messages and all-or-nothing decoding may require additional information of a different sort for full confidence that a QSO is complete and will be logged at both ends. Most important is careful operator attention to what is sent and received, and the context of these messages. For example: Did the QSO involve repeat requests? Was anything copied from my QSO partner after I sent RR73? Is his signal weak, perhaps close to the decoding threshold? Is my signal strength likely similar to his, or stronger, or weaker? Is his frequency clear? Many NILs are created by operators trusting too much in the optional, partially automated message sequencing and logging features provided by some software. An operator may reasonably decide to log a contest QSO with station “X” when all of the following are true: 1. X has called me (in other words, I received a message containing my callsign followed by X) 2. I have called X (I sent a message with X followed by my callsign) 3. I have received contest exchange from X 4. I have received Roger from X, so I know X has copied my exchange 5. I have sent Roger to X Note, however, that no fixed set of logical rules can ensure that the last transmitted Roger has been received. A final message with 73 following reception of RR73 is sometimes sent in the hope that it will add confidence, but this can slow down a run of QSOs and is certainly not a requirement. A human decision to log a QSO — or to allow a software controlled logging step to proceed, based on the overall context — is always best. In these first-run contests many operators were still gaining experience with the FT modes, especially under contest conditions. With increased skills there is good reason to expect lower NIL rates in future contests, especially now that some attention has been focused on the matter. Already in the 2019 WW-Digi contest there were three “Golden Logs” with more than 100 QSOs, scores over 10,000, and zero NILs, and 15 such logs with fewer than 1% NILs. Our examination of ALL.TXT files shows that avoidable human error has likely been responsible for the majority of recorded NILs. Blind reliance on a software package’s automatic logging features is never a good strategy. As a general set of guidelines for logging in FT4/FT8 contests we recommend the following: • In WSJT-X, activate and learn to use the Alternate F1-F6 bindings selectable on the Settings | General tab. • Always log a QSO when you have received RRR, RR73, or 73 from a station you are working. • Log a QSO when you send RR73 if you are reasonably confident it will be copied. But be sure to watch for any indication that it was not copied, and then take appropriate action. For example, if you receive the Tx3 message (R plus contest exchange) again, hit F4 to re-send your RR73. Steve Franke, K9AN, Don Hill, AA5AU, Ed Muns, W0YK, Iztok Saje, S52D, and Joe Taylor, K1JT February 25, 2020
will also appear in the May-June 2020 issue of NCJ, the National Contest Journal.
73 from the authors of the study:
Steve Franke, K9AN
Don Hill, AA5AU
Ed Muns, W0YK
Iztok Saje S52D
Joe Taylor, K1JT