Four of the five Green Hills port tests cannot pass, because the golden output
file each one compares against is not in the tree.
How the test works
test/ports/azrtos_test_tx_ghs_<core>.bat runs the ThreadX demo on the Green
Hills simarm simulator under MULTI, replaying a .rc script that sets
breakpoints and dumps the demo's thread counters to test.log. It then compares
that capture byte for byte against a committed golden file:
%AZRTOS_GHS_MULTI_MULTI% -connectfile sample_threadx.con -connect="Simulator" ^
-p %AZRTOS_TEST_DIR%\azrtos_test_tx_ghs_cortex_m3.rc sample_threadx
COMP test.log %AZRTOS_TEST_DIR%\azrtos_test_tx_ghs_cortex_m3.valid.log /M /L /A
IF %ERRORLEVEL% NEQ 0 (
ECHO Test results are invalid!
EXIT /B 1
)
The assertion is a real one: the golden file records each of the eight demo
threads' counters at two sampling points, so a scheduling regression moves the
numbers and the comparison fails.
thread_0_counter = 0x00000000 ... thread_0_counter = 0x00000005
thread_1_counter = 0x00000000 ... thread_1_counter = 0x00003695
thread_2_counter = 0x00000000 ... thread_2_counter = 0x0000363f
The gap
All five cores have the harness script and the .rc playback file. Only
cortex_m3 has a golden log.
| core |
.bat |
.rc |
.valid.log |
| cortex_m3 |
yes |
yes |
yes |
| cortex_a5 |
yes |
yes |
missing |
| cortex_a7 |
yes |
yes |
missing |
| cortex_a8 |
yes |
yes |
missing |
| cortex_a9 |
yes |
yes |
missing |
So those four runs reach COMP, fail on a file that is not there, and print
Test results are invalid! — which reads as a scheduling failure rather than a
missing fixture. That is the part worth fixing regardless of what happens to the
golden files themselves.
Two further observations
The committed golden file is pinned to one machine. Because COMP is an
exact match, the first lines of the capture have to agree too:
Target: Initializing `C:\ghs\comp_202014\simarm -cpu=cortexm3 -rom_use_entry'.
Playing back from C:\ghs\comp_202014\defaults\os_aware\txarm.rc
The test therefore only passes with Green Hills comp_202014 installed at
exactly C:\ghs\. Any other version or install path fails on the second line,
before a single counter is compared.
Two naming conventions are in use. The ac5, ac6 and gnu harnesses use
*.log.expected — 27 files — and compare the same way (COMP /A /L /M test.log ...gnu_cortex_m3.log.expected). Only the Green Hills set uses *.valid.log, and
it has one file. The split is a plausible reason the gap went unnoticed.
Possible resolutions
- Capture the four missing golden files on a machine with the toolchain, and
normalise the environment-dependent header lines out of the comparison so the
files are not pinned to one install path.
- Or, if the Green Hills tests are no longer exercised, remove the four scripts
and their .rc files so the suite does not advertise coverage it cannot
deliver, keeping cortex_m3 if it still runs.
Either way the four scripts should not fail with a message that blames the
scheduler for a missing file.
Regenerating the captures needs a Green Hills licence and the simulator, so this
is reported rather than fixed.
Four of the five Green Hills port tests cannot pass, because the golden output
file each one compares against is not in the tree.
How the test works
test/ports/azrtos_test_tx_ghs_<core>.batruns the ThreadX demo on the GreenHills
simarmsimulator under MULTI, replaying a.rcscript that setsbreakpoints and dumps the demo's thread counters to
test.log. It then comparesthat capture byte for byte against a committed golden file:
The assertion is a real one: the golden file records each of the eight demo
threads' counters at two sampling points, so a scheduling regression moves the
numbers and the comparison fails.
The gap
All five cores have the harness script and the
.rcplayback file. Onlycortex_m3has a golden log..bat.rc.valid.logSo those four runs reach
COMP, fail on a file that is not there, and printTest results are invalid!— which reads as a scheduling failure rather than amissing fixture. That is the part worth fixing regardless of what happens to the
golden files themselves.
Two further observations
The committed golden file is pinned to one machine. Because
COMPis anexact match, the first lines of the capture have to agree too:
The test therefore only passes with Green Hills
comp_202014installed atexactly
C:\ghs\. Any other version or install path fails on the second line,before a single counter is compared.
Two naming conventions are in use. The ac5, ac6 and gnu harnesses use
*.log.expected— 27 files — and compare the same way (COMP /A /L /M test.log ...gnu_cortex_m3.log.expected). Only the Green Hills set uses*.valid.log, andit has one file. The split is a plausible reason the gap went unnoticed.
Possible resolutions
normalise the environment-dependent header lines out of the comparison so the
files are not pinned to one install path.
and their
.rcfiles so the suite does not advertise coverage it cannotdeliver, keeping
cortex_m3if it still runs.Either way the four scripts should not fail with a message that blames the
scheduler for a missing file.
Regenerating the captures needs a Green Hills licence and the simulator, so this
is reported rather than fixed.