Skip to content

Green Hills port tests for Cortex-A5, A7, A8 and A9 compare against golden files that do not exist #619

Description

@fdesbiens

Four of the five Green Hills port tests cannot pass, because the golden output
file each one compares against is not in the tree.

How the test works

test/ports/azrtos_test_tx_ghs_<core>.bat runs the ThreadX demo on the Green
Hills simarm simulator under MULTI, replaying a .rc script that sets
breakpoints and dumps the demo's thread counters to test.log. It then compares
that capture byte for byte against a committed golden file:

%AZRTOS_GHS_MULTI_MULTI% -connectfile sample_threadx.con -connect="Simulator" ^
    -p %AZRTOS_TEST_DIR%\azrtos_test_tx_ghs_cortex_m3.rc sample_threadx
COMP test.log %AZRTOS_TEST_DIR%\azrtos_test_tx_ghs_cortex_m3.valid.log /M /L /A
IF %ERRORLEVEL% NEQ 0 (
    ECHO Test results are invalid!
    EXIT /B 1
)

The assertion is a real one: the golden file records each of the eight demo
threads' counters at two sampling points, so a scheduling regression moves the
numbers and the comparison fails.

thread_0_counter = 0x00000000     ...      thread_0_counter = 0x00000005
thread_1_counter = 0x00000000     ...      thread_1_counter = 0x00003695
thread_2_counter = 0x00000000     ...      thread_2_counter = 0x0000363f

The gap

All five cores have the harness script and the .rc playback file. Only
cortex_m3 has a golden log.

core .bat .rc .valid.log
cortex_m3 yes yes yes
cortex_a5 yes yes missing
cortex_a7 yes yes missing
cortex_a8 yes yes missing
cortex_a9 yes yes missing

So those four runs reach COMP, fail on a file that is not there, and print
Test results are invalid! — which reads as a scheduling failure rather than a
missing fixture. That is the part worth fixing regardless of what happens to the
golden files themselves.

Two further observations

The committed golden file is pinned to one machine. Because COMP is an
exact match, the first lines of the capture have to agree too:

Target: Initializing `C:\ghs\comp_202014\simarm -cpu=cortexm3 -rom_use_entry'.
Playing back from C:\ghs\comp_202014\defaults\os_aware\txarm.rc

The test therefore only passes with Green Hills comp_202014 installed at
exactly C:\ghs\. Any other version or install path fails on the second line,
before a single counter is compared.

Two naming conventions are in use. The ac5, ac6 and gnu harnesses use
*.log.expected — 27 files — and compare the same way (COMP /A /L /M test.log ...gnu_cortex_m3.log.expected). Only the Green Hills set uses *.valid.log, and
it has one file. The split is a plausible reason the gap went unnoticed.

Possible resolutions

  • Capture the four missing golden files on a machine with the toolchain, and
    normalise the environment-dependent header lines out of the comparison so the
    files are not pinned to one install path.
  • Or, if the Green Hills tests are no longer exercised, remove the four scripts
    and their .rc files so the suite does not advertise coverage it cannot
    deliver, keeping cortex_m3 if it still runs.

Either way the four scripts should not fail with a message that blames the
scheduler for a missing file.

Regenerating the captures needs a Green Hills licence and the simulator, so this
is reported rather than fixed.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions