The quality of the generated code

archec gen writes code that should build clean, that a reviewer can trace back to the model, and that is tested by the model's own test plans. This page is how to check each of these, from the model:

• Build and lint — archec gen --build --lint, each finding at the model element it comes from
• C unit tests from the test plans — archec gen --target c-tests, make test, coverage of states and transitions
• Defensive code — --defensive: ranges, stale inputs and end-to-end checks (CRC, counter) on receive
• The traceability matrix — archec gen --target trace-matrix: requirement → element → generated function → test case → result

Each tool archec runs that is not installed (cppcheck, clang-tidy, cargo clippy, gcov) is said when the command runs, and what it would check is left out — never pretended. test/codegen-quality.test.js runs what is installed for real.

Build and lint#

archec gen main.arche --target c -o generated --build            # build it
archec gen main.arche --target c -o generated --build --lint     # and lint it
archec gen main.arche --target embassy -o generated --lint --sarif gen.sarif

--build builds what was just written:

The generated C is meant to build clean, so with -Werror a warning is a build error: the command exits 1 when the build fails, 0 otherwise (a linter's finding alone does not fail it — read them, or gate on the SARIF in CI).

Each finding is said at the model element its generated line comes from, then where it is in the generated code:

main.arche:57 error [cc-unused-variable] unused variable 'x' — c/controller_ports.c:6 in controller_send_cmd (port cmd of Controller)
components.arche:172 warning [clippy-blocks_in_conditions] … — embassy/brake_fw/src/app.rs:26 in main (fn main of brakes)
components.arche:118 warning [misra-10.4] … — c/adcu.c:61 in adcu_fire_with (transition Manual -> Ready on self_test_ok of Adcu)

The rule is cc-<warning> for the compiler, misra-<rule> for a MISRA C:2012 rule, cppcheck-<id> for cppcheck's other checks, clang-tidy's check name, and clippy-<lint> for clippy. The element is found from the names the generators give and the comments they write:

A line that maps to nothing (a runtime file, main.c) is said at the generated file and line only.

--sarif <file> writes the findings as SARIF 2.1.0, as archec check --sarif does: each at the model element's file and line (so code scanning shows it on the .arche file), the generated place in its message, a rule per code.

C unit tests from the test plans#

archec gen main.arche --target c-tests -o generated            # write them
make -C generated/c-tests test                                  # build and run them
archec gen main.arche --target c-tests -o generated --run       # both
archec gen main.arche --target c-tests -o generated --coverage  # and gcov's coverage

--target c-tests writes, under c-tests/, the C of each component (as --target c writes it), arche_tests.c and a Makefile. arche_tests.c is the harness archec difftest drives the generated C with — the system's parts on a virtual clock, each message along its links with their latency, one thing at a time per part, in the order the specification gives — and a function per test case. No test framework: the harness is the test runner, and make test exits 1 when a case fails. make test CASES="TC-1 TP-2" runs some cases (by id or plan); ARCHE_TRACE=1 prints what the parts do, as archec difftest traces it.

A case runs as Arche's test runner runs it: from a fresh start, each step at its at, its do, its wait, then its expectations — an expression once the step has settled; within 40ms: true no later than 40 ms after the step began; at d:; eventually: within the case's timeout (100 ms); never: and always: watched after each event until the case ends, 10 ms after its last step.

What a case says in Python is translated where C can say it:

A case with anything else (print, received(…), a fault, a keyframe, a function test) is generated as SKIP, with what C could not say; nothing is guessed. The output is the one archec test reads from Arche:

PASS TP-BRK/TC-BRK-01 Brakes when fast
FAIL TP-BRK/TC-BRK-03 A wrong expectation
    ctrl.state == "Braking" — false at 1.000 ms (main.arche:111)
SKIP TP-BRK/TC-BRK-04 Python the C tests cannot say — do print("hello") (main.arche:114)
arche-tests: 2 passed, 1 failed, 1 skipped

--run builds and runs them, prints that and writes c-tests/results.json — the verdicts by PLAN/CASE and what they say of the requirements the cases verify, as archec test writes .archec/test-results.json. The traceability matrix reads it.

A body that waits (await wait(…)) runs in the harness, but its wait takes no time; one that loops forever on its waits (while True: await wait(WATCHDOG)) runs past 100000 events: its case is an ERROR (and the next case still runs). The car's watchdog loop does this today: in C it needs a thread, which the harness does not give a body.

Coverage#

With gcov (--coverage, or make coverage), each module is built with --coverage and its lines read back as the model:

and reported as archec test --coverage reports Arche's (also in c-tests/coverage.json):

Coverage (the generated C (gcov)):
  states           5/5     100 %
  transitions      3/4      75 %
      not covered: brakes: Braking -> Released on after 100ms
  functions        0/1       0 %
      not covered: sensor.scan

A part with one instance is named by its path (brakes), a module several instances share by its component (Brakes).

Defensive code#

archec gen main.arche --target c --defensive -o generated

--defensive adds to the C what the model says a message may be, checked where it comes in:

Why reject and not clamp: a value out of its range is a fault upstream — a sensor, a bus, a sender's bug — and a clamped value hides it as a plausible one. A rejected message never reaches the part, which keeps acting on the last good one; the stale flag says when that has gone on too long.

With a deployment, each binary's wiring.c checks every message it dispatches to a part (arche_accept_<link>_<msg>, its inputs listed in wiring.h), checks for stale inputs at each period of a periodic thread and at each message, and logs each rejection once until the input is accepted again (a stuck sensor does not flood the log). Without a deployment, <msg>_in_range() and arche_errors are there for your receive code. A thread that waits for messages and has no period only sees an input go stale at its next message, or when asked (<binary>_input_stale).

It is opt-in rather than on by default because it changes what the parts see: a rejected message never reaches them, so archec difftest would no longer compare like with like against the simulator, which checks none of this; and each checked input costs a few bytes of RAM and a few compares, which a small target should choose to pay. The checks are in C only: the Embassy crates do not have them yet.

The traceability matrix#

archec test main.arche                                          # Arche's simulator: arche-out/.archec/test-results.json
archec gen main.arche --target c-tests -o generated --run       # the C: generated/c-tests/results.json
archec gen main.arche --target trace-matrix -o generated        # generated/trace-matrix/matrix.md, .csv

For each requirement: the model elements that satisfy it (satisfies, satisfied-by, a trace), the C generated for each (--target c's file, line and function, read back as in build and lint), the test cases that verify it (a plan's or a case's verifies, verified-by, trace verifies, verify { tests … }), and what the last run of each case said:

A component is said by where its module starts (its init); a function, a state, a transition or a port traced to a requirement by its own generated function; a part by its own and its component's. The results are those of --results <file> (each a results JSON, repeatable), else those found: arche-out/.archec/test-results.json (sim) and <out>/c-tests/results.json (C); a case in neither is not run.

Below the table, the gaps: a requirement no element satisfies, whose elements have no generated C, that no case verifies, with a case that failed, or whose cases have not run. matrix.csv has a row per requirement, element and case (requirement,title,element,element_at,generated,test_case,result), for a spreadsheet or a certification tool.