Integration targets

archec gen --target <t> also writes what the tools around a firmware read: descriptions of its protocols and interfaces, the build systems that compile the generated code, the scripts that flash it and read its trace, and the configuration of the bench it is tested on. Each target is one generator, as the others are (archec gen --list): a pure function of the model and the deployment (the one --deployment names, else the active one), whose notes say what it could not write.

Each one writes only what the model says. Where a protocol needs a value the language has no word for (a J1939 SPN, a MAVLink id, a CANopen vendor id), it is read from an attr on the declaration it belongs to, and when it is missing the notes say so: a value is never made up silently.

One target per format, rather than one with sub-kinds: each has its own --check, its own notes and its own row in archec gen --list, and a CI job regenerates only what it uses.

Protocol descriptions#

CANopen (--target canopen)#

An electronic data sheet (EDS, CiA 306, version 4.0) per binary that sends or receives on a CAN bus of the deployment: canopen/<bus>/<binary>.eds.

• the mandatory objects: device type (0x1000), error register (0x1001), identity (0x1018);
• a transmit PDO per frame the binary sends on the bus (0x1800 + n, its mapping at 0x1A00 + n) and a receive PDO per frame it takes (0x1400 + n, 0x1600 + n). The COB-ID is the route's id (bit 29 set for a 29-bit one); a message with a period is sent on the event timer (transmission type 0xFE, the period in ms), the others when the application sends them;
• an object per message in the manufacturer area (0x2000 on, the same index in every node's EDS), a sub-index per signal named as in the model (TEMPERATURE.value), with its CANopen type.

The signals are the layout the DBC and the generated C and Rust use, so the PDO's bytes are theirs. A message with a wire format maps its fields at their bits, with dummy entries over the gaps. What a classic PDO cannot carry is left out, and said: a frame over 8 bytes (CANopen FD, CiA 1301, is not written), a big-endian field, a field of a bit count no CANopen type has (a 4-bit field: there is no UNSIGNED4).

The model has no CANopen identity. On the binary (or its processor):

binary pump_fw "pump-fw" on pumpb {
  attr canopen_node_id = 5      // [DeviceComissioning] NodeID
  attr vendor_id = 0x1234       // 0x1018 sub 1, VendorNumber
  attr product_code = 42        // 0x1018 sub 2
  attr revision = 1             // 0x1018 sub 3
  …
}

Without them the EDS says 0 and the notes name what is missing. A bit rate that is not one of CANopen's (10, 20, 50, 125, 250, 500, 800, 1000 kbit/s) is noted too.

Checked by reading each file with the canopen Python package (its object dictionary and the PDO maps it builds from it) in test/integration.test.js.

J1939 (--target j1939)#

For each CAN bus that carries frames with a 29-bit id: j1939/<bus>.json and the same as Markdown tables, j1939/<bus>.md.

Everything about a parameter group is in its identifier (J1939-21), so it is read from the route's id: the priority, the PGN (with the data page and the extended data page), the PDU format — PDU 1 with its destination address, PDU 2 with its group extension — and the source address. Then its length, its rate (the message's period), the binary that sends it and the one that receives it.

Each signal is an SPN, with J1939-71's columns: its start position (byte.bit, both from 1), its length in bits, resolution and offset (a wire format's scale and offset), unit and range. The SPN number is not in the identifier, so it is written on the field:

message EEC1 "Electronic engine controller 1" {
  attr pgn = 61444             // checked against the route's id
  period 10ms
  size 8
  torque_mode: uint8 bits 4 range 0..15 { attr spn = 899 }
  speed: float32 unit rpm range 0..8031 at 24 bits 16 scale 0.125 { attr spn = 190 }
}
…
route pump.engine -> gateway.engine over can_bus { id 0x0CF00405 }

A signal without attr spn is listed with "spn": null and named in the notes. Also noted: a frame over 8 bytes (it goes by the transport protocol, TP.BAM or TP.CMDT), a big-endian field, a binary that sends from more than one source address, or from another than its attr j1939_address, a PGN that two binaries send, and the frames of 11-bit ids on the bus, which are left out.

Checked against cantools.j1939 (the priority, PGN and source address it reads from the same id).

MAVLink (--target mavlink)#

The message catalog as a MAVLink message-definition file, the format mavgen reads: mavlink/<project>.xml, a dialect of its own (it includes nothing, so the model's HEARTBEAT does not clash with common.xml's).

• a <message> per message with a fixed layout, a <field> per signal of that layout: a struct is flattened (header_seq), an array of scalars is one array field (float[4]), an enum is a uint8_t with enum="MODE";
• an <enum> per enum the messages use, its entries MODE_RUN;
• the unit of a field when MAVLink's schema knows it (m/s, degC, rad, %, and the model's °, °/s, m/s² as deg, deg/s, m/s/s); another goes into the field's description;
• the message's name or description, and its period, as its description.

The MAVLink id is not in the model: attr mavlink_id = 30 on the message gives it. Messages without one are numbered from 65536 in the catalog's order — above every range the MAVLink registry gives dialects, so they clash with none — and the notes list them: give them their real ids before two systems rely on them. A payload over 255 bytes or with more than 64 fields is left out, and said.

Checked with mavgen --strict-units (the schema, units included), its C headers compiled and a message packed, and its Python round-tripping a message.

SOME/IP (--target someip)#

A service description as JSON, someip/<deployment>.json (FIBEX and ARXML carry the same: services, their methods, events and event groups, and where each instance runs), with the data types its payloads use.

• a service per interface a port provides: each operation a method — request/response when it returns something, fire and forget when it does not — with its parameters; the interface's messages that the provider sends are events (cyclic at the message's period, else on change) in one event group; those it only receives are fire-and-forget methods;
• a service per out port on an Ethernet link that sends messages and takes part in no interface: its messages as events;
• per service its instances: the part that provides it, the binary and processor it runs on, and the parts at the other end of its links;
• dataTypes: the structs, enums and unions the payloads use, scalars named as SOME/IP names them (sint16, float32, boolean).

The ids are not in the model: attr someip_service_id on the interface (or the port), attr someip_instance_id on the providing port, attr someip_method_id on an operation, attr someip_event_id on a message, attr someip_major_version and someip_minor_version on the interface. Without them services are numbered from 0x1000, methods from 0x0001 and events from 0x8001 in the model's order, and the notes list each one. An interface provided on a port that is not on IP (a CAN port) is noted. SOME/IP serializes big-endian; the codecs archec generates for CAN are little-endian, so they are not this wire format.

OpenAPI (--target openapi)#

An OpenAPI 3.1 document per component definition whose ports have HTTP handlers (handler GET "/api/v1/readings/{node}" -> reading): openapi/<Component>.json.

• an operation per handler, its operationId the function's name, its summary the function's name in quotes, its description the handler's (else the function's), tagged with the port;
• {node} in the path is a path parameter, typed by the function's parameter of that name; the other parameters are the query (GET, DELETE, HEAD) or a JSON body (POST, PUT, PATCH), required unless optional;
• the function's return type is the 200 answer as JSON; with none, 204;
• servers: each part of that definition with prop hostname = "…" (or attr url = "…").

The schemas (JSON Schema, shared with AsyncAPI): a struct is an object with its fields required (an optional one not) and no others, an enum its literal names (x-values gives their numbers), a fixed array its size as min and max items, an integer its type's bounds or its range, a float its range, a unit x-unit, an alias type (type Speed = float32 unit m/s range 0..70) its scalar with its unit and range, a union a oneOf of {case, value}.

Checked with openapi-spec-validator.

AsyncAPI (--target asyncapi)#

An AsyncAPI 3.0 document per system (the deployment's, when there is one): asyncapi/<system>.json.

• a channel per port that sends over a link: its address the link's or the port's topic, else <part>/<port> as the ROS 2 nodes name it; its messages what the links carry;
• a send operation for the part that publishes, a receive operation for each part at the other end;
• with a deployment, a server per bus a route takes (its protocol from the port type: can, serial, udp, mqtt…, its bit rate in the description), and the channel on it;
• the messages in components.messages, their payloads in the shared schemas.

Checked against the official AsyncAPI 3.0.0 JSON schema (@asyncapi/specs with ajv) when ARCHE_NODE_TOOLS names a folder that has them installed.

Build systems#

The build targets wrap the C that --target c writes, where it is: they refer to ../../c from their own folder, so both go into the same output folder.

archec gen main.arche --target c -o gen
archec gen main.arche --target cmake -o gen
cmake -S gen/cmake -B build && cmake --build build      # every C binary, on this host

A binary in Rust (--target embassy) or Go (--target go) has its own build, cargo or go: these targets note it and leave it out.

CMake (--target cmake)#

• cmake/<binary>/CMakeLists.txt: the binary's model as a static library (<binary>_model: the types, its parts, its wiring and drivers), then the executable with its main.c and its runtime — POSIX on a host, FreeRTOS when cross-compiled (ARCHE_RT chooses);
• cmake/<binary>/toolchain.cmake, for a binary on a microcontroller: GCC by its prefix (arm-none-eabi-, riscv32-esp-elf-, xtensa-esp32-elf-) and the core's flags from its core-type (Cortex-M4F: -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=hard);
• cmake/<binary>/port.cmake, written once and yours: the FreeRTOS kernel (FREERTOS_KERNEL_PATH, its port from the core: GCC_ARM_CM4F…), the chip's startup code and linker script. Without a kernel only the model library is built, with a warning;
• cmake/CMakeLists.txt: every C binary for this host, the microcontrollers' too (their logic on the POSIX runtime, as their Makefile builds it) — what CI builds and runs.

A core FreeRTOS has no upstream port for (Xtensa, the ESP32's) is noted: ESP-IDF brings its own, which --target platformio uses.

Checked: the rig's three C binaries configured, built and run with CMake on the host; the sensor node's model library cross-compiled for its STM32G4 with the toolchain file when an arm-none-eabi-gcc with newlib is installed.

Zephyr (--target west)#

For each C binary on a board Zephyr knows (the store's Nucleo, nRF52840 DK, Pico and ESP32 boards), a Zephyr application, west/<binary>/:

• CMakeLists.txt: the binary's C added to Zephyr's app, on Zephyr's POSIX API (arche_rt_posix.c);
• prj.conf, written once and yours: CONFIG_POSIX_API and the drivers the overlay's nodes need (CONFIG_CAN, CONFIG_SERIAL…);
• boards/<board>.overlay: what --target devicetree writes for that board (the linked CAN, UART, I2C and SPI on their pins, an alias per bus that the C opens);
• west.yml: a manifest pinning Zephyr (v4.1.0) and allowing only the modules the board's HAL needs (cmsis, hal_stm32…).

west init -l gen/west/node_fw && west update && west build -b nucleo_g474re gen/west/node_fw

Checked: the manifest is validated and loaded by west's own Python package.

PlatformIO (--target platformio)#

For each C binary on a board PlatformIO knows, platformio/<binary>/platformio.ini: the platform and board (ststm32/nucleo_g474re, espressif32/esp32dev, nordicnrf52/nrf52840_dk, raspberrypi/pico), a framework with threads under the runtime — Zephyr on STM32 and nRF52 (with zephyr/CMakeLists.txt, prj.conf and app.overlay), ESP-IDF's FreeRTOS on ESP32, the Earle Philhower Arduino core's FreeRTOS on the RP2040 — and the generated C as its sources (src_dir = ../../c, a build_src_filter of the binary's files). The notes say what is left to do by hand (ESP-IDF starts app_main, not main).

Checked: pio project config reads each file. Not built: that downloads each platform's toolchain.

CI (--target ci)#

A pipeline for GitHub Actions (ci/.github/workflows/archec.yml) and the same for GitLab CI (ci/.gitlab-ci.yml); copy ci/ to the root of the repository.

• model: archec check, then archec ci with its JUnit report (uploaded on GitHub, a test report on GitLab);
• generated: archec gen --check for each target the deployment uses (c and cmake for C binaries, embassy for Rust ones, go, dbc for its CAN buses), in a matrix: the generated code committed is what the model gives;
• build-c: the CMake project of --target cmake, built on the host, each binary run for a second;
• build-<binary> per Rust binary: cargo build --release for its chip's target (thumbv7em-none-eabihf…; the Espressif toolchain for an Xtensa ESP32).

The model's path (ARCHE_MAIN, the main file's name from the repository root), the folder the generated code is committed in (ARCHE_GEN, generated) and the deployment are variables at the top, to edit when the project is not at the root.

Checked: both files are valid YAML and valid against SchemaStore's GitHub workflow and GitLab CI schemas (check-jsonschema).

The lab#

Flash and monitor (--target flash)#

For each binary on a microcontroller, two scripts in flash/<binary>/ (written executable):

• flash.sh [image] flashes it with the first of its chip's tools that is installed, or the one TOOL= names:

  Chip	Tools, in order
  STM32	probe-rs (--chip STM32G474RETx), OpenOCD over the ST-LINK (target/stm32g4x.cfg)
  nRF52	probe-rs (nRF52840_xxAA), nrfjprog (the ELF made a HEX), OpenOCD over a J-Link
  RP2040	picotool (load -x), probe-rs, OpenOCD over a CMSIS-DAP probe
  ESP32	espflash, esptool (the ELF made an image, written at 0x10000)

  The image is $1, else where the build puts it: cargo's embassy/<binary>/target/<triple>/release/<crate> for Rust, build-<binary>/<binary>.elf for C (the cross build of --target cmake, from the folder archec gen wrote into).

• monitor.sh reads the binary's trace for ARCHE_DURATION seconds (10) into ARCHE_TRACE (<binary>.trace), then runs archec timing and archec live --replay on it with the deployment. A C binary's trace comes from the board's UART (its virtual COM port when it has one; ARCHE_PORT, default /dev/ttyACM0), at the bitrate of the link on it (else 115200), read with pyserial; a Rust binary's is defmt over RTT, through probe-rs attach. The model is ARCHE_MAIN (the main file's name, from where it runs).

The trace is what archec gen --trace makes the firmware write: the C built with -DARCHE_TRACE_LIVE, its c/arche_trace_port.c writing to that UART; the Rust with --features trace. archec timing skips whatever comes before arche-trace on a line, so a log prefix does no harm. A binary on Linux is not flashed (it is noted): copy it there.

Checked: each script parses (sh -n); flash.sh run against a stand-in probe-rs gets the chip and the image; OpenOCD reads the interface, transport and target of each OpenOCD line; monitor.sh reads arche-trace lines written to a pseudo-terminal, and archec timing and archec live measure the thread from them. Not run on a board here.

The bench (--target bench)#

The hardware test rack of a deployment, from its platform:

• bench/<deployment>.yaml:
  • buses: each bus with its type (can, can-fd, uart, ethernet), bitrate (and data_bitrate, from attr data_bitrate on a CAN FD bus — the language has no word for the data phase), the rack's interface on it (can0, can1… in order, /dev/ttyUSB0… for a serial bus), the nodes on it with their binaries, and for a CAN bus every frame it carries: id, message, length, sender, receiver, period;
  • duts: each binary on a microcontroller, with its board, chip, flash and monitor scripts (--target flash) and its console UART; hosts: the binaries on Linux;
  • power: each supply (an out … : power port) and what it feeds;
  • io: the GPIO, PWM and analog links;
  • what the model cannot say is null (the supply's and the IO's channels on your rack), and the notes say what else is missing (no bitrate, no power link);
• bench/setup_<bus>.sh per CAN bus: its SocketCAN interface at the bus's bitrate (dbitrate … fd on for CAN FD, restart-ms 100), or a vcan when the interface given is named so;
• bench/bench_<bus>.py per CAN bus, with python-can: it lists the frames that go by, named after the model, and fails (exit 1) when a periodic one is missing or off its period by more than half. --as <binary> stands in for a binary that is not on the rack: its frames are sent at their period; --send <MESSAGE> sends one; --interface virtual runs it all in one process, without hardware.

sh gen/bench/setup_can_bus.sh can0
python3 gen/bench/bench_can_bus.py --channel can0 --duration 10 --as gw_app

Checked: the YAML loads; the setup script parses; the python-can script on its virtual bus sees the sensor's and the pump's frames at their periods when it stands in for them, and fails with the pump's frame missing without it.