Generators of your own

archec writes code for about twenty-five targets — C, Rust, Go, DBC, ARXML, ROS 2 and the rest (archec gen --list). Your company has others: an RTE of its own, a test bench's configuration, a document in its own layout. A generator adds one, run like the built-in ones:

generator rte "Our RTE glue" {
  templates "generators/rte"
  options { prefix "kc_" }
  needs deployment
}

archec gen main.arche --target rte -o generated     # into generated/rte/

A generator reads the generator model and returns files. It comes in three kinds — templates, a script, a command — and the model is the same for all three.

A generator has two siblings: an importer reads a file of another tool into arche-lang, an exporter writes the model as one. All three can come together in a plugin — a folder the project, a Store package or archec itself ships (plugins).

archec generator new rte --templates     # or --script, --command: one that works, to start from
archec generator model main.arche        # the model it reads, as JSON

The generator model#

The project as plain JSON, version 1 (schemas/generator-model.v1.json). Every name is resolved and every value worked out, and what a generator would otherwise have to compute is already computed:

Each element has at: the file and line it is written at. A generator that finds something wrong says so on the element, and archec prints it at that line (and VS Code puts it in Problems).

An element in a package says it (package). One with attr key = values has them in attrs: what the language has no word for, such as another tool's id or name. An exporter writes them back (the Melody exporter keeps attr melody_id as the element's xmi:id). Parts, threads, binaries, processes, allocators, memories and placements have attrs too.

The model only grows within version 1: a field added keeps it version 1. A generator says which version it reads with api 1.

Bodies in C, Rust and Go#

Each body — a function's, a state's onEnter and onExit, a port's onReceive and onDeadline, a component's lifecycle — has its source, its python, and c, cpp, rust and go: the same translators the built-in targets use (cpp as --target cpp writes it: this->n). A body in Python is translated too when the action language can say it, as the C and Go targets do; otherwise those three are absent.

The translation cannot know your runtime, so what the body does to the outside is written as the built-in targets write it, and your generator provides those names:

Types are named as the targets name them: go_t in C, Go in Rust, GO in Go; durations are nanoseconds (uint64_t, u64, int64). A part without a state machine writes a goto as a comment.

The model is enough: examples/generators/dbc.js writes the same DBC files as --target dbc, byte for byte, reading nothing but the model (the tests check it).

Templates#

A folder of text files. Each one is a template: what is between {{ and }} is filled from the model, and the result has the template's name (without a .tpl suffix).

/* {{ project.name }}: its messages. */
{{#each messages where layout.bytes}}
typedef struct {
{{#each fields}}
  {{ type | ctype @root.options.prefix }} {{ id | snake }};{{#if unit}} /* {{ unit }} */{{/if}}
{{/each}}
} {{ @root.options.prefix }}{{ id | snake }}_t;
{{/each}}

A line that holds only a block tag ({{#each …}}, {{/if}}, {{else}}…) leaves nothing behind, not even its line break, so a template of code reads like the code it makes.

Filters: snake, camel, pascal, kebab, constant (BRAKE_CMD), upper, lower; ctype, rusttype, pytype (a type of the model in that language, through its aliases — ctype "kc_" writes a declared type as kc_name_t); hex (hex 4 → 0x00FF), json, quote, default "x", join ", ", map "id", sort "id", length, first, last, indent 4, replace "a" "b", trim, pad 12. A filter's argument written without quotes is a name read from the model (ctype @root.options.prefix).

A script#

generator can-db "CAN databases" { script "dbc.js"; needs deployment }

module.exports = function generate(model, ctx) {
  const files = {};
  for (const m of model.messages) {
    if (m.layout.problem) { ctx.warn(m, `${m.id} has no fixed layout`); continue; }
    files[`${ctx.filters.snake(m.id)}.txt`] = m.layout.signals.map((s) => s.name).join('\n');
  }
  return { files };   // { "path": "text" }, or { "path": { text, keep: true } }
};

export default function generate… works too. ctx gives:

• options;
• readFile(rel): a file of the script's own folder;
• render(template, data): the template language;
• region(id, default, style): a USER CODE region around the default code, kept when the file is written again (style: "c", "line", "hash", or a file name; "pycogen", PYCOGEN's markers, with the file name as a fifth argument; regeneration);
• filters;
• note(msg);
• warn(where, msg) and error(where, msg). where is an element of the model, a path or id, or { file, line };
• builtin(target, { deployment, trace, monitors }): a built-in target's files as { "path": "text" } (see below);
• as a hook (after c): target, files ({ "path": "text" }) and kept (the paths marked keep).

In TypeScript#

script "gen.ts" (or .mts, .cts) is TypeScript: archec strips the types and runs what is left like any script. Node does it itself from 22.13 and 23.2 (with the lines where they were, so an error's line is the .ts's; an enum or a namespace is written out, and its lines may move); on an older node, esbuild or typescript do it when the project or archec has them installed, and otherwise the error says so. The model and ctx are typed by generator-model.v1.d.ts — archec generator types writes it, and archec generator new rte --script --ts writes a script and it side by side:

import type { GeneratorContext, GeneratorModel, GeneratorResult } from './generator-model.v1';

export default function generate(model: GeneratorModel, ctx: GeneratorContext): GeneratorResult {
  const files: Record<string, string> = {};
  for (const m of model.messages) files[`${m.id}.txt`] = m.fields.map((f) => f.id).join('\n');
  return { files };
}

export function generate works too. Types are only stripped, never checked: tsc --noEmit in your editor or CI checks them.

The sandbox#

The script runs in a sandbox: plain JavaScript, with nothing of node or npm — no process, no file system beyond its own folder, and no network. It returns its files rather than writing them. A script that loops is stopped after 30 s.

A script may be several files: import { view } from './core/view' (or require('./core/view')) loads a module of its own, .ts or .js, with export as usual. Modules come from the script's folder, or, in a plugin or a library, from anywhere in it; one outside, or a module of node or npm (import fs from 'fs'), is an error.

A command, in any language#

generator icd-md "ICD" { command "python3 icd.py"; output "docs" }

The command runs in the folder of the file the generator is written in. It reads the model as JSON on its stdin and writes this JSON on its stdout:

{ "files": [{ "path": "icd.md", "text": "…", "keep": false }],
  "notes": ["9 messages"],
  "errors": [{ "where": "BRAKE_CMD", "message": "…", "severity": "warning" }] }

What it writes on stderr is shown as notes. A command that exits with an error, or writes something other than that JSON, is an error.

Options, and one generator from another#

options { … } is a record, and it is what the model's options holds. --option prefix=kc_ changes one for a run. A project that wants other values writes another generator from the first:

generator rte-bench "RTE for the bench" from rte { options { prefix "bench_" } }

An option the first generator does not have is a warning (generator-option).

Options declared, with their types#

A generator, above all one a library ships, says what its options are:

generator rte "Our RTE glue" {
  templates "generators/rte"
  option prefix "The prefix of the C names" { type string; default "kc_" }
  option os "The RTOS it runs on" { type string; values posix, freertos; default posix }
  option depth "Queue depth" { type int; default 8 }
}

type is string, int, float, bool or list (any value, when not said); values are the only ones it takes; default is its value when nothing sets it, and the model's options holds it. archec check then holds what a generator from it sets: an option it does not declare is a warning, a value of another type or not among its values an error, at the value (generator-option). --option is read as the option's type (--option prefix=12 is the string "12", --option list=a,b a list) and held to it the same way. archec gen --list lists each generator's options, and in VS Code options { } completes their names and then their values, and hovering over one says what it is.

A built-in target's files#

A generator often wants what a built-in target writes, plus a little: the C of --target c with a file of glue, the DBC in another layout. It reads them rather than writing them again.

A script calls ctx.builtin:

module.exports = function generate(model, ctx) {
  const c = ctx.builtin('c');                 // { "c/arche_types.h": "…", … }
  const files = {};
  for (const [p, text] of Object.entries(c)) files[p] = `#pragma once_ours\n${text}`;
  files['c/glue.c'] = ctx.render('…', model);
  return { files };
};

A template or a command says which targets it reads with builtins, and the model's builtins holds their files — { "c": [{ "path", "text", "keep" }] }:

generator listing "What C writes" { templates "listing"; builtins c, dbc }

{{#each builtins.c}}{{ path }}: {{ text | length }} characters
{{/each}}

The files are the target's as it writes them, before any hook, for the deployment of the run (--deployment). freertos, zephyr, posix and components, which Arche writes, are not available.

Hooks: after a target#

A company puts its license header on every file it ships. Rather than a generator of its own for that, or an edit by hand after each run, it writes a hook: a generator that runs after a target and changes its files.

generator license "Our license header" { script "license.js"; after c, go }

module.exports = function (model, ctx) {
  const files = {};
  for (const [p, text] of Object.entries(ctx.files))
    if (/\.[ch]$/.test(p)) files[p] = `/* (c) ACME. All rights reserved. */\n${text}`;
  files['LICENSE'] = ctx.readFile('LICENSE');
  return { files };
};

archec gen --target c then writes the target's files through the hook. What a hook returns is only what it changes:

• a path with its text replaces that file, or adds it;
• a path with null takes the file away;
• a file it does not name stays as it was.

A command hook reads the same in the model's after — { "target": "c", "files": [{ "path", "text", "keep" }] } — and writes { "files": [{ "path": "c/x.h", "text": null }] } to take one away.

• Order: the hooks on a target run in the order they are written (an included file's where it is included), each on the files the previous one left. archec gen --list shows them (after c).
• Paths are as written under -o (c/arche_types.h), not under a folder of the hook's own.
• keep: a file the target marks keep (yours once written) stays marked when a hook changes it, and what the hook does to it counts only when it is first written. ctx.kept lists them.
• A hook is a script or a command; it runs only through its target (archec gen --target license says so), on any built-in target or one of the project's own generators, and --no-hooks leaves them out. A library's hook runs only once approved, like its other scripts.

Where the files go#

They go into <-o>/<output>/: the generator's id by default, output "dir" for another folder, output "." for -o itself. A path that would leave that folder is refused. A file marked keep is written once and never again. --check says what would change and writes nothing.

Testing a generator#

archec generator test main.arche rte --update   # approve the output as it is
archec generator test main.arche rte            # later: missing, changed (at the first different line), new

The approved files are kept in expected/<id>/ beside the declaration. Commit them: a change to the model or to the generator then shows as a difference in CI.

From a library, from the Store#

A library can ship generators: its generator declarations, with their paths relative to the library's own files. A Store package can be of kind generator. uses acme-gen "^1.0" and include "acme-gen/generators.arche" make them targets of the project. A package of kind plugin needs no include: uses is enough, and it may bring importers and exporters too (plugins).

A library's templates run as they are, because they run no code. A library's script or command runs on the machine of whoever builds the project, so it runs only after the project approves it:

archec generator approve main.arche rte   # lists the code it runs, then records it

The approval goes into arche.lock, which is committed and reviewed with the rest, at the fingerprint the library has now. If the library changes, the generator is refused until it is approved again. --revoke withdraws an approval, and archec audit lists a generator's JavaScript with the rest of the code a project brings in. The sandbox is no security boundary (node's vm is not one), which is why the approval is what decides.

What is checked#

In VS Code#

• Running: a ▶ Generate lens sits above each generator, and Generate code… lists the project's generators after the built-in targets. What a generator says is wrong appears in Problems, at the element's line. A hook's lens (▶ Generate c) runs its target, and Generate code… runs a target through its hooks.
• Writing: after from, completion offers the project's generators; in options { }, the options declared, then their values, with what each is for on hover. In a file of a templates folder:
  • {{ offers the model's names;
  • {{# offers the blocks;
  • | offers the filters;
  • hovering over any of them says what it is.