npm-iface-gen

Protocols: Interface Gen

The npm interface generator (@rootintf/iface-gen). It reads the JSON Schemas in schemas.readonly and emits one granular, dual ESM+CJS npm package per schema, so a project generates only the interfaces it uses. It is the npm twin of cpp-iface-gen: both read the same ifacegenconfig.json, pin a schema to the same hash, and generate the same names.

Adding iface-gen to the Project

The commands in this section are run from the root of the consuming project, the directory that holds ifacegenconfig.json, with the submodule checked out as npm-iface-gen.git directly beneath it. Adjust the paths if the submodule lives elsewhere, for example under src/.

Adding the submodule

npm-iface-gen is added to the consuming project as a git submodule, next to the project’s other *.git submodules:

git submodule add git@gitlab.com:interop-protocols/npm-iface-gen.git npm-iface-gen.git
git submodule update --init --recursive

The generator runs on Node and has a single runtime dependency, ajv. Install it inside the submodule (the generator ships a lockfile, and nothing in it has install scripts):

cd npm-iface-gen.git && npm ci --ignore-scripts

The generator is then run as node npm-iface-gen.git/src/gen.cjs … (see CLI interface). Generated envelope packages import @rootintf/json-msg at runtime, so the consuming project lists that in its dependencies; plain-JSON interfaces have no runtime dependency.

The generator is also published as @rootintf/iface-gen. Added as a devDependency (npm i -D @rootintf/iface-gen), npm puts an iface-gen binary on the path for scripts, e.g. "gen": "iface-gen --config ifacegenconfig.json", and npx --no-install iface-gen … runs the installed copy without reaching the registry.

ifacegenconfig.json

One ifacegenconfig.json at the root of the consuming project. The file is shared with the C generator: a schema *registry* (each schema declared once, pinned by its SHA-256) plus a list of output *targets*. Every target the npm generator reads carries `out-npm`; the C generator’s targets carry out-cpp instead.

A starter config is printed by a bare --config:

node npm-iface-gen.git/src/gen.cjs --config > ifacegenconfig.json

An npm consumer (an Electron renderer participant with two interfaces) looks like:

{
	"schemas": {
		"Echo":      { "src": "~/protocols.git/schemas.readonly/schema_Discovery_objs/schema_Echo.json", "sha256": "<sha256 of the schema bytes>" },
		"Discovery": { "src": "~/protocols.git/schemas.readonly/schema_Discovery.json",                  "sha256": "<sha256 of the schema bytes>" }
	},
	"targets": [
		{
			"root": "src/renderer",
			"IPSME": { "main": "App.js", "object": "App", "runtime": "electron-renderer", "shape": "component", "out": "IPSME" },
			"schema-cache": "schemas",
			"out-npm": "generated",
			"scope": "@rootintf",
			"prefix": "iface-",
			"delete": true,
			"schemas": [ "Echo", "Discovery" ]
		}
	],
	"version": 4
}

All paths in the config are resolved relative to the config file, after a leading ~ and any $VAR / ${VAR} have been expanded. The expansion is done by the generator itself, not by the shell, so it behaves the same from PowerShell, cmd, bash and an IDE.

Home directory and slashes in src

A src is usually written home-relative, so one committed config works for every developer regardless of where the consuming project is checked out:

"Discovery": { "src": "~/protocols.git/schemas.readonly/schema_Discovery.json", "sha256": "<sha256 of the schema bytes>" }
  • ~ is the current user’s home directory: $HOME on macOS and Linux (/Users/<name>, /home/<name>), %USERPROFILE% on Windows (C:\Users\<name>). The same ~/protocols.git/… therefore resolves on every OS as long as the schemas repo is checked out directly under the home directory.

  • $VAR and ${VAR} expand to that environment variable. For the home directory prefer ~ over $HOME: on Windows HOME is normally unset outside Git Bash, and inside Git Bash it is the MSYS-style /c/Users/<name>, which a native program resolves against the current drive to a path that does not exist. $HOME is fine on macOS and Linux.

  • Use forward slashes. Both generators accept / on every OS, Windows included. Backslashes work only on Windows and must be doubled inside JSON ("~\\protocols.git\\…"), so they tie the config to one platform. The slash style has no effect on the pin: the hash is over the schema’s bytes, not its path.

  • A relative src (../../protocols.git/schemas.readonly/…) resolves against the directory holding ifacegenconfig.json and instead ties the config to the checkout layout (the schemas repo as a sibling). Either form is fine; what matters for a build is the pin, and once schema-cache is populated the src need not exist at all.

Table 1. Top level
key meaning

version

The config format version. Must be 3; a missing or different value is a hard error.

schemas

The registry: "<Name>": { "src", "sha256", "debug" }. A name is declared once and selected by targets by name.

targets

The list of outputs; each entry is described below.

Table 2. Registry entry
key meaning

src

Path to the schema (relative to the config, ~/$VAR expanded) or a URL. A URL is fetched on a cache miss, verified, and written to schema-cache.

sha256

The pin: the plain sha256sum of the schema bytes, identical to the C++ generator’s. Every schema in schemas.readonly ships a .sha256 sidecar holding exactly this value. A mismatch is a hard error. The pin is over the raw bytes, so the schema must be LF; a CRLF file is rejected.

debug

true reads the src live without verifying it (a loud warning is printed). --no-debug refuses to run if any entry has it set.

Table 3. Target
key meaning

root

Optional. A relative directory under the config dir; all of the target’s write paths (out-npm, schema-cache, the IPSME out) are placed under it, and it names the target (ifacegen-base/<root>/). Without it a target is named target-01, target-02, … and paths resolve against the config dir as-is. No absolute paths and no ...

schema-cache

Directory for content-addressed copies of the verified schemas, <sha256>.json. Commit it: a build then needs neither network nor the src path.

out-npm

Directory for the generated <prefix><Name>/ packages. Required on every target.

scope

The package scope, default @rootintf; "" emits unscoped packages.

prefix

Package and directory name prefix, default iface- → directory iface-Discovery, package @<scope>/iface-discovery.

delete

true prunes any generated package in out-npm (and cache blob in schema-cache) that this run did not produce. Only generator-shaped directories are touched.

schemas

The registry names to generate for this target. A name without a registry entry is a hard error.

IPSME

Optional. Its presence requests the editable IPSME participant scaffold: main (the participant file; default main.js), object (the exported participant name; default is the stem of main), out (directory for the bridge and responders; default IPSME) and the participant’s three axes: the required shape (class

flat

component: which template), the required runtime (node

electron-main

electron-renderer

browser: its context, documented in the scaffold; native is the C++ generator’s), and the optional msgenv-import (the ME it joins, the author’s literal the bridge imports; omitted leaves a commented placeholder). A missing or unknown shape or runtime is a hard error, and so is the v3 shell.

out-cpp, namespace, *-include

A run writes, per target:

  • out-npm: one package per schema plus a <prefix>introspection package. Regenerated wholesale; never hand-edit.

  • schema-cache: the verified schema copies. Commit.

  • main and the IPSME out directory: the participant scaffold. These are yours to edit; a re-run folds template changes into them with a 3-way merge.

  • ifacegen-base/ next to the config: the merge ancestors for the scaffold. Commit.

  • A * text eol=lf .gitattributes in each managed directory, so an autocrlf=true checkout cannot smudge the committed artifacts.

CLI interface

Two modes. Config mode (--config <file>) is the one a project uses; ad-hoc mode takes interface names or schema paths on the command line and reads them from --schemas with no pins or cache. A bare --config prints the starter config; -h and -v print help and the generator version.

node npm-iface-gen.git/src/gen.cjs --config ifacegenconfig.json       # from the submodule
npx --no-install iface-gen --config ifacegenconfig.json               # when installed as a devDependency

The generator exits non-zero on any error (a version mismatch, a pin mismatch, an unknown flag), so either line can gate a build step.

iface-gen --help
iface-gen (npm) — IPSME interface package generator (config v3)

Usage:
  iface-gen [Name ...] [options]     ad-hoc: generate from --schemas DIR
  iface-gen --config <file>          config-driven (schema registry + targets)
  iface-gen --config                 print a starter config template

Options:
  --schemas DIR      schema dir for ad-hoc mode (default: ./schema)
  --out-npm DIR      output dir for the generated packages (default: ./packages)
  --scope NAME       npm scope (default: @rootintf; "" = unscoped)
  --prefix STR       package/dir name prefix (default: iface-)
  --delete           prune packages not produced by this run
  --no-debug         fail if any registry entry has debug:true
  --seed-bases       adopt existing IPSME scaffold output that has no base
  --force            corrupt schema-cache blob: re-read the src, verify the pin, rewrite the blob
  --config [file]    config mode; bare prints a starter template
  -h, --help         show this help and exit
  -v, --version      print the generator version and exit

In config mode the per-target keys supply what the ad-hoc flags would (--out-npm, --scope, --prefix, --delete); the flags that remain meaningful alongside --config are the three guards --no-debug, --seed-bases and --force.

Generating the Bridge and Responders

Beyond the per-interface packages, a config target with an IPSME block also generates an editable IPSME participant scaffold: the wiring between the messaging environment and your application that would otherwise be written by hand. The packages are regenerated wholesale and never edited. The scaffold is the opposite, a skeleton you fill in, which is why every scaffold file is written through a 3-way merge (see Merge-on-gen: these files are yours to edit).

The IPSME block

{
	"root": "src/renderer",
	"IPSME": { "main": "App.js", "object": "App", "runtime": "electron-renderer", "shape": "component", "out": "IPSME" },
	"schema-cache": "schemas",
	"out-npm": "generated",
	"scope": "@rootintf",
	"prefix": "iface-",
	"delete": true,
	"schemas": [ "Echo", "Discovery", "MessagingEnv" ]
}
  • main: the participant file (default main.js), written at the target root.

  • object: the exported participant name (default: the stem of main), the class or function component that fills the $object$ token in the templates. App.js with object App gives function App and interface_App.

  • out: the directory for the bridge and the responders (default IPSME), relative to the target root.

  • shape: required. Which template main is generated from, each a different answer to the one hard requirement, construct the bridge exactly once per participant lifetime:

    • class: a plain class, a hosted object your entrypoint constructs once.

    • flat: flat top-level code, the entry module of a Node process or of Electron’s main process. A module body runs exactly once per process, so the construct-once guarantee needs no guard.

    • component: a React component, constructing the bridge exactly once in a useRef lazy-init.

  • runtime: required. The participant’s runtime context: node, electron-main, electron-renderer or browser (native belongs to the C++ generator). It selects nothing today: it is written into the scaffold’s header as documentation, and it is the identity future context differences hang off (which reflector the context must wire) without configs ever changing.

  • msgenv-import: optional. The ME this participant joins, as the package specifier the bridge imports, e.g. @ipsme/msgenv-mqtt. It is the author’s literal, filled into the bridge verbatim and never derived from runtime or from the OS: a private ME behind a reflector pair is as valid as the OS bus. Left out, the bridge carries a commented placeholder to fill by hand. A map keyed by platform is a hard error.

    The v3 shell is a hard error that names the migration: node-boot and electron-boot were { runtime: node | electron-main, shape: flat }, node and electron-main were shape class, electron-renderer and browser were shape component.

The templates live in src/IPSME/ and src/shells/ of the submodule.

What it emits

file what

<main> (e.g. App.js)

The participant, as a class, a flat entry module or a React component according to shape. It constructs the bridge once and destructures the responder chain from it.

<out>/IPSME_Bridge.js

Wires the chain: the responder imports and their logr aliases, the LOGR_.wire list, the responder declarations, the per-base dispatch (json_msg, json_eff, json_ack, each base confirmed once before a responder sees the message) and the process_msgs reprocess loop.

<out>/Responder_<iface>.js

One per envelope schema. A live handler per message, where your handling goes, plus the base dispatch handler_json_{msg,eff,ack} that validates each message with its generated validate and forwards it narrowed. It imports its interface package by relative path into out-npm, so the generated tree is self-contained without a publish or install step.

Only envelope schemas get a responder. A schema with at least one JSON_Msg/JSON_Eff/JSON_Ack message yields a Responder_<iface>; a plain-JSON schema (Echo, Hyperport) gets its package but no responder. A participant with no envelope schema at all is valid too: the chain collapses to { MsgEnv, process_msgs } and an empty responder list.

Merge-on-gen: these files are yours to edit

Re-running the generator must not clobber your edits, so each scaffold file is written through a git 3-way merge:

  • base: the pristine generated output of the previous run,

  • ours: the file on disk now, that is the base plus your edits,

  • theirs: the freshly generated output.

git merge-file folds template changes into your file and raises conflict markers only where a regenerated line overlaps a line you touched; everywhere else the merge is silent. The base is kept in a mirror tree next to the config, ifacegen-base/<target>/<out-rel>/<file>.v<version>, one snapshot per file stamped with the generator version that produced it (a non-source extension, so it is never bundled or compiled). <target> is the target’s root, or target-01, target-02, … for targets without one. Commit ifacegen-base/ together with the scaffold: it is what makes the next merge possible on any checkout. The merge needs the git binary on the path.

Every run reports one status per scaffold file:

    ipsme -> src/renderer/IPSME  main=App.js object=App runtime=electron-renderer shape=component msgenv-import=@ipsme/msgenv-broadcastchannel
    main   -> src/renderer/App.js  [unchanged]
    bridge -> src/renderer/IPSME/IPSME_Bridge.js  [unchanged]
    resp   -> src/renderer/IPSME/Responder_Discovery.js  [created]
  • created: first run, no output yet. Output and base are written together, identical.

  • unchanged: the generator’s output equals the base, so your file is left alone. On a generator version bump the base is re-stamped, shown as [base vX → vY].

  • merged: the template changed since the base and the change folded into your file cleanly. The base advanced.

  • CONFLICTS(n): in n regions both you and the generator changed the same lines. Resolve the markers as with any git conflict; the base has already advanced, so the next run reports unchanged.

  • seeded: only with --seed-bases, see Adopting a scaffold file that already exists.

Conflicts are resolved by hand on every run that produces them; there is no travelling resolution cache (git rerere is local-only and does not survive a clone, and a committed equivalent would reimplement it). That is held back until conflict volume justifies it.

Reading a conflict

The markers are git’s diff3 style (zdiff3 with git 2.35 or newer, which keeps lines common to both sides out of the conflict), so a conflict shows three sections, with the base in the middle:

<<<<<<< yours (edited)      <- on disk now (base + your edits)
   ...A...
||||||| base (generated)    <- the last pristine generated skeleton (the common ancestor)
   ...B...
=======
   ...C...                  <- the freshly generated output
>>>>>>> generated (new)

Resolve by comparing each side with the base B in the middle:

  • A equals B: you did not touch this region, only the generator did. Take C.

  • C equals B: the generator did not change it, only you did. Keep A.

  • A and C both differ from B: both of you changed it. Merge by hand.

That middle section is why diff3 is worth the extra markers: without it you would see only your version against the new one and have to guess who changed what.

Adopting a scaffold file that already exists

The invariant is strict: output present implies a base exists, otherwise the run stops with an error. The generator will not fabricate an ancestor under a file that never descended from one, because the first merge after that would silently duplicate or corrupt it. So an existing hand-written bridge is not retrofitted with a base; you reach the managed state through created, then bring your logic onto the generated skeleton:

cp src/renderer/IPSME/IPSME_Bridge.js /tmp/IPSME_Bridge.mine.js             # 1. preserve your current bridge
rm src/renderer/IPSME/IPSME_Bridge.js                                        # 2. remove it so gen takes the `created` path
node npm-iface-gen.git/src/gen.cjs --config ifacegenconfig.json              # 3. [created]: pristine bridge + its base, in sync
git difftool --no-index src/renderer/IPSME/IPSME_Bridge.js /tmp/IPSME_Bridge.mine.js   # 4. 2-way merge your logic INTO the generated file
git add src/renderer/IPSME/IPSME_Bridge.js ifacegen-base/                    # 5. commit the reconciled file AND its base
git commit -m "adopt IPSME_Bridge under iface-gen management"

After step 3 the base equals the pristine output; in step 4 you edit that file, so your bridge genuinely descends from the base and every later run is a clean 3-way merge.

Never paste your old file over the generated stub. The base must be a true ancestor. Drop a from-scratch file on top with the stub recorded as its base, and the first template change merges as a paste-over that silently duplicates or corrupts your code.

--seed-bases is the one-time opt-in for the reverse direction: it records a base for existing output that has none and leaves the output untouched (status seeded). Use it only when you are certain the output already descends from the generated skeleton, for example after a renamed target or a lost ifacegen-base/; otherwise future merges may duplicate.