$ find . -name protocol-Discovery -type d
v3 ./protocols.git/npm-protocol-Discovery.git ./protocols.git/antora-docs/build/root-interface.se/protocols/npm-protocol-Discovery
v3 ./protocols.git/cpp-protocol-Discovery.git ./IOI.git/app-uui-win.git/cpp-protocol-Discovery.git ./IOI.git/boilerplate-tx-cpp.git/cpp-protocol-Discovery.git ./IOI.git/docker-discovered.git/discovered/trivial-discovered/cpp-protocol-Discovery.git ./IOI.git/docker-discovered.git/trivial-discovered.git/cpp-protocol-Discovery.git ./IOI.git/docker-discovery-svc.git/discovery-svc/discovery-svc/cpp-protocol-Discovery.git ./IOI.git/docker-discovery-svc.git/discovery-svc.git/cpp-protocol-Discovery.git ./IOI.git/tx-Decentraland.git/cpp-protocol-Discovery.git ./IOI.git/tx-identity-broker.git/cpp-protocol-Discovery.git ./IOI.git/tx-xxxxxx.git/cpp-protocol-Discovery.git ./IPSME.git/reflector-cpp-MQTT—asio-client.git/cpp-protocol-Discovery.git
v0.8 still supports Subverse and IOI ./protocols.git/objc-protocol-Discovery.git ./IOI.git/app-uui.git/objc-protocol-Discovery.git ./IOI.git/default-runtime-manager.git/objc-protocol-Discovery.git ./IOI.git/docker-discovery-svc.git/objc-discovery-svc-query.git/objc-protocol-Discovery.git ./IOI.git/tx-authorization.git/objc-protocol-Discovery.git ./IOI.git/tx-http.git/objc-protocol-Discovery.git ./IOI.git/tx-sl.git/objc-protocol-Discovery.git ./IPSME.git/reflector-objc-NSDNC—asio-client.git/objc-protocol-Discovery.git ./IPSME.git/reflector-objc-ws-server—NSDNC-(renamed,new-origin).git/objc-protocol-Discovery.git
● Good question — and the answer is that mature shops don't put timeouts and message-vocabulary in the same place. They layer it, and the layers are owned
by different things. Here's the typical stack, from the wire up:
1. Schema / IDL layer — the interfaces
The .proto / Avro / JSON Schema files. Pure structure. Often kept in a dedicated repo or monorepo dir (/proto, a "schemas" package) so every service
consumes the same source of truth. This is your npm-protocol-Discovery and friends.
2. Codegen layer — typed classes + convenience constructors
This is where create_with_merge()-style helpers come from. People almost never hand-write them — the toolchain generates them:
- protoc / buf generate emits message classes with builders (Foo.newBuilder().setX().build() in Java, Foo{X: ...} in Go).
- ts-proto / protobuf-ts for TypeScript emit interfaces plus create(), fromPartial(), encode()/decode().
- For JSON Schema (your world): ajv compiles validators; json-schema-to-typescript emits the types. Your hand-written merge / create_with_cause_merge
are exactly the convenience layer codegen would otherwise give you.
Key point: builders/constructors live with the schema, because they're still just "make a well-formed message." That's why having merge() in your
interface package is fine and normal — it's codegen-adjacent, not protocol logic.
3. Transport / RPC layer — where timeouts actually live
This is the big one, and the answer to "where do timeouts go." Almost nobody hand-rolls timeout logic per-interface. It's provided generically by the
RPC stack or infra:
- gRPC has deadlines built in — every call carries a deadline that propagates across hops. You set it per-call (withDeadline), not per-message-type.
- Service mesh (Istio/Linkerd/Envoy) does timeouts, retries, circuit-breaking, and deadline propagation outside the code entirely, as config.
- Client libraries (gRPC interceptors, HTTP middleware, AbortController/AbortSignal in Node, Polly in .NET, resilience4j in Java) handle
timeout/retry/backoff as cross-cutting middleware that wraps every interface uniformly.
So a service with 40 interfaces has one timeout mechanism, applied generically — not 40 timeout implementations.
4. Protocol logic layer — only for the genuinely stateful ones
Timeouts that are part of the protocol's semantics (not just transport deadlines) — like 2PhW's "abort if the participant doesn't ACK the prepare in
time," or Discovery's "expire a peer whose TTL lapsed" — those are real state machines and live in their own module, depending on the interface package
but separate from it. This is the small minority. Most "protocols" in a microservice estate are just interface + generic transport timeout.
● You actually already built it — twice. EventLog-mem.mjs has two timeout mechanisms, and the "general framework" you're reaching for is mostly a matter
of unifying them. Let me show you what's there, then where the real gap is.
What you already have
Mechanism A — active timer (add_Event, lines 52–62): pass timeout_ms + timeout_callback and it arms a setTimeout that fires the callback on expiry.
Push-based, fires on its own.
Mechanism B — the dependency queue (lines 205–233): queue_Event(evt, abs_deadline_ms) parks an event with a message-anchored absolute deadline
(Date.parse(msg.dt) + ttl, not now()+duration — so it accounts for transit time, per your comment at 186–188). purge_dequeued() then drops anything
resolved or past deadline, checked once per pump tick.
So you have both the gRPC-style "active deadline" and the IPSME-native "park-until-or-expire." This is the layer-3 equivalent for a broadcast world.
The gap
The two paths are asymmetric in a way that matters:
┌─────────────┬─────────────────────────────────────────────────────────────────────┬──────────────────────────────────┬──────────────────────────┐
│ │ fires an action on timeout? │ clock │ runs… │
├─────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────┼──────────────────────────┤
│ A │ ✅ calls timeout_callback │ local now + timeout_ms │ out of band (timer │
│ setTimeout │ │ │ thread) │
├─────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────┼──────────────────────────┤
│ B queue │ ❌ just purges silently (line 228 logs 'timed-out' but invokes │ message-anchored (sender dt + │ in-band (pump tick) │
│ │ nothing) │ ttl) │ │
└─────────────┴─────────────────────────────────────────────────────────────────────┴──────────────────────────────────┴──────────────────────────┘
Mechanism B is the right one for IPSME — message-anchored, deterministic, re-evaluated against live state each tick — but on timeout it only drops the
event; it never publishes anything. So if 2PhW parks its coordinator event waiting for votes and the deadline lapses, the parked event is silently
purged and no abort is ever published. The protocol just forgets.
Mechanism A does fire a callback, but via setTimeout it runs out of band — a timer firing mid-handler mutates responder state concurrently with the
pump, which breaks the "one pump processes everything in order" model the whole IPSME_Bridge design rests on (the skill's note about idempotence and
single-threaded reprocessing).
What "general timeout framework" should mean here
Give mechanism B an on-timeout action, fired in-band by the pump — i.e. merge A's callback into B's deterministic, message-anchored path:
queue_Event(evt, abs_deadline_ms, on_timeout) // optional action
Then split purge_dequeued() into two steps in the pump (process_msgs, the skill's drive loop):
1. process_timeouts — for each pending entry where now >= abs_deadline_ms: invoke on_timeout({idx, evt}) then drop it. The action is where 2PhW
publishes its abort, Discovery publishes peer-expiry, Echo marks the request failed.
2. purge resolved — drop dequeued entries as today.
That gives you the single shared mechanism every interface sits on — exactly the gRPC-deadline role, adapted to broadcast semantics: the timeout is how
a sender finalizes its "0..n replies" decision (give up / best-of-what-arrived / abort). Each protocol supplies only its on_timeout closure; nobody
re-implements timers.
It also retires the setTimeout path for protocol logic (keep it, if you like, only for fire-and-forget local housekeeping), removing the out-of-band
reentrancy hazard.
Two things worth confirming before I'd touch code:
1. Which path does 2PhW currently use for its timeout — the setTimeout callback (A) or the queue (B)? If it's relying on B for an abort, that abort
isn't firing today, and this is a latent bug, not just a refactor.
2. Same mechanism in cpp-EventLog? Your comment at 197 says this mirrors the C++ InMemory_EventLog. A general framework should land in both, or they'll
drift.
● README is now fully consistent — no iface-base anywhere, dual ESM/CJS documented, Model B’s consumer package.json shows the single json-msg runtime dep. So the two tasks (remove iface-base, document Model B) are done and verified (npm-iface + app both green on ESM and CJS).
On your question — Model B vs file: in subverse-map
Good instinct, and worth separating two things:
The generated packages (iface-discovery, iface-echo) — yes, Model B is the better fit, even now. Right now subverse-map reaches into the generator's output dir via file:../../protocols.git/npm-iface.git/packages/iface-Discovery. That's fragile: that packages/ dir is regenerated wholesale (I rm -rf packages each run), so the app is coupled to whatever the generator last emitted. Model B flips it: subverse-map runs iface-gen Discovery Echo --out-npm ./src/generated and owns its generated code in its own tree, decoupled from the generator's workspace — and it's the exact pattern you'll publish with, so it's good dogfood.
json-msg — stays a normal dependency, not Model B. It's not generated; it's a real package. So it's file: now (local, unpublished) and becomes a registry ^2.2.0 once you publish. Model B doesn't touch it.
So "the relevant packages" splits cleanly: the generated ones (discovery/echo) → Model B local generation; the runtime one (json-msg) → ordinary dep.
My honest take on timing: file: for the generated packages works and is verified, so switching is optional churn while everything is local. But since (a) it's the published pattern, (b) it decouples the app from the wholesale-regenerated packages/, and (c) it proves the Model B flow end-to-end, I'd go ahead and convert subverse-map now. Concretely: - devDep @rootintf/iface-gen (file: to npm-iface), - add "gen": "iface-gen Discovery Echo --out-npm ./src/generated --no-cpp" + wire it before build, - drop the two file: iface deps, keep json-msg, - repoint the two imports (main.ts, Responder_Discovery.js) at ./src/generated/iface-Discovery/....
Want me to do that conversion now? It's the cleaner end state and lets you see Model B running in a real app — I'll re-verify ESM + CJS after.
● Yes — absolutely. That’s just Model A, and the generator supports it fully. Each generated @rootintf/iface-<name> is a complete, standalone, publishable npm package (own package.json, dual ESM/CJS, own deps, own version). You npm publish each one exactly like you publish @rootintf/protocol-discovery today.
The key realization
Your original gripe was "sick of creating npm packages for each interface" — i.e., hand-authoring the scaffolding (package.json, build, validate wrappers). The generator already killed that, independent of how you distribute. So per-package publishing now costs you only the publish step, not the authoring. And your consumers' experience stays identical to today: npm i @rootintf/iface-discovery # same shape as: npm i @rootintf/protocol-discovery No toolchain at the consumer, per-package semver, the works. Model B was never required — it's just the alternative that trades N publishes for one (publish only the generator). If you don't mind publishing each, Model A is perfectly good.
Two ways to organize it
-
Keep one git repo per protocol (your current topology). Point the generator’s output at each repo and publish from there: iface-gen Discovery --out-npm ../npm-protocol-discovery.git --no-cpp # then: cd ../npm-protocol-discovery.git && npm publish
-
The repo’s contents become generated instead of hand-written, but the per-repo + per-package model is untouched.
-
One monorepo (npm-iface/packages/* with workspaces) and publish each package from there (a tool like changesets publishes only the ones whose _v/version changed).
What stays separate
@rootintf/json-msg remains its own published package (the shared runtime + base types); every envelope iface-* package declares it as a dependency (^2.2.0), so it installs transitively. That's the one thing that's not per-protocol — it's the common base, published once.
Versioning fits this cleanly
Each package publishes as 0.1.0-api.N. Change a schema → bump that schema's _v → only that package's -api.N moves → republish just that one. Independent per-protocol releases, exactly like now.
So the honest summary: per-package publishing is fully viable and arguably the path of least surprise for your existing consumers. The generator just means you stop hand-writing those packages — you regenerate and publish. The only reason to prefer Model B is if publishing N packages per release becomes a chore you'd rather collapse into one.
One naming note worth deciding: the generator emits iface-<name>. If you want these to be drop-in replacements for the existing @rootintf/protocol-<name> packages (so consumers don't change import specifiers), you'd want the generator to emit that name instead — a small, deliberate change. Want me to make the package-name prefix configurable (--scope/--prefix) so you can publish them as protocol-* if you choose?