Architecture

libsig is a C11 library for building signal-processing applications out of small units of logic (“capsules”) that exchange values (“signals”) along configured connections. This document explains how the pieces fit together, who owns what, and which threads touch which data. For per-feature design history see docs/superpowers/specs/ (the design of each piece) and docs/superpowers/plans/ (how it was built).

Related documents:

• Architecture site: the explorable C4 model (how it is built)
• Project configuration: the on-disk project format
• Management API: the sig_mgmt operations for managing a running instance
• REST API: the management HTTP API (one transport over it)
• Writing capsules: adding your own logic
• Examples: runnable worked cases
• Web UI: the browser editor for a running host

Concepts

Term	Meaning
Value (sig_value)	The data that flows: int, bool, double, string, bytes, or nested map/list.
Signal	A sig_value emitted on an output terminal and delivered to input terminals.
Capsule (sig_capsule)	Compiled-in code: a name, input/output terminal tables, a table of the properties it accepts (sig_property) and optional start/stop hooks. Available to every project.
Capsule configuration (sig_capsule_configuration)	A named, configured capsule: a capsule plus properties (configurations/<Name>.cfg). Available to any service in the project.
Capsule instance (sig_capsule_instance)	One configuration as used by one service, with its own storage, state and routes. This is what runs, and what appears on the canvas.
Terminal	A named input (with a handler) or output on a capsule.
Service (sig_service)	A worker thread plus signal queue and job scheduler that owns a set of instances.
Project (sig_project)	A project loaded from a persistence store (a directory of .cfg files, with the file backend): capsule configurations and services, wired together.
Runtime (sig_runtime)	Owns a project and the optional runtime modules; drives the start/stop lifecycle.
Runtime module	An optional runtime feature (currently the HTTP server, management, the glue that serves management over HTTP and the signal feed).
Host	A program (sighost) that creates a runtime, registers capsules and runtime modules and waits for shutdown.

Naming

Three layers, in a 1 : n : n relationship:

• A capsule is the compiled-in implementation (simulator, counter, logger, filter, switch, state_change, weather). There is one of each, available to every project.
• A capsule configuration is a named configuration of one capsule, such as Sim (a simulator with interval_ms 250). It lives in configurations/<Name>.cfg, and any service in the project may use it. One capsule has many configurations.
• A capsule instance is one configuration used by one service. It is what runs, and what appears on the canvas. One configuration has an instance in each service that lists it.

In the files the layers appear as: the "type" key of a configuration names its capsule; a service’s execution entry names a configuration. In the REST API an instance reports "type" (its capsule) and "configuration".

“Module” on its own always means a runtime module (sig_runtime_module: the HTTP server, management, the glue between them). A capsule is never called a module.

Today an instance takes the name of its configuration, so a configuration appears at most once per service; two simulators in one service need two configurations.

System at a glance

flowchart LR
    subgraph Browser
        UI[Web UI<br/>React + React Flow]
    end
    subgraph sighost process
        HTTP[sig::http_server<br/>server thread] --> GLUE[sig::mgmt_http<br/>router + JSON + status mapping]
        GLUE --> MGMT[sig::mgmt<br/>operations on sig_value trees]
        MGMT --> RT[sig_runtime]
        RT --> PRJ[sig_project]
        PRJ --> SVC1[service A<br/>worker thread]
        PRJ --> SVC2[service B<br/>worker thread]
    end
    FILES[(project store<br/>configurations/*.cfg<br/>services/*.cfg)]
    UI -- REST/JSON --> HTTP
    PRJ <-- load / write<br/>via sig_persistence --> FILES
    CLI[curl / scripts] -- REST/JSON --> HTTP

The project store (a directory, with the file backend) is the source of truth. The runtime loads it into services; the management API reads the live services and rewrites the files; the UI is just a client of that API and holds no state of its own beyond an unsaved draft.

Layering

 ┌─────────────────────────────────────────────────────────────────┐
 │ host/posix  main.c, sigwait shutdown        (sighost binary)    │
 │ host/       sig_host_run: portable boot sequence                │
 ├─────────────────────────────────────────────────────────────────┤
 │ capsules/   one folder per capsule, and their registration      │
 ├─────────────────────────────────────────────────────────────────┤
 │ src/mgmt/   sig::mgmt (operations)   sig::mgmt_http (REST glue) │   optional
 │ src/http/   sig::http_server  sig::http_client (curl)           │   optional
 │ src/net/    sig::net (TCP sockets)                              │   optional
 ├─────────────────────────────────────────────────────────────────┤
 │ sig_runtime    lifecycle, runtime modules, service control      │
 │ sig_project    load a store -> services + configurations        │
 │ sig_config     parse .cfg JSON -> capsules, services, wiring    │
 │ sig_service    worker thread, signal queue, job scheduler       │
 │ sig_capsule    capsules, configurations, instances, routes      │   sig::sig
 │ sig_value  sig_json  sig_ht  sig_ll  sig_log  sig_persistence   │
 │ sig_thread  sig_time  sig_poller   OS-specific, behind headers  │
 │ sig_alloc      allocator (leak-tracking with SIG_DEBUG)         │
 └─────────────────────────────────────────────────────────────────┘

Build targets (CMake):

Target	Contains	Enabled by
sig::sig	the core above (including sig_json, the streaming JSON writer)	always
sig::net	src/net/sig_net_posix.c	SIG_NET
sig::http_server	src/http/sig_http_server.c, sig_http.c (parser/formatter)	SIG_HTTP_SERVER (needs SIG_NET)
sig::http_client	src/http/sig_http_client_curl.c	SIG_HTTP_CLIENT (needs libcurl)
sig::mgmt	src/mgmt/sig_mgmt.c: transport-neutral operations; no HTTP or socket dependency (links only sig::sig)	SIG_MGMT
sig::mgmt_http	src/mgmt/sig_mgmt_http.c (glue) and sig_mgmt_rest.c (private router, status mapping, JSON)	automatic when SIG_MGMT and SIG_HTTP_SERVER are both on
sig::host	host/sig_host.c	SIG_HOST
sig_capsule_<name>, sig_builtin_capsules	built-in capsules	SIG_HOST, SIG_CAPSULES
sighost	host/<platform>/ entry point	SIG_HOST

Everything OS-specific is behind sig/platform/sig_thread.h and sig_time.h (threads, mutexes, condition variables, the reader/writer lock sig_rwlock that sig_mgmt uses, clocks), sig_poller.h and sig/net/sig_net.h (TCP). Only POSIX backends exist today (src/platform/posix/); the rest of the code does not include OS headers.

Data flow

A signal travels from a capsule’s output terminal to the input terminal of another capsule:

1. A capsule calls sig_capsule_instance_notify(src, "out", &value).
2. src->routes maps the output terminal name to a list of (destination instance, input terminal) routes, set up at load time from the service file’s receivers. For every route:
3. sig_service_enqueue() on the destination’s service copies the value (sig_value_dup) into a queue entry and signals the service’s condition variable. The caller never blocks on the receiver.
4. The destination service’s worker thread dequeues the entry and calls terminal->handler(instance, &signal, terminal_name).

Consequences:

• Delivery is asynchronous and per-destination-service FIFO. There is no ordering guarantee between signals queued by different sources.
• Sender and receiver may be in the same service or different ones; the path is identical.
• Notifying an output terminal with no routes silently drops the signal. With several routes delivery is best-effort: one failing destination does not stop the rest, and the last failure status is returned.
• The queue copies values, so the sender keeps ownership of what it passed.

Scheduling

Capsules may schedule periodic or one-shot callbacks that run on their service’s worker thread (sig_capsule_instance_schedule). The worker uses a timed wait on its condition variable, so an idle service consumes no CPU. Minimum interval is 1 ms; a repeating job that overruns its interval skips missed fires rather than bursting to catch up; job nodes are retained until the service is destroyed, so do not schedule unbounded numbers of one-shot jobs.

Values

sig_value is a small tagged union. Strings, bytes, maps and lists have refcounted payloads: sig_value_copy shares, sig_value_dup deep-copies. Containers are mutable only while uniquely owned; once shared they are frozen and mutators return SIG_ERR_STATE. put/append move the child in. sig_value_to_json / sig_value_from_json convert to and from JSON (sorted keys, base64 for bytes, strict parser, nesting limited to SIG_VALUE_MAX_DEPTH = 32).

Lifecycle

Runtime state machine

 CREATED ──start──▶ STARTING ──▶ RUNNING ──stop──▶ STOPPING ──▶ CREATED
    ▲                                                              │
    └──────────────────────── may be started again ────────────────┘

sig_runtime_start():

1. Loads the project from the store (sig_project_load). Nothing is running yet.
2. Two phases over the registered runtime modules: create for every module in registration order, then start for every module in registration order.
3. Starts every service with "auto_start": true. A failure here is logged but does not fail the call.

If steps 1–2 fail, everything done so far is unwound in reverse order and the runtime returns to CREATED.

sig_runtime_stop() stops all services, then stops every module, then destroys every module (each in reverse order), then destroys the project.

Shutdown is requested with sig_runtime_request_shutdown() and observed by sig_runtime_wait(). The request is thread-safe but not async-signal-safe, so hosts block SIGINT/SIGTERM and call it from a dedicated sigwait() thread. POST /system/shutdown takes the same path.

Host boot sequence (sig_host_run)

open store → create runtime → register modules → arm platform shutdown trigger
  → sig_runtime_start → sig_runtime_wait → sig_runtime_stop
  → disarm trigger → destroy runtime → close store

A failure at any step unwinds the earlier ones. sig_host_platform is the seam for platform-specific shutdown triggers (host/posix/ provides the sigwait one). sig_runtime_start() clears any shutdown request made before it, which is why the trigger is armed first but only matters afterwards.

Service lifecycle

• sig_service_start() runs each owned instance’s start hook synchronously on the calling thread, then spawns the worker thread. By the time it returns the hooks have run.
• sig_service_stop() joins the worker thread first, then runs each instance’s stop hook on the calling thread.
• Starting a started service is SIG_ERR_EXISTS; stopping or destroying a never-started or stopped service is a safe no-op.

Editing and the dirty flag

Edits arrive through the management API (PUT/PATCH/DELETE) and are applied to the project under a runtime-wide edit lock:

sequenceDiagram
    participant C as Client (UI / curl)
    participant M as sig::mgmt
    participant P as sig_project
    participant D as Disk
    C->>M: PATCH /configurations/Sim
    M->>P: validate against capsule property table
    P->>D: save configurations/Sim (atomic write)
    P->>P: mark services using Sim dirty
    M-->>C: 200 (new document)
    Note over P: running services are untouched
    C->>M: POST /services/test/restart
    M->>P: stop, rebuild from stored definitions, start
    M-->>C: 200 state running (dirty false)

A stopped service is rebuilt immediately on edit; a running one keeps running its old instances until stopped, so an edit can never half-apply. Nothing validated is written if validation fails, and a failed disk write leaves the old file and definition in place.

Runtime modules

A sig_runtime_module is {name, create, start, stop, destroy} with every hook except name optional. Startup is two-phase: every module is created in registration order, then every module is started; shutdown stops every module, then destroys every module, in reverse. A module may therefore rely on the state of those registered before it (sig_runtime_module_state). To serve management over HTTP register http_server, then mgmt, then mgmt_http: the glue looks up both in its create. The shape deliberately mirrors sig_persistence_module; future runtime features (IPC, security) are meant to plug in the same way.

Ownership

Thing	Owner	Notes
Capsule registry	the host	borrowed by runtime/project; must outlive the runtime
Capsules	static data	borrowed by configurations; must outlive them
Capsule configurations	project	destroyed with the project
Capsule instances	project (via an owned list)	services do not own their instances
Services	project	stopped before destroy
Runtime module state	runtime	created/destroyed with the runtime lifecycle
Signals in flight	the destination service’s queue	duplicated on enqueue

sig_project_destroy() stops every service, then destroys instances, services and configurations, in that order.

Threading model

Threads in a running system:

Thread	Runs
Host main thread	sig_runtime_start/wait/stop; module create/start/stop hooks; capsule start/stop hooks
One worker per running service	terminal handlers and scheduled jobs of that service’s instances
http_server server thread	accepts one connection at a time and runs the registered handlers, for management sig_mgmt_http’s REST handler (absent in single-threaded mode: see below)
signal_feed thread	the feed’s WebSocket connections, when the runtime has no loop; with a loop (-s) they run on the loop thread. Absent unless SIG_SIGNAL_FEED
Platform shutdown thread	blocks in sigwait() and calls sig_runtime_request_shutdown

Rules:

1. An instance belongs to its service’s thread. Once added to a started service, only that thread may touch the instance’s storage and state (from handlers, jobs, or lifecycle hooks). The one thread-safe way to affect it from elsewhere is a signal (notify/enqueue). The sig_ht containers are not internally synchronized.
2. Never call runtime control from service threads. sig_runtime_service_* and sig_runtime_uptime_seconds take a control mutex that is held across service start/stop joins. Calling them from a service or capsule thread can deadlock. Call from module, transport or host threads only.
3. sig_mgmt operations need the runtime RUNNING (SIG_ERR_STATE otherwise) and must not be called from service or capsule threads (see Management API). Transports call them from threads their module’s stop hook joins (the HTTP server thread).
4. Initialize logging before spawning threads that log (sig_log_init).
5. sig_runtime_service_* calls are serialized and rejected unless the runtime is RUNNING, so they cannot race sig_runtime_stop().
6. Introspection (sig_project_walk_services, sig_service_walk_instances, sig_capsule_configuration_walk_properties) is safe from any thread once the project is loaded, because the sets and configuration properties are not modified after load.
7. sig_net_* timeouts: a negative timeout_ms waits forever (it goes to poll()).

The signal tap

sig_runtime_set_tap installs one observer that sees every signal crossing a terminal boundary; the signal feed is its only user. It fires in two places: for an output, in sig_capsule_instance_notify before the route lookup (so it sees signals on unrouted outputs too), and for an input, in the service just before the terminal’s handler runs. The callback therefore runs on whichever thread emits: a service worker, or the loop thread when services are driven, and it must not block or call into the runtime. The feed’s callback copies the value into a bounded per-subscription queue and counts drops instead of waiting.

The tap is installed only while at least one subscription exists, so with no subscribers the signal path pays one atomic load per signal. It is applied to every service, including ones started or restarted later, but a runtime restart forgets it (see Restart behaviour).

Single-threaded operation

The threads above are the default. A host can instead run everything on one thread (sighost -s, sig_host_config.single_threaded), for hardware where threads are scarce or absent. The portable pieces:

Piece	Role
sig_poller (sig_poller.h)	The readiness backend: add/modify/remove an fd, wait with a timeout, wake. Exactly one implementation is linked, chosen by -DSIG_POLLER=<name> (src/platform/posix/sig_poller_<name>.c). Three exist: poll (portable), kqueue (macOS/BSD, the default there; the wake is an EVFILT_USER event rather than a pipe) and epoll (Linux; opt in with -DSIG_POLLER=epoll, one eventfd for the wake). test/test_sig_poller.c is the contract every backend must pass, and it is built and run once per backend the system supports.
sig_loop (sig_loop.h)	fd watches, one-shot timers and a per-iteration step hook, over a poller. Backend-independent.
sig_service_set_driven() / sig_service_step()	A driven service starts no worker thread; each step delivers the queued signals and runs due jobs, then reports its next deadline.
sig_runtime_set_loop() / sig_runtime_step()	Gives the runtime a loop: every service (including ones started or restarted later) becomes driven, and the host’s step hook calls sig_runtime_step(). Cross-thread work and shutdown requests wake the loop.

sig_host_run creates the loop, installs the step hook and runs it where it would otherwise call sig_runtime_wait(). Runtime modules find the loop with sig_runtime_loop(): http_server then serves from it (non-blocking connections, one timer each, at most four at once) instead of its own thread, and weather needs no thread of its own in either mode. Handlers, jobs and management requests all run on the loop thread, so rule 1 still holds with one thread.

What still runs on its own thread in this mode: the platform shutdown thread (sigwait; a port can call sig_runtime_request_shutdown from its own trigger) and, while a request is in flight, the libcurl port of sig_http_get_async (one short-lived thread per request). A port with no curl or threads implements sig_http_get_async (sig_http_client.h) natively and calls its completion callback from its own loop.

Rules for code that runs in either mode: never block in a handler, job or completion callback; hand results from other threads back to the service with sig_service_schedule_job (thread-safe, wakes the loop); a runtime module that registers fds or timers on the loop removes them in its stop hook.

Management subsystem

sig_mgmt is the supported API for managing a running instance; nothing outside it should mutate a running project (a documented rule, not enforced). It is a set of transport-neutral operations returning sig_value trees and sig_status_t codes. Transports sit on top. Dependencies point downward only:

 HTTP client ──▶ sig::http_server ──▶ sig::mgmt_http ──operations──▶ sig::mgmt ──▶ sig_runtime API
                 (sockets, parsing,   (glue: router,   (sig_value     (validation, snapshots,
                  limits, CORS)        JSON, status     trees,          locking)
                                       mapping)         sig_status_t)
 REPL / ssh / IPC module (yours) ───────────────operations──────────▶ sig::mgmt

• sig::mgmt (sig_mgmt.h, docs/management-api.md) has no HTTP dependency: no paths, status codes or response format (definition documents go in and out as JSON text, parsed into trees), and it links only sig::sig. It reads state through the public runtime/project/service introspection API and changes it only through sig_runtime_edit, sig_runtime_service_* and sig_runtime_request_shutdown. It owns validation, view shaping and one reader/writer lock (sig_rwlock): reads build owned snapshots under the shared lock, edits take it exclusive. Starting a stopped, dirty service rebuilds it under the write lock and then starts with sig_runtime_service_start_prebuilt, which reports SIG_ERR_STALE rather than rebuild.
• sig::http_server (sig_http_server.h) is the shared HTTP/1.1 server: one server thread (or the runtime’s loop) that parses a minimal request, enforces size and time limits, applies CORS and dispatches to the handler registered for the longest matching path prefix. It owns the bind, port, cors_origin and timeout_ms configuration, and listens only if some module registered routes. Read the bound port with sig_http_server_get_port(). sig_http.c (parser/formatter) is private; sig_json (the streaming JSON writer, core in sig/value/sig_json.h) is public.
• sig::mgmt_http (sig_mgmt_http.h) is the glue: it registers the root path with the HTTP server and forwards requests to its private REST layer (sig_mgmt_rest.c), which holds the route table, 400/404/405 handling, the mapping from sig_status_t to HTTP statuses and error bodies, and the JSON rendering of result trees. The routes live at the root (/system, /services/...). It takes no config; a non-empty one is rejected so old port=... use fails loudly. It is built only when SIG_MGMT and SIG_HTTP_SERVER are both on; a SIG_MGMT-only build has the operations and their tests but no REST layer.
• Authorization has a single seam, request_authorized(). There is no authentication yet.

Another transport (a serial link, a REPL, an ssh text tool, an IPC executable) is a runtime module that calls sig_mgmt_from_runtime and the operations directly. See management-api.md, and rest-api.md for the REST endpoints.

Web UI

ui/ is a separate TypeScript application (React, React Flow, TanStack Query, Material UI) that talks only to the REST API; the C code has no knowledge of it beyond the optional CORS origin (sighost -o).

 api/ (zod-validated client) ─▶ state/ (queries, polling) ─▶ graph/ (model + ELK layout)
                                      │                              │
                                      ▼                              ▼
                               edit/ (draft, validate, save) ─▶ canvas/ + panels/

• It draws the stored definition (GET /services/{name}/config), not the live service, so a service with pending edits shows what it will run after a restart.
• Edits accumulate in a client-side draft. Save first re-reads the host to detect conflicts, then PUTs each new or changed configuration, then PATCHes the service. A failure part-way keeps what was written, and a retry rebases the draft onto it.
• Validation duplicates the capsule property tables from GET /capsules, so the form can show errors before the host would reject them. The host remains the authority.
• It polls every 10 s (the API has no push channel); canvas positions are kept in browser storage per host and service.

Persistence

sig_persistence is a pluggable store interface keyed by (namespace, key) (create, load, save, remove, list) with a registry. The project lives in it: sig_project_load(store, registry, out) reads the configurations and services namespaces, sig_runtime_config carries the store (borrowed), and every project edit saves through it. The file backend (sig_persistence_file_module, configured with a base directory) stores <base>/<ns>/<key>.cfg for configurations and services and .sig for any other namespace, writes atomically (temp file, fsync, rename) and creates missing directories on save. A namespace matches [A-Za-z0-9_-]{1,64}; a key is 1 to 128 bytes with no /, no .. and no control characters. Consequently, if the project directory is removed after boot, a runtime restart loads an empty project and the next save recreates the directories. The edit API’s own name rule ([A-Za-z0-9_-]{1,64}) is stricter than the key rule, so files with other sane names still load but cannot be created through the API.

Error handling

Functions return sig_status_t: SIG_OK (0) or a negative SIG_ERR_* (ALLOC, INVALID_ARG, NOT_FOUND, EXISTS, PLATFORM, TYPE_MISMATCH, PARSE, STATE, TIMEOUT, STALE). Constructors write their out-parameter only on success and leave nothing allocated on failure. Allocation goes through sig_alloc; building with -DSIG_DEBUG=ON enables leak tracking.

Repository map

include/sig/<area>/ public headers (API surface), one folder per function
src/<area>/         implementation, mirroring include/sig/<area>/
                    (base value runtime project persistence net http mgmt platform)
src/platform/posix/ OS backends (threads, clocks, poller)
src/net/, src/http/ POSIX socket backend, HTTP server/parser and curl client
capsules/<name>/    one folder per built-in capsule (source, header, tests)
capsules/           the registration shim (sig_builtin_capsules)
host/               sig_host (portable; header in host/include/sig/) and
                    host/<platform>/ (entry point)
examples/           runnable sample projects
test/               cmocka unit tests, one binary per module
docs/               this documentation; superpowers/ holds design specs and plans

Extension points

To add	Do
A capsule	writing-capsules.md
A runtime module	Implement a sig_runtime_module and register it in the host before sig_runtime_start
A management transport	A runtime module registered after mgmt that maps its framing onto the sig_mgmt_* operations (management-api.md; sig::mgmt_http does this for REST on sig::http_server)
A persistence backend	Implement a sig_persistence_module (namespaced load/save/remove/list); a host selects it as its store (sig_host_config.store_module)
A host platform	Add host/<platform>/ with a CMakeLists.txt producing an executable that links sig_host and sig_builtin_capsules, a sig_host_platform, and an entry point; build with -DSIG_HOST_PLATFORM=<platform>
A platform backend	Provide the sig_thread.h, sig_time.h and sig_poller.h functions (and optionally sig_net.h) for your OS