Legion L3 — Capability-Provider SDK (writing modules)

Status: DRAFT (Foundation 09). This is the first visible cut of the L3 module-author API. It is a draft, not a release candidate — names and signatures may change. Start with architecture.md for the layer model and abi.md for the linkage/boundary rules.

What L3 is

L3 is the provider surface: you write a module that offers one or more capabilities (e.g. kvstore, ai.stt.transcribe, audio.out). A module declares the capabilities it provides, implements a handler for its methods, and — once its node holds a certificate authorising those capabilities — they are automatically advertised to the cohort (declaring is advertising; there is no separate publish step).

L3 is an in-process header contract, not the dynamic ABI. A module includes a single header and is either compiled into the runtime or loaded at runtime; the L3 symbols it calls (legion_module_register, legion_l3_kv_set, …) are not exported by liblegion.so. They resolve against the host executable (the runtime/tools link the static library with --whole-archive --export-dynamic). See abi.md.

#include "legion/sdk.h"        /* the entire L3 module-author surface */
#include "legion/legion_pal.h" /* legion_pal->alloc / ->free, time, locks */

A module source may include only sdk.h, legion_pal.h, and the SDK allowlist; reaching below L3 (core/plugin.h, core/consensus.h, core/lnmp_*, …) is rejected by the Foundation 04.01 CI gate (check_module_includes.sh).

The shape of a module

A module is one legion_module_registration_t record (sdk.h:213):

The handler (sdk.h:176) — one function, dispatched for every method call, local or delegated:

int handler(const char *method_name, legion_map_t *params, int caller_tier,
            void **result, size_t *result_len, void *plugin_data);

Return LEGION_OK and optionally allocate *result (via legion_pal->alloc; the runtime frees it), or return a LEGION_ERR_* code and leave *result unset. Handlers may run concurrently — synchronise your own module state.

A method declaration (legion_module_method_decl_t, sdk.h:112) tells the runtime how to treat each method:

Worked example — a minimal capability-providing module

This is the real, tested fixture src/sdk_demo/sdk_demo_module.c (the smallest module that proves the full dlopen + signed-manifest + L3-KV path). It provides the capability sdk_demo.test with two methods: noop (non-mutating) and set_pair (mutating, writes through the L3 KV wrapper).

#include "legion/sdk.h"
#include "legion/legion_pal.h"
#include <string.h>

/* Per-instance data. pm is wired in at init so a mutating handler can reach
 * the L3 KV wrapper. */
typedef struct { legion_plugin_manager_t *pm; } sdk_demo_data_t;

static void *sdk_demo_create(legion_map_t *config) {
    (void)config;
    sdk_demo_data_t *d = legion_pal->alloc(sizeof(*d));
    if (d) d->pm = NULL;
    return d;
}
static void sdk_demo_destroy(void *data) { if (data) legion_pal->free(data); }

/* A non-mutating method: allocate the result with the PAL allocator. */
static int sdk_demo_noop(void **result, size_t *result_len) {
    char *r = legion_pal->alloc(3);
    if (!r) return LEGION_ERR_NOMEM;
    memcpy(r, "ok", 3);
    *result = r; *result_len = 3;
    return LEGION_OK;
}

/* A mutating method: store (key,value) via the L3 KV wrapper. */
static int sdk_demo_set_pair(sdk_demo_data_t *d, legion_map_t *params, int caller_tier) {
    if (!d || !d->pm || !params) return LEGION_ERR_INVALID_ARG;
    size_t klen = 0, vlen = 0;
    const char *key   = legion_map_get(params, "key", &klen);
    const void *value = legion_map_get(params, "value", &vlen);
    if (!key || !klen || !value) return LEGION_ERR_INVALID_ARG;
    return legion_l3_kv_set(d->pm, key, value, vlen, caller_tier);
}

/* The single handler routes by method name. */
static int sdk_demo_handler(const char *method, legion_map_t *params, int caller_tier,
                            void **result, size_t *result_len, void *plugin_data) {
    sdk_demo_data_t *d = plugin_data;
    if (!d) return LEGION_ERR_INTERNAL;
    if (!strcmp(method, "noop"))     return sdk_demo_noop(result, result_len);
    if (!strcmp(method, "set_pair")) return sdk_demo_set_pair(d, params, caller_tier);
    return LEGION_ERR_NOT_FOUND;
}

/* Method + parameter metadata. */
static const legion_method_param_t set_pair_params[] = {
    { "key",   "string", "Key to set",  true },
    { "value", "bytes",  "Value bytes", true },
};
static legion_module_method_decl_t sdk_demo_methods[] = {
    { .name="noop",     .mutates_state=false, .min_tier=3, .return_type="string",
      .description="Non-mutating no-op; returns \"ok\"" },
    { .name="set_pair", .mutates_state=true,  .min_tier=3,
      .params=set_pair_params, .param_count=2,
      .description="Mutating: stores (key,value) via legion_l3_kv_set" },
};

/* The capability this module provides — declaring it advertises it. */
static const char *sdk_demo_provided_caps[] = { "sdk_demo.test" };

const legion_module_registration_t legion_module_sdk_demo_reg = {
    .name = "sdk_demo", .version = "0.1.0",
    .provided_caps = sdk_demo_provided_caps, .provided_cap_count = 1,
    .methods = sdk_demo_methods, .method_count = 2,
    .handler = sdk_demo_handler,
    .create = sdk_demo_create, .destroy = sdk_demo_destroy,
    .category = LEGION_MODULE_LOADABLE,
};

/* Loadable-module entry point (only when built as a .so). */
#ifdef LEGION_BUILDING_LOADABLE_MODULE
int legion_module_init(legion_plugin_manager_t *pm) {
    int rc = legion_module_register(pm, &legion_module_sdk_demo_reg);
    if (rc != LEGION_OK) return rc;
    sdk_demo_data_t *d = legion_plugin_get_module_data(pm, "sdk_demo");
    if (d) d->pm = pm;          /* wire pm so set_pair can use the KV wrapper */
    return LEGION_OK;
}
void legion_module_fini(void) {}
#endif

Two ways this same source ships:

• Built-in (category aside, compiled into the runtime): registered by the runtime calling legion_module_register(pm, &legion_module_sdk_demo_reg) directly — no init export (define LEGION_BUILDING_LOADABLE_MODULE only for the .so).
• Loadable (.so): exports legion_module_init (or legion_module_init_v2 for a config map) + optional legion_module_fini; the loader resolves them (LEGION_MODULE_INIT_SYMBOL), verifies the module's signed manifest, calls init, and registers the result.

Once registered on a certified node, sdk_demo.test is advertised to the cohort automatically; a consumer (L4) on any cohort node can then discover and invoke it. You never call an "advertise" function.

Reference

Registration & lifecycle

Dispatch (calling other modules / capabilities)

L3 KV wrapper (sdk.h:497)

Typed facade over the kvstore module. Mutating ops route through consensus when the manager's submit_fn is wired. Tier rule: target_tier >= caller_tier (lower number = more privileged); _set is shorthand for _set_at_tier(…, caller_tier, caller_tier). Values from _get are legion_pal->alloc-owned — free with legion_pal->free.

LEGION_L3_KV_KEY_MAX = 64.

L3 capabilities & membership (sdk.h:628)

Reads are local (consensus-VM maps, no network); writes build/sign/submit the on-wire envelope through submit_fn (require it wired, else LEGION_ERR_NOT_INITIALIZED). No read-your-writes on writes (apply runs after commit). Read results are legion_pal->alloc-owned arrays.

Bounds: LEGION_L3_CAPS_MAX_CAPS = 64, LEGION_L3_CAPS_MAX_CAP_NAME_LEN = 64.

Futures (async delegation results) (sdk.h:269)

legion_future_create / _destroy / _is_ready / _wait(timeout_ms) / _get(result,len) / _on_complete(cb,ctx) — plus _set_on_part / _deliver_part for streaming responses. (_resolve is normally called by the runtime, not modules.)

State sync (legion_module_sync_t, sdk.h:144)

Set reg->sync if your module holds state that must survive epoch rollover / late-joiner catch-up: snapshot/restore (small states) or the chunked snapshot_begin/_chunk + restore_begin/_chunk/_finalize pair (large states).

Loadable-module ABI (sdk.h:805)

Export legion_module_init (legion_module_init_fn, receives pm) or legion_module_init_v2 (…_fn_v2, receives pm + a config map), plus optional legion_module_fini. The loader resolves them by the LEGION_MODULE_INIT_SYMBOL[_V2] / _FINI_SYMBOL names.

Key rules & gotchas

• Memory ownership: allocate results with legion_pal->alloc; free library-returned buffers with legion_pal->free — not libc free (the PAL may use HeapAlloc/HeapFree on Windows).
• mutates_state ⇒ consensus. A mutating method only converges across the cohort when the manager's submit_fn is wired (the node runtime does this); otherwise it applies locally and still returns LEGION_OK.
• Tiers are privilege levels 0(sovereign)…3(interface), enforced as target_tier >= caller_tier and per-method min_tier. Tiers are orthogonal to the cap semantic (the interaction shape).
• Declaring is advertising. Put a cap in provided_caps; a certified node advertises it automatically. Modules may also drive legion_l3_caps_advertise for dynamic cases. Apps never advertise.
• Concurrency: handlers can be called from multiple threads at once.
• Include discipline: only sdk.h + legion_pal.h + the SDK allowlist; the CI gate rejects below-L3 includes.

Status

DRAFT — surfaced for review. Expect changes. Companion docs: architecture.md (the layer model), abi.md (the ABI boundary), and an L4 application reference + guide (next).