The UnShell protocol is a tree-addressed packet protocol for remote procedure calls and bidirectional hook-backed data exchange across a hierarchy of connected endpoints.
Small means the core protocol stays narrow enough for constrained implementations. Extensible means new behavior is introduced through leaves, procedures, and payload schemas instead of frequent protocol redesign. Canonical means there should be one clearly defined way to express each core protocol behavior.
This document combines exact protocol definition with rationale. Rationale blocks explain why a rule exists, but do not define interoperability requirements.
> **Rationale:** This document uses a formal specification layout: descriptive sections first, exact protocol definition later, and rationale kept adjacent to the rules it explains.
The key words `MUST`, `MUST NOT`, `REQUIRED`, `SHALL`, `SHALL NOT`, `SHOULD`, `SHOULD NOT`, `RECOMMENDED`, `MAY`, and `OPTIONAL` in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) when, and only when, they appear in all capitals.
Unless a section is explicitly marked otherwise, sections labeled `Normative` define protocol requirements and sections labeled `Non-Normative` provide description, rationale, deployment guidance, or open design commentary.
The purpose of this specification is to define the set of protocol components required to assemble complete UnShell protocol packets and to provide a framework through which the protocol can be extended through leaves and procedure contracts.
The UnShell protocol assumes that a connection already exists, that the local implementation has decided whether a peer should be admitted into routing, and that any required authentication or authorization has already been handled by the surrounding system.
Every implementation is expected to maintain its own live connection set and its own ground truth about which peers are connected, admitted, and routable.
> **Rationale:** Authentication and handshakes were intentionally removed from the core scope. They are too deployment-specific to define canonically without bloating the protocol.
> **Rationale:** Packet serialization is in scope because independently authored endpoints need one canonical byte representation in order to interoperate. Transport selection remains out of scope because the same framed packet bytes can be carried over different transports.
If the caller wants output, it declares a hook inside the call. The recipient returns one or more `Data` packets toward the hook host. Once a hook exists, either side MAY continue exchanging `Data` packets associated with that hook until one side terminates the interaction. If normal execution cannot proceed, the endpoint MAY instead send a `Fault` packet upstream for that hook.
`procedure_id` is the canonical identifier for a procedure contract. A procedure contract includes the source library or namespace, the specific procedure identity, and the expected input and output schema pair.
`procedure_id` MUST use the canonical dotted form `org.product.vN.part.name`, except for the reserved empty string `""` used by the required introspection procedure defined in Section 12.1, where:
-`org` identifies the owning organization or namespace root
-`product` identifies the product or system namespace
-`vN` identifies the contract version, where `N` is a positive integer written in decimal form
-`part` identifies the subsystem, leaf family, or functional area
-`name` identifies the exact procedure or payload contract name
Each segment MUST be non-empty. Implementations SHOULD restrict segments to lowercase ASCII letters, digits, and underscores for portability. The version segment MUST appear exactly in the third position.
> **Rationale:** `procedure_id` is intentionally stricter than a method name or content type. It identifies a full callable contract, not just a label.
Each endpoint enforces authority only at the connections it directly maintains.
At a local routing boundary:
- a `Call` packet MUST be accepted only if it arrives from the direct parent connection permitted to issue downwards calls into the destination subtree represented by that boundary
- a `Call` packet that violates that rule MUST be dropped silently
- a `Data` packet MAY arrive from either direction if it belongs to a valid hook flow, routes correctly by path, and its `src_path` matches the expected peer recorded in local hook state
- a `Fault` packet MAY arrive only from the subordinate side of a hook-attributable call flow, and its `src_path` MUST match the expected subordinate peer recorded in local hook state or pending call context
> **Rationale:** `rkyv` does not define one single universal format independent of configuration. Its archived representation depends on format-control settings such as endianness, alignment, and pointer width. This specification therefore fixes those settings so "use rkyv" means one exact interoperable byte format rather than a family of related formats.
`Fault` is used for upstream protocol failure reporting associated with a hook.
> **Rationale:** `Fault` is separated from `Data` so ordinary application output does not need to share semantics with protocol failure signaling. A receiver can distinguish successful hook traffic from protocol failure immediately from `packet_type`, without inspecting `procedure_id` or the payload contract.
An endpoint's local subtree consists of the endpoint's own path and every descendant path whose segment sequence begins with the endpoint's path as a prefix.
A path `A` lies within the subtree of path `B` if and only if `B` is a prefix of `A`.
The root endpoint's path is the empty path, and its subtree contains all paths.
> **Rationale:** Longest-prefix routing is defined as a path-selection rule, not as a way to resolve duplicate ownership. The tree model assumes each endpoint path names exactly one place in the topology. If two child routes claim the same path, the local routing table is already invalid.
> **Rationale:** The upward-routing rule can look backwards at first glance because it is phrased from the perspective of the current endpoint rather than from the root. Defining subtree membership by path-prefix makes the decision mechanical: if the destination is not inside the current endpoint's subtree and no child owns a more specific prefix, the only remaining path is upward.
If the sender on that connection is not the direct parent permitted to issue downwards calls into the relevant subtree, the endpoint MUST drop the packet silently.
If the destination endpoint exists but `dst_leaf` names no local leaf, the endpoint MUST treat the declared `response_hook`, if present, as sufficient authority to emit a protocol fault upstream even though the requested procedure cannot be executed. If no hook exists, the endpoint MUST discard the `Call` silently.
If `procedure_id` is unknown or unsupported, the endpoint MUST treat the declared `response_hook`, if present, as sufficient authority to emit a protocol fault upstream even though the requested procedure cannot be executed. If no hook exists, the endpoint MUST discard the `Call` silently.
> **Rationale:** Fault reporting for an invalid call would be self-defeating if the callee first had to prove that the application procedure was valid before it could use the declared hook. The hook exists to carry either normal returned data or a protocol fault explaining why normal execution could not proceed.
Pending call context is local transient state created when an endpoint receives a `Call` that declares `response_hook` and before that call has either been accepted into active hook state, rejected with `Fault`, or discarded.
Each pending call context MUST contain at least:
- the caller `src_path`
- the declared `return_path`
- the declared `hook_id`
- the invoked `procedure_id`
- the destination `dst_leaf`
A pending call context MUST be keyed by the pair `(`return_path`, `hook_id`)`.
> **Rationale:** Pending call context exists because some failures are discovered before normal application execution begins. The callee still needs enough validated state to attribute an upstream `Fault` to the declared hook without pretending that the hook was fully active for ordinary bidirectional traffic.
> **Rationale:** Ordinary hook traffic is part of the same procedure contract that created the hook, so the returned `procedure_id` stays anchored to the originating `Call`. This keeps hook validation simple and avoids treating a response as a separate contract lookup. Introspection therefore uses `""` on both the `Call` and the `Data` it produces. Protocol faults are separate packets and therefore do not need to overload `Data` semantics.
A hook becomes active when the destination endpoint accepts the `Call` and allocates local hook state for the declared `response_hook`.
Once active, either side MAY send the first `Data` packet for that hook.
If an endpoint sends hook `Data` before the peer has activated local hook state for that hook, the peer MAY discard that packet as not yet attributable to an active hook.
> **Rationale:** The protocol allows symmetric hook traffic after activation, but it does not introduce a separate readiness or acknowledgment packet just to synchronize the first `Data` frame. Allowing early packets to be discarded keeps the core protocol small while making the race explicit. Higher-layer protocols that need stricter startup guarantees are expected to define their own handshake or first-packet discipline inside the hook.
After normal completion without a `Fault` packet, the protocol does not require immediate retirement or reuse of the `hook_id`. An implementation MAY retain inactive hook records for any implementation-defined period.
When allocating a new hook, an implementation SHOULD choose the lowest available inactive `hook_id`.
> **Rationale:** The protocol needs a clear uniqueness rule for active hooks, but it does not need to over-specify local allocator policy after normal completion. Some implementations may want to retain inactive entries briefly for diagnostics, duplicate suppression, or transport reordering tolerance. Recommending the lowest available inactive `hook_id` keeps allocation predictable without forcing immediate recycling.
- a sender MAY set `end_hook = true` when its application protocol has decided to end the hook interaction
- a receiver of `end_hook = true` SHOULD treat the sender as finished with that hook
- any finer-grained shutdown, acknowledgment, or cancellation sequencing MUST be defined by the application protocol carried in `procedure_id` and `data`
After normal completion without a `Fault` packet, the moment at which a hook becomes inactive is implementation-defined unless a higher-layer protocol carried on that hook defines a stricter rule.
> **Rationale:** `end_hook` only communicates that one sender is finished. The core protocol intentionally avoids defining a universal half-close or close-ack state machine because different applications want different shutdown behavior. A simple request-response hook can retire immediately after the final packet, while a richer bidirectional protocol can define a stricter end-of-stream sequence above this layer.
- a receiver of a `Fault` packet MUST validate `src_path` against the expected subordinate hook peer recorded in local hook state or pending call context
When an endpoint rejects a `Call` for `UnknownLeaf` or `UnknownProcedure` and the `Call` declared `response_hook`, the endpoint MUST send a `Fault` packet upstream using that declared `hook_id` and `return_path` even if normal application execution never began.
Sending a `Fault` packet ends the hook immediately. After sending or receiving a `Fault` packet, an implementation MUST remove that hook from active state.
If an endpoint receives a fault value it does not recognize, it MUST still treat the packet as a protocol fault and close the hook.
> **Rationale:** Protocol faults are part of interoperability, so they need a fixed canonical payload contract rather than a free-form error blob. A small enum with stable byte discriminants is cheap to encode, easy to evolve, and avoids coupling core protocol behavior to human-readable messages. Receivers can make deterministic decisions from the fault kind alone.
> **Rationale:** Separating `Fault` from `Data` keeps application output and protocol failure signaling visibly distinct on the wire. It also removes the need for a reserved fault `procedure_id`, because failure is already expressed by `packet_type`.
> **Rationale:** The fault set is intentionally small. Silent drop remains the canonical behavior for traffic that cannot be safely attributed to a valid call or hook, such as an unknown `hook_id`, malformed returned traffic, or a routing miss discovered by an intermediate router. `Fault` is reserved for failures that a receiver can attribute to a specific call flow and report upstream deterministically.
> **Rationale:** An unrecognized protocol fault still means the application contract has failed and the hook can no longer continue safely. Requiring unknown fault values to terminate the hook preserves forward compatibility: newer peers may introduce additional fault kinds without causing older peers to accidentally keep a broken hook alive.
> **Rationale:** Introspection is intentionally narrow. It tells a caller what can be called, not how to explain or rediscover the contract. `procedure_id` already names a pre-shared callable contract, so repeating human-readable descriptions, parameter metadata, or live state inside introspection would make the protocol less canonical rather than more useful.
- each listed procedure MUST be identified by its full canonical `procedure_id`, not by a leaf-local short name
- endpoint introspection and leaf introspection MUST use the same per-leaf discovery shape
> **Rationale:** Returning full `procedure_id` values avoids forcing the caller to reconstruct contract names from leaf-local fragments. Endpoint introspection and leaf introspection deliberately share the same leaf record shape so the endpoint-wide form is just a list of the leaf-specific form.
`procedure_id` is the main semantic anchor. In this design, the caller and callee are expected to share knowledge of what a procedure contract means. The protocol does not carry a global registry, but it does require a canonical dotted naming form so independently authored contracts remain distinguishable.
This document uses Rust-like `rkyv` struct notation to describe fields because it matches the current implementation language. The notation is explanatory, but the on-wire byte format is normatively fixed in Section 8.
