unshell/PROTOCOL.md

Tree Protocol Specification

Overview

The Tree Protocol is a lightweight, extensible message format designed for hierarchical message routing in a modular C2 (Command & Control) system. It provides RPC-like interactions, streaming support, event notifications, and peer-to-peer pivoting capabilities.

Design Principles

1. Loose typing - Actions and payloads are flexible JSON values, not strict enums
2. Namespacing - Use dot notation for action categorization (e.g., rpc.call, stream.data)
3. Extensibility - Add new capabilities without modifying core structure
4. Simplicity - Minimal required fields, optional metadata for flexibility

Message Structure

{
    "id": "uuid-string",
    "source": "path/to/sender",
    "target": "path/to/recipient",
    "action": "action.name",
    "payload": {},
    "routing": {},
    "meta": {}
}

Field Definitions

Field	Type	Required	Description
id	string	No	Unique message identifier for correlation
source	string/array	No	Origin path for responses
target	string/array	No	Destination path for routing
action	value	Yes	Operation to perform
payload	any	No	Data for the action
routing	object	No	P2P/pivoting metadata
meta	object	No	Extensible metadata

Action System

Actions are loose values - any JSON that components can interpret:

"action": "query"
"action": "rpc.call"
"action": "stream.open"
"action": {"name": "custom", "version": 1}

Common Action Categories

Category	Prefix	Purpose
Core	query, create, delete, update	CRUD operations
RPC	rpc.call, rpc.response	Remote procedure calls
Streams	stream.open, stream.data, stream.close	Bidirectional streams
Events	subscribe, event	Event notifications
Network	connect, disconnect, forward	Connection handling

Payload Formats

Payloads vary by action - components interpret them flexibly:

RPC Payload

{
    "action": "rpc.call",
    "target": ["endpoints", "ep1", "components", "tcp-client"],
    "payload": {
        "method": "connect",
        "params": {"address": "127.0.0.1", "port": 443}
    }
}

{
    "action": "rpc.response",
    "payload": {
        "success": true,
        "result": {"connected": true}
    }
}

Stream Payload

{
    "action": "stream.open",
    "payload": {
        "channel": "stdio",
        "mode": "bidirectional"
    }
}

{
    "action": "stream.data",
    "payload": {
        "channel": "stdout",
        "data": "SGVsbG8gd29ybGQ=",  // base64 encoded
        "chunk": 0,
        "total": 1
    }
}

{
    "action": "stream.close",
    "payload": {
        "channel": "stdio",
        "reason": "eof"
    }
}

Event Payload

{
    "action": "subscribe",
    "target": ["endpoints", "ep1", "logs"],
    "payload": {
        "event": "new_entry",
        "callback": "components/event-handler"
    }
}

{
    "action": "event",
    "source": ["endpoints", "ep1", "logs"],
    "payload": {
        "event": "new_entry",
        "data": "some log message"
    }
}

P2P Routing Payload

{
    "action": "forward",
    "target": ["peers", "peer-2", "endpoints", "ep2"],
    "routing": {
        "via": "peer-1",
        "hop": 1,
        "max_hops": 3
    },
    "payload": {...}
}

Response Pattern

All responses use a generic format:

{
    "id": "req-123",
    "action": "response",
    "source": ["endpoints", "ep1"],
    "payload": {
        "success": true,
        "result": {"connected": true},
        "error": null
    }
}

{
    "id": "req-123",
    "action": "response",
    "payload": {
        "success": false,
        "error": {
            "code": 404,
            "message": "not found"
        }
    }
}

Path Format

Paths can be represented in multiple ways:

"target": "components/tcp-client"
"target": ["components", "tcp-client"]
"target": ["endpoints", "ep1", "connections", "conn-1"]

Message Types (Optional Wrapper)

For transport-level distinction:

Type	Description
req	Request - expecting a response
resp	Response - reply to a request
event	Unsolicited notification
stream	Stream data message

Example with type wrapper:

{
    "id": "msg-123",
    "type": "req",
    "target": ["components", "tcp-client"],
    "action": "rpc.call",
    "payload": {"method": "connect", "params": {...}}
}

Binary Framing (Optional)

For bandwidth-constrained links:

[4 bytes: length][1 byte: flags][id (16 bytes, optional)][payload...]

• Length: Big-endian u32
• Flags: 0x01 = compressed, 0x02 = encrypted
• ID: UUID v4 (optional, for correlation)

Examples

Full RPC Call

{
    "id": "req-123",
    "source": "server",
    "target": ["endpoints", "ep1", "components", "tcp-client"],
    "action": "rpc.call",
    "payload": {
        "method": "connect",
        "params": {"address": "127.0.0.1", "port": 8080}
    }
}

Response

{
    "id": "resp-123",
    "source": ["endpoints", "ep1"],
    "target": "server",
    "action": "response",
    "payload": {
        "success": true,
        "result": {
            "connected": true,
            "local_addr": "192.168.1.100:12345",
            "remote_addr": "127.0.0.1:8080"
        }
    }
}

Log Subscription + Events

{
    "id": "sub-1",
    "source": "server",
    "target": ["endpoints", "ep1", "logs"],
    "action": "subscribe",
    "payload": {
        "event": "new_entry",
        "callback": ["components", "log-forwarder"]
    }
}

{
    "id": "evt-1",
    "source": ["endpoints", "ep1", "logs"],
    "target": ["components", "log-forwarder"],
    "action": "event",
    "payload": {
        "event": "new_entry",
        "data": "2024-01-15 10:30:45 - Connection established"
    }
}

P2P Forward

{
    "id": "req-456",
    "source": "server",
    "target": ["endpoints", "ep2", "components", "shell"],
    "action": "rpc.call",
    "routing": {
        "via": "peer-1",
        "hop": 1,
        "max_hops": 3
    },
    "payload": {
        "method": "execute",
        "params": {"command": "whoami"}
    }
}

Implementation Notes

Action Matching

impl TreeMessage {
    /// Check if action matches a pattern
    pub fn action_is(&self, action: &str) -> bool {
        match &self.action {
            Value::String(s) => s == action || s.ends_with(&format!(".{}", action)),
            _ => false,
        }
    }
    
    /// Get method name from RPC payload
    pub fn get_method(&self) -> Option<String> {
        self.payload.get("method")
            .and_then(|m| m.as_str())
            .map(String::from)
    }
    
    /// Get stream channel from payload
    pub fn get_channel(&self) -> Option<String> {
        self.payload.get("channel")
            .and_then(|c| c.as_str())
            .map(String::from)
    }
}

Component Interpretation

Components define their own action handlers:

impl TreeElement for TcpClient {
    fn send_message(&mut self, target: Value, message: Value) -> Value {
        match message.get("method").and_then(|m| m.as_str()) {
            Some("connect") => self.handle_connect(message),
            Some("send") => self.handle_send(message),
            Some("recv") => self.handle_recv(message),
            _ => json!({"error": "unknown method"}),
        }
    }
}

Comparison to Other Protocols

Aspect	Tree Protocol	OpenC2	OST-C2	Mythic
Typing	Loose/JSON	Strict enum	Binary enum	JSON
Actions	Namespaced strings	Fixed enum	Type/Code	Action strings
RPC	Yes	No	Limited	Yes
Streams	Yes	No	No	No
Events	Yes	No	No	Yes
P2P	Via paths	No	Yes	Via delegates
Extensibility	High	Medium	Low	Medium

Extensibility Points

1. New actions: Add any namespaced string - no code changes needed
2. Payload structure: Any JSON - components interpret as needed
3. Meta field: Add transport, timing, or custom metadata
4. Routing field: Add P2P-specific metadata as needed
5. Namespacing: Use prefixes like openc2., custom., vendor. for compatibility

Serialization

• Default: JSON (human-readable, debuggable)
• Optional: CBOR (binary, compact)
• Custom: Any format via meta.serialization hint