wgcloak/docs/PROTOCOL.md

Protocol overview

The application protocol, tunnel session, transport, and authentication remain independent. The application payload is opaque and is never parsed as WireGuard (or any other protocol).

Direct UDP wire protocol (target v1)

Direct messages use protocol version 1, network byte order, and a 12-byte common header:

+---------+---------+---------+---------+
| version | type    | flags   | hdr_len |
| u8      | u8      | u8      | u8      |
+---------------------------------------+
| session_id (u64)                      |
+---------------------------------------+

Packet types are HELLO (0x01), ACCEPT (0x02), REJECT (0x03), DATA (0x10), KEEPALIVE (0x11), and CLOSE (0x12). Unknown or malformed types are rejected or ignored according to context and never crash the daemon.

HELLO has session ID zero and carries a bounded client ID, timestamp, 16-byte client nonce, and a 32-byte HMAC-SHA256 authenticator over the canonical version/client/timestamp/nonce encoding. The server checks a default 60-second clock window and remembers accepted nonces for replay protection.

ACCEPT assigns a random, non-zero 64-bit session ID and returns a server nonce, idle timeout, maximum datagram size, and an authenticator. REJECT returns only a small reason class; it must not disclose secret material.

Authenticated DATA and KEEPALIVE packets contain the common header, a 32-bit direction-specific sequence number, a 128-bit truncated HMAC tag, and the optional payload. Session keys are derived with HKDF-SHA256 from the configured secret, both nonces, and the session ID. Implementations must use maintained cryptographic libraries and constant-time tag comparison.

Sequence windows provide duplicate/replay detection and diagnostics but do not reorder UDP. After MAC and replay checks, a valid packet may update the peer address, allowing NAT rebinding. A session expires after the configured idle timeout (target default 120 seconds), then its backend socket and replay state are released.

The core rejects oversized application datagrams, increments a counter, and logs at debug/warn level. v1 does not fragment datagrams; a conservative starting limit is 1200 bytes.

Encapsulated transports

• Direct UDP: native datagrams; uses the authenticated protocol above and provides integrity but not outer confidentiality.
• QUIC DATAGRAM: TLS 1.3 protects the connection; a client shared-secret challenge/response authenticates the configured client. Application datagrams use QUIC DATAGRAM frames.
• HTTP/3 CONNECT-UDP: uses extended CONNECT with :protocol=connect-udp and HTTP Datagrams with the RFC 9297 context ID. The request path is configurable. Capsules are used for the authentication exchange, not as the application-data carrier.
• TLS/TCP: uses the same client authentication model. Each application datagram is framed by a big-endian u16 payload length and opaque payload; the configured maximum applies. TCP head-of-line blocking is expected.

All four carriers are wired into the current client/server runtime. Client mode = "auto" probes its configured order, retries bounded probe attempts, and stops on authentication rejection. Reconnection after transport loss uses bounded exponential backoff with jitter; successful connection resets the backoff. This describes the current code path, not a claim that every v1 acceptance criterion has passed.

The control reload command is currently unsupported, including certificate or key reload. See the checklist for acceptance work that remains before a v1 release. The direct packet details and the encapsulated transport APIs are project-specific; interoperability requires matching this protocol.

Authentication and resource order

The server performs cheap bounds/version checks, authentication and rate-limit checks before allocating a backend UDP socket. Public listeners are hostile: limits, handshake timeouts, per-source caps, and malformed-packet handling are part of the protocol boundary.