mfahampshire
93 lines
4.1 KiB
Plaintext
93 lines
4.1 KiB
Plaintext
---
|
|
title: "Stream Module Architecture"
|
|
description: "Internal architecture of the Nym Stream subsystem: wire protocol, multiplexing, router task, and how concurrent byte channels share a single MixnetClient."
|
|
schemaType: "TechArticle"
|
|
section: "Developers"
|
|
lastUpdated: "2026-03-15"
|
|
---
|
|
|
|
# Stream Architecture
|
|
|
|
import { Callout } from 'nextra/components'
|
|
|
|
{/* Canonical source: sdk/rust/nym-sdk/src/mixnet/stream/ARCHITECTURE.md */}
|
|
|
|
## Overview
|
|
|
|
The stream subsystem gives each `MixnetClient` the ability to hold many concurrent byte channels (`AsyncRead + AsyncWrite`) to different remote peers, multiplexed over a single client connection.
|
|
|
|
```mermaid
|
|
---
|
|
config:
|
|
theme: neo-dark
|
|
---
|
|
flowchart TD
|
|
subgraph MixnetClient
|
|
SA["MixnetStream A"] -->|writes| CI["Client input channel"]
|
|
SB["MixnetStream B"] -->|writes| CI
|
|
CI --> MX["── Mixnet ──"]
|
|
MX --> RT["Router task"]
|
|
RT -->|Open messages| ML["MixnetListener.accept()"]
|
|
RT -->|Data messages| SM["Stream routing table"]
|
|
SM --> SA
|
|
SM --> SB
|
|
end
|
|
```
|
|
|
|
## Wire protocol
|
|
|
|
Every stream message has a fixed 16-byte LP frame header prepended to the payload:
|
|
|
|
```
|
|
[LpFrameKind: 2 bytes LE][StreamId: 8 bytes BE][MsgType: 1 byte][SequenceNum: 4 bytes BE][Reserved: 1 byte][payload ...]
|
|
```
|
|
|
|
- **LpFrameKind:** `3` (SphinxStream). Distinguishes stream traffic from other LP frame types (Opaque, Registration, Forward).
|
|
- **StreamId:** random `u64` generated by the opener, used to multiplex streams.
|
|
- **MsgType:** `Open` (0) or `Data` (1).
|
|
- **SequenceNum:** `u32` counter, incremented per write. Used by the receiver's per-stream reorder buffer to deliver data in the correct order.
|
|
- **Reserved:** must be `0x00`.
|
|
|
|
There is no `Close` message type; see [Known Limitations](#known-limitations) for why.
|
|
|
|
## Stream mode
|
|
|
|
Stream mode is activated lazily on the first call to `open_stream()` or `listener()`. This is a **one-way transition**:
|
|
|
|
1. The client's message receiver is handed off to a background router task
|
|
2. `stream_mode` flag is set to `true`
|
|
3. Message-based methods (`send_plain_message`, `wait_for_messages`) are disabled and return errors
|
|
|
|
There is no switching back without disconnecting and creating a new client.
|
|
|
|
## Opening and accepting streams
|
|
|
|
**Opening (outbound):**
|
|
1. `open_stream(recipient, surbs)` generates a random `StreamId`
|
|
2. An `Open` message is sent through the Mixnet to the recipient
|
|
3. A `MixnetStream` is returned, ready for writing and reading
|
|
|
|
**Accepting (inbound):**
|
|
1. `listener.accept()` waits for an `Open` message from a remote peer
|
|
2. A `MixnetStream` is created with the opener's `sender_tag` for anonymous replies
|
|
3. The stream is ready for bidirectional I/O
|
|
|
|
## Cleanup
|
|
|
|
- **On `drop`:** the stream deregisters from the routing table. No close message is sent over the wire.
|
|
- **Idle timeout:** streams idle for longer than the configured timeout (default: 30 minutes) are automatically cleaned up. Configure with [`MixnetClientBuilder::with_stream_idle_timeout()`](https://docs.rs/nym-sdk/latest/nym_sdk/mixnet/struct.MixnetClientBuilder.html).
|
|
|
|
## Known limitations
|
|
|
|
<Callout type="info">
|
|
**Sequence-based reordering.** The Mixnet does not guarantee message ordering at the transport level, but each stream write includes a `sequence_num` in the LP frame header. The receiver maintains a per-stream reorder buffer (BTreeMap keyed by sequence number) that buffers out-of-order messages and drains them in sequence. This means protocols that depend on byte ordering (HTTP, TLS, protobuf) work correctly over streams.
|
|
|
|
- **Buffer cap:** 256 messages per stream. If the buffer fills (e.g. a large gap in sequence numbers), the receiver skips ahead to the lowest buffered sequence.
|
|
- **Duplicates:** messages with a sequence number below the next expected are dropped.
|
|
- There is no `Close` message type, since a close could race ahead of in-flight data.
|
|
</Callout>
|
|
|
|
## Internal details
|
|
|
|
For the full implementation details (router task, `StreamMap`, `PollSender` usage, base-client type rationale), see the `ARCHITECTURE.md` file next to the module source code, or the [docs.rs](https://docs.rs/nym-sdk/latest/nym_sdk/) API reference.
|