f648349e82
* Diatixisify! * First pass at Typedoc generation for TS SDK * Remove overview pages * Fix typos and remove codebase references from docs Fix typos across network and developer docs: Quorum, available, cryptosystem, transaction, proportional, Standalone. Remove TODO placeholder from dVPN protocol page. Strip GitHub source links from network docs to decouple documentation from repo structure. * Expand thin landing pages across network and developer docs - Add intro content to network overview, infrastructure, and reference landing pages - Expand developer index with "where to start" guide - Add usage instructions and explanations to all five TS playground pages - Expand WebSocket client page with setup and message format examples * Restructure Rust SDK developer docs - Delete redundant mixnet example, message-helpers, and message-types subpages - Delete client-pool architecture and example subpages (content folded into landing) - Delete tcpproxy troubleshooting (folded into landing page) - Add deprecation notices to TcpProxy pages, pointing to Stream module - Add stream module docs: landing page, architecture, tutorial, and 4 example pages - Add mixnet and client-pool tutorials - Add SDK tour page - Update navigation and landing pages with docs.rs links * Restructure TS SDK developer docs - Merge overview, installation, and getting started into TS SDK landing page - Fold FAQ content into bundling/troubleshooting section - Delete redundant overview, installation, start, and FAQ pages - Update internal links in browsers.mdx and native.mdx - Update navigation and example page imports * Flatten and expand APIs section - Collapse nested API subpages into single pages with inline Redoc embeds - Rewrite introduction as landing page with decision table - Add endpoint categories, quick curl examples to each API page - Mark Explorer API as deprecated - Move NS API deployment guide to operators/performance-and-testing - Fix dangling /apis/nym-api/mainnet link in network-components - Remove sandbox endpoints from all API pages * Add redirects for moved and deleted pages - Add 25 redirects covering TS SDK, Rust SDK, APIs, and network sections - Fix dangling /developers/typescript/start link in operators changelog * Replace individual example doc pages with GitHub-linked tables, expand tutorials - replace individual example doc pages with GitHub-linked tables - expand mixnet tutorial with persistent identity and split_sender sections - add tcpproxy tutorial - rename "API Reference" to "TypeDoc Reference" in TS SDK sidebar - rename "Misc" to "Extras" in developer sidebar, move VPN CLI up - remove echo server from tools - update message-queue callout to reference actual modules - fix mixnet/examples redirect collision * Add SEO frontmatter, validate encryption standards, clean up URLs - add title/description/schemaType/section/lastUpdated frontmatter to 48 pages across developers, network, and APIs sections - remove network/.archive/ directory (compare against develop instead) - update nymtech.net → nym.com for website/blog links (keep infra URLs) - add native proxy "in progress" callout for Rust/C/Go * API-scraper update (#6598) * read nodes and locations * update python-prebuild.sh * Address PR #6494 review feedback - Use "mode" consistently instead of "role" on nym-nodes page - Replace "staking" with "bonding" for NYM token collateral - Wire up auto-scraped node counts via TimeNow + nodes-count.json - Fix broken licensing images: download CC icons locally, replace inline HTML - Fix 9 stale redirects pointing through deleted /network/architecture path * Fix linkcheck errors - Fix stale cross-links: /network/concepts/ → /network/mixnet-mode/ - Replace README.md references with globals.md in TypeDoc output - Add entryFileName: globals to typedoc.json configs to prevent recurrence * Fix remaining stale /network/architecture links - zk-nym-overview: architecture/nyx#nym-api → /network/infrastructure/nyx#nym-api - setup: network/architecture → /network/overview * Remove accidentally re-included architecture.md file from rebase * Standardize tutorials, document examples, add llms.txt, apply tone fixes - Expand Rust SDK tutorials with step-by-step structure; document all SDK examples across mixnet, client-pool, and tcpproxy pages - Add llms.txt generation script, wire into build and CI workflows - Apply tone/style fixes: deduplicate callouts, vary sentence structure, standardize voice consistency across changed pages * Consolidate redundant network overview docs * Trim dev docs: git-first imports, stream notice, collapse TcpProxy * Update tutorial * Refresh auto-generated API and command outputs * Update network section docs * Update developer and API docs: reusable components, stream protocol, conventions, tutorial fixes * Fix Rust SDK tutorial bugs: setup_env, port conflicts, logging, open_stream race condition * Update stream.mdx * Remove docs.rs link from Stream overview for the moment * add llms.txt and llms-full.txt note to readme --------- Co-authored-by: import this <97586125+serinko@users.noreply.github.com>
150 lines
8.9 KiB
Plaintext
150 lines
8.9 KiB
Plaintext
---
|
|
title: "Stream Module: AsyncRead/AsyncWrite Over the Mixnet"
|
|
description: "The Nym Stream module provides persistent, bidirectional byte channels over the mixnet with standard Rust AsyncRead and AsyncWrite traits."
|
|
schemaType: "TechArticle"
|
|
section: "Developers"
|
|
lastUpdated: "2026-03-15"
|
|
---
|
|
|
|
# Stream Module
|
|
|
|
import { Callout } from 'nextra/components'
|
|
import { CratesPaused } from '../../../components/crates-paused'
|
|
|
|
<CratesPaused />
|
|
|
|
The Mixnet is fundamentally message-based: no persistent connections, no guaranteed ordering, no TCP. The default [message API](./mixnet) works at this level, sending individual payloads independently through Mix Nodes. This is effective for privacy but unlike how most networking code is structured.
|
|
|
|
The **Stream module** bridges the gap by providing persistent, bidirectional byte channels that behave like TCP sockets. Each `MixnetStream` implements [`AsyncRead`](https://docs.rs/tokio/latest/tokio/io/trait.AsyncRead.html) and [`AsyncWrite`](https://docs.rs/tokio/latest/tokio/io/trait.AsyncWrite.html), so `tokio::io::copy`, codecs, `BufReader`/`BufWriter`, and any other async I/O consumer work without modification. **If you're coming from socket-based networking, start here.**
|
|
|
|
All streams are multiplexed over a single `MixnetClient`. A background router task reads a small header on each incoming message and dispatches the payload to the correct stream by ID, so multiple concurrent streams require no additional connections or gateways.
|
|
|
|
## How it works
|
|
|
|
The two sides of a stream connection follow a client/server pattern:
|
|
|
|
1. **Opener** calls `client.open_stream(recipient, surbs)`. This generates a random `StreamId`, registers the stream locally, and sends an `Open` message through the Mixnet.
|
|
2. **Listener** calls `listener.accept()`, which blocks until an `Open` arrives, registers the new stream, and returns a `MixnetStream` ready for reading and writing.
|
|
3. Both sides read and write using standard `AsyncRead`/`AsyncWrite`. Bytes are wrapped in a 16-byte LP frame header (stream ID, message type, sequence number), routed through the Mixnet, and demultiplexed on arrival.
|
|
4. **Cleanup** happens on `drop`. The stream deregisters from the local router. No close message is sent over the wire, since a close could race ahead of in-flight data.
|
|
|
|
```text
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ MixnetClient │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ MixnetStream │ │ MixnetStream │ ... │
|
|
│ │ (peer A) │ │ (peer B) │ │
|
|
│ └──────┬───────┘ └──────┬───────┘ │
|
|
│ │writes │writes │
|
|
│ ▼ ▼ │
|
|
│ ┌─────────────────────────────────┐ │
|
|
│ │ ClientInput.input_sender │ │
|
|
│ └──────────────┬──────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ── mixnet ── │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌─────────────────────────────────┐ │
|
|
│ │ reconstructed_receiver │ │
|
|
│ └──────────────┬──────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌─────────────────────────────────┐ │
|
|
│ │ Router task │ │
|
|
│ │ decode header → dispatch by ID │ │
|
|
│ └──┬──────────────────────────┬───┘ │
|
|
│ │ Open messages │ Data messages │
|
|
│ ▼ ▼ │
|
|
│ ┌──────────────┐ ┌──────────────────┐ │
|
|
│ │MixnetListener│ │ StreamMap lookup │ │
|
|
│ │ .accept() │ │ → per-stream tx │ │
|
|
│ └──────────────┘ └──────────────────┘ │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Complete example
|
|
|
|
A minimal example with two clients on the same machine: one opens a stream to the other, sends a message, and reads a reply.
|
|
|
|
```rust
|
|
use nym_sdk::mixnet;
|
|
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
|
use std::time::Duration;
|
|
|
|
const TIMEOUT: Duration = Duration::from_secs(60);
|
|
|
|
#[tokio::main]
|
|
async fn main() {
|
|
// Connect two ephemeral clients
|
|
let mut sender = mixnet::MixnetClient::connect_new().await.unwrap();
|
|
let mut receiver = mixnet::MixnetClient::connect_new().await.unwrap();
|
|
let receiver_addr = *receiver.nym_address();
|
|
|
|
// The receiver creates a listener (activates stream mode)
|
|
let mut listener = receiver.listener().unwrap();
|
|
|
|
// The sender opens a stream to the receiver's Nym address
|
|
let mut outbound = sender.open_stream(receiver_addr, None).await.unwrap();
|
|
|
|
// The receiver accepts the incoming stream
|
|
let mut inbound = tokio::time::timeout(TIMEOUT, listener.accept())
|
|
.await
|
|
.expect("timed out")
|
|
.expect("listener closed");
|
|
|
|
// Send data and read it back — just like a TCP socket
|
|
outbound.write_all(b"hello from sender").await.unwrap();
|
|
outbound.flush().await.unwrap();
|
|
|
|
let mut buf = vec![0u8; 1024];
|
|
let n = tokio::time::timeout(TIMEOUT, inbound.read(&mut buf))
|
|
.await
|
|
.expect("timed out")
|
|
.expect("read failed");
|
|
println!("Receiver got: {}", String::from_utf8_lossy(&buf[..n]));
|
|
|
|
// Reply back through the same stream
|
|
inbound.write_all(b"hello from receiver").await.unwrap();
|
|
inbound.flush().await.unwrap();
|
|
|
|
let n = tokio::time::timeout(TIMEOUT, outbound.read(&mut buf))
|
|
.await
|
|
.expect("timed out")
|
|
.expect("read failed");
|
|
println!("Sender got: {}", String::from_utf8_lossy(&buf[..n]));
|
|
|
|
// Streams deregister on drop, then disconnect clients
|
|
drop(outbound);
|
|
drop(inbound);
|
|
sender.disconnect().await;
|
|
receiver.disconnect().await;
|
|
}
|
|
```
|
|
|
|
<Callout type="info">
|
|
The receiver replies via **reply SURBs** (Single Use Reply Blocks) and never learns the sender's Nym address.
|
|
</Callout>
|
|
|
|
## When to use streams vs messages
|
|
|
|
| | Messages | Streams | TcpProxy |
|
|
|---|---|---|---|
|
|
| **Pattern** | Raw message payloads | Persistent bidirectional channels | TCP socket proxying |
|
|
| **API** | `send_plain_message()` / `wait_for_messages()` | `AsyncRead` + `AsyncWrite` | Localhost TCP socket |
|
|
| **Multiplexing** | N/A | Multiple streams per client | One client per TCP connection |
|
|
| **Ordering** | No guarantees | Sequence-based reordering | Session-based ordering |
|
|
| **Best for** | Simple notifications, one-shot requests | Interactive protocols, streaming data, any code expecting async I/O | Wrapping existing TCP applications |
|
|
| **Status** | Stable | New | Deprecated |
|
|
|
|
<Callout type="warning">
|
|
**Streams and messages are mutually exclusive.** Once you call `open_stream()` or `listener()`, the message-based API (`send_plain_message`, `wait_for_messages`) is permanently disabled on that client. This is a one-way transition: there is no switching back without disconnecting and reconnecting. See the [`stream_mode_guard.rs` example](https://github.com/nymtech/nym/blob/develop/sdk/rust/nym-sdk/examples/stream_mode_guard.rs) for details.
|
|
</Callout>
|
|
|
|
## Next steps
|
|
|
|
- [Tutorial: Build a private echo server](./stream/tutorial): server and client communicating over streams
|
|
- [Architecture](./stream/architecture): wire protocol, router task, data flow, stream cleanup, and known limitations
|
|
- [Examples](./stream/examples): annotated walkthroughs of the SDK examples (multi-stream, idle timeout, throughput testing)
|