Files
nym/documentation/docs/pages/developers/rust/stream.mdx
mfahampshire f648349e82 Max/docs-diataxis-ify (#6494)
* Diatixisify!

* First pass at Typedoc generation for TS SDK

* Remove overview pages

* Fix typos and remove codebase references from docs

Fix typos across network and developer docs: Quorum, available,
cryptosystem, transaction, proportional, Standalone. Remove TODO
placeholder from dVPN protocol page. Strip GitHub source links
from network docs to decouple documentation from repo structure.

* Expand thin landing pages across network and developer docs

- Add intro content to network overview, infrastructure, and reference landing pages
- Expand developer index with "where to start" guide
- Add usage instructions and explanations to all five TS playground pages
- Expand WebSocket client page with setup and message format examples

* Restructure Rust SDK developer docs

- Delete redundant mixnet example, message-helpers, and message-types subpages
- Delete client-pool architecture and example subpages (content folded into landing)
- Delete tcpproxy troubleshooting (folded into landing page)
- Add deprecation notices to TcpProxy pages, pointing to Stream module
- Add stream module docs: landing page, architecture, tutorial, and 4 example pages
- Add mixnet and client-pool tutorials
- Add SDK tour page
- Update navigation and landing pages with docs.rs links

* Restructure TS SDK developer docs

- Merge overview, installation, and getting started into TS SDK landing page
- Fold FAQ content into bundling/troubleshooting section
- Delete redundant overview, installation, start, and FAQ pages
- Update internal links in browsers.mdx and native.mdx
- Update navigation and example page imports

* Flatten and expand APIs section

- Collapse nested API subpages into single pages with inline Redoc embeds
- Rewrite introduction as landing page with decision table
- Add endpoint categories, quick curl examples to each API page
- Mark Explorer API as deprecated
- Move NS API deployment guide to operators/performance-and-testing
- Fix dangling /apis/nym-api/mainnet link in network-components
- Remove sandbox endpoints from all API pages

* Add redirects for moved and deleted pages

- Add 25 redirects covering TS SDK, Rust SDK, APIs, and network sections
- Fix dangling /developers/typescript/start link in operators changelog

* Replace individual example doc pages with GitHub-linked tables, expand tutorials

- replace individual example doc pages with GitHub-linked tables
- expand mixnet tutorial with persistent identity and split_sender sections
- add tcpproxy tutorial
- rename "API Reference" to "TypeDoc Reference" in TS SDK sidebar
- rename "Misc" to "Extras" in developer sidebar, move VPN CLI up
- remove echo server from tools
- update message-queue callout to reference actual modules
- fix mixnet/examples redirect collision

* Add SEO frontmatter, validate encryption standards, clean up URLs

- add title/description/schemaType/section/lastUpdated frontmatter to 48
  pages across developers, network, and APIs sections
- remove network/.archive/ directory (compare against develop instead)
- update nymtech.net → nym.com for website/blog links (keep infra URLs)
- add native proxy "in progress" callout for Rust/C/Go

* API-scraper update (#6598)

* read nodes and locations

* update python-prebuild.sh

* Address PR #6494 review feedback
- Use "mode" consistently instead of "role" on nym-nodes page
- Replace "staking" with "bonding" for NYM token collateral
- Wire up auto-scraped node counts via TimeNow + nodes-count.json
- Fix broken licensing images: download CC icons locally, replace inline HTML
- Fix 9 stale redirects pointing through deleted /network/architecture path

* Fix linkcheck errors
- Fix stale cross-links: /network/concepts/ → /network/mixnet-mode/
- Replace README.md references with globals.md in TypeDoc output
- Add entryFileName: globals to typedoc.json configs to prevent recurrence

* Fix remaining stale /network/architecture links
- zk-nym-overview: architecture/nyx#nym-api → /network/infrastructure/nyx#nym-api
- setup: network/architecture → /network/overview

* Remove accidentally re-included architecture.md file from rebase

* Standardize tutorials, document examples, add llms.txt, apply tone fixes

- Expand Rust SDK tutorials with step-by-step structure; document all SDK examples across mixnet, client-pool, and tcpproxy pages
- Add llms.txt generation script, wire into build and CI workflows
- Apply tone/style fixes: deduplicate callouts, vary sentence structure, standardize voice consistency across changed pages

* Consolidate redundant network overview docs

* Trim dev docs: git-first imports, stream notice, collapse TcpProxy

* Update tutorial

* Refresh auto-generated API and command outputs

* Update network section docs

* Update developer and API docs: reusable components, stream protocol, conventions, tutorial fixes

* Fix Rust SDK tutorial bugs: setup_env, port conflicts, logging,
open_stream race condition

* Update stream.mdx

* Remove docs.rs link from Stream overview for the moment

* add llms.txt and llms-full.txt note to readme

---------

Co-authored-by: import this <97586125+serinko@users.noreply.github.com>
2026-04-09 15:25:31 +00:00

150 lines
8.9 KiB
Plaintext

---
title: "Stream Module: AsyncRead/AsyncWrite Over the Mixnet"
description: "The Nym Stream module provides persistent, bidirectional byte channels over the mixnet with standard Rust AsyncRead and AsyncWrite traits."
schemaType: "TechArticle"
section: "Developers"
lastUpdated: "2026-03-15"
---
# Stream Module
import { Callout } from 'nextra/components'
import { CratesPaused } from '../../../components/crates-paused'
<CratesPaused />
The Mixnet is fundamentally message-based: no persistent connections, no guaranteed ordering, no TCP. The default [message API](./mixnet) works at this level, sending individual payloads independently through Mix Nodes. This is effective for privacy but unlike how most networking code is structured.
The **Stream module** bridges the gap by providing persistent, bidirectional byte channels that behave like TCP sockets. Each `MixnetStream` implements [`AsyncRead`](https://docs.rs/tokio/latest/tokio/io/trait.AsyncRead.html) and [`AsyncWrite`](https://docs.rs/tokio/latest/tokio/io/trait.AsyncWrite.html), so `tokio::io::copy`, codecs, `BufReader`/`BufWriter`, and any other async I/O consumer work without modification. **If you're coming from socket-based networking, start here.**
All streams are multiplexed over a single `MixnetClient`. A background router task reads a small header on each incoming message and dispatches the payload to the correct stream by ID, so multiple concurrent streams require no additional connections or gateways.
## How it works
The two sides of a stream connection follow a client/server pattern:
1. **Opener** calls `client.open_stream(recipient, surbs)`. This generates a random `StreamId`, registers the stream locally, and sends an `Open` message through the Mixnet.
2. **Listener** calls `listener.accept()`, which blocks until an `Open` arrives, registers the new stream, and returns a `MixnetStream` ready for reading and writing.
3. Both sides read and write using standard `AsyncRead`/`AsyncWrite`. Bytes are wrapped in a 16-byte LP frame header (stream ID, message type, sequence number), routed through the Mixnet, and demultiplexed on arrival.
4. **Cleanup** happens on `drop`. The stream deregisters from the local router. No close message is sent over the wire, since a close could race ahead of in-flight data.
```text
┌─────────────────────────────────────────────────────────┐
│ MixnetClient │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ MixnetStream │ │ MixnetStream │ ... │
│ │ (peer A) │ │ (peer B) │ │
│ └──────┬───────┘ └──────┬───────┘ │
│ │writes │writes │
│ ▼ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ ClientInput.input_sender │ │
│ └──────────────┬──────────────────┘ │
│ │ │
│ ▼ │
│ ── mixnet ── │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ reconstructed_receiver │ │
│ └──────────────┬──────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ Router task │ │
│ │ decode header → dispatch by ID │ │
│ └──┬──────────────────────────┬───┘ │
│ │ Open messages │ Data messages │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────────┐ │
│ │MixnetListener│ │ StreamMap lookup │ │
│ │ .accept() │ │ → per-stream tx │ │
│ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────┘
```
## Complete example
A minimal example with two clients on the same machine: one opens a stream to the other, sends a message, and reads a reply.
```rust
use nym_sdk::mixnet;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use std::time::Duration;
const TIMEOUT: Duration = Duration::from_secs(60);
#[tokio::main]
async fn main() {
// Connect two ephemeral clients
let mut sender = mixnet::MixnetClient::connect_new().await.unwrap();
let mut receiver = mixnet::MixnetClient::connect_new().await.unwrap();
let receiver_addr = *receiver.nym_address();
// The receiver creates a listener (activates stream mode)
let mut listener = receiver.listener().unwrap();
// The sender opens a stream to the receiver's Nym address
let mut outbound = sender.open_stream(receiver_addr, None).await.unwrap();
// The receiver accepts the incoming stream
let mut inbound = tokio::time::timeout(TIMEOUT, listener.accept())
.await
.expect("timed out")
.expect("listener closed");
// Send data and read it back — just like a TCP socket
outbound.write_all(b"hello from sender").await.unwrap();
outbound.flush().await.unwrap();
let mut buf = vec![0u8; 1024];
let n = tokio::time::timeout(TIMEOUT, inbound.read(&mut buf))
.await
.expect("timed out")
.expect("read failed");
println!("Receiver got: {}", String::from_utf8_lossy(&buf[..n]));
// Reply back through the same stream
inbound.write_all(b"hello from receiver").await.unwrap();
inbound.flush().await.unwrap();
let n = tokio::time::timeout(TIMEOUT, outbound.read(&mut buf))
.await
.expect("timed out")
.expect("read failed");
println!("Sender got: {}", String::from_utf8_lossy(&buf[..n]));
// Streams deregister on drop, then disconnect clients
drop(outbound);
drop(inbound);
sender.disconnect().await;
receiver.disconnect().await;
}
```
<Callout type="info">
The receiver replies via **reply SURBs** (Single Use Reply Blocks) and never learns the sender's Nym address.
</Callout>
## When to use streams vs messages
| | Messages | Streams | TcpProxy |
|---|---|---|---|
| **Pattern** | Raw message payloads | Persistent bidirectional channels | TCP socket proxying |
| **API** | `send_plain_message()` / `wait_for_messages()` | `AsyncRead` + `AsyncWrite` | Localhost TCP socket |
| **Multiplexing** | N/A | Multiple streams per client | One client per TCP connection |
| **Ordering** | No guarantees | Sequence-based reordering | Session-based ordering |
| **Best for** | Simple notifications, one-shot requests | Interactive protocols, streaming data, any code expecting async I/O | Wrapping existing TCP applications |
| **Status** | Stable | New | Deprecated |
<Callout type="warning">
**Streams and messages are mutually exclusive.** Once you call `open_stream()` or `listener()`, the message-based API (`send_plain_message`, `wait_for_messages`) is permanently disabled on that client. This is a one-way transition: there is no switching back without disconnecting and reconnecting. See the [`stream_mode_guard.rs` example](https://github.com/nymtech/nym/blob/develop/sdk/rust/nym-sdk/examples/stream_mode_guard.rs) for details.
</Callout>
## Next steps
- [Tutorial: Build a private echo server](./stream/tutorial): server and client communicating over streams
- [Architecture](./stream/architecture): wire protocol, router task, data flow, stream cleanup, and known limitations
- [Examples](./stream/examples): annotated walkthroughs of the SDK examples (multi-stream, idle timeout, throughput testing)