GRIN/nym
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

LP Registration + Telescoping + Gateway Probe Localnet Mode (#6286)

Browse Source 
* Add KKT cryptographic primitives

Post-quantum Key Encapsulation Mechanism (KEM) Key Transfer protocol.
Enables efficient distribution of post-quantum KEM public keys.

Squashed from georgio/noise-psq branch.

* Implement LP registration protocol with KKT/PSQ integration

Initial implementation of the Lewes Protocol (LP) for gateway registration:
- Add nym-lp crate with Noise protocol handshake
- Add LP listener to gateway for handling registrations
- Add LP client for registration flow
- Integrate KKT for post-quantum KEM key exchange
- Integrate PSQ for post-quantum PSK derivation
- Add Ed25519 authentication throughout
- Add docker/localnet support for testing

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>

* Add LP telescoping with nested sessions and subsession support

Extends LP protocol with telescoping architecture for nested sessions:
- Add nested session support with KKpsk0 rekeying
- Add subsession support with collision detection
- Implement unified packet format with outer header
- Refactor gateway handlers for single-packet forwarding
- Add TTL-based state cleanup for stale sessions
- Add outer AEAD encryption layer
- Refactor registration client for packet-per-connection model

* Add gateway-probe localnet mode with WireGuard tunnel support

Adds localnet testing mode to gateway-probe for LP development:
- Add TestMode enum for different probe configurations
- Add --gateway-ip flag for direct gateway testing
- Implement two-hop WireGuard tunnel for localnet
- Add mock ecash support for testing without real credentials
- Add netstack Go bindings for userspace networking
- Restructure probe with mode and common modules
- Update README with localnet mode documentation

* Increase KCP fragment limit from u8 to u16

- Change frg field from u8 to u16 in packet header (25 bytes total)
- Update encode/decode to use get_u16_le/put_u16_le
- Update Segment struct frg field to u16
- Remove truncating cast in session.rs
- Max message size now ~91MB (65,535 fragments × MTU)
- Internal protocol only, no interop concerns

Nym uses KCP for reliability and multiplexing, not standard real-time
use cases. The u8 limit (255 fragments, ~355KB) was insufficient.

Addresses: nym-yih9

* Zeroize Ed25519 key material in to_x25519 conversion

Wrap hash and x25519_bytes in zeroize::Zeroizing to ensure private
key material is cleared from memory after use.

Closes: nym-k55g

* Return Result from KCP session input() for error detection

Change KcpSession::input() to return Result<(), KcpError> so callers
can detect invalid packets instead of silently ignoring them.

- Add ConvMismatch error variant for conversation ID mismatches
- Update driver to propagate errors from session.input()
- Update all test and example callers

Closes: nym-n0kk

* Fix Zeroizing deref in ed25519 to_x25519 conversion

The from_bytes() function expects &[u8], need to deref the Zeroizing
wrapper to get the inner array.

* Add semaphore-based connection limiting for LP packet forwarding

Limits concurrent outbound connections when forwarding LP packets to
prevent file descriptor exhaustion under high load.

Key changes:
- Add max_concurrent_forwards config (default 1000)
- Add forward_semaphore to LpHandlerState
- Acquire semaphore permit before connecting in handle_forward_packet
- Return "Gateway at forward capacity" error when at limit

This provides load signaling so clients can choose another gateway
when the current one is overloaded.

Design note: Connection pooling was considered but provides minimal
benefit since telescope setup is one-time and targets are distributed
across many different gateways. See AIDEV-NOTE in LpHandlerState for
full analysis.

Closes: nym-xi3m

* Return error on session unavailable in handle_subsession_packet

Replace .session().ok() with proper error handling to fail fast when
session is Closed or Processing after state machine processing.

Previously, the code silently continued with outer_key = None, which
could cause protocol errors downstream.

Closes: nym-8de0

* Use explicit bincode Options helper in nested_session

Add bincode_options() helper that returns DefaultOptions with explicit
big_endian and varint_encoding configuration. This future-proofs against
bincode 1.x/2.x default changes and makes serialization format explicit.

Updated all 4 bincode usages in nested_session.rs to use the helper.

* Deduplicate outer_key lookup pattern in nested_session.rs

Extract common state_machine.session().ok().and_then(...) pattern into
two helper methods:
- get_send_key() for encryption (outer_aead_key_for_sending)
- get_recv_key() for decryption (outer_aead_key)

Updated 6 call sites to use the helpers, reducing verbosity.

* Add LpConfig struct and AIDEV-NOTE documentation for KKT+PSQ

- Create config.rs with LpConfig struct (kem_algorithm, psk_ttl, enable_kkt)
- Export LpConfig from lib.rs
- Add AIDEV-NOTE to psk.rs explaining:
  - Why PSQ is embedded in Noise (single round-trip, PSK binding)
  - KEM migration path (X25519 → MlKem768 → XWing)
- Add AIDEV-NOTE to state_machine.rs explaining protocol flow:
  - KKTExchange → Handshaking → Transport state transitions
  - PSK derivation formula (ECDH || PSQ || salt)

* Add forward_timeout to LP client config

Add forward_timeout (30s default) to LpConfig and wrap send_forward_packet's
connect_send_receive call with tokio::time::timeout, matching the pattern
used by register() with registration_timeout.

This prevents indefinite hangs when forwarding packets through entry gateway.

* Add negotiated_version field to LpSession

Add AtomicU8 field to store the protocol version from handshake packet
headers. Includes getter and setter methods for future version negotiation
and compatibility checks.

- negotiated_version() returns current version (defaults to 1)
- set_negotiated_version() allows setting during handshake
- Subsessions inherit version 1 (can be enhanced to inherit parent's)

* Change MessageType from u16 to u32

Breaking wire protocol change: MessageType field increased from 2 bytes
to 4 bytes in LP packets. This future-proofs the message type space and
aligns with other u32 fields.

Changes:
- message.rs: #[repr(u32)], from_u32(), to_u32()
- error.rs: InvalidMessageType(u32)
- codec.rs: All serialization/deserialization updated to 4-byte msg_type
  - Cleartext parsing: inner_bytes[4..8], content at [8..]
  - AEAD parsing: decrypted[4..8], content at [8..]
  - Serialization: 4 bytes for message type

* Various smaller fixes

* Refactor LP to stream-oriented TCP processing

Gateway (handler.rs):
- Add bound_receiver_idx field for session-affine connections
- Convert handle() from single-packet to loop with EOF detection
- Add validate_or_set_binding() for receiver_idx validation
- Set binding in handle_client_hello after collision check
- Centralize emit_lifecycle_metrics in main loop only
- Add is_connection_closed() helper for graceful EOF

Client (client.rs):
- Add stream field for persistent TCP connection
- Add ensure_connected(), send_packet(), receive_packet(), close() methods
- Modify perform_handshake_inner() to use persistent stream
- Modify register_with_credential() to use persistent stream
- Modify send_forward_packet() to use persistent stream
- Keep connect_send_receive() for reference (marked dead_code)

This reduces handshake overhead from ~5 TCP connections to 1.

Drive-by: Fix log::info! -> info! in wireguard peer_controller.rs

* Add persistent exit stream for entry→exit forwarding

Entry gateway now maintains a persistent TCP connection to the exit
gateway per client session, reusing it for all forward requests from
that client. This reduces TCP handshake overhead significantly.

Key changes:
- Add exit_stream: Option<(TcpStream, SocketAddr)> to LpConnectionHandler
- Modify handle_forward_packet() to open on first forward, reuse after
- Clear exit_stream on connection errors (auto-reconnect on next forward)
- Semaphore only acquired for connection opens, not reuse (sequential access)

* Fix code review issues for stream-oriented LP

- Add 30s timeout to exit stream I/O operations (nym-df31)
  Prevents handler from hanging on unresponsive exit gateway

- Return error on forward target address mismatch (nym-zegu)
  Previously warned and proceeded, which could mask bugs

- Close client stream on handshake error paths (nym-scvm)
  Prevents state machine inconsistency on timeout or failure

* Add LP registration idempotency and retry logic

Make LP registration resilient to network failures that could waste
credentials. When registration succeeds on the gateway but the response
is lost (e.g., network drop), clients can retry with the same WG key
and get the cached result instead of spending another credential.

Gateway-side:
- Add check_existing_registration() helper that looks up WG peer and
  returns cached GatewayData if already registered
- Add idempotency check in process_registration() dVPN branch
- Only return cached response if bandwidth > 0 (ensures registration
  was actually completed, not just peer created)
- Track idempotent registrations with lp_registration_dvpn_idempotent metric

Client-side:
- Add register_with_retry() to LpRegistrationClient that acquires
  credential once and retries handshake+registration on failure
- Add handshake_and_register_with_retry() to NestedLpSession for
  exit gateway registration via forwarding
- Add exponential backoff with jitter between retry attempts
- Verify outer session validity before nested session retry

Both retry methods clear state machine before retry to ensure fresh
handshake, and reuse the same credential across all attempts.

* Add no-mix-acks feature flag to nym-sphinx-framing

When enabled, mix nodes skip ack extraction and forwarding entirely.
The full payload (including ack portion) is returned as the message.

Closes: nym-3wrr

* Create nym-lp-speedtest crate scaffold

- Created tools/nym-lp-speedtest/ with Cargo.toml
- Added main.rs with CLI argument parsing
- Created stub modules: client.rs, speedtest.rs, topology.rs
- Added to workspace members
- Verified compilation with cargo check

* Implement topology fetching for nym-lp-speedtest

- Add topology.rs with NymTopology integration
- Fetch mix nodes and gateways from nym-api
- Build GatewayInfo with LP addresses (port 41264)
- Provide random_route_to_gateway() for Sphinx routing
- Add required Cargo.toml dependencies

* Implement LP+Sphinx+KCP client with SURB support

- Add send_data() and send_data_with_surbs() methods for mixnet data
- Integrate KCP reliable delivery with Sphinx packet construction
- Add x25519 encryption keypair for SURB reply mechanism
- Wire up main.rs to test LP handshake and data path
- Add NymRouteProvider support in topology for SURB construction
- Refactor send_data() to delegate to send_data_with_surbs(0) (DRY)

The client can now:
- Perform LP handshake with gateways
- Send data through the mixnet wrapped in KCP + Sphinx packets
- Attach SURBs for bidirectional communication
- Return encryption keys for decrypting replies

* Rename nym-lp-speedtest to nym-lp-client and fix KCP bug

- Rename crate from nym-lp-speedtest to nym-lp-client
- Fix KCP bug: add driver.update() call before fetch_outgoing()
  Without update(), KCP never moves segments from snd_queue to snd_buf
- Update CLI name, about string, and user agent to match new name

* Add LP mixnet mode registration with nym address return

- Extend RegistrationMode::Mixnet to include client_ed25519_pubkey
  and client_x25519_pubkey for nym address construction
- Add LpGatewayData struct containing gateway_identity and
  gateway_sphinx_key for SURB reply routing
- Add lp_gateway_data field to LpRegistrationResponse for mixnet mode
- Implement success_mixnet() constructor for mixnet registrations
- Update gateway registration to insert clients into ActiveClientsStore
  for SURB reply delivery, matching the websocket flow

* Implement LP data handler on UDP:51264

- Add LpDataHandler for UDP data plane (port 51264)
- Decrypt LP layer and forward Sphinx packets to mixnet
- Add outbound_mix_sender to LpHandlerState
- Integrate data handler spawn into LpListener::run()
- Add metrics for data packets received/forwarded/errors

Implements nym-yzzm

* Fix replay protection vulnerability in LP data handler

Use state machine process_input() instead of manual decryption to ensure
proper replay protection:
- Counter check against receiving window
- Counter marking after successful decryption

Also handle subsession actions gracefully (SendPacket ignored on UDP,
clients should use TCP control plane for rekeying).

Security fix for nym-yzzm implementation.

* feat(ipr): add KcpSessionManager for LP client KCP handling

- Add fetch_incoming() and recv() methods to KcpDriver for retrieving
  reassembled messages
- Create KcpSessionManager in ip-packet-router that manages KCP sessions
  keyed by conv_id (first 4 bytes of KCP packet header)
- Store ReplySurbs per session for sending anonymous replies
- Implement session timeout (5 min) and max sessions limit (10000)
- Add comprehensive tests for session lifecycle and KCP roundtrip

* feat(ipr): integrate KcpSessionManager into MixnetListener

- Add KcpSessionManager field to MixnetListener struct
- Add is_kcp_message() helper to detect KCP-wrapped payloads
- Add on_kcp_message() to process LP client KCP messages
- Refactor on_reconstructed_message() to route KCP vs regular IPR
- Add KCP tick timer (100ms) for session updates and cleanup
- Initialize KcpSessionManager in IpPacketRouter::run_service_provider()

KCP messages are detected by checking byte 4 for valid KCP commands
(81-84), which doesn't conflict with IPR protocol version bytes (6-8)
at position 0.

Closes: nym-96zl

* fix(ipr): prevent KCP detection false positives on IPR messages

Add secondary check in is_kcp_message() to exclude messages that match
IPR protocol header pattern (version 6-8 at byte 0, ServiceProviderType
0-2 at byte 1). This prevents false positives where IPR messages with
byte 4 in range 81-84 would be incorrectly routed to KCP processing.

Added 4 unit tests to validate the detection logic.

Closes: nym-6f3x

* fix(ipr): wrap KCP client responses in KCP before SURB reply

- Modify on_kcp_message to handle responses directly instead of returning them
- Add handle_kcp_response method that wraps response in KCP and sends via mixnet
- Ensures KCP clients receive KCP-wrapped responses for proper reassembly

Closes: nym-7oh2

* fix(ipr): send KCP protocol packets in tick instead of just logging

- Add get_sender_tag() and fetch_outgoing_for_conv() to KcpSessionManager
- Change handle_kcp_tick() to actually send ACKs/retransmissions via mixnet
- Reduce KCP tick interval from 100ms to 10ms for better responsiveness

This fixes the KCP reliability protocol which was broken because
protocol packets (ACKs, retransmissions) were generated but never sent.

* feat(lp-client): wrap payload in IpPacketRequest before KCP

- Add nym-ip-packet-requests and bytes dependencies
- Wrap payload in IpPacketRequest::new_data_request() before sending to KCP
- Add LP_DATA_PORT constant (51264) and lp_data_address field to GatewayInfo

This ensures IPR can properly parse incoming messages as DataRequest.
LP framing (wrapping Sphinx in LP before sending) is a separate task.

* feat(lp-client): add LP session management and UDP data plane support

- Add wrap_data() and session_id() to LpRegistrationClient for LP packet
  creation after handshake
- Add init_lp_session() and close_lp_session() to SpeedtestClient for
  managing LP sessions
- Extract prepare_sphinx_fragments() helper to reduce code duplication
  between send_data_with_surbs() and send_data_via_lp()
- Add send_data_via_lp() for sending Sphinx packets through LP's UDP
  data plane (port 51264)

The LP session is kept alive after TCP handshake closes, allowing
subsequent wrap_data() calls for UDP transmission without re-handshaking.

* random formatting

* replaced all instances of bincode::serialize and bincode::deserialize with explicit lp_bincode_serialiser() within the LP

* additional formatting

* removed source of possible panic from nym-kkt

invalid KEM mapping will now return an Err rather than panicking

* integration test for LP entry registration

This includes creation of mocks of various gateway-related components, such as the PeerController

* changed ClientHelloData serialisation

the old variant using bincode did not produce constant-length output in some cases

* Fixed generation of receiver index

removes the possible clash with the boostrap id

* Integration test for nested LP registration

- move `LpTransport` trait definition to shared `nym-lp-transport` crate
- make transport layer within `LpConnectionHandler` generic with respect to the forwarding target. it must, however, use the same type as the incoming client connection
- extracted explicit `LpConnectionHandler::establish_exit_stream` to more easily modify it in the future to fully protect the channel and disallow using untrusted egress points
- fix additional log-string interpolation nits

* resolved clippy issues pointed out by clippy 1.91

* added LP discovery into self-described endpoint:

- removed changes to the node bonding within the contract
- introduced '/api/v1/lewes-protocol' route within nym-node http api
- added 'lewes_protocol' field to 'NymNodeData' inside of NymNodeDescription
- refactored LpConfig to allow separate bind and announce addresses and used more strict typing

* chore: allow unwrap/expect within kkt benchmarking code

* chore: downgraded sha2 dep for cosmwasm compatibility

* clippy

* marking simd calls as unsafe

* fixed calls to '_mm_testz_si128'

* additional clippy fixes

---------

Co-authored-by: Georgio Nicolas <me@georgio.xyz>
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
This commit is contained in:
Drazen Urch
2026-01-14 10:06:02 +01:00
committed by GitHub
parent fc0b7189c7
commit 8a00ed6071
175 changed files with 39352 additions and 1439 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/LP_PROTOCOL.md
+990
View File
@@ -0,0 +1,990 @@
# Lewes Protocol (LP) - Technical Specification
## Overview
The Lewes Protocol (LP) is a direct TCP-based registration protocol for Nym gateways. It provides an alternative to mixnet-based registration with different trade-offs: lower latency at the cost of revealing client IP to the gateway.
**Design Goals:**
- **Low latency**: Direct TCP connection vs multi-hop mixnet routing
- **High reliability**: KCP protocol provides ordered, reliable delivery with ARQ
- **Strong security**: Noise XKpsk3 provides mutual authentication and forward secrecy
- **Replay protection**: Bitmap-based counter validation prevents replay attacks
- **Observability**: Prometheus metrics for production monitoring
**Non-Goals:**
- Network-level anonymity (use mixnet registration for that)
- Persistent connections (LP is registration-only, single-use)
- Backward compatibility with legacy protocols
## Architecture
### Protocol Stack
```
┌─────────────────────────────────────────┐
│ Application Layer │
│ - Registration Requests │
│ - E-cash Credential Verification │
│ - WireGuard Peer Management │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ LP Layer (Lewes Protocol) │
│ - Noise XKpsk3 Handshake │
│ - Replay Protection (1024-pkt window) │
│ - Counter-based Sequencing │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ KCP Layer (Reliability) │
│ - Ordered Delivery │
│ - ARQ with Selective ACK │
│ - Congestion Control │
│ - RTT Estimation │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ TCP Layer │
│ - Connection Establishment │
│ - Byte Stream Delivery │
└─────────────────────────────────────────┘
```
### Why This Layering?
**TCP**: Provides connection-oriented byte stream and handles network-level retransmission.
**KCP**: Adds application-level reliability optimized for low latency:
- **Fast retransmit**: Triggered after 2 duplicate ACKs (vs TCP's 3)
- **Selective ACK**: Acknowledges specific packets, not just cumulative
- **Configurable RTO**: Minimum RTO of 100ms (configurable)
- **No Nagle**: Immediate sending for low-latency applications
**LP**: Provides cryptographic security and session management:
- **Noise XKpsk3**: Mutual authentication with pre-shared key
- **Replay protection**: Prevents duplicate packet acceptance
- **Session isolation**: Each session has unique cryptographic state
**Application**: Business logic for registration and credential verification.
## Protocol Flow
### 1. Connection Establishment
```
Client Gateway
| |
|--- TCP SYN ---------------------------> |
|<-- TCP SYN-ACK ------------------------ |
|--- TCP ACK ----------------------------> |
| |
```
- **Control Port**: 41264 (default, configurable)
- **Data Port**: 51264 (reserved for future use, not currently used)
### 2. Session Initialization
Client generates session parameters:
```rust
// Client-side session setup
let client_lp_keypair = Keypair::generate(); // X25519 keypair
let gateway_lp_public = gateway.lp_public_key; // From gateway descriptor
let salt = [timestamp (8 bytes) || nonce (24 bytes)]; // 32-byte salt
// Derive PSK using ECDH + Blake3 KDF
let shared_secret = ECDH(client_private, gateway_public);
let psk = Blake3_derive_key(
context = "nym-lp-psk-v1",
input = shared_secret,
salt = salt
);
// Calculate session IDs (deterministic from keys)
let lp_id = hash(client_lp_public || 0xCC || gateway_lp_public) & 0xFFFFFFFF;
let kcp_conv_id = hash(client_lp_public || 0xFF || gateway_lp_public) & 0xFFFFFFFF;
```
**Session ID Properties:**
- **Deterministic**: Same key pair always produces same ID
- **Order-independent**: `ID(A, B) == ID(B, A)` due to sorted hashing
- **Collision-resistant**: Uses full hash, truncated to u32
- **Unique per protocol**: Different delimiters (0xCC for LP, 0xFF for KCP)
### 3. Noise Handshake (XKpsk3 Pattern)
```
Client (Initiator) Gateway (Responder)
| |
|--- e ----------------------------------> | [1] Client ephemeral
| |
|<-- e, ee, s, es --------------------- | [2] Gateway ephemeral + static
| |
|--- s, se, psk -------------------------> | [3] Client static + PSK mix
| |
[Transport mode established]
```
**Message Contents:**
**[1] Initiator → Responder: `e`**
- Payload: Client ephemeral public key (32 bytes)
- Encrypted: No (initial message)
**[2] Responder → Initiator: `e, ee, s, es`**
- `e`: Responder ephemeral public key
- `ee`: Mix ephemeral-ephemeral DH
- `s`: Responder static public key (encrypted)
- `es`: Mix ephemeral-static DH
- Encrypted: Yes (with keys from `ee`)
**[3] Initiator → Responder: `s, se, psk`**
- `s`: Initiator static public key (encrypted)
- `se`: Mix static-ephemeral DH
- `psk`: Mix pre-shared key (at position 3)
- Encrypted: Yes (with keys from `ee`, `es`)
**Security Properties:**
- ✅ **Mutual authentication**: Both sides prove identity via static keys
- ✅ **Forward secrecy**: Ephemeral keys provide PFS
- ✅ **PSK authentication**: Binds session to out-of-band PSK
- ✅ **Identity hiding**: Static keys encrypted after first message
**Handshake Characteristics:**
- **Messages**: 3 (1.5 round trips)
- **Minimum network RTTs**: 1.5
- **Cryptographic operations**: ECDH, ChaCha20-Poly1305, SHA-256
### 4. PSK Derivation Details
**Formula:**
```
shared_secret = X25519(client_private_lp, gateway_public_lp)
psk = Blake3_derive_key(
context = "nym-lp-psk-v1",
key_material = shared_secret (32 bytes),
salt = timestamp || nonce (32 bytes)
)
```
**Implementation** (from `common/nym-lp/src/psk.rs:48`):
```rust
pub fn derive_psk(
local_private: &PrivateKey,
remote_public: &PublicKey,
salt: &[u8; 32],
) -> [u8; 32] {
let shared_secret = local_private.diffie_hellman(remote_public);
nym_crypto::kdf::derive_key_blake3(PSK_CONTEXT, shared_secret.as_bytes(), salt)
}
```
**Why This Design:**
1. **Identity-bound**: PSK tied to LP keypairs, not ephemeral
- Prevents MITM without LP private key
- Links session to long-term identities
2. **Session-specific via salt**: Different registrations use different PSKs
- `timestamp`: 8-byte Unix timestamp (milliseconds)
- `nonce`: 24-byte random value
- Prevents PSK reuse across sessions
3. **Symmetric derivation**: Both sides derive same PSK
- Client: `ECDH(client_priv, gateway_pub)`
- Gateway: `ECDH(gateway_priv, client_pub)`
- Mathematical property: `ECDH(a, B) == ECDH(b, A)`
4. **Blake3 KDF with domain separation**:
- Context string prevents cross-protocol attacks
- Generates uniform 32-byte output suitable for Noise
**Salt Transmission:**
- Included in `ClientHello` message (cleartext)
- Gateway extracts salt before deriving PSK
- Timestamp validation rejects stale salts
### 5. Replay Protection
**Mechanism: Sliding Window with Bitmap** (from `common/nym-lp/src/replay/validator.rs:32`):
```rust
const WORD_SIZE: usize = 64;
const N_WORDS: usize = 16; // 1024 bits total
const N_BITS: usize = WORD_SIZE * N_WORDS; // 1024
pub struct ReceivingKeyCounterValidator {
next: u64, // Next expected counter
receive_cnt: u64, // Total packets received
bitmap: [u64; 16], // 1024-bit bitmap
}
```
**Algorithm:**
```
For each incoming packet with counter C:
1. Quick check (branchless):
- If C >= next: Accept (growing)
- If C + 1024 < next: Reject (too old, outside window)
- If bitmap[C % 1024] is set: Reject (duplicate)
- Else: Accept (out-of-order within window)
2. After successful processing, mark:
- Set bitmap[C % 1024] = 1
- If C >= next: Update next = C + 1
- Increment receive_cnt
```
**Performance Optimizations:**
1. **SIMD-accelerated bitmap operations** (from `common/nym-lp/src/replay/simd/`):
- AVX2 support (x86_64)
- SSE2 support (x86_64)
- NEON support (ARM)
- Scalar fallback (portable)
2. **Branchless execution** (constant-time):
```rust
// No early returns - prevents timing attacks
let result = if is_growing {
Some(Ok(()))
} else if too_far_back {
Some(Err(ReplayError::OutOfWindow))
} else if duplicate {
Some(Err(ReplayError::DuplicateCounter))
} else {
Some(Ok(()))
};
result.unwrap()
```
3. **Overflow-safe arithmetic**:
```rust
let too_far_back = if counter > u64::MAX - 1024 {
false // Can't overflow, so not too far back
} else {
counter + 1024 < self.next
};
```
**Memory Usage** (verified from `common/nym-lp/src/replay/validator.rs:738`):
```rust
// test_memory_usage()
size = size_of::<u64>() * 2 + // next + receive_cnt = 16 bytes
size_of::<u64>() * N_WORDS; // bitmap = 128 bytes
// Total: 144 bytes
```
### 6. Registration Request
After handshake completes, client sends encrypted registration request:
```rust
pub struct RegistrationRequest {
pub mode: RegistrationMode,
pub credential: EcashCredential,
pub gateway_identity: String,
}
pub enum RegistrationMode {
Dvpn {
wg_public_key: [u8; 32],
},
Mixnet {
client_id: String,
mix_address: Option<String>,
},
}
```
**Encryption:**
- Encrypted using Noise transport mode
- Includes 16-byte Poly1305 authentication tag
- Replay protection via LP counter
### 7. Credential Verification
Gateway verifies the e-cash credential:
```rust
// Gateway-side verification
pub async fn verify_credential(
&self,
credential: &EcashCredential,
) -> Result<VerifiedCredential, CredentialError> {
// 1. Check credential signature (BLS12-381)
verify_blinded_signature(&credential.signature)?;
// 2. Check credential not already spent (nullifier check)
if self.storage.is_nullifier_spent(&credential.nullifier).await? {
return Err(CredentialError::AlreadySpent);
}
// 3. Extract bandwidth allocation
let bandwidth_bytes = credential.bandwidth_value;
// 4. Mark nullifier as spent
self.storage.mark_nullifier_spent(&credential.nullifier).await?;
Ok(VerifiedCredential {
bandwidth_bytes,
expiry: credential.expiry,
})
}
```
**For dVPN Mode:**
```rust
let peer_config = WireguardPeerConfig {
public_key: request.wg_public_key,
allowed_ips: vec!["10.0.0.0/8"],
bandwidth_limit: verified.bandwidth_bytes,
};
self.wg_controller.add_peer(peer_config).await?;
```
### 8. Registration Response
```rust
pub enum RegistrationResponse {
Success {
bandwidth_allocated: u64,
expiry: u64,
gateway_data: GatewayData,
},
Error {
code: ErrorCode,
message: String,
},
}
pub enum ErrorCode {
InvalidCredential = 1,
CredentialExpired = 2,
CredentialAlreadyUsed = 3,
InsufficientBandwidth = 4,
WireguardPeerRegistrationFailed = 5,
InternalError = 99,
}
```
## State Machine and Security Protocol
### Protocol Components
The Lewes Protocol combines three cryptographic protocols for secure, post-quantum resistant communication:
1. **KKT (KEM Key Transfer)** - Dynamically fetches responder's KEM public key with Ed25519 authentication
2. **PSQ (Post-Quantum Secure PSK)** - Derives PSK using KEM-based protocol for HNDL resistance
3. **Noise XKpsk3** - Provides encrypted transport with mutual authentication and forward secrecy
### State Machine
The LP state machine orchestrates the complete protocol flow from connection to encrypted transport:
```
┌─────────────────────────────────────────────────────────────────────┐
│ LEWES PROTOCOL STATE MACHINE │
└─────────────────────────────────────────────────────────────────────┘
┌──────────────────┐
│ ReadyToHandshake │
│ │
│ • Keys loaded │
│ • Session ID set │
└────────┬─────────┘
│
StartHandshake input
│
▼
┌───────────────────────────────────────┐
│ KKTExchange │
│ │
│ Initiator: │
│ 1. Send KKT request (signed) │
│ 2. Receive KKT response │
│ 3. Validate Ed25519 signature │
│ 4. Extract KEM public key │
│ │
│ Responder: │
│ 1. Wait for KKT request │
│ 2. Validate signature │
│ 3. Send signed KEM key │
└───────────────┬───────────────────────┘
│
KKT Complete
│
▼
┌───────────────────────────────────────┐
│ Handshaking │
│ │
│ PSQ Protocol: │
│ 1. Initiator encapsulates PSK │
│ (embedded in Noise msg 1) │
│ 2. Responder decapsulates PSK │
│ (sends ctxt_B in Noise msg 2) │
│ 3. Both derive final PSK: │
│ KDF(ECDH || KEM_shared) │
│ │
│ Noise XKpsk3 Handshake: │
│ → msg 1: e, es, ss + PSQ payload │
│ ← msg 2: e, ee, se + ctxt_B │
│ → msg 3: s, se (handshake complete) │
└───────────────┬───────────────────────┘
│
Handshake Complete
│
▼
┌───────────────────────────────────────┐
│ Transport │
│ │
│ • Encrypted data transfer │
│ • AEAD with ChaCha20-Poly1305 │
│ • Replay protection (counters) │
│ • Bidirectional communication │
└───────────────┬───────────────────────┘
│
Close input
│
▼
┌──────────┐
│ Closed │
│ │
│ • Reason │
└──────────┘
```
### Message Sequence
Complete protocol flow from connection to encrypted transport:
```
Initiator Responder
│ │
│ ════════════════ KKT EXCHANGE ════════════════ │
│ │
│ KKTRequest (signed with Ed25519) │
├──────────────────────────────────────────────────────────>│
│ │ Validate
│ │ signature
│ KKTResponse (signed KEM key + hash) │
│<──────────────────────────────────────────────────────────┤
│ │
│ Validate signature │
│ Extract kem_pk │
│ │
│ ══════════════ PSQ + NOISE HANDSHAKE ══════════════ │
│ │
│ Noise msg 1: e, es, ss │
│ + PSQ InitiatorMsg (KEM encapsulation) │
├──────────────────────────────────────────────────────────>│
│ │
│ │ PSQ: Decapsulate
│ │ Derive PSK
│ │ Inject into Noise
│ Noise msg 2: e, ee, se │
│ + ctxt_B (encrypted PSK) │
│<──────────────────────────────────────────────────────────┤
│ │
│ Extract ctxt_B │
│ Store for re-registration │
│ Inject PSK into Noise │
│ │
│ Noise msg 3: s, se │
├──────────────────────────────────────────────────────────>│
│ │
│ Handshake Complete ✓ │ Handshake Complete ✓
│ Transport mode active │ Transport mode active
│ │
│ ═══════════════ TRANSPORT MODE ═══════════════ │
│ │
│ EncryptedData (AEAD, counter N) │
├──────────────────────────────────────────────────────────>│
│ │
│ EncryptedData (counter M) │
│<──────────────────────────────────────────────────────────┤
│ │
│ (bidirectional encrypted communication) │
│◄──────────────────────────────────────────────────────────►
│ │
```
### KKT (KEM Key Transfer) Protocol
**Purpose**: Securely obtain responder's KEM public key before PSQ can begin.
**Key Features**:
- Ed25519 signatures for authentication (both request and response signed)
- Optional hash validation for key pinning (future directory service integration)
- Currently signature-only mode (deployable without infrastructure)
- Easy upgrade path to hash-based key pinning
**Initiator Flow**:
```rust
1. Generate KKT request with Ed25519 signature
2. Send KKTRequest to responder
3. Receive KKTResponse with signed KEM key
4. Validate Ed25519 signature
5. (Optional) Validate key hash against directory
6. Store KEM key for PSQ encapsulation
```
**Responder Flow**:
```rust
1. Receive KKTRequest from initiator
2. Validate initiator's Ed25519 signature
3. Generate KKTResponse with:
- Responder's KEM public key
- Ed25519 signature over (key || timestamp)
- Blake3 hash of KEM key
4. Send KKTResponse to initiator
```
### PSQ (Post-Quantum Secure PSK) Protocol
**Purpose**: Derive a post-quantum secure PSK for Noise protocol.
**Security Properties**:
- **HNDL resistance**: PSK derived from KEM-based protocol
- **Forward secrecy**: Ephemeral KEM keypair per session
- **Authentication**: Ed25519 signatures prevent MitM
- **Algorithm agility**: Easy upgrade from X25519 to ML-KEM
**PSK Derivation**:
```
Classical ECDH:
ecdh_secret = X25519_DH(local_private, remote_public)
KEM Encapsulation (Initiator):
(kem_shared_secret, ciphertext) = KEM.Encap(responder_kem_pk)
KEM Decapsulation (Responder):
kem_shared_secret = KEM.Decap(kem_private, ciphertext)
Final PSK:
combined = ecdh_secret || kem_shared_secret || salt
psk = Blake3_KDF("nym-lp-psk-psq-v1", combined)
```
**Integration with Noise**:
- PSQ payload embedded in first Noise message (no extra round-trip)
- Responder sends encrypted PSK handle (ctxt_B) in second Noise message
- Both sides inject derived PSK before completing Noise handshake
- Noise validates PSK correctness during handshake
**PSK Handle (ctxt_B)**:
The responder's encrypted PSK handle allows future re-registration without repeating PSQ:
- Encrypted with responder's long-term key
- Can be presented in future sessions
- Enables fast re-registration for returning clients
### Security Guarantees
**Achieved Properties**:
- ✅ **Mutual authentication**: Ed25519 signatures in KKT and PSQ
- ✅ **Forward secrecy**: Ephemeral keys in Noise handshake
- ✅ **Post-quantum PSK**: KEM-based PSK derivation
- ✅ **HNDL resistance**: PSK safe even if private keys compromised later
- ✅ **Replay protection**: Monotonic counters with sliding window
- ✅ **Key confirmation**: Noise handshake validates PSK correctness
**Implementation Status**:
- 🔄 **Key pinning**: Hash validation via directory service (signature-only for now)
- 🔄 **ML-KEM support**: Easy config upgrade from X25519 to ML-KEM-768
- 🔄 **PSK re-use**: ctxt_B handle stored for future re-registration
### Algorithm Choices
**Current (Testing/Development)**:
- KEM: X25519 (DHKEM) - Classical ECDH, widely tested
- Hash: Blake3 - Fast, secure, parallel
- Signature: Ed25519 - Fast verification, compact
- AEAD: ChaCha20-Poly1305 - Fast, constant-time
**Future (Production)**:
- KEM: ML-KEM-768 - NIST-approved post-quantum KEM
- Hash: Blake3 - No change needed
- Signature: Ed25519 - No change needed (or upgrade to ML-DSA)
- AEAD: ChaCha20-Poly1305 - No change needed
**Migration Path**:
```toml
# Current deployment
[lp.crypto]
kem_algorithm = "x25519"
# Future upgrade (config change only)
[lp.crypto]
kem_algorithm = "ml-kem-768"
```
### Message Types
**KKT Messages**:
```rust
// Message Type 0x0004
struct KKTRequest {
timestamp: u64, // Unix timestamp (replay protection)
initiator_ed25519_pk: [u8; 32], // Initiator's public key
signature: [u8; 64], // Ed25519 signature
}
// Message Type 0x0005
struct KKTResponse {
kem_pk: Vec<u8>, // Responder's KEM public key
key_hash: [u8; 32], // Blake3 hash of KEM key
timestamp: u64, // Unix timestamp
signature: [u8; 64], // Ed25519 signature
}
```
**PSQ Embedding**:
- PSQ InitiatorMsg embedded in Noise message 1 payload (after 'e, es, ss')
- PSQ ResponderMsg (ctxt_B) embedded in Noise message 2 payload (after 'e, ee, se')
- No additional round-trips beyond standard 3-message Noise handshake
## KCP Protocol Details
### KCP Configuration
From `common/nym-kcp/src/session.rs`:
```rust
pub struct KcpSession {
conv: u32, // Conversation ID
mtu: usize, // Default: 1400 bytes
snd_wnd: u16, // Send window: 128 segments
rcv_wnd: u16, // Receive window: 128 segments
rx_minrto: u32, // Minimum RTO: 100ms (configurable)
}
```
### KCP Packet Format
```
┌────────────────────────────────────────────────┐
│ Conv ID (4 bytes) - Conversation identifier │
├────────────────────────────────────────────────┤
│ Cmd (1 byte) - PSH/ACK/WND/ERR │
├────────────────────────────────────────────────┤
│ Frg (1 byte) - Fragment number (reverse order) │
├────────────────────────────────────────────────┤
│ Wnd (2 bytes) - Receive window size │
├────────────────────────────────────────────────┤
│ Timestamp (4 bytes) - Send timestamp │
├────────────────────────────────────────────────┤
│ Sequence Number (4 bytes) - Packet sequence │
├────────────────────────────────────────────────┤
│ UNA (4 bytes) - Unacknowledged sequence │
├────────────────────────────────────────────────┤
│ Length (4 bytes) - Data length │
├────────────────────────────────────────────────┤
│ Data (variable) - Payload │
└────────────────────────────────────────────────┘
```
**Total header**: 24 bytes
### KCP Features
**Reliability Mechanisms:**
- **Sequence Numbers (sn)**: Track packet ordering
- **Fragment Numbers (frg)**: Handle message fragmentation
- **UNA (Unacknowledged)**: Cumulative ACK up to this sequence
- **Selective ACK**: Via individual ACK packets
- **Fast Retransmit**: Triggered by duplicate ACKs (configurable threshold)
- **RTO Calculation**: Smoothed RTT with variance
## LP Packet Format
### LP Header
```
┌────────────────────────────────────────────────┐
│ Protocol Version (1 byte) - Currently: 1 │
├────────────────────────────────────────────────┤
│ Session ID (4 bytes) - LP session identifier │
├────────────────────────────────────────────────┤
│ Counter (8 bytes) - Replay protection counter │
└────────────────────────────────────────────────┘
```
**Total header**: 13 bytes
### LP Message Types
```rust
pub enum LpMessage {
Handshake(Vec<u8>),
EncryptedData(Vec<u8>),
ClientHello {
client_lp_public: [u8; 32],
salt: [u8; 32],
timestamp: u64,
},
Busy,
}
```
### Complete Packet Structure
```
┌─────────────────────────────────────┐
│ LP Header (13 bytes) │
│ - Version, Session ID, Counter │
├─────────────────────────────────────┤
│ LP Message (variable) │
│ - Type tag (1 byte) │
│ - Message data │
├─────────────────────────────────────┤
│ Trailer (16 bytes) │
│ - Reserved for future MAC/tag │
└─────────────────────────────────────┘
```
## Security Properties
### Threat Model
**Protected Against:**
- ✅ **Passive eavesdropping**: Noise encryption (ChaCha20-Poly1305)
- ✅ **Active MITM**: Mutual authentication via static keys + PSK
- ✅ **Replay attacks**: Counter-based validation with 1024-packet window
- ✅ **Packet injection**: Poly1305 authentication tags
- ✅ **Timestamp replay**: 30-second window for ClientHello timestamps (configurable)
- ✅ **DoS (connection flood)**: Connection limit (default: 10,000, configurable)
- ✅ **Credential reuse**: Nullifier tracking in database
**Not Protected Against:**
- ❌ **Network-level traffic analysis**: LP is not anonymous (use mixnet for that)
- ❌ **Gateway compromise**: Gateway sees client registration data
- ⚠️ **Per-IP DoS**: No per-IP rate limiting (global limit only)
### Cryptographic Primitives
| Component | Algorithm | Key Size | Source |
|-----------|-----------|----------|--------|
| Key Exchange | X25519 | 256 bits | RustCrypto |
| Encryption | ChaCha20 | 256 bits | RustCrypto |
| Authentication | Poly1305 | 256 bits | RustCrypto |
| KDF | Blake3 | 256 bits | nym_crypto |
| Hash (Noise) | SHA-256 | 256 bits | snow crate |
| Signature (E-cash) | BLS12-381 | 381 bits | E-cash contract |
### Forward Secrecy
Noise XKpsk3 provides forward secrecy through ephemeral keys:
1. **Initial handshake**: Uses ephemeral + static keys
2. **Key compromise scenario**:
- Compromise of **static key**: Past sessions remain secure (ephemeral keys destroyed)
- Compromise of **PSK**: Attacker needs static key too (two-factor security)
- Compromise of **both**: Only future sessions affected, not past
3. **Session key lifetime**: Destroyed after single registration completes
### Timing Attack Resistance
**Constant-time operations:**
- ✅ Replay protection check (branchless)
- ✅ Bitmap bit operations (branchless)
- ✅ Noise crypto operations (via snow/RustCrypto)
**Variable-time operations:**
- ⚠️ Credential verification (database lookup time varies)
- ⚠️ WireGuard peer registration (filesystem operations)
## Configuration
### Gateway Configuration
From `gateway/src/node/lp_listener/mod.rs:78`:
```toml
[lp]
# Enable/disable LP listener
enabled = true
# Bind address
bind_address = "0.0.0.0"
# Control port (for LP handshake and registration)
control_port = 41264
# Data port (reserved for future use)
data_port = 51264
# Maximum concurrent connections
max_connections = 10000
# Timestamp validation window (seconds)
# ClientHello messages older than this are rejected
timestamp_tolerance_secs = 30
# Use mock e-cash verifier (TESTING ONLY!)
use_mock_ecash = false
```
### Firewall Rules
**Required inbound rules:**
```bash
# Allow TCP connections to LP control port
iptables -A INPUT -p tcp --dport 41264 -j ACCEPT
# Optional: Rate limiting
iptables -A INPUT -p tcp --dport 41264 -m state --state NEW \
-m recent --set --name LP_LIMIT
iptables -A INPUT -p tcp --dport 41264 -m state --state NEW \
-m recent --update --seconds 60 --hitcount 100 --name LP_LIMIT \
-j DROP
```
## Metrics
From `gateway/src/node/lp_listener/mod.rs:4`:
**Connection Metrics:**
- `active_lp_connections`: Gauge tracking current active LP connections
- `lp_connections_total`: Counter for total LP connections handled
- `lp_connection_duration_seconds`: Histogram of connection durations
- `lp_connections_completed_gracefully`: Counter for successful completions
- `lp_connections_completed_with_error`: Counter for error terminations
**Handshake Metrics:**
- `lp_handshakes_success`: Counter for successful handshakes
- `lp_handshakes_failed`: Counter for failed handshakes
- `lp_handshake_duration_seconds`: Histogram of handshake durations
- `lp_client_hello_failed`: Counter for ClientHello failures
**Registration Metrics:**
- `lp_registration_attempts_total`: Counter for all registration attempts
- `lp_registration_success_total`: Counter for successful registrations
- `lp_registration_failed_total`: Counter for failed registrations
- `lp_registration_duration_seconds`: Histogram of registration durations
**Mode-Specific:**
- `lp_registration_dvpn_attempts/success/failed`: dVPN mode counters
- `lp_registration_mixnet_attempts/success/failed`: Mixnet mode counters
**Credential Metrics:**
- `lp_credential_verification_attempts/success/failed`: Verification counters
- `lp_bandwidth_allocated_bytes_total`: Total bandwidth allocated
**Error Metrics:**
- `lp_errors_handshake`: Handshake errors
- `lp_errors_timestamp_too_old/too_far_future`: Timestamp validation errors
- `lp_errors_wg_peer_registration`: WireGuard peer registration failures
## Error Codes
### Handshake Errors
| Error | Description |
|-------|-------------|
| `NOISE_DECRYPT_ERROR` | Invalid ciphertext or wrong keys |
| `NOISE_PROTOCOL_ERROR` | Unexpected message or state |
| `REPLAY_DUPLICATE` | Counter already seen |
| `REPLAY_OUT_OF_WINDOW` | Counter outside 1024-packet window |
| `TIMESTAMP_TOO_OLD` | ClientHello > configured tolerance |
| `TIMESTAMP_FUTURE` | ClientHello from future |
### Registration Errors
| Code | Name | Description |
|------|------|-------------|
| `CREDENTIAL_INVALID` | Invalid credential | Signature verification failed |
| `CREDENTIAL_EXPIRED` | Credential expired | Past expiry timestamp |
| `CREDENTIAL_SPENT` | Already used | Nullifier already in database |
| `INSUFFICIENT_BANDWIDTH` | Not enough bandwidth | Requested > credential value |
| `WIREGUARD_FAILED` | Peer registration failed | Kernel error adding WireGuard peer |
## Limitations
### Current Limitations
1. **No persistent sessions**: Each registration is independent
2. **Single registration per session**: Connection closes after registration
3. **No streaming**: Protocol is request-response only
4. **No gateway discovery**: Client must know gateway's LP public key beforehand
5. **No version negotiation**: Protocol version fixed at 1
6. **No per-IP rate limiting**: Only global connection limit
### Testing Gaps
1. **No end-to-end integration tests**: Unit tests exist, integration tests pending
2. **No performance benchmarks**: Latency/throughput not measured
3. **No load testing**: Concurrent connection limits not stress-tested
4. **No security audit**: Cryptographic implementation not externally reviewed
## References
### Specifications
- **Noise Protocol Framework**: https://noiseprotocol.org/noise.html
- **XKpsk3 Pattern**: https://noiseexplorer.com/patterns/XKpsk3/
- **KCP Protocol**: https://github.com/skywind3000/kcp
- **Blake3**: https://github.com/BLAKE3-team/BLAKE3-specs
### Implementations
- **snow**: Rust Noise protocol implementation
- **RustCrypto**: Cryptographic primitives (ChaCha20-Poly1305, X25519)
- **tokio**: Async runtime for network I/O
### Security Audits
- [ ] Noise implementation audit (pending)
- [ ] Replay protection audit (pending)
- [ ] E-cash integration audit (pending)
- [ ] Penetration testing (pending)
## Changelog
### Version 1.1 (Post-Quantum PSK with KKT)
**Implemented:**
- KKTExchange state in state machine for pre-handshake KEM key transfer
- PSQ (Post-Quantum Secure PSK) protocol integration
- KKT (KEM Key Transfer) protocol with Ed25519 authentication
- Optional hash validation for KEM key pinning (signature-only mode active)
- PSK handle (ctxt_B) storage for future re-registration
- X25519 DHKEM support (ready for ML-KEM upgrade)
- Comprehensive state machine tests (7 test cases)
- generate_fresh_salt() utility for session creation
**Security Improvements:**
- Post-quantum PSK derivation (KEM-based)
- HNDL (Harvest Now, Decrypt Later) resistance
- Mutual authentication via Ed25519 signatures
- Easy migration path to ML-KEM-768
**Architecture:**
- State flow: ReadyToHandshake → KKTExchange → Handshaking → Transport
- PSQ embedded in Noise handshake (no extra round-trip)
- Automatic KKT on StartHandshake (no manual key distribution)
**Related Issues:**
- nym-4za: Add KKTExchange state to LpStateMachine
### Version 1.0 (Initial Implementation)
**Implemented:**
- Noise XKpsk3 handshake
- KCP reliability layer
- Replay protection (1024-packet window with SIMD)
- PSK derivation (ECDH + Blake3)
- dVPN and Mixnet registration modes
- E-cash credential verification
- WireGuard peer management
- Prometheus metrics
- DoS protection (connection limits, timestamp validation)
**Pending:**
- End-to-end integration tests
- Performance benchmarks
- Security audit
- Client implementation
- Gateway probe support
- Per-IP rate limiting
docs/LP_README.md
+470
View File
@@ -0,0 +1,470 @@
# Lewes Protocol (LP) - Fast Gateway Registration
## What is LP?
The Lewes Protocol (LP) is a direct TCP-based registration protocol for Nym gateways. It provides an alternative to mixnet-based registration with different trade-offs.
**Trade-offs:**
- **Faster**: Direct TCP connection vs multi-hop mixnet routing (fewer hops = lower latency)
- **Less Anonymous**: Client IP visible to gateway (mixnet hides IP)
- **More Reliable**: KCP provides ordered delivery with fast retransmission
- **Secure**: Noise XKpsk3 provides mutual authentication and forward secrecy
**Use LP when:**
- Fast registration is important
- Network anonymity is not required for the registration step
- You want reliable, ordered delivery
**Use mixnet registration when:**
- Network-level anonymity is essential
- IP address hiding is required
- Traffic analysis resistance is critical
## Quick Start
### For Gateway Operators
```bash
# 1. Enable LP in gateway config
cat >> ~/.nym/gateways/<id>/config/config.toml << EOF
[lp]
enabled = true
bind_address = "0.0.0.0"
control_port = 41264
max_connections = 10000
timestamp_tolerance_secs = 30
EOF
# 2. Open firewall
sudo ufw allow 41264/tcp
# 3. Restart gateway
systemctl restart nym-gateway
# 4. Verify LP listener
sudo netstat -tlnp | grep 41264
curl http://localhost:8080/metrics | grep lp_connections_total
```
### For Client Developers
```rust
use nym_registration_client::{RegistrationClient, RegistrationMode};
// Initialize client
let client = RegistrationClient::builder()
.gateway_identity("gateway-identity-key")
.gateway_lp_public_key(gateway_lp_pubkey) // From gateway descriptor
.gateway_lp_address("gateway-ip:41264")
.mode(RegistrationMode::Lp)
.build()?;
// Register with dVPN mode
let result = client.register_lp(
credential,
RegistrationMode::Dvpn {
wg_public_key: client_wg_pubkey,
}
).await?;
match result {
LpRegistrationResult::Success { gateway_data, bandwidth_allocated, .. } => {
// Use gateway_data to configure WireGuard tunnel
}
LpRegistrationResult::Error { code, message } => {
eprintln!("Registration failed: {} (code: {})", message, code);
}
}
```
## Architecture
```
┌─────────────────────────────────────────┐
│ Application │
│ - Registration Request │
│ - E-cash Verification │
│ - WireGuard Setup │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ LP Layer │
│ - Noise XKpsk3 Handshake │
│ - Replay Protection (1024 packets) │
│ - Counter-based Sequencing │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ KCP Layer │
│ - Ordered Delivery │
│ - Fast Retransmission │
│ - Congestion Control │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ TCP │
│ - Connection-oriented │
│ - Byte Stream │
└─────────────────────────────────────────┘
```
### Why This Stack?
**TCP**: Reliable connection establishment, handles network-level packet loss.
**KCP**: Application-level reliability optimized for low latency:
- Fast retransmit after 2 duplicate ACKs (vs TCP's 3)
- Selective acknowledgment (better than TCP's cumulative ACK)
- Minimum RTO of 100ms (configurable, vs TCP's typical 200ms+)
**LP**: Cryptographic security:
- **Noise XKpsk3**: Mutual authentication + forward secrecy
- **Replay Protection**: 1024-packet sliding window
- **Session Isolation**: Each registration has unique crypto state
**Application**: Credential verification and peer registration logic.
## Key Features
### Security
**Cryptographic Primitives:**
- **Noise XKpsk3**: Mutual authentication with PSK
- **ChaCha20-Poly1305**: Authenticated encryption
- **X25519**: Key exchange
- **Blake3**: KDF for PSK derivation
**Security Properties:**
- Mutual authentication (both client and gateway prove identity)
- Forward secrecy (past sessions remain secure if keys compromised)
- Replay protection (1024-packet sliding window with SIMD optimization)
- Timestamp validation (30-second window, configurable)
### Observability
**Prometheus metrics** (from `gateway/src/node/lp_listener/mod.rs:4`):
- Connection counts and durations
- Handshake success/failure rates
- Registration outcomes (dVPN vs Mixnet)
- Credential verification results
- Error categorization
- Latency histograms
### DoS Protection
From `gateway/src/node/lp_listener/mod.rs`:
- **Connection limits**: Configurable max concurrent connections (default: 10,000)
- **Timestamp validation**: Rejects messages outside configured window (default: 30s)
- **Replay protection**: Prevents packet replay attacks
## Components
### Core Modules
| Module | Path | Purpose |
|--------|------|---------|
| **nym-lp** | `common/nym-lp/` | Core LP protocol implementation |
| **nym-kcp** | `common/nym-kcp/` | KCP reliability protocol |
| **lp_listener** | `gateway/src/node/lp_listener/` | Gateway-side LP listener |
### Key Files
**Protocol:**
- `common/nym-lp/src/noise_protocol.rs` - Noise state machine
- `common/nym-lp/src/replay/validator.rs` - Replay protection
- `common/nym-lp/src/psk.rs` - PSK derivation
- `common/nym-lp/src/session.rs` - LP session management
**KCP:**
- `common/nym-kcp/src/session.rs` - KCP state machine
- `common/nym-kcp/src/packet.rs` - KCP packet format
**Gateway:**
- `gateway/src/node/lp_listener/mod.rs` - TCP listener
- `gateway/src/node/lp_listener/handler.rs` - Connection handler
- `gateway/src/node/lp_listener/handshake.rs` - Noise handshake
- `gateway/src/node/lp_listener/registration.rs` - Registration logic
## Protocol Flow
### 1. Connection Establishment
```
Client Gateway
|--- TCP SYN ------------> |
|<-- TCP SYN-ACK --------- |
|--- TCP ACK ------------> |
```
Port: 41264 (default, configurable)
### 2. Session Setup
```rust
// Client generates session parameters
let salt = [timestamp (8 bytes) || nonce (24 bytes)];
let shared_secret = ECDH(client_lp_private, gateway_lp_public);
let psk = Blake3_derive_key("nym-lp-psk-v1", shared_secret, salt);
// Deterministic session IDs (order-independent)
let lp_id = hash(client_pub || 0xCC || gateway_pub) & 0xFFFFFFFF;
let kcp_conv = hash(client_pub || 0xFF || gateway_pub) & 0xFFFFFFFF;
```
### 3. Noise Handshake (XKpsk3)
```
Client Gateway
|--- e ------------------------>| [1] Client ephemeral
|<-- e, ee, s, es -------------| [2] Gateway ephemeral + static
|--- s, se, psk -------------->| [3] Client static + PSK
[Transport mode established]
```
**Handshake characteristics:**
- 3 messages (1.5 round trips minimum)
- Cryptographic operations: ECDH, ChaCha20-Poly1305, SHA-256
### 4. Registration
```
Client Gateway
|--- RegistrationRequest ------>| (encrypted)
| | [Verify credential]
| | [Register WireGuard peer if dVPN]
|<-- RegistrationResponse ------| (encrypted)
```
### 5. Connection Close
After successful registration, connection is closed. LP is registration-only.
## Configuration
### Gateway
```toml
# ~/.nym/gateways/<id>/config/config.toml
[lp]
enabled = true
bind_address = "0.0.0.0"
control_port = 41264
data_port = 51264 # Reserved, not currently used
max_connections = 10000
timestamp_tolerance_secs = 30
use_mock_ecash = false # TESTING ONLY!
```
### Environment Variables
```bash
RUST_LOG=nym_gateway::node::lp_listener=debug
LP_ENABLED=true
LP_CONTROL_PORT=41264
LP_MAX_CONNECTIONS=20000
```
## Monitoring
### Key Metrics
**Connections:**
```promql
nym_gateway_active_lp_connections
rate(nym_gateway_lp_connections_total[5m])
rate(nym_gateway_lp_connections_completed_with_error[5m])
```
**Handshakes:**
```promql
rate(nym_gateway_lp_handshakes_success[5m])
rate(nym_gateway_lp_handshakes_failed[5m])
histogram_quantile(0.95, nym_gateway_lp_handshake_duration_seconds)
```
**Registrations:**
```promql
rate(nym_gateway_lp_registration_success_total[5m])
rate(nym_gateway_lp_registration_dvpn_success[5m])
rate(nym_gateway_lp_registration_mixnet_success[5m])
histogram_quantile(0.95, nym_gateway_lp_registration_duration_seconds)
```
### Recommended Alerts
```yaml
- alert: LPHighRejectionRate
expr: rate(nym_gateway_lp_connections_completed_with_error[5m]) > 10
for: 5m
- alert: LPHandshakeFailures
expr: rate(nym_gateway_lp_handshakes_failed[5m]) / rate(nym_gateway_lp_handshakes_success[5m]) > 0.05
for: 10m
```
## Testing
### Unit Tests
```bash
# Run all LP tests
cargo test -p nym-lp
cargo test -p nym-kcp
# Specific suites
cargo test -p nym-lp replay
cargo test -p nym-kcp session
```
**Test Coverage** (from code):
| Component | Tests | Focus Areas |
|-----------|-------|-------------|
| Replay Protection | 14 | Edge cases, concurrency, overflow |
| KCP Session | 12 | Out-of-order, retransmit, window |
| PSK Derivation | 5 | Determinism, symmetry, salt |
| LP Session | 10 | Handshake, encrypt/decrypt |
### Missing Tests
- [ ] End-to-end registration flow
- [ ] Network failure scenarios
- [ ] Credential verification integration
- [ ] Load testing (concurrent connections)
- [ ] Performance benchmarks
## Troubleshooting
### Connection Refused
```bash
# Check listener
sudo netstat -tlnp | grep 41264
# Check config
grep "lp.enabled" ~/.nym/gateways/<id>/config/config.toml
# Check firewall
sudo ufw status | grep 41264
```
### Handshake Failures
```bash
# Check logs
journalctl -u nym-gateway | grep "handshake.*failed"
# Common causes:
# - Wrong gateway LP public key
# - Clock skew > 30s (check with: timedatectl)
# - Replay detection (retry with fresh connection)
```
### High Rejection Rate
```bash
# Check metrics
curl http://localhost:8080/metrics | grep lp_connections_completed_with_error
# Check connection limit
curl http://localhost:8080/metrics | grep active_lp_connections
```
See [LP_DEPLOYMENT.md](./LP_DEPLOYMENT.md#troubleshooting) for detailed guide.
## Security
### Threat Model
**Protected Against:**
- ✅ Passive eavesdropping (Noise encryption)
- ✅ Active MITM (mutual authentication)
- ✅ Replay attacks (counter-based validation)
- ✅ Packet injection (Poly1305 MAC)
- ✅ DoS (connection limits, timestamp validation)
**Not Protected Against:**
- ❌ Network-level traffic analysis (IP visible)
- ❌ Gateway compromise (sees registration data)
- ⚠️ Per-IP DoS (global limit only, not per-IP)
**Key Properties:**
- **Forward Secrecy**: Past sessions secure if keys compromised
- **Mutual Authentication**: Both parties prove identity
- **Replay Protection**: 1024-packet sliding window (verified: 144 bytes memory)
- **Constant-Time**: Replay checks are branchless (timing-attack resistant)
See [LP_SECURITY.md](./LP_SECURITY.md) for complete security analysis.
### Known Limitations
1. **No network anonymity**: Client IP visible to gateway
2. **Not quantum-resistant**: X25519 vulnerable to Shor's algorithm
3. **Single-use sessions**: No session resumption
4. **No per-IP rate limiting**: Only global connection limit
## Implementation Status
### Implemented ✅
- Noise XKpsk3 handshake
- KCP reliability layer
- Replay protection (1024-packet window with SIMD)
- PSK derivation (ECDH + Blake3)
- dVPN and Mixnet registration modes
- E-cash credential verification
- WireGuard peer management
- Prometheus metrics
- DoS protection
### Pending ⏳
- End-to-end integration tests
- Performance benchmarks
- External security audit
- Client implementation
- Gateway probe support
- Per-IP rate limiting
## Documentation
- **[LP_PROTOCOL.md](./LP_PROTOCOL.md)**: Complete protocol specification
- **[LP_DEPLOYMENT.md](./LP_DEPLOYMENT.md)**: Deployment and operations guide
- **[LP_SECURITY.md](./LP_SECURITY.md)**: Security analysis and threat model
- **[CODEMAP.md](../CODEMAP.md)**: Repository structure
## Contributing
### Getting Started
1. Read [CODEMAP.md](../CODEMAP.md) for repository structure
2. Review [LP_PROTOCOL.md](./LP_PROTOCOL.md) for protocol details
3. Check [FUNCTION_LEXICON.md](../FUNCTION_LEXICON.md) for API reference
### Areas Needing Work
**High Priority:**
- Integration tests for end-to-end registration
- Performance benchmarks (latency, throughput, concurrent connections)
- Per-IP rate limiting
- Client-side implementation
**Medium Priority:**
- Gateway probe support
- Load testing framework
- Fuzzing for packet parsers
## License
Same as parent Nym repository.
## Support
- **GitHub Issues**: https://github.com/nymtech/nym/issues
- **Discord**: https://discord.gg/nym
---
**Protocol Version**: 1.0
**Status**: Draft (pending security audit and integration tests)
docs/LP_REGISTRATION_ARCHITECTURE.md
+1400
View File
File diff suppressed because it is too large Load Diff
docs/LP_REGISTRATION_SEQUENCES.md
+1441
View File
File diff suppressed because it is too large Load Diff
docs/LP_REGISTRATION_WALKTHROUGH.md
+261
View File
@@ -0,0 +1,261 @@
# LP Registration Protocol - Technical Walkthrough
**Branch**: `drazen/lp-reg`
**Status**: Implementation complete, testing in progress
**Audience**: Engineering team, technical demo
---
## Executive Summary
LP Registration is a **fast, direct registration protocol** that allows clients to connect to Nym gateways without traversing the mixnet. It's designed primarily for dVPN use cases where users need quick WireGuard peer setup with sub-second latency.
### Key Characteristics
| Aspect | LP Registration | Traditional Mixnet Registration |
|--------|----------------|--------------------------------|
| **Latency** | Sub-second (100ms-1s) | Multi-second (3-10s) |
| **Transport** | Direct TCP (port 41264) | Through mixnet layers |
| **Reliability** | Guaranteed delivery | Probabilistic delivery |
| **Anonymity** | Client IP visible to gateway | Network-level anonymity |
| **Use Case** | dVPN, low-latency services | Privacy-critical applications |
| **Security** | Noise XKpsk3 + ChaCha20-Poly1305 | Sphinx packet encryption |
### Protocol Stack
```
┌─────────────────────────────────────────────────────────────┐
│ Application Layer │
│ WireGuard Peer Registration (dVPN) / Mixnet Client. │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ LP Registration Layer │
│ LpRegistrationRequest / LpRegistrationResponse │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Noise XKpsk3 Protocol Layer │
│ ChaCha20-Poly1305 Encryption + Authentication │
│ Replay Protection (1024-pkt window) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Transport Layer │
│ TCP (length-prefixed packet framing) │
└─────────────────────────────────────────────────────────────┘
```
---
## Architecture Overview
### High-Level Component Diagram
```
┌──────────────────────────────────────────────────────────────────────┐
│ CLIENT SIDE │
├──────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ nym-registration-client (Client Library) │ │
│ │ nym-registration-client/src/lp_client/client.rs:39-62 │ │
│ │ │ │
│ │ • LpRegistrationClient │ │
│ │ • TCP connection management │ │
│ │ • Packet serialization/framing │ │
│ │ • Integration with BandwidthController │ │
│ └────────────────────┬────────────────────────────────────────┘ │
│ │ │
│ ┌────────────────────┴─────────────────────────────────────────┐ │
│ │ common/nym-lp (Protocol Library) │ │
│ │ common/nym-lp/src/ (multiple modules) │ │
│ │ │ │
│ │ • LpStateMachine (state_machine.rs:96-420) │ │
│ │ • Noise XKpsk3 (noise_protocol.rs:40-88) │ │
│ │ • PSK derivation (psk.rs:28-52) │ │
│ │ • ReplayValidator (replay/validator.rs:25-125) │ │
│ │ • Message types (message.rs, packet.rs) │ │
│ └──────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
│
│ TCP (port 41264)
│ Length-prefixed packets
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ GATEWAY SIDE │
├──────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ LpListener (TCP Accept Loop) │ │
│ │ gateway/src/node/lp_listener/mod.rs:226-270 │ │
│ │ │ │
│ │ • Binds to 0.0.0.0:41264 │ │
│ │ • Spawns LpConnectionHandler per connection │ │
│ │ • Metrics: active_lp_connections │ │
│ └────────────────────┬────────────────────────────────────────┘ │
│ │ │
│ ┌────────────────────▼─────────────────────────────────────────┐ │
│ │ LpConnectionHandler (Per-Connection) │ │
│ │ gateway/src/node/lp_listener/handler.rs:101-216 │ │
│ │ │ │
│ │ 1. Receive ClientHello & validate timestamp │ │
│ │ 2. Derive PSK from ECDH + salt │ │
│ │ 3. Perform Noise handshake │ │
│ │ 4. Receive encrypted registration request │ │
│ │ 5. Process registration (delegate to registration.rs) │ │
│ │ 6. Send encrypted response │ │
│ │ 7. Emit metrics & close │ │
│ └────────────────────┬─────────────────────────────────────────┘ │
│ │ │
│ ┌────────────────────▼─────────────────────────────────────────┐ │
│ │ Registration Processor (Business Logic) │ │
│ │ gateway/src/node/lp_listener/registration.rs:136-288 │ │
│ │ │ │
│ │ Mode: dVPN Mode: Mixnet │ │
│ │ ├─ register_wg_peer() ├─ (skip WireGuard) │ │
│ │ ├─ credential_verification() ├─ credential_verification() │ │
│ │ └─ return GatewayData └─ return bandwidth only │ │
│ └────────┬───────────────────────────────┬─────────────────────┘ │
│ │ │ │
│ ┌────────▼───────────────────┐ ┌───────▼─────────────────────┐ │
│ │ WireGuard Controller │ │ E-cash Verifier │ │
│ │ (PeerControlRequest) │ │ (EcashManager trait) │ │
│ │ │ │ │ │
│ │ • Add/Remove WG peers │ │ • Verify BLS signature │ │
│ │ • Manage peer lifecycle │ │ • Check nullifier spent │ │
│ │ • Monitor bandwidth usage │ │ • Allocate bandwidth │ │
│ └─────────────────────────────┘ └────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ GatewayStorage (Database) │ │
│ │ │ │
│ │ Tables: │ │
│ │ • wireguard_peers (public_key, client_id, ticket_type) │ │
│ │ • bandwidth (client_id, available) │ │
│ │ • spent_credentials (nullifier, expiry) │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
```
---
## Implementation Roadmap
### ✅ Completed Components
1. **Protocol Library** (`common/nym-lp/`)
- Noise XKpsk3 implementation
- PSK derivation (Blake3 KDF)
- Replay protection with SIMD optimization
- Message types and packet framing
2. **Gateway Listener** (`gateway/src/node/lp_listener/`)
- TCP accept loop with connection limits
- Per-connection handler with lifecycle management
- dVPN and Mixnet registration modes
- Comprehensive metrics
3. **Client Library** (`nym-registration-client/`)
- Connection management with timeouts
- Noise handshake as initiator
- E-cash credential integration
- Error handling and retries
4. **Testing Tools** (`nym-gateway-probe/`)
- LP-only test mode (`--only-lp-registration`)
- Mock e-cash mode (`--use-mock-ecash`)
- Detailed test results
## Detailed Documentation
### For Protocol Deep-Dive
📄 **[LP_REGISTRATION_SEQUENCES.md](./LP_REGISTRATION_SEQUENCES.md)**
- Complete sequence diagrams for all flows
- Happy path with byte-level message formats
- Error scenarios and recovery paths
- Noise handshake details
### For Architecture Understanding
📄 **[LP_REGISTRATION_ARCHITECTURE.md](./LP_REGISTRATION_ARCHITECTURE.md)**
- Component interaction diagrams
- Data flow through gateway modules
- Client-side architecture
- State transitions
---
## Code Navigation
### Key Entry Points
| Component | File Path | Description |
|-----------|-----------|-------------|
| **Gateway Listener** | `gateway/src/node/lp_listener/mod.rs:226` | `LpListener::run()` - main loop |
| **Connection Handler** | `gateway/src/node/lp_listener/handler.rs:101` | `LpConnectionHandler::handle()` - per-connection |
| **Registration Logic** | `gateway/src/node/lp_listener/registration.rs:136` | `process_registration()` - business logic |
| **Client Entry** | `nym-registration-client/src/lp_client/client.rs:39` | `LpRegistrationClient` struct |
| **Protocol Core** | `common/nym-lp/src/state_machine.rs:96` | `LpStateMachine` - Noise protocol |
| **Probe Test** | `nym-gateway-probe/src/lib.rs:861` | `lp_registration_probe()` - integration test |
---
## Metrics and Observability
### Prometheus Metrics
**Connection Metrics**:
- `lp_connections_total{result="success|error"}` - Counter
- `lp_active_lp_connections` - Gauge
- `lp_connection_duration_seconds` - Histogram (buckets: 0.01, 0.1, 1, 5, 10, 30)
**Handshake Metrics**:
- `lp_handshakes_success` - Counter
- `lp_handshakes_failed{reason="..."}` - Counter
- `lp_handshake_duration_seconds` - Histogram
**Registration Metrics**:
- `lp_registration_attempts_total` - Counter
- `lp_registration_success_total{mode="dvpn|mixnet"}` - Counter
- `lp_registration_failed_total{reason="..."}` - Counter
- `lp_registration_duration_seconds` - Histogram
**Bandwidth Metrics**:
- `lp_bandwidth_allocated_bytes_total` - Counter
- `lp_credential_verification_success` - Counter
- `lp_credential_verification_failed{reason="..."}` - Counter
## Performance Characteristics
### Latency Breakdown
```
Total Registration Time: ~221ms (typical)
├─ TCP Connect: 10-20ms
├─ Noise Handshake: 40-60ms (3 round-trips)
│ ├─ ClientHello send: <5ms
│ ├─ Msg 1 (-> e): <5ms
│ ├─ Msg 2 (<- e,ee,s,es): 20-30ms (crypto ops)
│ └─ Msg 3 (-> s,se,psk): 10-20ms
├─ Registration Request: 100-150ms
│ ├─ Request encrypt & send: <5ms
│ ├─ Gateway processing: 90-140ms
│ │ ├─ WireGuard peer setup: 20-40ms
│ │ ├─ Database operations: 30-50ms
│ │ ├─ E-cash verification: 40-60ms (or <1ms with mock)
│ │ └─ Response preparation: <5ms
│ └─ Response receive & decrypt: <5ms
└─ Connection cleanup: <5ms
```
### Resource Usage
- **Memory per session**: 144 bytes (state machine + replay window)
- **Max concurrent connections**: 10,000 (configurable)
- **CPU**: Minimal (ChaCha20 is efficient, SIMD optimizations)
- **Database**: 3-5 queries per registration (indexed lookups)
docs/LP_SECURITY.md
+729
View File
@@ -0,0 +1,729 @@
# LP (Lewes Protocol) Security Considerations
## Threat Model
### Attacker Capabilities
**Network Attacker (Dolev-Yao Model):**
- ✅ Can observe all network traffic
- ✅ Can inject, modify, drop, or replay packets
- ✅ Can perform active MITM attacks
- ✅ Cannot break cryptographic primitives (ChaCha20, Poly1305, X25519)
- ✅ Cannot forge digital signatures (BLS12-381)
**Gateway Compromise:**
- ✅ Attacker gains full access to gateway server
- ✅ Can read all gateway state (keys, credentials, database)
- ✅ Can impersonate gateway to clients
- ❌ Cannot decrypt past sessions (forward secrecy)
- ❌ Cannot impersonate clients without their keys
**Client Compromise:**
- ✅ Attacker gains access to client device
- ✅ Can read client LP private key
- ✅ Can impersonate client to gateways
- ❌ Cannot decrypt other clients' sessions
### Security Goals
**Confidentiality:**
- Registration requests encrypted end-to-end
- E-cash credentials protected from eavesdropping
- WireGuard keys transmitted securely
**Integrity:**
- All messages authenticated with Poly1305 MAC
- Tampering detected and rejected
- Replay attacks prevented
**Authentication:**
- Mutual authentication via Noise XKpsk3
- Gateway proves possession of LP private key
- Client proves possession of LP private key + PSK
**Forward Secrecy:**
- Compromise of long-term keys doesn't reveal past sessions
- Ephemeral keys provide PFS
- Session keys destroyed after use
**Non-Goals:**
- **Network anonymity**: LP reveals client IP to gateway (use mixnet for anonymity)
- **Traffic analysis resistance**: Packet timing visible to network observer
- **Deniability**: Parties can prove who they communicated with
## Cryptographic Design
### Noise Protocol XKpsk3
**Pattern:**
```
XKpsk3:
<- s
...
-> e
<- e, ee, s, es
-> s, se, psk
```
**Security Properties:**
| Property | Provided | Rationale |
|----------|----------|-----------|
| Confidentiality (forward) | ✅ Strong | Ephemeral keys + PSK |
| Confidentiality (backward) | ✅ Weak | PSK compromise affects future |
| Authentication (initiator) | ✅ Strong | Static key + PSK |
| Authentication (responder) | ✅ Strong | Static key known upfront |
| Identity hiding (initiator) | ✅ Yes | Static key encrypted |
| Identity hiding (responder) | ❌ No | Static key in handshake msg 2 |
**Why XKpsk3:**
1. **Known responder identity**: Client knows gateway's LP public key from descriptor
2. **Mutual authentication**: Both sides prove identity
3. **PSK binding**: Links session to out-of-band PSK (prevents MITM with compromised static key alone)
4. **Forward secrecy**: Ephemeral keys provide PFS even if static keys leaked
**Alternative patterns considered:**
- **IKpsk2**: No forward secrecy (rejected)
- **XXpsk3**: More round trips, unknown identities (not needed)
- **NKpsk0**: No client authentication (rejected)
### PSK Derivation Security
**Formula:**
```
shared_secret = X25519(client_lp_private, gateway_lp_public)
psk = Blake3_derive_key("nym-lp-psk-v1", shared_secret, salt)
```
**Security Analysis:**
1. **ECDH Security**: Based on Curve25519 hardness (128-bit security)
- Resistant to quantum attacks up to Grover's algorithm (64-bit post-quantum)
- Well-studied, no known vulnerabilities
2. **Blake3 KDF Security**:
- Output indistinguishable from random (PRF security)
- Domain separation via context string prevents cross-protocol attacks
- Collision resistance: 128 bits (birthday bound on 256-bit hash)
3. **Salt Freshness**:
- Timestamp component prevents long-term PSK reuse
- Nonce component provides per-session uniqueness
- Both transmitted in ClientHello (integrity protected by timestamp validation + Noise handshake)
**Attack Scenarios:**
| Attack | Feasibility | Mitigation |
|--------|-------------|------------|
| Brute force PSK | ❌ Infeasible | 2^128 operations (Curve25519 DL) |
| Quantum attack on ECDH | ⚠️ Future threat | Shor's algorithm breaks X25519 in polynomial time |
| Salt replay | ❌ Prevented | Timestamp validation (30s window) |
| Cross-protocol PSK reuse | ❌ Prevented | Domain separation ("nym-lp-psk-v1") |
**Quantum Resistance:**
LP is **not quantum-resistant** due to X25519 use. Future upgrade path:
```rust
// Hybrid PQ-KEM (future)
let classical_secret = X25519(client_priv, gateway_pub);
let pq_secret = Kyber768::encaps(gateway_pq_pub);
let psk = Blake3_derive_key(
"nym-lp-psk-v2-pq",
classical_secret || pq_secret,
salt
);
```
### Replay Protection Analysis
**Algorithm: Sliding Window with Bitmap**
```rust
Window size: 1024 packets
Bitmap: [u64; 16] = 1024 bits
For counter C:
- Accept if C >= next (new packet)
- Reject if C + 1024 < next (too old)
- Reject if bitmap[C % 1024] == 1 (duplicate)
- Otherwise accept and mark
```
**Security Properties:**
1. **Replay Window**: 1024 packets
- Sufficient for expected reordering in TCP+KCP
- Small enough to limit replay attack surface
2. **Memory Efficiency**: 128 bytes bitmap
- Tracks 1024 unique counters
- O(1) lookup and insertion
3. **Overflow Handling**: Wraps at u64::MAX
- Properly handles counter wraparound
- Unlikely to occur (2^64 packets = trillions)
**Attack Scenarios:**
| Attack | Feasibility | Mitigation |
|--------|-------------|------------|
| Replay within window | ❌ Prevented | Bitmap tracking |
| Replay outside window | ❌ Prevented | Window boundary check |
| Counter overflow | ⚠️ Theoretical | Wraparound handling + 2^64 limit |
| Timing attack | ❌ Mitigated | Branchless execution |
**Timing Attack Resistance:**
```rust
// Constant-time check (branchless)
pub fn will_accept_branchless(&self, counter: u64) -> ReplayResult<()> {
let is_growing = counter >= self.next;
let too_far_back = /* calculated */;
let duplicate = self.check_bit_branchless(counter);
// Single branch at end (constant-time up to this point)
let result = if is_growing { Ok(()) }
else if too_far_back { Err(OutOfWindow) }
else if duplicate { Err(Duplicate) }
else { Ok(()) };
result.unwrap()
}
```
**SIMD Optimizations:**
- AVX2, SSE2, NEON: SIMD clears are constant-time
- Scalar fallback: Also constant-time (no data-dependent branches)
- No timing channels revealed through replay check
## Denial of Service (DoS) Protection
### Connection-Level DoS
**Attack:** Flood gateway with TCP connections
**Mitigations:**
1. **Max connections limit** (default: 10,000):
```rust
if active_connections >= max_connections {
return; // Drop new connection
}
```
- Prevents memory exhaustion (~5 KB per connection)
- Configurable based on gateway capacity
2. **TCP SYN cookies** (kernel-level):
```bash
sysctl -w net.ipv4.tcp_syncookies=1
```
- Prevents SYN flood attacks
- No state allocated until 3-way handshake completes
3. **Connection rate limiting** (iptables):
```bash
iptables -A INPUT -p tcp --dport 41264 -m state --state NEW \
-m recent --update --seconds 60 --hitcount 100 -j DROP
```
- Limits new connections per IP
- 100 connections/minute threshold
**Residual Risk:**
- ⚠️ **No per-IP limit in application**: Current implementation only has global limit
- **Recommendation**: Add per-IP tracking:
```rust
let connections_from_ip = ip_tracker.get(remote_addr.ip());
if connections_from_ip >= per_ip_limit {
return; // Reject
}
```
### Handshake-Level DoS
**Attack:** Start handshakes but never complete them
**Mitigations:**
1. **Handshake timeout**: Noise state machine times out
- Implementation: Tokio task timeout (implicit)
- Recommended: Explicit 15-second timeout
2. **State cleanup**: Connection dropped if handshake fails
```rust
if handshake_fails {
drop(connection); // Frees memory immediately
}
```
3. **No resource allocation before handshake**:
- Replay validator created only after handshake
- Minimal memory usage during handshake (~200 bytes)
**Attack Scenarios:**
| Attack | Resource Consumed | Mitigation |
|--------|-------------------|------------|
| Half-open connections | TCP state (~4 KB) | SYN cookies |
| Incomplete handshakes | Noise state (~200 B) | Timeout + cleanup |
| Slow clients | Connection slot | Timeout + max connections |
### Timestamp-Based DoS
**Attack:** Replay old ClientHello messages
**Mitigation:**
```rust
let timestamp_age = now - client_hello.timestamp;
if timestamp_age > 30_seconds {
return Err(TimestampTooOld);
}
if timestamp_age < -30_seconds {
return Err(TimestampFromFuture);
}
```
**Properties:**
- 30-second window limits replay attack surface
- Clock skew tolerance: ±30 seconds (reasonable for NTP)
- Metrics track rejections: `lp_timestamp_validation_rejected`
**Residual Risk:**
- ⚠️ 30-second window allows replay of ClientHello within window
- **Mitigation**: Replay protection on post-handshake messages
### Credential Verification DoS
**Attack:** Flood gateway with fake credentials
**Mitigations:**
1. **Fast rejection path**:
```rust
// Check signature before database lookup
if !verify_bls_signature(&credential) {
return Err(InvalidSignature); // Fast path
}
// Only then check database
```
2. **Database indexing**:
```sql
CREATE INDEX idx_nullifiers ON spent_credentials(nullifier);
```
- O(log n) nullifier lookup instead of O(n)
3. **Rate limiting** (future):
- Limit credential verification attempts per IP
- Exponential backoff for repeated failures
**Performance Impact:**
- BLS signature verification: ~5ms per credential
- Database lookup: ~1ms (with index)
- Total: ~6ms per invalid credential
**Attack Cost:**
- Attacker must generate BLS signatures (computationally expensive)
- Invalid signatures rejected before database query
- Real cost is in valid-looking but fake credentials (still requires crypto)
## Threat Scenarios
### Scenario 1: Passive Eavesdropper
**Attacker:** Network observer (ISP, hostile network)
**Capabilities:**
- Observe all LP traffic (including ClientHello)
- Analyze packet sizes, timing, patterns
**Protections:**
- ✅ ClientHello metadata visible but not sensitive (timestamp, nonce)
- ✅ Noise handshake encrypts all subsequent messages
- ✅ Registration request fully encrypted (credential not visible)
- ✅ ChaCha20-Poly1305 provides IND-CCA2 security
**Leakage:**
- ⚠️ Client IP address visible (inherent to TCP)
- ⚠️ Packet timing reveals registration events
- ⚠️ Connection to known gateway suggests Nym usage
**Recommendation:** Use LP for fast registration, mixnet for anonymity-critical operations.
### Scenario 2: Active MITM
**Attacker:** On-path adversary (malicious router, hostile WiFi)
**Capabilities:**
- Intercept, modify, drop, inject packets
- Cannot break cryptography
**Protections:**
- ✅ Noise XKpsk3 mutual authentication prevents impersonation
- ✅ Client verifies gateway's LP static public key
- ✅ Gateway verifies client via PSK derivation
- ✅ Any packet modification detected via Poly1305 MAC
**Attack Attempts:**
1. **Impersonate Gateway**:
- Attacker doesn't have gateway's LP private key
- Cannot complete handshake (Noise fails at `es` mix)
- Client rejects connection
2. **Impersonate Client**:
- Attacker doesn't know client's LP private key
- Cannot derive correct PSK
- Noise fails at `psk` mix in message 3
- Gateway rejects connection
3. **Modify Messages**:
- Poly1305 MAC fails
- Noise decryption fails
- Connection aborted
**Residual Risk:**
- ⚠️ DoS possible (drop packets, connection killed)
- ✅ Cannot learn registration data or credentials
### Scenario 3: Gateway Compromise
**Attacker:** Full access to gateway server
**Capabilities:**
- Read all gateway state (keys, database, memory)
- Modify gateway behavior
- Impersonate gateway to clients
**Impact:**
1. **Current Sessions**: Compromised
- Attacker can decrypt ongoing registration requests
- Can steal credentials from current sessions
2. **Past Sessions**: Protected (forward secrecy)
- Ephemeral keys already destroyed
- Cannot decrypt recorded traffic
3. **Future Sessions**: Compromised until key rotation
- Attacker can impersonate gateway
- Can steal credentials from new registrations
**Mitigations:**
1. **Key Rotation**:
```bash
# Generate new LP keypair
./nym-node generate-lp-keypair
# Update gateway descriptor (automatic on restart)
```
- Invalidates attacker's stolen keys
- Clients fetch new public key from descriptor
2. **Monitoring**:
- Detect anomalous credential verification patterns
- Alert on unusual database access
- Monitor for key file modifications
3. **Defense in Depth**:
- E-cash credentials have limited value (time-bound, nullifiers)
- WireGuard keys rotatable by client
- No long-term sensitive data stored
**Credential Reuse Prevention:**
- Nullifier stored in database
- Nullifier = Hash(credential_data)
- Even with database access, attacker cannot create new credentials
- Can only steal credentials submitted during compromise window
### Scenario 4: Replay Attack
**Attacker:** Records past LP sessions, replays later
**Attack Attempts:**
1. **Replay ClientHello**:
- Timestamp validation rejects messages > 30s old
- Nonce in salt changes per session
- Cannot reuse old ClientHello
2. **Replay Handshake Messages**:
- Noise uses ephemeral keys (fresh each session)
- Replaying old handshake messages fails (wrong ephemeral key)
- Handshake fails, no session established
3. **Replay Post-Handshake Packets**:
- Counter-based replay protection
- Bitmap tracks last 1024 packets
- Duplicate counters rejected
- Cannot replay old encrypted messages
4. **Replay Entire Session**:
- Different ephemeral keys each time
- Cannot replay connection to gateway
- Even if gateway state reset, timestamp rejects old ClientHello
**Success Probability:** Negligible (< 2^-128)
### Scenario 5: Quantum Adversary (Future)
**Attacker:** Quantum computer with Shor's algorithm
**Capabilities:**
- Break X25519 ECDH in polynomial time
- Recover LP static private keys from public keys
- Does NOT break symmetric crypto (ChaCha20, Blake3)
**Impact:**
1. **Recorded Traffic**: Vulnerable
- Attacker records all LP traffic now
- Breaks X25519 later with quantum computer
- Recovers PSKs from recorded ClientHellos
- Decrypts recorded sessions
2. **Real-Time Interception**: Full compromise
- Can impersonate gateway (knows private key)
- Can decrypt all traffic
- Complete MITM attack
**Mitigations (Future):**
1. **Hybrid PQ-KEM**:
```rust
// Use both classical and post-quantum KEM
let classical = X25519(client_priv, gateway_pub);
let pq = Kyber768::encaps(gateway_pq_pub);
let psk = Blake3(classical || pq, salt);
```
2. **Post-Quantum Noise**:
- Noise specification supports PQ KEMs
- Can upgrade to Kyber, NTRU, or SIKE
- Requires protocol version 2
**Timeline:**
- Quantum threat: ~10-20 years away
- PQ upgrade: Can be deployed when threat becomes real
- Backward compatibility: Support both classical and PQ
## Security Recommendations
### For Gateway Operators
**High Priority:**
1. **Enable all DoS protections**:
```toml
[lp]
max_connections = 10000 # Adjust based on capacity
timestamp_tolerance_secs = 30 # Don't increase unnecessarily
```
2. **Secure key storage**:
```bash
chmod 600 ~/.nym/gateways/<id>/keys/lp_x25519.pem
# Encrypt disk if possible
```
3. **Monitor metrics**:
- Alert on high `lp_handshakes_failed`
- Alert on unusual `lp_timestamp_validation_rejected`
- Track `lp_credential_verification_failed` patterns
4. **Keep database secure**:
- Regular backups
- Index on `nullifier` column
- Periodic cleanup of old nullifiers
**Medium Priority:**
5. **Implement per-IP rate limiting** (future):
```rust
const MAX_CONNECTIONS_PER_IP: usize = 10;
```
6. **Regular key rotation**:
- Rotate LP keypair every 6-12 months
- Coordinate with network updates
7. **Firewall hardening**:
```bash
# Only allow LP port
ufw default deny incoming
ufw allow 41264/tcp
```
### For Client Developers
**High Priority:**
1. **Verify gateway LP public key**:
```rust
// Fetch from trusted source (network descriptor)
let gateway_lp_pubkey = fetch_gateway_descriptor(gateway_id)
.await?
.lp_public_key;
// Pin for future connections
save_pinned_key(gateway_id, gateway_lp_pubkey);
```
2. **Handle errors securely**:
```rust
match registration_result {
Err(LpError::Replay(_)) => {
// DO NOT retry immediately (might be replay attack)
log::warn!("Replay detected, waiting before retry");
tokio::time::sleep(Duration::from_secs(60)).await;
}
Err(e) => {
// Other errors safe to retry
}
}
```
3. **Use fresh credentials**:
- Don't reuse credentials across registrations
- Check credential expiry before attempting registration
**Medium Priority:**
4. **Implement connection timeout**:
```rust
tokio::time::timeout(
Duration::from_secs(30),
registration_client.register_lp(...)
).await?
```
5. **Secure local key storage**:
- Use OS keychain for LP private keys
- Don't log or expose keys
### For Network Operators
**High Priority:**
1. **Deploy monitoring infrastructure**:
- Prometheus + Grafana for metrics
- Alerting on security-relevant metrics
- Correlation of events across gateways
2. **Incident response plan**:
- Procedure for gateway compromise
- Key rotation workflow
- Client notification mechanism
3. **Regular security audits**:
- External audit of Noise implementation
- Penetration testing of LP endpoints
- Review of credential verification logic
**Medium Priority:**
4. **Threat intelligence**:
- Monitor for known attacks on Noise protocol
- Track quantum computing advances
- Plan PQ migration timeline
## Compliance Considerations
### Data Protection (GDPR, etc.)
**Personal Data Collected:**
- Client IP address (connection metadata)
- Credential nullifiers (pseudonymous identifiers)
- Timestamps (connection events)
**Data Retention:**
- IP addresses: Not stored beyond connection duration
- Nullifiers: Stored until credential expiry + grace period
- Logs: Configurable retention (default: 7 days)
**Privacy Protections:**
- Nullifiers pseudonymous (not linkable to real identity)
- No PII collected or stored
- Credentials use blind signatures (gateway doesn't learn identity)
### Security Compliance
**SOC 2 / ISO 27001 Requirements:**
1. **Access Control**:
- LP keys protected (file permissions)
- Database access restricted
- Principle of least privilege
2. **Encryption in Transit**:
- Noise protocol provides end-to-end encryption
- TLS for metrics endpoint (if exposed)
3. **Logging and Monitoring**:
- Security events logged
- Metrics for anomaly detection
- Audit trail for credential usage
4. **Incident Response**:
- Key rotation procedure
- Backup and recovery
- Communication plan
## Audit Checklist
Before production deployment:
- [ ] Noise implementation reviewed by cryptographer
- [ ] Replay protection tested with edge cases (overflow, concurrency)
- [ ] DoS limits tested (connection flood, credential spam)
- [ ] Timing attack resistance verified (replay check, credential verification)
- [ ] Key storage secured (file permissions, encryption at rest)
- [ ] Monitoring and alerting configured
- [ ] Incident response plan documented
- [ ] Penetration testing performed
- [ ] Code review completed
- [ ] Dependencies audited (cargo-audit, cargo-deny)
## References
### Security Specifications
- **Noise Protocol Framework**: https://noiseprotocol.org/
- **XKpsk3 Analysis**: https://noiseexplorer.com/patterns/XKpsk3/
- **Curve25519**: https://cr.yp.to/ecdh.html
- **ChaCha20-Poly1305**: RFC 8439
- **Blake3**: https://github.com/BLAKE3-team/BLAKE3-specs
### Security Audits
- [ ] Noise implementation audit (pending)
- [ ] Cryptographic review (pending)
- [ ] Penetration test report (pending)
### Known Vulnerabilities
*None currently identified. This section will be updated as issues are discovered.*
## Responsible Disclosure
If you discover a security vulnerability in LP:
1. **DO NOT** publish vulnerability details publicly
2. Email security@nymtech.net with:
- Description of vulnerability
- Steps to reproduce
- Potential impact
- Suggested mitigation (if any)
3. Allow 90 days for patch development before public disclosure
4. Coordinate disclosure timeline with Nym team
**Bug Bounty**: Check https://nymtech.net/security for current bounty program.