Domain fronting integration (#5974)

* feat: unify HTTP client creation and enable domain fronting

Enhanced the base nym_http_api_client to reduce fragmentation and enable domain fronting:

- Added SerializationFormat enum for explicit JSON/bincode choice (no auto-detection)
- Added from_network() method to create clients from NymNetworkDetails with domain fronting
- Added with_bincode() builder method for explicit serialization configuration
- Set Accept header based on serialization preference
- Added deprecation paths for NymApiClient wrapper and nym_api::Client re-export
- Enabled domain fronting support via network defaults feature

This is part of a broader effort to consolidate HTTP client implementations across the codebase,
reducing ~500 lines of wrapper code and providing automatic domain fronting for censorship resistance.

* feat: migrate NymApiClient usage to unified HTTP client

- Wire up domain fronting configuration in NymNetworkDetails
- Implement NymApiClientExt trait for base nym_http_api_client::Client
- Migrate direct NymApiClient usage in multiple components:
  - nym-network-monitor
  - verloc measurements
  - connection tester
  - coconut/ecash client
  - validator rewarder
- Add Copy derive to ApiUrlConst to enable iteration
- Update error handling and Display implementations

This enables automatic domain fronting for all Nym API calls via the configured CDN front hosts.

* fix: resolve all compilation errors after NymApiClient migration

- Add missing nym-http-api-client dependencies to multiple crates
- Add NymApiClientExt trait imports where needed
- Fix type mismatches from NymApiClient to unified Client
- Add error conversions for NymAPIError in various error enums
- Implement missing trait methods (get_current_rewarded_set, get_all_basic_nodes_with_metadata, get_all_described_nodes)
- Fix type conversions for RewardedSetResponse in network monitor
- Update all API client instantiation to use new unified HTTP client

* feat: complete migration to unified HTTP client and fix all compilation errors

- Added missing NymApiClientExt trait methods (get_all_expanded_nodes, change_base_urls)
- Fixed all compilation errors across the workspace
- Updated nym-node to use unified client instead of deprecated NymApiClient
- Fixed type conversions for RewardedSetResponse → EpochRewardedSet
- Added nym-http-api-client dependency where needed
- Updated all examples and documentation to use new client API

* fix: provide all API URLs for automatic failover in endpoint rotation

Previously, when rotating API endpoints, only a single URL was provided to the
HTTP client, defeating the purpose of having multiple URLs for resilience.

Changes:
- NymApiTopologyProvider now provides all URLs in rotated order when switching endpoints
- NymApisClient similarly provides all URLs starting from the working endpoint
- Added clarifying comments for broadcast/exhaustive query methods where single URLs are intentionally used
- This enables the HTTP client's built-in failover mechanism while maintaining endpoint rotation behavior

The fix ensures that if the primary endpoint fails, the client can automatically
failover to alternative endpoints without manual intervention, improving overall
network resilience.

* Update common/client-core/src/client/base_client/mod.rs

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>

* Remove error generics, address PR comments

* Explicit warning on missing fronting configuration

* Assorted CI fixes

* Registry proc-macro

* Rename macro

* Syn workspace version

* Where do we need to put inventory

* Ergonomics and call sites, incept the builder

* fix: Address critical issues in client configuration registry implementation

- Fixed HeaderMapInit parsing bug that would cause compilation errors
- Added comprehensive documentation with usage examples and DSL reference
- Improved error handling with better error messages for invalid headers
- Added test coverage for both macro and registry functionality
- Added debug inspection capabilities for registered configurations
- Fixed module name conflicts in tests by using separate modules

All tests now passing:
- 7 macro tests validating DSL parsing and code generation
- 4 registry tests verifying configuration collection and application

* Use default value for the ports until api is deployed

* Feature/improved http error (#6025)

* use display impl for urls

* feat: attempt to add more details to reqwest errors

* temporarily restored GenericRequestFailure variant

* another restoration

* cleanup

* Some debug tooling, and default timeout fix

* Fix user-agent override

* Fix various wasm things

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Bogdan-Ștefan Neacşu <bogdan@nymtech.net>
This commit is contained in:
Drazen Urch
2025-09-15 14:32:15 +02:00
committed by GitHub
parent 0ee387d983
commit 90e9e3cff8
101 changed files with 2248 additions and 570 deletions
+9 -4
View File
@@ -27,8 +27,13 @@ pub enum NymNodeHttpError {
},
#[error("error building or using HTTP client: {source}")]
ClientError {
#[from]
source: HttpClientError,
},
ClientError { source: Box<HttpClientError> },
}
impl From<HttpClientError> for NymNodeHttpError {
fn from(source: HttpClientError) -> Self {
NymNodeHttpError::ClientError {
source: Box::new(source),
}
}
}
+25 -16
View File
@@ -10,7 +10,6 @@ use nym_task::ShutdownToken;
use nym_validator_client::client::NymApiClientExt;
use nym_validator_client::models::{KeyRotationInfoResponse, NodeRefreshBody};
use nym_validator_client::nym_api::error::NymAPIError;
use nym_validator_client::NymApiClient;
use rand::prelude::SliceRandom;
use rand::thread_rng;
use std::sync::Arc;
@@ -27,7 +26,7 @@ pub struct NymApisClient {
struct InnerClient {
// NOTE: this was implemented before the internal http client supported multiple URLs
active_client: NymApiClient,
active_client: Client,
available_urls: Vec<Url>,
shutdown_token: ShutdownToken,
currently_used_api: usize,
@@ -45,7 +44,7 @@ impl NymApisClient {
let mut urls = nym_apis.to_vec();
urls.shuffle(&mut thread_rng());
let active_client = nym_http_api_client::Client::builder(urls[0].clone())?
let active_client = Client::builder(urls[0].clone())?
.no_hickory_dns()
.with_user_agent(NymNode::user_agent())
.with_timeout(Duration::from_secs(5))
@@ -53,7 +52,7 @@ impl NymApisClient {
Ok(NymApisClient {
inner: Arc::new(RwLock::new(InnerClient {
active_client: NymApiClient::from(active_client),
active_client: active_client.clone(),
available_urls: urls,
shutdown_token,
currently_used_api: 0,
@@ -88,9 +87,19 @@ impl NymApisClient {
if guard.currently_used_api != last_working_endpoint {
drop(guard);
let mut guard = self.inner.write().await;
let next_url = guard.available_urls[last_working_endpoint].clone();
guard.currently_used_api = last_working_endpoint;
guard.active_client.change_nym_api(next_url);
// Provide all URLs starting from the working endpoint for automatic failover
let rotated_urls: Vec<_> = guard
.available_urls
.iter()
.cycle()
.skip(last_working_endpoint)
.take(guard.available_urls.len())
.map(|u| u.clone().into())
.collect();
guard.active_client.change_base_urls(rotated_urls);
}
Ok(res)
@@ -123,10 +132,10 @@ impl InnerClient {
{
let broadcast_fut =
stream::iter(self.available_urls.clone()).for_each_concurrent(None, |url| {
let nym_api = self
.active_client
.nym_api
.clone_with_new_url(url.clone().into());
let mut nym_api = self.active_client.clone();
// For broadcast, we intentionally set a single URL per client
// to ensure each endpoint receives the request
nym_api.change_base_urls(vec![url.clone().into()]);
let req_fut = req(nym_api, request_body);
async move {
if let Err(err) = req_fut.await {
@@ -172,10 +181,10 @@ impl InnerClient {
.skip(last_working)
.chain(self.available_urls.iter().enumerate().take(last_working))
{
let nym_api = self
.active_client
.nym_api
.clone_with_new_url(url.clone().into());
let mut nym_api = self.active_client.clone();
// For exhaustive query, we test each endpoint individually in sequence
// to find a working one - so single URL is correct here
nym_api.change_base_urls(vec![url.clone().into()]);
let timeout_fut = sleep(timeout_duration);
let query_fut = req(nym_api);
@@ -216,8 +225,8 @@ impl InnerClient {
}
}
impl AsRef<NymApiClient> for InnerClient {
fn as_ref(&self) -> &NymApiClient {
impl AsRef<Client> for InnerClient {
fn as_ref(&self) -> &Client {
&self.active_client
}
}
+7 -7
View File
@@ -7,6 +7,7 @@ use crate::node::routing_filter::network_filter::NetworkRoutingFilter;
use async_trait::async_trait;
use nym_crypto::asymmetric::ed25519;
use nym_gateway::node::UserAgent;
use nym_http_api_client::Client;
use nym_node_metrics::prometheus_wrapper::{PrometheusMetric, PROMETHEUS_METRICS};
use nym_noise::config::NoiseNetworkView;
use nym_task::ShutdownToken;
@@ -20,7 +21,7 @@ use nym_validator_client::nym_api::NymApiClientExt;
use nym_validator_client::nym_nodes::{
NodesByAddressesResponse, SemiSkimmedNode, SemiSkimmedNodesWithMetadata,
};
use nym_validator_client::{NymApiClient, ValidatorClientError};
use nym_validator_client::ValidatorClientError;
use std::collections::{HashMap, HashSet};
use std::net::{IpAddr, SocketAddr};
use std::ops::Deref;
@@ -35,7 +36,7 @@ use url::Url;
const LOCAL_NODE_ID: NodeId = 1234567890;
struct NodesQuerier {
client: NymApiClient,
client: Client,
nym_api_urls: Vec<Url>,
currently_used_api: usize,
}
@@ -49,7 +50,7 @@ impl NodesQuerier {
self.currently_used_api = (self.currently_used_api + 1) % self.nym_api_urls.len();
self.client
.change_nym_api(self.nym_api_urls[self.currently_used_api].clone())
.change_base_urls(self.nym_api_urls.iter().map(|u| u.clone().into()).collect())
}
async fn rewarded_set(&mut self) -> Result<EpochRewardedSet, ValidatorClientError> {
@@ -62,7 +63,7 @@ impl NodesQuerier {
if res.is_err() {
self.use_next_nym_api()
}
res
Ok(res?.into())
}
async fn current_nymnodes(
@@ -77,7 +78,7 @@ impl NodesQuerier {
if res.is_err() {
self.use_next_nym_api()
}
res
Ok(res?)
}
async fn query_for_info(
@@ -86,7 +87,6 @@ impl NodesQuerier {
) -> Result<NodesByAddressesResponse, ValidatorClientError> {
let res = self
.client
.nym_api
.nodes_by_addresses(ips)
.await
.inspect_err(|err| error!("failed to obtain node information: {err}"));
@@ -228,7 +228,7 @@ impl NetworkRefresher {
let mut this = NetworkRefresher {
querier: NodesQuerier {
client: NymApiClient::from(nym_api),
client: nym_api,
nym_api_urls,
currently_used_api: 0,
},