Compare commits

..

28 Commits

Author SHA1 Message Date
Bogdan-Ștefan Neacşu b5213f3908 Hide coconut runtime flags (#2990) 2023-02-08 13:00:58 +02:00
Bogdan-Ștefan Neacşu 590a4644e2 Skip errors on blind sign within threshold (#2976) 2023-02-08 11:54:12 +02:00
Bogdan-Ștefan Neacşu f8345e03c6 Feature/dkg reshare (#2936)
* Add resharing parameter

* Fix equality of dealers and members

* Contract resharing handling

* Dealer verification unit test

* Dealing commit unit test

* Epoch state unit tests

* Fix clippy

* Fmt

* Query initial dealer data

* Resharing nym-api changes

* Implement the mockups for nym-api dkg tests

* Dealing test

* Vk unit test

* Fix skipping vk submission

* Fix clippy

* Missing dealing for noninitial resharing dealer

* Check master vk holds after resharing on nym-apis

* Update changelog

* Fix clippy
2023-02-07 12:56:08 +02:00
Jon Häggblad 6675ea0249 Remove all the .DS_Store files and add to gitignore (#2974) 2023-02-07 11:45:40 +01:00
Jędrzej Stuczyński 56f48e6e41 Feature/service provider interface (#2934)
* added additional workspace-wide dependencies

* Added conditional serialization on 'BinaryBuildInformationOwned'

* initial framework for service provider messages

* updated request/response tags to account for existing variants of Socks5Message

* handling legacy deserialization

* another serialization revamp to account for legacy version

* legacy client working with versioned network requester

* socks5 client deserializing responses into updated structures

* using new structures for sending in socks5 client

* SendRequest wrapper for Request::Send variant

* created named fields for all variants of 'ControllerCommand'

* Versioning socks5 requests + moving to proper Socks5Request struct

* Updated backwards compatible Socks5Response

* unused imports

* poc ServiceProvider trait

* wip

* implemented 'ServiceProvider' trait on the Network Requester

* Socks5RequestError

* added properly serialized ErrorResponse to ControlResponse

* fixed version serialization + feature selection

* handling of version control requests

* improved SocksProxyError by providing more concrete variants and removing generic case

* got rid of ServiceProviderClient trait and wrote simple example showing control requests

* tests for serialisation backwards compatibility

* post-merge fixes due to method renaming

* enum boxing to make clippy happier

* making sure to not drop buffer channel when starting `DirectClient`

* Using nym-sdk in the example

* Replaced printing version to stdout with proper log call
2023-02-07 09:49:19 +00:00
Pierre Dommerc 1733aaa4cc NC Android - setup APK publish (#2967)
* ci(nc-android): init workflow apk build & release

* ci(nc-android): setup nc android project for APK publish

* docs(nc-android): update readme

* ci(nc-android): add note in the workflow
2023-02-06 10:56:39 +01:00
Mark Sinclair 78aa07360c Update build-and-upload-binaries-ci.yml 2023-02-03 17:10:13 +00:00
Mark Sinclair 4e52478c7a Update build-and-upload-binaries-ci.yml 2023-02-03 16:54:49 +00:00
Mark Sinclair 7c77665a37 Update build-and-upload-binaries-ci.yml 2023-02-03 16:36:44 +00:00
Mark Sinclair 72880b9764 Update build-and-upload-binaries-ci.yml 2023-02-03 16:14:29 +00:00
Mark Sinclair fd1c0ae62b Update build-and-upload-binaries-ci.yml 2023-02-03 12:03:13 +00:00
Mark Sinclair bd7399091b Update build-and-upload-binaries-ci.yml 2023-02-03 11:57:32 +00:00
Mark Sinclair 00fad44e2e Update build-and-upload-binaries-ci.yml 2023-02-03 11:46:07 +00:00
Mark Sinclair ea33c332ee GitHub Actions: add action to build and upload binaries to CI server 2023-02-03 11:35:03 +00:00
Bogdan-Ștefan Neacșu b39f8af8d0 Fix flaky dkg test 2023-02-03 12:29:21 +02:00
dependabot[bot] a400463b7e build(deps): bump ua-parser-js in /nym-wallet/webdriver (#2909)
Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.28 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](https://github.com/faisalman/ua-parser-js/compare/0.7.28...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 10:00:09 +00:00
dependabot[bot] 82928af64c build(deps): bump http-cache-semantics in /nym-wallet/webdriver (#2960)
Bumps [http-cache-semantics](https://github.com/kornelski/http-cache-semantics) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/kornelski/http-cache-semantics/releases)
- [Commits](https://github.com/kornelski/http-cache-semantics/commits)

---
updated-dependencies:
- dependency-name: http-cache-semantics
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 09:59:08 +00:00
dependabot[bot] c5891f546c build(deps): bump http-cache-semantics from 4.1.0 to 4.1.1 (#2961)
Bumps [http-cache-semantics](https://github.com/kornelski/http-cache-semantics) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/kornelski/http-cache-semantics/releases)
- [Commits](https://github.com/kornelski/http-cache-semantics/commits)

---
updated-dependencies:
- dependency-name: http-cache-semantics
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 09:58:34 +00:00
Tommy Verrall 580b677489 Binary checker, introduction 2023-02-03 08:46:40 -01:00
Jędrzej Stuczyński 2093789c5c Added an option to set custom 'host' for the native client (#2939)
* Added an option to set custom 'host' for the native client

* Changelog entry
2023-02-02 16:38:39 +00:00
Pierre Dommerc d09864b99f build(nc-android): prepare for apk release (#2943)
* chore(nc-android): prepare for production build

* refactor(nc-android): remove dead code

* feat(nc-android): update native color theme

* feat(nc-android): update native color theme

* build(nc-android): fix rfd version issue

* build(nc-android): fix dist dir no such file error

* fix(nc-android): post rebase changes
2023-02-02 15:18:58 +01:00
Bogdan-Ștefan Neacşu f6e8278592 Fix typo during merge back to develop (#2956) 2023-02-02 13:55:58 +02:00
cgi-bin/ b085a8a636 typo: electrum (#2954) 2023-02-02 09:34:08 +00:00
farbanas e6ce531aeb fix: formatting 2023-02-01 17:16:53 +01:00
farbanas 1a860eb3f5 fix: wrong parameter type for addresses in generator commands (should be AccountId instead of String) 2023-02-01 12:45:51 +01:00
Fran Arbanas 09cfd9ff05 Update CHANGELOG.md 2023-01-31 14:42:16 +00:00
Bogdan-Ștefan Neacșu 2c88ca137a Resolve merge conflict 2023-01-31 16:19:34 +02:00
farbanas ca3bfc859a merge conflicts 2023-01-31 15:09:35 +01:00
1825 changed files with 57609 additions and 79404 deletions
-16
View File
@@ -1,16 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
commit-message:
prefix: build
prefix-development: chore
include: scope
+7 -3
View File
@@ -39,16 +39,20 @@ jobs:
uses: actions/setup-node@v3
with:
node-version: 16
- name: Matrix - Node Install
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: security
NYM_PROJECT_NAME: "Daily security report"
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "security"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_AUDIT }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
@@ -34,7 +34,7 @@ on:
- 'tools/ts-rs-cli/**'
env:
NETWORK: mainnet
NETWORK: mainnet
jobs:
publish-nym:
@@ -46,7 +46,7 @@ jobs:
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Prepare build output directory
shell: bash
env:
@@ -69,7 +69,7 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --release --all
args: --workspace --release
- name: Install Rust stable
uses: actions-rs/toolchain@v1
@@ -79,9 +79,6 @@ jobs:
override: true
components: rustfmt, clippy
- name: Install wasm-opt
run: cargo install --version 0.112.0 wasm-opt
- name: Build release contracts
run: make wasm
@@ -98,15 +95,9 @@ jobs:
cp target/release/nym-network-requester $OUTPUT_DIR
cp target/release/nym-network-statistics $OUTPUT_DIR
cp target/release/nym-cli $OUTPUT_DIR
cp target/release/nym-credential-client $OUTPUT_DIR
cp target/release/explorer-api $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_bandwidth.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
- name: Deploy branch to CI www
continue-on-error: true
@@ -119,3 +110,4 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/
EXCLUDE: "/dist/, /node_modules/"
+6 -2
View File
@@ -43,10 +43,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ts-${{ env.GITHUB_REF_SLUG }}-example
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: ts-packages
NYM_PROJECT_NAME: "ts-packages"
@@ -54,6 +54,10 @@ jobs:
NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-ts-packages"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
-76
View File
@@ -1,76 +0,0 @@
name: CD dev-portal
on:
push:
branches: master
paths:
- 'documentation/dev-portal/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck
- name: Clean website
run: cd documentation/dev-portal && mdbook clean
- name: Build website
run: cd documentation/dev-portal && mdbook build
- name: Deploy branch master to dev
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_DEV }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET_DEVP }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Deploy branch master to prod
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_PROD }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET_DEVP }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: cd-dev
NYM_PROJECT_NAME: "Dev portal CD"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CD_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DEVP }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
-82
View File
@@ -1,82 +0,0 @@
name: CD docs
on:
push:
branches: master
paths:
- 'documentation/docs/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --release --all
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck && \
cargo install --vers "^0.5.0" mdbook-cmdrun
- name: Clean website
run: cd documentation/docs && mdbook clean
- name: Build website
run: cd documentation/docs && mdbook build
- name: Deploy branch master to dev
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_DEV }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Deploy branch master to prod
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_PROD }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: cd-docs
NYM_PROJECT_NAME: "Docs CD"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CD_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DOCS }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
@@ -1,61 +0,0 @@
name: check-merge-conflicts
# Check that the latest release branch merges into master and develop without
# any conflicts that git is not able to resolve
on:
workflow_dispatch:
schedule:
- cron: '5 6 * * *'
jobs:
get_release:
runs-on: ubuntu-latest
outputs:
output1: ${{ steps.step2.outputs.latest_release }}
steps:
- name: Check out repository code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set output variable to latest release branch
id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 1)" >> $GITHUB_OUTPUT
check-merge-release-into-master:
name: Check that the release branch merges into master
needs: get_release
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup git user
run: |
git config --global user.name "ci"
git config --global user.email "ci@localhost"
- name: Check merge release branch into master
run: |
./.github/workflows/support-files/git-merge-check.sh origin/master $branch1
env:
branch1: ${{needs.get_release.outputs.output1}}
check-merge-release-into-develop:
name: Check that the release branch merges into develop
needs: get_release
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup git user
run: |
git config --global user.name "ci"
git config --global user.email "ci@localhost"
- name: Check merge release branch into develop
run: |
./.github/workflows/support-files/git-merge-check.sh origin/develop $branch1
env:
branch1: ${{needs.get_release.outputs.output1}}
-56
View File
@@ -1,56 +0,0 @@
name: Run config checks on all binaries
on:
workflow_dispatch:
release:
types: [created]
push:
paths:
- 'clients/**'
- 'common/**'
- 'contracts/**'
- 'integrations/**'
- 'mixnode/**'
- 'sdk/rust/nym-sdk/**'
- 'service-providers/**'
pull_request:
paths:
- 'clients/**'
- 'common/**'
- 'gateway/**'
- 'integrations/**'
- 'mixnode/**'
- 'sdk/rust/nym-sdk/**'
- 'service-providers/**'
env:
NETWORK: mainnet
jobs:
publish-nym:
strategy:
fail-fast: false
matrix:
platform: [custom-runner-linux]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install jq vim libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools
continue-on-error: true
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Branch name
run: echo running on branch ${GITHUB_REF##*/}
- name: Run tests against binaries
run: ./build_and_run.sh ${{ github.head_ref || github.ref_name }}
working-directory: tests/
-66
View File
@@ -1,66 +0,0 @@
name: CI dev-portal
on:
push:
branches-ignore: master
paths:
- 'documentation/dev-portal/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" mdbook-theme \
&& cargo install --vers "^0.7.7" mdbook-linkcheck
- name: Clean website
run: cd documentation/dev-portal && mdbook clean
- name: Build website
run: cd documentation/dev-portal && mdbook build
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/dev-portal-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: ci-dev
NYM_PROJECT_NAME: "Dev portal CI"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CI_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "dev-portal-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DEVP }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
-72
View File
@@ -1,72 +0,0 @@
name: CI docs
on:
push:
branches-ignore: master
paths:
- 'documentation/docs/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --release --all
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck && \
cargo install --vers "^0.5.0" mdbook-cmdrun
- name: Clean website
run: cd documentation/docs && mdbook clean
- name: Build website
run: cd documentation/docs && mdbook build
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/docs-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: ci-docs
NYM_PROJECT_NAME: "Docs CI"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CI_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "docs-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DOCS }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
@@ -1,11 +1,9 @@
name: Nym Connect - mobile (Rust)
name: Nym Connect for Android (rust)
on:
push:
paths:
- "nym-connect/mobile/src-tauri/**"
- "nym-connect/mobile/src-tauri/Cargo.toml"
- "!nym-connect/mobile/src-tauri/gen/**"
- "nym-connect-android/src-tauri/**"
- "clients/client-core/**"
- "clients/socks5/**"
- "common/**"
@@ -14,9 +12,7 @@ on:
- "nym-api/nym-api-requests/**"
pull_request:
paths:
- "nym-connect/mobile/src-tauri/**"
- "nym-connect/mobile/src-tauri/Cargo.toml"
- "!nym-connect/mobile/src-tauri/gen/**"
- "nym-connect-android/src-tauri/**"
- "clients/client-core/**"
- "clients/socks5/**"
- "common/**"
@@ -32,9 +28,10 @@ jobs:
#RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache
#defaults:
#run:
#working-directory: nym-connect/mobile/src-tauri/
#working-directory: nym-connect-android/src-tauri/
steps:
- name: Install Dependencies (Linux)
if: ${{ !env.ACT }}
run: |
sudo apt-get update
sudo apt-get -y install \
@@ -49,6 +46,7 @@ jobs:
librsvg2-dev \
libsoup-3.0-dev \
libjavascriptcoregtk-4.1-dev
#continue-on-error: true
- name: Checkout
uses: actions/checkout@v3
@@ -60,13 +58,13 @@ jobs:
components: clippy, rustfmt
- name: Check formatting
run: cargo fmt --manifest-path nym-connect/mobile/src-tauri/Cargo.toml -- --check
run: cargo fmt --manifest-path nym-connect-android/src-tauri/Cargo.toml -- --check
- name: Build all binaries
run: cargo build --manifest-path nym-connect/mobile/src-tauri/Cargo.toml
run: cargo build --manifest-path nym-connect-android/src-tauri/Cargo.toml
- name: Run all tests
run: cargo test --manifest-path nym-connect/mobile/src-tauri/Cargo.toml
run: cargo test --manifest-path nym-connect-android/src-tauri/Cargo.toml
- name: Clippy
run: cargo clippy --manifest-path nym-connect/mobile/src-tauri/Cargo.toml --all-targets -- -D warnings
run: cargo clippy --manifest-path nym-connect-android/src-tauri/Cargo.toml --all-targets -- -D warnings
@@ -1,30 +1,28 @@
name: Nym Connect - desktop (Rust)
name: Nym Connect (rust)
on:
push:
paths:
- "nym-connect/desktop/src-tauri/**"
- "nym-connect/desktop/src-tauri/Cargo.toml"
- "clients/client-core/**"
- "clients/socks5/**"
- "common/**"
- "gateway/gateway-requests/**"
- "contracts/vesting/**"
- "nym-api/nym-api-requests/**"
- 'nym-connect/**'
- 'clients/client-core/**'
- 'clients/socks5/**'
- 'common/**'
- 'gateway/gateway-requests/**'
- 'contracts/vesting/**'
- 'nym-api/nym-api-requests/**'
pull_request:
paths:
- "nym-connect/desktop/src-tauri/**"
- "nym-connect/desktop/src-tauri/Cargo.toml"
- "clients/client-core/**"
- "clients/socks5/**"
- "common/**"
- "gateway/gateway-requests/**"
- "contracts/vesting/**"
- "nym-api/nym-api-requests/**"
- 'nym-connect/**'
- 'clients/client-core/**'
- 'clients/socks5/**'
- 'common/**'
- 'gateway/gateway-requests/**'
- 'contracts/vesting/**'
- 'nym-api/nym-api-requests/**'
jobs:
build:
runs-on: [self-hosted, custom-linux]
runs-on: [ self-hosted, custom-linux ]
env:
RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache
steps:
@@ -47,29 +45,29 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: build
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace
args: --manifest-path nym-connect/Cargo.toml --workspace
- name: Run all tests
uses: actions-rs/cargo@v1
with:
command: test
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace
args: --manifest-path nym-connect/Cargo.toml --workspace
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --manifest-path nym-connect/desktop/Cargo.toml --all -- --check
args: --manifest-path nym-connect/Cargo.toml --all -- --check
- uses: actions-rs/clippy-check@v1
name: Clippy checks
continue-on-error: true
with:
token: ${{ secrets.GITHUB_TOKEN }}
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace --all-features
args: --manifest-path nym-connect/Cargo.toml --workspace --all-features
- name: Run clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace --all-features -- -D warnings
args: --manifest-path nym-connect/Cargo.toml --workspace --all-features -- -D warnings
+1 -4
View File
@@ -6,7 +6,7 @@ on:
jobs:
build:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-contracts-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-contracts-') && github.event_name == 'release' }}
runs-on: [self-hosted, custom-runner-linux]
steps:
- uses: actions/checkout@v2
@@ -19,9 +19,6 @@ jobs:
override: true
components: rustfmt, clippy
- name: Install wasm-opt
run: cargo install --version 0.112.0 wasm-opt
- name: Build release contracts
run: make wasm
-13
View File
@@ -1,13 +0,0 @@
name: Greetings
on: [pull_request_target, issues]
jobs:
greeting:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for raising this issue'
pr-message: 'Thank you for making this first PR'
@@ -1,4 +1,4 @@
name: Nym Connect - Android APK Build
name: NC Android APK Release
on:
workflow_dispatch:
@@ -9,11 +9,11 @@ on:
jobs:
build:
name: Build APK
runs-on: custom-runner-linux
runs-on: ubuntu-latest
env:
ANDROID_HOME: ${{ github.workspace }}/android-sdk
NDK_VERSION: 25.1.8937393
NDK_HOME: ${{ github.workspace }}/android-sdk/ndk/25.1.8937393
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/${{ env.NDK_VERSION }}
SDK_PLATFORM_VERSION: android-33
SDK_BUILDTOOLS_VERSION: 33.0.1
@@ -23,17 +23,16 @@ jobs:
run: |
sudo apt-get update
sudo apt-get -y install \
libwebkit2gtk-4.0-dev \
build-essential \
unzip \
curl \
wget \
libssl-dev \
libgtk-3-dev \
squashfs-tools \
libayatana-appindicator3-dev \
librsvg2-dev
- name: Checkout
uses: actions/checkout@v3
- name: Install Java
uses: actions/setup-java@v3
with:
@@ -61,11 +60,6 @@ jobs:
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
# TODO this step takes a considerable amount of time
# We could avoid to compile from source tauri-cli and use instead
# pre-compiled binary provided by the node package `@tauri-apps/cli`
# But when using the later the build fails for some reason
# so keep installing and using tauri-cli
- name: Install tauri cli
run: cargo install tauri-cli --version "^2.0.0-alpha.2"
@@ -86,53 +80,48 @@ jobs:
npm i -g yarn
yarn --version
- name: Checkout
uses: actions/checkout@v3
- name: Build frontend code
run: |
yarn install --frozen-lockfile
yarn build
yarn workspace @nym/nym-connect-mobile webpack:prod
yarn workspace @nym/nym-connect-android webpack:prod
- name: Build APK
working-directory: nym-connect/mobile
working-directory: nym-connect-android
env:
# NODE_TAURI_CLI=${{ github.workspace }}/nym-connect/mobile/node_modules/.bin/tauri
ANDROID_SDK_ROOT: ${{ env.ANDROID_HOME }}
WRY_ANDROID_PACKAGE: net.nymtech.nym_connect
WRY_ANDROID_LIBRARY: nym_connect
WRY_ANDROID_PACKAGE: net.nymtech.nym_connect_android
WRY_ANDROID_LIBRARY: nym_connect_android
# TODO build with release profile (--release), it will requires
# to sign the APK. For now build with debug profile to avoid that
# TODO build using `yarn tauri`, provide NODE_TAURI_CLI, see TODO notes above
run: cargo tauri android build --debug --apk --split-per-abi -t aarch64
run: cargo tauri android build --debug --apk
# TODO add the version number to APK name
- name: Rename APK artifact
run: |
mkdir apk/
mv nym-connect/mobile/src-tauri/gen/android/nym_connect/app/build/outputs/apk/arm64/debug/app-arm64-debug.apk \
apk/nym-connect-arm64-debug.apk
mv nym-connect/mobile/src-tauri/gen/android/nym_connect/app/build/outputs/apk/x86_64/debug/app-x86_64-debug.apk \
apk/nym-connect-x86_64-debug.apk
mv nym-connect-android/src-tauri/gen/android/nym_connect_android/app/build/outputs/apk/universal/debug/app-universal-debug.apk \
nym-connect-debug.apk
- name: Upload APK artifact
uses: actions/upload-artifact@v3
with:
name: nc-apk-debug
path: |
apk/nym-connect-arm64-debug.apk
apk/nym-connect-x86_64-debug.apk
path: nym-connect-debug.apk
# publish:
# name: Publish APK
# needs: build
# runs-on: ubuntu-latest
# steps:
# - name: Checkout
# uses: actions/checkout@v3
# - name: Download binary artifact
# uses: actions/download-artifact@v3
# with:
# name: nc-apk-debug
# path: apk
# # TODO add a step to upload the APK somewhere
# - name: Publish
# uses: ???
publish:
name: Publish APK
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Download binary artifact
uses: actions/download-artifact@v3
with:
name: nc-apk-debug
path: apk
# TODO add a step to upload the APK somewhere
# - name: Publish
# uses: ???
@@ -0,0 +1,57 @@
name: CI for Network Explorer API
on:
workflow_dispatch:
release:
types: [created]
env:
NETWORK: mainnet
jobs:
publish-nym:
if: ${{ startsWith(github.ref, 'refs/tags/nym-explorer-api-') && (github.event_name == 'release' || github.event_name == 'workflow_dispatch') }}
strategy:
fail-fast: false
matrix:
platform: [ubuntu-20.04]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools
continue-on-error: true
- name: Check the release tag starts with `nym-explorer-api-`
uses: actions/github-script@v3
with:
script: |
core.setFailed('Release tag did not start with nym-explorer-api-...')
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all explorer-api
uses: actions-rs/cargo@v1
with:
command: build
args: --manifest-path explorer-api/Cargo.toml --workspace --release
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: my-artifact
path: |
target/release/explorer-api
retention-days: 30
- name: Upload to release based on tag name
uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with:
files: |
target/release/explorer-api
+6 -2
View File
@@ -57,10 +57,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ne-sb-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: network-explorer
NYM_PROJECT_NAME: "Network Explorer"
@@ -69,6 +69,10 @@ jobs:
NYM_CI_WWW_LOCATION_STORYBOOK: "ne-sb-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-network-explorer"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
+17 -13
View File
@@ -10,7 +10,7 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
# creates the matrix strategy from nightly_build_matrix_includes.json
- uses: actions/checkout@v3
- uses: actions/checkout@v2
- id: set-matrix
uses: JoshuaTheMiller/conditional-build-matrix@main
with:
@@ -29,7 +29,7 @@ jobs:
if: matrix.os == 'ubuntu-20.04'
- name: Check out repository code
uses: actions/checkout@v3
uses: actions/checkout@v2
- name: Install rust toolchain
uses: actions-rs/toolchain@v1
@@ -51,9 +51,9 @@ jobs:
command: build
args: --workspace
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -63,9 +63,9 @@ jobs:
command: build
args: --workspace --examples
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -75,7 +75,7 @@ jobs:
command: test
args: --workspace
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
with:
@@ -88,9 +88,9 @@ jobs:
command: test
args: --workspace -- --ignored
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -147,17 +147,17 @@ jobs:
- name: Collect jobs status
uses: technote-space/workflow-conclusion-action@v2
- name: Check out repository code
uses: actions/checkout@v3
uses: actions/checkout@v2
- name: install npm
uses: actions/setup-node@v3
if: env.WORKFLOW_CONCLUSION == 'failure'
with:
node-version: 16
- name: Matrix - Node Install
- name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure'
env:
NYM_NOTIFICATION_KIND: nightly
@@ -165,9 +165,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
+22 -30
View File
@@ -28,7 +28,7 @@ jobs:
run: git fetch --all
- name: Set output variable to latest release branch
id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+' | tail -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
build:
needs: [get_release,matrix_prep]
strategy:
@@ -54,33 +54,15 @@ jobs:
override: true
components: rustfmt, clippy
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
with:
command: clean
- name: Build all examples
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Reclaim some disk space
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -90,9 +72,9 @@ jobs:
command: test
args: --workspace
- name: Reclaim some disk space
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -101,11 +83,17 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: test
args: --workspace -- --ignored
args: --workspace --all-features -- --ignored
- name: Reclaim some disk space
- name: Check formatting
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
with:
command: fmt
args: --all -- --check
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -114,7 +102,7 @@ jobs:
continue-on-error: true
with:
token: ${{ secrets.GITHUB_TOKEN }}
args: --workspace
args: --all-features
- name: Run clippy
uses: actions-rs/cargo@v1
@@ -168,11 +156,11 @@ jobs:
if: env.WORKFLOW_CONCLUSION == 'failure'
with:
node-version: 16
- name: Matrix - Node Install
- name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure'
env:
NYM_NOTIFICATION_KIND: nightly
@@ -180,9 +168,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly-release"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
+17 -25
View File
@@ -28,7 +28,7 @@ jobs:
run: git fetch --all
- name: Set output variable to latest release branch
id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 2 | head -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+' | tail -n 2 | head -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
build:
needs: [get_release,matrix_prep]
strategy:
@@ -54,12 +54,6 @@ jobs:
override: true
components: rustfmt, clippy
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
@@ -72,18 +66,6 @@ jobs:
with:
command: clean
- name: Build all examples
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
- name: Run all tests
uses: actions-rs/cargo@v1
with:
@@ -92,7 +74,7 @@ jobs:
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
@@ -101,7 +83,13 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: test
args: --workspace -- --ignored
args: --workspace --all-features -- --ignored
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
@@ -114,7 +102,7 @@ jobs:
continue-on-error: true
with:
token: ${{ secrets.GITHUB_TOKEN }}
args: --workspace
args: --all-features
- name: Run clippy
uses: actions-rs/cargo@v1
@@ -168,11 +156,11 @@ jobs:
if: env.WORKFLOW_CONCLUSION == 'failure'
with:
node-version: 16
- name: Matrix - Node Install
- name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure'
env:
NYM_NOTIFICATION_KIND: nightly
@@ -180,9 +168,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly-release"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
+50
View File
@@ -0,0 +1,50 @@
name: Publish Nym CLI binaries
on:
workflow_dispatch:
release:
types: [created]
env:
NETWORK: mainnet
jobs:
publish-nym-cli:
if: ${{ startsWith(github.ref, 'refs/tags/nym-cli-') && (github.event_name == 'release' || github.event_name = 'workflow_dispatch') }}
strategy:
fail-fast: false
matrix:
platform: [ubuntu-20.04, windows-latest, macos-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Check the release tag starts with `nym-cli-`
uses: actions/github-script@v3
with:
script: |
core.setFailed('Release tag did not start with nym-cli-...')
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build binary
run: make build-nym-cli
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: nym-cli-${{ matrix.platform }}
path: |
target/release/nym-cli*
retention-days: 30
- name: Upload to release based on tag name
uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with:
files: |
target/release/nym-cli
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (MacOS)
name: Publish Nym Connect (MacOS)
on:
workflow_dispatch:
release:
@@ -6,11 +6,11 @@ on:
defaults:
run:
working-directory: nym-connect/desktop
working-directory: nym-connect
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -73,7 +73,7 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: nym-connect_1.0.0_x64.dmg
path: nym-connect/desktop/target/release/bundle/dmg/nym-connect_1.0.0_x64.dmg
path: nym-connect/target/release/bundle/dmg/nym-connect_1.0.0_x64.dmg
retention-days: 30
- name: Clean up keychain
@@ -86,5 +86,5 @@ jobs:
if: github.event_name == 'release'
with:
files: |
nym-connect/desktop/target/release/bundle/dmg/*.dmg
nym-connect/desktop/target/release/bundle/macos/*.app.tar.gz*
nym-connect/target/release/bundle/dmg/*.dmg
nym-connect/target/release/bundle/macos/*.app.tar.gz*
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (Ubuntu)
name: Publish Nym Connect (Ubuntu)
on:
workflow_dispatch:
release:
@@ -6,11 +6,11 @@ on:
defaults:
run:
working-directory: nym-connect/desktop
working-directory: nym-connect
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -53,7 +53,7 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: nym-connect.AppImage.tar.gz
path: nym-connect/desktop/target/release/bundle/appimage/nym-connect_1.0.0_amd64.AppImage
path: nym-connect/target/release/bundle/appimage/nym-connect_1.0.0_amd64.AppImage
retention-days: 30
- name: Upload to release based on tag name
@@ -61,5 +61,5 @@ jobs:
if: github.event_name == 'release'
with:
files: |
nym-connect/desktop/target/release/bundle/appimage/*.AppImage
nym-connect/desktop/target/release/bundle/appimage/*.AppImage.tar.gz*
nym-connect/target/release/bundle/appimage/*.AppImage
nym-connect/target/release/bundle/appimage/*.AppImage.tar.gz*
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (Windows 10)
name: Publish Nym Connect (Windows 10)
on:
workflow_dispatch:
release:
@@ -6,11 +6,11 @@ on:
defaults:
run:
working-directory: nym-connect/desktop
working-directory: nym-connect
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -56,7 +56,7 @@ jobs:
encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }}
- name: Install app dependencies
run: yarn --network-timeout 100000
run: yarn
- name: Build and sign it
env:
@@ -72,7 +72,7 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: nym-connect_1.0.0_x64_en-US.msi
path: nym-connect/desktop/target/release/bundle/msi/nym-connect_1.0.0_x64_en-US.msi
path: nym-connect/target/release/bundle/msi/nym-connect_1.0.0_x64_en-US.msi
retention-days: 30
- name: Upload to release based on tag name
@@ -80,5 +80,5 @@ jobs:
if: github.event_name == 'release'
with:
files: |
nym-connect/desktop/target/release/bundle/msi/*.msi
nym-connect/desktop/target/release/bundle/msi/*.msi.zip*
nym-connect/target/release/bundle/msi/*.msi
nym-connect/target/release/bundle/msi/*.msi.zip*
@@ -1,13 +1,13 @@
name: CI for nym-connect - Desktop
name: CI for nym-connect
on:
push:
paths:
- 'nym-connect/desktop/**'
- 'nym-connect/**'
defaults:
run:
working-directory: nym-connect/desktop
working-directory: nym-connect
jobs:
build:
@@ -34,15 +34,15 @@ jobs:
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "nym-connect/desktop/storybook-static/"
SOURCE: "nym-connect/storybook-static/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/nym-connect-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: nym-connect
NYM_PROJECT_NAME: "nym-connect"
@@ -50,6 +50,10 @@ jobs:
NYM_CI_WWW_LOCATION: "nym-connect-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nym-connect"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
+1 -1
View File
@@ -16,7 +16,7 @@ env:
jobs:
publish-nym:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -10,7 +10,7 @@ defaults:
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -9,7 +9,7 @@ defaults:
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -1,6 +1,5 @@
name: Publish Nym Wallet (Windows 10)
on:
workflow_dispatch:
release:
types: [created]
@@ -10,7 +9,7 @@ defaults:
jobs:
publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy:
fail-fast: false
matrix:
@@ -56,7 +55,7 @@ jobs:
encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }}
- name: Install app dependencies
run: yarn --network-timeout 100000
run: yarn
- name: Build and sign it
env:
@@ -68,16 +67,8 @@ jobs:
TAURI_KEY_PASSWORD: ${{ secrets.TAURI_KEY_PASSWORD }}
run: yarn build
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: nym-wallet_1.0.0_x64_en-US.msi
path: nym-wallet/target/release/bundle/msi/nym-wallet_1.*.msi
retention-days: 30
- name: Upload to release based on tag name
uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with:
files: |
nym-wallet/target/release/bundle/msi/*.msi
+6 -2
View File
@@ -35,10 +35,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/wallet-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: nym-wallet
NYM_PROJECT_NAME: "nym-wallet"
@@ -46,6 +46,10 @@ jobs:
NYM_CI_WWW_LOCATION: "wallet-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nym-wallet"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
-36
View File
@@ -1,36 +0,0 @@
name: CI for Nym API Tests
on:
workflow_dispatch:
push:
paths:
- "nym-api/**"
defaults:
run:
working-directory: nym-api/tests
jobs:
test:
name: nym-api tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install npm
run: npm install
- name: Node v18
uses: actions/setup-node@v3
with:
node-version: 18.1.0
- name: Install yarn
run: yarn install
- name: Run yarn
run: yarn
- name: Run tests
run: yarn test:qa
working-directory: nym-api/tests
+5 -1
View File
@@ -1,3 +1,7 @@
KEYBASE_NYM_CHANNEL=
KEYBASE_NYMBOT_USERNAME=
KEYBASE_NYMBOT_PAPERKEY=
MATRIX_SERVER=
MATRIX_ROOM=
MATRIX_ROOM_OF_SHAME=
@@ -35,4 +39,4 @@ NYM_CI_WWW_BASE=example.com
# Nightly builds
WORKFLOW_CONCLUSION=success
SHOW_DEBUG=true
SHOW_DEBUG=true
+9 -10
View File
@@ -4,7 +4,7 @@ This is a collection of scripts and files to support GitHub Actions.
## Sending Notifications
These scripts send CI notifications to Matrix by creating messages from templates and env vars passed from GitHub Actions.
These scripts send CI notifications to Keybase by creating messages from templates and env vars passed from GitHub Actions.
### Adding notifications to a GitHub Action
@@ -19,11 +19,10 @@ jobs:
env:
NYM_NOTIFICATION_KIND: "my-component"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-network-explorer"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
@@ -35,8 +34,8 @@ Notifications are run by adding the snippet above to a GitHub Action, and:
1. Installing node packages needed at run time
2. Set the env vars as required:
- `NYM_NOTIFICATION_KIND` matches the directory in `.github/workflows/support-files/${NYM_NOTIFICATION_KIND}` to provide the templates and extra scripting in `index.js`
- Matrix credentials, room and other env vars for the status of the build and repo
3. Replacing the default entry point shell script on the `keybaseio/client:stable-node` docker image to run `.github/workflows/support-files/notifications/entry_point.sh`
- Keybase credentials, channel and other env vars for the status of the build and repo
3. Replacing the default entry point shell script on the `keybaseio/client:stable-node` docker image to run `.github/workflows/support-files/notifications/entry_point.sh`
### Running locally
@@ -44,7 +43,7 @@ You will need:
- Node 16 LTS
- npm
Copy `.github/workflows/support-files/.env.example` to `.github/workflows/support-files/.env` and valid Matrix credentials.
Copy `.github/workflows/support-files/.env.example` to `.github/workflows/support-files/.env` and valid Keybase credentials.
Then run `npm install` to get dependencies.
@@ -56,4 +55,4 @@ npm install
cp .env.example .env
vi .env
npm run dev
```
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_BASE }}/developers/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,11 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
> 🔴 **FAILURE** :cry:
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_BASE }}/docs/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,62 +0,0 @@
#!/usr/bin/env bash
#
# Basic usage:
# ./git-merge-check origin/develop origin/release/v1.1.9
#
set -x
set -o nounset
set -o pipefail
# Start from branch ...
branch1=$1
# ... and try to merge in
branch2=$2
echo "Checking if $branch2 merges without conflicts into $branch1..."
git checkout -q $branch1 || exit 1
# There are two options here as far as I see on what we should check for. Either
#
# (A) check for CONFLICT in any file except whitelist (such as .lock files)
# (B) check for "Automatic merge failed"
#
# Both of these options have drawbacks.
#
# The first (A) has the problem in that maybe we don't want to fail if the
# changes can be automatically merged (duh), but at least we are not pestered
# about constant lock file changes.
# The second (B) has the problem that it's difficult to filter out automatic
# merge fails for files we don't care about (.lock files).
#
# The ideal solution would be to check for automatic merge fails for files
# except those on a whitelist (e.g. lock files).
# For now I choose to use (B) here, because I hope it might be less noisy
# Alternative A
#output=$(git merge --no-commit --no-ff $branch2 | grep -v .lock)
#merge_failed=$(echo $output | grep -v "CONFLICT")
#return_code=$?
# Alternative B
output=$(git merge --no-commit --no-ff $branch2)
merge_failed=$(echo $output | grep -v "Automatic merge failed")
return_code=$?
# Restore
git merge --abort
git checkout -q -
if [ $return_code -eq 0 ]; then
echo "Merge check success"
else
echo "Merge check failed"
fi
exit $return_code
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
@@ -1,16 +1,10 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION_STORYBOOK }}.{{ env.NYM_CI_WWW_BASE }}
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
@@ -151,7 +151,7 @@ async function getMessageBody(context) {
return `${icon} ${job.conclusion}: ${job.name} - ${job.html_url}`;
})
// and join with newlines for display in the template
.join('\n\n');
.join('\n');
return template({ ...context, jobResults });
}
@@ -1,15 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `when` {{ timestamp }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
{{ jobResults }}
@@ -1,15 +1,9 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS**
>
> `when` {{ timestamp }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
{{ jobResults }}
{{ jobResults }}
@@ -1,9 +1,10 @@
require('dotenv').config();
const Bot = require('keybase-bot');
const { sendMatrixMessage } = require('./send_message_to_matrix');
let context = {
kinds: ['nym-wallet', 'ts-packages', 'network-explorer', 'nightly', 'nym-connect','security','ci-docs','cd-docs','ci-dev','cd-dev'],
kinds: ['nym-wallet', 'ts-packages', 'network-explorer', 'nightly', 'nym-connect','security'],
};
/**
@@ -23,6 +24,21 @@ function validateContext() {
'Please set env var NYM_PROJECT_NAME with the project name for displaying in notification messages',
);
}
if (!context.env.KEYBASE_NYM_CHANNEL) {
throw new Error(
'Please set env var KEYBASE_NYM_CHANNEL with the channel name for the notification message',
);
}
if (!context.env.KEYBASE_NYMBOT_USERNAME) {
throw new Error(
'Username is not defined. Please set env var KEYBASE_NYMBOT_USERNAME',
);
}
if (!context.env.KEYBASE_NYMBOT_PAPERKEY) {
throw new Error(
'Paperkey is not defined. Please set env var KEYBASE_NYMBOT_PAPERKEY',
);
}
if (context.env.MATRIX_ROOM) {
if (!context.env.MATRIX_SERVER) {
throw new Error(
@@ -68,6 +84,12 @@ function createTemplateContext() {
context.kind = context.env.NYM_NOTIFICATION_KIND;
context.keybase = {
channel: context.env.KEYBASE_NYM_CHANNEL,
username: context.env.KEYBASE_NYMBOT_USERNAME,
paperkey: context.env.KEYBASE_NYMBOT_PAPERKEY,
};
if (!context.env.GIT_BRANCH_NAME) {
context.env.GIT_BRANCH_NAME = context.env.GITHUB_REF.split('/')
.slice(2)
@@ -77,6 +99,40 @@ function createTemplateContext() {
context.status = process.env.IS_SUCCESS === 'true' ? 'success' : 'failure';
}
async function sendKeybaseMessage(messageBody) {
const bot = new Bot();
try {
console.log(
`Initialising keybase with user "${
context.keybase.username
}" and key: "${'*'.repeat(context.keybase.paperkey.length)}"...`,
);
await bot.init(context.keybase.username, context.keybase.paperkey, {
verbose: false,
});
const channel = {
name: context.env.KEYBASE_NYMBOT_TEAM || 'nymtech_bot',
membersType: 'team',
topicName: context.keybase.channel,
topic_type: 'CHAT',
};
const message = {
body: messageBody,
};
console.log(`Sending to ${channel.name}#${channel.topicName}...`);
await bot.chat.send(channel, message);
console.log('Message sent!');
} catch (error) {
console.error(error);
process.exitCode = -1;
} finally {
await bot.deinit();
}
}
/**
* Uses the `kind` set in the context to process the context and generate a notification message
* @returns {Promise<string>} A string notification message body
@@ -113,6 +169,7 @@ async function main() {
console.log(messageBody);
console.log('-----------------------------------------');
}
await sendKeybaseMessage(messageBody);
if(context.env.MATRIX_ROOM) {
await sendMatrixMessage(context, messageBody, context.env.MATRIX_ROOM)
}
@@ -5,7 +5,12 @@ const localStorage = new LocalStorage('./scratch');
const {
LocalStorageCryptoStore,
} = require('matrix-js-sdk/lib/crypto/store/localStorage-crypto-store');
var showdown = require('showdown');
const vfile = require('to-vfile');
const unified = require('unified');
const remarkParse = require('remark-parse');
const remarkHtml = require('remark-html');
const emoji = require('remark-emoji');
// hide all matrix client output
console.error = (error) => console.log('❌ error: ', error);
@@ -54,9 +59,17 @@ function createClient(context, room, message) {
return client;
}
async function markdownToHtml(messageAsMarkdown) {
const file = await unified()
.use(emoji)
.use(remarkParse)
.use(remarkHtml)
.process(await vfile({ path: 'test.md', contents: messageAsMarkdown}));
return String(file);
}
async function sendMatrixMessage(contextArg, messageAsMarkdown, roomId) {
const converter = new showdown.Converter();
const messageAsHtml = converter.makeHtml(messageAsMarkdown);
const messageAsHtml = await markdownToHtml(messageAsMarkdown);
const client = createClient(contextArg, roomId, messageAsHtml);
await client.initCrypto();
await client.startClient({ initialSyncLimit: 1 });
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
@@ -1,14 +1,9 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View storybook:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
@@ -1,18 +1,13 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS**
>
> ➡️➡️➡️➡️➡️ **View output:**
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
+4 -4
View File
@@ -1,6 +1,6 @@
{
"name": "send-matrix-message",
"description": "Sends a notification message with the matrix sdk",
"name": "send-keybase-message",
"description": "Sends a notification message with the keybase package that fails when piped into the keybase CLI",
"version": "1.0.0",
"private": true,
"scripts": {
@@ -10,6 +10,7 @@
"dependencies": {
"dotenv": "^16.0.0",
"handlebars": "^4.7.7",
"keybase-bot": "^3.6.1",
"matrix-js-sdk": "^9.3.0",
"node-localstorage": "^2.1.6",
"octokit": "^1.7.1",
@@ -17,11 +18,10 @@
"remark-emoji": "^2.2.0",
"remark-html": "^13.0.2",
"remark-parse": "^9.0.0",
"showdown": "^2.1.0",
"to-vfile": "^6.1.0",
"unified": "^9.2.2"
},
"devDependencies": {
"prettier": "^2.8.7"
"prettier": "2.3.2"
}
}
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
@@ -1,20 +1,14 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS**
>
> ➡️➡️➡️➡️➡️ **View output:**
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}
>
> `example`: https://{{ env.NYM_CI_WWW_LOCATION }}-example.{{ env.NYM_CI_WWW_BASE }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
-46
View File
@@ -1,46 +0,0 @@
name: tag-and-release
on:
workflow_dispatch:
inputs:
nym_binaries_version:
description: 'Version of the nym-binaries tag'
required: false
type: string
nym_wallet_version:
description: 'Version of the nym-wallet tag'
required: false
type: string
nym_connect_version:
description: 'Version of the nym-connect tag'
required: false
type: string
jobs:
tag-components:
uses: nymtech/reusable-workflows/.github/workflows/tag-components.yml@master
with:
nym_binaries_version: ${{ input.nym_binaries_version }}
nym_wallet_version: ${{ input.nym_wallet_version }}
nym_connect_version: ${{ input.nym_connect_version }}
create-nym-binaries-release:
if: ${{ input.nym_binaries_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-binaries-release.yml@master
with:
version: ${{ input.nym_binaries_version }}
needs: ["tag-components"]
create-nym-wallet-release:
if: ${{ input.nym_wallet_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-wallet-release.yml@master
with:
version: ${{ input.nym_wallet_version }}
needs: ["tag-components"]
create-nym-connect-release:
if: ${{ input.nym_connect_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-connect-release.yml@master
with:
version: ${{ input.nym_connect_version }}
needs: ["tag-components"]
+10 -23
View File
@@ -5,22 +5,9 @@ on:
paths:
- 'ts-packages/**'
- 'sdk/typescript/**'
- 'nym-connect/desktop/src/**'
- 'nym-connect/desktop/package.json'
- 'nym-connect/mobile/src/**'
- 'nym-connect/mobile/package.json'
- 'nym-wallet/src/**'
- 'nym-wallet/package.json'
pull_request:
paths:
- 'ts-packages/**'
- 'sdk/typescript/**'
- 'nym-connect/desktop/src/**'
- 'nym-connect/desktop/package.json'
- 'nym-connect/mobile/src/**'
- 'nym-connect/mobile/package.json'
- 'nym-wallet/src/**'
- 'nym-wallet/package.json'
- nym-connect
- nym-connect-android
- nym-wallet
jobs:
build:
@@ -36,16 +23,12 @@ jobs:
node-version: 16
- name: Setup yarn
run: npm install -g yarn
- name: Install
run: yarn
- name: Build packages
run: yarn build
- name: Lint
run: yarn lint && yarn tsc
- name: Matrix - Node Install
run: yarn && yarn lint && yarn tsc
- name: Keybase - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
- name: Keybase - Send Notification
env:
NYM_NOTIFICATION_KIND: ts-packages
NYM_PROJECT_NAME: "ts-packages"
@@ -53,6 +36,10 @@ jobs:
NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-ts-packages"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
@@ -1,95 +0,0 @@
name: update-versions-and-changelog
on:
workflow_dispatch:
inputs:
release_version:
description: "Release version, usually the milestone title"
required: true
type: string
milestone_id:
description: "Milestone ID, check the URL when you're on the specific milestone page"
required: true
type: string
env:
CI_BOT_AUTHOR: "Nym bot"
CI_BOT_EMAIL: "nym-bot@users.noreply.github.com"
jobs:
update-versions:
runs-on: ubuntu-22.04
steps:
- name: checkout-source
uses: actions/checkout@v3
with:
ref: "release/${{ inputs.release_version }}"
path: "nym"
- name: checkout-ci-tools-repo
uses: actions/checkout@v3
with:
repository: "nymtech/ci-tools"
ref: "master"
path: "ci-tools"
token: "${{ secrets.ACCESS_TOKEN_PRIVATE_REPOS }}"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: install-version-bumper
run: "cargo install --path ."
working-directory: "ci-tools/version-bumper"
- name: run-version-bumper
run: "version-bumper bump binaries --nym-repo-directory nym"
- name: push-changes-to-branch
run: |
git config --global user.name "${{ env.CI_BOT_AUTHOR }}"
git config --global user.email "${{ env.CI_BOT_EMAIL }}"
git checkout -b release/${{ inputs.release_version }}-preparation
git commit -am "chore: version bump in preparation for release"
git push -u origin release/${{ inputs.release_version }}-preparation
working-directory: "nym"
update-changelog:
runs-on: ubuntu-22.04
needs: [update-versions]
steps:
- name: checkout-source
uses: actions/checkout@v3
with:
ref: "release/${{ inputs.release_version }}"
path: "nym"
- name: checkout-ci-tools-repo
uses: actions/checkout@v3
with:
repository: "nymtech/ci-tools"
ref: "master"
path: "ci-tools"
token: "${{ secrets.ACCESS_TOKEN_PRIVATE_REPOS }}"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: install-changelog-updater
run: "cargo install --path ."
working-directory: "ci-tools/changelog-updater"
- name: run-changelog-updater
run: "changelog-updater mix ${{ inputs.milestone_id }} release/${{ inputs.release_version }}"
- name: push-changes-to-branch
run: |
git config --global user.name "${{ env.CI_BOT_AUTHOR }}"
git config --global user.email "${{ env.CI_BOT_EMAIL }}"
git checkout release/${{ inputs.release_version }}-preparation
git commit -am "chore: update changelog preparation for release"
git push
working-directory: "nym"
+2 -1
View File
@@ -39,6 +39,7 @@ validator-api-config.toml
dist
storybook-static
envs/qwerty.env
Cargo.lock
nym-connect/Cargo.lock
.parcel-cache
**/.DS_Store
cpu-cycles/libcpucycles/build
+15 -142
View File
@@ -2,152 +2,19 @@
Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
- nym-network-statistics properly handles signals ([#3209])
- add socks5 support for Rust SDK ([#3226], [#3255])
- add coconut bandwidth credential support for Rust SDK ([#3273])
[#3209]: https://github.com/nymtech/nym/issues/3209
[#3226]: https://github.com/nymtech/nym/pull/3226
[#3255]: https://github.com/nymtech/nym/pull/3255
[#3273]: https://github.com/nymtech/nym/pull/3273
## [v1.1.15] (2023-04-18)
- Fix verloc being stuck waiting for shutdown signal ([#3250])
- Introduce `--output json` flag to `sign` command to allow to more easily capture the output ([#3249])
- Explorer - Dont fetch Service Provider list on Testnet ([#3245])
- When determining active set, rather than weighting the nodes by just the `stake`, use `stake * performance` ([#3234])
- Introduce dual packet sizes to our clients (as in use two packet sizes at the same time depending on message size) ([#3189])
- Experiment with offline signing in our validator client ([#3174])
- Modify network requester binary to reload `allowed.list` periodically to pull in any changes made upstream without having to restart the service ([#3149])
- Standardise all `--output json` on binary inits, we pass the output json at different points for different binaries. ([#3080])
- Service provider directory contract: initial version ([#2759])
- Fix issue where network-requester run failed on fresh init due to missing allow file ([#3316])
[#3250]: https://github.com/nymtech/nym/issues/3250
[#3249]: https://github.com/nymtech/nym/issues/3249
[#3245]: https://github.com/nymtech/nym/issues/3245
[#3234]: https://github.com/nymtech/nym/issues/3234
[#3189]: https://github.com/nymtech/nym/issues/3189
[#3174]: https://github.com/nymtech/nym/issues/3174
[#3149]: https://github.com/nymtech/nym/issues/3149
[#3080]: https://github.com/nymtech/nym/issues/3080
[#2759]: https://github.com/nymtech/nym/issues/2759
[#3316]: https://github.com/nymtech/nym/pull/3316
## [v1.1.14] (2023-04-04)
- Investigate cause of qwerty validator being in invalid rewarding state ([#3224])
- Fix NR config due to changes in #3199 ([#3223])
- [Issue] Mixnodes and gateway do not close connections properly ([#3187])
- disable sign-ext when using wasm-opt + update wasm-opt ([#3203])
- chore: tidy up client `Debug` config section ([#3199])
[#3224]: https://github.com/nymtech/nym/issues/3224
[#3223]: https://github.com/nymtech/nym/issues/3223
[#3187]: https://github.com/nymtech/nym/issues/3187
[#3203]: https://github.com/nymtech/nym/pull/3203
[#3199]: https://github.com/nymtech/nym/pull/3199
>>>>>>> master
## [v1.1.13] (2023-03-15)
- NE - instead of throwing a "Mixnode/Gateway not found" error for blacklisted nodes due to bad performance, show their history but tag them as "Having poor performance" ([#2979])
- NE - Upgrade Sandbox and make below changes: ([#2332])
- Explorer - Updates ([#3168])
- Website v2 - deploy infrastructure for strapi and CI ([#2213])
- add blockstream green to sp list ([#3180])
- mock-nym-api: fix .storybook lint error ([#3178])
- Validating new interval config parameters to prevent division by zero ([#3153])
[#2979]: https://github.com/nymtech/nym/issues/2979
[#2332]: https://github.com/nymtech/nym/issues/2332
[#3168]: https://github.com/nymtech/nym/issues/3168
[#2213]: https://github.com/nymtech/nym/issues/2213
[#3180]: https://github.com/nymtech/nym/pull/3180
[#3178]: https://github.com/nymtech/nym/pull/3178
[#3153]: https://github.com/nymtech/nym/pull/3153
## [v1.1.12] (2023-03-07)
- Fix generated docs for mixnet and vesting contract on docs.rs ([#3093])
- Introduce a way of injecting topology into the client ([#3044])
- Update mixnet TypeScript client methods #1 ([#2783])
- Update tooltips for routing and average score ([#3133])
- update selected service provider description style ([#3128])
[#3093]: https://github.com/nymtech/nym/issues/3093
[#3044]: https://github.com/nymtech/nym/issues/3044
[#2783]: https://github.com/nymtech/nym/issues/2783
[#3133]: https://github.com/nymtech/nym/pull/3133
[#3128]: https://github.com/nymtech/nym/pull/3128
## [v1.1.11] (2023-02-28)
- Fix empty dealer set loop ([#3105])
- The nym-api db.sqlite is broken when trying to run against it it in `enabled-credentials-mode true` there is an ordering issue with migrations when using the credential binary to purchase bandwidth ([#3100])
- Feature/latency based gateway selection ([#3081])
- Fix the credential binary to handle transactions to sleep when in non-inProgress epochs ([#3057])
- Publish mixnet contract to crates.io ([#1919])
- Publish vesting contract to crates.io ([#1920])
- Feature/update checker to use master ([#3097])
- Feature/improve binary checks ([#3094])
[#3105]: https://github.com/nymtech/nym/issues/3105
[#3100]: https://github.com/nymtech/nym/issues/3100
[#3081]: https://github.com/nymtech/nym/pull/3081
[#3057]: https://github.com/nymtech/nym/issues/3057
[#1919]: https://github.com/nymtech/nym/issues/1919
[#1920]: https://github.com/nymtech/nym/issues/1920
[#3097]: https://github.com/nymtech/nym/pull/3097
[#3094]: https://github.com/nymtech/nym/pull/3094
## [v1.1.10] (2023-02-21)
- Verloc listener causing mixnode unexpected shutdown ([#3038])
- rust-sdk - update API following implementation experience with the network-requester ([#3001])
- Prevent coconut deposits in incompatible states ([#2991])
- Support unavailable signer within threshold ([#2987])
- Implement DKG re-sharing ([#2935])
- contracts: add nym prefix to mixnet and vesting contract packages ([#2855])
- Introduce common interface for all service providers to allow obtaining information such as whether they're online, what binary version they're running, etc. ([#2758])
- Add client functionality to nym-network-requester ([#1900])
- nym-api: uptime rework ([#3053])
- ci: update typescript-lint.yml ([#3035])
- contracts: add nym prefix to mixnet and vesting contract packages ([#2855])
[#3038]: https://github.com/nymtech/nym/issues/3038
[#3001]: https://github.com/nymtech/nym/issues/3001
[#2991]: https://github.com/nymtech/nym/issues/2991
[#2987]: https://github.com/nymtech/nym/issues/2987
[#2935]: https://github.com/nymtech/nym/issues/2935
[#2855]: https://github.com/nymtech/nym/pull/2855
[#2758]: https://github.com/nymtech/nym/issues/2758
[#1900]: https://github.com/nymtech/nym/issues/1900
[#3053]: https://github.com/nymtech/nym/pull/3053
[#3035]: https://github.com/nymtech/nym/pull/3035
[#2855]: https://github.com/nymtech/nym/pull/2855
## [v1.1.9] (2023-02-07)
# [Unreleased]
### Added
- Remove Coconut feature flag ([#2793])
- Separate `nym-api` endpoints with values of "total-supply" and "circulating-supply" in `nym` ([#2964])
- remove coconut feature and unify builds ([#2890])
- native-client: is now capable of listening for requests on sockets different than `127.0.0.1` ([#2939]). This can be specified via `--host` flag during `init` or `run`. Alternatively a custom `host` can be set in `config.toml` file under `socket` section.
- dkg resharing mode ([#2936])
### Changed
[#2890]: https://github.com/nymtech/nym/pull/2890
[#2939]: https://github.com/nymtech/nym/pull/2939
[#2936]: https://github.com/nymtech/nym/pull/2936
- native-client: is now capable of listening for requests on sockets different than `127.0.0.1` ([#2912]). This can be specified via `--host` flag during `init` or `run`. Alternatively a custom `host` can be set in `config.toml` file under `socket` section.
- mixnode, gateway: fix unexpected shutdown on corrupted connection ([#2963])
[#2793]: https://github.com/nymtech/nym/issues/2793
[#2912]: https://github.com/nymtech/nym/issues/2912
[#2964]: https://github.com/nymtech/nym/issues/2964
[#2963]: https://github.com/nymtech/nym/issues/3017
## [v1.1.8] (2023-01-31)
# [v1.1.8] (2023-01-31)
### Added
@@ -167,7 +34,7 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
[#2873]: https://github.com/nymtech/nym/issues/2873
## [v1.1.7] (2023-01-24)
# [v1.1.7] (2023-01-24)
### Added
@@ -223,6 +90,12 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
### Added
### Changed
## [v1.1.5] (2023-01-10)
### Added
- socks5: send status message for service ready, and network-requester error response in https://github.com/nymtech/nym/pull/2715
### Changed
Generated
+1595 -1679
View File
File diff suppressed because it is too large Load Diff
+11 -30
View File
@@ -17,35 +17,36 @@ opt-level = 3
resolver = "2"
members = [
"clients/client-core",
"clients/credential",
"clients/native",
"clients/native/websocket-requests",
"clients/socks5",
"common/async-file-watcher",
"common/bandwidth-controller",
"common/bin-common",
"common/client-core",
"common/bandwidth-claim-contract",
"common/client-libs/gateway-client",
"common/client-libs/mixnet-client",
"common/client-libs/validator-client",
"common/client-connections",
"common/coconut-interface",
"common/commands",
"common/config",
"common/build-information",
"common/cosmwasm-smart-contracts/coconut-bandwidth-contract",
"common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/service-provider-directory",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/mobile-storage",
"common/credential-storage",
"common/credentials",
"common/crypto",
"common/dkg",
"common/crypto/dkg",
"common/execute",
"common/inclusion-probability",
"common/ledger",
"common/logging",
"common/mixnode-common",
"common/network-defaults",
"common/nonexhaustive-delayqueue",
@@ -61,7 +62,6 @@ members = [
"common/nymsphinx/params",
"common/nymsphinx/types",
"common/pemstore",
"common/socks5-client-core",
"common/socks5/proxy-helpers",
"common/socks5/requests",
"common/statistics",
@@ -69,6 +69,7 @@ members = [
"common/topology",
"common/types",
"common/wasm-utils",
"common/completions",
"explorer-api",
"gateway",
"gateway/gateway-requests",
@@ -96,38 +97,18 @@ default-members = [
"explorer-api",
]
exclude = ["explorer", "contracts", "clients/webassembly", "nym-wallet", "nym-connect/mobile/src-tauri", "nym-connect/desktop", "cpu-cycles"]
exclude = ["explorer", "contracts", "clients/webassembly", "nym-wallet", "nym-connect", "nym-connect-android"]
[workspace.package]
authors = ["Nym Technologies SA"]
repository = "https://github.com/nymtech/nym"
homepage = "https://nymtech.net"
documentation = "https://nymtech.net"
edition = "2021"
license = "Apache-2.0"
[workspace.dependencies]
async-trait = "0.1.64"
bip39 = { version = "2.0.0", features = ["zeroize"] }
cfg-if = "1.0.0"
cosmwasm-derive = "=1.0.0"
cosmwasm-schema = "=1.0.0"
cosmwasm-std = "=1.0.0"
cosmwasm-storage = "=1.0.0"
cw-utils = "=0.13.4"
cw-storage-plus = "=0.13.4"
cw2 = { version = "=0.13.4" }
cw3 = { version = "=0.13.4" }
cw3-fixed-multisig = { version = "=0.13.4" }
cw4 = { version = "=0.13.4" }
dotenvy = "0.15.6"
lazy_static = "1.4.0"
async-trait = "0.1.63"
log = "0.4"
once_cell = "1.7.2"
rand = "0.8.5"
thiserror = "1.0.38"
serde = "1.0.152"
serde_json = "1.0.91"
tap = "1.0.1"
thiserror = "1.0.38"
tokio = "1.24.1"
url = "2.2"
+75 -83
View File
@@ -1,116 +1,108 @@
# Default target
all: test
test: clippy-all cargo-test wasm fmt
test-all: test cargo-test-expensive
no-clippy: build cargo-test wasm fmt
happy: fmt clippy-happy test
clippy-all: clippy-main clippy-all-contracts clippy-all-wallet clippy-all-connect clippy-all-wasm-client
clippy-happy: clippy-happy-main clippy-happy-contracts clippy-happy-wallet clippy-happy-connect
cargo-test: test-main test-contracts test-wallet test-connect
cargo-test-expensive: test-main-expensive test-contracts-expensive test-wallet-expensive test-connect-expensive
build: build-contracts build-wallet build-main build-connect build-wasm-client
fmt: fmt-main fmt-contracts fmt-wallet fmt-connect fmt-wasm-client
# Building release binaries is a little manual as we can't just build --release
# on all workspaces.
build-release: build-release-main wasm
clippy-happy-main:
cargo clippy
# -----------------------------------------------------------------------------
# Define targets for a given workspace
# $(1): name
# $(2): path to workspace
# $(3): extra arguments to cargo
# -----------------------------------------------------------------------------
define add_cargo_workspace
clippy-happy-contracts:
cargo clippy --manifest-path contracts/Cargo.toml --target wasm32-unknown-unknown
clippy-happy-$(1):
cargo clippy --manifest-path $(2)/Cargo.toml $(3)
clippy-happy-wallet:
cargo clippy --manifest-path nym-wallet/Cargo.toml
clippy-$(1):
cargo clippy --manifest-path $(2)/Cargo.toml --workspace $(3) -- -D warnings
clippy-happy-connect:
cargo clippy --manifest-path nym-connect/Cargo.toml
clippy-examples-$(1):
cargo clippy --manifest-path $(2)/Cargo.toml --workspace --examples -- -D warnings
clippy-main:
cargo clippy --workspace -- -D warnings
check-$(1):
cargo check --manifest-path $(2)/Cargo.toml --workspace $(3)
clippy-wasm:
cargo clippy --manifest-path clients/webassembly/Cargo.toml --target wasm32-unknown-unknown --workspace -- -D warnings
test-$(1):
cargo test --manifest-path $(2)/Cargo.toml --workspace
test-expensive-$(1):
cargo test --manifest-path $(2)/Cargo.toml --workspace -- --ignored
clippy-all-contracts:
cargo clippy --workspace --manifest-path contracts/Cargo.toml --all-features --target wasm32-unknown-unknown -- -D warnings
build-$(1):
cargo build --manifest-path $(2)/Cargo.toml --workspace $(3)
clippy-all-wallet:
cargo clippy --workspace --manifest-path nym-wallet/Cargo.toml --all-features -- -D warnings
build-examples-$(1):
cargo build --manifest-path $(2)/Cargo.toml --workspace --examples
clippy-all-connect:
cargo clippy --workspace --manifest-path nym-connect/Cargo.toml --all-features -- -D warnings
build-release-$(1):
cargo build --manifest-path $(2)/Cargo.toml --workspace --release $(3)
clippy-all-wasm-client:
cargo clippy --workspace --manifest-path clients/webassembly/Cargo.toml --all-features --target wasm32-unknown-unknown -- -D warnings
fmt-$(1):
cargo fmt --manifest-path $(2)/Cargo.toml --all
test-main:
cargo test --workspace
clippy-happy: clippy-happy-$(1)
clippy-all: clippy-$(1) clippy-examples-$(1)
check: check-$(1)
cargo-test: test-$(1)
cargo-test-expensive: test-expensive-$(1)
build: build-$(1) build-$(1)-examples
build-release-all: build-release-$(1)
fmt: fmt-$(1)
test-main-expensive:
cargo test --workspace -- --ignored
endef
test-contracts:
cargo test --manifest-path contracts/Cargo.toml --all-features
# -----------------------------------------------------------------------------
# Rust workspaces
# -----------------------------------------------------------------------------
test-contracts-expensive:
cargo test --manifest-path contracts/Cargo.toml --all-features -- --ignored
# Generate targets for the various cargo workspaces
test-wallet:
cargo test --manifest-path nym-wallet/Cargo.toml --all-features
$(eval $(call add_cargo_workspace,main,.))
$(eval $(call add_cargo_workspace,contracts,contracts,--target wasm32-unknown-unknown))
$(eval $(call add_cargo_workspace,wasm-client,clients/webassembly,--target wasm32-unknown-unknown))
$(eval $(call add_cargo_workspace,wallet,nym-wallet,))
$(eval $(call add_cargo_workspace,connect,nym-connect/desktop))
ifdef NYM_MOBILE
$(eval $(call add_cargo_workspace,connect-mobile,nym-connect/mobile/src-tauri))
endif
test-wallet-expensive:
cargo test --manifest-path nym-wallet/Cargo.toml --all-features -- --ignored
# -----------------------------------------------------------------------------
# Convenience targets for crates that are already part of the main workspace
# -----------------------------------------------------------------------------
test-connect:
cargo test --manifest-path nym-connect/Cargo.toml --all-features
test-connect-expensive:
cargo test --manifest-path nym-connect/Cargo.toml --all-features -- --ignored
build-main:
cargo build --workspace
build-contracts:
cargo build --manifest-path contracts/Cargo.toml --workspace
build-wallet:
cargo build --manifest-path nym-wallet/Cargo.toml --workspace
build-connect:
cargo build --manifest-path nym-connect/Cargo.toml --workspace
build-explorer-api:
cargo build -p explorer-api
cargo build --manifest-path explorer-api/Cargo.toml --workspace
build-wasm-client:
cargo build --manifest-path clients/webassembly/Cargo.toml --workspace --target wasm32-unknown-unknown
build-nym-cli:
cargo build -p nym-cli --release
cargo build --release --manifest-path tools/nym-cli/Cargo.toml
# -----------------------------------------------------------------------------
# Build contracts ready for deploy
# -----------------------------------------------------------------------------
fmt-main:
cargo fmt --all
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release
VESTING_CONTRACT=$(CONTRACTS_OUT_DIR)/vesting_contract.wasm
MIXNET_CONTRACT=$(CONTRACTS_OUT_DIR)/mixnet_contract.wasm
SERVICE_PROVIDER_DIRECTORY_CONTRACT=$(CONTRACTS_OUT_DIR)/nym_service_provider_directory.wasm
fmt-contracts:
cargo fmt --manifest-path contracts/Cargo.toml --all
wasm: wasm-build wasm-opt
fmt-wallet:
cargo fmt --manifest-path nym-wallet/Cargo.toml --all
wasm-build:
fmt-connect:
cargo fmt --manifest-path nym-connect/Cargo.toml --all
fmt-wasm-client:
cargo fmt --manifest-path clients/webassembly/Cargo.toml --all
wasm:
RUSTFLAGS='-C link-arg=-s' cargo build --manifest-path contracts/Cargo.toml --release --target wasm32-unknown-unknown
wasm-opt:
wasm-opt --disable-sign-ext -Os $(VESTING_CONTRACT) -o $(VESTING_CONTRACT)
wasm-opt --disable-sign-ext -Os $(MIXNET_CONTRACT) -o $(MIXNET_CONTRACT)
wasm-opt --disable-sign-ext -Os $(SERVICE_PROVIDER_DIRECTORY_CONTRACT) -o $(SERVICE_PROVIDER_DIRECTORY_CONTRACT)
# -----------------------------------------------------------------------------
# Misc
# -----------------------------------------------------------------------------
# NOTE: this seems deprecated an not needed anymore?
mixnet-opt: wasm
cd contracts/mixnet && make opt
@@ -118,5 +110,5 @@ generate-typescript:
cd tools/ts-rs-cli && cargo run && cd ../..
yarn types:lint:fix
run-api-tests:
cd nym-api/tests/functional_test && yarn test:qa
run-validator-tests:
cd nym-api/tests/functional_test && yarn test:qa
+2 -2
View File
@@ -16,12 +16,12 @@ The platform is composed of multiple Rust crates. Top-level executable binary cr
* nym-wallet - a desktop wallet implemented using the [Tauri](https://tauri.studio/en/docs/about/intro) framework.
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0)
[![Build Status](https://img.shields.io/github/actions/workflow/status/nymtech/nym/build.yml?branch=develop&style=for-the-badge&logo=github-actions)](https://github.com/nymtech/nym/actions?query=branch%3Adevelop)
[![Build Status](https://img.shields.io/github/workflow/status/nymtech/nym/Continuous%20integration/develop?style=for-the-badge&logo=github-actions)](https://github.com/nymtech/nym/actions?query=branch%3Adevelop)
### Building
Platform build instructions are available on [our docs site](https://nymtech.net/docs/binaries/building-nym.html).
Platform build instructions are available on [our docs site](https://nymtech.net/docs/stable/run-nym-nodes/build-nym).
Wallet build instructions are also available on [our docs site](https://nymtech.net/docs/stable/nym-apps/wallet#for-developers).
### Developing
+3 -3
View File
@@ -3,8 +3,8 @@ Critical bug or security issue 💥
If you're here because you're trying to figure out how to notify us of a security issue, go to Discord, and alert the core engineers:
Dave Hrycyszyn futurechimp#5430
Drazen Urch drazen#4873
Jedrzej Stuczynski "Jedrzej | Nym#5666"
Fran Arbanas | franarbanas#0995
Mark Sinclair | marknym#8088
Please avoid opening public issues on GitHub that contain information about a potential security vulnerability as this makes it difficult to reduce the impact and harm of valid security issues.
Please avoid opening public issues on GitHub that contain information about a potential security vulnerability as this makes it difficult to reduce the impact and harm of valid security issues.
@@ -1,6 +1,6 @@
[package]
name = "nym-client-core"
version = "1.1.14"
name = "client-core"
version = "1.1.8"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2021"
rust-version = "1.66"
@@ -8,40 +8,34 @@ rust-version = "1.66"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait = { workspace = true }
async-trait = { version = "0.1.58" }
dirs = "4.0"
dashmap = "5.4.0"
futures = "0.3"
humantime-serde = "1.0"
log = { workspace = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.89"
tap = "1.0.1"
thiserror = "1.0.34"
url = { version ="2.2", features = ["serde"] }
tungstenite = { version = "0.13.0", default-features = false }
tokio = { version = "1.24.1", features = ["macros"]}
time = "0.3.17"
# internal
nym-bandwidth-controller = { path = "../bandwidth-controller" }
nym-config = { path = "../config" }
nym-crypto = { path = "../crypto" }
nym-gateway-client = { path = "../client-libs/gateway-client" }
config = { path = "../../common/config" }
client-connections = { path = "../../common/client-connections" }
crypto = { path = "../../common/crypto" }
gateway-client = { path = "../../common/client-libs/gateway-client" }
#gateway-client = { path = "../../common/client-libs/gateway-client", default-features = false, features = ["wasm", "coconut"] }
nym-gateway-requests = { path = "../../gateway/gateway-requests" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" }
nym-pemstore = { path = "../pemstore" }
nym-topology = { path = "../topology" }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
nym-task = { path = "../task" }
nym-credential-storage = { path = "../credential-storage" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.nym-validator-client]
path = "../client-libs/validator-client"
features = ["nyxd-client"]
gateway-requests = { path = "../../gateway/gateway-requests" }
nonexhaustive-delayqueue = { path = "../../common/nonexhaustive-delayqueue" }
nymsphinx = { path = "../../common/nymsphinx" }
pemstore = { path = "../../common/pemstore" }
topology = { path = "../../common/topology" }
validator-client = { path = "../../common/client-libs/validator-client", default-features = false }
task = { path = "../../common/task" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream]
version = "0.1.11"
@@ -51,10 +45,7 @@ features = ["time"]
version = "1.24.1"
features = ["time"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-tungstenite]
version = "0.14"
[target."cfg(all(not(target_arch = \"wasm32\"),not(target_os = \"android\")))".dependencies.sqlx]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx]
version = "0.6.2"
features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"]
optional = true
@@ -74,8 +65,7 @@ version = "0.2.4"
features = ["futures"]
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils]
path = "../wasm-utils"
features = ["websocket"]
path = "../../common/wasm-utils"
[target."cfg(target_arch = \"wasm32\")".dependencies.time]
version = "0.3.17"
@@ -91,5 +81,5 @@ sqlx = { version = "0.6.2", features = ["runtime-tokio-rustls", "sqlite", "macro
[features]
default = []
fs-surb-storage = ["sqlx"]
wasm = ["nym-gateway-client/wasm"]
wasm = ["gateway-client/wasm"]
@@ -5,11 +5,7 @@ use crate::{client::replies::reply_storage, config::DebugConfig};
pub fn setup_empty_reply_surb_backend(debug_config: &DebugConfig) -> reply_storage::Empty {
reply_storage::Empty {
min_surb_threshold: debug_config
.reply_surbs
.minimum_reply_surb_storage_threshold,
max_surb_threshold: debug_config
.reply_surbs
.maximum_reply_surb_storage_threshold,
min_surb_threshold: debug_config.minimum_reply_surb_storage_threshold,
max_surb_threshold: debug_config.maximum_reply_surb_storage_threshold,
}
}
@@ -1,7 +1,6 @@
// Copyright 2022-2023 - Nym Technologies SA <contact@nymtech.net>
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::received_buffer::ReceivedBufferMessage;
use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender};
use crate::client::key_manager::KeyManager;
@@ -16,61 +15,47 @@ use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyCon
use crate::client::replies::reply_storage::{
CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys,
};
use crate::client::topology_control::nym_api_provider::NymApiTopologyProvider;
use crate::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
};
use crate::config::{Config, DebugConfig, GatewayEndpointConfig};
use crate::error::ClientCoreError;
use crate::spawn_future;
use client_connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use crypto::asymmetric::{encryption, identity};
use futures::channel::mpsc;
use log::{debug, info};
use nym_bandwidth_controller::BandwidthController;
use nym_crypto::asymmetric::{encryption, identity};
use nym_gateway_client::{
use gateway_client::bandwidth::BandwidthController;
#[cfg(target_arch = "wasm32")]
use gateway_client::wasm_mockups::CosmWasmClient;
use gateway_client::{
AcknowledgementReceiver, AcknowledgementSender, GatewayClient, MixnetMessageReceiver,
MixnetMessageSender,
};
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver};
use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use nym_task::{TaskClient, TaskManager};
use nym_topology::provider_trait::TopologyProvider;
use log::{debug, info};
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::addressing::nodes::NodeIdentity;
use nymsphinx::receiver::ReconstructedMessage;
use std::sync::Arc;
use std::time::Duration;
use tap::TapFallible;
use task::{TaskClient, TaskManager};
use url::Url;
use nym_credential_storage::storage::Storage;
#[cfg(not(target_arch = "wasm32"))]
use nym_validator_client::nyxd::traits::DkgQueryClient;
use validator_client::nyxd::CosmWasmClient;
#[cfg(target_arch = "wasm32")]
use nym_bandwidth_controller::wasm_mockups::DkgQueryClient;
use super::received_buffer::ReceivedBufferMessage;
#[cfg(all(not(target_arch = "wasm32"), feature = "fs-surb-storage"))]
pub mod non_wasm_helpers;
pub mod helpers;
#[derive(Clone)]
pub struct ClientInput {
pub connection_command_sender: ConnectionCommandSender,
pub input_sender: InputMessageSender,
}
impl ClientInput {
pub async fn send(
&self,
message: InputMessage,
) -> Result<(), tokio::sync::mpsc::error::SendError<InputMessage>> {
self.input_sender.send(message).await
}
}
#[derive(Clone)]
pub struct ClientOutput {
pub received_buffer_request_sender: ReceivedBufferRequestSender,
}
@@ -91,11 +76,9 @@ impl ClientOutput {
}
}
#[derive(Clone, Debug)]
pub struct ClientState {
pub shared_lane_queue_lengths: LaneQueueLengths,
pub reply_controller_sender: ReplyControllerSender,
pub topology_accessor: TopologyAccessor,
}
pub enum ClientInputStatus {
@@ -152,7 +135,7 @@ impl From<bool> for CredentialsToggle {
}
}
pub struct BaseClientBuilder<'a, B, C, St: Storage> {
pub struct BaseClientBuilder<'a, B, C: Clone> {
// due to wasm limitations I had to split it like this : (
gateway_config: &'a GatewayEndpointConfig,
debug_config: &'a DebugConfig,
@@ -160,23 +143,21 @@ pub struct BaseClientBuilder<'a, B, C, St: Storage> {
nym_api_endpoints: Vec<Url>,
reply_storage_backend: B,
custom_topology_provider: Option<Box<dyn TopologyProvider>>,
bandwidth_controller: Option<BandwidthController<C, St>>,
bandwidth_controller: Option<BandwidthController<C>>,
key_manager: KeyManager,
}
impl<'a, B, C, St> BaseClientBuilder<'a, B, C, St>
impl<'a, B, C> BaseClientBuilder<'a, B, C>
where
B: ReplyStorageBackend + Send + Sync + 'static,
C: DkgQueryClient + Sync + Send + 'static,
St: Storage + 'static,
C: CosmWasmClient + Sync + Send + Clone + 'static,
{
pub fn new_from_base_config<T>(
base_config: &'a Config<T>,
key_manager: KeyManager,
bandwidth_controller: Option<BandwidthController<C, St>>,
bandwidth_controller: Option<BandwidthController<C>>,
reply_storage_backend: B,
) -> BaseClientBuilder<'a, B, C, St> {
) -> BaseClientBuilder<'a, B, C> {
BaseClientBuilder {
gateway_config: base_config.get_gateway_endpoint_config(),
debug_config: base_config.get_debug_config(),
@@ -185,7 +166,6 @@ where
bandwidth_controller,
reply_storage_backend,
key_manager,
custom_topology_provider: None,
}
}
@@ -193,28 +173,22 @@ where
gateway_config: &'a GatewayEndpointConfig,
debug_config: &'a DebugConfig,
key_manager: KeyManager,
bandwidth_controller: Option<BandwidthController<C, St>>,
bandwidth_controller: Option<BandwidthController<C>>,
reply_storage_backend: B,
credentials_toggle: CredentialsToggle,
nym_api_endpoints: Vec<Url>,
) -> BaseClientBuilder<'a, B, C, St> {
) -> BaseClientBuilder<'a, B, C> {
BaseClientBuilder {
gateway_config,
debug_config,
disabled_credentials: credentials_toggle.is_disabled(),
nym_api_endpoints,
reply_storage_backend,
custom_topology_provider: None,
bandwidth_controller,
key_manager,
}
}
pub fn with_topology_provider(mut self, provider: Box<dyn TopologyProvider>) -> Self {
self.custom_topology_provider = Some(provider);
self
}
pub fn as_mix_recipient(&self) -> Recipient {
Recipient::new(
*self.key_manager.identity_keypair().public_key(),
@@ -237,16 +211,21 @@ where
) {
info!("Starting loop cover traffic stream...");
let stream = LoopCoverTrafficStream::new(
let mut stream = LoopCoverTrafficStream::new(
ack_key,
debug_config.acknowledgements.average_ack_delay,
debug_config.average_ack_delay,
debug_config.average_packet_delay,
debug_config.loop_cover_traffic_average_delay,
mix_tx,
self_address,
topology_accessor,
debug_config.traffic,
debug_config.cover_traffic,
);
if let Some(size) = debug_config.use_extended_packet_size {
log::debug!("Setting extended packet size: {:?}", size);
stream.set_custom_packet_size(size.into());
}
stream.start_with_shutdown(shutdown);
}
@@ -292,15 +271,14 @@ where
shutdown: TaskClient,
) {
info!("Starting received messages buffer controller...");
let controller: ReceivedMessagesBufferController<SphinxMessageReceiver> =
ReceivedMessagesBufferController::new(
local_encryption_keypair,
query_receiver,
mixnet_receiver,
reply_key_storage,
reply_controller_sender,
);
controller.start_with_shutdown(shutdown)
ReceivedMessagesBufferController::new(
local_encryption_keypair,
query_receiver,
mixnet_receiver,
reply_key_storage,
reply_controller_sender,
)
.start_with_shutdown(shutdown)
}
async fn start_gateway_client(
@@ -308,14 +286,14 @@ where
mixnet_message_sender: MixnetMessageSender,
ack_sender: AcknowledgementSender,
shutdown: TaskClient,
) -> Result<GatewayClient<C, St>, ClientCoreError> {
) -> Result<GatewayClient<C>, ClientCoreError> {
let gateway_id = self.gateway_config.gateway_id.clone();
if gateway_id.is_empty() {
return Err(ClientCoreError::GatewayIdUnknown);
}
let gateway_address = self.gateway_config.gateway_listener.clone();
if gateway_address.is_empty() {
return Err(ClientCoreError::GatewayAddressUnknown);
return Err(ClientCoreError::GatwayAddressUnknown);
}
let gateway_identity = identity::PublicKey::from_base58_string(gateway_id)
@@ -325,7 +303,6 @@ where
let shared_key = if self.key_manager.is_gateway_key_set() {
Some(self.key_manager.gateway_shared_key())
} else {
log::info!("Gateway key not set! Will proceed anyway.");
None
};
@@ -336,9 +313,7 @@ where
shared_key,
mixnet_message_sender,
ack_sender,
self.debug_config
.gateway_connection
.gateway_response_timeout,
self.debug_config.gateway_response_timeout,
self.bandwidth_controller.take(),
shutdown,
);
@@ -354,38 +329,25 @@ where
Ok(gateway_client)
}
fn setup_topology_provider(
custom_provider: Option<Box<dyn TopologyProvider>>,
nym_api_urls: Vec<Url>,
) -> Box<dyn TopologyProvider> {
// if no custom provider was ... provided ..., create one using nym-api
custom_provider.unwrap_or_else(|| {
Box::new(NymApiTopologyProvider::new(
nym_api_urls,
env!("CARGO_PKG_VERSION").to_string(),
))
})
}
// future responsible for periodically polling directory server and updating
// the current global view of topology
async fn start_topology_refresher(
topology_provider: Box<dyn TopologyProvider>,
nym_api_urls: Vec<Url>,
refresh_rate: Duration,
topology_accessor: TopologyAccessor,
shutdown: TaskClient,
) -> Result<(), ClientCoreError> {
let topology_refresher_config = TopologyRefresherConfig::new(refresh_rate);
let mut topology_refresher = TopologyRefresher::new(
topology_refresher_config,
topology_accessor,
topology_provider,
let topology_refresher_config = TopologyRefresherConfig::new(
nym_api_urls,
refresh_rate,
env!("CARGO_PKG_VERSION").to_string(),
);
let mut topology_refresher =
TopologyRefresher::new(topology_refresher_config, topology_accessor);
// before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view
info!("Obtaining initial network topology");
topology_refresher.try_refresh().await;
topology_refresher.refresh().await;
if let Err(err) = topology_refresher.ensure_topology_is_routable().await {
log::error!(
@@ -405,7 +367,7 @@ where
// over it. Perhaps GatewayClient needs to be thread-shareable or have some channel for
// requests?
fn start_mix_traffic_controller(
gateway_client: GatewayClient<C, St>,
gateway_client: GatewayClient<C>,
shutdown: TaskClient,
) -> BatchMixMessageSender {
info!("Starting mix traffic controller...");
@@ -494,13 +456,9 @@ where
)
.await?;
let topology_provider = Self::setup_topology_provider(
self.custom_topology_provider.take(),
self.nym_api_endpoints,
);
Self::start_topology_refresher(
topology_provider,
self.debug_config.topology.topology_refresh_rate,
self.nym_api_endpoints.clone(),
self.debug_config.topology_refresh_rate,
shared_topology_accessor.clone(),
task_manager.subscribe(),
)
@@ -530,12 +488,17 @@ where
// primarily to throttle incoming connections (e.g socks5 for attached network-requesters)
let shared_lane_queue_lengths = LaneQueueLengths::new();
let controller_config = real_messages_control::Config::new(
let mut controller_config = real_messages_control::Config::new(
self.debug_config,
self.key_manager.ack_key(),
self_address,
);
if let Some(size) = self.debug_config.use_extended_packet_size {
log::debug!("Setting extended packet size: {:?}", size);
controller_config.set_custom_packet_size(size.into());
}
Self::start_real_traffic_controller(
controller_config,
shared_topology_accessor.clone(),
@@ -550,16 +513,12 @@ where
task_manager.subscribe(),
);
if !self
.debug_config
.cover_traffic
.disable_loop_cover_traffic_stream
{
if !self.debug_config.disable_loop_cover_traffic_stream {
Self::start_cover_traffic_stream(
self.debug_config,
self.key_manager.ack_key(),
self_address,
shared_topology_accessor.clone(),
shared_topology_accessor,
sphinx_message_sender,
task_manager.subscribe(),
);
@@ -583,7 +542,6 @@ where
client_state: ClientState {
shared_lane_queue_lengths,
reply_controller_sender,
topology_accessor: shared_topology_accessor,
},
task_manager,
})
@@ -30,12 +30,8 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
// it will only be happening on the very first run and in practice won't incur huge
// costs since the storage is going to be empty
let mem_store = CombinedReplyStorage::new(
debug_config
.reply_surbs
.minimum_reply_surb_storage_threshold,
debug_config
.reply_surbs
.maximum_reply_surb_storage_threshold,
debug_config.minimum_reply_surb_storage_threshold,
debug_config.maximum_reply_surb_storage_threshold,
);
storage_backend
.init_fresh(&mem_store)
@@ -50,12 +46,8 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
fn setup_inactive_backend(debug_config: &DebugConfig) -> fs_backend::Backend {
info!("creating inactive surb database");
fs_backend::Backend::new_inactive(
debug_config
.reply_surbs
.minimum_reply_surb_storage_threshold,
debug_config
.reply_surbs
.maximum_reply_surb_storage_threshold,
debug_config.minimum_reply_surb_storage_threshold,
debug_config.maximum_reply_surb_storage_threshold,
)
}
@@ -3,15 +3,15 @@
use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::topology_control::TopologyAccessor;
use crate::{config, spawn_future};
use crate::spawn_future;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet;
use nym_sphinx::params::PacketSize;
use nym_sphinx::utils::sample_poisson_duration;
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::cover::generate_loop_cover_packet;
use nymsphinx::params::PacketSize;
use nymsphinx::utils::sample_poisson_duration;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
@@ -34,8 +34,11 @@ where
/// Average delay an acknowledgement packet is going to get delay at a single mixnode.
average_ack_delay: Duration,
/// Defines configuration options related to cover traffic.
cover_traffic: config::CoverTraffic,
/// Average delay a data packet is going to get delay at a single mixnode.
average_packet_delay: Duration,
/// Average delay between sending subsequent cover packets.
average_cover_message_sending_delay: Duration,
/// Internal state, determined by `average_message_sending_delay`,
/// used to keep track of when a next packet should be sent out.
@@ -58,11 +61,8 @@ where
/// Accessor to the common instance of network topology.
topology_access: TopologyAccessor,
/// Primary predefined packet size used for the loop cover messages.
primary_packet_size: PacketSize,
/// Optional secondary predefined packet size used for the loop cover messages.
secondary_packet_size: Option<PacketSize>,
/// Predefined packet size used for the loop cover messages.
packet_size: PacketSize,
}
impl<R> Stream for LoopCoverTrafficStream<R>
@@ -83,7 +83,7 @@ where
// we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline
let avg_delay = self.cover_traffic.loop_cover_traffic_average_delay;
let avg_delay = self.average_cover_message_sending_delay;
let next_poisson_delay = sample_poisson_duration(&mut self.rng, avg_delay);
// The next interval value is `next_poisson_delay` after the one that just
@@ -107,14 +107,15 @@ where
// obviously when we finally make shared rng that is on 'higher' level, this should become
// generic `R`
impl LoopCoverTrafficStream<OsRng> {
#[allow(clippy::too_many_arguments)]
pub fn new(
ack_key: Arc<AckKey>,
average_ack_delay: Duration,
average_packet_delay: Duration,
average_cover_message_sending_delay: Duration,
mix_tx: BatchMixMessageSender,
our_full_destination: Recipient,
topology_access: TopologyAccessor,
traffic_config: config::Traffic,
cover_config: config::CoverTraffic,
) -> Self {
let rng = OsRng;
@@ -127,17 +128,21 @@ impl LoopCoverTrafficStream<OsRng> {
LoopCoverTrafficStream {
ack_key,
average_ack_delay,
cover_traffic: cover_config,
average_packet_delay,
average_cover_message_sending_delay,
next_delay,
mix_tx,
our_full_destination,
rng,
topology_access,
primary_packet_size: traffic_config.primary_packet_size,
secondary_packet_size: traffic_config.secondary_packet_size,
packet_size: Default::default(),
}
}
pub fn set_custom_packet_size(&mut self, packet_size: PacketSize) {
self.packet_size = packet_size;
}
fn set_next_delay(&mut self, amount: Duration) {
#[cfg(not(target_arch = "wasm32"))]
let next_delay = Box::pin(time::sleep(amount));
@@ -148,28 +153,9 @@ impl LoopCoverTrafficStream<OsRng> {
self.next_delay = next_delay;
}
fn loop_cover_message_size(&mut self) -> PacketSize {
let Some(secondary_packet_size) = self.secondary_packet_size else {
return self.primary_packet_size
};
let use_primary = self
.rng
.gen_bool(self.cover_traffic.cover_traffic_primary_size_ratio);
if use_primary {
self.primary_packet_size
} else {
secondary_packet_size
}
}
async fn on_new_message(&mut self) {
trace!("next cover message!");
let cover_traffic_packet_size = self.loop_cover_message_size();
trace!("the next loop cover message will be put in a {cover_traffic_packet_size} packet");
// TODO for way down the line: in very rare cases (during topology update) we might have
// to wait a really tiny bit before actually obtaining the permit hence messing with our
// poisson delay, but is it really a problem?
@@ -192,8 +178,8 @@ impl LoopCoverTrafficStream<OsRng> {
&self.ack_key,
&self.our_full_destination,
self.average_ack_delay,
self.cover_traffic.loop_cover_traffic_average_delay,
cover_traffic_packet_size,
self.average_packet_delay,
self.packet_size,
)
.expect("Somehow failed to generate a loop cover message with a valid topology");
@@ -203,6 +189,10 @@ impl LoopCoverTrafficStream<OsRng> {
// This isn't a problem, if the channel is full means we're already sending the
// max amount of messages downstream can handle.
log::debug!("Failed to send cover message - channel full");
// However it's still useful to alert the user that the gateway or the link to
// the gateway can't keep up. Either due to insufficient bandwidth on the
// client side, or that the gateway is overloaded.
log::warn!("Failed to send sphinx packet - gateway or connection to gatway can't keep up");
}
TrySendError::Closed(_) => {
log::warn!("Failed to send cover message - channel closed");
@@ -223,18 +213,10 @@ impl LoopCoverTrafficStream<OsRng> {
tokio::task::yield_now().await;
}
pub fn start_with_shutdown(mut self, mut shutdown: nym_task::TaskClient) {
if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with
// so panic and review the code that lead to this branch
panic!("attempted to start LoopCoverTrafficStream while config explicitly disabled it.")
}
pub fn start_with_shutdown(mut self, mut shutdown: task::TaskClient) {
// we should set initial delay only when we actually start the stream
let sampled = sample_poisson_duration(
&mut self.rng,
self.cover_traffic.loop_cover_traffic_average_delay,
);
let sampled =
sample_poisson_duration(&mut self.rng, self.average_cover_message_sending_delay);
self.set_next_delay(sampled);
spawn_future(async move {
@@ -1,6 +1,6 @@
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_task::connections::TransmissionLane;
use client_connections::TransmissionLane;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
pub type InputMessageSender = tokio::sync::mpsc::Sender<InputMessage>;
pub type InputMessageReceiver = tokio::sync::mpsc::Receiver<InputMessage>;
@@ -77,12 +77,4 @@ impl InputMessage {
lane,
}
}
pub fn lane(&self) -> &TransmissionLane {
match self {
InputMessage::Regular { lane, .. }
| InputMessage::Anonymous { lane, .. }
| InputMessage::Reply { lane, .. } => lane,
}
}
}
@@ -2,10 +2,10 @@
// SPDX-License-Identifier: Apache-2.0
use crate::config::persistence::key_pathfinder::ClientKeyPathfinder;
use crypto::asymmetric::{encryption, identity};
use gateway_requests::registration::handshake::SharedKeys;
use log::*;
use nym_crypto::asymmetric::{encryption, identity};
use nym_gateway_requests::registration::handshake::SharedKeys;
use nym_sphinx::acknowledgements::AckKey;
use nymsphinx::acknowledgements::AckKey;
use rand::{CryptoRng, RngCore};
use std::io;
use std::sync::Arc;
@@ -72,17 +72,17 @@ impl KeyManager {
/// Loads previously stored client keys from the disk.
fn load_client_keys(client_pathfinder: &ClientKeyPathfinder) -> io::Result<Self> {
let identity_keypair: identity::KeyPair =
nym_pemstore::load_keypair(&nym_pemstore::KeyPairPath::new(
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
))?;
let encryption_keypair: encryption::KeyPair =
nym_pemstore::load_keypair(&nym_pemstore::KeyPairPath::new(
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
))?;
let ack_key: AckKey = nym_pemstore::load_key(client_pathfinder.ack_key())?;
let ack_key: AckKey = pemstore::load_key(client_pathfinder.ack_key())?;
Ok(KeyManager {
identity_keypair: Arc::new(identity_keypair),
@@ -98,7 +98,7 @@ impl KeyManager {
let mut key_manager = Self::load_client_keys(client_pathfinder)?;
let gateway_shared_key: SharedKeys =
nym_pemstore::load_key(client_pathfinder.gateway_shared_key())?;
pemstore::load_key(client_pathfinder.gateway_shared_key())?;
key_manager.gateway_shared_key = Some(Arc::new(gateway_shared_key));
@@ -113,7 +113,7 @@ impl KeyManager {
let mut key_manager = Self::load_client_keys(client_pathfinder)?;
let gateway_shared_key: Result<SharedKeys, io::Error> =
nym_pemstore::load_key(client_pathfinder.gateway_shared_key());
pemstore::load_key(client_pathfinder.gateway_shared_key());
// It's ok if the gateway key was not found
let gateway_shared_key = match gateway_shared_key {
@@ -132,27 +132,27 @@ impl KeyManager {
// it is done so for the consistency sake so that you wouldn't require an rng instance
// during `load_keys` to generate the said key.
pub fn store_keys(&self, client_pathfinder: &ClientKeyPathfinder) -> io::Result<()> {
nym_pemstore::store_keypair(
pemstore::store_keypair(
self.identity_keypair.as_ref(),
&nym_pemstore::KeyPairPath::new(
&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
),
)?;
nym_pemstore::store_keypair(
pemstore::store_keypair(
self.encryption_keypair.as_ref(),
&nym_pemstore::KeyPairPath::new(
&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
),
)?;
nym_pemstore::store_key(self.ack_key.as_ref(), client_pathfinder.ack_key())?;
pemstore::store_key(self.ack_key.as_ref(), client_pathfinder.ack_key())?;
match self.gateway_shared_key.as_ref() {
None => debug!("No gateway shared key available to store!"),
Some(gate_key) => {
nym_pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
}
}
@@ -168,7 +168,7 @@ impl KeyManager {
))
}
Some(gate_key) => {
nym_pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
}
}
@@ -2,16 +2,13 @@
// SPDX-License-Identifier: Apache-2.0
use crate::spawn_future;
use log::*;
use nym_gateway_client::GatewayClient;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_credential_storage::storage::Storage;
#[cfg(not(target_arch = "wasm32"))]
use nym_validator_client::nyxd::traits::DkgQueryClient;
#[cfg(target_arch = "wasm32")]
use nym_bandwidth_controller::wasm_mockups::DkgQueryClient;
use gateway_client::wasm_mockups::CosmWasmClient;
use gateway_client::GatewayClient;
use log::*;
use nymsphinx::forwarding::packet::MixPacket;
#[cfg(not(target_arch = "wasm32"))]
use validator_client::nyxd::CosmWasmClient;
pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>;
pub type BatchMixMessageReceiver = tokio::sync::mpsc::Receiver<Vec<MixPacket>>;
@@ -20,10 +17,10 @@ pub type BatchMixMessageReceiver = tokio::sync::mpsc::Receiver<Vec<MixPacket>>;
pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32;
const MAX_FAILURE_COUNT: usize = 100;
pub struct MixTrafficController<C, St: Storage> {
pub struct MixTrafficController<C: Clone> {
// TODO: most likely to be replaced by some higher level construct as
// later on gateway_client will need to be accessible by other entities
gateway_client: GatewayClient<C, St>,
gateway_client: GatewayClient<C>,
mix_rx: BatchMixMessageReceiver,
// TODO: this is temporary work-around.
@@ -31,14 +28,13 @@ pub struct MixTrafficController<C, St: Storage> {
consecutive_gateway_failure_count: usize,
}
impl<C, St> MixTrafficController<C, St>
impl<C> MixTrafficController<C>
where
C: DkgQueryClient + Sync + Send + 'static,
St: Storage + 'static,
C: CosmWasmClient + Sync + Send + Clone + 'static,
{
pub fn new(
gateway_client: GatewayClient<C, St>,
) -> (MixTrafficController<C, St>, BatchMixMessageSender) {
gateway_client: GatewayClient<C>,
) -> (MixTrafficController<C>, BatchMixMessageSender) {
let (sphinx_message_sender, sphinx_message_receiver) =
tokio::sync::mpsc::channel(MIX_MESSAGE_RECEIVER_BUFFER_SIZE);
(
@@ -80,7 +76,7 @@ where
}
}
pub fn start_with_shutdown(mut self, mut shutdown: nym_task::TaskClient) {
pub fn start_with_shutdown(mut self, mut shutdown: task::TaskClient) {
spawn_future(async move {
debug!("Started MixTrafficController with graceful shutdown support");
@@ -3,9 +3,9 @@
use super::action_controller::{AckActionSender, Action};
use futures::StreamExt;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::{
use nymsphinx::{
acknowledgements::{identifier::recover_identifier, AckKey},
chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID},
};
@@ -65,7 +65,7 @@ impl AcknowledgementListener {
}
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started AcknowledgementListener with graceful shutdown support");
while !shutdown.is_shutdown() {
@@ -6,9 +6,9 @@ use crate::client::real_messages_control::acknowledgement_control::Retransmissio
use futures::channel::mpsc;
use futures::StreamExt;
use log::*;
use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::Delay as SphinxDelay;
use nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nymsphinx::chunking::fragment::FragmentIdentifier;
use nymsphinx::Delay as SphinxDelay;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
@@ -216,7 +216,7 @@ impl ActionController {
fn handle_expired_ack_timer(
&mut self,
expired_ack: Expired<FragmentIdentifier>,
task_client: &mut nym_task::TaskClient,
task_client: &mut task::TaskClient,
) {
// I'm honestly not sure how to handle it, because getting it means other things in our
// system are already misbehaving. If we ever see this panic, then I guess we should worry
@@ -260,7 +260,7 @@ impl ActionController {
}
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started ActionController with graceful shutdown support");
while !shutdown.is_shutdown() {
@@ -4,10 +4,10 @@
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver};
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::ReplyControllerSender;
use client_connections::TransmissionLane;
use log::*;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_task::connections::TransmissionLane;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use rand::{CryptoRng, Rng};
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
@@ -109,7 +109,7 @@ where
};
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started InputMessageListener with graceful shutdown support");
while !shutdown.is_shutdown() {
@@ -13,11 +13,11 @@ use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future;
use action_controller::AckActionReceiver;
use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::params::PacketSize;
use nym_sphinx::{
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use nymsphinx::params::PacketSize;
use nymsphinx::{
acknowledgements::AckKey,
addressing::clients::Recipient,
chunking::fragment::{Fragment, FragmentIdentifier},
@@ -249,7 +249,7 @@ where
}
}
pub(super) fn start_with_shutdown(self, shutdown: nym_task::TaskClient) {
pub(super) fn start_with_shutdown(self, shutdown: task::TaskClient) {
let mut acknowledgement_listener = self.acknowledgement_listener;
let mut input_message_listener = self.input_message_listener;
let mut retransmission_request_listener = self.retransmission_request_listener;
@@ -9,12 +9,12 @@ use crate::client::real_messages_control::acknowledgement_control::PacketDestina
use crate::client::real_messages_control::message_handler::{MessageHandler, PreparationError};
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender;
use client_connections::TransmissionLane;
use futures::StreamExt;
use log::*;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::chunking::fragment::Fragment;
use nym_sphinx::preparer::PreparedFragment;
use nym_task::connections::TransmissionLane;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::chunking::fragment::Fragment;
use nymsphinx::preparer::PreparedFragment;
use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak};
@@ -137,7 +137,7 @@ where
.await
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started RetransmissionRequestListener with graceful shutdown support");
while !shutdown.is_shutdown() {
@@ -5,7 +5,7 @@ use super::action_controller::{AckActionSender, Action};
use super::SentPacketNotificationReceiver;
use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
use nymsphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
/// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled
@@ -37,7 +37,7 @@ impl SentNotificationListener {
.unwrap();
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started SentNotificationListener with graceful shutdown support");
while !shutdown.is_shutdown() {
@@ -8,23 +8,23 @@ use crate::client::real_messages_control::real_traffic_stream::{
use crate::client::real_messages_control::{AckActionSender, Action};
use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags};
use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit};
use client_connections::TransmissionLane;
use log::{debug, error, info, trace, warn};
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_sphinx::message::NymMessage;
use nym_sphinx::params::{PacketSize, DEFAULT_NUM_MIX_HOPS};
use nym_sphinx::preparer::{MessagePreparer, PreparedFragment};
use nym_sphinx::Delay;
use nym_task::connections::TransmissionLane;
use nym_topology::{NymTopology, NymTopologyError};
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nymsphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nymsphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nymsphinx::message::NymMessage;
use nymsphinx::params::{PacketSize, DEFAULT_NUM_MIX_HOPS};
use nymsphinx::preparer::{MessagePreparer, PreparedFragment};
use nymsphinx::Delay;
use rand::{CryptoRng, Rng};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use thiserror::Error;
use topology::{NymTopology, NymTopologyError};
// TODO: move that error elsewhere since it seems to be contaminating different files
#[derive(Debug, Clone, Error)]
@@ -101,11 +101,8 @@ pub(crate) struct Config {
/// Note that it does not include gateway hops.
num_mix_hops: u8,
/// Primary predefined packet size used for the encapsulated messages.
primary_packet_size: PacketSize,
/// Optional secondary predefined packet size used for the encapsulated messages.
secondary_packet_size: Option<PacketSize>,
/// Predefined packet size used for the encapsulated messages.
packet_size: PacketSize,
}
impl Config {
@@ -121,8 +118,7 @@ impl Config {
average_packet_delay,
average_ack_delay,
num_mix_hops: DEFAULT_NUM_MIX_HOPS,
primary_packet_size: PacketSize::default(),
secondary_packet_size: None,
packet_size: PacketSize::default(),
}
}
@@ -134,14 +130,8 @@ impl Config {
}
/// Allows setting non-default size of the sphinx packets sent out.
pub fn with_custom_primary_packet_size(mut self, packet_size: PacketSize) -> Self {
self.primary_packet_size = packet_size;
self
}
/// Allows setting non-default size of the sphinx packets sent out.
pub fn with_custom_secondary_packet_size(mut self, packet_size: Option<PacketSize>) -> Self {
self.secondary_packet_size = packet_size;
pub fn with_custom_packet_size(mut self, packet_size: PacketSize) -> Self {
self.packet_size = packet_size;
self
}
}
@@ -180,6 +170,7 @@ where
config.average_packet_delay,
config.average_ack_delay,
)
.with_custom_real_message_packet_size(config.packet_size)
.with_mix_hops(config.num_mix_hops);
MessageHandler {
@@ -220,28 +211,6 @@ where
}
}
fn optimal_packet_size(&self, msg: &NymMessage) -> PacketSize {
// if secondary packet was never set, then it's obvious we have to use the primary packet
let Some(secondary_packet) = self.config.secondary_packet_size else {
trace!("only primary packet size is available");
return self.config.primary_packet_size
};
let primary_count =
msg.required_packets(self.config.primary_packet_size, self.config.num_mix_hops);
let secondary_count = msg.required_packets(secondary_packet, self.config.num_mix_hops);
trace!("This message would require: {primary_count} primary packets or {secondary_count} secondary packets...");
// if there would be no benefit in using the secondary packet - use the primary (duh)
if primary_count <= secondary_count {
trace!("so choosing primary for this message");
self.config.primary_packet_size
} else {
trace!("so choosing secondary for this message");
secondary_packet
}
}
async fn generate_reply_surbs_with_keys(
&mut self,
amount: usize,
@@ -268,13 +237,9 @@ where
reply_surb: ReplySurb,
is_extra_surb_request: bool,
) -> Result<(), SurbWrappedPreparationError> {
let msg = NymMessage::new_reply(message);
let packet_size = self.optimal_packet_size(&msg);
debug!("Using {packet_size} packets for {msg}");
let mut fragment = self
.message_preparer
.pad_and_split_message(msg, packet_size);
.pad_and_split_message(NymMessage::new_reply(message));
if fragment.len() > 1 {
// well, it's not a single surb message
return Err(SurbWrappedPreparationError {
@@ -324,12 +289,10 @@ where
// // TODO: this will require additional argument to make it use different variant of `ReplyMessage`
pub(crate) fn split_reply_message(&mut self, message: Vec<u8>) -> Vec<Fragment> {
let msg = NymMessage::new_reply(ReplyMessage::new_data_message(message));
let packet_size = self.optimal_packet_size(&msg);
debug!("Using {packet_size} packets for {msg}");
self.message_preparer
.pad_and_split_message(msg, packet_size)
.pad_and_split_message(NymMessage::new_reply(ReplyMessage::new_data_message(
message,
)))
}
pub(crate) async fn send_retransmission_reply_chunks(
@@ -425,11 +388,7 @@ where
let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?;
let packet_size = self.optimal_packet_size(&message);
debug!("Using {packet_size} packets for {message}");
let fragments = self
.message_preparer
.pad_and_split_message(message, packet_size);
let fragments = self.message_preparer.pad_and_split_message(message);
let mut pending_acks = Vec::with_capacity(fragments.len());
let mut real_messages = Vec::with_capacity(fragments.len());
@@ -1,4 +1,4 @@
// Copyright 2021-2023 - Nym Technologies SA <contact@nymtech.net>
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
// INPUT: InputMessage from user
@@ -21,14 +21,16 @@ use crate::{
},
spawn_future,
};
use client_connections::{ConnectionCommandReceiver, LaneQueueLengths};
use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths};
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::params::PacketSize;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc;
use std::time::Duration;
use crate::client::replies::reply_controller;
use crate::config;
@@ -43,29 +45,61 @@ pub struct Config {
/// Key used to decrypt contents of received SURBAcks
ack_key: Arc<AckKey>,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the additive part `b`
ack_wait_addition: Duration,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the multiplier `a`
ack_wait_multiplier: f64,
/// Address of `this` client.
self_recipient: Recipient,
/// Specifies all traffic related configuration options.
traffic: config::Traffic,
/// Average delay between sending subsequent packets from this client.
average_message_sending_delay: Duration,
/// Specifies all cover traffic related configuration options.
cover_traffic: config::CoverTraffic,
/// Average delay a data packet is going to get delayed at a single mixnode.
average_packet_delay_duration: Duration,
/// Specifies all acknowledgements related configuration options.
acks: config::Acknowledgements,
/// Average delay an acknowledgement packet is going to get delayed at a single mixnode.
average_ack_delay_duration: Duration,
/// Specifies all reply SURBs related configuration options.
reply_surbs: config::ReplySurbs,
/// Controls whether the main packet stream constantly produces packets according to the predefined
/// poisson distribution.
disable_main_poisson_packet_distribution: bool,
/// Predefined packet size used for the encapsulated messages.
packet_size: PacketSize,
/// Defines the minimum number of reply surbs the client would request.
minimum_reply_surb_request_size: u32,
/// Defines the maximum number of reply surbs the client would request.
maximum_reply_surb_request_size: u32,
/// Defines the maximum number of reply surbs a remote party is allowed to request from this client at once.
maximum_allowed_reply_surb_request_size: u32,
/// Defines maximum amount of time the client is going to wait for reply surbs before explicitly asking
/// for more even though in theory they wouldn't need to.
maximum_reply_surb_rerequest_waiting_period: Duration,
/// Defines maximum amount of time the client is going to wait for reply surbs before
/// deciding it's never going to get them and would drop all pending messages
maximum_reply_surb_drop_waiting_period: Duration,
/// Defines maximum amount of time given reply surb is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
maximum_reply_surb_age: Duration,
/// Defines maximum amount of time given reply key is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
maximum_reply_key_age: Duration,
}
impl<'a> From<&'a Config> for acknowledgement_control::Config {
fn from(cfg: &'a Config) -> Self {
acknowledgement_control::Config::new(
cfg.acks.ack_wait_addition,
cfg.acks.ack_wait_multiplier,
)
.with_custom_packet_size(cfg.traffic.primary_packet_size)
acknowledgement_control::Config::new(cfg.ack_wait_addition, cfg.ack_wait_multiplier)
.with_custom_packet_size(cfg.packet_size)
}
}
@@ -74,16 +108,26 @@ impl<'a> From<&'a Config> for real_traffic_stream::Config {
real_traffic_stream::Config::new(
Arc::clone(&cfg.ack_key),
cfg.self_recipient,
cfg.acks.average_ack_delay,
cfg.traffic,
cfg.cover_traffic.cover_traffic_primary_size_ratio,
cfg.average_ack_delay_duration,
cfg.average_packet_delay_duration,
cfg.average_message_sending_delay,
cfg.disable_main_poisson_packet_distribution,
)
.with_custom_cover_packet_size(cfg.packet_size)
}
}
impl<'a> From<&'a Config> for reply_controller::Config {
fn from(cfg: &'a Config) -> Self {
reply_controller::Config::new(cfg.reply_surbs)
reply_controller::Config::new(
cfg.minimum_reply_surb_request_size,
cfg.maximum_reply_surb_request_size,
cfg.maximum_allowed_reply_surb_request_size,
cfg.maximum_reply_surb_rerequest_waiting_period,
cfg.maximum_reply_surb_drop_waiting_period,
cfg.maximum_reply_surb_age,
cfg.maximum_reply_key_age,
)
}
}
@@ -92,11 +136,10 @@ impl<'a> From<&'a Config> for message_handler::Config {
message_handler::Config::new(
Arc::clone(&cfg.ack_key),
cfg.self_recipient,
cfg.traffic.average_packet_delay,
cfg.acks.average_ack_delay,
cfg.average_packet_delay_duration,
cfg.average_ack_delay_duration,
)
.with_custom_primary_packet_size(cfg.traffic.primary_packet_size)
.with_custom_secondary_packet_size(cfg.traffic.secondary_packet_size)
.with_custom_packet_size(cfg.packet_size)
}
}
@@ -109,12 +152,32 @@ impl Config {
Config {
ack_key,
self_recipient,
traffic: base_client_debug_config.traffic,
cover_traffic: base_client_debug_config.cover_traffic,
acks: base_client_debug_config.acknowledgements,
reply_surbs: base_client_debug_config.reply_surbs,
packet_size: Default::default(),
ack_wait_addition: base_client_debug_config.ack_wait_addition,
ack_wait_multiplier: base_client_debug_config.ack_wait_multiplier,
average_message_sending_delay: base_client_debug_config.message_sending_average_delay,
average_packet_delay_duration: base_client_debug_config.average_packet_delay,
average_ack_delay_duration: base_client_debug_config.average_ack_delay,
disable_main_poisson_packet_distribution: base_client_debug_config
.disable_main_poisson_packet_distribution,
minimum_reply_surb_request_size: base_client_debug_config
.minimum_reply_surb_request_size,
maximum_reply_surb_request_size: base_client_debug_config
.maximum_reply_surb_request_size,
maximum_allowed_reply_surb_request_size: base_client_debug_config
.maximum_allowed_reply_surb_request_size,
maximum_reply_surb_rerequest_waiting_period: base_client_debug_config
.maximum_reply_surb_rerequest_waiting_period,
maximum_reply_surb_drop_waiting_period: base_client_debug_config
.maximum_reply_surb_drop_waiting_period,
maximum_reply_surb_age: base_client_debug_config.maximum_reply_surb_age,
maximum_reply_key_age: base_client_debug_config.maximum_reply_key_age,
}
}
pub fn set_custom_packet_size(&mut self, packet_size: PacketSize) {
self.packet_size = packet_size;
}
}
pub(crate) struct RealMessagesController<R>
@@ -207,7 +270,7 @@ impl RealMessagesController<OsRng> {
}
}
pub fn start_with_shutdown(self, shutdown: nym_task::TaskClient) {
pub fn start_with_shutdown(self, shutdown: task::TaskClient) {
let mut out_queue_control = self.out_queue_control;
let ack_control = self.ack_control;
let mut reply_control = self.reply_control;
@@ -6,21 +6,20 @@ use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::real_messages_control::acknowledgement_control::SentPacketNotificationSender;
use crate::client::topology_control::TopologyAccessor;
use crate::client::transmission_buffer::TransmissionBuffer;
use crate::config;
use client_connections::{
ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane,
};
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::cover::generate_loop_cover_packet;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::params::PacketSize;
use nym_sphinx::preparer::PreparedFragment;
use nym_sphinx::utils::sample_poisson_duration;
use nym_task::connections::{
ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane,
};
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::chunking::fragment::FragmentIdentifier;
use nymsphinx::cover::generate_loop_cover_packet;
use nymsphinx::forwarding::packet::MixPacket;
use nymsphinx::params::PacketSize;
use nymsphinx::preparer::PreparedFragment;
use nymsphinx::utils::sample_poisson_duration;
use rand::{CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
@@ -28,6 +27,7 @@ use std::time::Duration;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time;
#[cfg(target_arch = "wasm32")]
use wasm_timer;
@@ -44,12 +44,18 @@ pub(crate) struct Config {
/// Average delay an acknowledgement packet is going to get delay at a single mixnode.
average_ack_delay: Duration,
/// Defines all configuration options related to this traffic stream.
traffic: config::Traffic,
/// Average delay a data packet is going to get delay at a single mixnode.
average_packet_delay: Duration,
/// Specifies the ratio of `primary_packet_size` to `secondary_packet_size` used in cover traffic.
/// Only applicable if `secondary_packet_size` is enabled.
cover_traffic_primary_size_ratio: f64,
/// Average delay between sending subsequent packets.
average_message_sending_delay: Duration,
/// Controls whether the stream constantly produces packets according to the predefined
/// poisson distribution.
disable_poisson_packet_distribution: bool,
/// Predefined packet size used for the loop cover messages.
cover_packet_size: PacketSize,
}
impl Config {
@@ -57,17 +63,25 @@ impl Config {
ack_key: Arc<AckKey>,
our_full_destination: Recipient,
average_ack_delay: Duration,
traffic: config::Traffic,
cover_traffic_primary_size_ratio: f64,
average_packet_delay: Duration,
average_message_sending_delay: Duration,
disable_poisson_packet_distribution: bool,
) -> Self {
Config {
ack_key,
our_full_destination,
average_ack_delay,
traffic,
cover_traffic_primary_size_ratio,
average_packet_delay,
average_message_sending_delay,
disable_poisson_packet_distribution,
cover_packet_size: Default::default(),
}
}
pub fn with_custom_cover_packet_size(mut self, packet_size: PacketSize) -> Self {
self.cover_packet_size = packet_size;
self
}
}
pub(crate) struct OutQueueControl<R>
@@ -198,30 +212,11 @@ where
self.sent_notifier.unbounded_send(frag_id).unwrap();
}
fn loop_cover_message_size(&mut self) -> PacketSize {
let Some(secondary_packet_size) = self.config.traffic.secondary_packet_size else {
return self.config.traffic.primary_packet_size
};
let use_primary = self
.rng
.gen_bool(self.config.cover_traffic_primary_size_ratio);
if use_primary {
self.config.traffic.primary_packet_size
} else {
secondary_packet_size
}
}
async fn on_message(&mut self, next_message: StreamMessage) {
trace!("created new message");
let (next_message, fragment_id) = match next_message {
StreamMessage::Cover => {
let cover_traffic_packet_size = self.loop_cover_message_size();
trace!("the next loop cover message will be put in a {cover_traffic_packet_size} packet");
// TODO for way down the line: in very rare cases (during topology update) we might have
// to wait a really tiny bit before actually obtaining the permit hence messing with our
// poisson delay, but is it really a problem?
@@ -245,8 +240,8 @@ where
&self.config.ack_key,
&self.config.our_full_destination,
self.config.average_ack_delay,
self.config.traffic.average_packet_delay,
cover_traffic_packet_size,
self.config.average_packet_delay,
self.config.cover_packet_size,
)
.expect(
"Somehow failed to generate a loop cover message with a valid topology",
@@ -291,7 +286,7 @@ where
}
fn current_average_message_sending_delay(&self) -> Duration {
self.config.traffic.message_sending_average_delay
self.config.average_message_sending_delay
* self.sending_delay_controller.current_multiplier()
}
@@ -302,34 +297,24 @@ where
self.sending_delay_controller.current_multiplier()
);
if self
.sending_delay_controller
.is_backpressure_currently_detected(used_slots)
{
// Even just a single used slot is enough to signal backpressure
if used_slots > 0 {
log::trace!("Backpressure detected");
self.sending_delay_controller.record_backpressure_detected();
}
// If the buffer is running out, slow down the sending rate by increasing the delay
// multiplier.
// If the buffer is running out, slow down the sending rate
if self.mix_tx.capacity() == 0
&& self.sending_delay_controller.not_increased_delay_recently()
{
self.sending_delay_controller.increase_delay_multiplier();
}
// If it looks like we are sending reliably, increase the sending rate by decreasing the
// sending delay multiplier.
if !self
.sending_delay_controller
.was_backpressure_detected_recently()
&& self.sending_delay_controller.not_decreased_delay_recently()
{
// Very carefully step up the sending rate in case it seems like we can solidly handle the
// current rate.
if self.sending_delay_controller.is_sending_reliable() {
self.sending_delay_controller.decrease_delay_multiplier();
}
// Keep track of multiplier changes, and log if necessary.
self.sending_delay_controller.record_delay_multiplier();
}
fn pop_next_message(&mut self) -> Option<RealMessage> {
@@ -357,6 +342,7 @@ where
if let Poll::Ready(Some(id)) = Pin::new(&mut self.client_connection_rx).poll_next(cx) {
match id {
ConnectionCommand::Close(id) => self.on_close_connection(id),
ConnectionCommand::ActiveConnections(_) => panic!(),
}
}
@@ -415,10 +401,8 @@ where
// we never set an initial delay - let's do it now
cx.waker().wake_by_ref();
let sampled = sample_poisson_duration(
&mut self.rng,
self.config.traffic.message_sending_average_delay,
);
let sampled =
sample_poisson_duration(&mut self.rng, self.config.average_message_sending_delay);
#[cfg(not(target_arch = "wasm32"))]
let next_delay = Box::pin(time::sleep(sampled));
@@ -437,6 +421,7 @@ where
if let Poll::Ready(Some(id)) = Pin::new(&mut self.client_connection_rx).poll_next(cx) {
match id {
ConnectionCommand::Close(id) => self.on_close_connection(id),
ConnectionCommand::ActiveConnections(_) => panic!(),
}
}
@@ -469,7 +454,7 @@ where
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<StreamMessage>> {
if self.config.traffic.disable_main_poisson_packet_distribution {
if self.config.disable_poisson_packet_distribution {
self.poll_immediate(cx)
} else {
self.poll_poisson(cx)
@@ -477,7 +462,7 @@ where
}
#[cfg(not(target_arch = "wasm32"))]
fn log_status(&self, shutdown: &mut nym_task::TaskClient) {
fn log_status(&self, shutdown: &mut task::TaskClient) {
use crate::error::ClientCoreStatusMessage;
let packets = self.transmission_buffer.total_size();
@@ -485,7 +470,7 @@ where
let lanes = self.transmission_buffer.num_lanes();
let mult = self.sending_delay_controller.current_multiplier();
let delay = self.current_average_message_sending_delay().as_millis();
let status_str = if self.config.traffic.disable_main_poisson_packet_distribution {
let status_str = if self.config.disable_poisson_packet_distribution {
format!("Status: {lanes} lanes, backlog: {backlog:.2} kiB ({packets}), no delay")
} else {
format!(
@@ -508,12 +493,23 @@ where
}
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
#[cfg(not(target_arch = "wasm32"))]
fn log_status_infrequent(&self) {
if self.sending_delay_controller.current_multiplier() > 1 {
log::warn!(
"Unable to send packets at the default rate - rate reduced by setting the delay multiplier set to: {}",
self.sending_delay_controller.current_multiplier()
);
}
}
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started OutQueueControl with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
{
let mut status_timer = tokio::time::interval(Duration::from_secs(5));
let mut infrequent_status_timer = tokio::time::interval(Duration::from_secs(60));
while !shutdown.is_shutdown() {
tokio::select! {
@@ -524,6 +520,9 @@ where
_ = status_timer.tick() => {
self.log_status(&mut shutdown);
}
_ = infrequent_status_timer.tick() => {
self.log_status_infrequent();
}
next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await;
} else {
@@ -11,20 +11,15 @@ const INCREASE_DELAY_MIN_CHANGE_INTERVAL_SECS: u64 = 1;
// The minimum time between decreasing the average delay between packets. We don't want to change
// to quickly to keep things somewhat stable. Also there are buffers downstreams meaning we need to
// wait a little to see the effect before we decrease further.
const DECREASE_DELAY_MIN_CHANGE_INTERVAL_SECS: u64 = 2;
// The queue length that is required for us to register that backpressure occured. If there are
// more than this many packets waiting to be sent, we consider the channel to be under
// backpressure.
const BACKPRESSURE_THRESHOLD: usize = 10;
const DECREASE_DELAY_MIN_CHANGE_INTERVAL_SECS: u64 = 30;
// If we enough time passes without any sign of backpressure in the channel, we can consider
// lowering the average delay.
const ACCEPTABLE_TIME_WITHOUT_BACKPRESSURE_SECS: u64 = 2;
// lowering the average delay. The goal is to keep somewhat stable, rather than maxing out
// bandwidth at all times.
const ACCEPTABLE_TIME_WITHOUT_BACKPRESSURE_SECS: u64 = 30;
// The maximum multiplier we apply to the base average Poisson delay.
const MAX_DELAY_MULTIPLIER: u32 = 6;
// The minium multiplier we apply to the base average Poisson delay.
const MIN_DELAY_MULTIPLIER: u32 = 1;
// If the multipler increases we log it, but we don't want to log about it too often.
const INTERVAL_BETWEEN_WARNING_ABOUT_ELEVATED_MULTIPLIER_SECS: u64 = 60;
pub(crate) struct SendingDelayController {
/// Multiply the average sending delay.
@@ -38,14 +33,6 @@ pub(crate) struct SendingDelayController {
/// Minimum delay multiplier
lower_bound: u32,
/// We counter the number of times the multiplier has been elevated. If it is elevated for long
/// enough we need to log about it.
multiplier_elevated_counter: u32,
/// We can't log about the elevated multiplier too often, so we keep track of the last time we
/// did,
time_when_logged_about_elevated_multiplier: Instant,
/// To make sure we don't change the multiplier to fast, we limit a change to some duration
time_when_changed: Instant,
@@ -68,9 +55,6 @@ impl SendingDelayController {
current_multiplier: MIN_DELAY_MULTIPLIER,
upper_bound,
lower_bound,
multiplier_elevated_counter: 0,
time_when_logged_about_elevated_multiplier: now
- Duration::from_secs(INTERVAL_BETWEEN_WARNING_ABOUT_ELEVATED_MULTIPLIER_SECS),
time_when_changed: now,
time_when_backpressure_detected: now,
}
@@ -95,7 +79,7 @@ impl SendingDelayController {
self.current_multiplier =
(self.current_multiplier + 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now();
log::debug!(
log::warn!(
"Increasing sending delay multiplier to: {}",
self.current_multiplier
);
@@ -116,58 +100,22 @@ impl SendingDelayController {
}
}
pub(crate) fn record_backpressure_detected(&mut self) {
self.time_when_backpressure_detected = get_time_now();
}
pub(crate) fn not_increased_delay_recently(&self) -> bool {
get_time_now()
> self.time_when_changed + Duration::from_secs(INCREASE_DELAY_MIN_CHANGE_INTERVAL_SECS)
}
pub(crate) fn not_decreased_delay_recently(&self) -> bool {
get_time_now()
> self.time_when_changed + Duration::from_secs(DECREASE_DELAY_MIN_CHANGE_INTERVAL_SECS)
}
pub(crate) fn is_backpressure_currently_detected(&self, queue_length: usize) -> bool {
queue_length > BACKPRESSURE_THRESHOLD
}
pub(crate) fn record_backpressure_detected(&mut self) {
self.time_when_backpressure_detected = get_time_now();
}
pub(crate) fn was_backpressure_detected_recently(&self) -> bool {
get_time_now()
< self.time_when_backpressure_detected
+ Duration::from_secs(ACCEPTABLE_TIME_WITHOUT_BACKPRESSURE_SECS)
}
pub(crate) fn record_delay_multiplier(&mut self) {
// Count the number of times the multiplier has been elevated.
let multiplier_elevated = self.current_multiplier - self.lower_bound;
if multiplier_elevated == 0 {
self.multiplier_elevated_counter = 0;
} else {
self.multiplier_elevated_counter += 1;
}
// If needed, log about the elevated multiplier.
pub(crate) fn is_sending_reliable(&self) -> bool {
let now = get_time_now();
if self.multiplier_elevated_counter > 20
&& now
> self.time_when_logged_about_elevated_multiplier
+ Duration::from_secs(INTERVAL_BETWEEN_WARNING_ABOUT_ELEVATED_MULTIPLIER_SECS)
{
let status_str = format!(
"Poisson delay currently scaled by: {}",
self.current_multiplier()
);
if self.current_multiplier() > 0 {
log::debug!("{}", status_str);
} else if self.current_multiplier() > 1 {
log::info!("{}", status_str);
} else if self.current_multiplier() > 2 {
log::warn!("{}", status_str);
}
self.time_when_logged_about_elevated_multiplier = now;
}
let delay_change_interval = Duration::from_secs(DECREASE_DELAY_MIN_CHANGE_INTERVAL_SECS);
let acceptable_time_without_backpressure =
Duration::from_secs(ACCEPTABLE_TIME_WITHOUT_BACKPRESSURE_SECS);
now > self.time_when_backpressure_detected + acceptable_time_without_backpressure
&& now > self.time_when_changed + delay_change_interval
}
}
@@ -4,20 +4,20 @@
use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::client::replies::reply_storage::SentReplyKeys;
use crate::spawn_future;
use crypto::asymmetric::encryption;
use crypto::Digest;
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::StreamExt;
use gateway_client::MixnetMessageReceiver;
use log::*;
use nym_crypto::asymmetric::encryption;
use nym_crypto::Digest;
use nym_gateway_client::MixnetMessageReceiver;
use nym_sphinx::anonymous_replies::requests::{
use nymsphinx::anonymous_replies::requests::{
RepliableMessage, RepliableMessageContent, ReplyMessage, ReplyMessageContent,
};
use nym_sphinx::anonymous_replies::{encryption_key::EncryptionKeyDigest, SurbEncryptionKey};
use nym_sphinx::message::{NymMessage, PlainMessage};
use nym_sphinx::params::ReplySurbKeyDigestAlgorithm;
use nym_sphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use nymsphinx::anonymous_replies::{encryption_key::EncryptionKeyDigest, SurbEncryptionKey};
use nymsphinx::message::{NymMessage, PlainMessage};
use nymsphinx::params::ReplySurbKeyDigestAlgorithm;
use nymsphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use std::collections::HashSet;
use std::sync::Arc;
@@ -30,13 +30,13 @@ pub type ReceivedBufferRequestReceiver = mpsc::UnboundedReceiver<ReceivedBufferM
pub type ReconstructedMessagesSender = mpsc::UnboundedSender<Vec<ReconstructedMessage>>;
pub type ReconstructedMessagesReceiver = mpsc::UnboundedReceiver<Vec<ReconstructedMessage>>;
struct ReceivedMessagesBufferInner<R: MessageReceiver> {
struct ReceivedMessagesBufferInner {
messages: Vec<ReconstructedMessage>,
local_encryption_keypair: Arc<encryption::KeyPair>,
// TODO: looking how it 'looks' here, perhaps `MessageReceiver` should be renamed to something
// else instead.
message_receiver: R,
message_receiver: MessageReceiver,
message_sender: Option<ReconstructedMessagesSender>,
// TODO: this will get cleared upon re-running the client
@@ -45,9 +45,9 @@ struct ReceivedMessagesBufferInner<R: MessageReceiver> {
recently_reconstructed: HashSet<i32>,
}
impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
impl ReceivedMessagesBufferInner {
fn recover_from_fragment(&mut self, fragment_data: &[u8]) -> Option<NymMessage> {
if nym_sphinx::cover::is_cover(fragment_data) {
if nymsphinx::cover::is_cover(fragment_data) {
trace!("The message was a loop cover message! Skipping it");
return None;
}
@@ -102,13 +102,13 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
&mut self,
reply_ciphertext: &mut [u8],
reply_key: SurbEncryptionKey,
) -> Result<Option<NymMessage>, MessageRecoveryError> {
) -> Option<NymMessage> {
// note: this performs decryption IN PLACE without extra allocation
self.message_receiver
.recover_plaintext_from_reply(reply_ciphertext, reply_key)?;
.recover_plaintext_from_reply(reply_ciphertext, reply_key);
let fragment_data = reply_ciphertext;
Ok(self.recover_from_fragment(fragment_data))
self.recover_from_fragment(fragment_data)
}
fn process_received_regular_packet(&mut self, mut raw_fragment: Vec<u8>) -> Option<NymMessage> {
@@ -130,13 +130,13 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
#[derive(Debug, Clone)]
// Note: you should NEVER create more than a single instance of this using 'new()'.
// You should always use .clone() to create additional instances
struct ReceivedMessagesBuffer<R: MessageReceiver> {
inner: Arc<Mutex<ReceivedMessagesBufferInner<R>>>,
struct ReceivedMessagesBuffer {
inner: Arc<Mutex<ReceivedMessagesBufferInner>>,
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
}
impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
impl ReceivedMessagesBuffer {
fn new(
local_encryption_keypair: Arc<encryption::KeyPair>,
reply_key_storage: SentReplyKeys,
@@ -146,7 +146,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
messages: Vec::new(),
local_encryption_keypair,
message_receiver: R::new(),
message_receiver: MessageReceiver::new(),
message_sender: None,
recently_reconstructed: HashSet::new(),
})),
@@ -328,10 +328,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
})
}
async fn handle_new_received(
&mut self,
msgs: Vec<Vec<u8>>,
) -> Result<(), MessageRecoveryError> {
async fn handle_new_received(&mut self, msgs: Vec<Vec<u8>>) {
trace!(
"Processing {:?} new message that might get added to the buffer!",
msgs.len()
@@ -347,7 +344,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
// if yes - this is a reply message
let completed_message =
if let Some((reply_key, reply_message)) = self.get_reply_key(&mut msg) {
inner_guard.process_received_reply(reply_message, reply_key)?
inner_guard.process_received_reply(reply_message, reply_key)
} else {
inner_guard.process_received_regular_packet(msg)
};
@@ -363,7 +360,6 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
if !completed_messages.is_empty() {
self.handle_reconstructed_messages(completed_messages).await
}
Ok(())
}
}
@@ -376,14 +372,14 @@ pub enum ReceivedBufferMessage {
ReceiverDisconnect,
}
struct RequestReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>,
struct RequestReceiver {
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
}
impl<R: MessageReceiver> RequestReceiver<R> {
impl RequestReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer<R>,
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
) -> Self {
RequestReceiver {
@@ -403,7 +399,7 @@ impl<R: MessageReceiver> RequestReceiver<R> {
}
}
async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started RequestReceiver with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
@@ -426,14 +422,14 @@ impl<R: MessageReceiver> RequestReceiver<R> {
}
}
struct FragmentedMessageReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>,
struct FragmentedMessageReceiver {
received_buffer: ReceivedMessagesBuffer,
mixnet_packet_receiver: MixnetMessageReceiver,
}
impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
impl FragmentedMessageReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer<R>,
received_buffer: ReceivedMessagesBuffer,
mixnet_packet_receiver: MixnetMessageReceiver,
) -> Self {
FragmentedMessageReceiver {
@@ -442,16 +438,13 @@ impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
}
}
async fn run_with_shutdown(
&mut self,
mut shutdown: nym_task::TaskClient,
) -> Result<(), MessageRecoveryError> {
async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started FragmentedMessageReceiver with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
new_messages = self.mixnet_packet_receiver.next() => {
if let Some(new_messages) = new_messages {
self.received_buffer.handle_new_received(new_messages).await?;
self.received_buffer.handle_new_received(new_messages).await;
} else {
log::trace!("FragmentedMessageReceiver: Stopping since channel closed");
break;
@@ -464,16 +457,15 @@ impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
}
shutdown.recv_timeout().await;
log::debug!("FragmentedMessageReceiver: Exiting");
Ok(())
}
}
pub(crate) struct ReceivedMessagesBufferController<R: MessageReceiver> {
fragmented_message_receiver: FragmentedMessageReceiver<R>,
request_receiver: RequestReceiver<R>,
pub(crate) struct ReceivedMessagesBufferController {
fragmented_message_receiver: FragmentedMessageReceiver,
request_receiver: RequestReceiver,
}
impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferController<R> {
impl ReceivedMessagesBufferController {
pub(crate) fn new(
local_encryption_keypair: Arc<encryption::KeyPair>,
query_receiver: ReceivedBufferRequestReceiver,
@@ -496,19 +488,15 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
}
}
pub fn start_with_shutdown(self, shutdown: nym_task::TaskClient) {
pub fn start_with_shutdown(self, shutdown: task::TaskClient) {
let mut fragmented_message_receiver = self.fragmented_message_receiver;
let mut request_receiver = self.request_receiver;
let shutdown_handle = shutdown.clone();
spawn_future(async move {
match fragmented_message_receiver
fragmented_message_receiver
.run_with_shutdown(shutdown_handle)
.await
{
Ok(_) => {}
Err(e) => error!("{e}"),
}
.await;
});
spawn_future(async move {
request_receiver.run_with_shutdown(shutdown).await;
@@ -4,14 +4,14 @@
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use crate::client::real_messages_control::message_handler::{MessageHandler, PreparationError};
use crate::client::replies::reply_storage::CombinedReplyStorage;
use client_connections::{ConnectionId, TransmissionLane};
use futures::channel::oneshot;
use futures::StreamExt;
use log::{debug, error, info, trace, warn};
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurb;
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_task::connections::{ConnectionId, TransmissionLane};
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use nymsphinx::anonymous_replies::ReplySurb;
use nymsphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use rand::{CryptoRng, Rng};
use std::cmp::{max, min};
use std::collections::btree_map::Entry;
@@ -22,21 +22,38 @@ use time::OffsetDateTime;
use crate::client::helpers::new_interval_stream;
use crate::client::transmission_buffer::TransmissionBuffer;
use crate::config;
pub(crate) use requests::{ReplyControllerMessage, ReplyControllerReceiver, ReplyControllerSender};
pub mod requests;
// this is still left as a separate config so I wouldn't need to replace it everywhere
// plus its not unreasonable to think that we might need something outside config::ReplySurbs struct
pub struct Config {
reply_surbs: config::ReplySurbs,
min_surb_request_size: u32,
max_surb_request_size: u32,
maximum_allowed_reply_surb_request_size: u32,
max_surb_rerequest_waiting_period: Duration,
max_surb_drop_waiting_period: Duration,
max_reply_surb_age: Duration,
max_reply_key_age: Duration,
}
impl Config {
pub(crate) fn new(reply_surbs_cfg: config::ReplySurbs) -> Self {
pub(crate) fn new(
min_surb_request_size: u32,
max_surb_request_size: u32,
maximum_allowed_reply_surb_request_size: u32,
max_surb_rerequest_waiting_period: Duration,
max_surb_drop_waiting_period: Duration,
max_reply_surb_age: Duration,
max_reply_key_age: Duration,
) -> Self {
Self {
reply_surbs: reply_surbs_cfg,
min_surb_request_size,
max_surb_request_size,
maximum_allowed_reply_surb_request_size,
max_surb_rerequest_waiting_period,
max_surb_drop_waiting_period,
max_reply_surb_age,
max_reply_key_age,
}
}
}
@@ -492,17 +509,9 @@ where
}
// 2. check whether the requested amount is within sane range
if amount
> self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size
{
warn!("The requested reply surb amount is larger than our maximum allowed ({amount} > {}). Lowering it to a more sane value...", self.config.reply_surbs.maximum_allowed_reply_surb_request_size);
amount = self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size;
if amount > self.config.maximum_allowed_reply_surb_request_size {
warn!("The requested reply surb amount is larger than our maximum allowed ({amount} > {}). Lowering it to a more sane value...", self.config.maximum_allowed_reply_surb_request_size);
amount = self.config.maximum_allowed_reply_surb_request_size;
}
// 3. construct and send the surbs away
@@ -698,11 +707,8 @@ where
}
let request_size = min(
self.config.reply_surbs.maximum_reply_surb_request_size,
max(
total_queue,
self.config.reply_surbs.minimum_reply_surb_request_size,
),
self.config.max_surb_request_size,
max(total_queue, self.config.min_surb_request_size),
);
if let Err(err) = self
@@ -738,17 +744,9 @@ where
};
let diff = now - last_received_time;
let max_rerequest_wait = self
.config
.reply_surbs
.maximum_reply_surb_rerequest_waiting_period;
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
if diff > max_rerequest_wait {
if diff > max_drop_wait {
if diff > self.config.max_surb_rerequest_waiting_period {
if diff > self.config.max_surb_drop_waiting_period {
to_remove.push(*pending_reply_target)
} else {
debug!("We haven't received any surbs in {:?} from {pending_reply_target}. Going to explicitly ask for more", diff);
@@ -795,7 +793,7 @@ where
};
let diff = now - last_received_time;
if diff > self.config.reply_surbs.maximum_reply_surb_age {
if diff > self.config.max_reply_surb_age {
info!("it's been {diff:?} since we last received any reply surb from {sender}. Going to remove all stored entries...");
to_remove_surbs.push(*sender);
@@ -815,7 +813,7 @@ where
let diff = now - sent_at;
if diff > self.config.reply_surbs.maximum_reply_key_age {
if diff > self.config.max_reply_key_age {
debug!("it's been {diff:?} since we created this reply key. it's probably never going to get used, so we're going to purge it...");
to_remove_keys.push(*digest);
}
@@ -837,15 +835,14 @@ where
// todo!()
// }
pub(crate) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
pub(crate) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started ReplyController with graceful shutdown support");
let polling_rate = Duration::from_secs(5);
let mut stale_inspection = new_interval_stream(polling_rate);
// this is in the order of hours/days so we don't have to poll it that often
let polling_rate =
Duration::from_secs(self.config.reply_surbs.maximum_reply_surb_age.as_secs() / 10);
let polling_rate = Duration::from_secs(self.config.max_reply_surb_age.as_secs() / 10);
let mut invalidation_inspection = new_interval_stream(polling_rate);
while !shutdown.is_shutdown() {
@@ -2,12 +2,12 @@
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use client_connections::{ConnectionId, TransmissionLane};
use futures::channel::{mpsc, oneshot};
use log::error;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurb;
use nym_task::connections::{ConnectionId, TransmissionLane};
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use nymsphinx::anonymous_replies::ReplySurb;
use std::sync::Weak;
pub(crate) fn new_control_channels() -> (ReplyControllerSender, ReplyControllerReceiver) {
@@ -5,8 +5,6 @@ use crate::client::replies::reply_storage::backend::Empty;
use crate::client::replies::reply_storage::{CombinedReplyStorage, ReplyStorageBackend};
use async_trait::async_trait;
use std::path::PathBuf;
// well, right now we don't have the browser storage : (
// so we keep everything in memory
#[derive(Debug)]
@@ -29,22 +27,6 @@ impl Backend {
impl ReplyStorageBackend for Backend {
type StorageError = <Empty as ReplyStorageBackend>::StorageError;
async fn new(
debug_config: &crate::config::DebugConfig,
_db_path: Option<PathBuf>,
) -> Result<Self, Self::StorageError> {
Ok(Backend {
empty: Empty {
min_surb_threshold: debug_config
.reply_surbs
.minimum_reply_surb_storage_threshold,
max_surb_threshold: debug_config
.reply_surbs
.maximum_reply_surb_storage_threshold,
},
})
}
async fn flush_surb_storage(
&mut self,
storage: &CombinedReplyStorage,
@@ -56,9 +56,4 @@ pub enum StorageError {
details: String,
// err: Option<Box<dyn std::error::Error>>
},
#[error("failed to create storage")]
FailedToCreateStorage {
source: Box<dyn std::error::Error + Send + Sync>,
},
}

Some files were not shown because too many files have changed in this diff Show More