Compare commits

..

28 Commits

Author SHA1 Message Date
Bogdan-Ștefan Neacşu b5213f3908 Hide coconut runtime flags (#2990) 2023-02-08 13:00:58 +02:00
Bogdan-Ștefan Neacşu 590a4644e2 Skip errors on blind sign within threshold (#2976) 2023-02-08 11:54:12 +02:00
Bogdan-Ștefan Neacşu f8345e03c6 Feature/dkg reshare (#2936)
* Add resharing parameter

* Fix equality of dealers and members

* Contract resharing handling

* Dealer verification unit test

* Dealing commit unit test

* Epoch state unit tests

* Fix clippy

* Fmt

* Query initial dealer data

* Resharing nym-api changes

* Implement the mockups for nym-api dkg tests

* Dealing test

* Vk unit test

* Fix skipping vk submission

* Fix clippy

* Missing dealing for noninitial resharing dealer

* Check master vk holds after resharing on nym-apis

* Update changelog

* Fix clippy
2023-02-07 12:56:08 +02:00
Jon Häggblad 6675ea0249 Remove all the .DS_Store files and add to gitignore (#2974) 2023-02-07 11:45:40 +01:00
Jędrzej Stuczyński 56f48e6e41 Feature/service provider interface (#2934)
* added additional workspace-wide dependencies

* Added conditional serialization on 'BinaryBuildInformationOwned'

* initial framework for service provider messages

* updated request/response tags to account for existing variants of Socks5Message

* handling legacy deserialization

* another serialization revamp to account for legacy version

* legacy client working with versioned network requester

* socks5 client deserializing responses into updated structures

* using new structures for sending in socks5 client

* SendRequest wrapper for Request::Send variant

* created named fields for all variants of 'ControllerCommand'

* Versioning socks5 requests + moving to proper Socks5Request struct

* Updated backwards compatible Socks5Response

* unused imports

* poc ServiceProvider trait

* wip

* implemented 'ServiceProvider' trait on the Network Requester

* Socks5RequestError

* added properly serialized ErrorResponse to ControlResponse

* fixed version serialization + feature selection

* handling of version control requests

* improved SocksProxyError by providing more concrete variants and removing generic case

* got rid of ServiceProviderClient trait and wrote simple example showing control requests

* tests for serialisation backwards compatibility

* post-merge fixes due to method renaming

* enum boxing to make clippy happier

* making sure to not drop buffer channel when starting `DirectClient`

* Using nym-sdk in the example

* Replaced printing version to stdout with proper log call
2023-02-07 09:49:19 +00:00
Pierre Dommerc 1733aaa4cc NC Android - setup APK publish (#2967)
* ci(nc-android): init workflow apk build & release

* ci(nc-android): setup nc android project for APK publish

* docs(nc-android): update readme

* ci(nc-android): add note in the workflow
2023-02-06 10:56:39 +01:00
Mark Sinclair 78aa07360c Update build-and-upload-binaries-ci.yml 2023-02-03 17:10:13 +00:00
Mark Sinclair 4e52478c7a Update build-and-upload-binaries-ci.yml 2023-02-03 16:54:49 +00:00
Mark Sinclair 7c77665a37 Update build-and-upload-binaries-ci.yml 2023-02-03 16:36:44 +00:00
Mark Sinclair 72880b9764 Update build-and-upload-binaries-ci.yml 2023-02-03 16:14:29 +00:00
Mark Sinclair fd1c0ae62b Update build-and-upload-binaries-ci.yml 2023-02-03 12:03:13 +00:00
Mark Sinclair bd7399091b Update build-and-upload-binaries-ci.yml 2023-02-03 11:57:32 +00:00
Mark Sinclair 00fad44e2e Update build-and-upload-binaries-ci.yml 2023-02-03 11:46:07 +00:00
Mark Sinclair ea33c332ee GitHub Actions: add action to build and upload binaries to CI server 2023-02-03 11:35:03 +00:00
Bogdan-Ștefan Neacșu b39f8af8d0 Fix flaky dkg test 2023-02-03 12:29:21 +02:00
dependabot[bot] a400463b7e build(deps): bump ua-parser-js in /nym-wallet/webdriver (#2909)
Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.28 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](https://github.com/faisalman/ua-parser-js/compare/0.7.28...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 10:00:09 +00:00
dependabot[bot] 82928af64c build(deps): bump http-cache-semantics in /nym-wallet/webdriver (#2960)
Bumps [http-cache-semantics](https://github.com/kornelski/http-cache-semantics) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/kornelski/http-cache-semantics/releases)
- [Commits](https://github.com/kornelski/http-cache-semantics/commits)

---
updated-dependencies:
- dependency-name: http-cache-semantics
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 09:59:08 +00:00
dependabot[bot] c5891f546c build(deps): bump http-cache-semantics from 4.1.0 to 4.1.1 (#2961)
Bumps [http-cache-semantics](https://github.com/kornelski/http-cache-semantics) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/kornelski/http-cache-semantics/releases)
- [Commits](https://github.com/kornelski/http-cache-semantics/commits)

---
updated-dependencies:
- dependency-name: http-cache-semantics
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 09:58:34 +00:00
Tommy Verrall 580b677489 Binary checker, introduction 2023-02-03 08:46:40 -01:00
Jędrzej Stuczyński 2093789c5c Added an option to set custom 'host' for the native client (#2939)
* Added an option to set custom 'host' for the native client

* Changelog entry
2023-02-02 16:38:39 +00:00
Pierre Dommerc d09864b99f build(nc-android): prepare for apk release (#2943)
* chore(nc-android): prepare for production build

* refactor(nc-android): remove dead code

* feat(nc-android): update native color theme

* feat(nc-android): update native color theme

* build(nc-android): fix rfd version issue

* build(nc-android): fix dist dir no such file error

* fix(nc-android): post rebase changes
2023-02-02 15:18:58 +01:00
Bogdan-Ștefan Neacşu f6e8278592 Fix typo during merge back to develop (#2956) 2023-02-02 13:55:58 +02:00
cgi-bin/ b085a8a636 typo: electrum (#2954) 2023-02-02 09:34:08 +00:00
farbanas e6ce531aeb fix: formatting 2023-02-01 17:16:53 +01:00
farbanas 1a860eb3f5 fix: wrong parameter type for addresses in generator commands (should be AccountId instead of String) 2023-02-01 12:45:51 +01:00
Fran Arbanas 09cfd9ff05 Update CHANGELOG.md 2023-01-31 14:42:16 +00:00
Bogdan-Ștefan Neacșu 2c88ca137a Resolve merge conflict 2023-01-31 16:19:34 +02:00
farbanas ca3bfc859a merge conflicts 2023-01-31 15:09:35 +01:00
2120 changed files with 70514 additions and 127214 deletions
+22 -17
View File
@@ -11,25 +11,30 @@
# In each subsection folders are ordered first by depth, then alphabetically. # In each subsection folders are ordered first by depth, then alphabetically.
# This should make it easy to add new rules without breaking existing ones. # This should make it easy to add new rules without breaking existing ones.
# contracts # Something weird not covered by anything else
/contracts/mixnet @durch @jstuczyn * @futurechimp @mmsinclair
/contracts/vesting @durch @jstuczyn
/contracts/service-provider-directory @octol
# crypto code # Rust rules:
/common/crypto/ @jstuczyn *.rs @durch @futurechimp @jstuczyn @neacsu @octol
/common/nymcoconut/ @jstuczyn Cargo.* @durch @futurechimp @jstuczyn @neacsu @octol
/common/dkg/ @jstuczyn
/common/nymsphinx/ @jstuczyn
# rust sdk # JS rules:
/sdk/rust/ @octol *.js @mmsinclair @fmtabbara
*.ts @mmsinclair @fmtabbara
*.tsx @mmsinclair @fmtabbara
*.jsx @mmsinclair @fmtabbara
# nym-connect (rust) # Something looking like possible documentation rules:
/nym-connect/desktop/src-tauri/ @octol *.md @mfahampshire
# nym-wallet (rust) # our docker scripts
/nym-wallet/src-tauri/ @octol /docker/ @neacsu
# documentation # if there are any changes in the core crypto, I feel like Ania should take a look:
/documentation @mfahampshire /common/crypto/ @aniampio
/common/nymsphinx/ @aniampio
# Explorer and wallet should probably get looked by the product team
/explorer/ @nymtech/product
/nym-wallet/ @nymtech/product
/wallet-web/ @nymtech/product
-14
View File
@@ -1,14 +0,0 @@
---
name: 'Documentation'
about: Suggest a fix or enhancement to the documentation or developer portal content
title: "[DOCS]"
labels: documentation
assignees: mfahampshire
---
Is your issue either:
- [ ] a fix to existing documentation (e.g. fixing a broken link or incorrect command)
- [ ] an enhancement (e.g. adding a description for an undocumented feature)
Please briefly describe your issue:
-16
View File
@@ -1,16 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
commit-message:
prefix: build
prefix-development: chore
include: scope
+7 -3
View File
@@ -39,16 +39,20 @@ jobs:
uses: actions/setup-node@v3 uses: actions/setup-node@v3
with: with:
node-version: 16 node-version: 16
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: security NYM_NOTIFICATION_KIND: security
NYM_PROJECT_NAME: "Daily security report" NYM_PROJECT_NAME: "Daily security report"
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "security"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_AUDIT }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}" MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}" MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}" MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
@@ -34,7 +34,7 @@ on:
- 'tools/ts-rs-cli/**' - 'tools/ts-rs-cli/**'
env: env:
NETWORK: mainnet NETWORK: mainnet
jobs: jobs:
publish-nym: publish-nym:
@@ -46,7 +46,7 @@ jobs:
runs-on: ${{ matrix.platform }} runs-on: ${{ matrix.platform }}
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Prepare build output directory - name: Prepare build output directory
shell: bash shell: bash
env: env:
@@ -63,25 +63,22 @@ jobs:
- name: Install Rust stable - name: Install Rust stable
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
with: with:
toolchain: 1.69.0 toolchain: stable
- name: Build all binaries - name: Build all binaries
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: build command: build
args: --workspace --release --all args: --workspace --release
- name: Install Rust stable - name: Install Rust stable
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
with: with:
toolchain: 1.69.0 toolchain: stable
target: wasm32-unknown-unknown target: wasm32-unknown-unknown
override: true override: true
components: rustfmt, clippy components: rustfmt, clippy
- name: Install wasm-opt
run: cargo install --version 0.112.0 wasm-opt
- name: Build release contracts - name: Build release contracts
run: make wasm run: make wasm
@@ -98,17 +95,9 @@ jobs:
cp target/release/nym-network-requester $OUTPUT_DIR cp target/release/nym-network-requester $OUTPUT_DIR
cp target/release/nym-network-statistics $OUTPUT_DIR cp target/release/nym-network-statistics $OUTPUT_DIR
cp target/release/nym-cli $OUTPUT_DIR cp target/release/nym-cli $OUTPUT_DIR
cp target/release/nym-credential-client $OUTPUT_DIR
cp target/release/explorer-api $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_bandwidth.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_service_provider_directory.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_name_service.wasm $OUTPUT_DIR
- name: Deploy branch to CI www - name: Deploy branch to CI www
continue-on-error: true continue-on-error: true
@@ -121,3 +110,4 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }} REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/ TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/
EXCLUDE: "/dist/, /node_modules/" EXCLUDE: "/dist/, /node_modules/"
+6 -2
View File
@@ -43,10 +43,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }} REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ts-${{ env.GITHUB_REF_SLUG }}-example TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ts-${{ env.GITHUB_REF_SLUG }}-example
EXCLUDE: "/dist/, /node_modules/" EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: ts-packages NYM_NOTIFICATION_KIND: ts-packages
NYM_PROJECT_NAME: "ts-packages" NYM_PROJECT_NAME: "ts-packages"
@@ -54,6 +54,10 @@ jobs:
NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}" NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-ts-packages"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
-4
View File
@@ -9,14 +9,12 @@ on:
- 'gateway/**' - 'gateway/**'
- 'integrations/**' - 'integrations/**'
- 'mixnode/**' - 'mixnode/**'
- 'sdk/lib/socks5-listener/**'
- 'sdk/rust/nym-sdk/**' - 'sdk/rust/nym-sdk/**'
- 'service-providers/**' - 'service-providers/**'
- 'nym-api/**' - 'nym-api/**'
- 'nym-outfox/**' - 'nym-outfox/**'
- 'tools/nym-cli/**' - 'tools/nym-cli/**'
- 'tools/ts-rs-cli/**' - 'tools/ts-rs-cli/**'
- 'Cargo.toml'
pull_request: pull_request:
paths: paths:
- 'clients/**' - 'clients/**'
@@ -25,14 +23,12 @@ on:
- 'gateway/**' - 'gateway/**'
- 'integrations/**' - 'integrations/**'
- 'mixnode/**' - 'mixnode/**'
- 'sdk/lib/socks5-listener/**'
- 'sdk/rust/nym-sdk/**' - 'sdk/rust/nym-sdk/**'
- 'service-providers/**' - 'service-providers/**'
- 'nym-api/**' - 'nym-api/**'
- 'nym-outfox/**' - 'nym-outfox/**'
- 'tools/nym-cli/**' - 'tools/nym-cli/**'
- 'tools/ts-rs-cli/**' - 'tools/ts-rs-cli/**'
- 'Cargo.toml'
jobs: jobs:
build: build:
-76
View File
@@ -1,76 +0,0 @@
name: CD dev-portal
on:
push:
branches: master
paths:
- 'documentation/dev-portal/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck
- name: Clean website
run: cd documentation/dev-portal && mdbook clean
- name: Build website
run: cd documentation/dev-portal && mdbook build
- name: Deploy branch master to dev
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_DEV }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET_DEVP }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Deploy branch master to prod
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_PROD }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET_DEVP }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: cd-dev
NYM_PROJECT_NAME: "Dev portal CD"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CD_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DEVP }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
-82
View File
@@ -1,82 +0,0 @@
name: CD docs
on:
push:
branches: master
paths:
- 'documentation/docs/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --release --all
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck && \
cargo install --vers "^0.5.0" mdbook-cmdrun
- name: Clean website
run: cd documentation/docs && mdbook clean
- name: Build website
run: cd documentation/docs && mdbook build
- name: Deploy branch master to dev
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_DEV }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Deploy branch master to prod
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CD_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CD_WWW_REMOTE_HOST_PROD }}
REMOTE_USER: ${{ secrets.CD_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CD_WWW_REMOTE_TARGET }}/
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: cd-docs
NYM_PROJECT_NAME: "Docs CD"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CD_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DOCS }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
@@ -1,61 +0,0 @@
name: check-merge-conflicts
# Check that the latest release branch merges into master and develop without
# any conflicts that git is not able to resolve
on:
workflow_dispatch:
schedule:
- cron: '5 6 * * *'
jobs:
get_release:
runs-on: ubuntu-latest
outputs:
output1: ${{ steps.step2.outputs.latest_release }}
steps:
- name: Check out repository code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set output variable to latest release branch
id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 1)" >> $GITHUB_OUTPUT
check-merge-release-into-master:
name: Check that the release branch merges into master
needs: get_release
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup git user
run: |
git config --global user.name "ci"
git config --global user.email "ci@localhost"
- name: Check merge release branch into master
run: |
./.github/workflows/support-files/git-merge-check.sh origin/master $branch1
env:
branch1: ${{needs.get_release.outputs.output1}}
check-merge-release-into-develop:
name: Check that the release branch merges into develop
needs: get_release
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup git user
run: |
git config --global user.name "ci"
git config --global user.email "ci@localhost"
- name: Check merge release branch into develop
run: |
./.github/workflows/support-files/git-merge-check.sh origin/develop $branch1
env:
branch1: ${{needs.get_release.outputs.output1}}
-56
View File
@@ -1,56 +0,0 @@
name: Run config checks on all binaries
on:
workflow_dispatch:
release:
types: [created]
push:
paths:
- 'clients/**'
- 'common/**'
- 'contracts/**'
- 'integrations/**'
- 'mixnode/**'
- 'sdk/rust/nym-sdk/**'
- 'service-providers/**'
pull_request:
paths:
- 'clients/**'
- 'common/**'
- 'gateway/**'
- 'integrations/**'
- 'mixnode/**'
- 'sdk/rust/nym-sdk/**'
- 'service-providers/**'
env:
NETWORK: mainnet
jobs:
publish-nym:
strategy:
fail-fast: false
matrix:
platform: [custom-runner-linux]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install jq vim libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools
continue-on-error: true
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Branch name
run: echo running on branch ${GITHUB_REF##*/}
- name: Run tests against binaries
run: ./build_and_run.sh ${{ github.head_ref || github.ref_name }}
working-directory: tests/
-66
View File
@@ -1,66 +0,0 @@
name: CI dev-portal
on:
push:
branches-ignore: master
paths:
- 'documentation/dev-portal/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" mdbook-theme \
&& cargo install --vers "^0.7.7" mdbook-linkcheck
- name: Clean website
run: cd documentation/dev-portal && mdbook clean
- name: Build website
run: cd documentation/dev-portal && mdbook build
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/dev-portal/book/html/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/dev-portal-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: ci-dev
NYM_PROJECT_NAME: "Dev portal CI"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CI_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "dev-portal-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DEVP }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
-72
View File
@@ -1,72 +0,0 @@
name: CI docs
on:
push:
branches-ignore: master
paths:
- 'documentation/docs/**'
jobs:
build:
runs-on: custom-runner-linux
steps:
- uses: actions/checkout@v3
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v3
with:
node-version: "16"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --release --all
- name: Install mdbook
run: (test -x $HOME/.cargo/bin/mdbook || cargo install --vers "^0.4" mdbook)
- name: Install mdbook plugins
run: |
cargo install --vers "^0.2.0" mdbook-variables && cargo install \
--vers "^1.8.0" mdbook-admonish && cargo install --vers \
"^0.1.2" mdbook-last-changed && cargo install --vers "^0.1.2" \
mdbook-theme && cargo install --vers "^0.7.7" mdbook-linkcheck && \
cargo install --vers "^0.5.0" mdbook-cmdrun
- name: Clean website
run: cd documentation/docs && mdbook clean
- name: Build website
run: cd documentation/docs && mdbook build
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "documentation/docs/book/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/docs-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: ci-docs
NYM_PROJECT_NAME: "Docs CI"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CI_WWW_BASE }}"
NYM_CI_WWW_LOCATION: "docs-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_DOCS }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
@@ -1,11 +1,9 @@
name: Nym Connect - mobile (Rust) name: Nym Connect for Android (rust)
on: on:
push: push:
paths: paths:
- "nym-connect/mobile/src-tauri/**" - "nym-connect-android/src-tauri/**"
- "nym-connect/mobile/src-tauri/Cargo.toml"
- "!nym-connect/mobile/src-tauri/gen/**"
- "clients/client-core/**" - "clients/client-core/**"
- "clients/socks5/**" - "clients/socks5/**"
- "common/**" - "common/**"
@@ -14,9 +12,7 @@ on:
- "nym-api/nym-api-requests/**" - "nym-api/nym-api-requests/**"
pull_request: pull_request:
paths: paths:
- "nym-connect/mobile/src-tauri/**" - "nym-connect-android/src-tauri/**"
- "nym-connect/mobile/src-tauri/Cargo.toml"
- "!nym-connect/mobile/src-tauri/gen/**"
- "clients/client-core/**" - "clients/client-core/**"
- "clients/socks5/**" - "clients/socks5/**"
- "common/**" - "common/**"
@@ -32,9 +28,10 @@ jobs:
#RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache #RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache
#defaults: #defaults:
#run: #run:
#working-directory: nym-connect/mobile/src-tauri/ #working-directory: nym-connect-android/src-tauri/
steps: steps:
- name: Install Dependencies (Linux) - name: Install Dependencies (Linux)
if: ${{ !env.ACT }}
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get -y install \ sudo apt-get -y install \
@@ -49,6 +46,7 @@ jobs:
librsvg2-dev \ librsvg2-dev \
libsoup-3.0-dev \ libsoup-3.0-dev \
libjavascriptcoregtk-4.1-dev libjavascriptcoregtk-4.1-dev
#continue-on-error: true
- name: Checkout - name: Checkout
uses: actions/checkout@v3 uses: actions/checkout@v3
@@ -60,13 +58,13 @@ jobs:
components: clippy, rustfmt components: clippy, rustfmt
- name: Check formatting - name: Check formatting
run: cargo fmt --manifest-path nym-connect/mobile/src-tauri/Cargo.toml -- --check run: cargo fmt --manifest-path nym-connect-android/src-tauri/Cargo.toml -- --check
- name: Build all binaries - name: Build all binaries
run: cargo build --manifest-path nym-connect/mobile/src-tauri/Cargo.toml run: cargo build --manifest-path nym-connect-android/src-tauri/Cargo.toml
- name: Run all tests - name: Run all tests
run: cargo test --manifest-path nym-connect/mobile/src-tauri/Cargo.toml run: cargo test --manifest-path nym-connect-android/src-tauri/Cargo.toml
- name: Clippy - name: Clippy
run: cargo clippy --manifest-path nym-connect/mobile/src-tauri/Cargo.toml --all-targets -- -D warnings run: cargo clippy --manifest-path nym-connect-android/src-tauri/Cargo.toml --all-targets -- -D warnings
@@ -1,30 +1,28 @@
name: Nym Connect - desktop (Rust) name: Nym Connect (rust)
on: on:
push: push:
paths: paths:
- "nym-connect/desktop/src-tauri/**" - 'nym-connect/**'
- "nym-connect/desktop/src-tauri/Cargo.toml" - 'clients/client-core/**'
- "clients/client-core/**" - 'clients/socks5/**'
- "clients/socks5/**" - 'common/**'
- "common/**" - 'gateway/gateway-requests/**'
- "gateway/gateway-requests/**" - 'contracts/vesting/**'
- "contracts/vesting/**" - 'nym-api/nym-api-requests/**'
- "nym-api/nym-api-requests/**"
pull_request: pull_request:
paths: paths:
- "nym-connect/desktop/src-tauri/**" - 'nym-connect/**'
- "nym-connect/desktop/src-tauri/Cargo.toml" - 'clients/client-core/**'
- "clients/client-core/**" - 'clients/socks5/**'
- "clients/socks5/**" - 'common/**'
- "common/**" - 'gateway/gateway-requests/**'
- "gateway/gateway-requests/**" - 'contracts/vesting/**'
- "contracts/vesting/**" - 'nym-api/nym-api-requests/**'
- "nym-api/nym-api-requests/**"
jobs: jobs:
build: build:
runs-on: [self-hosted, custom-linux] runs-on: [ self-hosted, custom-linux ]
env: env:
RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache RUSTC_WRAPPER: /home/ubuntu/.cargo/bin/sccache
steps: steps:
@@ -47,29 +45,29 @@ jobs:
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: build command: build
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace args: --manifest-path nym-connect/Cargo.toml --workspace
- name: Run all tests - name: Run all tests
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: test command: test
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace args: --manifest-path nym-connect/Cargo.toml --workspace
- name: Check formatting - name: Check formatting
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: fmt command: fmt
args: --manifest-path nym-connect/desktop/Cargo.toml --all -- --check args: --manifest-path nym-connect/Cargo.toml --all -- --check
- uses: actions-rs/clippy-check@v1 - uses: actions-rs/clippy-check@v1
name: Clippy checks name: Clippy checks
continue-on-error: true continue-on-error: true
with: with:
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace --all-features args: --manifest-path nym-connect/Cargo.toml --workspace --all-features
- name: Run clippy - name: Run clippy
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: clippy command: clippy
args: --manifest-path nym-connect/desktop/Cargo.toml --workspace --all-features -- -D warnings args: --manifest-path nym-connect/Cargo.toml --workspace --all-features -- -D warnings
+1 -4
View File
@@ -6,7 +6,7 @@ on:
jobs: jobs:
build: build:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-contracts-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-contracts-') && github.event_name == 'release' }}
runs-on: [self-hosted, custom-runner-linux] runs-on: [self-hosted, custom-runner-linux]
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@@ -19,9 +19,6 @@ jobs:
override: true override: true
components: rustfmt, clippy components: rustfmt, clippy
- name: Install wasm-opt
run: cargo install --version 0.112.0 wasm-opt
- name: Build release contracts - name: Build release contracts
run: make wasm run: make wasm
+1 -2
View File
@@ -30,7 +30,6 @@ jobs:
continue-on-error: ${{ matrix.rust == 'nightly' }} continue-on-error: ${{ matrix.rust == 'nightly' }}
needs: matrix_prep needs: matrix_prep
strategy: strategy:
fail-fast: false
matrix: ${{fromJson(needs.matrix_prep.outputs.matrix)}} matrix: ${{fromJson(needs.matrix_prep.outputs.matrix)}}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@@ -64,4 +63,4 @@ jobs:
if: ${{ matrix.rust != 'nightly' }} if: ${{ matrix.rust != 'nightly' }}
with: with:
command: clippy command: clippy
args: --manifest-path contracts/Cargo.toml --workspace --all-targets -- -D warnings args: --manifest-path contracts/Cargo.toml --workspace -- -D warnings
-13
View File
@@ -1,13 +0,0 @@
name: Greetings
on: [pull_request_target, issues]
jobs:
greeting:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for raising this issue'
pr-message: 'Thank you for making this first PR'
+127
View File
@@ -0,0 +1,127 @@
name: NC Android APK Release
on:
workflow_dispatch:
push:
branches:
- "release/nc-android-v[0-9].[0-9].[0-9]*"
jobs:
build:
name: Build APK
runs-on: ubuntu-latest
env:
ANDROID_HOME: ${{ github.workspace }}/android-sdk
NDK_VERSION: 25.1.8937393
NDK_HOME: ${{ env.ANDROID_HOME }}/ndk/${{ env.NDK_VERSION }}
SDK_PLATFORM_VERSION: android-33
SDK_BUILDTOOLS_VERSION: 33.0.1
steps:
- name: Install Dependencies (Linux)
# https://next--tauri.netlify.app/next/guides/getting-started/prerequisites/linux/#1-system-dependencies
run: |
sudo apt-get update
sudo apt-get -y install \
libwebkit2gtk-4.0-dev \
build-essential \
curl \
wget \
libssl-dev \
libgtk-3-dev \
squashfs-tools \
libayatana-appindicator3-dev \
librsvg2-dev
- name: Install Java
uses: actions/setup-java@v3
with:
distribution: "temurin"
java-version: "17"
- name: Install Android SDK manager
# https://developer.android.com/studio/command-line/sdkmanager
run: |
curl -sS https://dl.google.com/android/repository/commandlinetools-linux-9477386_latest.zip -o cmdline-tools.zip
unzip cmdline-tools.zip
mkdir -p $ANDROID_HOME/cmdline-tools/latest
mv cmdline-tools/* $ANDROID_HOME/cmdline-tools/latest
rm -rf cmdline-tools
- name: Install Android S/NDK
run: |
echo y | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager --licenses
echo y | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager \
"platforms;$SDK_PLATFORM_VERSION" \
"platform-tools" \
"ndk;$NDK_VERSION" \
"build-tools;$SDK_BUILDTOOLS_VERSION"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Install tauri cli
run: cargo install tauri-cli --version "^2.0.0-alpha.2"
- name: Install rust android targets
run: |
rustup target add aarch64-linux-android \
armv7-linux-androideabi \
i686-linux-android \
x86_64-linux-android
- name: Setup Nodejs
uses: actions/setup-node@v3
with:
node-version: 18
- name: Install yarn
run: |
npm i -g yarn
yarn --version
- name: Checkout
uses: actions/checkout@v3
- name: Build frontend code
run: |
yarn install --frozen-lockfile
yarn build
yarn workspace @nym/nym-connect-android webpack:prod
- name: Build APK
working-directory: nym-connect-android
env:
WRY_ANDROID_PACKAGE: net.nymtech.nym_connect_android
WRY_ANDROID_LIBRARY: nym_connect_android
# TODO build with release profile (--release), it will requires
# to sign the APK. For now build with debug profile to avoid that
run: cargo tauri android build --debug --apk
# TODO add the version number to APK name
- name: Rename APK artifact
run: |
mv nym-connect-android/src-tauri/gen/android/nym_connect_android/app/build/outputs/apk/universal/debug/app-universal-debug.apk \
nym-connect-debug.apk
- name: Upload APK artifact
uses: actions/upload-artifact@v3
with:
name: nc-apk-debug
path: nym-connect-debug.apk
publish:
name: Publish APK
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Download binary artifact
uses: actions/download-artifact@v3
with:
name: nc-apk-debug
path: apk
# TODO add a step to upload the APK somewhere
# - name: Publish
# uses: ???
@@ -0,0 +1,57 @@
name: CI for Network Explorer API
on:
workflow_dispatch:
release:
types: [created]
env:
NETWORK: mainnet
jobs:
publish-nym:
if: ${{ startsWith(github.ref, 'refs/tags/nym-explorer-api-') && (github.event_name == 'release' || github.event_name == 'workflow_dispatch') }}
strategy:
fail-fast: false
matrix:
platform: [ubuntu-20.04]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools
continue-on-error: true
- name: Check the release tag starts with `nym-explorer-api-`
uses: actions/github-script@v3
with:
script: |
core.setFailed('Release tag did not start with nym-explorer-api-...')
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build all explorer-api
uses: actions-rs/cargo@v1
with:
command: build
args: --manifest-path explorer-api/Cargo.toml --workspace --release
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: my-artifact
path: |
target/release/explorer-api
retention-days: 30
- name: Upload to release based on tag name
uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with:
files: |
target/release/explorer-api
+6 -2
View File
@@ -57,10 +57,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }} REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ne-sb-${{ env.GITHUB_REF_SLUG }} TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/ne-sb-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/" EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: network-explorer NYM_NOTIFICATION_KIND: network-explorer
NYM_PROJECT_NAME: "Network Explorer" NYM_PROJECT_NAME: "Network Explorer"
@@ -69,6 +69,10 @@ jobs:
NYM_CI_WWW_LOCATION_STORYBOOK: "ne-sb-${{ env.GITHUB_REF_SLUG }}" NYM_CI_WWW_LOCATION_STORYBOOK: "ne-sb-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-network-explorer"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
+17 -13
View File
@@ -10,7 +10,7 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }} matrix: ${{ steps.set-matrix.outputs.matrix }}
steps: steps:
# creates the matrix strategy from nightly_build_matrix_includes.json # creates the matrix strategy from nightly_build_matrix_includes.json
- uses: actions/checkout@v3 - uses: actions/checkout@v2
- id: set-matrix - id: set-matrix
uses: JoshuaTheMiller/conditional-build-matrix@main uses: JoshuaTheMiller/conditional-build-matrix@main
with: with:
@@ -29,7 +29,7 @@ jobs:
if: matrix.os == 'ubuntu-20.04' if: matrix.os == 'ubuntu-20.04'
- name: Check out repository code - name: Check out repository code
uses: actions/checkout@v3 uses: actions/checkout@v2
- name: Install rust toolchain - name: Install rust toolchain
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
@@ -51,9 +51,9 @@ jobs:
command: build command: build
args: --workspace args: --workspace
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -63,9 +63,9 @@ jobs:
command: build command: build
args: --workspace --examples args: --workspace --examples
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -75,7 +75,7 @@ jobs:
command: test command: test
args: --workspace args: --workspace
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
with: with:
@@ -88,9 +88,9 @@ jobs:
command: test command: test
args: --workspace -- --ignored args: --workspace -- --ignored
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -147,17 +147,17 @@ jobs:
- name: Collect jobs status - name: Collect jobs status
uses: technote-space/workflow-conclusion-action@v2 uses: technote-space/workflow-conclusion-action@v2
- name: Check out repository code - name: Check out repository code
uses: actions/checkout@v3 uses: actions/checkout@v2
- name: install npm - name: install npm
uses: actions/setup-node@v3 uses: actions/setup-node@v3
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
with: with:
node-version: 16 node-version: 16
- name: Matrix - Node Install - name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
env: env:
NYM_NOTIFICATION_KIND: nightly NYM_NOTIFICATION_KIND: nightly
@@ -165,9 +165,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}" IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}" MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}" MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}" MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
@@ -6,7 +6,7 @@
}, },
{ {
"os":"windows10", "os":"windows-latest",
"rust":"stable", "rust":"stable",
"runOnEvent":"schedule" "runOnEvent":"schedule"
}, },
@@ -22,7 +22,7 @@
"runOnEvent":"schedule" "runOnEvent":"schedule"
}, },
{ {
"os":"windows10", "os":"windows-latest",
"rust":"beta", "rust":"beta",
"runOnEvent":"schedule" "runOnEvent":"schedule"
}, },
@@ -38,7 +38,7 @@
"runOnEvent":"schedule" "runOnEvent":"schedule"
}, },
{ {
"os":"windows10", "os":"windows-latest",
"rust":"nightly", "rust":"nightly",
"runOnEvent":"schedule" "runOnEvent":"schedule"
}, },
+22 -30
View File
@@ -28,7 +28,7 @@ jobs:
run: git fetch --all run: git fetch --all
- name: Set output variable to latest release branch - name: Set output variable to latest release branch
id: step2 id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+' | tail -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
build: build:
needs: [get_release,matrix_prep] needs: [get_release,matrix_prep]
strategy: strategy:
@@ -54,33 +54,15 @@ jobs:
override: true override: true
components: rustfmt, clippy components: rustfmt, clippy
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Build all binaries - name: Build all binaries
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: build command: build
args: --workspace args: --workspace
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
- name: Build all examples
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Reclaim some disk space
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }}
with: with:
command: clean command: clean
@@ -90,9 +72,9 @@ jobs:
command: test command: test
args: --workspace args: --workspace
- name: Reclaim some disk space - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -101,11 +83,17 @@ jobs:
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: test command: test
args: --workspace -- --ignored args: --workspace --all-features -- --ignored
- name: Reclaim some disk space - name: Check formatting
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} with:
command: fmt
args: --all -- --check
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -114,7 +102,7 @@ jobs:
continue-on-error: true continue-on-error: true
with: with:
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
args: --workspace args: --all-features
- name: Run clippy - name: Run clippy
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
@@ -168,11 +156,11 @@ jobs:
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
with: with:
node-version: 16 node-version: 16
- name: Matrix - Node Install - name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
env: env:
NYM_NOTIFICATION_KIND: nightly NYM_NOTIFICATION_KIND: nightly
@@ -180,9 +168,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}" GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly-release"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}" IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}" MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}" MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}" MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
+17 -25
View File
@@ -28,7 +28,7 @@ jobs:
run: git fetch --all run: git fetch --all
- name: Set output variable to latest release branch - name: Set output variable to latest release branch
id: step2 id: step2
run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n 2 | head -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT run: echo "latest_release=$(git branch -r | grep -E 'release/v[0-9]+\.[0-9]+\.[0-9]+' | tail -n 2 | head -n 1 | sed 's/ origin\///')" >> $GITHUB_OUTPUT
build: build:
needs: [get_release,matrix_prep] needs: [get_release,matrix_prep]
strategy: strategy:
@@ -54,12 +54,6 @@ jobs:
override: true override: true
components: rustfmt, clippy components: rustfmt, clippy
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Build all binaries - name: Build all binaries
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
@@ -72,18 +66,6 @@ jobs:
with: with:
command: clean command: clean
- name: Build all examples
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' }}
with:
command: clean
- name: Run all tests - name: Run all tests
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
@@ -92,7 +74,7 @@ jobs:
- name: Reclaim some disk space (because Windows is being annoying) - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
if: ${{ matrix.os == 'windows-latest' || matrix.os == 'ubuntu-20.04' }} if: ${{ matrix.os == 'windows-latest' }}
with: with:
command: clean command: clean
@@ -101,7 +83,13 @@ jobs:
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: test command: test
args: --workspace -- --ignored args: --workspace --all-features -- --ignored
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Reclaim some disk space (because Windows is being annoying) - name: Reclaim some disk space (because Windows is being annoying)
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
@@ -114,7 +102,7 @@ jobs:
continue-on-error: true continue-on-error: true
with: with:
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
args: --workspace args: --all-features
- name: Run clippy - name: Run clippy
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
@@ -168,11 +156,11 @@ jobs:
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
with: with:
node-version: 16 node-version: 16
- name: Matrix - Node Install - name: Keybase - Node Install
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
if: env.WORKFLOW_CONCLUSION == 'failure' if: env.WORKFLOW_CONCLUSION == 'failure'
env: env:
NYM_NOTIFICATION_KIND: nightly NYM_NOTIFICATION_KIND: nightly
@@ -180,9 +168,13 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}" GIT_BRANCH_NAME: "${{needs.get_release.outputs.output1}}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nightly-release"
IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}" IS_SUCCESS: "${{ env.WORKFLOW_CONCLUSION == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM_NIGHTLY }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}" MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}" MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}" MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
+50
View File
@@ -0,0 +1,50 @@
name: Publish Nym CLI binaries
on:
workflow_dispatch:
release:
types: [created]
env:
NETWORK: mainnet
jobs:
publish-nym-cli:
if: ${{ startsWith(github.ref, 'refs/tags/nym-cli-') && (github.event_name == 'release' || github.event_name = 'workflow_dispatch') }}
strategy:
fail-fast: false
matrix:
platform: [ubuntu-20.04, windows-latest, macos-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Check the release tag starts with `nym-cli-`
uses: actions/github-script@v3
with:
script: |
core.setFailed('Release tag did not start with nym-cli-...')
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Build binary
run: make build-nym-cli
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: nym-cli-${{ matrix.platform }}
path: |
target/release/nym-cli*
retention-days: 30
- name: Upload to release based on tag name
uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with:
files: |
target/release/nym-cli
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (MacOS) name: Publish Nym Connect (MacOS)
on: on:
workflow_dispatch: workflow_dispatch:
release: release:
@@ -6,11 +6,11 @@ on:
defaults: defaults:
run: run:
working-directory: nym-connect/desktop working-directory: nym-connect
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -73,7 +73,7 @@ jobs:
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: nym-connect_1.0.0_x64.dmg name: nym-connect_1.0.0_x64.dmg
path: nym-connect/desktop/target/release/bundle/dmg/nym-connect_1.0.0_x64.dmg path: nym-connect/target/release/bundle/dmg/nym-connect_1.0.0_x64.dmg
retention-days: 30 retention-days: 30
- name: Clean up keychain - name: Clean up keychain
@@ -86,5 +86,5 @@ jobs:
if: github.event_name == 'release' if: github.event_name == 'release'
with: with:
files: | files: |
nym-connect/desktop/target/release/bundle/dmg/*.dmg nym-connect/target/release/bundle/dmg/*.dmg
nym-connect/desktop/target/release/bundle/macos/*.app.tar.gz* nym-connect/target/release/bundle/macos/*.app.tar.gz*
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (Ubuntu) name: Publish Nym Connect (Ubuntu)
on: on:
workflow_dispatch: workflow_dispatch:
release: release:
@@ -6,11 +6,11 @@ on:
defaults: defaults:
run: run:
working-directory: nym-connect/desktop working-directory: nym-connect
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -53,7 +53,7 @@ jobs:
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: nym-connect.AppImage.tar.gz name: nym-connect.AppImage.tar.gz
path: nym-connect/desktop/target/release/bundle/appimage/nym-connect_1.0.0_amd64.AppImage path: nym-connect/target/release/bundle/appimage/nym-connect_1.0.0_amd64.AppImage
retention-days: 30 retention-days: 30
- name: Upload to release based on tag name - name: Upload to release based on tag name
@@ -61,5 +61,5 @@ jobs:
if: github.event_name == 'release' if: github.event_name == 'release'
with: with:
files: | files: |
nym-connect/desktop/target/release/bundle/appimage/*.AppImage nym-connect/target/release/bundle/appimage/*.AppImage
nym-connect/desktop/target/release/bundle/appimage/*.AppImage.tar.gz* nym-connect/target/release/bundle/appimage/*.AppImage.tar.gz*
@@ -1,4 +1,4 @@
name: Publish Nym Connect - desktop (Windows 10) name: Publish Nym Connect (Windows 10)
on: on:
workflow_dispatch: workflow_dispatch:
release: release:
@@ -6,11 +6,11 @@ on:
defaults: defaults:
run: run:
working-directory: nym-connect/desktop working-directory: nym-connect
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-connect-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -56,7 +56,7 @@ jobs:
encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }} encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }}
- name: Install app dependencies - name: Install app dependencies
run: yarn --network-timeout 100000 run: yarn
- name: Build and sign it - name: Build and sign it
env: env:
@@ -72,7 +72,7 @@ jobs:
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: nym-connect_1.0.0_x64_en-US.msi name: nym-connect_1.0.0_x64_en-US.msi
path: nym-connect/desktop/target/release/bundle/msi/nym-connect_1.0.0_x64_en-US.msi path: nym-connect/target/release/bundle/msi/nym-connect_1.0.0_x64_en-US.msi
retention-days: 30 retention-days: 30
- name: Upload to release based on tag name - name: Upload to release based on tag name
@@ -80,5 +80,5 @@ jobs:
if: github.event_name == 'release' if: github.event_name == 'release'
with: with:
files: | files: |
nym-connect/desktop/target/release/bundle/msi/*.msi nym-connect/target/release/bundle/msi/*.msi
nym-connect/desktop/target/release/bundle/msi/*.msi.zip* nym-connect/target/release/bundle/msi/*.msi.zip*
@@ -1,13 +1,13 @@
name: CI for nym-connect - Desktop name: CI for nym-connect
on: on:
push: push:
paths: paths:
- 'nym-connect/desktop/**' - 'nym-connect/**'
defaults: defaults:
run: run:
working-directory: nym-connect/desktop working-directory: nym-connect
jobs: jobs:
build: build:
@@ -34,15 +34,15 @@ jobs:
env: env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }} SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete" ARGS: "-rltgoDzvO --delete"
SOURCE: "nym-connect/desktop/storybook-static/" SOURCE: "nym-connect/storybook-static/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }} REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }} REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/nym-connect-${{ env.GITHUB_REF_SLUG }} TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/nym-connect-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/" EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: nym-connect NYM_NOTIFICATION_KIND: nym-connect
NYM_PROJECT_NAME: "nym-connect" NYM_PROJECT_NAME: "nym-connect"
@@ -50,6 +50,10 @@ jobs:
NYM_CI_WWW_LOCATION: "nym-connect-${{ env.GITHUB_REF_SLUG }}" NYM_CI_WWW_LOCATION: "nym-connect-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nym-connect"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
+1 -1
View File
@@ -16,7 +16,7 @@ env:
jobs: jobs:
publish-nym: publish-nym:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -10,7 +10,7 @@ defaults:
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -9,7 +9,7 @@ defaults:
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -1,6 +1,5 @@
name: Publish Nym Wallet (Windows 10) name: Publish Nym Wallet (Windows 10)
on: on:
workflow_dispatch:
release: release:
types: [created] types: [created]
@@ -10,7 +9,7 @@ defaults:
jobs: jobs:
publish-tauri: publish-tauri:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
@@ -56,7 +55,7 @@ jobs:
encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }} encodedString: ${{ secrets.WALLET_ADMIN_ADDRESS }}
- name: Install app dependencies - name: Install app dependencies
run: yarn --network-timeout 100000 run: yarn
- name: Build and sign it - name: Build and sign it
env: env:
@@ -68,16 +67,8 @@ jobs:
TAURI_KEY_PASSWORD: ${{ secrets.TAURI_KEY_PASSWORD }} TAURI_KEY_PASSWORD: ${{ secrets.TAURI_KEY_PASSWORD }}
run: yarn build run: yarn build
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: nym-wallet_1.0.0_x64_en-US.msi
path: nym-wallet/target/release/bundle/msi/nym-wallet_1.*.msi
retention-days: 30
- name: Upload to release based on tag name - name: Upload to release based on tag name
uses: softprops/action-gh-release@v1 uses: softprops/action-gh-release@v1
if: github.event_name == 'release'
with: with:
files: | files: |
nym-wallet/target/release/bundle/msi/*.msi nym-wallet/target/release/bundle/msi/*.msi
+6 -2
View File
@@ -35,10 +35,10 @@ jobs:
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }} REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/wallet-${{ env.GITHUB_REF_SLUG }} TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/wallet-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/" EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: nym-wallet NYM_NOTIFICATION_KIND: nym-wallet
NYM_PROJECT_NAME: "nym-wallet" NYM_PROJECT_NAME: "nym-wallet"
@@ -46,6 +46,10 @@ jobs:
NYM_CI_WWW_LOCATION: "wallet-${{ env.GITHUB_REF_SLUG }}" NYM_CI_WWW_LOCATION: "wallet-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-nym-wallet"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
-110
View File
@@ -1,110 +0,0 @@
name: Nyms5 Android
# unsigned APKs only, supported archs:
# - arm64-v8a (arm64)
# - x86_64
on:
workflow_dispatch:
push:
tags:
- nyms5-android-v*
jobs:
build:
name: Build APK
runs-on: custom-runner-linux
env:
ANDROID_HOME: ${{ github.workspace }}/android-sdk
NDK_VERSION: 25.2.9519653
NDK_HOME: ${{ github.workspace }}/android-sdk/ndk/25.2.9519653
SDK_PLATFORM_VERSION: android-33
SDK_BUILDTOOLS_VERSION: 33.0.2
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install Java
uses: actions/setup-java@v3
with:
distribution: "temurin"
java-version: "17"
- name: Install Android SDK manager
# https://developer.android.com/studio/command-line/sdkmanager
run: |
curl -sS https://dl.google.com/android/repository/commandlinetools-linux-9477386_latest.zip -o cmdline-tools.zip
unzip cmdline-tools.zip
mkdir -p $ANDROID_HOME/cmdline-tools/latest
mv cmdline-tools/* $ANDROID_HOME/cmdline-tools/latest
rm -rf cmdline-tools
- name: Install Android S/NDK
run: |
echo y | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager --licenses
echo y | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager \
"platforms;$SDK_PLATFORM_VERSION" \
"platform-tools" \
"ndk;$NDK_VERSION" \
"build-tools;$SDK_BUILDTOOLS_VERSION"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.70.0
- name: Install rust android targets
run: |
rustup target add aarch64-linux-android \
x86_64-linux-android
- name: Build lib nym-socks5-listener
working-directory: sdk/lib/socks5-listener/
env:
RELEASE: true
RUSTFLAGS: "-C link-args=-Wl,--hash-style=gnu"
# build for arm64 and x86_64
run: ./build-android.sh aarch64 x86_64
- name: Build APKs (unsigned)
working-directory: nym-connect/native/android
env:
ANDROID_SDK_ROOT: ${{ env.ANDROID_HOME }}
# build for arm64 and x86_64
run: |
./gradlew :app:assembleArch64Debug
./gradlew :app:assembleArch64Release
- name: Prepare APKs
run: |
mkdir apk
mv nym-connect/native/android/app/build/outputs/apk/arch64/debug/app-arch64-debug.apk \
apk/nyms5-arch64-debug.apk
mv nym-connect/native/android/app/build/outputs/apk/arch64/release/app-arch64-release-unsigned.apk \
apk/nyms5-arch64-release.apk
- name: Upload APKs
uses: actions/upload-artifact@v3
with:
name: nyms5-apk-arch64
path: |
apk/nyms5-arch64-debug.apk
apk/nyms5-arch64-release.apk
gh-release:
name: Publish APK (GH release)
needs: build
runs-on: custom-runner-linux
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Download binary artifact
uses: actions/download-artifact@v3
with:
name: nyms5-apk-arch64
path: apk
- name: Release
uses: softprops/action-gh-release@v1
with:
files: |
apk/nyms5-arch64-debug.apk
apk/nyms5-arch64-release.apk
-36
View File
@@ -1,36 +0,0 @@
name: CI for Nym API Tests
on:
workflow_dispatch:
push:
paths:
- "nym-api/**"
defaults:
run:
working-directory: nym-api/tests
jobs:
test:
name: nym-api tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install npm
run: npm install
- name: Node v18
uses: actions/setup-node@v3
with:
node-version: 18.1.0
- name: Install yarn
run: yarn install
- name: Run yarn
run: yarn
- name: Run tests
run: yarn test:qa
working-directory: nym-api/tests
+5 -1
View File
@@ -1,3 +1,7 @@
KEYBASE_NYM_CHANNEL=
KEYBASE_NYMBOT_USERNAME=
KEYBASE_NYMBOT_PAPERKEY=
MATRIX_SERVER= MATRIX_SERVER=
MATRIX_ROOM= MATRIX_ROOM=
MATRIX_ROOM_OF_SHAME= MATRIX_ROOM_OF_SHAME=
@@ -35,4 +39,4 @@ NYM_CI_WWW_BASE=example.com
# Nightly builds # Nightly builds
WORKFLOW_CONCLUSION=success WORKFLOW_CONCLUSION=success
SHOW_DEBUG=true SHOW_DEBUG=true
+9 -10
View File
@@ -4,7 +4,7 @@ This is a collection of scripts and files to support GitHub Actions.
## Sending Notifications ## Sending Notifications
These scripts send CI notifications to Matrix by creating messages from templates and env vars passed from GitHub Actions. These scripts send CI notifications to Keybase by creating messages from templates and env vars passed from GitHub Actions.
### Adding notifications to a GitHub Action ### Adding notifications to a GitHub Action
@@ -19,11 +19,10 @@ jobs:
env: env:
NYM_NOTIFICATION_KIND: "my-component" NYM_NOTIFICATION_KIND: "my-component"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}" KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}" KEYBASE_NYM_CHANNEL: "ci-network-explorer"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
uses: docker://keybaseio/client:stable-node uses: docker://keybaseio/client:stable-node
with: with:
@@ -35,8 +34,8 @@ Notifications are run by adding the snippet above to a GitHub Action, and:
1. Installing node packages needed at run time 1. Installing node packages needed at run time
2. Set the env vars as required: 2. Set the env vars as required:
- `NYM_NOTIFICATION_KIND` matches the directory in `.github/workflows/support-files/${NYM_NOTIFICATION_KIND}` to provide the templates and extra scripting in `index.js` - `NYM_NOTIFICATION_KIND` matches the directory in `.github/workflows/support-files/${NYM_NOTIFICATION_KIND}` to provide the templates and extra scripting in `index.js`
- Matrix credentials, room and other env vars for the status of the build and repo - Keybase credentials, channel and other env vars for the status of the build and repo
3. Replacing the default entry point shell script on the `keybaseio/client:stable-node` docker image to run `.github/workflows/support-files/notifications/entry_point.sh` 3. Replacing the default entry point shell script on the `keybaseio/client:stable-node` docker image to run `.github/workflows/support-files/notifications/entry_point.sh`
### Running locally ### Running locally
@@ -44,7 +43,7 @@ You will need:
- Node 16 LTS - Node 16 LTS
- npm - npm
Copy `.github/workflows/support-files/.env.example` to `.github/workflows/support-files/.env` and valid Matrix credentials. Copy `.github/workflows/support-files/.env.example` to `.github/workflows/support-files/.env` and valid Keybase credentials.
Then run `npm install` to get dependencies. Then run `npm install` to get dependencies.
@@ -56,4 +55,4 @@ npm install
cp .env.example .env cp .env.example .env
vi .env vi .env
npm run dev npm run dev
``` ```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_BASE }}/developers/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,11 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
> 🔴 **FAILURE** :cry:
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_BASE }}/docs/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,29 +0,0 @@
const Handlebars = require('handlebars');
const fs = require('fs');
const path = require('path');
async function addToContextAndValidate(context) {
if (!context.env.NYM_CI_WWW_LOCATION) {
throw new Error('Please ensure the env var NYM_CI_WWW_LOCATION is set');
}
if (!context.env.NYM_CI_WWW_BASE) {
throw new Error('Please ensure the env var NYM_CI_WWW_BASE is set');
}
}
async function getMessageBody(context) {
const source = fs
.readFileSync(
context.env.IS_SUCCESS === 'true'
? path.resolve(__dirname, 'templates', 'success')
: path.resolve(__dirname, 'templates', 'failure'),
)
.toString();
const template = Handlebars.compile(source);
return template(context);
}
module.exports = {
addToContextAndValidate,
getMessageBody,
};
@@ -1,17 +0,0 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,17 +0,0 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
```
{{ env.GIT_COMMIT_MESSAGE }}
```
@@ -1,62 +0,0 @@
#!/usr/bin/env bash
#
# Basic usage:
# ./git-merge-check origin/develop origin/release/v1.1.9
#
set -x
set -o nounset
set -o pipefail
# Start from branch ...
branch1=$1
# ... and try to merge in
branch2=$2
echo "Checking if $branch2 merges without conflicts into $branch1..."
git checkout -q $branch1 || exit 1
# There are two options here as far as I see on what we should check for. Either
#
# (A) check for CONFLICT in any file except whitelist (such as .lock files)
# (B) check for "Automatic merge failed"
#
# Both of these options have drawbacks.
#
# The first (A) has the problem in that maybe we don't want to fail if the
# changes can be automatically merged (duh), but at least we are not pestered
# about constant lock file changes.
# The second (B) has the problem that it's difficult to filter out automatic
# merge fails for files we don't care about (.lock files).
#
# The ideal solution would be to check for automatic merge fails for files
# except those on a whitelist (e.g. lock files).
# For now I choose to use (B) here, because I hope it might be less noisy
# Alternative A
#output=$(git merge --no-commit --no-ff $branch2 | grep -v .lock)
#merge_failed=$(echo $output | grep -v "CONFLICT")
#return_code=$?
# Alternative B
output=$(git merge --no-commit --no-ff $branch2)
merge_failed=$(echo $output | grep -v "Automatic merge failed")
return_code=$?
# Restore
git merge --abort
git checkout -q -
if [ $return_code -eq 0 ]; then
echo "Merge check success"
else
echo "Merge check failed"
fi
exit $return_code
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry: > 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message: Commit message:
``` ```
@@ -1,16 +1,10 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/ > :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View output:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION_STORYBOOK }}.{{ env.NYM_CI_WWW_BASE }} > `storybook`: https://{{ env.NYM_CI_WWW_LOCATION_STORYBOOK }}.{{ env.NYM_CI_WWW_BASE }}
>
> ✅ **SUCCESS** > ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}: Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
``` ```
@@ -151,7 +151,7 @@ async function getMessageBody(context) {
return `${icon} ${job.conclusion}: ${job.name} - ${job.html_url}`; return `${icon} ${job.conclusion}: ${job.name} - ${job.html_url}`;
}) })
// and join with newlines for display in the template // and join with newlines for display in the template
.join('\n\n'); .join('\n');
return template({ ...context, jobResults }); return template({ ...context, jobResults });
} }
@@ -1,15 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry: > 🔴 **FAILURE** :cry:
>
> `when` {{ timestamp }} > `when` {{ timestamp }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
{{ jobResults }} {{ jobResults }}
@@ -1,15 +1,9 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS** > ✅ **SUCCESS**
>
> `when` {{ timestamp }} > `when` {{ timestamp }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
{{ jobResults }} {{ jobResults }}
@@ -1,9 +1,10 @@
require('dotenv').config(); require('dotenv').config();
const Bot = require('keybase-bot');
const { sendMatrixMessage } = require('./send_message_to_matrix'); const { sendMatrixMessage } = require('./send_message_to_matrix');
let context = { let context = {
kinds: ['nym-wallet', 'ts-packages', 'network-explorer', 'nightly', 'nym-connect','security','ci-docs','cd-docs','ci-dev','cd-dev'], kinds: ['nym-wallet', 'ts-packages', 'network-explorer', 'nightly', 'nym-connect','security'],
}; };
/** /**
@@ -23,6 +24,21 @@ function validateContext() {
'Please set env var NYM_PROJECT_NAME with the project name for displaying in notification messages', 'Please set env var NYM_PROJECT_NAME with the project name for displaying in notification messages',
); );
} }
if (!context.env.KEYBASE_NYM_CHANNEL) {
throw new Error(
'Please set env var KEYBASE_NYM_CHANNEL with the channel name for the notification message',
);
}
if (!context.env.KEYBASE_NYMBOT_USERNAME) {
throw new Error(
'Username is not defined. Please set env var KEYBASE_NYMBOT_USERNAME',
);
}
if (!context.env.KEYBASE_NYMBOT_PAPERKEY) {
throw new Error(
'Paperkey is not defined. Please set env var KEYBASE_NYMBOT_PAPERKEY',
);
}
if (context.env.MATRIX_ROOM) { if (context.env.MATRIX_ROOM) {
if (!context.env.MATRIX_SERVER) { if (!context.env.MATRIX_SERVER) {
throw new Error( throw new Error(
@@ -68,6 +84,12 @@ function createTemplateContext() {
context.kind = context.env.NYM_NOTIFICATION_KIND; context.kind = context.env.NYM_NOTIFICATION_KIND;
context.keybase = {
channel: context.env.KEYBASE_NYM_CHANNEL,
username: context.env.KEYBASE_NYMBOT_USERNAME,
paperkey: context.env.KEYBASE_NYMBOT_PAPERKEY,
};
if (!context.env.GIT_BRANCH_NAME) { if (!context.env.GIT_BRANCH_NAME) {
context.env.GIT_BRANCH_NAME = context.env.GITHUB_REF.split('/') context.env.GIT_BRANCH_NAME = context.env.GITHUB_REF.split('/')
.slice(2) .slice(2)
@@ -77,6 +99,40 @@ function createTemplateContext() {
context.status = process.env.IS_SUCCESS === 'true' ? 'success' : 'failure'; context.status = process.env.IS_SUCCESS === 'true' ? 'success' : 'failure';
} }
async function sendKeybaseMessage(messageBody) {
const bot = new Bot();
try {
console.log(
`Initialising keybase with user "${
context.keybase.username
}" and key: "${'*'.repeat(context.keybase.paperkey.length)}"...`,
);
await bot.init(context.keybase.username, context.keybase.paperkey, {
verbose: false,
});
const channel = {
name: context.env.KEYBASE_NYMBOT_TEAM || 'nymtech_bot',
membersType: 'team',
topicName: context.keybase.channel,
topic_type: 'CHAT',
};
const message = {
body: messageBody,
};
console.log(`Sending to ${channel.name}#${channel.topicName}...`);
await bot.chat.send(channel, message);
console.log('Message sent!');
} catch (error) {
console.error(error);
process.exitCode = -1;
} finally {
await bot.deinit();
}
}
/** /**
* Uses the `kind` set in the context to process the context and generate a notification message * Uses the `kind` set in the context to process the context and generate a notification message
* @returns {Promise<string>} A string notification message body * @returns {Promise<string>} A string notification message body
@@ -113,6 +169,7 @@ async function main() {
console.log(messageBody); console.log(messageBody);
console.log('-----------------------------------------'); console.log('-----------------------------------------');
} }
await sendKeybaseMessage(messageBody);
if(context.env.MATRIX_ROOM) { if(context.env.MATRIX_ROOM) {
await sendMatrixMessage(context, messageBody, context.env.MATRIX_ROOM) await sendMatrixMessage(context, messageBody, context.env.MATRIX_ROOM)
} }
@@ -5,7 +5,12 @@ const localStorage = new LocalStorage('./scratch');
const { const {
LocalStorageCryptoStore, LocalStorageCryptoStore,
} = require('matrix-js-sdk/lib/crypto/store/localStorage-crypto-store'); } = require('matrix-js-sdk/lib/crypto/store/localStorage-crypto-store');
var showdown = require('showdown');
const vfile = require('to-vfile');
const unified = require('unified');
const remarkParse = require('remark-parse');
const remarkHtml = require('remark-html');
const emoji = require('remark-emoji');
// hide all matrix client output // hide all matrix client output
console.error = (error) => console.log('❌ error: ', error); console.error = (error) => console.log('❌ error: ', error);
@@ -54,9 +59,17 @@ function createClient(context, room, message) {
return client; return client;
} }
async function markdownToHtml(messageAsMarkdown) {
const file = await unified()
.use(emoji)
.use(remarkParse)
.use(remarkHtml)
.process(await vfile({ path: 'test.md', contents: messageAsMarkdown}));
return String(file);
}
async function sendMatrixMessage(contextArg, messageAsMarkdown, roomId) { async function sendMatrixMessage(contextArg, messageAsMarkdown, roomId) {
const converter = new showdown.Converter(); const messageAsHtml = await markdownToHtml(messageAsMarkdown);
const messageAsHtml = converter.makeHtml(messageAsMarkdown);
const client = createClient(contextArg, roomId, messageAsHtml); const client = createClient(contextArg, roomId, messageAsHtml);
await client.initCrypto(); await client.initCrypto();
await client.startClient({ initialSyncLimit: 1 }); await client.startClient({ initialSyncLimit: 1 });
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry: > 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message: Commit message:
``` ```
@@ -1,14 +1,9 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View storybook:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/ > :rocket: {{ env.NYM_PROJECT_NAME }} ➡️➡️➡️➡️➡️ **View storybook:** https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}/
>
> ✅ **SUCCESS** > ✅ **SUCCESS**
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}: Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
``` ```
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry: > 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message: Commit message:
``` ```
@@ -1,18 +1,13 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS** > ✅ **SUCCESS**
>
> ➡️➡️➡️➡️➡️ **View output:** > ➡️➡️➡️➡️➡️ **View output:**
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }} > `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}: Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
``` ```
+4 -4
View File
@@ -1,6 +1,6 @@
{ {
"name": "send-matrix-message", "name": "send-keybase-message",
"description": "Sends a notification message with the matrix sdk", "description": "Sends a notification message with the keybase package that fails when piped into the keybase CLI",
"version": "1.0.0", "version": "1.0.0",
"private": true, "private": true,
"scripts": { "scripts": {
@@ -10,6 +10,7 @@
"dependencies": { "dependencies": {
"dotenv": "^16.0.0", "dotenv": "^16.0.0",
"handlebars": "^4.7.7", "handlebars": "^4.7.7",
"keybase-bot": "^3.6.1",
"matrix-js-sdk": "^9.3.0", "matrix-js-sdk": "^9.3.0",
"node-localstorage": "^2.1.6", "node-localstorage": "^2.1.6",
"octokit": "^1.7.1", "octokit": "^1.7.1",
@@ -17,11 +18,10 @@
"remark-emoji": "^2.2.0", "remark-emoji": "^2.2.0",
"remark-html": "^13.0.2", "remark-html": "^13.0.2",
"remark-parse": "^9.0.0", "remark-parse": "^9.0.0",
"showdown": "^2.1.0",
"to-vfile": "^6.1.0", "to-vfile": "^6.1.0",
"unified": "^9.2.2" "unified": "^9.2.2"
}, },
"devDependencies": { "devDependencies": {
"prettier": "^2.8.7" "prettier": "2.3.2"
} }
} }
@@ -1,14 +1,9 @@
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> 🔴 **FAILURE** :cry: > 🔴 **FAILURE** :cry:
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message: Commit message:
``` ```
@@ -1,20 +1,14 @@
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
> :rocket: {{ env.NYM_PROJECT_NAME }} > :rocket: {{ env.NYM_PROJECT_NAME }}
>
> ✅ **SUCCESS** > ✅ **SUCCESS**
>
> ➡️➡️➡️➡️➡️ **View output:** > ➡️➡️➡️➡️➡️ **View output:**
>
> `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }} > `storybook`: https://{{ env.NYM_CI_WWW_LOCATION }}.{{ env.NYM_CI_WWW_BASE }}
>
> `example`: https://{{ env.NYM_CI_WWW_LOCATION }}-example.{{ env.NYM_CI_WWW_BASE }} > `example`: https://{{ env.NYM_CI_WWW_LOCATION }}-example.{{ env.NYM_CI_WWW_BASE }}
>
> `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }} > `branch` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/tree/{{ env.GIT_BRANCH_NAME }}
>
> `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }} > `commit` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/commit/{{ env.GITHUB_SHA }}
>
> `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }} > `build ` {{ env.GITHUB_SERVER_URL }}/{{ env.GITHUB_REPOSITORY }}/actions/runs/{{ env.GITHUB_RUN_ID }}
>
Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}: Commit message by `{{ env.GITHUB_ACTOR }}` at {{ timestamp }}:
``` ```
-46
View File
@@ -1,46 +0,0 @@
name: tag-and-release
on:
workflow_dispatch:
inputs:
nym_binaries_version:
description: 'Version of the nym-binaries tag'
required: false
type: string
nym_wallet_version:
description: 'Version of the nym-wallet tag'
required: false
type: string
nym_connect_version:
description: 'Version of the nym-connect tag'
required: false
type: string
jobs:
tag-components:
uses: nymtech/reusable-workflows/.github/workflows/tag-components.yml@master
with:
nym_binaries_version: ${{ input.nym_binaries_version }}
nym_wallet_version: ${{ input.nym_wallet_version }}
nym_connect_version: ${{ input.nym_connect_version }}
create-nym-binaries-release:
if: ${{ input.nym_binaries_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-binaries-release.yml@master
with:
version: ${{ input.nym_binaries_version }}
needs: ["tag-components"]
create-nym-wallet-release:
if: ${{ input.nym_wallet_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-wallet-release.yml@master
with:
version: ${{ input.nym_wallet_version }}
needs: ["tag-components"]
create-nym-connect-release:
if: ${{ input.nym_connect_version }}
uses: nymtech/reusable-workflows/.github/workflows/create-connect-release.yml@master
with:
version: ${{ input.nym_connect_version }}
needs: ["tag-components"]
+10 -23
View File
@@ -5,22 +5,9 @@ on:
paths: paths:
- 'ts-packages/**' - 'ts-packages/**'
- 'sdk/typescript/**' - 'sdk/typescript/**'
- 'nym-connect/desktop/src/**' - nym-connect
- 'nym-connect/desktop/package.json' - nym-connect-android
- 'nym-connect/mobile/src/**' - nym-wallet
- 'nym-connect/mobile/package.json'
- 'nym-wallet/src/**'
- 'nym-wallet/package.json'
pull_request:
paths:
- 'ts-packages/**'
- 'sdk/typescript/**'
- 'nym-connect/desktop/src/**'
- 'nym-connect/desktop/package.json'
- 'nym-connect/mobile/src/**'
- 'nym-connect/mobile/package.json'
- 'nym-wallet/src/**'
- 'nym-wallet/package.json'
jobs: jobs:
build: build:
@@ -36,16 +23,12 @@ jobs:
node-version: 16 node-version: 16
- name: Setup yarn - name: Setup yarn
run: npm install -g yarn run: npm install -g yarn
- name: Install
run: yarn
- name: Build packages
run: yarn build
- name: Lint - name: Lint
run: yarn lint && yarn tsc run: yarn && yarn lint && yarn tsc
- name: Matrix - Node Install - name: Keybase - Node Install
run: npm install run: npm install
working-directory: .github/workflows/support-files working-directory: .github/workflows/support-files
- name: Matrix - Send Notification - name: Keybase - Send Notification
env: env:
NYM_NOTIFICATION_KIND: ts-packages NYM_NOTIFICATION_KIND: ts-packages
NYM_PROJECT_NAME: "ts-packages" NYM_PROJECT_NAME: "ts-packages"
@@ -53,6 +36,10 @@ jobs:
NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}" NYM_CI_WWW_LOCATION: "ts-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}" GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}" GIT_BRANCH: "${GITHUB_REF##*/}"
KEYBASE_NYMBOT_USERNAME: "${{ secrets.KEYBASE_NYMBOT_USERNAME }}"
KEYBASE_NYMBOT_PAPERKEY: "${{ secrets.KEYBASE_NYMBOT_PAPERKEY }}"
KEYBASE_NYMBOT_TEAM: "${{ secrets.KEYBASE_NYMBOT_TEAM }}"
KEYBASE_NYM_CHANNEL: "ci-ts-packages"
IS_SUCCESS: "${{ job.status == 'success' }}" IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}" MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}" MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
@@ -1,95 +0,0 @@
name: update-versions-and-changelog
on:
workflow_dispatch:
inputs:
release_version:
description: "Release version, usually the milestone title"
required: true
type: string
milestone_id:
description: "Milestone ID, check the URL when you're on the specific milestone page"
required: true
type: string
env:
CI_BOT_AUTHOR: "Nym bot"
CI_BOT_EMAIL: "nym-bot@users.noreply.github.com"
jobs:
update-versions:
runs-on: ubuntu-22.04
steps:
- name: checkout-source
uses: actions/checkout@v3
with:
ref: "release/${{ inputs.release_version }}"
path: "nym"
- name: checkout-ci-tools-repo
uses: actions/checkout@v3
with:
repository: "nymtech/ci-tools"
ref: "master"
path: "ci-tools"
token: "${{ secrets.ACCESS_TOKEN_PRIVATE_REPOS }}"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: install-version-bumper
run: "cargo install --path ."
working-directory: "ci-tools/version-bumper"
- name: run-version-bumper
run: "version-bumper bump binaries --nym-repo-directory nym"
- name: push-changes-to-branch
run: |
git config --global user.name "${{ env.CI_BOT_AUTHOR }}"
git config --global user.email "${{ env.CI_BOT_EMAIL }}"
git checkout -b release/${{ inputs.release_version }}-preparation
git commit -am "chore: version bump in preparation for release"
git push -u origin release/${{ inputs.release_version }}-preparation
working-directory: "nym"
update-changelog:
runs-on: ubuntu-22.04
needs: [update-versions]
steps:
- name: checkout-source
uses: actions/checkout@v3
with:
ref: "release/${{ inputs.release_version }}"
path: "nym"
- name: checkout-ci-tools-repo
uses: actions/checkout@v3
with:
repository: "nymtech/ci-tools"
ref: "master"
path: "ci-tools"
token: "${{ secrets.ACCESS_TOKEN_PRIVATE_REPOS }}"
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: install-changelog-updater
run: "cargo install --path ."
working-directory: "ci-tools/changelog-updater"
- name: run-changelog-updater
run: "changelog-updater mix ${{ inputs.milestone_id }} release/${{ inputs.release_version }}"
- name: push-changes-to-branch
run: |
git config --global user.name "${{ env.CI_BOT_AUTHOR }}"
git config --global user.email "${{ env.CI_BOT_EMAIL }}"
git checkout release/${{ inputs.release_version }}-preparation
git commit -am "chore: update changelog preparation for release"
git push
working-directory: "nym"
-79
View File
@@ -1,79 +0,0 @@
name: Upload nyxd to CI
on:
workflow_dispatch:
jobs:
publish-nyxd:
strategy:
fail-fast: false
matrix:
platform: [ubuntu-20.04]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v3
- name: Prepare build output directory
shell: bash
env:
OUTPUT_DIR: ci-builds/nyxd
run: |
rm -rf ci-builds || true
mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools git
continue-on-error: true
- name: Update env variables to include go
run: |
sudo rm -rf /usr/local/go
curl https://dl.google.com/go/go1.19.2.linux-amd64.tar.gz | sudo tar -C/usr/local -zxvf -
cat <<'EOF' >>$HOME/.profile
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export GO111MODULE=on
export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin
EOF
source $HOME/.profile
- name: Verify Go is installed
run: go version
- name: Clone nyxd repo
run: |
git clone https://github.com/tommyv1987/nyxd
cd nyxd
git checkout release/v0.30.2
- name: Run nyxd
run: |
pwd
cd nyxd && make build
sleep 10
ls /home/runner/work/nym/nym/nyxd/build
- name: Prepare build output
shell: bash
env:
OUTPUT_DIR: ci-builds/nyxd
run: |
cp /home/runner/work/nym/nym/nyxd/build/nyxd $OUTPUT_DIR
WASMVM_SO=$(ldd /home/runner/work/nym/nym/nyxd/build/nyxd | grep "libwasm*" | awk '{ print $3 }')
ls $WASMVM_SO
sleep 3
cp $(echo $WASMVM_SO) $OUTPUT_DIR
- name: Deploy nyxd to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-avzr"
SOURCE: "ci-builds/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/
EXCLUDE: "/dist/, /node_modules/"
+1 -1
View File
@@ -64,4 +64,4 @@ jobs:
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: clippy command: clippy
args: --manifest-path nym-wallet/Cargo.toml --workspace --all-features --all-targets -- -D warnings args: --manifest-path nym-wallet/Cargo.toml --workspace --all-features -- -D warnings
+2 -3
View File
@@ -8,7 +8,6 @@
.idea .idea
target target
.env .env
.env.dev
/.vscode/settings.json /.vscode/settings.json
validator/.vscode validator/.vscode
sample-configs/validator-config.toml sample-configs/validator-config.toml
@@ -40,7 +39,7 @@ validator-api-config.toml
dist dist
storybook-static storybook-static
envs/qwerty.env envs/qwerty.env
Cargo.lock
nym-connect/Cargo.lock
.parcel-cache .parcel-cache
**/.DS_Store **/.DS_Store
cpu-cycles/libcpucycles/build
foxyfox.env
+15 -245
View File
@@ -2,255 +2,19 @@
Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] # [Unreleased]
## [v1.1.21] (2023-06-13)
- mixFetch: Change socks5 `SendRequest` to include OrderedMessage index as a field rather than making it serialized inside the `data` field
([#3534])
- Explorer - add more data columns to the Service Provider section: ([#3474])
- network-requester: support report if they run an open proxy using `ControlRequest` API ([#3461])
- Refactor client configs (London discussion) ([#3444])
- Increase `DEFAULT_MAXIMUM_CONNECTION_BUFFER_SIZE` to 2000 to improve reliability ([#3433])
- socks5: sender waits for lanes to clear even though the connection is closed ([#3366])
- version bump for variables ([#3545])
[#3534]: https://github.com/nymtech/nym/issues/3534
[#3474]: https://github.com/nymtech/nym/issues/3474
[#3461]: https://github.com/nymtech/nym/issues/3461
[#3444]: https://github.com/nymtech/nym/issues/3444
[#3433]: https://github.com/nymtech/nym/issues/3433
[#3366]: https://github.com/nymtech/nym/issues/3366
[#3545]: https://github.com/nymtech/nym/pull/3545
## [v1.1.20] (2023-06-06)
- Explorer - Fix SP supported apps list ([#3458])
- Investigate if we need to lower `SHUTDOWN_TIMEOUT` in socks5 to zero (or almost zero) ([#3438])
- Explorer - show all gateways in the default view regardless of the version number ([#3427])
- service-provider-directory: add signature check when announcing ([#3360])
- Support functionality for nym-name-service (nym-api, nym-cli, etc) ([#3355])
- Edit the nym-network-requester to support the enabled-credentials-mode flag ([#3101])
- [BUG] network requester documentation update ([#3493])
- removing hardcoded version numbers ([#3485])
- [BUG] network requester documentation update ([#3481])
- [BUG] network requester documentation update ([#3469])
- Sign when announcing service providers to the directory contract ([#3459])
- mixnode documentation update ([#3435])
- updated readme with new developer chat links + new docs links ([#3141])
[#3458]: https://github.com/nymtech/nym/issues/3458
[#3438]: https://github.com/nymtech/nym/issues/3438
[#3427]: https://github.com/nymtech/nym/issues/3427
[#3360]: https://github.com/nymtech/nym/issues/3360
[#3355]: https://github.com/nymtech/nym/issues/3355
[#3101]: https://github.com/nymtech/nym/issues/3101
[#3493]: https://github.com/nymtech/nym/pull/3493
[#3485]: https://github.com/nymtech/nym/pull/3485
[#3481]: https://github.com/nymtech/nym/pull/3481
[#3469]: https://github.com/nymtech/nym/pull/3469
[#3459]: https://github.com/nymtech/nym/pull/3459
[#3435]: https://github.com/nymtech/nym/pull/3435
[#3141]: https://github.com/nymtech/nym/pull/3141
## [v1.1.19] (2023-05-16)
- nym-name-service endpoint in nym-api ([#3403])
- Implement key storage for WASM client using IndexedDB (for browser) ([#3329])
- Initial version of nym-name-service contract providing name aliases for nym-addresses ([#3274])
- Update Cargo.lock ([#3410])
[#3403]: https://github.com/nymtech/nym/issues/3403
[#3329]: https://github.com/nymtech/nym/issues/3329
[#3274]: https://github.com/nymtech/nym/issues/3274
[#3410]: https://github.com/nymtech/nym/pull/3410
## [v1.1.18] (2023-05-09)
- Implement heartbeat messages between socks5 proxy and network requester ([#3215])
[#3215]: https://github.com/nymtech/nym/issues/3215
## [v1.1.17] (2023-05-02)
- Add service-provider-directory-contract support to nym-cli ([#3334])
- Start using the node-testing-utils (implemented in #3270) in nym-api Network monitor to simplify the logic there ([#3312])
- Add service-provider-directory support to validator-client ([#3296])
- Allow topology injection in our WASM client ('test my node' feature) ([#3270])
- Expose service-provider-directory contract data in nym-api endpoints ([#3242])
- Cache service provider contract in nym-api ([#3241])
- Feature/1 1 17 docs ([#3370])
- adding a test for SP endpoint ([#3367])
- Feature/store cipher ([#3350])
[#3334]: https://github.com/nymtech/nym/issues/3334
[#3312]: https://github.com/nymtech/nym/issues/3312
[#3296]: https://github.com/nymtech/nym/issues/3296
[#3270]: https://github.com/nymtech/nym/issues/3270
[#3242]: https://github.com/nymtech/nym/issues/3242
[#3241]: https://github.com/nymtech/nym/issues/3241
[#3370]: https://github.com/nymtech/nym/pull/3370
[#3367]: https://github.com/nymtech/nym/pull/3367
[#3350]: https://github.com/nymtech/nym/pull/3350
## [v1.1.16] (2023-04-25)
- Explorer - Fix sorting function on Stake Saturation. It is currently working per page and not globally ([#3320])
- Poisson process gets stuck at too slow rate. Rework to more aggressively up-regulate ([#3309])
- decrease the logging level of warnings associated with clients dropping packets due to gateway being overloaded (I'd say reduce it to debug/trace) - there are few sources of those, e.g. in real and cover traffic streams ([#3299])
- Make the buffer size in `AvailableReader` depend on packet sizes the client is using + introduce read timeouts ([#3213])
- Rust SDK - Support coconut, credential storage etc ([#2755])
- version bump for next release ([#3349])
- added coconut credential generation example ([#3339])
- update mix-node setup docs with node description ([#3325])
- exposed missing gateway commands in nym-cli ([#3324])
- make sure to clear inner 'ack_map' in 'GatewaysReader' ([#3300])
[#3320]: https://github.com/nymtech/nym/issues/3320
[#3309]: https://github.com/nymtech/nym/issues/3309
[#3299]: https://github.com/nymtech/nym/issues/3299
[#3213]: https://github.com/nymtech/nym/issues/3213
[#2755]: https://github.com/nymtech/nym/issues/2755
[#3349]: https://github.com/nymtech/nym/pull/3349
[#3339]: https://github.com/nymtech/nym/pull/3339
[#3325]: https://github.com/nymtech/nym/pull/3325
[#3324]: https://github.com/nymtech/nym/pull/3324
[#3300]: https://github.com/nymtech/nym/pull/3300
## [v1.1.15] (2023-04-18)
- Fix verloc being stuck waiting for shutdown signal ([#3250])
- Introduce `--output json` flag to `sign` command to allow to more easily capture the output ([#3249])
- Explorer - Dont fetch Service Provider list on Testnet ([#3245])
- When determining active set, rather than weighting the nodes by just the `stake`, use `stake * performance` ([#3234])
- Introduce dual packet sizes to our clients (as in use two packet sizes at the same time depending on message size) ([#3189])
- Experiment with offline signing in our validator client ([#3174])
- Modify network requester binary to reload `allowed.list` periodically to pull in any changes made upstream without having to restart the service ([#3149])
- Standardise all `--output json` on binary inits, we pass the output json at different points for different binaries. ([#3080])
- Service provider directory contract: initial version ([#2759])
- Fix issue where network-requester run failed on fresh init due to missing allow file ([#3316])
[#3250]: https://github.com/nymtech/nym/issues/3250
[#3249]: https://github.com/nymtech/nym/issues/3249
[#3245]: https://github.com/nymtech/nym/issues/3245
[#3234]: https://github.com/nymtech/nym/issues/3234
[#3189]: https://github.com/nymtech/nym/issues/3189
[#3174]: https://github.com/nymtech/nym/issues/3174
[#3149]: https://github.com/nymtech/nym/issues/3149
[#3080]: https://github.com/nymtech/nym/issues/3080
[#2759]: https://github.com/nymtech/nym/issues/2759
[#3316]: https://github.com/nymtech/nym/pull/3316
## [v1.1.14] (2023-04-04)
- Investigate cause of qwerty validator being in invalid rewarding state ([#3224])
- Fix NR config due to changes in #3199 ([#3223])
- [Issue] Mixnodes and gateway do not close connections properly ([#3187])
- disable sign-ext when using wasm-opt + update wasm-opt ([#3203])
- chore: tidy up client `Debug` config section ([#3199])
[#3224]: https://github.com/nymtech/nym/issues/3224
[#3223]: https://github.com/nymtech/nym/issues/3223
[#3187]: https://github.com/nymtech/nym/issues/3187
[#3203]: https://github.com/nymtech/nym/pull/3203
[#3199]: https://github.com/nymtech/nym/pull/3199
>>>>>>> master
## [v1.1.13] (2023-03-15)
- NE - instead of throwing a "Mixnode/Gateway not found" error for blacklisted nodes due to bad performance, show their history but tag them as "Having poor performance" ([#2979])
- NE - Upgrade Sandbox and make below changes: ([#2332])
- Explorer - Updates ([#3168])
- Website v2 - deploy infrastructure for strapi and CI ([#2213])
- add blockstream green to sp list ([#3180])
- mock-nym-api: fix .storybook lint error ([#3178])
- Validating new interval config parameters to prevent division by zero ([#3153])
[#2979]: https://github.com/nymtech/nym/issues/2979
[#2332]: https://github.com/nymtech/nym/issues/2332
[#3168]: https://github.com/nymtech/nym/issues/3168
[#2213]: https://github.com/nymtech/nym/issues/2213
[#3180]: https://github.com/nymtech/nym/pull/3180
[#3178]: https://github.com/nymtech/nym/pull/3178
[#3153]: https://github.com/nymtech/nym/pull/3153
## [v1.1.12] (2023-03-07)
- Fix generated docs for mixnet and vesting contract on docs.rs ([#3093])
- Introduce a way of injecting topology into the client ([#3044])
- Update mixnet TypeScript client methods #1 ([#2783])
- Update tooltips for routing and average score ([#3133])
- update selected service provider description style ([#3128])
[#3093]: https://github.com/nymtech/nym/issues/3093
[#3044]: https://github.com/nymtech/nym/issues/3044
[#2783]: https://github.com/nymtech/nym/issues/2783
[#3133]: https://github.com/nymtech/nym/pull/3133
[#3128]: https://github.com/nymtech/nym/pull/3128
## [v1.1.11] (2023-02-28)
- Fix empty dealer set loop ([#3105])
- The nym-api db.sqlite is broken when trying to run against it it in `enabled-credentials-mode true` there is an ordering issue with migrations when using the credential binary to purchase bandwidth ([#3100])
- Feature/latency based gateway selection ([#3081])
- Fix the credential binary to handle transactions to sleep when in non-inProgress epochs ([#3057])
- Publish mixnet contract to crates.io ([#1919])
- Publish vesting contract to crates.io ([#1920])
- Feature/update checker to use master ([#3097])
- Feature/improve binary checks ([#3094])
[#3105]: https://github.com/nymtech/nym/issues/3105
[#3100]: https://github.com/nymtech/nym/issues/3100
[#3081]: https://github.com/nymtech/nym/pull/3081
[#3057]: https://github.com/nymtech/nym/issues/3057
[#1919]: https://github.com/nymtech/nym/issues/1919
[#1920]: https://github.com/nymtech/nym/issues/1920
[#3097]: https://github.com/nymtech/nym/pull/3097
[#3094]: https://github.com/nymtech/nym/pull/3094
## [v1.1.10] (2023-02-21)
- Verloc listener causing mixnode unexpected shutdown ([#3038])
- rust-sdk - update API following implementation experience with the network-requester ([#3001])
- Prevent coconut deposits in incompatible states ([#2991])
- Support unavailable signer within threshold ([#2987])
- Implement DKG re-sharing ([#2935])
- contracts: add nym prefix to mixnet and vesting contract packages ([#2855])
- Introduce common interface for all service providers to allow obtaining information such as whether they're online, what binary version they're running, etc. ([#2758])
- Add client functionality to nym-network-requester ([#1900])
- nym-api: uptime rework ([#3053])
- ci: update typescript-lint.yml ([#3035])
- contracts: add nym prefix to mixnet and vesting contract packages ([#2855])
[#3038]: https://github.com/nymtech/nym/issues/3038
[#3001]: https://github.com/nymtech/nym/issues/3001
[#2991]: https://github.com/nymtech/nym/issues/2991
[#2987]: https://github.com/nymtech/nym/issues/2987
[#2935]: https://github.com/nymtech/nym/issues/2935
[#2855]: https://github.com/nymtech/nym/pull/2855
[#2758]: https://github.com/nymtech/nym/issues/2758
[#1900]: https://github.com/nymtech/nym/issues/1900
[#3053]: https://github.com/nymtech/nym/pull/3053
[#3035]: https://github.com/nymtech/nym/pull/3035
[#2855]: https://github.com/nymtech/nym/pull/2855
### Added ### Added
- remove coconut feature and unify builds ([#2890])
- Remove Coconut feature flag ([#2793]) - native-client: is now capable of listening for requests on sockets different than `127.0.0.1` ([#2939]). This can be specified via `--host` flag during `init` or `run`. Alternatively a custom `host` can be set in `config.toml` file under `socket` section.
- dkg resharing mode ([#2936])
[#2890]: https://github.com/nymtech/nym/pull/2890
[#2939]: https://github.com/nymtech/nym/pull/2939
[#2936]: https://github.com/nymtech/nym/pull/2936
# [v1.1.8] (2023-01-31)
- native-client: is now capable of listening for requests on sockets different than `127.0.0.1` ([#2912]). This can be specified via `--host` flag during `init` or `run`. Alternatively a custom `host` can be set in `config.toml` file under `socket` section.
- mixnode, gateway: fix unexpected shutdown on corrupted connection ([#2963])
[#2793]: https://github.com/nymtech/nym/issues/2793
[#2912]: https://github.com/nymtech/nym/issues/2912
[#2964]: https://github.com/nymtech/nym/issues/2964
[#2963]: https://github.com/nymtech/nym/issues/3017
### Added ### Added
@@ -270,7 +34,7 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
[#2873]: https://github.com/nymtech/nym/issues/2873 [#2873]: https://github.com/nymtech/nym/issues/2873
# [v1.1.7] (2023-01-24)
### Added ### Added
@@ -326,6 +90,12 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
### Added ### Added
### Changed
## [v1.1.5] (2023-01-10)
### Added
- socks5: send status message for service ready, and network-requester error response in https://github.com/nymtech/nym/pull/2715 - socks5: send status message for service ready, and network-requester error response in https://github.com/nymtech/nym/pull/2715
### Changed ### Changed
Generated
+1770 -2621
View File
File diff suppressed because it is too large Load Diff
+11 -41
View File
@@ -17,39 +17,38 @@ opt-level = 3
resolver = "2" resolver = "2"
members = [ members = [
"clients/client-core",
"clients/credential", "clients/credential",
"clients/native", "clients/native",
"clients/native/websocket-requests", "clients/native/websocket-requests",
"clients/socks5", "clients/socks5",
"common/async-file-watcher", "common/bandwidth-claim-contract",
"common/bandwidth-controller",
"common/bin-common",
"common/client-core",
"common/client-libs/gateway-client", "common/client-libs/gateway-client",
"common/client-libs/mixnet-client", "common/client-libs/mixnet-client",
"common/client-libs/validator-client", "common/client-libs/validator-client",
"common/client-connections",
"common/coconut-interface", "common/coconut-interface",
"common/commands", "common/commands",
"common/config", "common/config",
"common/build-information",
"common/cosmwasm-smart-contracts/coconut-bandwidth-contract", "common/cosmwasm-smart-contracts/coconut-bandwidth-contract",
"common/cosmwasm-smart-contracts/coconut-dkg", "common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common", "common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/group-contract", "common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract", "common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract", "common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/name-service",
"common/cosmwasm-smart-contracts/service-provider-directory",
"common/cosmwasm-smart-contracts/vesting-contract", "common/cosmwasm-smart-contracts/vesting-contract",
"common/mobile-storage",
"common/credential-storage", "common/credential-storage",
"common/credentials", "common/credentials",
"common/crypto", "common/crypto",
"common/dkg", "common/crypto/dkg",
"common/execute", "common/execute",
"common/inclusion-probability", "common/inclusion-probability",
"common/ledger", "common/ledger",
"common/logging",
"common/mixnode-common", "common/mixnode-common",
"common/network-defaults", "common/network-defaults",
"common/node-tester-utils",
"common/nonexhaustive-delayqueue", "common/nonexhaustive-delayqueue",
"common/nymcoconut", "common/nymcoconut",
"common/nymsphinx", "common/nymsphinx",
@@ -61,24 +60,21 @@ members = [
"common/nymsphinx/forwarding", "common/nymsphinx/forwarding",
"common/nymsphinx/framing", "common/nymsphinx/framing",
"common/nymsphinx/params", "common/nymsphinx/params",
"common/nymsphinx/routing",
"common/nymsphinx/types", "common/nymsphinx/types",
"common/pemstore", "common/pemstore",
"common/socks5-client-core",
"common/socks5/proxy-helpers", "common/socks5/proxy-helpers",
"common/socks5/requests", "common/socks5/requests",
"common/statistics", "common/statistics",
"common/store-cipher",
"common/task", "common/task",
"common/topology", "common/topology",
"common/types", "common/types",
"common/wasm-utils", "common/wasm-utils",
"common/completions",
"explorer-api", "explorer-api",
"gateway", "gateway",
"gateway/gateway-requests", "gateway/gateway-requests",
"integrations/bity", "integrations/bity",
"mixnode", "mixnode",
"sdk/lib/socks5-listener",
"sdk/rust/nym-sdk", "sdk/rust/nym-sdk",
"service-providers/common", "service-providers/common",
"service-providers/network-requester", "service-providers/network-requester",
@@ -87,7 +83,6 @@ members = [
"nym-api/nym-api-requests", "nym-api/nym-api-requests",
"nym-outfox", "nym-outfox",
"tools/nym-cli", "tools/nym-cli",
"tools/nym-nr-query",
"tools/ts-rs-cli" "tools/ts-rs-cli"
] ]
@@ -102,43 +97,18 @@ default-members = [
"explorer-api", "explorer-api",
] ]
exclude = ["explorer", "contracts", "clients/webassembly", "nym-wallet", "nym-connect/mobile/src-tauri", "nym-connect/desktop", "cpu-cycles"] exclude = ["explorer", "contracts", "clients/webassembly", "nym-wallet", "nym-connect", "nym-connect-android"]
[workspace.package] [workspace.package]
authors = ["Nym Technologies SA"] authors = ["Nym Technologies SA"]
repository = "https://github.com/nymtech/nym" repository = "https://github.com/nymtech/nym"
homepage = "https://nymtech.net" homepage = "https://nymtech.net"
documentation = "https://nymtech.net"
edition = "2021" edition = "2021"
license = "Apache-2.0"
[workspace.dependencies] [workspace.dependencies]
anyhow = "1.0.71" async-trait = "0.1.63"
async-trait = "0.1.64"
bip39 = { version = "2.0.0", features = ["zeroize"] }
cfg-if = "1.0.0"
cosmwasm-derive = "=1.0.0"
cosmwasm-schema = "=1.0.0"
cosmwasm-std = "=1.0.0"
cosmwasm-storage = "=1.0.0"
cw-controllers = "=0.13.4"
cw-storage-plus = "=0.13.4"
cw-utils = "=0.13.4"
cw2 = { version = "=0.13.4" }
cw3 = { version = "=0.13.4" }
cw3-fixed-multisig = { version = "=0.13.4" }
cw4 = { version = "=0.13.4" }
dotenvy = "0.15.6"
generic-array = "0.14.7"
getrandom = "0.2.10"
lazy_static = "1.4.0"
log = "0.4" log = "0.4"
once_cell = "1.7.2" thiserror = "1.0.38"
rand = "0.8.5"
serde = "1.0.152" serde = "1.0.152"
serde_json = "1.0.91" serde_json = "1.0.91"
tap = "1.0.1"
thiserror = "1.0.38"
tokio = "1.24.1" tokio = "1.24.1"
url = "2.2"
zeroize = "1.6.0"
+74 -88
View File
@@ -1,122 +1,108 @@
# Default target
all: test
test: clippy-all cargo-test wasm fmt test: clippy-all cargo-test wasm fmt
test-all: test cargo-test-expensive test-all: test cargo-test-expensive
no-clippy: build cargo-test wasm fmt no-clippy: build cargo-test wasm fmt
happy: fmt clippy-happy test happy: fmt clippy-happy test
clippy-all: clippy-main clippy-all-contracts clippy-all-wallet clippy-all-connect clippy-all-wasm-client
clippy-happy: clippy-happy-main clippy-happy-contracts clippy-happy-wallet clippy-happy-connect
cargo-test: test-main test-contracts test-wallet test-connect
cargo-test-expensive: test-main-expensive test-contracts-expensive test-wallet-expensive test-connect-expensive
build: build-contracts build-wallet build-main build-connect build-wasm-client
fmt: fmt-main fmt-contracts fmt-wallet fmt-connect fmt-wasm-client
# Building release binaries is a little manual as we can't just build --release clippy-happy-main:
# on all workspaces. cargo clippy
build-release: build-release-main wasm
# Deprecated clippy-happy-contracts:
# For backwards compatibility cargo clippy --manifest-path contracts/Cargo.toml --target wasm32-unknown-unknown
clippy-all: clippy
# ----------------------------------------------------------------------------- clippy-happy-wallet:
# Define targets for a given workspace cargo clippy --manifest-path nym-wallet/Cargo.toml
# $(1): name
# $(2): path to workspace
# $(3): extra arguments to cargo
# -----------------------------------------------------------------------------
define add_cargo_workspace
clippy-happy-$(1): clippy-happy-connect:
cargo clippy --manifest-path $(2)/Cargo.toml $(3) cargo clippy --manifest-path nym-connect/Cargo.toml
clippy-$(1): clippy-main:
cargo clippy --manifest-path $(2)/Cargo.toml --workspace $(3) -- -D warnings cargo clippy --workspace -- -D warnings
clippy-examples-$(1): clippy-wasm:
cargo clippy --manifest-path $(2)/Cargo.toml --workspace --examples -- -D warnings cargo clippy --manifest-path clients/webassembly/Cargo.toml --target wasm32-unknown-unknown --workspace -- -D warnings
check-$(1):
cargo check --manifest-path $(2)/Cargo.toml --workspace $(3)
test-$(1): clippy-all-contracts:
cargo test --manifest-path $(2)/Cargo.toml --workspace cargo clippy --workspace --manifest-path contracts/Cargo.toml --all-features --target wasm32-unknown-unknown -- -D warnings
test-expensive-$(1): clippy-all-wallet:
cargo test --manifest-path $(2)/Cargo.toml --workspace -- --ignored cargo clippy --workspace --manifest-path nym-wallet/Cargo.toml --all-features -- -D warnings
build-$(1): clippy-all-connect:
cargo build --manifest-path $(2)/Cargo.toml --workspace $(3) cargo clippy --workspace --manifest-path nym-connect/Cargo.toml --all-features -- -D warnings
build-examples-$(1): clippy-all-wasm-client:
cargo build --manifest-path $(2)/Cargo.toml --workspace --examples cargo clippy --workspace --manifest-path clients/webassembly/Cargo.toml --all-features --target wasm32-unknown-unknown -- -D warnings
build-release-$(1): test-main:
cargo build --manifest-path $(2)/Cargo.toml --workspace --release $(3) cargo test --workspace
fmt-$(1): test-main-expensive:
cargo fmt --manifest-path $(2)/Cargo.toml --all cargo test --workspace -- --ignored
clippy-happy: clippy-happy-$(1) test-contracts:
clippy: clippy-$(1) clippy-examples-$(1) cargo test --manifest-path contracts/Cargo.toml --all-features
check: check-$(1)
cargo-test: test-$(1)
cargo-test-expensive: test-expensive-$(1)
build: build-$(1) build-examples-$(1)
build-release-all: build-release-$(1)
fmt: fmt-$(1)
endef test-contracts-expensive:
cargo test --manifest-path contracts/Cargo.toml --all-features -- --ignored
# ----------------------------------------------------------------------------- test-wallet:
# Rust workspaces cargo test --manifest-path nym-wallet/Cargo.toml --all-features
# -----------------------------------------------------------------------------
# Generate targets for the various cargo workspaces test-wallet-expensive:
cargo test --manifest-path nym-wallet/Cargo.toml --all-features -- --ignored
$(eval $(call add_cargo_workspace,main,.)) test-connect:
$(eval $(call add_cargo_workspace,contracts,contracts,--target wasm32-unknown-unknown)) cargo test --manifest-path nym-connect/Cargo.toml --all-features
$(eval $(call add_cargo_workspace,wasm-client,clients/webassembly,--target wasm32-unknown-unknown))
$(eval $(call add_cargo_workspace,wallet,nym-wallet,))
$(eval $(call add_cargo_workspace,connect,nym-connect/desktop))
ifdef NYM_MOBILE
$(eval $(call add_cargo_workspace,connect-mobile,nym-connect/mobile/src-tauri))
endif
# ----------------------------------------------------------------------------- test-connect-expensive:
# Convenience targets for crates that are already part of the main workspace cargo test --manifest-path nym-connect/Cargo.toml --all-features -- --ignored
# -----------------------------------------------------------------------------
build-main:
cargo build --workspace
build-contracts:
cargo build --manifest-path contracts/Cargo.toml --workspace
build-wallet:
cargo build --manifest-path nym-wallet/Cargo.toml --workspace
build-connect:
cargo build --manifest-path nym-connect/Cargo.toml --workspace
build-explorer-api: build-explorer-api:
cargo build -p explorer-api cargo build --manifest-path explorer-api/Cargo.toml --workspace
build-wasm-client:
cargo build --manifest-path clients/webassembly/Cargo.toml --workspace --target wasm32-unknown-unknown
build-nym-cli: build-nym-cli:
cargo build -p nym-cli --release cargo build --release --manifest-path tools/nym-cli/Cargo.toml
# ----------------------------------------------------------------------------- fmt-main:
# Build contracts ready for deploy cargo fmt --all
# -----------------------------------------------------------------------------
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release fmt-contracts:
VESTING_CONTRACT=$(CONTRACTS_OUT_DIR)/vesting_contract.wasm cargo fmt --manifest-path contracts/Cargo.toml --all
MIXNET_CONTRACT=$(CONTRACTS_OUT_DIR)/mixnet_contract.wasm
SERVICE_PROVIDER_DIRECTORY_CONTRACT=$(CONTRACTS_OUT_DIR)/nym_service_provider_directory.wasm
NAME_SERVICE_CONTRACT=$(CONTRACTS_OUT_DIR)/nym_name_service.wasm
wasm: wasm-build wasm-opt fmt-wallet:
cargo fmt --manifest-path nym-wallet/Cargo.toml --all
wasm-build: fmt-connect:
cargo fmt --manifest-path nym-connect/Cargo.toml --all
fmt-wasm-client:
cargo fmt --manifest-path clients/webassembly/Cargo.toml --all
wasm:
RUSTFLAGS='-C link-arg=-s' cargo build --manifest-path contracts/Cargo.toml --release --target wasm32-unknown-unknown RUSTFLAGS='-C link-arg=-s' cargo build --manifest-path contracts/Cargo.toml --release --target wasm32-unknown-unknown
wasm-opt:
wasm-opt --disable-sign-ext -Os $(VESTING_CONTRACT) -o $(VESTING_CONTRACT)
wasm-opt --disable-sign-ext -Os $(MIXNET_CONTRACT) -o $(MIXNET_CONTRACT)
wasm-opt --disable-sign-ext -Os $(SERVICE_PROVIDER_DIRECTORY_CONTRACT) -o $(SERVICE_PROVIDER_DIRECTORY_CONTRACT)
wasm-opt --disable-sign-ext -Os $(NAME_SERVICE_CONTRACT) -o $(NAME_SERVICE_CONTRACT)
# -----------------------------------------------------------------------------
# Misc
# -----------------------------------------------------------------------------
# NOTE: this seems deprecated an not needed anymore?
mixnet-opt: wasm mixnet-opt: wasm
cd contracts/mixnet && make opt cd contracts/mixnet && make opt
@@ -124,5 +110,5 @@ generate-typescript:
cd tools/ts-rs-cli && cargo run && cd ../.. cd tools/ts-rs-cli && cargo run && cd ../..
yarn types:lint:fix yarn types:lint:fix
run-api-tests: run-validator-tests:
cd nym-api/tests/functional_test && yarn test:qa cd nym-api/tests/functional_test && yarn test:qa
+6 -10
View File
@@ -16,13 +16,13 @@ The platform is composed of multiple Rust crates. Top-level executable binary cr
* nym-wallet - a desktop wallet implemented using the [Tauri](https://tauri.studio/en/docs/about/intro) framework. * nym-wallet - a desktop wallet implemented using the [Tauri](https://tauri.studio/en/docs/about/intro) framework.
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0)
[![Build Status](https://img.shields.io/github/actions/workflow/status/nymtech/nym/build.yml?branch=develop&style=for-the-badge&logo=github-actions)](https://github.com/nymtech/nym/actions?query=branch%3Adevelop) [![Build Status](https://img.shields.io/github/workflow/status/nymtech/nym/Continuous%20integration/develop?style=for-the-badge&logo=github-actions)](https://github.com/nymtech/nym/actions?query=branch%3Adevelop)
### Building ### Building
Platform build instructions are available on [our docs site](https://nymtech.net/docs/binaries/build-nym.html). Platform build instructions are available on [our docs site](https://nymtech.net/docs/stable/run-nym-nodes/build-nym).
Wallet build instructions are also available on [our docs site](https://nymtech.net/docs/wallet/desktop-wallet.html). Wallet build instructions are also available on [our docs site](https://nymtech.net/docs/stable/nym-apps/wallet#for-developers).
### Developing ### Developing
@@ -32,11 +32,7 @@ For Typescript components, please see [ts-packages](./ts-packages).
### Developer chat ### Developer chat
> We used to use Keybase for developer chats, but we have since migrated to Matrix and Discord. We no longer check the old **nymtech.friends** Keybase team. You can chat to us in [Keybase](https://keybase.io). Download their chat app, then click **Teams -> Join a team**. Type **nymtech.friends** into the team name and hit **continue**. For general chat, hang out in the **#general** channel. Our development takes places in the **#dev** channel. Node operators should be in the **#node-operators** channel.
You can chat to us in two places:
* The #dev channel on [Matrix](https://matrix.to/#/#dev:nymtech.chat)
* The various developer channels on [Discord](https://discord.gg/nym)
### Rewards ### Rewards
@@ -50,7 +46,7 @@ Node, node operator and delegator rewards are determined according to the princi
|<img src="https://render.githubusercontent.com/render/math?math=\lambda_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\lambda_{i}#gh-dark-mode-only">|ratio of stake operator has pledged to their node to the token circulating supply. |<img src="https://render.githubusercontent.com/render/math?math=\lambda_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\lambda_{i}#gh-dark-mode-only">|ratio of stake operator has pledged to their node to the token circulating supply.
|<img src="https://render.githubusercontent.com/render/math?math=\omega_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\omega_{i}#gh-dark-mode-only">|fraction of total effort undertaken by node `i`, set to `1/k`. |<img src="https://render.githubusercontent.com/render/math?math=\omega_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\omega_{i}#gh-dark-mode-only">|fraction of total effort undertaken by node `i`, set to `1/k`.
|<img src="https://render.githubusercontent.com/render/math?math=k#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}k#gh-dark-mode-only">|number of nodes stakeholders are incentivised to create, set by the validators, a matter of governance. Currently determined by the `reward set` size, and set to 720 in testnet Sandbox. |<img src="https://render.githubusercontent.com/render/math?math=k#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}k#gh-dark-mode-only">|number of nodes stakeholders are incentivised to create, set by the validators, a matter of governance. Currently determined by the `reward set` size, and set to 720 in testnet Sandbox.
|<img src="https://render.githubusercontent.com/render/math?math=\alpha#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\alpha#gh-dark-mode-only">|Sybil attack resistance parameter - the higher this parameter is set the stronger the reduction in competitiveness gets for a Sybil attacker. |<img src="https://render.githubusercontent.com/render/math?math=\alpha#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}\alpha#gh-dark-mode-only">|Sybil attack resistance parameter - the higher this parameter is set the stronger the reduction in competitivness gets for a Sybil attacker.
|<img src="https://render.githubusercontent.com/render/math?math=PM_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PM_{i}#gh-dark-mode-only">|declared profit margin of operator `i`, defaults to 10% in. |<img src="https://render.githubusercontent.com/render/math?math=PM_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PM_{i}#gh-dark-mode-only">|declared profit margin of operator `i`, defaults to 10% in.
|<img src="https://render.githubusercontent.com/render/math?math=PF_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PF_{i}#gh-dark-mode-only">|uptime of node `i`, scaled to 0 - 1, for the rewarding epoch |<img src="https://render.githubusercontent.com/render/math?math=PF_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PF_{i}#gh-dark-mode-only">|uptime of node `i`, scaled to 0 - 1, for the rewarding epoch
|<img src="https://render.githubusercontent.com/render/math?math=PP_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PP_{i}#gh-dark-mode-only">|cost of operating node `i` for the duration of the rewarding epoch, set to 40 NYMT. |<img src="https://render.githubusercontent.com/render/math?math=PP_{i}#gh-light-mode-only"><img src="https://render.githubusercontent.com/render/math?math=\color{white}PP_{i}#gh-dark-mode-only">|cost of operating node `i` for the duration of the rewarding epoch, set to 40 NYMT.
@@ -74,7 +70,7 @@ Operator of node `i` is credited with the following amount:
<img src="https://render.githubusercontent.com/render/math?math=min\{PP_{i},R_{i})\} %2b max\{0, (PM_{i} %2b (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only"> <img src="https://render.githubusercontent.com/render/math?math=min\{PP_{i},R_{i})\} %2b max\{0, (PM_{i} %2b (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only">
<img src="https://render.githubusercontent.com/render/math?math=\color{white}min\{PP_{i},R_{i})\} %2b max\{0, (PM_{i} %2b (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only"> <img src="https://render.githubusercontent.com/render/math?math=\color{white}min\{PP_{i},R_{i})\} %2b max\{0, (PM_{i} %2b (1 - PM_{i}) \cdot \lambda_{i}/\delta_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only">
Delegate with stake `s` receives: Delegate with stake `s` recieves:
<img src="https://render.githubusercontent.com/render/math?math=max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only"> <img src="https://render.githubusercontent.com/render/math?math=max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-light-mode-only">
<img src="https://render.githubusercontent.com/render/math?math=\color{white}max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only"> <img src="https://render.githubusercontent.com/render/math?math=\color{white}max\{0, (1-PM_{i}) \cdot (s^'/\sigma_{i}) \cdot (R_{i} - PP_{i})\}#gh-dark-mode-only">
+3 -3
View File
@@ -3,8 +3,8 @@ Critical bug or security issue 💥
If you're here because you're trying to figure out how to notify us of a security issue, go to Discord, and alert the core engineers: If you're here because you're trying to figure out how to notify us of a security issue, go to Discord, and alert the core engineers:
Dave Hrycyszyn futurechimp#5430 Dave Hrycyszyn futurechimp#5430
Drazen Urch drazen#4873
Jedrzej Stuczynski "Jedrzej | Nym#5666" Jedrzej Stuczynski "Jedrzej | Nym#5666"
Fran Arbanas | franarbanas#0995
Mark Sinclair | marknym#8088
Please avoid opening public issues on GitHub that contain information about a potential security vulnerability as this makes it difficult to reduce the impact and harm of valid security issues.
Please avoid opening public issues on GitHub that contain information about a potential security vulnerability as this makes it difficult to reduce the impact and harm of valid security issues.
@@ -1,6 +1,6 @@
[package] [package]
name = "nym-client-core" name = "client-core"
version = "1.1.14" version = "1.1.8"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"] authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2021" edition = "2021"
rust-version = "1.66" rust-version = "1.66"
@@ -8,44 +8,34 @@ rust-version = "1.66"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies] [dependencies]
async-trait = { workspace = true } async-trait = { version = "0.1.58" }
base64 = "0.21.2"
dirs = "4.0" dirs = "4.0"
dashmap = "5.4.0" dashmap = "5.4.0"
futures = "0.3" futures = "0.3"
humantime-serde = "1.0" humantime-serde = "1.0"
log = { workspace = true } log = { workspace = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"] } rand = { version = "0.7.3", features = ["wasm-bindgen"] }
serde = { workspace = true, features = ["derive"] } serde = { version = "1.0", features = ["derive"] }
serde_json = { workspace = true } serde_json = "1.0.89"
sha2 = "0.10.6"
tap = "1.0.1" tap = "1.0.1"
thiserror = "1.0.34" thiserror = "1.0.34"
url = { version ="2.2", features = ["serde"] } url = { version ="2.2", features = ["serde"] }
tungstenite = { version = "0.13.0", default-features = false }
tokio = { version = "1.24.1", features = ["macros"]} tokio = { version = "1.24.1", features = ["macros"]}
time = "0.3.17" time = "0.3.17"
zeroize = { workspace = true }
# internal # internal
nym-bandwidth-controller = { path = "../bandwidth-controller" } config = { path = "../../common/config" }
nym-config = { path = "../config" } client-connections = { path = "../../common/client-connections" }
nym-crypto = { path = "../crypto" } crypto = { path = "../../common/crypto" }
nym-gateway-client = { path = "../client-libs/gateway-client" } gateway-client = { path = "../../common/client-libs/gateway-client" }
#gateway-client = { path = "../../common/client-libs/gateway-client", default-features = false, features = ["wasm", "coconut"] } #gateway-client = { path = "../../common/client-libs/gateway-client", default-features = false, features = ["wasm", "coconut"] }
nym-gateway-requests = { path = "../../gateway/gateway-requests" } gateway-requests = { path = "../../gateway/gateway-requests" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" } nonexhaustive-delayqueue = { path = "../../common/nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" } nymsphinx = { path = "../../common/nymsphinx" }
nym-pemstore = { path = "../pemstore" } pemstore = { path = "../../common/pemstore" }
nym-topology = { path = "../topology" } topology = { path = "../../common/topology" }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false } validator-client = { path = "../../common/client-libs/validator-client", default-features = false }
nym-task = { path = "../task" } task = { path = "../../common/task" }
nym-credential-storage = { path = "../credential-storage" }
nym-network-defaults = { path = "../network-defaults" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.nym-validator-client]
path = "../client-libs/validator-client"
features = ["nyxd-client"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream] [target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream]
version = "0.1.11" version = "0.1.11"
@@ -55,9 +45,6 @@ features = ["time"]
version = "1.24.1" version = "1.24.1"
features = ["time"] features = ["time"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-tungstenite]
version = "0.14"
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx] [target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx]
version = "0.6.2" version = "0.6.2"
features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"]
@@ -78,8 +65,7 @@ version = "0.2.4"
features = ["futures"] features = ["futures"]
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils] [target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils]
path = "../wasm-utils" path = "../../common/wasm-utils"
features = ["websocket"]
[target."cfg(target_arch = \"wasm32\")".dependencies.time] [target."cfg(target_arch = \"wasm32\")".dependencies.time]
version = "0.3.17" version = "0.3.17"
@@ -95,5 +81,5 @@ sqlx = { version = "0.6.2", features = ["runtime-tokio-rustls", "sqlite", "macro
[features] [features]
default = [] default = []
fs-surb-storage = ["sqlx"] fs-surb-storage = ["sqlx"]
wasm = ["nym-gateway-client/wasm"] wasm = ["gateway-client/wasm"]
@@ -1,15 +1,11 @@
// Copyright 2023 - Nym Technologies SA <contact@nymtech.net> // Copyright 2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
//
use crate::{client::replies::reply_storage, config::DebugConfig}; use crate::{client::replies::reply_storage, config::DebugConfig};
pub fn setup_empty_reply_surb_backend(debug_config: &DebugConfig) -> reply_storage::Empty { pub fn setup_empty_reply_surb_backend(debug_config: &DebugConfig) -> reply_storage::Empty {
reply_storage::Empty { reply_storage::Empty {
min_surb_threshold: debug_config min_surb_threshold: debug_config.minimum_reply_surb_storage_threshold,
.reply_surbs max_surb_threshold: debug_config.maximum_reply_surb_storage_threshold,
.minimum_reply_surb_storage_threshold,
max_surb_threshold: debug_config
.reply_surbs
.maximum_reply_surb_storage_threshold,
} }
} }
@@ -1,12 +1,9 @@
// Copyright 2022-2023 - Nym Technologies SA <contact@nymtech.net> // Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use super::received_buffer::ReceivedBufferMessage;
use crate::client::base_client::storage::MixnetClientStorage;
use crate::client::cover_traffic_stream::LoopCoverTrafficStream; use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender}; use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender};
use crate::client::key_manager::persistence::KeyStore; use crate::client::key_manager::KeyManager;
use crate::client::key_manager::ManagedKeys;
use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController}; use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController};
use crate::client::real_messages_control; use crate::client::real_messages_control;
use crate::client::real_messages_control::RealMessagesController; use crate::client::real_messages_control::RealMessagesController;
@@ -18,64 +15,47 @@ use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyCon
use crate::client::replies::reply_storage::{ use crate::client::replies::reply_storage::{
CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys, CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys,
}; };
use crate::client::topology_control::nym_api_provider::NymApiTopologyProvider;
use crate::client::topology_control::{ use crate::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig, TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
}; };
use crate::config::{Config, DebugConfig, GatewayEndpointConfig}; use crate::config::{Config, DebugConfig, GatewayEndpointConfig};
use crate::error::ClientCoreError; use crate::error::ClientCoreError;
use crate::{config, spawn_future}; use crate::spawn_future;
use client_connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use crypto::asymmetric::{encryption, identity};
use futures::channel::mpsc; use futures::channel::mpsc;
use log::{debug, info}; use gateway_client::bandwidth::BandwidthController;
use nym_bandwidth_controller::BandwidthController; #[cfg(target_arch = "wasm32")]
use nym_credential_storage::storage::Storage as CredentialStorage; use gateway_client::wasm_mockups::CosmWasmClient;
use nym_crypto::asymmetric::{encryption, identity}; use gateway_client::{
use nym_gateway_client::{
AcknowledgementReceiver, AcknowledgementSender, GatewayClient, MixnetMessageReceiver, AcknowledgementReceiver, AcknowledgementSender, GatewayClient, MixnetMessageReceiver,
MixnetMessageSender, MixnetMessageSender,
}; };
use nym_sphinx::acknowledgements::AckKey; use log::{debug, info};
use nym_sphinx::addressing::clients::Recipient; use nymsphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::nodes::NodeIdentity; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::params::PacketType; use nymsphinx::addressing::nodes::NodeIdentity;
use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver}; use nymsphinx::receiver::ReconstructedMessage;
use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use nym_task::{TaskClient, TaskManager};
use nym_topology::provider_trait::TopologyProvider;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration;
use tap::TapFallible; use tap::TapFallible;
use task::{TaskClient, TaskManager};
use url::Url; use url::Url;
#[cfg(target_arch = "wasm32")]
use nym_bandwidth_controller::wasm_mockups::DkgQueryClient;
use crate::client::base_client::storage::gateway_details::GatewayDetailsStore;
use crate::init::{setup_gateway, GatewaySetup, InitialisationDetails};
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
use nym_validator_client::nyxd::traits::DkgQueryClient; use validator_client::nyxd::CosmWasmClient;
use super::received_buffer::ReceivedBufferMessage;
#[cfg(all(not(target_arch = "wasm32"), feature = "fs-surb-storage"))] #[cfg(all(not(target_arch = "wasm32"), feature = "fs-surb-storage"))]
pub mod non_wasm_helpers; pub mod non_wasm_helpers;
pub mod helpers; pub mod helpers;
pub mod storage;
#[derive(Clone)]
pub struct ClientInput { pub struct ClientInput {
pub connection_command_sender: ConnectionCommandSender, pub connection_command_sender: ConnectionCommandSender,
pub input_sender: InputMessageSender, pub input_sender: InputMessageSender,
} }
impl ClientInput {
pub async fn send(
&self,
message: InputMessage,
) -> Result<(), tokio::sync::mpsc::error::SendError<InputMessage>> {
self.input_sender.send(message).await
}
}
#[derive(Clone)]
pub struct ClientOutput { pub struct ClientOutput {
pub received_buffer_request_sender: ReceivedBufferRequestSender, pub received_buffer_request_sender: ReceivedBufferRequestSender,
} }
@@ -96,11 +76,9 @@ impl ClientOutput {
} }
} }
#[derive(Clone, Debug)]
pub struct ClientState { pub struct ClientState {
pub shared_lane_queue_lengths: LaneQueueLengths, pub shared_lane_queue_lengths: LaneQueueLengths,
pub reply_controller_sender: ReplyControllerSender, pub reply_controller_sender: ReplyControllerSender,
pub topology_accessor: TopologyAccessor,
} }
pub enum ClientInputStatus { pub enum ClientInputStatus {
@@ -157,58 +135,67 @@ impl From<bool> for CredentialsToggle {
} }
} }
pub struct BaseClientBuilder<'a, C, S: MixnetClientStorage> { pub struct BaseClientBuilder<'a, B, C: Clone> {
config: &'a Config, // due to wasm limitations I had to split it like this : (
client_store: S, gateway_config: &'a GatewayEndpointConfig,
dkg_query_client: Option<C>, debug_config: &'a DebugConfig,
custom_topology_provider: Option<Box<dyn TopologyProvider + Send + Sync>>, disabled_credentials: bool,
setup_method: GatewaySetup, nym_api_endpoints: Vec<Url>,
reply_storage_backend: B,
bandwidth_controller: Option<BandwidthController<C>>,
key_manager: KeyManager,
} }
impl<'a, C, S> BaseClientBuilder<'a, C, S> impl<'a, B, C> BaseClientBuilder<'a, B, C>
where where
S: MixnetClientStorage + 'static, B: ReplyStorageBackend + Send + Sync + 'static,
C: DkgQueryClient + Send + Sync + 'static, C: CosmWasmClient + Sync + Send + Clone + 'static,
{ {
pub fn new( pub fn new_from_base_config<T>(
base_config: &'a Config, base_config: &'a Config<T>,
client_store: S, key_manager: KeyManager,
dkg_query_client: Option<C>, bandwidth_controller: Option<BandwidthController<C>>,
) -> BaseClientBuilder<'a, C, S> { reply_storage_backend: B,
) -> BaseClientBuilder<'a, B, C> {
BaseClientBuilder { BaseClientBuilder {
config: base_config, gateway_config: base_config.get_gateway_endpoint_config(),
client_store, debug_config: base_config.get_debug_config(),
dkg_query_client, disabled_credentials: base_config.get_disabled_credentials_mode(),
custom_topology_provider: None, nym_api_endpoints: base_config.get_nym_api_endpoints(),
setup_method: GatewaySetup::MustLoad, bandwidth_controller,
reply_storage_backend,
key_manager,
} }
} }
pub fn with_gateway_setup(mut self, setup: GatewaySetup) -> Self { pub fn new(
self.setup_method = setup; gateway_config: &'a GatewayEndpointConfig,
self debug_config: &'a DebugConfig,
key_manager: KeyManager,
bandwidth_controller: Option<BandwidthController<C>>,
reply_storage_backend: B,
credentials_toggle: CredentialsToggle,
nym_api_endpoints: Vec<Url>,
) -> BaseClientBuilder<'a, B, C> {
BaseClientBuilder {
gateway_config,
debug_config,
disabled_credentials: credentials_toggle.is_disabled(),
nym_api_endpoints,
reply_storage_backend,
bandwidth_controller,
key_manager,
}
} }
pub fn with_topology_provider( pub fn as_mix_recipient(&self) -> Recipient {
mut self,
provider: Box<dyn TopologyProvider + Send + Sync>,
) -> Self {
self.custom_topology_provider = Some(provider);
self
}
// note: do **NOT** make this method public as its only valid usage is from within `start_base`
// because it relies on the crypto keys being already loaded
fn mix_address(
managed_keys: &ManagedKeys,
gateway_config: &GatewayEndpointConfig,
) -> Recipient {
Recipient::new( Recipient::new(
*managed_keys.identity_public_key(), *self.key_manager.identity_keypair().public_key(),
*managed_keys.encryption_public_key(), *self.key_manager.encryption_keypair().public_key(),
// TODO: below only works under assumption that gateway address == gateway id // TODO: below only works under assumption that gateway address == gateway id
// (which currently is true) // (which currently is true)
NodeIdentity::from_base58_string(&gateway_config.gateway_id).unwrap(), NodeIdentity::from_base58_string(&self.gateway_config.gateway_id).unwrap(),
) )
} }
@@ -224,16 +211,21 @@ where
) { ) {
info!("Starting loop cover traffic stream..."); info!("Starting loop cover traffic stream...");
let stream = LoopCoverTrafficStream::new( let mut stream = LoopCoverTrafficStream::new(
ack_key, ack_key,
debug_config.acknowledgements.average_ack_delay, debug_config.average_ack_delay,
debug_config.average_packet_delay,
debug_config.loop_cover_traffic_average_delay,
mix_tx, mix_tx,
self_address, self_address,
topology_accessor, topology_accessor,
debug_config.traffic,
debug_config.cover_traffic,
); );
if let Some(size) = debug_config.use_extended_packet_size {
log::debug!("Setting extended packet size: {:?}", size);
stream.set_custom_packet_size(size.into());
}
stream.start_with_shutdown(shutdown); stream.start_with_shutdown(shutdown);
} }
@@ -250,7 +242,6 @@ where
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver, client_connection_rx: ConnectionCommandReceiver,
shutdown: TaskClient, shutdown: TaskClient,
packet_type: PacketType,
) { ) {
info!("Starting real traffic stream..."); info!("Starting real traffic stream...");
@@ -266,7 +257,7 @@ where
lane_queue_lengths, lane_queue_lengths,
client_connection_rx, client_connection_rx,
) )
.start_with_shutdown(shutdown, packet_type); .start_with_shutdown(shutdown);
} }
// buffer controlling all messages fetched from provider // buffer controlling all messages fetched from provider
@@ -280,96 +271,83 @@ where
shutdown: TaskClient, shutdown: TaskClient,
) { ) {
info!("Starting received messages buffer controller..."); info!("Starting received messages buffer controller...");
let controller: ReceivedMessagesBufferController<SphinxMessageReceiver> = ReceivedMessagesBufferController::new(
ReceivedMessagesBufferController::new( local_encryption_keypair,
local_encryption_keypair, query_receiver,
query_receiver, mixnet_receiver,
mixnet_receiver, reply_key_storage,
reply_key_storage, reply_controller_sender,
reply_controller_sender, )
); .start_with_shutdown(shutdown)
controller.start_with_shutdown(shutdown)
} }
async fn start_gateway_client( async fn start_gateway_client(
config: &Config, &mut self,
gateway_config: GatewayEndpointConfig,
managed_keys: &ManagedKeys,
bandwidth_controller: Option<BandwidthController<C, S::CredentialStore>>,
mixnet_message_sender: MixnetMessageSender, mixnet_message_sender: MixnetMessageSender,
ack_sender: AcknowledgementSender, ack_sender: AcknowledgementSender,
shutdown: TaskClient, shutdown: TaskClient,
) -> Result<GatewayClient<C, S::CredentialStore>, ClientCoreError> ) -> Result<GatewayClient<C>, ClientCoreError> {
where let gateway_id = self.gateway_config.gateway_id.clone();
<S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static, if gateway_id.is_empty() {
<S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static, return Err(ClientCoreError::GatewayIdUnknown);
{ }
let gateway_address = gateway_config.gateway_listener.clone(); let gateway_address = self.gateway_config.gateway_listener.clone();
let gateway_id = gateway_config.gateway_id; if gateway_address.is_empty() {
return Err(ClientCoreError::GatwayAddressUnknown);
}
// TODO: in theory, at this point, this should be infallible
let gateway_identity = identity::PublicKey::from_base58_string(gateway_id) let gateway_identity = identity::PublicKey::from_base58_string(gateway_id)
.map_err(ClientCoreError::UnableToCreatePublicKeyFromGatewayId)?; .map_err(ClientCoreError::UnableToCreatePublicKeyFromGatewayId)?;
// disgusting wasm workaround since there's no key persistence there (nor `client init`)
let shared_key = if self.key_manager.is_gateway_key_set() {
Some(self.key_manager.gateway_shared_key())
} else {
None
};
let mut gateway_client = GatewayClient::new( let mut gateway_client = GatewayClient::new(
gateway_address, gateway_address,
managed_keys.identity_keypair(), self.key_manager.identity_keypair(),
gateway_identity, gateway_identity,
Some(managed_keys.must_get_gateway_shared_key()), shared_key,
mixnet_message_sender, mixnet_message_sender,
ack_sender, ack_sender,
config.debug.gateway_connection.gateway_response_timeout, self.debug_config.gateway_response_timeout,
bandwidth_controller, self.bandwidth_controller.take(),
shutdown, shutdown,
); );
gateway_client.set_disabled_credentials_mode(config.client.disabled_credentials_mode); gateway_client.set_disabled_credentials_mode(self.disabled_credentials);
let shared_key = gateway_client gateway_client
.authenticate_and_start() .authenticate_and_start()
.await .await
.tap_err(|err| { .tap_err(|err| {
log::error!("Could not authenticate and start up the gateway connection - {err}") log::error!("Could not authenticate and start up the gateway connection - {err}")
})?; })?;
managed_keys.ensure_gateway_key(shared_key);
Ok(gateway_client) Ok(gateway_client)
} }
fn setup_topology_provider(
custom_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
nym_api_urls: Vec<Url>,
) -> Box<dyn TopologyProvider + Send + Sync> {
// if no custom provider was ... provided ..., create one using nym-api
custom_provider.unwrap_or_else(|| {
Box::new(NymApiTopologyProvider::new(
nym_api_urls,
env!("CARGO_PKG_VERSION").to_string(),
))
})
}
// future responsible for periodically polling directory server and updating // future responsible for periodically polling directory server and updating
// the current global view of topology // the current global view of topology
async fn start_topology_refresher( async fn start_topology_refresher(
topology_provider: Box<dyn TopologyProvider + Send + Sync>, nym_api_urls: Vec<Url>,
topology_config: config::Topology, refresh_rate: Duration,
topology_accessor: TopologyAccessor, topology_accessor: TopologyAccessor,
mut shutdown: TaskClient, shutdown: TaskClient,
) -> Result<(), ClientCoreError> { ) -> Result<(), ClientCoreError> {
let topology_refresher_config = let topology_refresher_config = TopologyRefresherConfig::new(
TopologyRefresherConfig::new(topology_config.topology_refresh_rate); nym_api_urls,
refresh_rate,
let mut topology_refresher = TopologyRefresher::new( env!("CARGO_PKG_VERSION").to_string(),
topology_refresher_config,
topology_accessor,
topology_provider,
); );
let mut topology_refresher =
TopologyRefresher::new(topology_refresher_config, topology_accessor);
// before returning, block entire runtime to refresh the current network view so that any // before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view // components depending on topology would see a non-empty view
info!("Obtaining initial network topology"); info!("Obtaining initial network topology");
topology_refresher.try_refresh().await; topology_refresher.refresh().await;
if let Err(err) = topology_refresher.ensure_topology_is_routable().await { if let Err(err) = topology_refresher.ensure_topology_is_routable().await {
log::error!( log::error!(
@@ -379,101 +357,65 @@ where
return Err(ClientCoreError::InsufficientNetworkTopology(err)); return Err(ClientCoreError::InsufficientNetworkTopology(err));
} }
if topology_config.disable_refreshing { info!("Starting topology refresher...");
// if we're not spawning the refresher, don't cause shutdown immediately topology_refresher.start_with_shutdown(shutdown);
info!("The topology refesher is not going to be started");
shutdown.mark_as_success();
} else {
// don't spawn the refresher if we don't want to be refreshing the topology.
// only use the initial values obtained
info!("Starting topology refresher...");
topology_refresher.start_with_shutdown(shutdown);
}
Ok(()) Ok(())
} }
// controller for sending packets to mixnet (either real traffic or cover traffic) // controller for sending sphinx packets to mixnet (either real traffic or cover traffic)
// TODO: if we want to send control messages to gateway_client, this CAN'T take the ownership // TODO: if we want to send control messages to gateway_client, this CAN'T take the ownership
// over it. Perhaps GatewayClient needs to be thread-shareable or have some channel for // over it. Perhaps GatewayClient needs to be thread-shareable or have some channel for
// requests? // requests?
fn start_mix_traffic_controller( fn start_mix_traffic_controller(
gateway_client: GatewayClient<C, S::CredentialStore>, gateway_client: GatewayClient<C>,
shutdown: TaskClient, shutdown: TaskClient,
) -> BatchMixMessageSender ) -> BatchMixMessageSender {
where
<S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static,
{
info!("Starting mix traffic controller..."); info!("Starting mix traffic controller...");
let (mix_traffic_controller, mix_tx) = MixTrafficController::new(gateway_client); let (mix_traffic_controller, mix_tx) = MixTrafficController::new(gateway_client);
mix_traffic_controller.start_with_shutdown(shutdown); mix_traffic_controller.start_with_shutdown(shutdown);
mix_tx mix_tx
} }
// TODO: rename it as it implies the data is persistent whilst one can use InMemBackend
async fn setup_persistent_reply_storage( async fn setup_persistent_reply_storage(
backend: S::ReplyStore, backend: B,
shutdown: TaskClient, shutdown: TaskClient,
) -> Result<CombinedReplyStorage, ClientCoreError> ) -> Result<CombinedReplyStorage, ClientCoreError>
where where
<S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send, <B as ReplyStorageBackend>::StorageError: Sync + Send,
S::ReplyStore: Send + Sync,
{ {
log::trace!("Setup persistent reply storage"); if backend.is_active() {
let persistent_storage = PersistentReplyStorage::new(backend); log::trace!("Setup persistent reply storage");
let mem_store = persistent_storage let persistent_storage = PersistentReplyStorage::new(backend);
.load_state_from_backend() let mem_store = persistent_storage
.await .load_state_from_backend()
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
let store_clone = mem_store.clone();
spawn_future(async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await .await
}); .map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
Ok(mem_store) let store_clone = mem_store.clone();
} spawn_future(async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
});
async fn initialise_keys_and_gateway(&self) -> Result<InitialisationDetails, ClientCoreError> Ok(mem_store)
where } else {
<S::KeyStore as KeyStore>::StorageError: Sync + Send, log::trace!("Setup inactive reply storage");
<S::GatewayDetailsStore as GatewayDetailsStore>::StorageError: Sync + Send, Ok(backend
{ .get_inactive_storage()
setup_gateway( .map_err(|err| ClientCoreError::SurbStorageError {
&self.setup_method, source: Box::new(err),
self.client_store.key_store(), })?)
self.client_store.gateway_details_store(), }
false,
Some(&self.config.client.nym_api_urls),
)
.await
} }
pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError> pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError>
where where
S::ReplyStore: Send + Sync, <B as ReplyStorageBackend>::StorageError: Sync + Send,
<S::KeyStore as KeyStore>::StorageError: Send + Sync,
<S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send,
<S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static,
<S::GatewayDetailsStore as GatewayDetailsStore>::StorageError: Sync + Send,
{ {
info!("Starting nym client"); info!("Starting nym client");
// derive (or load) client keys and gateway configuration
let details = self.initialise_keys_and_gateway().await?;
let gateway_config = details.gateway_details;
let managed_keys = details.managed_keys;
let (reply_storage_backend, credential_store) = self.client_store.into_runtime_stores();
let bandwidth_controller = self
.dkg_query_client
.map(|client| BandwidthController::new(credential_store, client));
// channels for inter-component communication // channels for inter-component communication
// TODO: make the channels be internally created by the relevant components // TODO: make the channels be internally created by the relevant components
// rather than creating them here, so say for example the buffer controller would create the request channels // rather than creating them here, so say for example the buffer controller would create the request channels
@@ -500,39 +442,30 @@ where
let (reply_controller_sender, reply_controller_receiver) = let (reply_controller_sender, reply_controller_receiver) =
reply_controller::requests::new_control_channels(); reply_controller::requests::new_control_channels();
let self_address = Self::mix_address(&managed_keys, &gateway_config); let self_address = self.as_mix_recipient();
// the components are started in very specific order. Unless you know what you are doing, // the components are started in very specific order. Unless you know what you are doing,
// do not change that. // do not change that.
let gateway_client = Self::start_gateway_client( let gateway_client = self
self.config, .start_gateway_client(mixnet_messages_sender, ack_sender, task_manager.subscribe())
gateway_config, .await?;
&managed_keys,
bandwidth_controller, let reply_storage = Self::setup_persistent_reply_storage(
mixnet_messages_sender, self.reply_storage_backend,
ack_sender,
task_manager.subscribe(), task_manager.subscribe(),
) )
.await?; .await?;
let reply_storage =
Self::setup_persistent_reply_storage(reply_storage_backend, task_manager.subscribe())
.await?;
let topology_provider = Self::setup_topology_provider(
self.custom_topology_provider.take(),
self.config.get_nym_api_endpoints(),
);
Self::start_topology_refresher( Self::start_topology_refresher(
topology_provider, self.nym_api_endpoints.clone(),
self.config.debug.topology, self.debug_config.topology_refresh_rate,
shared_topology_accessor.clone(), shared_topology_accessor.clone(),
task_manager.subscribe(), task_manager.subscribe(),
) )
.await?; .await?;
Self::start_received_messages_buffer_controller( Self::start_received_messages_buffer_controller(
managed_keys.encryption_keypair(), self.key_manager.encryption_keypair(),
received_buffer_request_receiver, received_buffer_request_receiver,
mixnet_messages_receiver, mixnet_messages_receiver,
reply_storage.key_storage(), reply_storage.key_storage(),
@@ -540,11 +473,11 @@ where
task_manager.subscribe(), task_manager.subscribe(),
); );
// The message_sender is the transmitter for any component generating sphinx packets // The sphinx_message_sender is the transmitter for any component generating sphinx packets
// that are to be sent to the mixnet. They are used by cover traffic stream and real // that are to be sent to the mixnet. They are used by cover traffic stream and real
// traffic stream. // traffic stream.
// The MixTrafficController then sends the actual traffic // The MixTrafficController then sends the actual traffic
let message_sender = let sphinx_message_sender =
Self::start_mix_traffic_controller(gateway_client, task_manager.subscribe()); Self::start_mix_traffic_controller(gateway_client, task_manager.subscribe());
// Channels that the websocket listener can use to signal downstream to the real traffic // Channels that the websocket listener can use to signal downstream to the real traffic
@@ -555,39 +488,38 @@ where
// primarily to throttle incoming connections (e.g socks5 for attached network-requesters) // primarily to throttle incoming connections (e.g socks5 for attached network-requesters)
let shared_lane_queue_lengths = LaneQueueLengths::new(); let shared_lane_queue_lengths = LaneQueueLengths::new();
let controller_config = real_messages_control::Config::new( let mut controller_config = real_messages_control::Config::new(
&self.config.debug, self.debug_config,
managed_keys.ack_key(), self.key_manager.ack_key(),
self_address, self_address,
); );
if let Some(size) = self.debug_config.use_extended_packet_size {
log::debug!("Setting extended packet size: {:?}", size);
controller_config.set_custom_packet_size(size.into());
}
Self::start_real_traffic_controller( Self::start_real_traffic_controller(
controller_config, controller_config,
shared_topology_accessor.clone(), shared_topology_accessor.clone(),
ack_receiver, ack_receiver,
input_receiver, input_receiver,
message_sender.clone(), sphinx_message_sender.clone(),
reply_storage, reply_storage,
reply_controller_sender.clone(), reply_controller_sender.clone(),
reply_controller_receiver, reply_controller_receiver,
shared_lane_queue_lengths.clone(), shared_lane_queue_lengths.clone(),
client_connection_rx, client_connection_rx,
task_manager.subscribe(), task_manager.subscribe(),
self.config.debug.traffic.packet_type,
); );
if !self if !self.debug_config.disable_loop_cover_traffic_stream {
.config
.debug
.cover_traffic
.disable_loop_cover_traffic_stream
{
Self::start_cover_traffic_stream( Self::start_cover_traffic_stream(
&self.config.debug, self.debug_config,
managed_keys.ack_key(), self.key_manager.ack_key(),
self_address, self_address,
shared_topology_accessor.clone(), shared_topology_accessor,
message_sender, sphinx_message_sender,
task_manager.subscribe(), task_manager.subscribe(),
); );
} }
@@ -596,7 +528,6 @@ where
debug!("The address of this client is: {self_address}"); debug!("The address of this client is: {self_address}");
Ok(BaseClient { Ok(BaseClient {
address: self_address,
client_input: ClientInputStatus::AwaitingProducer { client_input: ClientInputStatus::AwaitingProducer {
client_input: ClientInput { client_input: ClientInput {
connection_command_sender: client_connection_tx, connection_command_sender: client_connection_tx,
@@ -611,7 +542,6 @@ where
client_state: ClientState { client_state: ClientState {
shared_lane_queue_lengths, shared_lane_queue_lengths,
reply_controller_sender, reply_controller_sender,
topology_accessor: shared_topology_accessor,
}, },
task_manager, task_manager,
}) })
@@ -619,7 +549,6 @@ where
} }
pub struct BaseClient { pub struct BaseClient {
pub address: Recipient,
pub client_input: ClientInputStatus, pub client_input: ClientInputStatus,
pub client_output: ClientOutputStatus, pub client_output: ClientOutputStatus,
pub client_state: ClientState, pub client_state: ClientState,
@@ -0,0 +1,100 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::replies::reply_storage::{
fs_backend, CombinedReplyStorage, ReplyStorageBackend,
};
use crate::config::DebugConfig;
use crate::error::ClientCoreError;
use log::{error, info};
use std::path::Path;
use std::{fs, io};
use time::OffsetDateTime;
async fn setup_fresh_backend<P: AsRef<Path>>(
db_path: P,
debug_config: &DebugConfig,
) -> Result<fs_backend::Backend, ClientCoreError> {
info!("creating fresh surb database");
let mut storage_backend = match fs_backend::Backend::init(db_path).await {
Ok(backend) => backend,
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}");
return Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
});
}
};
// while I kinda hate that we're going to be creating `CombinedReplyStorage` twice,
// it will only be happening on the very first run and in practice won't incur huge
// costs since the storage is going to be empty
let mem_store = CombinedReplyStorage::new(
debug_config.minimum_reply_surb_storage_threshold,
debug_config.maximum_reply_surb_storage_threshold,
);
storage_backend
.init_fresh(&mem_store)
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
Ok(storage_backend)
}
fn setup_inactive_backend(debug_config: &DebugConfig) -> fs_backend::Backend {
info!("creating inactive surb database");
fs_backend::Backend::new_inactive(
debug_config.minimum_reply_surb_storage_threshold,
debug_config.maximum_reply_surb_storage_threshold,
)
}
fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
let db_path = db_path.as_ref();
debug_assert!(db_path.exists());
let now = OffsetDateTime::now_utc().unix_timestamp();
let suffix = format!("_{now}.corrupted");
let new_extension =
if let Some(existing_extension) = db_path.extension().and_then(|ext| ext.to_str()) {
format!("{existing_extension}.{suffix}")
} else {
suffix
};
let mut renamed = db_path.to_owned();
renamed.set_extension(new_extension);
fs::rename(db_path, renamed)
}
pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
db_path: Option<P>,
debug_config: &DebugConfig,
) -> Result<fs_backend::Backend, ClientCoreError> {
if let Some(db_path) = db_path {
// if the database file doesnt exist, initialise fresh storage, otherwise attempt to load
// the existing one
let db_path = db_path.as_ref();
if db_path.exists() {
info!("loading existing surb database");
match fs_backend::Backend::try_load(db_path).await {
Ok(backend) => Ok(backend),
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path)?;
setup_fresh_backend(db_path, debug_config).await
}
}
} else {
setup_fresh_backend(db_path, debug_config).await
}
} else {
Ok(setup_inactive_backend(debug_config))
}
}
@@ -3,15 +3,15 @@
use crate::client::mix_traffic::BatchMixMessageSender; use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::topology_control::TopologyAccessor; use crate::client::topology_control::TopologyAccessor;
use crate::{config, spawn_future}; use crate::spawn_future;
use futures::task::{Context, Poll}; use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt}; use futures::{Future, Stream, StreamExt};
use log::*; use log::*;
use nym_sphinx::acknowledgements::AckKey; use nymsphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet; use nymsphinx::cover::generate_loop_cover_packet;
use nym_sphinx::params::PacketSize; use nymsphinx::params::PacketSize;
use nym_sphinx::utils::sample_poisson_duration; use nymsphinx::utils::sample_poisson_duration;
use rand::{rngs::OsRng, CryptoRng, Rng}; use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin; use std::pin::Pin;
use std::sync::Arc; use std::sync::Arc;
@@ -34,8 +34,11 @@ where
/// Average delay an acknowledgement packet is going to get delay at a single mixnode. /// Average delay an acknowledgement packet is going to get delay at a single mixnode.
average_ack_delay: Duration, average_ack_delay: Duration,
/// Defines configuration options related to cover traffic. /// Average delay a data packet is going to get delay at a single mixnode.
cover_traffic: config::CoverTraffic, average_packet_delay: Duration,
/// Average delay between sending subsequent cover packets.
average_cover_message_sending_delay: Duration,
/// Internal state, determined by `average_message_sending_delay`, /// Internal state, determined by `average_message_sending_delay`,
/// used to keep track of when a next packet should be sent out. /// used to keep track of when a next packet should be sent out.
@@ -45,7 +48,7 @@ where
#[cfg(target_arch = "wasm32")] #[cfg(target_arch = "wasm32")]
next_delay: Pin<Box<wasm_timer::Delay>>, next_delay: Pin<Box<wasm_timer::Delay>>,
/// Channel used for sending prepared nym packets to `MixTrafficController` that sends them /// Channel used for sending prepared sphinx packets to `MixTrafficController` that sends them
/// out to the network without any further delays. /// out to the network without any further delays.
mix_tx: BatchMixMessageSender, mix_tx: BatchMixMessageSender,
@@ -58,11 +61,8 @@ where
/// Accessor to the common instance of network topology. /// Accessor to the common instance of network topology.
topology_access: TopologyAccessor, topology_access: TopologyAccessor,
/// Primary predefined packet size used for the loop cover messages. /// Predefined packet size used for the loop cover messages.
primary_packet_size: PacketSize, packet_size: PacketSize,
/// Optional secondary predefined packet size used for the loop cover messages.
secondary_packet_size: Option<PacketSize>,
} }
impl<R> Stream for LoopCoverTrafficStream<R> impl<R> Stream for LoopCoverTrafficStream<R>
@@ -83,7 +83,7 @@ where
// we know it's time to send a message, so let's prepare delay for the next one // we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline // Get the `now` by looking at the current `delay` deadline
let avg_delay = self.cover_traffic.loop_cover_traffic_average_delay; let avg_delay = self.average_cover_message_sending_delay;
let next_poisson_delay = sample_poisson_duration(&mut self.rng, avg_delay); let next_poisson_delay = sample_poisson_duration(&mut self.rng, avg_delay);
// The next interval value is `next_poisson_delay` after the one that just // The next interval value is `next_poisson_delay` after the one that just
@@ -107,14 +107,15 @@ where
// obviously when we finally make shared rng that is on 'higher' level, this should become // obviously when we finally make shared rng that is on 'higher' level, this should become
// generic `R` // generic `R`
impl LoopCoverTrafficStream<OsRng> { impl LoopCoverTrafficStream<OsRng> {
#[allow(clippy::too_many_arguments)]
pub fn new( pub fn new(
ack_key: Arc<AckKey>, ack_key: Arc<AckKey>,
average_ack_delay: Duration, average_ack_delay: Duration,
average_packet_delay: Duration,
average_cover_message_sending_delay: Duration,
mix_tx: BatchMixMessageSender, mix_tx: BatchMixMessageSender,
our_full_destination: Recipient, our_full_destination: Recipient,
topology_access: TopologyAccessor, topology_access: TopologyAccessor,
traffic_config: config::Traffic,
cover_config: config::CoverTraffic,
) -> Self { ) -> Self {
let rng = OsRng; let rng = OsRng;
@@ -127,17 +128,21 @@ impl LoopCoverTrafficStream<OsRng> {
LoopCoverTrafficStream { LoopCoverTrafficStream {
ack_key, ack_key,
average_ack_delay, average_ack_delay,
cover_traffic: cover_config, average_packet_delay,
average_cover_message_sending_delay,
next_delay, next_delay,
mix_tx, mix_tx,
our_full_destination, our_full_destination,
rng, rng,
topology_access, topology_access,
primary_packet_size: traffic_config.primary_packet_size, packet_size: Default::default(),
secondary_packet_size: traffic_config.secondary_packet_size,
} }
} }
pub fn set_custom_packet_size(&mut self, packet_size: PacketSize) {
self.packet_size = packet_size;
}
fn set_next_delay(&mut self, amount: Duration) { fn set_next_delay(&mut self, amount: Duration) {
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
let next_delay = Box::pin(time::sleep(amount)); let next_delay = Box::pin(time::sleep(amount));
@@ -148,28 +153,9 @@ impl LoopCoverTrafficStream<OsRng> {
self.next_delay = next_delay; self.next_delay = next_delay;
} }
fn loop_cover_message_size(&mut self) -> PacketSize {
let Some(secondary_packet_size) = self.secondary_packet_size else {
return self.primary_packet_size
};
let use_primary = self
.rng
.gen_bool(self.cover_traffic.cover_traffic_primary_size_ratio);
if use_primary {
self.primary_packet_size
} else {
secondary_packet_size
}
}
async fn on_new_message(&mut self) { async fn on_new_message(&mut self) {
trace!("next cover message!"); trace!("next cover message!");
let cover_traffic_packet_size = self.loop_cover_message_size();
trace!("the next loop cover message will be put in a {cover_traffic_packet_size} packet");
// TODO for way down the line: in very rare cases (during topology update) we might have // TODO for way down the line: in very rare cases (during topology update) we might have
// to wait a really tiny bit before actually obtaining the permit hence messing with our // to wait a really tiny bit before actually obtaining the permit hence messing with our
// poisson delay, but is it really a problem? // poisson delay, but is it really a problem?
@@ -192,9 +178,8 @@ impl LoopCoverTrafficStream<OsRng> {
&self.ack_key, &self.ack_key,
&self.our_full_destination, &self.our_full_destination,
self.average_ack_delay, self.average_ack_delay,
self.cover_traffic.loop_cover_traffic_average_delay, self.average_packet_delay,
cover_traffic_packet_size, self.packet_size,
nym_sphinx::params::PacketType::Mix,
) )
.expect("Somehow failed to generate a loop cover message with a valid topology"); .expect("Somehow failed to generate a loop cover message with a valid topology");
@@ -204,6 +189,10 @@ impl LoopCoverTrafficStream<OsRng> {
// This isn't a problem, if the channel is full means we're already sending the // This isn't a problem, if the channel is full means we're already sending the
// max amount of messages downstream can handle. // max amount of messages downstream can handle.
log::debug!("Failed to send cover message - channel full"); log::debug!("Failed to send cover message - channel full");
// However it's still useful to alert the user that the gateway or the link to
// the gateway can't keep up. Either due to insufficient bandwidth on the
// client side, or that the gateway is overloaded.
log::warn!("Failed to send sphinx packet - gateway or connection to gatway can't keep up");
} }
TrySendError::Closed(_) => { TrySendError::Closed(_) => {
log::warn!("Failed to send cover message - channel closed"); log::warn!("Failed to send cover message - channel closed");
@@ -224,18 +213,10 @@ impl LoopCoverTrafficStream<OsRng> {
tokio::task::yield_now().await; tokio::task::yield_now().await;
} }
pub fn start_with_shutdown(mut self, mut shutdown: nym_task::TaskClient) { pub fn start_with_shutdown(mut self, mut shutdown: task::TaskClient) {
if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with
// so panic and review the code that lead to this branch
panic!("attempted to start LoopCoverTrafficStream while config explicitly disabled it.")
}
// we should set initial delay only when we actually start the stream // we should set initial delay only when we actually start the stream
let sampled = sample_poisson_duration( let sampled =
&mut self.rng, sample_poisson_duration(&mut self.rng, self.average_cover_message_sending_delay);
self.cover_traffic.loop_cover_traffic_average_delay,
);
self.set_next_delay(sampled); self.set_next_delay(sampled);
spawn_future(async move { spawn_future(async move {
@@ -0,0 +1,80 @@
use client_connections::TransmissionLane;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
pub type InputMessageSender = tokio::sync::mpsc::Sender<InputMessage>;
pub type InputMessageReceiver = tokio::sync::mpsc::Receiver<InputMessage>;
#[derive(Debug)]
pub enum InputMessage {
/// The simplest message variant where no additional information is attached.
/// You're simply sending your `data` to specified `recipient` without any tagging.
///
/// Ends up with `NymMessage::Plain` variant
Regular {
recipient: Recipient,
data: Vec<u8>,
lane: TransmissionLane,
},
/// Creates a message used for a duplex anonymous communication where the recipient
/// will never learn of our true identity. This is achieved by carefully sending `reply_surbs`.
///
/// Note that if reply_surbs is set to zero then
/// this variant requires the client having sent some reply_surbs in the past
/// (and thus the recipient also knowing our sender tag).
///
/// Ends up with `NymMessage::Repliable` variant
Anonymous {
recipient: Recipient,
data: Vec<u8>,
reply_surbs: u32,
lane: TransmissionLane,
},
/// Attempt to use our internally received and stored `ReplySurb` to send the message back
/// to specified recipient whilst not knowing its full identity (or even gateway).
///
/// Ends up with `NymMessage::Reply` variant
Reply {
recipient_tag: AnonymousSenderTag,
data: Vec<u8>,
lane: TransmissionLane,
},
}
impl InputMessage {
pub fn new_regular(recipient: Recipient, data: Vec<u8>, lane: TransmissionLane) -> Self {
InputMessage::Regular {
recipient,
data,
lane,
}
}
pub fn new_anonymous(
recipient: Recipient,
data: Vec<u8>,
reply_surbs: u32,
lane: TransmissionLane,
) -> Self {
InputMessage::Anonymous {
recipient,
data,
reply_surbs,
lane,
}
}
pub fn new_reply(
recipient_tag: AnonymousSenderTag,
data: Vec<u8>,
lane: TransmissionLane,
) -> Self {
InputMessage::Reply {
recipient_tag,
data,
lane,
}
}
}
@@ -0,0 +1,227 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::config::persistence::key_pathfinder::ClientKeyPathfinder;
use crypto::asymmetric::{encryption, identity};
use gateway_requests::registration::handshake::SharedKeys;
use log::*;
use nymsphinx::acknowledgements::AckKey;
use rand::{CryptoRng, RngCore};
use std::io;
use std::sync::Arc;
// Note: to support key rotation in the future, all keys will require adding an extra smart pointer,
// most likely an AtomicCell, or if it doesn't work as I think it does, a Mutex. Although I think
// AtomicCell includes a Mutex implicitly if the underlying type does not work atomically.
// And I guess there will need to be some mechanism for a grace period when you can still
// use the old key after new one was issued.
// Remember that Arc<T> has Deref implementation for T
#[derive(Clone)]
pub struct KeyManager {
/// identity key associated with the client instance.
identity_keypair: Arc<identity::KeyPair>,
/// encryption key associated with the client instance.
encryption_keypair: Arc<encryption::KeyPair>,
/// shared key derived with the gateway during "registration handshake"
gateway_shared_key: Option<Arc<SharedKeys>>,
/// key used for producing and processing acknowledgement packets.
ack_key: Arc<AckKey>,
}
// The expected flow of a KeyManager "lifetime" is as follows:
/*
1. ::new() is called during client-init
2. after gateway registration is completed [in init] ::insert_gateway_shared_key() is called
3. ::store_keys() is called before init finishes execution.
4. ::load_keys() is called at the beginning of each subsequent client-run
5. [not implemented] ::rotate_keys() is called periodically during client-run I presume?
*/
impl KeyManager {
/// Creates new instance of a [`KeyManager`]
pub fn new<R>(rng: &mut R) -> Self
where
R: RngCore + CryptoRng,
{
KeyManager {
identity_keypair: Arc::new(identity::KeyPair::new(rng)),
encryption_keypair: Arc::new(encryption::KeyPair::new(rng)),
gateway_shared_key: None,
ack_key: Arc::new(AckKey::new(rng)),
}
}
pub fn from_keys(
id_keypair: identity::KeyPair,
enc_keypair: encryption::KeyPair,
gateway_shared_key: SharedKeys,
ack_key: AckKey,
) -> Self {
Self {
identity_keypair: Arc::new(id_keypair),
encryption_keypair: Arc::new(enc_keypair),
gateway_shared_key: Some(Arc::new(gateway_shared_key)),
ack_key: Arc::new(ack_key),
}
}
/// Loads previously stored client keys from the disk.
fn load_client_keys(client_pathfinder: &ClientKeyPathfinder) -> io::Result<Self> {
let identity_keypair: identity::KeyPair =
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
))?;
let encryption_keypair: encryption::KeyPair =
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
))?;
let ack_key: AckKey = pemstore::load_key(client_pathfinder.ack_key())?;
Ok(KeyManager {
identity_keypair: Arc::new(identity_keypair),
encryption_keypair: Arc::new(encryption_keypair),
gateway_shared_key: None,
ack_key: Arc::new(ack_key),
})
}
/// Loads previously stored keys from the disk. Fails if not all, including the shared gateway
/// key, is available.
pub fn load_keys(client_pathfinder: &ClientKeyPathfinder) -> io::Result<Self> {
let mut key_manager = Self::load_client_keys(client_pathfinder)?;
let gateway_shared_key: SharedKeys =
pemstore::load_key(client_pathfinder.gateway_shared_key())?;
key_manager.gateway_shared_key = Some(Arc::new(gateway_shared_key));
Ok(key_manager)
}
/// Loads previously stored keys from the disk. Fails if client keys are not availabe, but the
/// shared gateway key is optional.
pub fn load_keys_but_gateway_is_optional(
client_pathfinder: &ClientKeyPathfinder,
) -> io::Result<Self> {
let mut key_manager = Self::load_client_keys(client_pathfinder)?;
let gateway_shared_key: Result<SharedKeys, io::Error> =
pemstore::load_key(client_pathfinder.gateway_shared_key());
// It's ok if the gateway key was not found
let gateway_shared_key = match gateway_shared_key {
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(None),
Err(err) => Err(err),
Ok(key) => Ok(Some(key)),
}?;
key_manager.gateway_shared_key = gateway_shared_key.map(Arc::new);
Ok(key_manager)
}
/// Stores all available keys on the disk.
// While perhaps there is no much point in storing the `AckKey` on the disk,
// it is done so for the consistency sake so that you wouldn't require an rng instance
// during `load_keys` to generate the said key.
pub fn store_keys(&self, client_pathfinder: &ClientKeyPathfinder) -> io::Result<()> {
pemstore::store_keypair(
self.identity_keypair.as_ref(),
&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
),
)?;
pemstore::store_keypair(
self.encryption_keypair.as_ref(),
&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
),
)?;
pemstore::store_key(self.ack_key.as_ref(), client_pathfinder.ack_key())?;
match self.gateway_shared_key.as_ref() {
None => debug!("No gateway shared key available to store!"),
Some(gate_key) => {
pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
}
}
Ok(())
}
pub fn store_gateway_key(&self, client_pathfinder: &ClientKeyPathfinder) -> io::Result<()> {
match self.gateway_shared_key.as_ref() {
None => {
return Err(io::Error::new(
io::ErrorKind::Other,
"trying to store a non-existing key",
))
}
Some(gate_key) => {
pemstore::store_key(gate_key.as_ref(), client_pathfinder.gateway_shared_key())?
}
}
Ok(())
}
/// Overwrite the existing identity keypair
pub fn set_identity_keypair(&mut self, id_keypair: identity::KeyPair) {
self.identity_keypair = Arc::new(id_keypair);
}
/// Gets an atomically reference counted pointer to [`identity::KeyPair`].
pub fn identity_keypair(&self) -> Arc<identity::KeyPair> {
Arc::clone(&self.identity_keypair)
}
/// Overwrite the existing encryption keypair
pub fn set_encryption_keypair(&mut self, enc_keypair: encryption::KeyPair) {
self.encryption_keypair = Arc::new(enc_keypair);
}
/// Gets an atomically reference counted pointer to [`encryption::KeyPair`].
pub fn encryption_keypair(&self) -> Arc<encryption::KeyPair> {
Arc::clone(&self.encryption_keypair)
}
/// Overwrite the existing ack key
pub fn set_ack_key(&mut self, ack_key: AckKey) {
self.ack_key = Arc::new(ack_key);
}
/// Gets an atomically reference counted pointer to [`AckKey`].
pub fn ack_key(&self) -> Arc<AckKey> {
Arc::clone(&self.ack_key)
}
/// After shared key with the gateway is derived, puts its ownership to this instance of a [`KeyManager`].
pub fn insert_gateway_shared_key(&mut self, gateway_shared_key: Arc<SharedKeys>) {
self.gateway_shared_key = Some(gateway_shared_key)
}
/// Gets an atomically reference counted pointer to [`SharedKey`].
// since this function is not fully public, it is not expected to be used externally and
// hence it's up to us to ensure it's called in correct context
pub fn gateway_shared_key(&self) -> Arc<SharedKeys> {
Arc::clone(
self.gateway_shared_key
.as_ref()
.expect("tried to unwrap empty gateway key!"),
)
}
pub fn is_gateway_key_set(&self) -> bool {
self.gateway_shared_key.is_some()
}
}
@@ -2,16 +2,13 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::spawn_future; use crate::spawn_future;
use log::*;
use nym_gateway_client::GatewayClient;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_credential_storage::storage::Storage;
#[cfg(not(target_arch = "wasm32"))]
use nym_validator_client::nyxd::traits::DkgQueryClient;
#[cfg(target_arch = "wasm32")] #[cfg(target_arch = "wasm32")]
use nym_bandwidth_controller::wasm_mockups::DkgQueryClient; use gateway_client::wasm_mockups::CosmWasmClient;
use gateway_client::GatewayClient;
use log::*;
use nymsphinx::forwarding::packet::MixPacket;
#[cfg(not(target_arch = "wasm32"))]
use validator_client::nyxd::CosmWasmClient;
pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>; pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>;
pub type BatchMixMessageReceiver = tokio::sync::mpsc::Receiver<Vec<MixPacket>>; pub type BatchMixMessageReceiver = tokio::sync::mpsc::Receiver<Vec<MixPacket>>;
@@ -20,10 +17,10 @@ pub type BatchMixMessageReceiver = tokio::sync::mpsc::Receiver<Vec<MixPacket>>;
pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32; pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32;
const MAX_FAILURE_COUNT: usize = 100; const MAX_FAILURE_COUNT: usize = 100;
pub struct MixTrafficController<C, St: Storage> { pub struct MixTrafficController<C: Clone> {
// TODO: most likely to be replaced by some higher level construct as // TODO: most likely to be replaced by some higher level construct as
// later on gateway_client will need to be accessible by other entities // later on gateway_client will need to be accessible by other entities
gateway_client: GatewayClient<C, St>, gateway_client: GatewayClient<C>,
mix_rx: BatchMixMessageReceiver, mix_rx: BatchMixMessageReceiver,
// TODO: this is temporary work-around. // TODO: this is temporary work-around.
@@ -31,24 +28,22 @@ pub struct MixTrafficController<C, St: Storage> {
consecutive_gateway_failure_count: usize, consecutive_gateway_failure_count: usize,
} }
impl<C, St> MixTrafficController<C, St> impl<C> MixTrafficController<C>
where where
C: DkgQueryClient + Sync + Send + 'static, C: CosmWasmClient + Sync + Send + Clone + 'static,
St: Storage + 'static,
<St as Storage>::StorageError: Send + Sync + 'static,
{ {
pub fn new( pub fn new(
gateway_client: GatewayClient<C, St>, gateway_client: GatewayClient<C>,
) -> (MixTrafficController<C, St>, BatchMixMessageSender) { ) -> (MixTrafficController<C>, BatchMixMessageSender) {
let (message_sender, message_receiver) = let (sphinx_message_sender, sphinx_message_receiver) =
tokio::sync::mpsc::channel(MIX_MESSAGE_RECEIVER_BUFFER_SIZE); tokio::sync::mpsc::channel(MIX_MESSAGE_RECEIVER_BUFFER_SIZE);
( (
MixTrafficController { MixTrafficController {
gateway_client, gateway_client,
mix_rx: message_receiver, mix_rx: sphinx_message_receiver,
consecutive_gateway_failure_count: 0, consecutive_gateway_failure_count: 0,
}, },
message_sender, sphinx_message_sender,
) )
} }
@@ -81,7 +76,7 @@ where
} }
} }
pub fn start_with_shutdown(mut self, mut shutdown: nym_task::TaskClient) { pub fn start_with_shutdown(mut self, mut shutdown: task::TaskClient) {
spawn_future(async move { spawn_future(async move {
debug!("Started MixTrafficController with graceful shutdown support"); debug!("Started MixTrafficController with graceful shutdown support");
@@ -3,9 +3,9 @@
use super::action_controller::{AckActionSender, Action}; use super::action_controller::{AckActionSender, Action};
use futures::StreamExt; use futures::StreamExt;
use gateway_client::AcknowledgementReceiver;
use log::*; use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nymsphinx::{
use nym_sphinx::{
acknowledgements::{identifier::recover_identifier, AckKey}, acknowledgements::{identifier::recover_identifier, AckKey},
chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID}, chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID},
}; };
@@ -65,13 +65,13 @@ impl AcknowledgementListener {
} }
} }
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) { pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started AcknowledgementListener with graceful shutdown support"); debug!("Started AcknowledgementListener with graceful shutdown support");
while !shutdown.is_shutdown() { while !shutdown.is_shutdown() {
tokio::select! { tokio::select! {
acks = self.ack_receiver.next() => match acks { acks = self.ack_receiver.next() => match acks {
Some(acks) => {self.handle_ack_receiver_item(acks).await} Some(acks) => self.handle_ack_receiver_item(acks).await,
None => { None => {
log::trace!("AcknowledgementListener: Stopping since channel closed"); log::trace!("AcknowledgementListener: Stopping since channel closed");
break; break;
@@ -6,9 +6,9 @@ use crate::client::real_messages_control::acknowledgement_control::Retransmissio
use futures::channel::mpsc; use futures::channel::mpsc;
use futures::StreamExt; use futures::StreamExt;
use log::*; use log::*;
use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey}; use nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nym_sphinx::chunking::fragment::FragmentIdentifier; use nymsphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::Delay as SphinxDelay; use nymsphinx::Delay as SphinxDelay;
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
@@ -216,7 +216,7 @@ impl ActionController {
fn handle_expired_ack_timer( fn handle_expired_ack_timer(
&mut self, &mut self,
expired_ack: Expired<FragmentIdentifier>, expired_ack: Expired<FragmentIdentifier>,
task_client: &mut nym_task::TaskClient, task_client: &mut task::TaskClient,
) { ) {
// I'm honestly not sure how to handle it, because getting it means other things in our // I'm honestly not sure how to handle it, because getting it means other things in our
// system are already misbehaving. If we ever see this panic, then I guess we should worry // system are already misbehaving. If we ever see this panic, then I guess we should worry
@@ -260,7 +260,7 @@ impl ActionController {
} }
} }
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) { pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started ActionController with graceful shutdown support"); debug!("Started ActionController with graceful shutdown support");
while !shutdown.is_shutdown() { while !shutdown.is_shutdown() {
@@ -3,14 +3,11 @@
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver}; use crate::client::inbound_messages::{InputMessage, InputMessageReceiver};
use crate::client::real_messages_control::message_handler::MessageHandler; use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender; use crate::client::replies::reply_controller::ReplyControllerSender;
use client_connections::TransmissionLane;
use log::*; use log::*;
use nym_sphinx::addressing::clients::Recipient; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag; use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::params::PacketType;
use nym_task::connections::TransmissionLane;
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements, /// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
@@ -44,18 +41,6 @@ where
} }
} }
async fn handle_premade_packets(&mut self, packets: Vec<MixPacket>, lane: TransmissionLane) {
self.message_handler
.send_premade_mix_packets(
packets
.into_iter()
.map(|p| RealMessage::new(p, None))
.collect(),
lane,
)
.await
}
async fn handle_reply( async fn handle_reply(
&mut self, &mut self,
recipient_tag: AnonymousSenderTag, recipient_tag: AnonymousSenderTag,
@@ -72,11 +57,10 @@ where
recipient: Recipient, recipient: Recipient,
content: Vec<u8>, content: Vec<u8>,
lane: TransmissionLane, lane: TransmissionLane,
packet_type: PacketType,
) { ) {
if let Err(err) = self if let Err(err) = self
.message_handler .message_handler
.try_send_plain_message(recipient, content, lane, packet_type) .try_send_plain_message(recipient, content, lane)
.await .await
{ {
warn!("failed to send a plain message - {err}") warn!("failed to send a plain message - {err}")
@@ -89,11 +73,10 @@ where
content: Vec<u8>, content: Vec<u8>,
reply_surbs: u32, reply_surbs: u32,
lane: TransmissionLane, lane: TransmissionLane,
packet_type: PacketType,
) { ) {
if let Err(err) = self if let Err(err) = self
.message_handler .message_handler
.try_send_message_with_reply_surbs(recipient, content, reply_surbs, lane, packet_type) .try_send_message_with_reply_surbs(recipient, content, reply_surbs, lane)
.await .await
{ {
warn!("failed to send a repliable message - {err}") warn!("failed to send a repliable message - {err}")
@@ -106,17 +89,14 @@ where
recipient, recipient,
data, data,
lane, lane,
} => { } => self.handle_plain_message(recipient, data, lane).await,
self.handle_plain_message(recipient, data, lane, PacketType::Mix)
.await
}
InputMessage::Anonymous { InputMessage::Anonymous {
recipient, recipient,
data, data,
reply_surbs, reply_surbs,
lane, lane,
} => { } => {
self.handle_repliable_message(recipient, data, reply_surbs, lane, PacketType::Mix) self.handle_repliable_message(recipient, data, reply_surbs, lane)
.await .await
} }
InputMessage::Reply { InputMessage::Reply {
@@ -126,45 +106,10 @@ where
} => { } => {
self.handle_reply(recipient_tag, data, lane).await; self.handle_reply(recipient_tag, data, lane).await;
} }
InputMessage::Premade { msgs, lane } => self.handle_premade_packets(msgs, lane).await,
InputMessage::MessageWrapper {
message,
packet_type,
} => match *message {
InputMessage::Regular {
recipient,
data,
lane,
} => {
self.handle_plain_message(recipient, data, lane, packet_type)
.await
}
InputMessage::Anonymous {
recipient,
data,
reply_surbs,
lane,
} => {
self.handle_repliable_message(recipient, data, reply_surbs, lane, packet_type)
.await
}
InputMessage::Reply {
recipient_tag,
data,
lane,
} => {
self.handle_reply(recipient_tag, data, lane).await;
}
InputMessage::Premade { msgs, lane } => {
self.handle_premade_packets(msgs, lane).await
}
// MessageWrappers can't be nested
InputMessage::MessageWrapper { .. } => unimplemented!(),
},
}; };
} }
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) { pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started InputMessageListener with graceful shutdown support"); debug!("Started InputMessageListener with graceful shutdown support");
while !shutdown.is_shutdown() { while !shutdown.is_shutdown() {
@@ -13,11 +13,11 @@ use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future; use crate::spawn_future;
use action_controller::AckActionReceiver; use action_controller::AckActionReceiver;
use futures::channel::mpsc; use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*; use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nymsphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag; use nymsphinx::params::PacketSize;
use nym_sphinx::params::{PacketSize, PacketType}; use nymsphinx::{
use nym_sphinx::{
acknowledgements::AckKey, acknowledgements::AckKey,
addressing::clients::Recipient, addressing::clients::Recipient,
chunking::fragment::{Fragment, FragmentIdentifier}, chunking::fragment::{Fragment, FragmentIdentifier},
@@ -249,11 +249,7 @@ where
} }
} }
pub(super) fn start_with_shutdown( pub(super) fn start_with_shutdown(self, shutdown: task::TaskClient) {
self,
shutdown: nym_task::TaskClient,
packet_type: PacketType,
) {
let mut acknowledgement_listener = self.acknowledgement_listener; let mut acknowledgement_listener = self.acknowledgement_listener;
let mut input_message_listener = self.input_message_listener; let mut input_message_listener = self.input_message_listener;
let mut retransmission_request_listener = self.retransmission_request_listener; let mut retransmission_request_listener = self.retransmission_request_listener;
@@ -279,7 +275,7 @@ where
let shutdown_handle = shutdown.clone(); let shutdown_handle = shutdown.clone();
spawn_future(async move { spawn_future(async move {
retransmission_request_listener retransmission_request_listener
.run_with_shutdown(shutdown_handle, packet_type) .run_with_shutdown(shutdown_handle)
.await; .await;
debug!("The retransmission request listener has finished execution!"); debug!("The retransmission request listener has finished execution!");
}); });
@@ -9,12 +9,12 @@ use crate::client::real_messages_control::acknowledgement_control::PacketDestina
use crate::client::real_messages_control::message_handler::{MessageHandler, PreparationError}; use crate::client::real_messages_control::message_handler::{MessageHandler, PreparationError};
use crate::client::real_messages_control::real_traffic_stream::RealMessage; use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender; use crate::client::replies::reply_controller::ReplyControllerSender;
use client_connections::TransmissionLane;
use futures::StreamExt; use futures::StreamExt;
use log::*; use log::*;
use nym_sphinx::chunking::fragment::Fragment; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::preparer::PreparedFragment; use nymsphinx::chunking::fragment::Fragment;
use nym_sphinx::{addressing::clients::Recipient, params::PacketType}; use nymsphinx::preparer::PreparedFragment;
use nym_task::connections::TransmissionLane;
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak}; use std::sync::{Arc, Weak};
@@ -48,20 +48,17 @@ where
&mut self, &mut self,
packet_recipient: Recipient, packet_recipient: Recipient,
chunk_data: Fragment, chunk_data: Fragment,
packet_type: PacketType,
) -> Result<PreparedFragment, PreparationError> { ) -> Result<PreparedFragment, PreparationError> {
debug!("retransmitting normal packet..."); debug!("retransmitting normal packet...");
// TODO: Figure out retransmission packet type signaling
self.message_handler self.message_handler
.try_prepare_single_chunk_for_sending(packet_recipient, chunk_data, packet_type) .try_prepare_single_chunk_for_sending(packet_recipient, chunk_data)
.await .await
} }
async fn on_retransmission_request( async fn on_retransmission_request(
&mut self, &mut self,
weak_timed_out_ack: Weak<PendingAcknowledgement>, weak_timed_out_ack: Weak<PendingAcknowledgement>,
packet_type: PacketType,
) { ) {
let timed_out_ack = match weak_timed_out_ack.upgrade() { let timed_out_ack = match weak_timed_out_ack.upgrade() {
Some(timed_out_ack) => timed_out_ack, Some(timed_out_ack) => timed_out_ack,
@@ -88,7 +85,6 @@ where
self.prepare_normal_retransmission_chunk( self.prepare_normal_retransmission_chunk(
**recipient, **recipient,
timed_out_ack.message_chunk.clone(), timed_out_ack.message_chunk.clone(),
packet_type,
) )
.await .await
} }
@@ -135,26 +131,19 @@ where
// send to `OutQueueControl` to eventually send to the mix network // send to `OutQueueControl` to eventually send to the mix network
self.message_handler self.message_handler
.forward_messages( .forward_messages(
vec![RealMessage::new( vec![RealMessage::new(prepared_fragment.mix_packet, frag_id)],
prepared_fragment.mix_packet,
Some(frag_id),
)],
TransmissionLane::Retransmission, TransmissionLane::Retransmission,
) )
.await .await
} }
pub(super) async fn run_with_shutdown( pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
&mut self,
mut shutdown: nym_task::TaskClient,
packet_type: PacketType,
) {
debug!("Started RetransmissionRequestListener with graceful shutdown support"); debug!("Started RetransmissionRequestListener with graceful shutdown support");
while !shutdown.is_shutdown() { while !shutdown.is_shutdown() {
tokio::select! { tokio::select! {
timed_out_ack = self.request_receiver.next() => match timed_out_ack { timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await, Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack).await,
None => { None => {
log::trace!("RetransmissionRequestListener: Stopping since channel closed"); log::trace!("RetransmissionRequestListener: Stopping since channel closed");
break; break;
@@ -5,7 +5,7 @@ use super::action_controller::{AckActionSender, Action};
use super::SentPacketNotificationReceiver; use super::SentPacketNotificationReceiver;
use futures::StreamExt; use futures::StreamExt;
use log::*; use log::*;
use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID}; use nymsphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
/// Module responsible for starting up retransmission timers. /// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled /// It is required because when we send our packet to the `real traffic stream` controlled
@@ -37,7 +37,7 @@ impl SentNotificationListener {
.unwrap(); .unwrap();
} }
pub(super) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) { pub(super) async fn run_with_shutdown(&mut self, mut shutdown: task::TaskClient) {
debug!("Started SentNotificationListener with graceful shutdown support"); debug!("Started SentNotificationListener with graceful shutdown support");
while !shutdown.is_shutdown() { while !shutdown.is_shutdown() {
@@ -8,26 +8,26 @@ use crate::client::real_messages_control::real_traffic_stream::{
use crate::client::real_messages_control::{AckActionSender, Action}; use crate::client::real_messages_control::{AckActionSender, Action};
use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags}; use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags};
use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit}; use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit};
use client_connections::TransmissionLane;
use log::{debug, error, info, trace, warn}; use log::{debug, error, info, trace, warn};
use nym_sphinx::acknowledgements::AckKey; use nymsphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage}; use nymsphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey}; use nymsphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier}; use nymsphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_sphinx::message::NymMessage; use nymsphinx::message::NymMessage;
use nym_sphinx::params::{PacketSize, PacketType, DEFAULT_NUM_MIX_HOPS}; use nymsphinx::params::{PacketSize, DEFAULT_NUM_MIX_HOPS};
use nym_sphinx::preparer::{MessagePreparer, PreparedFragment}; use nymsphinx::preparer::{MessagePreparer, PreparedFragment};
use nym_sphinx::Delay; use nymsphinx::Delay;
use nym_task::connections::TransmissionLane;
use nym_topology::{NymTopology, NymTopologyError};
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use thiserror::Error; use thiserror::Error;
use topology::{NymTopology, NymTopologyError};
// TODO: move that error elsewhere since it seems to be contaminating different files // TODO: move that error elsewhere since it seems to be contaminating different files
#[derive(Debug, Error)] #[derive(Debug, Clone, Error)]
pub enum PreparationError { pub enum PreparationError {
#[error(transparent)] #[error(transparent)]
NymTopologyError(#[from] NymTopologyError), NymTopologyError(#[from] NymTopologyError),
@@ -101,11 +101,8 @@ pub(crate) struct Config {
/// Note that it does not include gateway hops. /// Note that it does not include gateway hops.
num_mix_hops: u8, num_mix_hops: u8,
/// Primary predefined packet size used for the encapsulated messages. /// Predefined packet size used for the encapsulated messages.
primary_packet_size: PacketSize, packet_size: PacketSize,
/// Optional secondary predefined packet size used for the encapsulated messages.
secondary_packet_size: Option<PacketSize>,
} }
impl Config { impl Config {
@@ -121,8 +118,7 @@ impl Config {
average_packet_delay, average_packet_delay,
average_ack_delay, average_ack_delay,
num_mix_hops: DEFAULT_NUM_MIX_HOPS, num_mix_hops: DEFAULT_NUM_MIX_HOPS,
primary_packet_size: PacketSize::default(), packet_size: PacketSize::default(),
secondary_packet_size: None,
} }
} }
@@ -134,14 +130,8 @@ impl Config {
} }
/// Allows setting non-default size of the sphinx packets sent out. /// Allows setting non-default size of the sphinx packets sent out.
pub fn with_custom_primary_packet_size(mut self, packet_size: PacketSize) -> Self { pub fn with_custom_packet_size(mut self, packet_size: PacketSize) -> Self {
self.primary_packet_size = packet_size; self.packet_size = packet_size;
self
}
/// Allows setting non-default size of the sphinx packets sent out.
pub fn with_custom_secondary_packet_size(mut self, packet_size: Option<PacketSize>) -> Self {
self.secondary_packet_size = packet_size;
self self
} }
} }
@@ -180,6 +170,7 @@ where
config.average_packet_delay, config.average_packet_delay,
config.average_ack_delay, config.average_ack_delay,
) )
.with_custom_real_message_packet_size(config.packet_size)
.with_mix_hops(config.num_mix_hops); .with_mix_hops(config.num_mix_hops);
MessageHandler { MessageHandler {
@@ -220,28 +211,6 @@ where
} }
} }
fn optimal_packet_size(&self, msg: &NymMessage) -> PacketSize {
// if secondary packet was never set, then it's obvious we have to use the primary packet
let Some(secondary_packet) = self.config.secondary_packet_size else {
trace!("only primary packet size is available");
return self.config.primary_packet_size
};
let primary_count =
msg.required_packets(self.config.primary_packet_size, self.config.num_mix_hops);
let secondary_count = msg.required_packets(secondary_packet, self.config.num_mix_hops);
trace!("This message would require: {primary_count} primary packets or {secondary_count} secondary packets...");
// if there would be no benefit in using the secondary packet - use the primary (duh)
if primary_count <= secondary_count {
trace!("so choosing primary for this message");
self.config.primary_packet_size
} else {
trace!("so choosing secondary for this message");
secondary_packet
}
}
async fn generate_reply_surbs_with_keys( async fn generate_reply_surbs_with_keys(
&mut self, &mut self,
amount: usize, amount: usize,
@@ -268,13 +237,9 @@ where
reply_surb: ReplySurb, reply_surb: ReplySurb,
is_extra_surb_request: bool, is_extra_surb_request: bool,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
let msg = NymMessage::new_reply(message);
let packet_size = self.optimal_packet_size(&msg);
debug!("Using {packet_size} packets for {msg}");
let mut fragment = self let mut fragment = self
.message_preparer .message_preparer
.pad_and_split_message(msg, packet_size); .pad_and_split_message(NymMessage::new_reply(message));
if fragment.len() > 1 { if fragment.len() > 1 {
// well, it's not a single surb message // well, it's not a single surb message
return Err(SurbWrappedPreparationError { return Err(SurbWrappedPreparationError {
@@ -291,10 +256,8 @@ where
.try_prepare_single_reply_chunk_for_sending(reply_surb, chunk_clone) .try_prepare_single_reply_chunk_for_sending(reply_surb, chunk_clone)
.await?; .await?;
let real_messages = RealMessage::new( let real_messages =
prepared_fragment.mix_packet, RealMessage::new(prepared_fragment.mix_packet, chunk.fragment_identifier());
Some(chunk.fragment_identifier()),
);
let delay = prepared_fragment.total_delay; let delay = prepared_fragment.total_delay;
let pending_ack = let pending_ack =
PendingAcknowledgement::new_anonymous(chunk, delay, target, is_extra_surb_request); PendingAcknowledgement::new_anonymous(chunk, delay, target, is_extra_surb_request);
@@ -326,12 +289,10 @@ where
// // TODO: this will require additional argument to make it use different variant of `ReplyMessage` // // TODO: this will require additional argument to make it use different variant of `ReplyMessage`
pub(crate) fn split_reply_message(&mut self, message: Vec<u8>) -> Vec<Fragment> { pub(crate) fn split_reply_message(&mut self, message: Vec<u8>) -> Vec<Fragment> {
let msg = NymMessage::new_reply(ReplyMessage::new_data_message(message));
let packet_size = self.optimal_packet_size(&msg);
debug!("Using {packet_size} packets for {msg}");
self.message_preparer self.message_preparer
.pad_and_split_message(msg, packet_size) .pad_and_split_message(NymMessage::new_reply(ReplyMessage::new_data_message(
message,
)))
} }
pub(crate) async fn send_retransmission_reply_chunks( pub(crate) async fn send_retransmission_reply_chunks(
@@ -386,8 +347,7 @@ where
let lane = raw.0; let lane = raw.0;
let fragment = raw.1; let fragment = raw.1;
let real_message = let real_message = RealMessage::new(prepared.mix_packet, prepared.fragment_identifier);
RealMessage::new(prepared.mix_packet, Some(prepared.fragment_identifier));
let delay = prepared.total_delay; let delay = prepared.total_delay;
let pending_ack = PendingAcknowledgement::new_anonymous(fragment, delay, target, false); let pending_ack = PendingAcknowledgement::new_anonymous(fragment, delay, target, false);
@@ -404,23 +364,14 @@ where
Ok(()) Ok(())
} }
pub(crate) async fn send_premade_mix_packets(
&mut self,
msgs: Vec<RealMessage>,
lane: TransmissionLane,
) {
self.forward_messages(msgs, lane).await;
}
pub(crate) async fn try_send_plain_message( pub(crate) async fn try_send_plain_message(
&mut self, &mut self,
recipient: Recipient, recipient: Recipient,
message: Vec<u8>, message: Vec<u8>,
lane: TransmissionLane, lane: TransmissionLane,
packet_type: PacketType,
) -> Result<(), PreparationError> { ) -> Result<(), PreparationError> {
let message = NymMessage::new_plain(message); let message = NymMessage::new_plain(message);
self.try_split_and_send_non_reply_message(message, recipient, lane, packet_type) self.try_split_and_send_non_reply_message(message, recipient, lane)
.await .await
} }
@@ -429,9 +380,7 @@ where
message: NymMessage, message: NymMessage,
recipient: Recipient, recipient: Recipient,
lane: TransmissionLane, lane: TransmissionLane,
packet_type: PacketType,
) -> Result<(), PreparationError> { ) -> Result<(), PreparationError> {
debug!("Sending non-reply message with packet type {packet_type}");
// TODO: I really dislike existence of this assertion, it implies code has to be re-organised // TODO: I really dislike existence of this assertion, it implies code has to be re-organised
debug_assert!(!matches!(message, NymMessage::Reply(_))); debug_assert!(!matches!(message, NymMessage::Reply(_)));
@@ -439,15 +388,7 @@ where
let topology_permit = self.topology_access.get_read_permit().await; let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?; let topology = self.get_topology(&topology_permit)?;
let packet_size = if packet_type == PacketType::Outfox { let fragments = self.message_preparer.pad_and_split_message(message);
PacketSize::OutfoxRegularPacket
} else {
self.optimal_packet_size(&message)
};
debug!("Using {packet_size} packets for {message}");
let fragments = self
.message_preparer
.pad_and_split_message(message, packet_size);
let mut pending_acks = Vec::with_capacity(fragments.len()); let mut pending_acks = Vec::with_capacity(fragments.len());
let mut real_messages = Vec::with_capacity(fragments.len()); let mut real_messages = Vec::with_capacity(fragments.len());
@@ -460,13 +401,10 @@ where
topology, topology,
&self.config.ack_key, &self.config.ack_key,
&recipient, &recipient,
packet_type,
)?; )?;
let real_message = RealMessage::new( let real_message =
prepared_fragment.mix_packet, RealMessage::new(prepared_fragment.mix_packet, fragment.fragment_identifier());
Some(fragment.fragment_identifier()),
);
let delay = prepared_fragment.total_delay; let delay = prepared_fragment.total_delay;
let pending_ack = PendingAcknowledgement::new_known(fragment, delay, recipient); let pending_ack = PendingAcknowledgement::new_known(fragment, delay, recipient);
@@ -484,9 +422,7 @@ where
&mut self, &mut self,
recipient: Recipient, recipient: Recipient,
amount: u32, amount: u32,
packet_type: PacketType,
) -> Result<(), PreparationError> { ) -> Result<(), PreparationError> {
debug!("Sending additional reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient); let sender_tag = self.get_or_create_sender_tag(&recipient);
let (reply_surbs, reply_keys) = let (reply_surbs, reply_keys) =
self.generate_reply_surbs_with_keys(amount as usize).await?; self.generate_reply_surbs_with_keys(amount as usize).await?;
@@ -500,7 +436,6 @@ where
message, message,
recipient, recipient,
TransmissionLane::AdditionalReplySurbs, TransmissionLane::AdditionalReplySurbs,
packet_type,
) )
.await?; .await?;
@@ -516,9 +451,7 @@ where
message: Vec<u8>, message: Vec<u8>,
num_reply_surbs: u32, num_reply_surbs: u32,
lane: TransmissionLane, lane: TransmissionLane,
packet_type: PacketType,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
debug!("Sending message with reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient); let sender_tag = self.get_or_create_sender_tag(&recipient);
let (reply_surbs, reply_keys) = self let (reply_surbs, reply_keys) = self
.generate_reply_surbs_with_keys(num_reply_surbs as usize) .generate_reply_surbs_with_keys(num_reply_surbs as usize)
@@ -527,7 +460,7 @@ where
let message = let message =
NymMessage::new_repliable(RepliableMessage::new_data(message, sender_tag, reply_surbs)); NymMessage::new_repliable(RepliableMessage::new_data(message, sender_tag, reply_surbs));
self.try_split_and_send_non_reply_message(message, recipient, lane, packet_type) self.try_split_and_send_non_reply_message(message, recipient, lane)
.await?; .await?;
log::trace!("storing {} reply keys", reply_keys.len()); log::trace!("storing {} reply keys", reply_keys.len());
@@ -540,21 +473,13 @@ where
&mut self, &mut self,
recipient: Recipient, recipient: Recipient,
chunk: Fragment, chunk: Fragment,
packet_type: PacketType,
) -> Result<PreparedFragment, PreparationError> { ) -> Result<PreparedFragment, PreparationError> {
debug!("Sending single chunk with packet type {packet_type}");
let topology_permit = self.topology_access.get_read_permit().await; let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?; let topology = self.get_topology(&topology_permit)?;
let prepared_fragment = self let prepared_fragment = self
.message_preparer .message_preparer
.prepare_chunk_for_sending( .prepare_chunk_for_sending(chunk, topology, &self.config.ack_key, &recipient)
chunk,
topology,
&self.config.ack_key,
&recipient,
packet_type,
)
.unwrap(); .unwrap();
Ok(prepared_fragment) Ok(prepared_fragment)
@@ -565,7 +490,7 @@ where
fragments: Vec<Fragment>, fragments: Vec<Fragment>,
reply_surbs: Vec<ReplySurb>, reply_surbs: Vec<ReplySurb>,
) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> { ) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> {
debug_assert_eq!( debug_assert_ne!(
fragments.len(), fragments.len(),
reply_surbs.len(), reply_surbs.len(),
"attempted to send {} fragments with {} reply surbs", "attempted to send {} fragments with {} reply surbs",
@@ -590,7 +515,6 @@ where
topology, topology,
&self.config.ack_key, &self.config.ack_key,
reply_surb, reply_surb,
PacketType::Mix,
) )
.unwrap() .unwrap()
}) })
@@ -610,13 +534,7 @@ where
let prepared_fragment = self let prepared_fragment = self
.message_preparer .message_preparer
.prepare_reply_chunk_for_sending( .prepare_reply_chunk_for_sending(chunk, topology, &self.config.ack_key, reply_surb)
chunk,
topology,
&self.config.ack_key,
reply_surb,
PacketType::Mix,
)
.unwrap(); .unwrap();
Ok(prepared_fragment) Ok(prepared_fragment)
@@ -1,4 +1,4 @@
// Copyright 2021-2023 - Nym Technologies SA <contact@nymtech.net> // Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
// INPUT: InputMessage from user // INPUT: InputMessage from user
@@ -21,15 +21,16 @@ use crate::{
}, },
spawn_future, spawn_future,
}; };
use client_connections::{ConnectionCommandReceiver, LaneQueueLengths};
use futures::channel::mpsc; use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*; use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nymsphinx::acknowledgements::AckKey;
use nym_sphinx::acknowledgements::AckKey; use nymsphinx::addressing::clients::Recipient;
use nym_sphinx::addressing::clients::Recipient; use nymsphinx::params::PacketSize;
use nym_sphinx::params::PacketType;
use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths};
use rand::{rngs::OsRng, CryptoRng, Rng}; use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration;
use crate::client::replies::reply_controller; use crate::client::replies::reply_controller;
use crate::config; use crate::config;
@@ -44,29 +45,61 @@ pub struct Config {
/// Key used to decrypt contents of received SURBAcks /// Key used to decrypt contents of received SURBAcks
ack_key: Arc<AckKey>, ack_key: Arc<AckKey>,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the additive part `b`
ack_wait_addition: Duration,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the multiplier `a`
ack_wait_multiplier: f64,
/// Address of `this` client. /// Address of `this` client.
self_recipient: Recipient, self_recipient: Recipient,
/// Specifies all traffic related configuration options. /// Average delay between sending subsequent packets from this client.
traffic: config::Traffic, average_message_sending_delay: Duration,
/// Specifies all cover traffic related configuration options. /// Average delay a data packet is going to get delayed at a single mixnode.
cover_traffic: config::CoverTraffic, average_packet_delay_duration: Duration,
/// Specifies all acknowledgements related configuration options. /// Average delay an acknowledgement packet is going to get delayed at a single mixnode.
acks: config::Acknowledgements, average_ack_delay_duration: Duration,
/// Specifies all reply SURBs related configuration options. /// Controls whether the main packet stream constantly produces packets according to the predefined
reply_surbs: config::ReplySurbs, /// poisson distribution.
disable_main_poisson_packet_distribution: bool,
/// Predefined packet size used for the encapsulated messages.
packet_size: PacketSize,
/// Defines the minimum number of reply surbs the client would request.
minimum_reply_surb_request_size: u32,
/// Defines the maximum number of reply surbs the client would request.
maximum_reply_surb_request_size: u32,
/// Defines the maximum number of reply surbs a remote party is allowed to request from this client at once.
maximum_allowed_reply_surb_request_size: u32,
/// Defines maximum amount of time the client is going to wait for reply surbs before explicitly asking
/// for more even though in theory they wouldn't need to.
maximum_reply_surb_rerequest_waiting_period: Duration,
/// Defines maximum amount of time the client is going to wait for reply surbs before
/// deciding it's never going to get them and would drop all pending messages
maximum_reply_surb_drop_waiting_period: Duration,
/// Defines maximum amount of time given reply surb is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
maximum_reply_surb_age: Duration,
/// Defines maximum amount of time given reply key is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
maximum_reply_key_age: Duration,
} }
impl<'a> From<&'a Config> for acknowledgement_control::Config { impl<'a> From<&'a Config> for acknowledgement_control::Config {
fn from(cfg: &'a Config) -> Self { fn from(cfg: &'a Config) -> Self {
acknowledgement_control::Config::new( acknowledgement_control::Config::new(cfg.ack_wait_addition, cfg.ack_wait_multiplier)
cfg.acks.ack_wait_addition, .with_custom_packet_size(cfg.packet_size)
cfg.acks.ack_wait_multiplier,
)
.with_custom_packet_size(cfg.traffic.primary_packet_size)
} }
} }
@@ -75,16 +108,26 @@ impl<'a> From<&'a Config> for real_traffic_stream::Config {
real_traffic_stream::Config::new( real_traffic_stream::Config::new(
Arc::clone(&cfg.ack_key), Arc::clone(&cfg.ack_key),
cfg.self_recipient, cfg.self_recipient,
cfg.acks.average_ack_delay, cfg.average_ack_delay_duration,
cfg.traffic, cfg.average_packet_delay_duration,
cfg.cover_traffic.cover_traffic_primary_size_ratio, cfg.average_message_sending_delay,
cfg.disable_main_poisson_packet_distribution,
) )
.with_custom_cover_packet_size(cfg.packet_size)
} }
} }
impl<'a> From<&'a Config> for reply_controller::Config { impl<'a> From<&'a Config> for reply_controller::Config {
fn from(cfg: &'a Config) -> Self { fn from(cfg: &'a Config) -> Self {
reply_controller::Config::new(cfg.reply_surbs) reply_controller::Config::new(
cfg.minimum_reply_surb_request_size,
cfg.maximum_reply_surb_request_size,
cfg.maximum_allowed_reply_surb_request_size,
cfg.maximum_reply_surb_rerequest_waiting_period,
cfg.maximum_reply_surb_drop_waiting_period,
cfg.maximum_reply_surb_age,
cfg.maximum_reply_key_age,
)
} }
} }
@@ -93,11 +136,10 @@ impl<'a> From<&'a Config> for message_handler::Config {
message_handler::Config::new( message_handler::Config::new(
Arc::clone(&cfg.ack_key), Arc::clone(&cfg.ack_key),
cfg.self_recipient, cfg.self_recipient,
cfg.traffic.average_packet_delay, cfg.average_packet_delay_duration,
cfg.acks.average_ack_delay, cfg.average_ack_delay_duration,
) )
.with_custom_primary_packet_size(cfg.traffic.primary_packet_size) .with_custom_packet_size(cfg.packet_size)
.with_custom_secondary_packet_size(cfg.traffic.secondary_packet_size)
} }
} }
@@ -110,12 +152,32 @@ impl Config {
Config { Config {
ack_key, ack_key,
self_recipient, self_recipient,
traffic: base_client_debug_config.traffic, packet_size: Default::default(),
cover_traffic: base_client_debug_config.cover_traffic, ack_wait_addition: base_client_debug_config.ack_wait_addition,
acks: base_client_debug_config.acknowledgements, ack_wait_multiplier: base_client_debug_config.ack_wait_multiplier,
reply_surbs: base_client_debug_config.reply_surbs, average_message_sending_delay: base_client_debug_config.message_sending_average_delay,
average_packet_delay_duration: base_client_debug_config.average_packet_delay,
average_ack_delay_duration: base_client_debug_config.average_ack_delay,
disable_main_poisson_packet_distribution: base_client_debug_config
.disable_main_poisson_packet_distribution,
minimum_reply_surb_request_size: base_client_debug_config
.minimum_reply_surb_request_size,
maximum_reply_surb_request_size: base_client_debug_config
.maximum_reply_surb_request_size,
maximum_allowed_reply_surb_request_size: base_client_debug_config
.maximum_allowed_reply_surb_request_size,
maximum_reply_surb_rerequest_waiting_period: base_client_debug_config
.maximum_reply_surb_rerequest_waiting_period,
maximum_reply_surb_drop_waiting_period: base_client_debug_config
.maximum_reply_surb_drop_waiting_period,
maximum_reply_surb_age: base_client_debug_config.maximum_reply_surb_age,
maximum_reply_key_age: base_client_debug_config.maximum_reply_key_age,
} }
} }
pub fn set_custom_packet_size(&mut self, packet_size: PacketSize) {
self.packet_size = packet_size;
}
} }
pub(crate) struct RealMessagesController<R> pub(crate) struct RealMessagesController<R>
@@ -208,7 +270,7 @@ impl RealMessagesController<OsRng> {
} }
} }
pub fn start_with_shutdown(self, shutdown: nym_task::TaskClient, packet_type: PacketType) { pub fn start_with_shutdown(self, shutdown: task::TaskClient) {
let mut out_queue_control = self.out_queue_control; let mut out_queue_control = self.out_queue_control;
let ack_control = self.ack_control; let ack_control = self.ack_control;
let mut reply_control = self.reply_control; let mut reply_control = self.reply_control;
@@ -224,6 +286,6 @@ impl RealMessagesController<OsRng> {
debug!("The reply controller has finished execution!"); debug!("The reply controller has finished execution!");
}); });
ack_control.start_with_shutdown(shutdown, packet_type); ack_control.start_with_shutdown(shutdown);
} }
} }

Some files were not shown because too many files have changed in this diff Show More