Add Playwright CI tests for mix-fetch internal-dev harness

Per-PR smoke test (ci-sdk-wasm.yml): loads WASM runtimes and inits MixFetch on Chromium, Firefox, and WebKit. Nightly stress test (nightly-mix-fetch-stress.yml): connects to mainnet via random gateway, runs 10 mixed-size fetches, asserts >= 80% pass.
2026-04-01 18:02:18 +01:00
parent f3c6faee83
commit 82598fcd87
8 changed files with 1509 additions and 0 deletions
@@ -54,6 +54,30 @@ jobs:
      - name: "Build"
        run: make sdk-wasm-build

+      - name: "Build mix-fetch WASM (debug)"
+        run: |
+          make -C wasm/mix-fetch/go-mix-conn build-debug-dev
+          make -C wasm/mix-fetch build-rust-debug
+
+      - name: "Build mix-fetch internal-dev harness"
+        working-directory: wasm/mix-fetch/internal-dev
+        run: npm install && npm run build
+
+      - name: "Install Playwright browsers"
+        working-directory: wasm/mix-fetch/tests
+        run: npm install && npx playwright install --with-deps # --with-deps assumes Ubuntu/Debian, see note in wasm/mix-fetch/tests/README.md
+
+      - name: "Smoke-test mix-fetch internal-dev (headless)"
+        working-directory: wasm/mix-fetch/tests
+        run: npm run test:smoke
+
+      - name: Upload Playwright traces on failure
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: mix-fetch-playwright-traces
+          path: wasm/mix-fetch/tests/test-results/
+
      - name: "Test"
        run: make sdk-wasm-test

@@ -0,0 +1,61 @@
+name: nightly-mix-fetch-stress
+
+on:
+  schedule:
+    - cron: '0 3 * * *'
+  workflow_dispatch:
+
+jobs:
+  stress:
+    runs-on: arc-linux-latest
+    env:
+      CARGO_TERM_COLOR: always
+      RUSTUP_PERMIT_COPY_RENAME: 1
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - uses: actions-rs/toolchain@v1
+        with:
+          profile: minimal
+          toolchain: ${{ vars.REQUIRED_RUSTC_VERSION }}
+          target: wasm32-unknown-unknown
+          override: true
+
+      - name: Set up Go
+        uses: actions/setup-go@v6
+        with:
+          go-version: "1.24.6"
+
+      - name: Install wasm-pack
+        run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+
+      - name: Install wasm-bindgen-cli
+        run: cargo install wasm-bindgen-cli
+
+      - name: "Build mix-fetch WASM (debug)"
+        run: |
+          make -C wasm/mix-fetch/go-mix-conn build-debug-dev
+          make -C wasm/mix-fetch build-rust-debug
+
+      - name: "Build internal-dev harness"
+        working-directory: wasm/mix-fetch/internal-dev
+        run: npm install && npm run build
+
+      - name: "Install Playwright browsers"
+        working-directory: wasm/mix-fetch/tests
+        run: npm install && npx playwright install --with-deps
+
+      - name: "Stress-test mix-fetch through mainnet"
+        working-directory: wasm/mix-fetch/tests
+        run: npm run test:stress
+
+      - name: Upload Playwright traces on failure
+        if: failure()
+        uses: actions/upload-artifact@v4
+        with:
+          name: mix-fetch-stress-traces
+          path: wasm/mix-fetch/tests/test-results/
@@ -0,0 +1,81 @@
+# mix-fetch Playwright Tests
+
+Automated browser tests for the mix-fetch internal-dev harness. Tests run against the webpack-built `internal-dev/dist/` served locally.
+
+## Prerequisites
+
+WASM build artifacts must exist (Go first, then Rust). For local dev, use the debug targets:
+
+```bash
+# Builds to go-mix-conn/build/
+make -C wasm/mix-fetch/go-mix-conn build-debug-dev
+# Builds to pkg/ (needs Go bindings)
+make -C wasm/mix-fetch build-rust-debug
+```
+
+CI uses the same debug targets.
+
+Build the internal-dev webpack bundle:
+
+```bash
+cd wasm/mix-fetch/internal-dev && npm install && npm run build
+```
+
+Install Playwright and browsers:
+
+```bash
+cd wasm/mix-fetch/tests && npm install && npx playwright install --with-deps
+```
+
+## Running
+
+```bash
+# Smoke tests (all browsers, WASM load + MixFetch init)
+npm run test:smoke
+
+# Stress tests (all browsers, stresstest on mainnet)
+npm run test:stress
+
+# Single browser
+npx playwright test --project=smoke-chromium
+npx playwright test --project=stress-firefox
+```
+
+## Test tiers
+
+### Smoke (`smoke.spec.mjs`)
+
+Verifies the internal-dev harness loads in a headless browser: Rust WASM + Go WASM initialise, the worker signals readiness, MixFetch connects to a random Entry Gateway, and no console errors are emitted. Runs on Chromium, Firefox, and WebKit.
+
+- **CI workflow**: `ci-sdk-wasm.yml`
+- **Trigger**: every PR that touches `wasm/**`, `clients/client-core/**`, `common/**`, or the workflow itself
+- **Timeout**: 1 minute
+
+### Stress (`stress.spec.mjs`)
+
+Connects to mainnet via a random Entry Gateway, fires 10 concurrent mixed-size fetches through the mixnet, and asserts >= 80% succeed. Runs on all three browsers.
+
+- **CI workflow**: `nightly-mix-fetch-stress.yml`
+- **Trigger**: daily at 03:00 UTC via cron, also available via `workflow_dispatch` for manual runs
+- **Timeout**: 2 minutes per browser, 2 retries
+
+## Arch/Manjaro note
+
+Playwright's WebKit is built for Ubuntu 24.04 and links against specific soname versions that don't match Arch's (e.g. `libicu*.so.74` vs `.so.78`, `libxml2.so.2` vs `.so.16`). `playwright install --with-deps` also fails because it uses `apt-get`.
+
+Chromium and Firefox work without any workarounds. To skip WebKit locally:
+
+```bash
+npx playwright test --project=smoke-chromium --project=smoke-firefox
+npx playwright test --project=stress-chromium --project=stress-firefox
+```
+
+All three browsers work on CI (Ubuntu runners with `--with-deps`).
+
+> **TODO**: investigate getting WebKit running on Arch/Manjaro (soname symlinks or alternative Playwright WebKit build).
+
+## TODO
+
+- [ ] Add Playwright CI for `wasm/client/` (nym-client-wasm) via the chat-app examples after WASM cleanup
+- [ ] Add Playwright CI for other SDK examples once stale dependencies are resolved
+- [ ] Consider WebKit system deps in CI runner setup (currently relies on `playwright install --with-deps` on Ubuntu)
@@ -0,0 +1,13 @@
+{
+  "name": "mix-fetch-playwright-tests",
+  "private": true,
+  "scripts": {
+    "test": "playwright test",
+    "test:smoke": "playwright test --project=smoke-chromium --project=smoke-firefox --project=smoke-webkit",
+    "test:stress": "playwright test --project=stress-chromium --project=stress-firefox --project=stress-webkit"
+  },
+  "devDependencies": {
+    "@playwright/test": "^1.52.0",
+    "serve": "^14.2.4"
+  }
+}
@@ -0,0 +1,53 @@
+import { defineConfig } from "@playwright/test";
+
+export default defineConfig({
+  testDir: "./tests",
+  timeout: 60_000,
+  retries: 1,
+  use: {
+    trace: "on-first-retry",
+  },
+  webServer: {
+    command: "npx serve ../internal-dev/dist -l 8001 --no-clipboard",
+    port: 8001,
+    reuseExistingServer: true,
+  },
+  projects: [
+    {
+      name: "smoke-chromium",
+      testMatch: "smoke.spec.mjs",
+      use: { browserName: "chromium" },
+    },
+    {
+      name: "smoke-firefox",
+      testMatch: "smoke.spec.mjs",
+      use: { browserName: "firefox" },
+    },
+    {
+      name: "smoke-webkit",
+      testMatch: "smoke.spec.mjs",
+      use: { browserName: "webkit" },
+    },
+    {
+      name: "stress-chromium",
+      testMatch: "stress.spec.mjs",
+      timeout: 120_000,
+      retries: 2,
+      use: { browserName: "chromium" },
+    },
+    {
+      name: "stress-firefox",
+      testMatch: "stress.spec.mjs",
+      timeout: 120_000,
+      retries: 2,
+      use: { browserName: "firefox" },
+    },
+    {
+      name: "stress-webkit",
+      testMatch: "stress.spec.mjs",
+      timeout: 120_000,
+      retries: 2,
+      use: { browserName: "webkit" },
+    },
+  ],
+});
@@ -0,0 +1,71 @@
+// Smoke test: verify the internal-dev harness loads both WASM runtimes
+// and successfully initialises a MixFetch connection to mainnet.
+
+import { test, expect } from "@playwright/test";
+
+function waitForConsole(page, predicate, timeoutMs = 60_000) {
+  return new Promise((resolve, reject) => {
+    const timer = setTimeout(
+      () =>
+        reject(
+          new Error(`Timed out waiting for console message (${timeoutMs}ms)`)
+        ),
+      timeoutMs
+    );
+    page.on("console", function handler(msg) {
+      if (predicate(msg.text())) {
+        clearTimeout(timer);
+        page.removeListener("console", handler);
+        resolve(msg.text());
+      }
+    });
+  });
+}
+
+test("internal-dev harness loads and MixFetch initialises", async ({
+  page,
+}) => {
+  const errors = [];
+
+  // Forward worker lifecycle + errors to test output
+  page.on("console", (msg) => {
+    const text = msg.text();
+    if (msg.type() === "error") {
+      if (!text.includes("favicon.ico")) {
+        errors.push(text);
+      }
+      console.log(`[ERROR] ${text}`);
+    } else if (
+      text.includes("Worker") ||
+      text.includes("MixFetch") ||
+      text.includes("Setting up") ||
+      text.includes("gateway")
+    ) {
+      console.log(text);
+    }
+  });
+
+  page.on("pageerror", (err) => {
+    errors.push(`pageerror: ${err.message}`);
+  });
+
+  const workerReady = waitForConsole(
+    page,
+    (text) => text.includes("Worker ready"),
+    30_000
+  );
+  await page.goto("http://localhost:8001");
+  await workerReady;
+
+  // Init MixFetch with a random gateway
+  const mixFetchReady = waitForConsole(
+    page,
+    (text) => text.includes("MixFetch ready!"),
+    120_000
+  );
+  await page.check('input[name="gateway-mode"][value="random"]');
+  await page.click("#start-mixfetch");
+  await mixFetchReady;
+
+  expect(errors).toEqual([]);
+});
@@ -0,0 +1,87 @@
+// Stress test: connect to mainnet via a random Entry Gateway and run concurrent fetches.
+// Pass criteria: >= 80% of requests succeed.
+
+import { test, expect } from "@playwright/test";
+
+const STRESS_COUNT = 10;
+const MIN_SUCCESS_RATE = 0.8;
+
+function waitForConsole(page, predicate, timeoutMs = 60_000) {
+  return new Promise((resolve, reject) => {
+    const timer = setTimeout(
+      () =>
+        reject(
+          new Error(`Timed out waiting for console message (${timeoutMs}ms)`)
+        ),
+      timeoutMs
+    );
+    page.on("console", function handler(msg) {
+      if (predicate(msg.text())) {
+        clearTimeout(timer);
+        page.removeListener("console", handler);
+        resolve(msg.text());
+      }
+    });
+  });
+}
+
+test("stress test: mixed-size fetches through mainnet", async ({ page }) => {
+  // Forward warnings, errors, and worker lifecycle messages to test output
+  page.on("console", (msg) => {
+    const text = msg.text();
+    if (msg.type() === "warning" || msg.type() === "error") {
+      console.log(`[${msg.type().toUpperCase()}] ${text}`);
+    } else if (
+      text.includes("Worker") ||
+      text.includes("MixFetch") ||
+      text.includes("stress") ||
+      text.includes("COMPLETE") ||
+      text.includes("FAIL")
+    ) {
+      console.log(text);
+    }
+  });
+
+  const workerReady = waitForConsole(
+    page,
+    (text) => text.includes("Worker ready"),
+    30_000
+  );
+  await page.goto("http://localhost:8001");
+  await workerReady;
+
+  const mixFetchReady = waitForConsole(
+    page,
+    (text) => text.includes("MixFetch ready!"),
+    120_000
+  );
+  await page.check('input[name="gateway-mode"][value="random"]');
+  await page.click("#start-mixfetch");
+  await mixFetchReady;
+
+  const stressComplete = waitForConsole(
+    page,
+    (text) => text.includes("=== COMPLETE:"),
+    90_000
+  );
+  await page.fill("#stress-test-count", String(STRESS_COUNT));
+  await page.selectOption("#stress-test-mode", "mixed");
+  await page.click("#stress-test-button");
+  const completionMsg = await stressComplete;
+
+  const match = completionMsg.match(/OK (\d+)\/(\d+)/);
+  expect(
+    match,
+    `Could not parse completion message: ${completionMsg}`
+  ).toBeTruthy();
+
+  const [, succeeded, total] = match;
+  const successRate = parseInt(succeeded, 10) / parseInt(total, 10);
+
+  console.log(
+    `Stress test result: ${succeeded}/${total} succeeded (${(
+      successRate * 100
+    ).toFixed(0)}%)`
+  );
+  expect(successRate).toBeGreaterThanOrEqual(MIN_SUCCESS_RATE);
+});