NYM-1199: Finalize Node Families E2E design for mock and native tests
Resolves open questions, detailing runtime persona selection for the mock server (D2), consolidating the Playwright suite (D10), scoping the native leg to Linux-only (D8), and introducing a manual sandbox smoke test (D9). Updates related tasks accordingly.
This commit is contained in:
@@ -35,8 +35,8 @@ This design supersedes an earlier framing that made WebdriverIO + `tauri-driver`
|
||||
**D1 — Playwright against the mock-wired dev server is the PRIMARY suite.**
|
||||
Point Playwright at `http://localhost:9000` with the mock flag on, and replay the journeys against the real app shell + router. Rationale: runs on every OS (incl. the developer's Mac), reuses the already-present `@playwright/test` dep and the existing selectors/specs, and tests the actual app chrome rather than Storybook iframes. *Alternative considered:* keep driving Storybook stories — lower fidelity (no router/app shell) and no closer to the real app. *Alternative considered:* Cypress — no advantage over the Playwright suite already in the repo.
|
||||
|
||||
**D2 — Build-time env flag selects the provider (not runtime).**
|
||||
A webpack `DefinePlugin` constant (e.g. `WALLET_MOCK_FAMILIES`, tri-state `owner|operator|off`) gates the import of `MockFamiliesContextProvider` vs `FamiliesContextProvider` behind a `const` check; with the flag off (default) the dead branch and its transitive mock imports tree-shake out. The flag both selects the mock provider and chooses the persona seed (`buildOwnerFlowStore` / `buildOperatorFlowStore`). *Alternatives considered:* runtime URL/query flag (ships mock code in prod unless guarded; rejected) and a separate entry point (heavier; unnecessary since the page is already provider-agnostic).
|
||||
**D2 — Build-time flag gates mock-vs-real; persona is chosen at runtime *within* the mock build.**
|
||||
A webpack `DefinePlugin` boolean (`WALLET_MOCK_FAMILIES=on|off`, default `off`) gates the import of `MockFamiliesContextProvider` vs `FamiliesContextProvider` behind a `const` check; with the flag off the dead branch and its transitive mock imports tree-shake out, so no mock code ships. The **persona** (`buildOwnerFlowStore` / `buildOperatorFlowStore` + sender) is selected at *runtime* from a URL param (`?persona=owner|operator`, default `owner`) read only by the mock route wrapper. This stays prod-safe — the runtime reader lives inside the already-build-gated mock branch — and means a **single** dev server serves both personas, so Playwright just navigates to different URLs (resolves the persona open question). *Alternatives considered:* tri-state build flag with one persona per build (forces two dev servers / two builds; rejected as heavier and slower); pure runtime URL flag for mock-vs-real (ships mock code in prod unless guarded; rejected); separate entry point (unnecessary — the page is already provider-agnostic).
|
||||
|
||||
**D3 — Reuse the existing fixtures and selectors verbatim.**
|
||||
The mock-wired dev server seeds the same fixtures as the Storybook flows, and the page already exposes the journey `data-testid`s, so the Playwright journeys mirror `FamilyFlows.stories.tsx` step-for-step. The same selectors then carry over to the optional WebdriverIO leg, keeping all suites observably equivalent.
|
||||
@@ -53,6 +53,15 @@ The merged `ci-nym-wallet-frontend.yml` has one `build` job (ubuntu-22.04: insta
|
||||
**D7 — Sandbox real-IPC smoke is a separate, optional, READ-only tier.**
|
||||
The sandbox contract (one family/one member) lets us smoke the real `FamiliesContextProvider` + `src/requests/families.ts` against a live chain — validating the IPC wiring that the mock deliberately stands in for (parent-change 9.4). Keep it read-only and separate from the deterministic mock e2e: a shared sandbox is a poor place to run create/kick/disband lifecycles, and live-network reads are inherently flakier. *Alternative considered:* fold sandbox into the main e2e — rejected (non-determinism + shared-state mutation).
|
||||
|
||||
**D8 — Native leg is Linux-only initially (no Windows leg yet).**
|
||||
Tier 2 ships as a single Ubuntu job. Adding the Windows/WebView2 leg (also supported by `tauri-driver`) doubles CI cost and maintenance for a tier that is already optional/`continue-on-error`; WebKitGTK on Linux is the higher-value target since it is closest to the Linux desktop builds. Revisit Windows only if a WebView2-specific regression surfaces.
|
||||
|
||||
**D9 — Sandbox smoke ships as a documented MANUAL step first, pinning the known family id.**
|
||||
A live sandbox read needs a connected, funded wallet account, which is not provisionable non-interactively in CI today (mnemonic in secrets + network + chain availability). So D7's smoke starts as a documented manual procedure that pins the known sandbox family id and asserts render/shape, not exact contents. It graduates to a CI job only once a sandbox test account can be provisioned headlessly — tracked as a follow-up, not a blocker.
|
||||
|
||||
**D10 — One Playwright suite: repoint to the dev server, retire the Storybook-iframe specs.**
|
||||
Rather than maintain two Playwright suites, repoint the single `e2e/families.spec.ts` at the mock-wired dev server (`:9000`). The Storybook `play` functions remain as Storybook-level interaction coverage (runnable via the test-runner), but we do not keep a parallel Playwright-against-Storybook suite — it would be lower fidelity and a drift source for no added coverage.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Browser fidelity ≠ packaged app** → The primary Playwright suite tests the web frontend, not the native binary or real `invoke`; the optional WebdriverIO leg covers the binary, and D7 covers real IPC. The three tiers together close the gap the mock leaves.
|
||||
@@ -69,7 +78,10 @@ Additive only. Rollout: (1) build-time flag + provider seam; (2) repoint Playwri
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Persona handling for the dev server: one server per persona run (re-launch with a different `WALLET_MOCK_FAMILIES`) vs a single launch with a runtime persona switch — default to per-run env, resolve during tasks.
|
||||
- Whether to add a Windows leg to the native job (supported by `tauri-driver`) or keep Linux-only initially — start Linux-only.
|
||||
- Sandbox smoke placement: CI job vs a documented manual step, and whether it pins a known family id — resolve during tasks.
|
||||
- Whether to retire the Storybook-iframe Playwright specs once the dev-server suite lands, or keep both — lean toward repointing (one suite).
|
||||
All four prior open questions are now resolved:
|
||||
- **Persona handling** → single dev server; persona via runtime `?persona=` URL param inside the build-gated mock branch (**D2**).
|
||||
- **Windows native leg** → no; Linux-only initially (**D8**).
|
||||
- **Sandbox smoke placement / family id** → documented manual step first, pinning the known sandbox family id; CI only once a headless test account exists (**D9**).
|
||||
- **Retire Storybook-iframe Playwright specs?** → yes; one suite, repointed at the dev server (**D10**).
|
||||
|
||||
Residual (a follow-up, not a blocker): provisioning a headless sandbox test account would let D9's smoke graduate from manual to CI.
|
||||
|
||||
@@ -1,20 +1,20 @@
|
||||
## 1. Build-time mock provider seam
|
||||
|
||||
- [ ] 1.1 Add a webpack `DefinePlugin` constant for the families mock flag (`WALLET_MOCK_FAMILIES`, tri-state `owner|operator|off`, default `off`) in the dev/mock webpack config.
|
||||
- [ ] 1.1 Add a webpack `DefinePlugin` boolean for the families mock gate (`WALLET_MOCK_FAMILIES=on|off`, default `off`) in the dev/mock webpack config (mock-vs-real only; persona is runtime, see 1.3).
|
||||
- [ ] 1.2 Introduce a provider-selection module that exports either `FamiliesContextProvider` (real) or `MockFamiliesContextProvider` (mock) based on the compile-time flag, behind a `const` guard so the unused branch tree-shakes. Keep this seam in its **own** module — do NOT make `FamilyPage.tsx` depend on the flag or on Tauri, so the merged `FamilyPage.figma.tsx` Code Connect mapping (`example: () => <FamilyPage />`) still imports it in isolation.
|
||||
- [ ] 1.3 Have the Family route/entry consume the selection module; in mock mode seed `buildOwnerFlowStore` / `buildOperatorFlowStore` per the persona flag (reuse `families.fixtures.ts`).
|
||||
- [ ] 1.3 Have the Family route/entry consume the selection module; in mock mode, read the persona at runtime from a `?persona=owner|operator` URL param (default `owner`) and seed `buildOwnerFlowStore` / `buildOperatorFlowStore` accordingly (reuse `families.fixtures.ts`). The persona reader lives only inside the build-gated mock branch.
|
||||
- [ ] 1.4 Ensure the Family page is reachable via normal in-app navigation on the dev server (`:9000`) and renders inside the app shell + router (not a Storybook iframe), keeping the existing `data-testid`s.
|
||||
- [ ] 1.5 Verify a default (flag `off`) production build excludes families mock-engine code (inspect bundle / add a guard); confirm the real provider still wires unchanged.
|
||||
- [ ] 1.6 Add npm scripts to launch the mock-wired dev server per persona (e.g. `WALLET_MOCK_FAMILIES=owner pnpm webpack:dev`).
|
||||
- [ ] 1.6 Add an npm script to launch the single mock-wired dev server (e.g. `WALLET_MOCK_FAMILIES=on pnpm webpack:dev`); both personas are reached on the one server via `?persona=`.
|
||||
|
||||
## 2. Primary e2e — Playwright against the mock-wired dev server
|
||||
|
||||
- [ ] 2.1 Repoint `playwright.config.ts` from Storybook (:6006) to the mock-wired dev server (`baseURL http://localhost:9000`), with a `webServer` that launches the dev server with the mock flag (per persona) and `reuseExistingServer` locally.
|
||||
- [ ] 2.2 Port the owner lifecycle journey (create → invite → accept → kick → disband) to drive the app-shell route, reusing the existing `data-testid` selectors and the Storybook flow steps.
|
||||
- [ ] 2.3 Port the operator lifecycle journey (accept → leave, then reject), asserting the reject-node invite group ends empty.
|
||||
- [ ] 2.1 Repoint `playwright.config.ts` from Storybook (:6006) to the mock-wired dev server (`baseURL http://localhost:9000`), with a single `webServer` (`WALLET_MOCK_FAMILIES=on`) and `reuseExistingServer` locally; tests pick persona via the `?persona=` URL.
|
||||
- [ ] 2.2 Port the owner lifecycle journey (create → invite → accept → kick → disband) against `/family?persona=owner`, reusing the existing `data-testid` selectors and the Storybook flow steps.
|
||||
- [ ] 2.3 Port the operator lifecycle journey against `/family?persona=operator` (accept → leave, then reject), asserting the reject-node invite group ends empty.
|
||||
- [ ] 2.4 Port the multi-node operator invite-states assertion (`node-invite-group-201` present, `node-invite-group-203-empty`).
|
||||
- [ ] 2.5 Handle portalled confirmation dialogs via their global test ids (mirror the Storybook `screen`-scoped queries).
|
||||
- [ ] 2.6 Decide whether to retire the old Storybook-iframe specs or keep both; if repointing, update `e2e/families.spec.ts` accordingly and factor shared selector/step constants for parity.
|
||||
- [ ] 2.6 Retire the old Storybook-iframe specs (D10): replace `e2e/families.spec.ts` with the dev-server journeys and factor shared selector/step constants for parity (Storybook `play` functions stay as Storybook-level coverage).
|
||||
- [ ] 2.7 Confirm the suite runs green locally on macOS (Chromium; optionally WebKit project).
|
||||
|
||||
## 3. CI — wire the primary suite in
|
||||
@@ -30,10 +30,11 @@
|
||||
- [ ] 4.4 Reuse the journey steps/selectors from §2 (shared constants) so the native leg asserts identical outcomes.
|
||||
- [ ] 4.5 Add a **separate** CI job (ubuntu-22.04) following the Tauri WebDriver-in-CI flow: install `libwebkit2gtk-4.1-dev` + `webkit2gtk-driver` + `xvfb`, set up Rust + cache, `cargo install tauri-driver --locked`, build the mock-wired binary, run the suite under `xvfb-run`. Start it `continue-on-error` until stable.
|
||||
|
||||
## 5. Optional — sandbox real-IPC read smoke
|
||||
## 5. Optional — sandbox real-IPC read smoke (manual first, D9)
|
||||
|
||||
- [ ] 5.1 Add a read-only smoke (Playwright or a small harness) that loads the app against the real `FamiliesContextProvider` connected to sandbox and asserts the Family page renders the sandbox family/member via real IPC.
|
||||
- [ ] 5.2 Keep it read-only (no create/invite/kick/disband) and assert shape (or pin the known family id) rather than exact contents; mark the job non-blocking and separate from the mock suites.
|
||||
- [ ] 5.1 Document a manual read-only smoke: connect the app to a sandbox account, open the Family page, and confirm it renders the known sandbox family/member via the real `FamiliesContextProvider` + `requests/families.ts` (no state-changing transaction).
|
||||
- [ ] 5.2 Pin the known sandbox family id and assert render/shape rather than exact contents, so a contract redeploy doesn't hard-fail; keep it separate from and non-blocking relative to the mock suites.
|
||||
- [ ] 5.3 (Follow-up, not a blocker) If/when a sandbox test account can be provisioned headlessly (mnemonic in CI secrets), promote the smoke to a non-blocking CI job.
|
||||
|
||||
## 6. Verification & docs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user