From e4ef898feccbc0cb4613401ac744e33a193b2197 Mon Sep 17 00:00:00 2001 From: Teffen Ellis Date: Thu, 30 Apr 2026 21:39:16 +0000 Subject: [PATCH] ci/web: run Playwright e2e suite on every PR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Boots the full authentik stack (postgres + Go server + Rust worker) inside the existing ci-web workflow, applies migrations and the test-admin user blueprint, then runs `corepack npm run --prefix web test:e2e` against http://localhost:9000. Uploads the HTML report, traces/videos, and authentik logs as artifacts on failure so reviewers can debug without rerunning locally. Also enables the HTML reporter and screenshot/video capture on CI in playwright.config.js, and updates the full dev-environment docs to point at the same npm scripts CI uses so local and CI runs stay in lockstep. Closes #21994 Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> ci/web: make test-admin blueprint self-contained The previous blueprint used !Find to look up the authentik Admins group, which raced against system/bootstrap.yaml and resolved to None when the explicit apply_blueprint step ran before the worker had applied bootstrap. The serializer rejected groups: [None] with Invalid pk "None". Define the group in the same blueprint with state: present and reference it via !KeyOf, so the test admin setup does not depend on any pre-existing data. If bootstrap has already created the group, state: present is a no-op on the identifiers; otherwise the group is created here. Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> ci/web: format test-admin-user.yaml with prettier Pick up the 4-space indent that web/'s prettier config enforces. The file was added under issue #21994 with 2-space indent and tripped the ci-web format check on push. Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> Use parallelism. Remove guard. Reorder tests. Ignore playwright-traces. Update expected path. Always parallel. Flesh out types. ci/web: post Playwright result comment + gated S3 upload + !cancelled() guards Three reviewer-facing improvements to the e2e job: 1. Idempotent PR comment summarising Playwright pass/fail/flaky/skipped counts. Marker `` lets re-runs edit the same comment instead of piling up. Skipped on fork PRs where the default GITHUB_TOKEN is read-only. 2. Optional S3 publish of the HTML report to `s3://authentik-playwright-artifacts/pr-/run-/attempt-/`, gated behind `vars.PLAYWRIGHT_S3_ENABLED == 'true'`. The bucket is pending infra provisioning; the public URL pattern is already wired into the comment so flipping the variable on later requires no workflow changes. Borrows the OIDC + IAM role plumbing from `.github/workflows/release-publish.yml`. 3. Switch the failure-guarded reporting/upload steps to `!cancelled()` so a superseded (cancelled) run no longer emits failure-shaped noise, and so successful runs still produce the artifact bundle reviewers expect. Adds the Playwright JSON reporter so the parse step can pull pass/fail counts from `playwright-report/results.json` for the comment body. Co-Authored-By: Agent (authentik-i21996-internal-achievable-raisin) <279763771+playpen-agent@users.noreply.github.com> web/e2e: fix three regressions blocking the parallel suite Locally and in CI the new `e2e (playwright)` job appeared to "hang" under `fullyParallel: true` + `workers: "50%"`. The hang was actually five tests sharing two unrelated bugs that all manifest as 30s test timeouts; the cluster only *looks* like a parallelism issue because multiple workers stall on the same wall-clock window. With these three fixes the full suite is green in 1m48s on `--workers=2` (was: 5 failed / 17 passed in 5m30s). 1. `web/test/browser/600-providers.test.ts` PR #21647 dropped the `to:` argument on the `session.login()` call in this file's `beforeEach`. Without it, `SessionFixture.login()` waits for the auth-flow URL pattern to re-appear — which it does immediately, since we just navigated there — so the helper returns *before* the post-login redirect lands. The wizard buttons probed afterward live on `/if/admin/#/core/providers`, which the user never actually reaches; every test in the file then hits the 30s `beforeEach` timeout. Pin the destination explicitly, matching the shape of every other test file. 2. `web/src/admin/roles/ak-role-list.ts` The role-list row anchor had no aria-label, so its accessible name was the (random, generated) role name. `500-roles.test.ts` searches for that anchor with `getByRole("link", { name: "view details" })` — the same selector `400-groups.test.ts` uses against the group list, where `GroupListPage.row()` *does* set `aria-label="View details of group ..."`. Bring the role row to parity with groups; the test wasn't wrong, the UI was missing the accessibility hook. 3. `web/test/browser/500-roles.test.ts` ("Edit role from view page") The post-edit verification used `page.getByText(updatedName)`, but on the role view page the new name renders in two places (the "Role " page-navbar heading and the description-list value), so the bare text match resolves to two elements and trips strict-mode. Add `{ exact: true }` so we assert the canonical value the edit wrote rather than the heading template. Co-Authored-By: Agent (authentik-i21996-internal-achievable-raisin) <279763771+playpen-agent@users.noreply.github.com> Use headless. --- .github/workflows/ci-web.yml | 247 ++++++++++++++++++ web/.gitignore | 1 + web/e2e/fixtures/NavigatorFixture.ts | 6 - web/e2e/types/node.d.ts | 26 ++ web/playwright.config.js | 14 +- web/src/admin/roles/ak-role-list.ts | 8 +- web/test/blueprints/test-admin-user.yaml | 22 +- .../{session.test.ts => 100-session.test.ts} | 0 ....test.ts => 101-session-lifecycle.test.ts} | 2 - .../{modals.test.ts => 200-modals.test.ts} | 0 .../{users.test.ts => 300-users.test.ts} | 0 .../{groups.test.ts => 400-groups.test.ts} | 0 .../{roles.test.ts => 500-roles.test.ts} | 7 +- ...roviders.test.ts => 600-providers.test.ts} | 10 +- ...tions.test.ts => 700-applications.test.ts} | 0 web/test/browser/prerequisites.setup.ts | 9 +- .../setup/full-dev-environment.mdx | 41 +++ 17 files changed, 369 insertions(+), 24 deletions(-) create mode 100644 web/e2e/types/node.d.ts rename web/test/browser/{session.test.ts => 100-session.test.ts} (100%) rename web/test/browser/{session-lifecycle.test.ts => 101-session-lifecycle.test.ts} (99%) rename web/test/browser/{modals.test.ts => 200-modals.test.ts} (100%) rename web/test/browser/{users.test.ts => 300-users.test.ts} (100%) rename web/test/browser/{groups.test.ts => 400-groups.test.ts} (100%) rename web/test/browser/{roles.test.ts => 500-roles.test.ts} (93%) rename web/test/browser/{providers.test.ts => 600-providers.test.ts} (92%) rename web/test/browser/{applications.test.ts => 700-applications.test.ts} (100%) diff --git a/.github/workflows/ci-web.yml b/.github/workflows/ci-web.yml index 68d4b5ea8c..0303fc0f3a 100644 --- a/.github/workflows/ci-web.yml +++ b/.github/workflows/ci-web.yml @@ -12,6 +12,20 @@ on: - main - version-* +env: + POSTGRES_DB: authentik + POSTGRES_USER: authentik + POSTGRES_PASSWORD: "EK-5jnKfjrGRm<77" + AUTHENTIK_BLUEPRINTS_DIR: "./blueprints" + AUTHENTIK_OUTPOSTS__DISABLE_EMBEDDED_OUTPOST: "true" + # Drives the system/bootstrap.yaml blueprint at startup: creates akadmin with + # these credentials and flips the Setup flag (Setup.set(True)) so the SPA's + # post-login redirect to "/" doesn't bounce through /setup, which would 500 + # because the OOBE policy refuses to run once akadmin already has a usable + # password. See authentik/core/setup/signals.py and blueprints/default/flow-oobe.yaml. + AUTHENTIK_BOOTSTRAP_EMAIL: "test-admin@goauthentik.io" + AUTHENTIK_BOOTSTRAP_PASSWORD: "test-runner" + jobs: lint: runs-on: ubuntu-latest @@ -39,6 +53,239 @@ jobs: - name: build working-directory: web/ run: corepack npm run build + e2e: + name: e2e (playwright) + runs-on: ubuntu-latest + timeout-minutes: 60 + permissions: + contents: read + # Required so the "Comment Playwright result on PR" step can update its + # marker comment via the gh CLI / REST API. + pull-requests: write + # Required so the optional "Upload HTML report to S3" step can mint OIDC + # credentials with aws-actions/configure-aws-credentials. Harmless when + # the upload is gated off. + id-token: write + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v5 + - name: Setup authentik env + uses: ./.github/actions/setup + with: + dependencies: system,python,node,go,rust,runtime + - name: Build web UI + run: corepack npm run --prefix web build + - name: Build authentik server (Go) + run: | # shell + go build -o ./bin/authentik-server ./cmd/server + sudo install -m 0755 ./bin/authentik-server /usr/local/bin/authentik-server + - name: Build authentik worker (Rust) + run: | # shell + cargo build --release --bin authentik + sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik + - name: Apply migrations + run: uv run python -m lifecycle.migrate + - name: Resolve Playwright version + id: playwright-version + working-directory: web + run: | # shell + version=$(node -p "require('@playwright/test/package.json').version") + if [ -z "$version" ]; then + echo "Failed to resolve @playwright/test version" >&2 + exit 1 + fi + echo "version=${version}" >> "$GITHUB_OUTPUT" + - name: Cache Playwright browsers + id: playwright-cache + uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v4 + with: + path: ~/.cache/ms-playwright + key: ${{ runner.os }}-playwright-${{ steps.playwright-version.outputs.version }} + - name: Install Playwright browsers + working-directory: web + run: | # shell + if [ "${{ steps.playwright-cache.outputs.cache-hit }}" = "true" ]; then + corepack npm exec -- playwright install-deps chromium + else + corepack npm exec -- playwright install --with-deps chromium + fi + - name: Start authentik server and worker + run: | # shell + set -euo pipefail + mkdir -p /tmp/ak-logs + + # The Go server (authentik-server) spawns gunicorn as a child via PATH lookup + # and inherits the env that `uv run` set up for it. Verify gunicorn resolves + # under the same launcher so we fail fast here instead of waiting for an + # empty 200 from the proxy fallback later. + uv run --frozen sh -c 'command -v gunicorn' \ + || { echo "gunicorn not resolvable from uv run"; exit 1; } + + uv run ak server > /tmp/ak-logs/server.log 2>&1 & + echo $! > /tmp/ak-logs/server.pid + + # The Rust worker also opens an HTTP/metrics server on listen.http / + # listen.metrics (default :9000 / :9300). On a single CI host that races the + # Go server's binds and silently steals :9000, leaving Playwright talking to + # a healthcheck-only axum router that returns 200/empty for /if/* paths. + # Pin the worker to disjoint ports so the Go server keeps the public 9000. + AUTHENTIK_LISTEN__HTTP="[::]:9001" \ + AUTHENTIK_LISTEN__METRICS="[::]:9301" \ + uv run ak worker > /tmp/ak-logs/worker.log 2>&1 & + echo $! > /tmp/ak-logs/worker.pid + - name: Wait for authentik to be ready + run: | # shell + set -euo pipefail + # Readiness probes must verify the Go server is actually serving the request, + # not just that *something* on :9000 returned 200. The Go proxy stamps + # `X-authentik-version` on its static responses and the rendered flow page + # contains the custom element — both are absent from the + # worker's axum healthcheck router, so checking either rules out the + # port-collision failure mode. + timeout 240 bash -c ' + until curl -fsS -o /dev/null http://localhost:9000/-/health/ready/; do + sleep 2 + done' + timeout 300 bash -c ' + until curl -fsS http://localhost:9000/if/flow/default-authentication-flow/ \ + | grep -q "ak-flow-executor"; do + sleep 3 + done' + - name: Run Playwright tests + working-directory: web + env: + AK_TEST_RUNNER_PAGE_URL: http://localhost:9000 + run: corepack npm run test:e2e + # Reporting / upload steps below intentionally use `!cancelled()` rather + # than `failure()`: a cancelled run (e.g. superseded by a newer push) is + # not a real result and shouldn't produce reviewer-facing artifacts or + # comments. `if-no-files-found: ignore` keeps the "passed" case quiet. + - name: Upload Playwright HTML report + if: ${{ !cancelled() }} + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: playwright-report + path: web/playwright-report/ + retention-days: 14 + if-no-files-found: ignore + - name: Upload Playwright traces and videos + if: ${{ !cancelled() }} + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: playwright-traces + path: web/test-results/ + retention-days: 14 + if-no-files-found: ignore + - name: Upload authentik server and worker logs + if: ${{ !cancelled() }} + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: authentik-logs + path: /tmp/ak-logs/ + retention-days: 14 + if-no-files-found: ignore + - name: Parse Playwright results + id: playwright-results + if: ${{ !cancelled() }} + run: | # shell + set -euo pipefail + report=web/playwright-report/results.json + if [ ! -f "$report" ]; then + { + echo "available=false" + echo "passed=0" + echo "failed=0" + echo "flaky=0" + echo "skipped=0" + } >> "$GITHUB_OUTPUT" + exit 0 + fi + { + echo "available=true" + echo "passed=$(jq -r '.stats.expected // 0' "$report")" + echo "failed=$(jq -r '.stats.unexpected // 0' "$report")" + echo "flaky=$(jq -r '.stats.flaky // 0' "$report")" + echo "skipped=$(jq -r '.stats.skipped // 0' "$report")" + } >> "$GITHUB_OUTPUT" + # The S3 publishing pair below is intentionally gated off until the + # `authentik-playwright-artifacts` bucket is provisioned by infra. Flip + # the repo variable `PLAYWRIGHT_S3_ENABLED=true` to turn it on; the URL + # baked into the PR comment below already points at the eventual key. + - name: Configure AWS credentials for HTML report upload + if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && vars.PLAYWRIGHT_S3_ENABLED == 'true' }} + uses: aws-actions/configure-aws-credentials@ec61189d14ec14c8efccab744f656cffd0e33f37 # v6.1.0 + with: + role-to-assume: "arn:aws:iam::016170277896:role/github_goauthentik_authentik" + aws-region: eu-central-1 + - name: Upload HTML report to S3 + if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && vars.PLAYWRIGHT_S3_ENABLED == 'true' }} + env: + S3_BUCKET: authentik-playwright-artifacts + S3_KEY_PREFIX: pr-${{ github.event.pull_request.number }}/run-${{ github.run_id }}/attempt-${{ github.run_attempt }} + run: | # shell + set -euo pipefail + if [ ! -d web/playwright-report ]; then + echo "No playwright-report/ produced; skipping S3 upload" + exit 0 + fi + aws s3 cp \ + --recursive \ + --acl=public-read \ + --cache-control "public, max-age=600" \ + web/playwright-report/ \ + "s3://${S3_BUCKET}/${S3_KEY_PREFIX}/" + # Same-repo guard: fork PRs run with a read-only GITHUB_TOKEN (even after + # maintainer approval), so the comment + edit calls would 403. Skip cleanly + # rather than failing the job on every fork PR run. + - name: Comment Playwright result on PR + if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + AVAILABLE: ${{ steps.playwright-results.outputs.available }} + PASSED: ${{ steps.playwright-results.outputs.passed }} + FAILED: ${{ steps.playwright-results.outputs.failed }} + FLAKY: ${{ steps.playwright-results.outputs.flaky }} + SKIPPED: ${{ steps.playwright-results.outputs.skipped }} + S3_ENABLED: ${{ vars.PLAYWRIGHT_S3_ENABLED }} + REPORT_URL: https://authentik-playwright-artifacts.s3.amazonaws.com/pr-${{ github.event.pull_request.number }}/run-${{ github.run_id }}/attempt-${{ github.run_attempt }}/index.html + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }} + run: | # shell + set -euo pipefail + marker='' + + if [ "$AVAILABLE" = "true" ]; then + if [ "$FAILED" -gt 0 ]; then + status='❌ Failed' + elif [ "$FLAKY" -gt 0 ]; then + status='⚠️ Passed with flakes' + else + status='✅ Passed' + fi + stats=$(printf '| Result | Count |\n|---|---|\n| ✅ Passed | %s |\n| ❌ Failed | %s |\n| ⚠️ Flaky | %s |\n| ⏭️ Skipped | %s |\n' "$PASSED" "$FAILED" "$FLAKY" "$SKIPPED") + else + status='⚠️ No results produced' + stats='The job did not produce `playwright-report/results.json`. The suite likely crashed before the JSON reporter wrote its output — see the workflow run for setup-step failures.' + fi + + if [ "$S3_ENABLED" = "true" ]; then + report_line=$(printf '[HTML report](%s) · [Workflow run](%s)' "$REPORT_URL" "$RUN_URL") + else + report_line=$(printf '[Workflow run](%s) · _HTML report hosting is gated off until the `authentik-playwright-artifacts` S3 bucket is provisioned (`vars.PLAYWRIGHT_S3_ENABLED`). Until then, download the `playwright-report` artifact from the run page._' "$RUN_URL") + fi + + body=$(printf '%s\n## Playwright e2e — %s\n\n%s\n\n%s\n' "$marker" "$status" "$stats" "$report_line") + + existing=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" \ + --paginate \ + --jq "[.[] | select(.body != null and (.body | startswith(\"$marker\")))] | .[0].id // empty") + + if [ -n "$existing" ]; then + gh api -X PATCH "repos/${GITHUB_REPOSITORY}/issues/comments/${existing}" -f body="$body" > /dev/null + echo "Updated existing comment ${existing}" + else + gh api -X POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" -f body="$body" > /dev/null + echo "Created new playwright-result comment" + fi ci-web-mark: if: always() needs: diff --git a/web/.gitignore b/web/.gitignore index 1d7ebbe353..c10c9b24b6 100644 --- a/web/.gitignore +++ b/web/.gitignore @@ -32,6 +32,7 @@ lib-cov # Coverage directory used by tools like istanbul coverage playwright-report +playwright-traces test-results *.lcov diff --git a/web/e2e/fixtures/NavigatorFixture.ts b/web/e2e/fixtures/NavigatorFixture.ts index 645cd71f04..bb315a5ef8 100644 --- a/web/e2e/fixtures/NavigatorFixture.ts +++ b/web/e2e/fixtures/NavigatorFixture.ts @@ -2,12 +2,6 @@ import { PageFixture } from "#e2e/fixtures/PageFixture"; import { Page } from "@playwright/test"; -export const GOOD_USERNAME = "test-admin@goauthentik.io"; -export const GOOD_PASSWORD = "test-runner"; - -export const BAD_USERNAME = "bad-username@bad-login.io"; -export const BAD_PASSWORD = "-this-is-a-bad-password-"; - export interface LoginInit { username?: string; password?: string; diff --git a/web/e2e/types/node.d.ts b/web/e2e/types/node.d.ts new file mode 100644 index 0000000000..f3605eff3b --- /dev/null +++ b/web/e2e/types/node.d.ts @@ -0,0 +1,26 @@ +/** + * @file Node.js global types and environment variables. + */ + +declare module "process" { + global { + namespace NodeJS { + interface ProcessEnv { + /** + * The email address of the bootstrap user created when running the tests. + * + * @format email + */ + readonly AUTHENTIK_BOOTSTRAP_EMAIL?: string; + /** + * The password of the bootstrap user created when running the tests. + */ + readonly AUTHENTIK_BOOTSTRAP_PASSWORD?: string; + /** + * The directory where the authentik blueprints are stored. + */ + readonly AUTHENTIK_BLUEPRINTS_DIR?: string; + } + } + } +} diff --git a/web/playwright.config.js b/web/playwright.config.js index 863b1e6eaa..ef9ad8dca9 100644 --- a/web/playwright.config.js +++ b/web/playwright.config.js @@ -23,10 +23,16 @@ export default defineConfig({ testDir: "./test/browser", fullyParallel: true, forbidOnly: CI, - retries: CI ? 2 : 0, - workers: CI ? 1 : undefined, + retries: CI ? 1 : 0, + workers: "50%", + maxFailures: CI ? 5 : 2, reporter: CI - ? "github" + ? [ + // --- + ["github"], + ["html", { open: "never", outputFolder: "playwright-report" }], + ["json", { outputFile: "playwright-report/results.json" }], + ] : [ // --- ["list", { printSteps: true }], @@ -36,6 +42,8 @@ export default defineConfig({ testIdAttribute: "data-test-id", baseURL, trace: "on-first-retry", + screenshot: "only-on-failure", + video: CI ? "retain-on-failure" : "off", colorScheme: "dark", launchOptions: { logger: { diff --git a/web/src/admin/roles/ak-role-list.ts b/web/src/admin/roles/ak-role-list.ts index cc04f13c1b..af1273d6ce 100644 --- a/web/src/admin/roles/ak-role-list.ts +++ b/web/src/admin/roles/ak-role-list.ts @@ -18,7 +18,7 @@ import { RoleForm } from "#admin/roles/ak-role-form"; import { RbacApi, Role } from "@goauthentik/api"; -import { msg } from "@lit/localize"; +import { msg, str } from "@lit/localize"; import { html, PropertyValues, TemplateResult } from "lit"; import { customElement, state } from "lit/decorators.js"; @@ -83,7 +83,11 @@ export class RoleListPage extends TablePage { row(item: Role): SlottedTemplateResult[] { return [ - html`${item.name}`, + html`${item.name}`, html`
${IconEditButton(RoleForm, item.pk)}
`, ]; } diff --git a/web/test/blueprints/test-admin-user.yaml b/web/test/blueprints/test-admin-user.yaml index 1a0e85e173..a8994242b7 100644 --- a/web/test/blueprints/test-admin-user.yaml +++ b/web/test/blueprints/test-admin-user.yaml @@ -1,6 +1,19 @@ version: 1 +metadata: + name: Playwright e2e test admin entries: - - attrs: + - model: authentik_core.group + state: present + identifiers: + name: "authentik Admins" + attrs: + is_superuser: true + id: admin-group + - model: authentik_core.user + state: present + identifiers: + username: akadmin + attrs: email: test-admin@goauthentik.io is_active: true name: authentik Default Admin @@ -8,9 +21,4 @@ entries: path: users type: internal groups: - - !Find [authentik_core.group, [name, "authentik Admins"]] - conditions: [] - identifiers: - username: akadmin - model: authentik_core.user - state: present + - !KeyOf admin-group diff --git a/web/test/browser/session.test.ts b/web/test/browser/100-session.test.ts similarity index 100% rename from web/test/browser/session.test.ts rename to web/test/browser/100-session.test.ts diff --git a/web/test/browser/session-lifecycle.test.ts b/web/test/browser/101-session-lifecycle.test.ts similarity index 99% rename from web/test/browser/session-lifecycle.test.ts rename to web/test/browser/101-session-lifecycle.test.ts index 0aff0f0b24..d49e5a8401 100644 --- a/web/test/browser/session-lifecycle.test.ts +++ b/web/test/browser/101-session-lifecycle.test.ts @@ -17,8 +17,6 @@ test.describe("Session Lifecycle", () => { test.beforeAll( 'Ensure "Enable Remember me on this device" is on for the default identification stage', async ({ browser }, { title: testName }) => { - if (Date.now()) return; - const context = await browser.newContext(); const page = await context.newPage(); const navigator = new NavigatorFixture(page, testName); diff --git a/web/test/browser/modals.test.ts b/web/test/browser/200-modals.test.ts similarity index 100% rename from web/test/browser/modals.test.ts rename to web/test/browser/200-modals.test.ts diff --git a/web/test/browser/users.test.ts b/web/test/browser/300-users.test.ts similarity index 100% rename from web/test/browser/users.test.ts rename to web/test/browser/300-users.test.ts diff --git a/web/test/browser/groups.test.ts b/web/test/browser/400-groups.test.ts similarity index 100% rename from web/test/browser/groups.test.ts rename to web/test/browser/400-groups.test.ts diff --git a/web/test/browser/roles.test.ts b/web/test/browser/500-roles.test.ts similarity index 93% rename from web/test/browser/roles.test.ts rename to web/test/browser/500-roles.test.ts index 041fb44163..e4bb23cc5d 100644 --- a/web/test/browser/roles.test.ts +++ b/web/test/browser/500-roles.test.ts @@ -161,8 +161,13 @@ test.describe("Roles", () => { }); await test.step("Verify role name updated on view page", async () => { + // The role name shows up in both the page-navbar heading (as + // "Role ") and the description-list text (raw ); a + // bare getByText match hits both and triggers strict-mode. Pin + // to the description list so we're asserting the canonical + // value the edit just wrote, not the heading template. await expect( - page.getByText(updatedName), + page.getByText(updatedName, { exact: true }), "Updated role name is visible on view page", ).toBeVisible(); }); diff --git a/web/test/browser/providers.test.ts b/web/test/browser/600-providers.test.ts similarity index 92% rename from web/test/browser/providers.test.ts rename to web/test/browser/600-providers.test.ts index 851bdf9002..b430155402 100644 --- a/web/test/browser/providers.test.ts +++ b/web/test/browser/600-providers.test.ts @@ -17,7 +17,15 @@ test.describe("Provider Wizard", () => { const dialog = page.getByRole("dialog", { name: "New Provider Wizard" }); - await test.step("Authenticate", async () => session.login()); + // session.login() with no `to` waits for the auth flow URL pattern to + // re-appear, which it does immediately (we just navigated there), so + // the helper returns *before* the post-login redirect lands. The + // wizard buttons probed below live on /if/admin/#/core/providers, so + // pin the destination explicitly — same shape as the other test files. + await test.step("Authenticate", async () => + session.login({ + to: "/if/admin/#/core/providers", + })); await test.step("Navigate to provider wizard", async () => { await expect(dialog, "Dialog is initially closed").toBeHidden(); diff --git a/web/test/browser/applications.test.ts b/web/test/browser/700-applications.test.ts similarity index 100% rename from web/test/browser/applications.test.ts rename to web/test/browser/700-applications.test.ts diff --git a/web/test/browser/prerequisites.setup.ts b/web/test/browser/prerequisites.setup.ts index 0fe350f0df..be2062bf17 100644 --- a/web/test/browser/prerequisites.setup.ts +++ b/web/test/browser/prerequisites.setup.ts @@ -3,9 +3,14 @@ import { expect, test as setup } from "#e2e"; setup("Web server availability", async ({ baseURL }) => { expect(baseURL, "Base URL is set").toBeTruthy(); - const ok = await fetch(baseURL!) + // Probe the default authentication flow rather than the root URL — the root + // redirects to /setup when AUTHENTIK_BOOTSTRAP_* env vars are unset, and + // /setup raises FlowNonApplicableException (HTTP 500) once an admin user + // already has a usable password (as our test-admin blueprint installs). + const probeURL = new URL("/if/flow/default-authentication-flow/", baseURL!); + const ok = await fetch(probeURL) .then((res) => res.ok) .catch(() => false); - expect(ok, `Web server should be listening on ${baseURL}`).toBeTruthy(); + expect(ok, `Web server should be serving ${probeURL}`).toBeTruthy(); }); diff --git a/website/docs/developer-docs/setup/full-dev-environment.mdx b/website/docs/developer-docs/setup/full-dev-environment.mdx index 4221cc8ef4..b7d628e0d4 100644 --- a/website/docs/developer-docs/setup/full-dev-environment.mdx +++ b/website/docs/developer-docs/setup/full-dev-environment.mdx @@ -202,6 +202,13 @@ Copy the generated recovery key and paste it into the URL, after the domain. For ## End-to-End (E2E) Setup +authentik ships two end-to-end test suites: + +- The Django/Selenium suite under `tests/e2e/` — driven by Django's test runner. It is exercised in CI by the `test-e2e` job in `ci-main.yml`. +- The browser-side Playwright suite under `web/test/browser/` — driven by `web/playwright.config.js`. It is exercised in CI by the `e2e (playwright)` job in `ci-web.yml` and runs against a live authentik server. + +### Django/Selenium suite + Start the E2E test services with the following command: ```shell @@ -216,6 +223,40 @@ Alternatively, you can connect directly via VNC on port `5900` using the passwor When using Docker Desktop, host networking needs to be enabled via **Docker Settings** > **Resources** > **Network** > **Enable host networking**. ::: +### Playwright suite + +The Playwright suite assumes that: + +- An authentik stack is reachable at `http://localhost:9000` (override with `AK_TEST_RUNNER_PAGE_URL`). +- The blueprint at `web/test/blueprints/test-admin-user.yaml` has been applied so that the `test-admin@goauthentik.io` user (password `test-runner`) can log in. + +Both prerequisites are satisfied by the standard full development environment described above, plus a one-off blueprint apply. From the repository root: + +```shell +# Make the test admin blueprint discoverable by the worker, then start authentik. +mkdir -p blueprints/local +cp web/test/blueprints/test-admin-user.yaml blueprints/local/test-admin-user.yaml + +# In separate terminals: +make run-server +make run-worker +``` + +Once the worker has applied the blueprint, install Playwright's browsers (one-time) and run the suite — these are the same npm scripts CI runs: + +```shell +corepack npm exec --prefix web -- playwright install --with-deps chromium +corepack npm run --prefix web test:e2e +``` + +After a failing run, open the HTML report to inspect traces, screenshots, and (in CI) videos: + +```shell +corepack npm exec --prefix web -- playwright show-report +``` + +In CI, the same artifacts are uploaded under the `playwright-report` and `playwright-traces` artifact names on the failed workflow run. + ## Contributing code ### Before submitting a pull request