* ci(pr): run E2E when a PR touches tests/e2e paths
Regression specs under tests/e2e never ran on PR CI — only schedule and
release called e2e.yml — so a red regression test could merge green.
Path-filter and workflow_call the E2E suite when E2E-relevant files change.
Use merge-base diffs so base-branch drift does not false-trigger E2E, fail
the detector when git diff cannot compute the PR range, and pin
least-privilege contents:read on both the detector and reusable E2E workflow.
Closes#10518
Co-authored-by: Wooseong Kim <innocarpe@gmail.com>
Co-authored-by: Orca <help@stably.ai>
* ci(pr): make the E2E path gate actually block, and match the real config path
Two fixes to the new path-filtered E2E job.
The gate did not gate. pr.yml's `verify` job is the required check, and it
enumerates its dependencies explicitly — `e2e` was in neither `needs` nor the
result list, so a failing shard left `verify` green. That reproduces the exact
hole this job exists to close: a red spec merges green, just with a red box
further down the page. Add `e2e` to both.
Because the job is path-filtered, `skipped` is the normal result on a PR that
touches no E2E files and has to keep passing. That allowance is checked after
the strict loop rather than inside it, so it can never leak to the six jobs
that are always required.
The `playwright.` pattern matched nothing. The config is
tests/playwright.config.ts — beside tests/e2e/, not inside it — so no tracked
file starts with `playwright.` and editing the runner config would silently
skip E2E. Anchor it at `tests/playwright.`.
Adds a contract test alongside the existing release-e2e one. Verified it fails
when either fix is reverted, and simulated the gate across
success/skipped/failure/cancelled plus the skip-must-not-mask-a-real-failure
case.
* test(ci): close two gaps in the E2E gate contract
CodeRabbit was right on both counts — verified by reverting each and watching
the contract stay green.
The path filter was unasserted, so `e2e` could lose its `if:` and run on every
PR — the cost the filter exists to avoid — without failing anything.
The strict-loop check hardcoded four of the six required jobs, so dropping
GIT_COMPATIBILITY or SHELL_CONTRACTS left them unenforced while the contract
passed. Derive the list from verify.needs instead, so a newly added required
job that misses the loop fails here rather than silently going unchecked.
* ci(pr): land the E2E path gate advisory instead of blocking
The E2E suite is currently failing every scheduled run on main — 22 of the last
22 — so making verify depend on it would block any PR touching tests/e2e/**,
including the PRs that fix the suite. This PR's own run reproduced that: 3 of 12
shards failed on specs unrelated to it (agent-session resume, Jira linking,
plugin containment, terminal artifacts).
So the job runs and reports on E2E-path PRs but is left out of verify.needs for
now. The detector, the tests/playwright. path fix, and the contract tests are
unaffected — those stand on their own and were the substance of the review.
Flipping to blocking is a three-line change once the suite is green; the exact
wiring, including why the skipped allowance must sit outside the strict loop, is
recorded on verify's Require-successful-checks step. The contract test pins the
advisory choice so it reads as deliberate rather than as the unwired-gate bug it
originally caught, and still fails if the path filter, the strict-loop coverage,
or the config path regress.
---------
Co-authored-by: Wooseong Kim <innocarpe@gmail.com>
Co-authored-by: Orca <help@stably.ai>
1fa9ffb5ea