Repository navigation
Port Selenium ITs to Playwright - #3097
Open
vladbailescu wants to merge 80 commits into
Open
vladbailescu wants to merge 80 commits into
vladbailescu wants to merge 80 commits into
Conversation
Adds a shared orchestrator (testing/it/docker/run-it.sh) plus a GitHub Actions job (.github/workflows/maven-it.yml) that run the testing/it/http suite against an AEM author instance started from the private circleci-aem-cloudready QuickProvider image, using the cloud (-cloud) core-components all package. Phase 1 is author-only: only the four IT classes that do not use a publish instance are selected. The image is a qp server, so the author is started with a qp client command inside the same container (no --network host needed); packages are installed via the CRX Package Manager. Local arm64 boots run under emulation and are best-effort; native amd64 CI is the authoritative run. See the README for the local flow, the confirmed image behavior, and the phased roadmap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Revert before merging to main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy the running instance's crx-quickstart logs out of the container on exit and upload them, so failures like the AdaptiveImageServlet/TableOfContentsFilter 500s can be diagnosed from the run artifacts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mage - Cache ~/.m2 with explicit restore/save (persists on failure). - Cache the AEM Docker image (docker save/load, keyed on the image ref) so runs after the first skip the multi-GB pull; log in + pull only on a cache miss. - Bump the cloud-ready image 26125-openjdk21 -> 27830-v2-openjdk21 (latest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…space qp's default -XX:MaxMetaspaceSize=256m is too small for the cloud-ready instance on Java 21: metaspace filled mid-run and later requests 500'd with OOM (surfaced as AdaptiveImageServlet/TableOfContents failures). Pass --vm-options to qp start (-Xmx4g -XX:MaxMetaspaceSize=1g), overridable via QP_VM_OPTIONS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
qp.sh runs the quickstart via eval "java ... $*", which word-split the multi-token --vm-options value (qp errored: Invalid option: -XX:MaxMetaspaceSize). Wrap the value in literal single quotes so it survives the eval as one argument (verified via qp --dry-run). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Install the core.wcm.components.it.core OSGi bundle (TestTransformerFactory / 'core-components-test-transformer' rewriter) via the Felix console during provisioning; TableOfContentsFilterIT 500'd without it. Local run is now 10/10. - Switch the AEM Docker image cache from the combined actions/cache (whose post-save is skipped on job failure) to split restore/save with if: always(), matching the Maven cache, so the image tarball persists across runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- run-it.sh: WITH_PUBLISH starts/provisions a publish instance (port 4503) and runs the full *IT.java suite against author + publish; publish gets the same content packages (tests GET pre-deployed content, no replication needed). - Exclude @category(IgnoreOnCloud) classes on the cloud target via excludedGroups (SeoIT, TableOfContentsFilterIT, ClientlibsIncludeIT), matching the pipeline; new IT_EXCLUDED_GROUPS knob. Author-only default trimmed accordingly. - maven-it.yml: run author+publish with per-instance heaps sized for the 16 GB public-repo runner; timeout 90m. Per-instance AEM log capture. Verified locally (author + publish, cloud-ready image): BUILD SUCCESS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- run-it.sh: WITH_SELENIUM mode runs the e2e-selenium suite with a host-local browser (-Dsel.jup.default.browser=chrome), reaching AEM at the published localhost:4502 (the module pins the author URL there; a Selenoid browser in a separate container cannot reach it). Wraps mvn in xvfb-run for headless CI; runs headed on macOS (no xvfb-run), the locally-validated path. - maven-it.yml: add a workflow_dispatch-only 'selenium' job (Xvfb + preinstalled Chrome, warm image + Maven caches), with a selenium_it_test input. Verified locally (cloud-ready image, ListIT): browser drove the author editor, 14/15 passing, BUILD SUCCESS. The one error is an emulation-speed Selenium timeout, not a wiring/code defect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- The selenium base lib calls window.maximize() per test, which fails under bare Xvfb (no window manager: 'Runtime.evaluate wasn't found'). Provide a virtual display via Xvfb (1920x1080x24) + fluxbox and export DISPLAY so host-local Chrome has a real windowed environment. - e2e-selenium binds only failsafe:integration-test (not verify), so mvn never fails on test errors; set the selenium report action fail_on_failure: true so the job status reflects actual results. Note: the Xvfb/WM path is Linux-only and cannot be exercised on the macOS dev box (local Chrome runs headed there); validated on CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- run-it.sh: SEL_GROUPS -> -Dgroups (JUnit tag include) and SEL_EXCLUDED_GROUPS -> -DexcludedGroups (default failing,nested,IgnoreOnSDK; IgnoreOnSDK mirrors the pipeline's cloud/SDK skip). Verified locally that -Dgroups filters selection. - maven-it.yml: selenium job becomes a fail-fast:false matrix over group1..group4 plus an 'ungrouped' leg (pdfviewer/separator/download v1/v2 carry no group tag), so all e2e-selenium tests run and the legs execute in parallel. Per-group check_name and artifact name. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ition
- run-it.sh: the smoke-class default no longer applies when SEL_GROUPS is set. An
empty SEL_IT_TEST with a group must stay empty; the previous ${SEL_IT_TEST:-<class>}
re-injected ListIT, so each group leg ran '-Dit.test=ListIT AND -Dgroups=groupN'
- only group4 (which contains ListIT) ran anything; group1-3 ran 0 tests and
falsely passed. Now group legs run the entire group.
- DownloadIT v1/v2: accept the RFC 5987 'filename*=UTF-8''<name>' parameter newer
AEM appends to Content-Disposition (startsWith the filename directive instead of
exact match). Verified locally: the doc-type assertions now pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
UI tests are prone to transient timing flakiness, and search tests can miss the async Oak index on the first attempt. Pass -Dfailsafe.rerunFailingTestsCount so failsafe re-runs failed tests before marking them failed. Effect measured on CI (timing-dependent). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r group - Add a 'prep' job that builds the deployable packages and primes the AEM image cache ONCE, publishing them as an artifact + a generated matrix. The http and selenium jobs now 'needs: prep' and only download the packages + docker load the cached image - no per-leg reactor build or multi-GB pull. - gen-selenium-matrix.sh splits each @tag group (group1..4) into 4 class buckets plus an ungrouped leg (17 legs, all 56 classes, verified locally) for finer parallelism. selenium job matrix comes from fromJSON(prep output). - Note: bash's built-in GROUPS array collides with a normal var name; the generator uses TAG_GROUPS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Bump actions/checkout, setup-java, cache/restore, cache/save, upload-artifact, download-artifact from v4 -> v5 to clear the Node20 deprecation warnings (v5 is the first Node24 major; artifact upload/download kept at the same major). - Add a 'summary' job (needs http + selenium, if: always) that downloads every leg's failsafe reports and writes a consolidated Step Summary via summarize-it.js: totals (tests/passing/failing/errors/skipped) and the list of failing tests grouped by class. Rerun-aware (flaky-but-passed tests excluded). Verified locally against a full run's artifacts: 453 total, 429 pass, 22 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ach other While the temporary it-docker-aem push trigger coexists with manual workflow_dispatch, both fired on the same ref and cancel-in-progress made them cancel each other. Add github.event_name to the concurrency group so push and dispatch runs are independent (a newer run of the same event still supersedes). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gen-selenium-matrix.sh now emits exactly 5 legs: one per @tag group (group1..group4, selected via -Dgroups so the whole group runs as a unit) plus one ungrouped leg for the untagged classes - reverting the earlier 4x-per-group split (17 legs) back to simple tag-controlled sharding. The shared prep job (build once, prime image cache once) is unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
All four jobs (prep, integration-test, selenium, summary) now use runs-on: self-hosted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Remove the Linux-only Xvfb/fluxbox/apt-get virtual-display step: the runner has a real logged-in GUI session, so Chrome runs headed natively against the WindowServer (same as the validated local Mac flow) - no virtual display needed. - Bump integration-test/selenium job timeouts (120m/180m) and document that the self-hosted runner is Apple Silicon, so the linux/amd64 AEM image runs under Docker emulation in CI too (not just locally) - author/publish boot alone took ~3.5 min each under emulation in prior local testing. - Update README's platform note: CI is no longer framed as the 'native amd64, authoritative' run since there is currently no native-amd64 execution path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI failed with 'mvn: command not found' (exit 127) - GitHub-hosted ubuntu-latest ships Maven preinstalled, but a self-hosted runner isn't guaranteed to. Add a 'Setup Maven (if missing)' step (via Homebrew, no-op if already present) after 'Setup JDK 11' in every job that invokes mvn (prep, integration-test, selenium). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ons) Runner is switching from bare macOS to an Ubuntu Docker image/container: - Restore the Xvfb + fluxbox virtual display step for the selenium job (no GUI session inside a container, unlike macOS's WindowServer). - Switch 'Setup Maven (if missing)' from Homebrew to apt-get in all three mvn- invoking jobs, with a root-first/sudo-fallback so it works whether the container runs as root or a regular user. - Update comments/README: no longer assert Apple Silicon/GUI-session for CI; flag that the runner container needs its own Docker daemon access (bind- mounted socket or DinD) for docker run/exec/pull/load to work at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Revert the self-hosted-runner-specific adaptations from this session, now that we're running on GitHub-hosted ubuntu-latest again: - runs-on: self-hosted -> ubuntu-latest for all four jobs. - Remove the 'Setup Maven (if missing)' steps (ubuntu-latest ships Maven preinstalled - the self-hosted runner did not). - Simplify the Xvfb + fluxbox step back to plain 'sudo apt-get' (ubuntu-latest has guaranteed passwordless sudo; the root-first/sudo-fallback logic was only needed because the self-hosted container's user/permissions were unknown). - Revert integration-test/selenium timeouts to the values validated earlier on native GitHub-hosted runs (90m/60m, down from 120m/180m padded for the self-hosted runner's uncertain architecture/emulation). - Update comments/README to describe ubuntu-latest again instead of the self-hosted Ubuntu-in-Docker runner. The prep-job cache/artifact sharing, concurrency event-scoping, actions v5 bump, and results-summary job from this session are unrelated to runner choice and are kept as-is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…warning Verified via the actions' own action.yml (using: field) rather than assuming: upload-artifact@v5 and download-artifact@v5/v6 still declare node20 - only v6+ (upload) / v7+ (download) declare node24. The earlier v5 bump fixed checkout/setup-java/cache (already node24 at v5) but not these two. actions/checkout@v5: node24 actions/setup-java@v5: node24 actions/cache@v5: node24 actions/upload-artifact@v5: node20 (v6+: node24) actions/download-artifact@v5/v6: node20 (v7+: node24) Bump both upload-artifact and download-artifact to v7 (same major, both node24; v4+ share the same artifact storage API so cross-version compat is unaffected). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified via a fresh run's logs: after fixing actions/upload-artifact and download-artifact, one Node20 warning remained - 'mikepenz/action-junit-report@v5' targets node20; v6+ targets node24 (checked directly against the action's own action.yml). Bump both usages to v6. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add hybrid runner routing so a single run can spread legs across GitHub-hosted and self-hosted runners (the Dockerized amd64 aem-it runner). A single `runs-on` can't span both pools (ubuntu-latest is a reserved hosted label), so routing is per-job/per-leg, driven by SELF_HOSTED_COUNT (workflow_dispatch input self_hosted_count, else vars.SELF_HOSTED_COUNT, else 0 = all hosted). - prep resolves the count (no GitHub API probe: listing self-hosted runners needs admin scope the built-in GITHUB_TOKEN can't get) and emits http_runs_on + a matrix whose legs each carry runs_on. - gen-selenium-matrix.sh assigns the first N legs to [self-hosted, aem-it], the rest to [ubuntu-latest]. - integration-test and selenium consume the routed runs-on; summary stays hosted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a self-hosted Dockerized runner, run-it.sh runs inside the runner container and
launches AEM as a sibling via the mounted host Docker socket. `docker run -p` then
publishes AEM's ports to the HOST, not to the runner's localhost, so the
localhost:4502/4503 contract that Maven and the Selenium browser rely on breaks -
wait_for_aem timed out after 600s even though AEM was up (verified: from inside the
container localhost:PORT is unreachable, --network container:<runner> reaches it).
Detect in-container execution (/.dockerenv, override AEM_IN_CONTAINER) and start
AEM with --network container:${HOSTNAME} instead of -p, keeping localhost working
for curl, Maven and Chrome. On GitHub-hosted runners / local macOS (bare host) the
behavior is unchanged (publish ports).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- run-it.sh: trap INT/TERM (not just EXIT) so a cancelled CI job tears down its AEM container before SIGKILL instead of orphaning it on the shared self-hosted daemon; disarm the trap on entry to avoid re-entry. Also reap stale AEM orphans attached to THIS runner's netns from a prior hard-killed job (safe with parallel runners - other runners' containers use a different netns). - maven-it.yml: on the http + selenium jobs, skip the ~7GB AEM-image cache restore and docker load when the image is already resident in the daemon (self-hosted reuse). Hosted runners start fresh so behaviour is unchanged; the image tag pins an exact version so "present" implies correct. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The AEM Selenium lib (Network.getRebasedURL) rebases localhost test URLs to the host LAN IP via getFirstLocalIP(), which only accepts 10.*/192.168.* and orElseThrow()s "No value present" otherwise. A Docker-bridge runner is 172.x, so every Selenium test errored in setup (ParameterResolutionException) with Chrome otherwise working. The lib honours an IP env override; since AEM shares our netns, 127.0.0.1 reaches it. Gated on IN_CONTAINER so GitHub-hosted (10.*) is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full @tag group leg finishes in ~7-13 min on GitHub-hosted amd64, but under Rosetta emulation on the self-hosted runner it approached the old 60m limit and got cancelled mid-run (it was passing 114/116 when cut off). Bump to 150m for emulation headroom; hosted legs still finish in minutes so this costs them nothing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each rerun re-launches Chrome and re-runs the test, which is costly under Rosetta emulation on the self-hosted runner (it pushed a @tag group leg past the timeout). One rerun still absorbs most transient UI/Oak-index flakiness; override SEL_RERUN to raise it where needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Exclude the Playwright POC branch from the long HTTP job for both push and pull_request refs so only the isolated browser comparison runs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add Playwright counterparts for component authoring ITs across the remaining test groups and ungrouped cases. Run matched Selenium-only and Playwright-only group jobs with retries disabled to compare the same coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The picker autocomplete can contain both the open-picker trigger and a Coral tag remove button. Scope the Playwright click to the named open-selection button to avoid ambiguous locator failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vladbailescu
changed the title
…uage navigation ITs - route toolbar actions through the robust overlay selection helper - resolve Coral select popovers via aria-controls to avoid strict mode violations - replay multifield reorder with mouse steps, Coral ignores atomic dragTo - reload the editor after policies are created so clientlibs are applied - read generated accordion item node names back from the repository - run carousel keyboard and accordion expansion checks with wcmmode=disabled - match the Selenium expectations for breadcrumb validation and the language navigation placeholder Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…list and TOC ITs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eas, dialog tabs) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Teaser, Tabs, Accordion reorder, Search, StyleTab, Embed, Text, Title and Navigation fixes mirroring the Selenium equivalents. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… untagged ITs - Delete the Selenium (seljup) IT classes and gen-selenium-matrix.sh; the Playwright ITs cover the same tests. - Tag PdfViewer, Separator, Download V1/V2 with playwright-group2 (fastest group on CI) and drop the ungrouped CI job. - gen-browser-matrix.sh derives one job per playwright-groupN tag and fails CI prep if any *IT class is untagged; ItGroupTagTest enforces the same in the module build (surefire). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pin the JUnit report action to its v6 commit and apply the shell and JavaScript reliability recommendations. Skill-Version: aem-sites-ai-toolkit/implement-review-findings v0.7.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Addressed all nine open SonarQube findings in 8536d85, including both inline action-pinning comments (replied and resolved). Shell/JavaScript checks, summary regression checks, matrix generation, and the Maven module build passed locally. No reactions or owner guidance were present on the review threads. Two additional sequential CI runs of this revision will check stability with automatic test reruns disabled; the PR description will record their final outcomes. |
Run four browser groups and HTTP author/publish across SDK, 6.5.24.0 (Java 11), and LTS 6.6.2 (Java 21). Prime each image separately, select classic/cloud packages and product exclusions, verify the runtime product, and keep reports distinct. Add mocked runner routing regression checks. Skill-Version: aem-sites-ai-toolkit/implement-change v0.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Provision core components and IT content in a per-product prepare job, stop the instances, and export a self-contained image consumed by all IT jobs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… wait Prepared images cut per-job setup from ~3.5m to ~1m, but the serial prepare stage raised wall time from 38m to 57m. Keep the 6.5 publish service-pack alignment, bundle activation wait, and per-product failure isolation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- support the Coral 3 Insert Component dialog, reopen multi-selects and wait for the panel selector popover to settle - set navigation structure depth explicitly, merge the clear-asset image tests - exclude Cloud-only NGDM smart crop and PWA service worker tests via IgnoreOn65/IgnoreOnLTS Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…f expanding secrets in run scripts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- keep Playwright versions of workflow, run-it.sh and README; drop Selenium ITs and gen-selenium-matrix.sh - pin org.apache.sling.testing.clients to 3.0.22 (no downgrade vs aem-cloud-testing-clients 1.2.3) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- cache AEM images as zstd (-12 --long=30) archives; log out of the registry after pulls - least-privilege workflow permissions; only cancel superseded pull request runs - save the Maven cache only on a miss and without this repo's SNAPSHOTs; build the parent pom and e2e-selenium-utils from source - fail the browser build on test failures (failsafe verify) and require tests in both report checks - distinct report check names; drop the no-op keep_aem input - run-it.sh: bind AEM ports to 127.0.0.1, check package manager status, keep this repo's bundle by exact version and refresh packages - summarize-it.js: ignore output text, report per leg, list flaky tests - log PdfViewer reloads; fix filter.xml comment and stale README sections Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
- TeaserV1PwIT: wait for the editor's async action title autofill before setting a custom title, and assert the value stuck - maven-it.yml: remove the it-playwright-poc push trigger; PR runs cover the branch and duplicate ref-scoped caches exceeded the 10 GB quota Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- maven-it.yml: resolve the test-all failsafe plugins/providers in prep so test jobs never download them from Maven Central (a transient download failure broke an LTS job); separate M2_CACHE_VERSION to rebuild the Maven cache - PlaywrightAuthorBaseTest: add addChildrenEditorItem, which waits for the new item before titling it; used by Tabs, Accordion and Carousel tests (an empty required title blocked Done on SDK) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- maven-it.yml: grant checks: write only to the jobs that publish JUnit reports (SonarCloud) - Replace deprecated RandomStringUtils.randomAlphabetic, add missing @OverRide annotations, drop an unread local and an unused parameter (CodeQL) - EmbedV1PwIT: assert the first rendered match; Twitter's widget briefly keeps the original blockquote next to the rendered tweet Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Migrate the browser integration tests from Selenium to Playwright and run them against the Dockerized AEM SDK locally and in GitHub Actions.
Add 44 Playwright IT classes covering all four browser groups, with shared authoring/component fixtures for editor dialogs, Coral controls, policies, page properties, assets, nested components, and panel reordering.
Stabilize asynchronous editor operations, autocomplete selection, tag/asset pickers, search fixtures, multifields, and native panel dragging. Retry the Teaser action-link suggestion query when newly created pages are not yet suggested.
Remove the 62 legacy Selenium test classes. Retain the historical
e2e-seleniummodule name and shared utility dependencies used by the Playwright tests.Move Download V1/V2, PDF Viewer, and Separator into
playwright-group2, the fastest group before redistribution. Remove the ungrouped job.Generate one CI job per
playwright-groupNtag. Fail matrix generation on untagged IT sources, and enforce grouping in normal Maven builds withItGroupTagTest.Build deployable packages once in
prep; prime each AEM image in a separate cache job and give every test job its own instance. Upload product-specific Failsafe reports, Playwright traces, and AEM logs, and aggregate results.Keep Docker provisioning shared between local and CI runs, including SDK bundle deduplication to avoid product/repository Core Components cross-wiring. Import tags before DAM fixtures so FileVault preserves asset tag references.
Address all nine currently reported SonarQube findings: pin both JUnit report action references to the verified v6 commit, remove the unused shell variable, use Bash
[[, use Node-prefixed built-ins andNumber.parseInt, separate the summary assignment, and sort class names withlocaleCompare.Run all four browser groups plus HTTP author/publish against SDK, AEM 6.5, and LTS (12 browser jobs and 3 HTTP jobs). New images:
docker-adobe-cif-release.dr-uw2.adobeitc.com/circleci-aem:6.5.24.0-openjdk11anddocker-adobe-cif-release.dr-uw2.adobeitc.com/circleci-aem-lts:6.6.2-openjdk21.Verify the product from runtime run modes/product info before provisioning. Install classic packages on 6.5/LTS, apply only the matching product's exclusions, and run LTS without
IgnoreOn65exclusions. Keep SDK bundle deduplication SDK-only.Add
test-run-it.shregression checks for image/product inference, runtime mismatch rejection, package/exclusion routing, explicit empty exclusions, browser host ports, and HTTP author/publish wiring; run them in CI prep.Align the 6.5 publish with the author's service pack: the image ships an SP24 author but a GA 6.5.0 publish.
align_publish_with_authorcopies theaem-service-pkgfrom the author, installs it on publish, and waits for matching product versions and settled bundles. Without it, Core Components can't resolve on publish (TeaserIT500).Wait for provisioning (
wait_for_provisioning) on author and publish before tests start. Isolate failures per product, so one product's cache job can't skip the others' tests.Fix cross-version Playwright failures:
insertableComponent).openSelectListidempotent, because Coral 3 multi-selects close after each pick.Add an
IgnoreOnLTSHTTP category and browser tag. Exclude the Cloud-only NGDM smart crop test and the PWA service worker test (PageIT.testServiceWorkerConfiguration) on 6.5/LTS.Merge of
mainand PR-3093 late-review fixesmain(includes Run http + Selenium ITs against Dockerized AEM Cloud SDK on GitHub Actions; fix flaky ITs #3093, [SITES-49051] [Core Components] Sanitize authored panel title before rendering in the panel selector #3088, GRANITE-71475: ContentAI Supported Search v2 - tabbed Search Results / AI Mode #3075). Drop the Selenium tests changed on main, keep this branch's workflow/runner, and bumporg.apache.sling.testing.clientsto 3.0.22, matchingaem-cloud-testing-clients1.2.3.zstd -12 --long=30). The restore is lookup-only, the save runs only on a miss, anddocker logoutruns after each pull. Archive sizes: 6.5 5.04 GB to 2.79 GB, SDK 2.41 GB to 2.09 GB, LTS 2.23 GB to 1.52 GB.pushtrigger. Caches are scoped per ref, so push and PR runs each stored about 6.4 GB, exceeding the 10 GB quota and evicting the m2 caches. PR runs also read caches frommain.permissions; cancel in progress only for PRs; dropkeep_aem; save m2 only on a miss and without this repo's SNAPSHOTs;require_testson the report steps; clearer check names.run-it.sh<status code="200">from the package manager.allpackage, then refreshes packages and waits for bundles to settle, with no stop/start or sleeps.e2e-selenium-utils.summarize-it.js: ignore failure text inside CDATA,system-outand rerun elements; report per leg; list flaky tests (passed on rerun) separately.e2e-selenium/pom.xml: bind failsafeverifyso test failures fail the build.test-allfailsafe plugins and JUnit providers inprep, so test jobs never download them from Maven Central. A transient download failure had broken an LTS job. The Maven cache now has its ownM2_CACHE_VERSION.addChildrenEditorItem, used by the Tabs, Accordion and Carousel tests. It waits for the children editor to append the new item before titling it. Otherwise the previous item could be titled instead, leaving the new required title empty and blocking Done (seen on SDK).checks: writeonly to the report-publishing jobs (SonarCloud); replace the deprecatedrandomAlphabetic, add missing@Overrideannotations, and drop an unread local and an unused parameter (CodeQL).EmbedV1PwIT: assert the first rendered match, because Twitter's widget briefly keeps the original blockquote next to the rendered tweet.Possible product issue (not fixed here): on 6.5/LTS the Navigation dialog renders the
structureDepthnumberfield default EL expression literally instead of evaluating it, so saving without editing posts an empty depth.Experiment (reverted): pre-provisioned per-product images (
docker export/importafter installing Core Components) cut per-job setup from about 3.5 to about 1 min. But the serial prepare stage (6.5 took about 20 min) and the 4.4 GB download per job raised wall time from 38 to 57 min (run 37028369958). Only the publish alignment and the provisioning waits were kept.The browser uses host-installed Chrome. Existing
WITH_SELENIUM/SEL_*runner knobs remain for compatibility; select groups withSEL_GROUPS=playwright-groupN. HTTP ITs now run on this branch too, on all three products.Validation
Latest revision
a4b7a2c70: CI run 37117830413 - passed all 15 IT jobs on SDK, 6.5 and LTS. The previous revision256630d7cpassed in run 37112925731; its rerun failed once on the Twitter embed race fixed above. Caches fit in the quota (about 7.7 GB including the m2 cache).Local SDK, from the repo root:
SEL_IT_TEST='EmbedV1PwIT,TeaserV2PwIT,ListV2PwIT,ListV3PwIT,PageV3PwIT,TabsV1PwIT'- passed, 89 tests.WITH_SELENIUM=true SEL_RERUN=0 AEM_AUTHOR_PORT=4504 SEL_IT_TEST='TabsV1PwIT,AccordionPwIT,CarouselPwIT,TeaserV2PwIT' bash testing/it/docker/run-it.sh- passed, 65 tests.SEL_IT_TEST='TeaserV*PwIT'- passed, 33 tests.mvn -o -f testing/it/{http,e2e-selenium}/pom.xml verify -Ptest-all -Dit.test=NoSuchPrimeIT -Dfailsafe.failIfNoSpecifiedTests=false -DfailIfNoTests=false- passed without AEM (the plugin-priming command).bash testing/it/docker/test-run-it.sh- passed. actionlint onmaven-it.yml- passed.mvn -o -q -f testing/it/e2e-selenium/pom.xml clean test- passed on the review-fix revision, including the grouping guard.bash -n testing/it/docker/run-it.sh testing/it/docker/gen-browser-matrix.sh- passed.node --check testing/it/docker/summarize-it.js- passed.bash testing/it/docker/gen-browser-matrix.sh- passed; generated exactly four Playwright jobs, no Selenium or ungrouped jobs.git diff --check- passed.Summary fixture regression checks - passed for counters, alphabetical class ordering, and exclusion of successful flaky retries from the failure list.
From
testing/it/e2e-selenium:mvn -o verify -Ptest-all -Dgroups=playwright-group3 -DexcludedGroups=failing,nested,IgnoreOnSDK -DfailIfNoTests=false -Dit.test='TeaserV1PwIT,TeaserV2PwIT'- passed locally against AEM SDK: 33 tests, no failures or errors.Grouping negative checks - adding a temporary untagged
DummyPwITcaused bothmvn -o -q testandbash ../docker/gen-browser-matrix.shto fail; the temporary class was removed andmvn -o clean testpassed afterward.CI run 36975073434 - passed all four Playwright groups on
0d4ca1d39, with reruns disabled. HTTP ITs were intentionally skipped on this branch.Two sequential attempts of run 37006941647 on
8536d8517: attempt 1 failed in group2 (CarouselPwIT.testReorderItem, dialog visibility timeout); attempt 2 passed all four groups. These are not three consecutive green runs.SonarQube reanalysis of the review-fix revision reported zero open issues.
bash testing/it/docker/test-run-it.sh- passed for all three products and explicit empty exclusion overrides.docker run --rm -v "$PWD:/repo" -w /repo rhysd/actionlint:latest -shellcheck= .github/workflows/maven-it.yml- passed.docker run --rm -v "$PWD:/repo" -w /repo koalaman/shellcheck:stable -S error testing/it/docker/run-it.sh testing/it/docker/test-run-it.sh- passed.AEM_IMAGE=docker-adobe-cif-release.dr-uw2.adobeitc.com/circleci-aem-lts:6.6.2-openjdk21 AEM_AUTHOR_PORT=4504 AEM_PUBLISH_PORT=4505 WITH_PUBLISH=true IT_TEST='ComponentsIT,SeoIT' KEEP_AEM=true QP_VM_OPTIONS='-Xmx3g -XX:MaxMetaspaceSize=1g -Djava.awt.headless=true' bash testing/it/docker/run-it.sh- passed locally: both LTS author/publish products verified, classic packages installed, 13 HTTP tests passed.From
testing/it/e2e-selenium:mvn -o verify -Ptest-all -Dgroups=playwright-group2,playwright-group4 -DexcludedGroups=failing,nested -Dit.test='ListV2PwIT,PageV3PwIT#testAdvancedSeoPageProperties' -Dsling.it.instance.url.1=http://localhost:4504 -Dgranite.it.author.url=http://localhost:4504- passed locally on LTS: 16 Playwright tests. The local LTS container was removed afterward.AEM 6.5 local startup failed twice inside QuickProvider before tests began. Native-amd64 GitHub validation of the full 6.5 suite is pending; no local 6.5 pass is claimed.
From
testing/it/e2e-selenium, against local LTS on port 4504:mvn -o verify -Ptest-all -Dgroups=playwright-group1,playwright-group2,playwright-group3,playwright-group4 -DexcludedGroups=failing,nested,IgnoreOnLTS -Dit.test='AccordionPwIT,TabsV1PwIT,CarouselPwIT'- passed: 45 tests. Earlier LTS runs of NavigationV1/V2 and ImagePwIT passed: 52 tests.CI run 37046428549 on
21cc2beb4- passed all 15 jobs (4 Playwright groups and HTTP author/publish on each of SDK, 6.5, and LTS) in about 26 min wall time, with no flaky retries recorded.Stability check: three sequential attempts of CI run 37050909756 on
9b085f155- all passed (15/15 jobs each, about 30 min per attempt). No test failures and no flaky retries were found in the Failsafe reports (1,559 test cases per attempt).SonarQube: the "secrets expanded in run blocks" findings were fixed in
9b085f155; the quality gate now passes.Timing comparison
Before removal, run 36892587993 ran matching Selenium and Playwright groups separately. These are whole job durations, including SDK provisioning/reporting, not browser execution alone:
Cross-version run 37046428549: HTTP 7 min (SDK/LTS) and 12 min (6.5); Playwright jobs 18–22 min each; about 26 min wall time for all 15 jobs.
After moving ungrouped tests into group2, the successful Playwright-only run above took group1 23m 01s, group2 22m 49s, group3 23m 44s, and group4 21m 51s.
Generated by aem-sites-ai-toolkit: implement-review-findings v0.7.1, update-pr-description v0.3.1, commit-and-pr v0.4.3, implement-change v0.4.1