Skip to content

chore(jp): add JP region E2E workflows mirroring EU - #1851

Merged
pranav-new-relic merged 6 commits into
mainfrom
feature/jp-region-tests
Jul 7, 2026
Merged

pranav-new-relic merged 6 commits into
mainfrom
feature/jp-region-tests

Conversation

@pranav-new-relic

@pranav-new-relic pranav-new-relic commented Jun 11, 2026 •

Copy link
Copy Markdown
Member

Summary

Wires Japan as a first-class region in the newrelic-cli CI test path, mirroring the existing EU pattern.

What's added:

  • .github/workflows/e2e-jp.yml — clone of e2e-eu.yml. Pulls ghcr.io/newrelic/deployer:latest, fetches test/definitions-jp/smoke from open-install-library, and runs against the JP test account via the GIT_DEPLOYER_DOCKER_USER_CONFIG_JP secret. Same trigger conditions as EU (PR + push to main).
  • .github/workflows/cleanup-jp.yml — clone of cleanup-eu.yml; reaps resources after End to end testing (JP) and Release workflows.
  • .github/workflows/e2e-dp-jp.yml — clone of e2e-dp-eu.yml (kept inactive, mirroring EU's currently-inactive DP variant).

The CLI's own region handling already covers JP (via newrelic-client-go's region.JP constant + the JP branch in internal/install/execution/platform_link_generator.go). No Go source changes are required for this rollout.

Prerequisites — all satisfied ✅

# Prereq Status
1 Deployer jp: block under newRelicUrls in src/app_config.yml ✅ Merged as deployer #32 on 2026-06-11 (originally drafted as #31, which was closed and superseded by #32).
2 ghcr.io/newrelic/deployer:latest refreshed with JP-aware image ✅ Published — verified empirically by CI provisioning EC2s in ap-northeast-1 and passing 36/36 checks.
3 OIL test/definitions-jp/smoke/ populated ✅ Merged as open-install-library #1377 on 2026-06-22 (1d9923d8 on main). 6 JP smoke test definitions present (awslinux2-infra, awslinuxarm64-infra, debian11-infra, nginx-linux2-ami, ubuntu20-infra, windows2019).
4 GitHub repo secret GIT_DEPLOYER_DOCKER_USER_CONFIG_JP ✅ Present in this repo since 2026-06-11; last updated 2026-06-16 to fix a JSON-shape issue where the secrets block needed to be nested under credentials.secrets.* (not root secrets.*) for the deployer's [credential:secrets:KEY] placeholder substitution to resolve.
5 gitdeployerCanada keypair imported into AWS ap-northeast-1 ✅ Imported — verified empirically by successful JP EC2 launches in CI.

Secret JSON shape (for reference)

{
  "credentials": {
    "aws": {
      "apiKey": "<AWS access key>",
      "secretKey": "<AWS secret key>",
      "secretKeyPath": "/mnt/deployer/configs/gitdeployerCanada.pem",
      "region": "ap-northeast-1",
      "secretKeyName": "gitdeployerCanada"
    },
    "newrelic": {
      "licenseKey": "<jpx*…NRAL ingest license key>",
      "nrPersonalApiKey": "<NRAK-… JP user API key>",
      "accountId": "<JP test account ID>",
      "nrRegion": "JP",
      "nrOrganizationId": "<JP org UUID>"
    },
    "secrets": {
      "nrHostFleetId": "<JP host fleet UUID>",
      "nrWindowsHostFleetId": "<JP Windows fleet UUID>"
    }
  }
}

Verification

  • CI: 36/36 checks pass on the latest push (2d9b3baf). The JP e2e workflow provisions EC2s in ap-northeast-1, runs the 6 smoke test definitions from test/definitions-jp/smoke/ against ghcr.io/newrelic/deployer:latest with the JP secret, and tears down cleanly.
  • Prior runs on this branch surfaced a transient upstream Windows infra-agent install regression (US/EU/JP windows* tests all timing out identically at ==> Installing Infrastructure Agent); resolved itself between 2026-06-22 and 2026-07-06. No code changes in this PR touched the Windows install path. Context: issuecomment-4768646167.

Companion PRs

Wires Japan as a first-class region in the CI test path.

- e2e-jp.yml — clone of e2e-eu.yml, pulls test/definitions-jp/smoke
- cleanup-jp.yml — clone of cleanup-eu.yml
- e2e-dp-jp.yml — clone of e2e-dp-eu.yml (kept inactive, like EU)

Region routing already supported by newrelic-client-go's region.JP
constant; no Go source changes needed.

Prerequisites: deployer PR newrelic-experimental/deployer#31
must merge and ghcr.io/newrelic/deployer:latest must be republished;
OIL companion PR newrelic/open-install-library#1377
must merge so test/definitions-jp/smoke/ exists; GitHub repo secret
GIT_DEPLOYER_DOCKER_USER_CONFIG_JP must be added (JP test account 8155661,
nrRegion: "jp", aws.region: "ap-northeast-1"); import gitdeployerCanada
keypair (public half) into AWS region ap-northeast-1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverted before PR review. Lets us validate the CLI's JP E2E path
against the OIL feature branch (which has definitions-jp/) without
merging OIL #1377 first.
Validation run completed successfully (all 6 jobs green); reverting
to upstream-clean state so PR is review-ready.
pranav-new-relic added a commit that referenced this pull request Jun 16, 2026
…olve CVEs

Resolves CVEs flagged in NR-576378 (newrelic-cli v0.112.10) plus the
broader CVE list from the daily Trivy/Grype scan.

Changes:

- go directive: 1.26.3 -> 1.26.4
  Fixes (stdlib): CVE-2026-42504, CVE-2026-42507, CVE-2026-27145,
  GO-2026-5037, GO-2026-5038, GO-2026-5039.

- golang.org/x/net: v0.38.0 -> v0.56.0 (latest)
  Fixes: CVE-2026-39821 (Critical, idna), CVE-2026-33814 (High, http2),
  GO-2026-5024, GO-2026-5025, GO-2026-5027/8/9/30, GO-2026-4440/1.

- golang.org/x/crypto (indirect): v0.51.0 -> v0.53.0 (latest)
- golang.org/x/exp: digest -> latest (matches renovate PR #1769)
- Companion bumps via go mod tidy:
    x/term v0.30.0 -> v0.44.0
    x/text v0.23.0 -> v0.38.0
    x/sync v0.12.0 -> v0.21.0
    x/sys  v0.31.0 -> v0.46.0  (covers GO-2026-5024)

- Base image: alpine 3.23.4 -> 3.24.0
  Fixes: ~24 libcrypto3/libssl3 CVEs (CVE-2026-45447 High;
  CVE-2026-34182/3, CVE-2026-42764, CVE-2026-45445 Medium;
  CVE-2026-34180/1, CVE-2026-42766/7/8/9/70, CVE-2026-45446,
  CVE-2026-7383, CVE-2026-9076 Low).

CI note: EU-windows2019 and infra-win2016 e2e jobs are red on this PR.
The failure is not caused by these dep bumps; it reproduces on every
PR that modifies go.mod/go.sum after 2026-06-11 ~08:19 UTC, when the
ghcr.io/newrelic/deployer:latest image was repushed without pinned
floating apt/pip/Galaxy deps. PRs that don't touch go.mod (e.g. #1851)
pass Windows e2e under the same deployer image. Tracking separately;
not a blocker for this security PR.
pranav-new-relic added a commit to newrelic/open-install-library that referenced this pull request Jun 22, 2026
## What this PR does

- **JP region test infrastructure** — clones `test/definitions-eu/` → `test/definitions-jp/` and adds matching workflows (`smoke-jp.yml`, `nonregression-jp.yml`, `cleanup-jp.yml`, `*-dp` variants) with the region rewrites (URLs, `*_JP` secret refs, `open-install-library-e2e-jp` identity name, `ap-northeast-1`).

- **Region-aware `validation.yml`** — splits the per-PR test-deploy matrix by path: `definitions/*` → US, `definitions-eu/*` → EU NerdGraph + EU secret, `definitions-jp/*` → JP NerdGraph + JP secret. Adds `create-system-identity-{eu,jp}` and `cleanup-system-identity-{eu,jp}` jobs; matrix builder now uses `--diff-filter=AMR` so deleted files don't enter the matrix.

- **Cross-region coverage parity** — added modern infra-agent OSes (Debian 11–13, Ubuntu 20/22/24, RHEL 9/10, Amazon Linux 2023, SUSE 15.4/5/7), OHI (Redis, RabbitMQ, Elasticsearch), and modern PHP (Ubuntu 22 fpm-wordpress) to both EU and JP. Bumped Debian 10→12 where deprecated, replaced Oracle 8 with RHEL 9, bumped redhat9-infra to `t3.small` for boot reliability, bumped dotnet ASP.NET runtime 5.0→8.0.

- **CrowdStrike removal** — deletes the no-op install stub + its only test consumer + the env-block references in `nonregression*.yml` (companion to deployer #33; [NR-520645](https://new-relic.atlassian.net/browse/NR-520645)). Backing deployer bundle is being removed separately; repo `CROWDSTRIKE_*` secrets can be deleted afterwards.

- **Flake/transient resilience** — retry-wrappers around `docker pull newrelic/deployer:latest`, the Apache `a2enmod` step in the dotnet deploy (start.yml), and `apt install` on Ubuntu 18 EOL mirrors in PHP-WordPress (configure.yml). Same `register`/`until`/`retries`/`delay` pattern used for the mvn retry in github.com/newrelic/demo-javatron/issues/48.

- **End-to-end NRQL verification** — run [`27671298270`](https://github.com/newrelic/open-install-library/actions/runs/27671298270) is fully green (94/96, 0 failures). AGENT entities + `FLEET → AGENT (CONTAINS)` relationships were observed via `NrAuditEvent` in EU NR (account 6729572) and JP NR (account 8163026), each inside the matching test window. Evidence in the [PR comment](#1377 (comment)).

- **Synced with `main`** — pulled agent-control 1.16.1 → 1.17.0 from #1381 (clean auto-merge).

## Prerequisites

1. newrelic-experimental/deployer#32 (merged) + image republish so `newrelic/deployer:latest` includes the `jp:` block under `newRelicUrls`.
2. GitHub repo secrets: `GIT_DEPLOYER_DOCKER_USER_CONFIG_JP` (with `credentials.secrets.{nrHostFleetId, nrWindowsHostFleetId}` populated for the JP fleets) — verified live in run 27671298270.
3. `gitdeployerCanada` keypair imported into AWS region `ap-northeast-1`.

## Companion PRs

- Deployer JP region: newrelic-experimental/deployer#32 (merged)
- Deployer CrowdStrike removal: newrelic-experimental/deployer#33
- newrelic-cli JP workflows: newrelic/newrelic-cli#1851


[NR-520645]: https://new-relic.atlassian.net/browse/NR-520645?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
OIL #1377 (Prereq 3 from the PR body) landed in
newrelic/open-install-library@main as 1d9923d8 today
(feat(ci): JP region test infrastructure + region-aware
validation.yml). All four prerequisites for this PR are now
satisfied; retrigger CI so reviewers can see the current state.
@pranav-new-relic
pranav-new-relic marked this pull request as ready for review June 22, 2026 12:50
Pulls in 4 days of CLI maintenance commits since 2026-06-18:
  bcb2b9c, f882453, 48c13ee  scoop/changelog for v0.112.12 / v0.112.13
  9fa45de  chore(workflow): auto-open PR when newer alpine base detected
  5440fd5  fix(deps): bump alpine 3.24.0 -> 3.24.1
  69181d5, 96e1242  scoop/changelog for v0.112.11
  dbd9cc1  fix(deps): bump golang.org/x/net (NR-576378 CVEs)

Auto-merged cleanly — no overlap with the e2e-jp/cleanup-jp/e2e-dp-jp
workflows added on this branch.
Pulls in the 4 new release commits since yesterday's sync:
  3966a2e, 5ed8a55  scoop/changelog for v0.112.15
  1bf7b49, 18c2c38  scoop/changelog for v0.112.14

All release-metadata commits — no code changes that overlap this
branch's e2e-jp / cleanup-jp / e2e-dp-jp workflow additions.
@pranav-new-relic
pranav-new-relic merged commit 9633e99 into main Jul 7, 2026
37 checks passed
@pranav-new-relic
pranav-new-relic deleted the feature/jp-region-tests branch July 7, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants