Repository navigation
[9.4](backport #8276) Fix doubled route metrics and missing request logs when OpAMP is enabled - #8294
Conversation
|
Cherry-pick of af0dc1e has failed: To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
|
This pull request is merged. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved merge conflicts, missing imports, and undefined metric and agent-tag symbols prevent the API and affected test packages from compiling.
5 open issues
What changed in this PR
Backports #8276 to 9.4 to address #8167: duplicate route metrics and missing rejection logs when OpAMP is enabled. The change places OpAMP behind Fleet Server’s shared API middleware.
Changes:
- Registers
/v1/opampin the main router and removes the outer router’s duplicate limiter. - Adds unit and integration coverage for request counts, logging, and rate-limit rejections.
- Adds a changelog entry describing the fix.
| File | Description |
|---|---|
internal/pkg/server/fleet_integration_test.go |
Adds route-monitoring integration coverage; also includes incoming agent-tag tests. |
internal/pkg/api/server.go |
Passes the OpAMP handler to the main router instead of wrapping it. |
internal/pkg/api/server_test.go |
Adds route-monitoring unit coverage and an incoming TLS reload test. |
internal/pkg/api/router.go |
Registers OpAMP behind shared middleware; includes a throttle-counting helper. |
internal/pkg/api/router_test.go |
Adds throttle-counter tests and lint-suppression comments. |
changelog/fragments/1795003967-fix-doubled-route-metrics-and-missing-request-logs-when-opamp-is-enabled.yaml |
Documents corrected route metrics and rejection logging. |
Review effort: Balanced
|
Dependency review finished without posting an analysis comment. If you expected a review, push a new commit to this pull request (or close and reopen it) to try again. Workflow run: https://github.com/elastic/fleet-server/actions/runs/44521343527 |
TL;DRBuildkite failed due to unresolved cherry-pick conflict markers committed in backport SHA Remediation
Investigation detailsRoot CauseThe backport commit message itself reports cherry-pick conflicts, and the patch for SHA Evidence
Verification
Follow-up
What is this? | From workflow: Observability Agentic Workflow — Status Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
9.4 serves OpAMP through a second router and does not have throttleWithCount. Keep the single-router OpAMP handler and drop the connection-rejection counter and the unrelated TLS reload test that the cherry-pick pulled in. Drop the unused staticcheck nolint so the touched integration test file passes lint. Co-authored-by: Cursor <cursoragent@cursor.com>
…egration test Lint checks the whole file because this backport changes it. NewClient and Config are deprecated in the v9 client. Co-authored-by: Cursor <cursoragent@cursor.com>

What is the problem this PR solves?
When the OpAMP endpoint is enabled, Fleet Server applies the route limits two times to every request. The
activeandtotalroute metrics, such ashttp_server_routes_checkin_active, count each request two times.How does this PR solve the problem?
Fleet Server now serves the
/v1/opampendpoint from the same API router as the other endpoints, behind the same middleware. Each request goes through the route limits one time so the route metrics count each request one time. Requests that a route limit rejects now show in the request log and in APM with their status code.OpAMP requests now also go through the request log, APM and the
server.limits.max_connectionslimit like the other endpoints.How to test this PR locally
Run the following tests:
Design Checklist
I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.Checklist
I have commented my code, particularly in hard-to-understand areasI have made corresponding changes to the documentationI have made corresponding change to the default configuration files./changelog/fragmentsusing the changelog toolRelated issues
This is an automatic backport of pull request #8276 done by [Mergify](https://mergify.com).