[AIT-1106] fix(uts): correct objects proxy specs and harden proxy rule/timeout guidance#501
Merged
Merged
Conversation
…uidance
Fixes found while evaluating the derived ably-java ObjectsFaultsTest against
the sandbox through uts-proxy:
- objects_faults.md: restructure the RTO20e test to the RTO20e1 sequence
(mutation in flight while SYNCING, then channel enters FAILED -> 92008).
The previous steps mutated after FAILED, which per RTO26b fails with 90001
and never reaches publishAndApply - confirmed against objects-features.md,
ably-js source and both ably-js/ably-java behaviour.
- objects_faults.md: rule match.action values must be strings ("20"), not
JSON numbers (uts-proxy rejects numbers with HTTP 400); bound every await
with a timeout; extend the action-number table (ACK, ERROR); align the
Common Cleanup with multi-client tests.
- connection_resume.md: RTN15g rule used invalid action type "refuse"
(silently passes through) - corrected to "refuse_connection".
- helpers/proxy.md: document the match.action string format, the v0.3.0
action-name resolution limit (names beyond AUTH need numeric strings), and
add a wall-clock timeout convention.
- docs: add wall-clock timeout guidance for integration specs
(writing-derived-tests.md new section, integration-testing.md and
writing-test-specs.md pointers) - virtual-time test frameworks (e.g.
kotlinx-coroutines runTest) fire spec timeouts instantly unless waits run
on the real clock.
- PLAN.md: update the objects_faults.md summary row to the corrected RTO20e
scenario.
sacOO7
force-pushed
the
fix/uts-proxy-liveobjects-spec
branch
from
c077c25 to
233d3a3
Compare
proxy.md is cross-module infrastructure documentation, not a realtime helper: it is consumed by the realtime (7 proxy specs), rest (rest_fallback.md) and objects (objects_faults.md) modules plus both process docs, unlike the genuinely module-local helper specs (mock_websocket.md, mock_http.md, standard_test_pool.md) which are each used only by their own module. Its old location forced cross-module reach-ins and implied it was realtime-specific. - git mv uts/realtime/integration/helpers/proxy.md -> uts/docs/proxy.md, leaving a one-line stub at the old path so external deep links keep resolving - update all 19 references across 12 files, normalising the pre-existing stale `uts/test/` prefix carried by the realtime proxy specs' references and replacing two mentions of the long-gone in-repo `uts/test/proxy/` directory with a link to the ably/uts-proxy repo - update the directory-tree diagrams in uts/README.md, docs/integration-testing.md and docs/writing-test-specs.md - adjust proxy.md's own cross-reference to sibling style (writing-derived-tests.md) No content changes to the spec itself beyond that one internal reference. Note: ~80 unit/proxy spec files still carry stale `uts/test/...` legacy paths unrelated to proxy.md — left for a separate cleanup PR.
sacOO7
force-pushed
the
fix/uts-proxy-liveobjects-spec
branch
from
ab649f3 to
3dbee55
Compare
sacOO7
added a commit
to ably/ably-java
that referenced
this pull request
Jul 4, 2026
…rrected spec Proxy session creation failed with HTTP 400 for any rule using a frame-action match: the uts-proxy's MatchConfig.match.action is a Go string (action name or numeric string), but the rule builders serialised messageAction as a JSON number. wsFrameToClientRule/wsFrameToServerRule now send it as a string. Add withRealTimeout to the shared infra: inside runTest, a bare withTimeout measures virtual (kotlinx.coroutines.test) time, which fast-forwards while the test idles - a timeout wrapping a real network await fires instantly. All real-network waits in integration tests must run on a real-thread dispatcher, matching the existing awaitState/awaitChannelState/pollUntil helpers. Rework ObjectsFaultsTest against the corrected source spec (objects/integration/proxy/objects_faults.md, ably/specification#501): - bound every object.get() with withRealTimeout instead of virtual-time withTimeout (all five tests previously failed instantly) - RTO7/RTO8: fetch the channel handle once and reuse it - re-calling Channels.get() with mode options on an attached channel throws - RTO20e: exercise the RTO20e1 sequence (mutation in flight while SYNCING, then the channel enters FAILED -> 92008/400 with the injected 90000 channel error as cause), mirroring ably-js's sync-wait rejection test; the spec's original mutate-after-FAILED steps hit the RTO26b precondition (90001) instead and were fixed at source - implement the spec's Common Cleanup: close clients and await CLOSED (with the connection-state guard) before tearing down the proxy session Update proxy spec path references (ObjectsFaultsTest/AuthReauthTest KDocs, uts-to-kotlin skill) to uts/docs/proxy.md per the relocation in ably/specification#501; uts/README.md's GitHub URLs are left until that PR reaches main. See PROXY_UTS_SPEC_FIX.md for the full evidence chain. All five ObjectsFaultsTest tests pass against the sandbox through uts-proxy.
There was a problem hiding this comment.
Pull request overview
This PR updates the UTS proxy-based integration specs and supporting documentation to (1) correct the Objects proxy fault scenario for RTO20e/RTO20e1, (2) harden proxy rule format guidance (notably match.action string typing and action-name limitations), and (3) clarify that integration/proxy timeouts are wall-clock to avoid virtual-time framework pitfalls in derived SDK tests.
Changes:
- Corrects
objects_faults.mdto test the RTO20e1 “in-flight operation fails during sync-wait” scenario (92008) rather than mutating after FAILED (which should be 90001 per write preconditions). - Fixes/clarifies uts-proxy rule formats and action typing, including correcting an invalid
refuseaction type and documentingmatch.actionrequirements/limitations. - Adds explicit wall-clock timeout guidance across docs and bounds previously unbounded awaits in the Objects proxy spec.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| uts/rest/integration/proxy/rest_fallback.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/rest_faults.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/presence_reentry.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/heartbeat.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/connection_resume.md | Updates proxy reference; fixes proxy rule action type to refuse_connection. |
| uts/realtime/integration/proxy/connection_open_failures.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/channel_faults.md | Updates proxy infrastructure reference path. |
| uts/realtime/integration/proxy/auth_reauth.md | Updates proxy reference and points to external uts-proxy repo. |
| uts/README.md | Updates tree/docs pointers to the cross-module proxy spec location. |
| uts/objects/PLAN.md | Updates summary row to reflect corrected RTO20e scenario wording. |
| uts/objects/integration/proxy/objects_faults.md | Corrects RTO20e/RTO20e1 test flow; fixes match.action typing; adds wall-clock waits/timeouts and cleanup guidance. |
| uts/docs/writing-test-specs.md | Updates proxy doc references; clarifies integration timeout semantics. |
| uts/docs/writing-derived-tests.md | Adds wall-clock timeout guidance for derived integration/proxy tests (virtual-time trap). |
| uts/docs/proxy.md | Documents match.action as string (and name-resolution limits); adds wall-clock timeout convention. |
| uts/docs/integration-testing.md | Updates proxy doc reference; clarifies timeout strategy as wall-clock. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sacOO7
changed the title
Member
|
@sacOO7 can you also check please whether or not these changes require a change in ably/ably-js#2219 ? |
Collaborator
Author
There was exact same recorded deviation for proxy tests. Also, integration tests were only running in |
sacOO7
added a commit
to ably/ably-js
that referenced
this pull request
Jul 6, 2026
…son+msgpack variants - objects_faults: restructure the RTO20e proxy test to the corrected spec sequence (ably/specification#501) — force SYNCING via injected ATTACHED with HAS_OBJECTS, mutate while SYNCING, wait for the publish ACK in the proxy log, then inject channel ERROR and assert the pending operation fails with 92008 / statusCode 400 / cause 90000 (RTO20e1). Removes the 90001 DEVIATION, which documented the spec error now fixed at source. - objects_gc: replace the deviation comments with the real spec references — undefined reads are RTLM5d2h-compliant (spec says undefined/null), and the timer-based GC sweep is covered at the unit tier (RTO10/RTO10b1/RTLM19 unit tests), so only observable tombstone semantics belong here. - objects_sync: align the RTO4 subscribe-only test title with the corrected spec wording and drop its DEVIATION comment. - Add describeWithProtocols helper and run objects_lifecycle, objects_sync and objects_gc once per protocol variant (json, msgpack) as their specs require, without any command-line or environment parameter. All suites verified against the nonprod sandbox: objects_faults 5/5, lifecycle/sync/gc 24/24 across both protocol variants.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
While evaluating the derived ably-java test suite (
ObjectsFaultsTest) against the real sandbox throughuts-proxy, every test inobjects/integration/proxy/objects_faults.mdfailed. Root-causing those failures surfaced one genuine spec error (RTO20e), several spec/tooling format mismatches that fail silently or with opaque errors, and a recurring translation trap around timeouts. This PR fixes the specs at source and adds the guidance so other deriving SDKs don't rediscover these the hard way.Every behavioural claim below was cross-validated against the features spec (
objects-features.md), ably-js source (the reference SDK), and a passing ably-java derived test run against the sandbox.Changes
1.
objects/integration/proxy/objects_faults.md— RTO20e spec error (the substantive fix)The test
objects/proxy/RTO20e/publish-fails-on-channel-failed-0mutated after the channel was already FAILED, yet asserted error 92008:publishAndApply.throwIfInvalidWriteApiConfiguration→invalidStateError()= 90001/400; the only 92008 is the sync-wait rejection inpublishAndApply(). Its own test ("publishAndApply rejects when channel state changes during sync wait") never mutates after FAILED. ably-java behaves identically.objects/unit/RTO20e1) already describe the correct sequence — the proxy steps were the outlier.The steps are restructured to the RTO20e1 sequence: force SYNCING via an injected ATTACHED carrying HAS_OBJECTS (
flags: 128, RTO4c), mutate while SYNCING, wait for the publish ACK in the proxy log plus a brief real-time yield, inject the channel ERROR, and assert the pending operation fails with 92008 / statusCode 400 / cause 90000. A note explains the RTO26b-vs-RTO20e1 distinction so the inversion doesn't creep back in. The requirement table is split into RTO20e (wait) and RTO20e1 (fail) rows, mirroring the unit spec.2. Proxy rule format fixes (silent/opaque failure modes)
objects_faults.md: two rules wrote"match": { "action": 20 }as a JSON number — uts-proxy rejects this at session creation with HTTP 400 (cannot unmarshal number into Go struct field MatchConfig.match.action of type string). Now"20"(string).connection_resume.md(RTN15g): rule action type"refuse"does not exist (refuse_connectiondoes). Unknown action types hit a silent passthrough in the proxy, so the secondws_connectwas never refused and the test premise broke with no error. Corrected.helpers/proxy.md: the Match Conditions section now documents theactionstring format explicitly, plus the v0.3.0 limitation that action names resolve only up toAUTH(17) — unresolvable names (e.g."OBJECT_SYNC") silently never match, so objects actions need numeric strings until the proxy's name table is extended.uts/objects,uts/realtimeanduts/rest(all match types, action types,trigger_actiontypes and match fields validated against the Go proxy's structs) found no further instances.3. Wall-clock timeout guidance (recurring translation trap)
All
WITH timeout/poll_until/WAITdurations in integration specs are wall-clock time, but frameworks that virtualise time by default (e.g. kotlinx-coroutinesrunTest) fast-forward the clock while the test idles — a naively translated 15s timeout around a real network await fires instantly with a misleading error. Added:docs/writing-derived-tests.md: new section Integration timeouts are wall-clock (beware virtual-time frameworks) — the counterpart to the existing No real timers in unit tests note — with the failure mode and the shared-helper remedy.docs/integration-testing.md(Timeout Strategy) anddocs/writing-test-specs.md(State Transitions): short pointers to it.helpers/proxy.md: new convention (Specify token revocation -Auth#revokeTokensmethod #10).objects_faults.md: the two previously unboundedobject.get()awaits now carryWITH timeout: 15 seconds— every await in the file is bounded, and the Common Cleanup covers multi-client tests explicitly.4.
objects/PLAN.mdSummary row updated to the corrected RTO20e scenario ("in-flight publish failing when channel enters FAILED during sync wait").
Validation
objects_faults.mdis fully implemented by the ably-java derived suite: all 5 tests pass against the sandbox through uts-proxy, including the restructured RTO20e (92008 with cause 90000 observed end-to-end).UTS:tags remain valid.Follow-ups (not in this PR)
ably/uts-proxy: extendactionNameswithOBJECT/OBJECT_SYNC/ANNOTATIONso name-based matching works for objects actions.realtime/integration/helpers/proxy.mdtouts/docs/— it is cross-module infrastructure consumed by realtime, rest and objects; best done as a dedicated rename commit with a reference sweep.publish-fails-on-channel-failed-0) now under-describes the scenario; kept stable for traceability, but a coordinated rename could accompany review feedback.🤖 Generated with Claude Code