Skip to content

docs: say how to serialize attribution runs and list the limitations - #16

Merged
esnunes merged 2 commits into
mainfrom
docs/readme
Oct 4, 2026
Merged

esnunes merged 2 commits into
mainfrom
docs/readme

Conversation

@esnunes

@esnunes esnunes commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Review item 20. Each change below was checked against the code or a run before writing it.

Serializing runs

The README suggested .serialized on suites. It isn't enough: .serialized orders only the tests of its own suite and nested suites, and Swift Testing still runs other suites in parallel. Two .serialized suites whose tests wait on each other through gates ran at the same time, and both records came out overlapped.

The README now says to turn off parallel testing for the run, and the Usage example drops .serialized. With either flag, the Fixtures/Concurrency gated tests run one at a time and both records are clean:

  • swift test --no-parallel
  • xcodebuild test -parallel-testing-enabled NO

The trait's doc comment says the same.

Other changes

  • Examples use a fresh directory from mktemp -d instead of a fixed /tmp/attribution (review item 16). The path is kept in a variable and listed afterwards, and parallel testing is off in both commands. I ran the swift test one as written against Fixtures/Example: it records into one directory per test process.
  • Guarantees:
  • Record layout: records are in the host's byte order, little-endian on every Apple platform.
  • A new Limitations section:
    • XCTest and Swift Testing run one after the other. Xcode runs a process's XCTest tests before its Swift Testing tests (observed with Xcode 26.3), SwiftPM uses separate processes, and attribution relies on it (from review item 12).
    • UI tests record nothing about the app, whose code runs in a process that doesn't link the package.
    • Images loaded late aren't observed. This moved from Guarantees, since it's a limitation.
    • The package's own code shows up in the records. Under xcodebuild the observer is instrumented, and the trait is instrumented everywhere.

Not changed: the install line stays on branch: "main" until there's a release tag (review items 22–23).

mise run lint passes.

Code review

Fixed:

  • Inlined $(mktemp -d). It lost the output path, so readers couldn't find their records.
  • .serialized in the Usage example. It contradicted the new text.
  • Xcode version. This description named Xcode 26.1 for the XCTest-before-Swift-Testing ordering; it was observed with 26.3.

Not changed:

  • "Two test bundles loaded by the same host" as the two-copies example can't be confirmed here.
  • The overlap sentence reads as applying to XCTest too, but Limitations already says the two frameworks aren't checked against each other.

🤖 Generated with Claude Code

@esnunes
esnunes requested a review from a team as a code owner October 3, 2026 16:50
@esnunes
esnunes requested review from fortmarek and removed request for a team October 3, 2026 16:50
esnunes added a commit that referenced this pull request Oct 4, 2026
The message before this PR printed them, and they are what diagnosed
#16's failure: they showed the iOS platform installed with no simulator
on it. The lookup now returns them alongside the simulator it found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
esnunes added a commit that referenced this pull request Oct 4, 2026
#16's Test on macOS failed on a tuist-macos-26-3 runner whose image has
the iOS simulator runtime but no simulators: Xcode listed only the "Any
iOS Simulator Device" placeholder. #18 had passed on another runner of
the same profile that did have them.

When Xcode lists no iPhone, the test now creates one with simctl on the
newest iOS runtime installed, using an iPhone model it supports, and
asks Xcode again.

execute now returns standard output, kept apart from standard error,
which drops the temporary log files its callers managed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
esnunes added a commit that referenced this pull request Oct 4, 2026
The message before this PR printed them, and they are what diagnosed
#16's failure: they showed the iOS platform installed with no simulator
on it. The lookup now returns them alongside the simulator it found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
esnunes and others added 2 commits October 3, 2026 23:36
- `.serialized` orders only its own suite: Swift Testing still runs
  other suites in parallel, so their tests come out overlapped. Say to
  turn parallel testing off for the run instead (`swift test
  --no-parallel`, `-parallel-testing-enabled NO`), which both serialize
  Swift Testing too. The trait's doc comment says the same.
- Use `mktemp -d` in the examples rather than a fixed path in /tmp.
- The coverage-report guarantee covers SwiftPM as well as Xcode.
- Records are in the host's byte order, little-endian on Apple platforms.
- Note the Objective-C runtime's harmless duplicate-class warning when
  two copies load.
- A Limitations section: XCTest and Swift Testing run one after the
  other, UI tests record nothing about the app, late-loaded images
  aren't observed, and the package's own code shows up in the records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Inlining "$(mktemp -d)" into the command threw the path away, so the
  reader couldn't find what was recorded. The examples now keep it in a
  variable and list it, with parallel testing off as the text advises.
- The Usage example no longer shows `.serialized`, which the text right
  below says isn't enough.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@esnunes
esnunes merged commit 2c5f640 into main Oct 4, 2026
7 checks passed
@esnunes
esnunes deleted the docs/readme branch October 4, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant