Repository navigation
Codex review: measure fork pull requests, outside triggers and the automatic review #2251
Description
Codex's GitHub review is on for this repository, and PR #2249 (a measurement branch, closed unmerged) answered part of what ISS #2000 asked. Three points stay open, and each is read off a live pull request.
- Fork pull requests. Whether Codex reviews a pull request opened from a fork, automatically or on
@codex review. No fork pull request has been opened since the review was turned on; the first external one answers it. - Who may trigger it.
@codex reviewfrom the maintainer's login started a review in 12 s. A comment from a login outside the organisation has not been tried; the first external contributor's comment, or an outside reviewer's, answers it. - The automatic review. None started when Measure Codex's automatic review (issue #2000): not to merge #2249 was opened (01:47:59Z) nor when it went from draft to ready (02:19:58Z, after the manual review of the same head); 20 and 15 minutes were watched. The review summary's own text says opening a pull request for review and marking a draft ready both trigger one. Whether the automatic review is off in Codex's settings or skipped a head already reviewed is read from the next pull request opened here, and from the trees' pull requests for ISS A tree with Codex's review enabled carries the Code Review Rules section in its AGENTS.md #2009.
Done when each point has a measurement in a comment here: the pull request, the login, what Codex posted and on which commit.
gh api repos/btclib-org/<repo>/issues/<n>/comments \
--jq '.[]|select(.user.login=="chatgpt-codex-connector[bot]")|.created_at'
Activity
fametrano commentedon Nov 14, 2026
Measured on 2026-11-14, with the command in the body and the pulls API.
Fork pull requests. Codex starts by itself when a pull request is opened from a fork, and the review then fails on its usage limit. Five of five, each within 10 s of opening:
dbdefault/functional-testsfailedqiaobochi040726-source/seed-scanfailedjay23606/btclib-nodefailedYurii201811/btclibfailedluanpv2003/.githubfailedEach carries a second comment two seconds later: "Codex usage limits have been reached for code reviews." The same happened on PR btclib-org/ai-agents-process#434, opened by
app/claudefrom a branch of the repository (11:56:13Z,failed). In the same hours Codex completed reviews on pull requests opened by the maintainer's login (PR #2274 at 14:59:43Z, PR btclib-org/ai-agents-process#437 at 15:28:07Z). So the limit follows who opened the pull request, not the repository or the hour; which setting decides it was not read.A failed review writes
"has_unresolved_findings":true,"status":"failed"in its state: a reader must teststatusbefore the findings flag.The automatic review. It is on: the five above show "PR opened" as trigger, and PR #2274 shows a security review "Automatically triggered" at 15:00:37Z after a push.
Who may trigger it. Still open: no comment
@codex reviewfrom a login outside the organisation was found on these pull requests.chatgpt-codex-connector commentedon Nov 14, 2026
Summary
statusbefore interpretinghas_unresolved_findings, because failed reviews can retain a true findings flag.0d591a9with a signed-off commit and created the follow-up pull-request metadata.Testing
git diff --checkPYENV_VERSION=3.14.4 python -m pytest tests/citations_test.py tests/conventions_test.py -q(the unmanaged environment lacks the pytest plugin providing the configuredtimeoutoption)uv run --locked --only-group lint pre-commit run --all-files markdownlint-cli2(installed uv is0.7.22, older than the repository-required>=0.13.4)git status --short --branch(clean working tree after commit)View task →