Skip to content

fix: Swap ReportStatistics timestamp args in PT2/AOTI backend - #203

Merged
whoisj merged 1 commit into
triton-inference-server:mainfrom
hg1112:fix-pt2-report-statistics-arg-order
Aug 27, 2026
Merged

whoisj merged 1 commit into
triton-inference-server:mainfrom
hg1112:fix-pt2-report-statistics-arg-order

Conversation

@hg1112

@hg1112 hg1112 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

TRITONBACKEND_ModelInstanceReportStatistics expects timestamps in the order
(exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns), but
ModelInstanceState::ProcessRequests in the PT2/AOTI instance state passed
exec_end_ns before compute_start_ns/compute_end_ns. Triton core then
interprets exec_end_ns as compute_input_end_ns, and the resulting
subtraction underflows as an unsigned 64-bit value, so
nv_inference_compute_infer_summary_us reports an enormous bogus number
(~18446744074s) instead of the real compute latency.

This reorders the arguments to match the documented header signature and the
legacy (non-PT2) PyTorch backend's call convention, which already does this
correctly. The batch-level TRITONBACKEND_ModelInstanceReportBatchStatistics
call a few lines below was already in the correct order, so no other changes
are needed.

Ref: triton-inference-server/server#8874

Test plan

  • pre-commit run (clang-format, codespell, etc.) passes on the changed file
  • Verified new argument order against TRITONBACKEND_ModelInstanceReportStatistics's
    declaration in triton-inference-server/core's tritonbackend.h

TRITONBACKEND_ModelInstanceReportStatistics expects timestamps in the
order (exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns),
but ModelInstanceState::ProcessRequests passed exec_end_ns before
compute_start_ns/compute_end_ns. Triton core then interprets
exec_end_ns as compute_input_end_ns, and the resulting subtraction
underflows as an unsigned 64-bit value, reporting
nv_inference_compute_infer_summary_us as an enormous bogus number
(~18446744074s).

Reorders the arguments to match the documented signature and the
legacy PyTorch backend's call convention.

Ref: triton-inference-server/server#8874
@greptile-apps

greptile-apps Bot commented Aug 2, 2026

Copy link
Copy Markdown

Greptile Summary

Corrects the PT2/AOTI per-request statistics call so timestamps follow the expected execution and computation sequence.

  • Moves exec_end_ns after the compute timestamps.
  • Aligns PT2/AOTI reporting with the legacy backend and batch-statistics call convention.

Confidence Score: 5/5

The PR appears safe to merge because the corrected timestamp order is consistent with the established sibling statistics calls.

The change only reorders same-typed timestamp arguments and now aligns the PT2/AOTI per-request call with both the legacy backend and adjacent batch-statistics reporting.

Important Files Changed

Filename Overview
src/pt2/model_instance_state.cc Reorders the statistics timestamps to exec_start_ns, compute_start_ns, compute_end_ns, and exec_end_ns; no actionable issue found.

Reviews (1): Last reviewed commit: "Fix swapped ReportStatistics timestamp a..." | Re-trigger Greptile

@mattwittwer

Copy link
Copy Markdown
Contributor

Hi @hg1112 - Thank you for the report and the fix!

Confirmed on our side: the API expects (exec_start, compute_start, compute_end, exec_end), so core computed compute_infer as compute_start_ns - exec_end_ns, which underflows uint64_t. compute_input/compute_output and trace timestamps were also skewed. Your reordering matches the API contract.

Validated through our internal CI (Pipeline ID: 63772955, branch mwittwer/pr203_ci_validation mirroring your commit 3d7c7ee), including a new L0_torch_aoti stats-sanity regression test that fails without this fix and passes with it.

One prerequisite before we can merge: Triton requires all contributors (or their corporate entity) to have a signed Contributor License Agreement on file with triton-cla@nvidia.com. Do you know if you or your employer already have one? Happy to help get that sorted if not.

@hg1112

hg1112 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Hi @mattwittwer , I have sent an email to triton-cla@nvidia.com.

I am doing this work from my personal interest in this project :)

@yinggeh yinggeh changed the title Fix swapped ReportStatistics timestamp args in PT2/AOTI backend fix: Swap ReportStatistics timestamp args in PT2/AOTI backend Aug 25, 2026
@yinggeh yinggeh self-assigned this Aug 25, 2026
@yinggeh yinggeh added the bug Something isn't working label Aug 25, 2026
@hg1112

hg1112 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Hi @mattwittwer , @yinggeh

I have received approval from triton cla

@mattwittwer
mattwittwer self-requested a review August 27, 2026 16:33
@whoisj

whoisj commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Waiting for CI 63772955 to complete before merging.

@whoisj
whoisj merged commit 16e0762 into triton-inference-server:main Aug 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Development

Successfully merging this pull request may close these issues.

4 participants