Repository navigation
Conversation
TRITONBACKEND_ModelInstanceReportStatistics expects timestamps in the order (exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns), but ModelInstanceState::ProcessRequests passed exec_end_ns before compute_start_ns/compute_end_ns. Triton core then interprets exec_end_ns as compute_input_end_ns, and the resulting subtraction underflows as an unsigned 64-bit value, reporting nv_inference_compute_infer_summary_us as an enormous bogus number (~18446744074s). Reorders the arguments to match the documented signature and the legacy PyTorch backend's call convention. Ref: triton-inference-server/server#8874
Greptile SummaryCorrects the PT2/AOTI per-request statistics call so timestamps follow the expected execution and computation sequence.
Confidence Score: 5/5The PR appears safe to merge because the corrected timestamp order is consistent with the established sibling statistics calls. The change only reorders same-typed timestamp arguments and now aligns the PT2/AOTI per-request call with both the legacy backend and adjacent batch-statistics reporting. Important Files Changed
Reviews (1): Last reviewed commit: "Fix swapped ReportStatistics timestamp a..." | Re-trigger Greptile |
|
Hi @hg1112 - Thank you for the report and the fix! Confirmed on our side: the API expects Validated through our internal CI (Pipeline ID: 63772955, branch One prerequisite before we can merge: Triton requires all contributors (or their corporate entity) to have a signed Contributor License Agreement on file with triton-cla@nvidia.com. Do you know if you or your employer already have one? Happy to help get that sorted if not. |
|
Hi @mattwittwer , I have sent an email to triton-cla@nvidia.com. I am doing this work from my personal interest in this project :) |
|
Hi @mattwittwer , @yinggeh I have received approval from triton cla |
|
Waiting for CI 63772955 to complete before merging. |
Summary
TRITONBACKEND_ModelInstanceReportStatisticsexpects timestamps in the order(exec_start_ns, compute_start_ns, compute_end_ns, exec_end_ns), butModelInstanceState::ProcessRequestsin the PT2/AOTI instance state passedexec_end_nsbeforecompute_start_ns/compute_end_ns. Triton core theninterprets
exec_end_nsascompute_input_end_ns, and the resultingsubtraction underflows as an unsigned 64-bit value, so
nv_inference_compute_infer_summary_usreports an enormous bogus number(~18446744074s) instead of the real compute latency.
This reorders the arguments to match the documented header signature and the
legacy (non-PT2) PyTorch backend's call convention, which already does this
correctly. The batch-level
TRITONBACKEND_ModelInstanceReportBatchStatisticscall a few lines below was already in the correct order, so no other changes
are needed.
Ref: triton-inference-server/server#8874
Test plan
pre-commit run(clang-format, codespell, etc.) passes on the changed fileTRITONBACKEND_ModelInstanceReportStatistics'sdeclaration in
triton-inference-server/core'stritonbackend.h