Is there anyway to specify a URL Prefix for hosting Triton?

[michaeltinsley]

Say I want to host Triton on a load balancer with a path prefix like my-domain.com/triton such that I can hit it with my-domain.com/triton/v2/health for example, is there any way to specify this path prefix in the Triton server?

[demirciberk]

Short answer: No. Triton's HTTP server does not support configuring a URL path prefix natively. The /v2/... KServe protocol paths are hardcoded in the server implementation.

Confirmed facts:

• Triton's HTTP endpoint registration lives in src/http_server.cc in this repo (see the HTTPAPIServer class), where routes like /v2/health/ready, /v2/health/live, /v2/models/..., /v2/models/{model}/infer, etc. are registered as fixed string paths. There is no configuration option, env var, or CLI flag that rewrites or prefixes these paths.
• The tritonserver CLI only exposes address/port-level HTTP options — --http-address, --http-port, --http-thread-count, --http-header-forward-pattern, --reuse-http-port, --http-restricted-api — none of which touch the URL path structure. See tritonserver --help output and the Customization Guide / Inference Protocols doc.
• The KServe v2 protocol Triton implements defines these endpoint paths (/v2/health/live, /v2/health/ready, /v2/models/{name}[/versions/{version}], etc.) as part of the spec itself: https://github.com/kserve/kserve/tree/master/docs/predict-api/v2. Since Triton aims for protocol compatibility, changing the path scheme server-side isn't offered.

Workaround (standard practice, confirmed via NVIDIA's own deployment docs): put a reverse proxy in front of Triton that strips the prefix before forwarding. NVIDIA's own docs use NGINX/Traefik/HAProxy in front of Triton for load balancing (e.g. deploy/k8s-onprem/README.md, and the TensorRT-LLM multi-instance guide). None of those examples rewrite the path prefix (they proxy / as-is), but the same proxies can strip a prefix.

Minimal NGINX example that lets you hit my-domain.com/triton/v2/health/ready while Triton itself only knows about /v2/health/ready:

nginx
location /triton/ {
rewrite ^/triton/(.*)$ /$1 break;
proxy_pass http://triton_backend; # e.g. http://127.0.0.1:8000
}

Caveat: any client SDK (e.g. tritonclient.http) that constructs the /v2/... path internally will need endpoint overrides or the prefix stripped at the proxy layer as above — those clients don't know about the prefix either, since it doesn't exist in Triton's protocol.

Not confirmed / needs more info if the above doesn't fit your case: whether a newer unreleased branch adds a path-prefix option — I did not find one in main as of this research. If your load balancer can't do path rewriting, that would need to be resolved on the LB side, not in Triton.