Skip to content

server: add --reasoning and --alias CLI options - #1017

Open
aaa2015 wants to merge 1 commit into
antirez:mainfrom
aaa2015:feat/reasoning-alias-options
Open

aaa2015 wants to merge 1 commit into
antirez:mainfrom
aaa2015:feat/reasoning-alias-options

Conversation

@aaa2015

@aaa2015 aaa2015 commented Sep 9, 2026

Copy link
Copy Markdown

Motivation

Clients that expose a model picker can inject thinking defaults that conflict
with the mode a user selected. This adds server-side defaults plus a way to
advertise an instance under a custom model id.

Changes

  • --reasoning on|off / --thinking on|off — default thinking state when a
    request omits one. The two are synonyms; the error message reports the flag
    the user actually typed.
  • --no-thinking — shortcut for --reasoning off.
  • --reasoning-effort none|minimal|low|medium|high|xhigh|max — default effort.
  • --alias NAME — advertise an extra model id in the catalogue so a picker can
    target this instance by name. The id is also served by GET /v1/models/NAME.

Alias suffixes select the default thinking mode: -chat/-nothink disable it,
-reasoner enables it. The match is on the suffix rather than a substring, so a
local quant name such as local-glm-5.3-flash-q2 stays unaffected while a
...-chat alias still opts out.

Turning the default off must not disable thinking permanently

think_mode_from_enabled() returns NONE whenever the effort is NONE, so an
earlier revision that set default_think_mode = NONE on --reasoning off
pinned every request to non-thinking. A client that opted back in through a
-reasoner alias or an explicit thinking: true was silently ignored.

Off now flips only the boolean and keeps the configured effort:

--reasoning off                  → enabled=false, effort=HIGH
  request without thinking field → NONE
  request with thinking:true     → HIGH   (was NONE before this fix)

--reasoning off --reasoning-effort max
  request with thinking:true     → MAX    (was NONE before this fix)

--reasoning-effort none          → deliberate opt-out, stays NONE

Catalogue bug fix

send_models() picked the advertised ids with a family test, but GLM-5.3 and
GLM-5.2 share DS4_MODEL_FAMILY_GLM_DSA. A GLM-5.3 engine therefore advertised
glm-5.2, glm-5.2-chat and glm-5.2-reasoner, misreporting the loaded model
to a picker. The ids are now selected by variant, and a single list drives both
the catalogue and the alias-deduplication check.

Testing

Machine: Apple M4, macOS 26.4.1, 16 GB. Backend: Metal (default build).

make ds4_test
./ds4_test --server
→ server: OK

--server covers the request parsing, chat rendering and thinking-control paths
this change touches. It needs no model file. The model-backed suites
(--logprob-vectors, --long-context, ...) were not run because this machine
has no GGUF checked out.

New tests:

  • test_server_reasoning_options — flag parsing, the off/on effort combinations
    above, plus negative alias-suffix cases (my-chat-model,
    local-chat-assistant must not disable thinking).
  • test_model_catalog_matches_loaded_variant — the three variant catalogues,
    and every advertised id is accepted by server_model_alias_known().
  • test_send_models_json_shape — drives the real /v1/models body over a
    socketpair and checks the JSON shape, plus that a built-in alias is not
    advertised twice.

No inference-backend code is touched, so no speed regression is expected.

Clients that expose a model picker can inject thinking defaults that conflict
with the mode a user selected.  --reasoning/--thinking set the server-side
default used when a request omits one, --reasoning-effort picks the default
effort, and --alias advertises an extra model id in the model catalogue so a
picker can target this instance by name.

Alias suffixes select the default thinking mode: -chat/-nothink disable it and
-reasoner enables it.  Match on the suffix rather than the whole string, so a
local quant name such as local-glm-5.3-flash-q2 stays unaffected while a
...-chat alias still opts out.

Turning the default off only flips the boolean.  Setting the effort to NONE as
well would pin think_mode_from_enabled() to NONE, so a client that opts back in
through a -reasoner alias or an explicit thinking:true was silently ignored.
Keep the configured effort so opting in still yields a usable mode; an explicit
--reasoning-effort none remains a deliberate opt-out.

While touching the catalogue, fix the ids it advertises.  GLM-5.3 and GLM-5.2
share DS4_MODEL_FAMILY_GLM_DSA, so a family test made a 5.3 engine list glm-5.2
and misreport the loaded model to a picker.  Select the ids by variant, and
drive both the catalogue and the alias-deduplication check from that one list so
a built-in alias is never advertised twice.
@aaa2015
aaa2015 force-pushed the feat/reasoning-alias-options branch from ea4740c to 4554267 Compare September 10, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant