fix(copilot): VS Code Copilot always reports the account default model instead of the user-selected mode#1560
Open
jorgegarciarey wants to merge 3 commits into
Conversation
VS Code Copilot transcripts contain no model field, so git-ai always fell back to models.json which returns the account default model (is_chat_default == true) instead of the user-selected model. Read the real model from VS Code's OTel SQLite DB (agent-traces.db), querying the spans table by chat_session_id (transcript file stem), preferring request_model over response_model, ordered by most recent end_time_ms. The OTel lookup is inserted between transcript extraction and the models.json fallback in the VS Code native hooks parser.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Devin review (git-ai-project#1560): response_model was not filtered for empty strings, so a row with NULL/empty request_model and an empty-string response_model returned Some("") and short-circuited the models.json fallback. Now both columns filter empty strings, consistent with the rest of the OTel queries. Adds a regression test.
jorgegarciarey
changed the title
Author
|
@svarlamov , what do you think about this? |
Member
|
Thank you for the contribution, @jorgegarciarey. Will do our best to review this week for an upcoming release! |
Author
|
Friendly reminder @svarlamov 👋🏼🙂 |
Author
|
Hello @svarlamov . Other pull requests with this fix was opened here #1638 It seems the problem is known. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1559
Problem
When using GitHub Copilot Chat from VS Code (agent mode), git-ai always attributes commits to the account's default model (e.g.
gpt-5.3-codex) regardless of which model the user actually selected in the chat picker.Root cause
VS Code's chat transcripts contain no model field in any entry (
session.start,user.message,assistant.message, etc.). Thesession.model_changeevent that git-ai relies on is only emitted by the Copilot CLI, not by VS Code. As a resultmodel_extraction::extract_modelreturnsNonefor VS Code transcripts and the resolution chain falls through to themodels.jsonfallback, which returns the entry withis_chat_default == true— the account default, not the user-selected model.Fix
VS Code's Copilot extension records the real per-request model in its OTel SQLite DB
agent-traces.db, located at.../User/globalStorage/github.copilot-chat/agent-traces.db. Thespanstable has achat_session_idcolumn matching the transcript file stem, plusrequest_model/response_modelcolumns.This PR:
extract_model_from_copilot_otel_db(and helperresolve_copilot_otel_db_path) insrc/streams/model_extraction.rs, which queriesspansbychat_session_id, preferringrequest_modeloverresponse_model, ordered by most recentend_time_ms. The DB path can be overridden viaGIT_AI_COPILOT_OTEL_DB_PATHfor testing.src/commands/checkpoint_agent/presets/github_copilot/ide.rs, between transcript extraction and themodels.jsonfallback, so the correct model is resolved before the account-default fallback kicks in.Testing
request_model, falls back toresponse_model, uses most recent span, ignores other sessions, and missing DB.cargo test --lib model_extraction— 24 passed, 0 failed.cargo fmtandcargo clippy --all-targets -- -D warnings— clean.cargo build— success.The change is read-only against the OTel DB (opened read-only) and degrades gracefully (returns
None) when the DB or session is missing, preserving the previousmodels.jsonfallback behavior.