Skip to content

[Index Management] Use the data plugin for doc counts and sample documents - #306623

Open
mattkime wants to merge 1 commit into
elastic:mainfrom
mattkime:index_mgmt_data_plugin_search
Open

mattkime wants to merge 1 commit into
elastic:mainfrom
mattkime:index_mgmt_data_plugin_search

Conversation

@mattkime

Copy link
Copy Markdown
Contributor

Summary

Index Management ran two queries through its own server routes. The browser already has a way to run them: the data plugin's search service. This PR removes both routes and runs the queries from the browser with data.search.

Removed route Replacement
POST /internal/index_management/index_doc_count data.search with the ES|QL search strategy (esql)
GET /internal/index_management/indices/{indexName}/sample data.search with the Elasticsearch search strategy (es)

The queries themselves are unchanged:

  • Document counts use the same FROM ... METADATA _index | STATS count() BY _index query with allow_partial_results: true. The query builder and response mapping now live in common/lib/index_doc_count_query.ts. quoteEsqlIndexName moved from server/lib to common/lib so the browser can use it.
  • Sample documents use the same search request: 10 docs, _source including _inference_fields, all fields, sorted by _doc descending. The hook now uses useQuery, with retries and refetch on window focus turned off to match the previous useRequest behavior.

data is now a required plugin of Index Management, and the data plugin's start contract is passed to components through the app context (plugins.data).

This PR is split out of #246293 and has no dependency on the es_client plugin.

Behavior notes

  • The es search strategy adds ignore_unavailable: true to every request. If the index is unavailable, the sample now comes back empty instead of returning an error like the old route did.
  • Errors now come from the data plugin's search errors instead of handleEsError. The overview and quick stats only read the error message, or fall back to the metadata doc count.

Testing

  • New unit tests for fetchIndexDocCounts and useLoadIndexDocumentsSample (request shape, count mapping, errors).
  • Updated the docCountApi, QuickStats and client integration tests to mock /internal/search/{strategy}.
  • The Scout API spec index_doc_count.spec.ts now sends the browser's query through internal/search/esql.

…ments

Replace the internal index_doc_count and indices/{indexName}/sample routes with
browser-side data plugin searches: the ES|QL search strategy for document counts
and the es search strategy for the sample documents on the index details page.

Co-authored-by: Cursor <cursoragent@cursor.com>
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown
Contributor
🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!
  • Click to trigger kibana-entity-store-performance-from-pr for this PR!
  • Click to trigger kibana-storybooks-from-pr for this PR!
  • Click to trigger kibana-static-checks for this PR!

@mattkime mattkime self-assigned this Nov 20, 2026
@mattkime mattkime added Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// Feature:Index Management Index and index templates UI backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes labels Nov 20, 2026
@mattkime
mattkime marked this pull request as ready for review November 20, 2026 04:19
@mattkime
mattkime requested a review from a team as a code owner November 20, 2026 04:19
@mattkime
mattkime requested a review from SoniaSanzV November 20, 2026 04:19
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-management (Team:Kibana Management)

@@ -92,7 +92,7 @@ describe('<IndexDetailsPage />', () => {
testUserStartPrivilegesResponse
);
httpRequestsMockHelpers.setLoadIndexDocCountResponse({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P3 (Low)

Register a sample search response in this suite and assert that the Overview renders it. Nothing sets up /internal/search/es, so the HTTP mock returns {} and the sample hook hits a rawResponse.hits error. The page tests can pass without ever exercising a successful preview through the new search wiring.

Generated by Libra. React with 👍 or 👎 to give feedback on this comment.

@@ -96,8 +102,8 @@ export const DetailsPageContent: FunctionComponent<Props> = ({
data: documentsSampleData,
isLoading: isDocumentsSampleLoading,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P3 (Low)

isLoading from useQuery is only true when there's no cached data yet, so it stays false during refetch(). useRequest used to set isLoading to true on every resend. As a result, clicking Refresh in the Data preview (or the onIndexRefresh refetch after an index action) no longer shows the button spinner or the EuiProgress bar. After a failed load, the error banner also stays up while the retry runs, because showErrorBanner is error && !isLoading. Use isFetching, or isLoading || isRefetching, to keep the old loading feedback. The shared module-level QueryClient also serves cached samples when you reopen an index, and with this change nothing shows that they are being refreshed.

Generated by Libra. Specialist: libra-complexity. React with 👍 or 👎 to give feedback on this comment.

search<IEsSearchRequest, IEsSearchResponse>(
{
params: {
index: indexName,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P3 (Low)

Set ignore_unavailable: false for this single-index preview. The es strategy defaults it to true, so a closed or recently deleted index can look like an empty one: the preview disappears instead of telling the user why the documents could not be loaded.

Generated by Libra. React with 👍 or 👎 to give feedback on this comment.

responseType: 'json',
body: JSON.stringify({ indexNames }),
apiTest.beforeAll(async ({ requestAuth }) => {
credentials = await requestAuth.getApiKey('admin');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P3 (Low)

Use a scoped interactive session for the replacement search endpoint. The admin API key can return the expected counts even if the new route rejects browser sessions or requires privileges beyond read access to the test indices. Keep setup on esClient, but make the assertions with a samlAuth cookie for a role scoped to the test indices.

Generated by Libra. Specialist: scout. React with 👍 or 👎 to give feedback on this comment.

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 5.5MB 5.5MB +1.0B
shared-plugins 12.4MB 12.4MB +762.0B
vendors-heavy 2.0MB 2.0MB +2.0B
total +765.0B
Unknown metric groups

shared async chunks total size

id before after diff
all 16.8MB 16.8MB +12.0B

total optimizer output size

id before after diff
all 69.1MB 69.1MB +776.0B

warm start memory

id before after diff
post forced gc heap baseline - 630261866 +630261866
post forced gc heap delta - -2295724 -2295724
post forced gc heap delta standard deviation - 1470659 +1470659
post forced gc heap target - 627966142 +627966142
tail heap delta - -23545296 -23545296
total +1233857647

cc @mattkime

mattkime added a commit that referenced this pull request Nov 20, 2026
…306623

Restore the index_doc_count and documents sample routes and their callers so
this PR only contains the es_client changes, and commit the user status route
files restored when reverting the security API usage.

Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:Index Management Index and index templates UI release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t//

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant