PS-11203 Vector Index Syntax by percona-mhansson · Pull Request #5987 · percona/percona-server

percona-mhansson · 2026-05-29T12:26:48Z

Stacked on #6000

catalinbp · 2026-07-22T08:43:07Z

+  eng "Vector index can only be created in tables with a BIGINT UNSIGNED primary key."
+
+ER_ONLY_SINGLE_VECTOR_INDEX_ALLOWED
+  eng "A table can have at most one vector index."


why is this the case? I've seen examples where having two indexed vector columns make a lot of sense. Is this only for the MVP?

Was not documented properly, but now is:

https://app.notion.com/p/percona/Vector-Search-MVP-design-for-potential-users-388674d091f3809fb332c1a9f0736ce6?source=copy_link#38a674d091f380c5b277c7e080d04dbf

catalinbp · 2026-07-22T08:53:16Z

 inline constexpr const decltype(handlerton::flags)
    HTON_SECONDARY_SUPPORTS_TEMPORARY_TABLE(1 << 25);

-


please revert these changes to the end of the file as they are not related to this commit

Good catch. These were actually part of the underlying PR, so I'll update that one, too.

catalinbp · 2026-07-22T09:02:31Z

+);
+SHOW CREATE TABLE t1;
+SHOW INDEXES FROM t1;
+


I think we don't have any test cases where you try creating a VECTOR index on a table without BIGINT UNSIGNED, also one where you create the table with such primary key but then you alter the table by droping it or perhaps change data type(though I think the second part of this idea should not be possible)

The latter I have: https://github.com/percona/percona-server/pull/5987/changes#diff-5daf5ca2072c4bcd935fb51611868b7a7a550bd8b541804f20556541446216e0R18-R19

Added the rest.

catalinbp · 2026-07-22T10:44:46Z

+        }
+        hnsw_param.M = std::atoi(p.value.str);
+      } else if (my_strcasecmp(system_charset_info, p.key.str, "metric") == 0) {
+        if (my_strcasecmp(system_charset_info, p.value.str, "euclidean") == 0) {


how will this tie into the distance functions? does it need to support all the metrics?

I also wonder what visibility do we have from the optimizer if an index is defined on one metric and we query it using distance function using another metric. Shouldn't this be more "static" since there are only a few available options?

Indeed. I changed the code to make it more obvious how to add new metrics. If you have a list of metrics already, feel free to drop them!

Not sure what you mean by static? You mean coded into the Bison parser?

catalinbp · 2026-07-22T11:52:33Z

+          }
+        ;
+
+index_construction_parameter:


I wonder if it's a good idea to expect IDENT_QUOTED instead of IDENT. As far as I know the former accepts non-ASCII bytes/multi-byte. This should be a simple config and I don't expect anything "exotic" here, it's easier to start with something restricted, and relax requirements later

There's some ugly lexer hack around IDENT, I never managed to get it to work

mysql> CREATE TABLE t1 ( id BIGINT UNSIGNED PRIMARY KEY, v1 VECTOR( 1234 ), VECTOR KEY( v1 ) TYPE hnsw WITH ( m=a ) ); ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'm=a ) )' at line 1

The rest of the string is eaten up by the IDENT above

Changing to ident for now. That seems to be what people do

catalinbp · 2026-07-23T21:30:25Z

        | XML_SYM
        | YEAR_SYM
        | ZONE_SYM
-        | VECTOR_SYM


Is this necessary? this makes VECTOR a reserved keyword so this will break existing schema using this name

I think it will be almost impossible to support the VECTOR INDEX syntax we have without it. Currently it's perfectly legal to define a column named vector:

CREATE TABLE t ( ..., vector char(1) )

Heck, you can even do CREATE TABLE t ( ..., vector VECTOR(1) )!

We want to support the syntax to create an in-line vector definition

CREATE TABLE t ( ..., vector index ix (col) )

You can probably see where this is going.
We have these options afaiu:

Re-shuffle the rules so we never reduce a until we have the full set of items. Only then can we know whether it's a column definition or an index definition
Not do the VECTOR INDEX syntax at all, only INDEX and let type TYPE/USING clause decide. Fwiw, this seems to be what Oracle are considering.
Do as above and make VECTOR a properly reserved word
We could also in theory create another category of reserved words which can be used as anything but column names :P

catalinbp · 2026-07-23T21:56:50Z

        key_create_info.comment = key_info->comment;
-
+      if (key_info->vector_index_type.str != nullptr)
+        key_create_info.vector_index_type = key_info->vector_index_type;


I think this where the other parameters of the index config gets lots and triggers this bug:

CREATE TABLE t1 ( id BIGINT UNSIGNED PRIMARY KEY, v1 VECTOR( 8 ), VECTOR KEY( v1 ) TYPE hnsw WITH ( M = 6, metric = euclidean ) ); SHOW CREATE TABLE t1; # Unrelated ALTER ALTER TABLE t1 ADD COLUMN c INT; # see how the parameters gets lost SHOW CREATE TABLE t1;

Ouch. This was a particularly gnarly issue. It made me re-implement where the whole serialization and de-serialization happens. But it's probably cleaner this way.

catalinbp · 2026-07-23T21:59:31Z

  assert((key_info->flags & flags_before_switch) == flags_before_switch);
  if (key->generated) key_info->flags |= HA_GENERATED_KEY;

+  // Serialize vector index construction params (WITH clause).


CREATE TABLE t2 ( a INT, KEY k ( a ) TYPE hnsw WITH ( M = 6 ) ); should not be accepted

Isn't now 😄

mysql> CREATE TABLE t2 ( a INT, KEY k ( a ) TYPE hnsw WITH ( M = 6 ) ); ERROR 7034 (HY000): A VECTOR index may only contain a vector type column.

Vector indexes have the type (algorithm) SE_SPECIFIC, and we add a column option in the data dictionary saying `vector_index=1;` which gets picked up by dedicated code in the data dictionary and the handler part of InnoDB. In the SQL layer, the vector index is very much a thing; there is an `HA_KEY_ALG_VECTOR`, an `HA_VECTOR` and a `KEYTYPE_VECTOR`. Extra SQL is added to display the type of a vector index as VECTOR rather than SE_SPECIFIC.

Adding the syntax `TYPE <ident> WITH ( <ident> = <ident>...) ` to index creation syntax. E.g.: `CREATE INDEX <name> ( <table> ) TYPE hnsw WITH ( M = 6 )` The syntax in the `WITH` list, christened Index Construction Parameters in this commit, must be verified by the storage engine. There were no hooks for this in InnoDB so one has been added in check_engine(). We have to do it fairly early so that we can prevent table creation in the DD in case of errors. Hence, the index type and parameter list are validated inside the storage engine using a new interface validate_engine_attributes(). The index construction parameters are serialized as a string of key-value pairs inside the index's `option` field. The serialization and de-serialization happen entirely inside the Data Dictionary. To do: We will probably still need a hook to handle the case of an already-existing table with invalid attributes; there are no hooks for this.

percona-mhansson changed the title ~~Vector mvp syntax~~ PS-11203 Vector Index Syntax May 29, 2026

percona-mhansson force-pushed the vector-mvp-syntax branch 3 times, most recently from c055ddc to bac48eb Compare June 9, 2026 13:47

percona-mhansson force-pushed the vector-mvp-syntax branch from bac48eb to 6f38434 Compare June 9, 2026 14:21

percona-mhansson closed this Jun 9, 2026

percona-mhansson reopened this Jun 9, 2026

percona-mhansson force-pushed the vector-mvp-syntax branch 2 times, most recently from a985587 to 96481cb Compare June 10, 2026 14:24

percona-mhansson force-pushed the vector-mvp-syntax branch 5 times, most recently from 3c65f43 to 67722d7 Compare June 26, 2026 13:52

percona-mhansson force-pushed the vector-mvp-syntax branch 2 times, most recently from 09426a6 to b22f12a Compare July 21, 2026 06:29