googleVertexAiIndex

A representation of a collection of database items organized in a way that allows for approximate nearest neighbor (a.k.a ANN) algorithms search.

To get more information about Index, see:

API documentation

Example Usage - Vertex Ai Index

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleStorageBucketBucket = new google.storageBucket.StorageBucket(
  this,
  "bucket",
  {
    location: "us-central1",
    name: "vertex-ai-index-test",
    uniform_bucket_level_access: true,
  }
);
new google.storageBucketObject.StorageBucketObject(this, "data", {
  bucket: googleStorageBucketBucket.name,
  content:
    '{"id": "42", "embedding": [0.5, 1.0], "restricts": [{"namespace": "class", "allow": ["cat", "pet"]},{"namespace": "category", "allow": ["feline"]}]}\n{"id": "43", "embedding": [0.6, 1.0], "restricts": [{"namespace": "class", "allow": ["dog", "pet"]},{"namespace": "category", "allow": ["canine"]}]}\n',
  name: "contents/data.json",
});
new google.vertexAiIndex.VertexAiIndex(this, "index", {
  description: "index for test",
  display_name: "test-index",
  index_update_method: "BATCH_UPDATE",
  labels: [
    {
      foo: "bar",
    },
  ],
  metadata: [
    {
      config: [
        {
          algorithm_config: [
            {
              tree_ah_config: [
                {
                  leaf_node_embedding_count: 500,
                  leaf_nodes_to_search_percent: 7,
                },
              ],
            },
          ],
          approximate_neighbors_count: 150,
          dimensions: 2,
          distance_measure_type: "DOT_PRODUCT_DISTANCE",
        },
      ],
      contents_delta_uri: `gs://\${${googleStorageBucketBucket.name}}/contents`,
    },
  ],
  region: "us-central1",
});

Example Usage - Vertex Ai Index Streaming

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleStorageBucketBucket = new google.storageBucket.StorageBucket(
  this,
  "bucket",
  {
    location: "us-central1",
    name: "vertex-ai-index-test",
    uniform_bucket_level_access: true,
  }
);
new google.storageBucketObject.StorageBucketObject(this, "data", {
  bucket: googleStorageBucketBucket.name,
  content:
    '{"id": "42", "embedding": [0.5, 1.0], "restricts": [{"namespace": "class", "allow": ["cat", "pet"]},{"namespace": "category", "allow": ["feline"]}]}\n{"id": "43", "embedding": [0.6, 1.0], "restricts": [{"namespace": "class", "allow": ["dog", "pet"]},{"namespace": "category", "allow": ["canine"]}]}\n',
  name: "contents/data.json",
});
new google.vertexAiIndex.VertexAiIndex(this, "index", {
  description: "index for test",
  display_name: "test-index",
  index_update_method: "STREAM_UPDATE",
  labels: [
    {
      foo: "bar",
    },
  ],
  metadata: [
    {
      config: [
        {
          algorithm_config: [
            {
              brute_force_config: [{}],
            },
          ],
          dimensions: 2,
          distance_measure_type: "COSINE_DISTANCE",
          feature_norm_type: "UNIT_L2_NORM",
        },
      ],
      contents_delta_uri: `gs://\${${googleStorageBucketBucket.name}}/contents`,
    },
  ],
  region: "us-central1",
});

Argument Reference

The following arguments are supported:

displayName - (Required) The display name of the Index. The name can be up to 128 characters long and can consist of any UTF-8 characters.

description - (Optional) The description of the Index.
metadata - (Optional) An additional information about the Index Structure is documented below.
labels - (Optional) The labels with user-defined metadata to organize your Indexes.
indexUpdateMethod - (Optional) The update method to use with this Index. The value must be the followings. If not set, BATCH_UPDATE will be used by default.
- BATCH_UPDATE: user can call indexes.patch with files on Cloud Storage of datapoints to update.
- STREAM_UPDATE: user can call indexes.upsertDatapoints/DeleteDatapoints to update the Index and the updates will be applied in corresponding DeployedIndexes in nearly real-time.
region - (Optional) The region of the index. eg us-central1
project - (Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

The metadata block supports:

contentsDeltaUri - (Optional) Allows inserting, updating or deleting the contents of the Matching Engine Index. The string must be a valid Cloud Storage directory path. If this field is set when calling IndexService.UpdateIndex, then no other Index field can be also updated as part of the same call. The expected structure and format of the files this URI points to is described at https://cloud.google.com/vertex-ai/docs/matching-engine/using-matching-engine#input-data-format
isCompleteOverwrite - (Optional) If this field is set together with contentsDeltaUri when calling IndexService.UpdateIndex, then existing content of the Index will be replaced by the data from the contentsDeltaUri.
config - (Optional) The configuration of the Matching Engine Index. Structure is documented below.

The config block supports:

dimensions - (Required) The number of dimensions of the input vectors.
approximateNeighborsCount - (Optional) The default number of neighbors to find via approximate search before exact reordering is performed. Exact reordering is a procedure where results returned by an approximate search algorithm are reordered via a more expensive distance computation. Required if tree-AH algorithm is used.
distanceMeasureType - (Optional) The distance measure used in nearest neighbor search. The value must be one of the followings:
- SQUARED_L2_DISTANCE: Euclidean (L_2) Distance
- L1_DISTANCE: Manhattan (L_1) Distance
- COSINE_DISTANCE: Cosine Distance. Defined as 1 - cosine similarity.
- DOT_PRODUCT_DISTANCE: Dot Product Distance. Defined as a negative of the dot product
featureNormType - (Optional) Type of normalization to be carried out on each vector. The value must be one of the followings:
- UNIT_L2_NORM: Unit L2 normalization type
- NONE: No normalization type is specified.
algorithmConfig - (Optional) The configuration with regard to the algorithms used for efficient search. Structure is documented below.

The algorithmConfig block supports:

treeAhConfig - (Optional) Configuration options for using the tree-AH algorithm (Shallow tree + Asymmetric Hashing). Please refer to this paper for more details: https://arxiv.org/abs/1908.10396 Structure is documented below.
bruteForceConfig - (Optional) Configuration options for using brute force search, which simply implements the standard linear search in the database for each query.

The treeAhConfig block supports:

leafNodeEmbeddingCount - (Optional) Number of embeddings on each leaf node. The default value is 1000 if not set.
leafNodesToSearchPercent - (Optional) The default percentage of leaf nodes that any query may be searched. Must be in range 1-100, inclusive. The default value is 10 (means 10%) if not set.

Attributes Reference

In addition to the arguments listed above, the following computed attributes are exported:

id - an identifier for the resource with format projects/{{project}}/locations/{{region}}/indexes/{{name}}
name - The resource name of the Index.
metadataSchemaUri - Points to a YAML file stored on Google Cloud Storage describing additional information about the Index, that is specific to it. Unset if the Index does not have any additional information.
deployedIndexes - The pointers to DeployedIndexes created from this Index. An Index can be only deleted if all its DeployedIndexes had been undeployed first. Structure is documented below.
etag - Used to perform consistent read-modify-write updates.
createTime - The timestamp of when the Index was created in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits.
updateTime - The timestamp of when the Index was last updated in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits.
indexStats - Stats of the index resource. Structure is documented below.

The deployedIndexes block contains:

indexEndpoint - (Output) A resource name of the IndexEndpoint.
deployedIndexId - (Output) The ID of the DeployedIndex in the above IndexEndpoint.

The indexStats block contains:

vectorsCount - (Output) The number of vectors in the Index.
shardsCount - (Output) The number of shards in the Index.

Timeouts

This resource provides the following Timeouts configuration options:

create - Default is 60 minutes.
update - Default is 60 minutes.
delete - Default is 60 minutes.

Import

Index can be imported using any of these accepted formats:

$ terraform import google_vertex_ai_index.default projects/{{project}}/locations/{{region}}/indexes/{{name}}
$ terraform import google_vertex_ai_index.default {{project}}/{{region}}/{{name}}
$ terraform import google_vertex_ai_index.default {{region}}/{{name}}
$ terraform import google_vertex_ai_index.default {{name}}

User Project Overrides

This resource supports User Project Overrides.