Skip to content

googleBigqueryDataset

Datasets allow you to organize and control access to your tables.

To get more information about Dataset, see:

\~> Warning: You must specify the role field using the legacy format owner instead of roles/bigqueryDataOwner. The API does accept both formats but it will always return the legacy format which results in Terraform showing permanent diff on each plan and apply operation.

Open in Cloud Shell

Example Usage - Bigquery Dataset Basic

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleServiceAccountBqowner = new google.serviceAccount.ServiceAccount(
  this,
  "bqowner",
  {
    account_id: "bqowner",
  }
);
new google.bigqueryDataset.BigqueryDataset(this, "dataset", {
  access: [
    {
      role: "OWNER",
      user_by_email: googleServiceAccountBqowner.email,
    },
    {
      domain: "hashicorp.com",
      role: "READER",
    },
  ],
  dataset_id: "example_dataset",
  default_table_expiration_ms: 3600000,
  description: "This is a test description",
  friendly_name: "test",
  labels: [
    {
      env: "default",
    },
  ],
  location: "EU",
});

Example Usage - Bigquery Dataset Cmek

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleKmsKeyRingKeyRing = new google.kmsKeyRing.KmsKeyRing(
  this,
  "key_ring",
  {
    location: "us",
    name: "example-keyring",
  }
);
const googleKmsCryptoKeyCryptoKey = new google.kmsCryptoKey.KmsCryptoKey(
  this,
  "crypto_key",
  {
    key_ring: googleKmsKeyRingKeyRing.id,
    name: "example-key",
  }
);
new google.bigqueryDataset.BigqueryDataset(this, "dataset", {
  dataset_id: "example_dataset",
  default_encryption_configuration: [
    {
      kms_key_name: googleKmsCryptoKeyCryptoKey.id,
    },
  ],
  default_table_expiration_ms: 3600000,
  description: "This is a test description",
  friendly_name: "test",
  location: "US",
});
Open in Cloud Shell

Example Usage - Bigquery Dataset Authorized Dataset

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleServiceAccountBqowner = new google.serviceAccount.ServiceAccount(
  this,
  "bqowner",
  {
    account_id: "bqowner",
  }
);
const googleBigqueryDatasetPublic = new google.bigqueryDataset.BigqueryDataset(
  this,
  "public",
  {
    access: [
      {
        role: "OWNER",
        user_by_email: googleServiceAccountBqowner.email,
      },
      {
        domain: "hashicorp.com",
        role: "READER",
      },
    ],
    dataset_id: "public",
    default_table_expiration_ms: 3600000,
    description: "This dataset is public",
    friendly_name: "test",
    labels: [
      {
        env: "default",
      },
    ],
    location: "EU",
  }
);
new google.bigqueryDataset.BigqueryDataset(this, "dataset", {
  access: [
    {
      role: "OWNER",
      user_by_email: googleServiceAccountBqowner.email,
    },
    {
      domain: "hashicorp.com",
      role: "READER",
    },
    {
      dataset: [
        {
          dataset: [
            {
              dataset_id: googleBigqueryDatasetPublic.datasetId,
              project_id: googleBigqueryDatasetPublic.project,
            },
          ],
          target_types: ["VIEWS"],
        },
      ],
    },
  ],
  dataset_id: "private",
  default_table_expiration_ms: 3600000,
  description: "This dataset is private",
  friendly_name: "test",
  labels: [
    {
      env: "default",
    },
  ],
  location: "EU",
});

Example Usage - Bigquery Dataset Authorized Routine

/*Provider bindings are generated by running cdktf get.
See https://cdk.tf/provider-generation for more details.*/
import * as google from "./.gen/providers/google";
/*The following providers are missing schema information and might need manual adjustments to synthesize correctly: google.
For a more precise conversion please use the --provider flag in convert.*/
const googleBigqueryDatasetPublic = new google.bigqueryDataset.BigqueryDataset(
  this,
  "public",
  {
    dataset_id: "public_dataset",
    description: "This dataset is public",
  }
);
const googleBigqueryRoutinePublic = new google.bigqueryRoutine.BigqueryRoutine(
  this,
  "public_1",
  {
    arguments: [
      {
        argument_kind: "FIXED_TYPE",
        data_type: '${jsonencode({ "typeKind" = "INT64" })}',
        name: "value",
      },
    ],
    dataset_id: googleBigqueryDatasetPublic.datasetId,
    definition_body: "SELECT 1 + value AS value\n",
    language: "SQL",
    return_table_type:
      '${jsonencode({ "columns" = [\n    { "name" = "value", "type" = { "typeKind" = "INT64" } },\n  ] })}',
    routine_id: "public_routine",
    routine_type: "TABLE_VALUED_FUNCTION",
  }
);
/*This allows the Terraform resource name to match the original name. You can remove the call if you don't need them to match.*/
googleBigqueryRoutinePublic.overrideLogicalId("public");
new google.bigqueryDataset.BigqueryDataset(this, "private", {
  access: [
    {
      role: "OWNER",
      user_by_email: "my@service-account.com",
    },
    {
      routine: [
        {
          dataset_id: googleBigqueryRoutinePublic.datasetId,
          project_id: googleBigqueryRoutinePublic.project,
          routine_id: googleBigqueryRoutinePublic.routineId,
        },
      ],
    },
  ],
  dataset_id: "private_dataset",
  description: "This dataset is private",
});

Argument Reference

The following arguments are supported:

  • datasetId - (Required) A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

  • maxTimeTravelHours - (Optional) Defines the time travel window in hours. The value can be from 48 to 168 hours (2 to 7 days).

  • access - (Optional) An array of objects that define dataset access for one or more entities. Structure is documented below.

  • defaultTableExpirationMs - (Optional) The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour).

    Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property.

  • defaultPartitionExpirationMs - (Optional) The default partition expiration for all partitioned tables in the dataset, in milliseconds.

    Once this property is set, all newly-created partitioned tables in the dataset will have an expirationMs property in the timePartitioning settings set to this value, and changing the value will only affect new tables, not existing ones. The storage in a partition will have an expiration time of its partition time plus this value. Setting this property overrides the use of defaultTableExpirationMs for partitioned tables: only one of defaultTableExpirationMs and defaultPartitionExpirationMs will be used for any new partitioned table. If you provide an explicit timePartitioningExpirationMs when creating or updating a partitioned table, that value takes precedence over the default partition expiration time indicated by this property.

  • description - (Optional) A user-friendly description of the dataset

  • friendlyName - (Optional) A descriptive name for the dataset

  • labels - (Optional) The labels associated with this dataset. You can use these to organize and group your datasets

  • location - (Optional) The geographic location where the dataset should reside. See official docs.

    There are two types of locations, regional or multi-regional. A regional location is a specific geographic place, such as Tokyo, and a multi-regional location is a large geographic area, such as the United States, that contains at least two geographic places.

    The default value is multi-regional location us. Changing this forces a new resource to be created.

  • defaultEncryptionConfiguration - (Optional) The default encryption key for all tables in the dataset. Once this property is set, all newly-created partitioned tables in the dataset will have encryption key set to this value, unless table creation request (or query) overrides the key. Structure is documented below.

  • isCaseInsensitive - (Optional) TRUE if the dataset and its table names are case-insensitive, otherwise FALSE. By default, this is FALSE, which means the dataset and its table names are case-sensitive. This field does not affect routine references.

  • defaultCollation - (Optional) Defines the default collation specification of future tables created in the dataset. If a table is created in this dataset without table-level default collation, then the table inherits the dataset default collation, which is applied to the string fields that do not have explicit collation specified. A change to this field affects only tables created afterwards, and does not alter the existing tables. The following values are supported:

    • 'und:ci': undetermined locale, case insensitive.
    • '': empty string. Default to case-sensitive behavior.

  • project - (Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • deleteContentsOnDestroy - (Optional) If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present.

The access block supports:

  • domain - (Optional) A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • groupByEmail - (Optional) An email address of a Google Group to grant access to.

  • role - (Optional) Describes the rights granted to the user specified by the other member of the access object. Basic, predefined, and custom roles are supported. Predefined roles that have equivalent basic roles are swapped by the API to their basic counterparts. See official docs.

  • specialGroup - (Optional) A special group to grant access to. Possible values include:

    • projectOwners: Owners of the enclosing project.

    • projectReaders: Readers of the enclosing project.

    • projectWriters: Writers of the enclosing project.

    • allAuthenticatedUsers: All authenticated BigQuery users.

  • userByEmail - (Optional) An email address of a user to grant access to. For example: fred@example.com

  • view - (Optional) A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

  • dataset - (Optional) Grants all resources of particular types in a particular dataset read access to the current dataset. Structure is documented below.

  • routine - (Optional) A routine from a different dataset to grant access to. Queries executed against that routine will have read access to tables in this dataset. The role field is not required when this field is set. If that routine is updated by any user, access to the routine needs to be granted again via an update operation. Structure is documented below.

The view block supports:

  • datasetId - (Required) The ID of the dataset containing this table.

  • projectId - (Required) The ID of the project containing this table.

  • tableId - (Required) The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

The dataset block supports:

  • dataset - (Required) The dataset this entry applies to Structure is documented below.

  • targetTypes - (Required) Which resources in the dataset this entry applies to. Currently, only views are supported, but additional target types may be added in the future. Possible values: VIEWS

The dataset block supports:

  • datasetId - (Required) The ID of the dataset containing this table.

  • projectId - (Required) The ID of the project containing this table.

The routine block supports:

  • datasetId - (Required) The ID of the dataset containing this table.

  • projectId - (Required) The ID of the project containing this table.

  • routineId - (Required) The ID of the routine. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 256 characters.

The defaultEncryptionConfiguration block supports:

  • kmsKeyName - (Required) Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

Attributes Reference

In addition to the arguments listed above, the following computed attributes are exported:

  • id - an identifier for the resource with format projects/{{project}}/datasets/{{datasetId}}

  • creationTime - The time when this dataset was created, in milliseconds since the epoch.

  • etag - A hash of the resource.

  • lastModifiedTime - The date when this dataset or any of its tables was last modified, in milliseconds since the epoch.

  • selfLink - The URI of the created resource.

Timeouts

This resource provides the following Timeouts configuration options:

  • create - Default is 20 minutes.
  • update - Default is 20 minutes.
  • delete - Default is 20 minutes.

Import

Dataset can be imported using any of these accepted formats:

$ terraform import google_bigquery_dataset.default projects/{{project}}/datasets/{{dataset_id}}
$ terraform import google_bigquery_dataset.default {{project}}/{{dataset_id}}
$ terraform import google_bigquery_dataset.default {{dataset_id}}

User Project Overrides

This resource supports User Project Overrides.