BigQuery

STEP 1

Step 1

Navigate to Integrations section of our portal and click Add Integration: Integrations.

STEP 2

Step 2

Select BigQuery and fill in the following fields:

• Schedule: How often Cloud Avenue should refresh metadata from BigQuery.
• API Secret: Paste the full contents of your GCP service account JSON key.
• Project ID: Your GCP project ID (field key: project_id).

Optional Filters

Depending on your account configuration, you may also see optional filters to scope what gets ingested. These accept comma-separated patterns:

• Schema Filter
• Database Filter
• Table Filter
• View Filter

Creating a Service Account, Custom Role, and JSON Key (GCP)

This integration uses a GCP service account to authenticate. The recommended approach is:

1. Create a custom IAM role with the minimum required permissions
2. Create a service account
3. Grant the custom role to the service account
4. Create a JSON key and paste it into API Secret

1) Create a custom IAM role

In the Google Cloud Console:

1. Go to IAM & Admin -> Roles
2. Click Create role
3. Set a title (for example: CloudAvenue BigQuery Metadata Reader)
4. Add the following permissions:

#	GCP Permission	Required For
1	bigquery.datasets.get
2	bigquery.tables.get
3	bigquery.tables.getData
4	bigquery.tables.list
5	resourcemanager.projects.get
6	bigquery.jobs.create
7	bigquery.jobs.listAll
8	bigquery.routines.get
9	bigquery.routines.list
10	datacatalog.taxonomies.get
11	datacatalog.taxonomies.list
12	bigquery.readsessions.create
13	bigquery.readsessions.getData
14	logging.operations.list

Reference (GCP):

• https://docs.cloud.google.com/iam/docs/creating-custom-roles#iam-custom-roles-get-metadata-console

2) Create a service account

1. Go to IAM & Admin -> Service Accounts
2. Click Create service account
3. Choose a service account name and create it

Reference (GCP):

• https://docs.cloud.google.com/iam/docs/service-accounts-create

3) Grant the custom role to the service account

1. Go to IAM & Admin -> IAM
2. Click Grant access
3. Add the service account principal (the ...@...iam.gserviceaccount.com email)
4. Select the custom role you created
5. Click Save

Reference (GCP):

• https://docs.cloud.google.com/iam/docs/grant-role-console

4) Create and download a JSON key

1. Go to IAM & Admin -> Service Accounts
2. Select the service account
3. Go to the Keys tab
4. Click Add key -> Create new key -> JSON
5. Download the key, open it, and copy the full JSON contents

Paste the full JSON into the integration's API Secret field.