Google BigQuery

The table export modal Integrations tab allows export of Redivis tables to Google BigQuery.

To export data to BigQuery, select Export, then fill out the subsequent modal.

You'll need to authenticate a Google account, and then specify the BigQuery project, BigQuery dataset, and BigQuery table.

Once you confirm the export, your data will begin transferring to the specified BigQuery location and will provide a live-updating status of the transfer.

Exporting to BigQuery table in a VPC

If the Google Cloud project containing your desired BigQuery table is associated with a Virtual Private Cloud (VPC), you will need to configure an ingress policy on your Google Cloud organization to allow data export from Redivis to that table.

Configure on Google Cloud Console

To configure your VPC to allow Redivis to export to BigQuery:

1. Go to your Google Cloud Console's VPC Service Controls dashboard and select your VPC

   1. Select your Google organization in the top-left project picker (rather than a specific project) to list your organization's VPCs. You'll need a VPC Service Controls admin role to make changes.

2. Select the perimeter that contains the desired bucket's project to see the Service perimeter details

   1. If BigQuery API is listed under Restricted Services, you will need to configure an ingress policy to write to the bucket.

   2. Confirm the BigQuery table's project is listed under Projects.

   
3. Select Edit, then under Ingress policy, select Add an ingress rule, and configure the following options:

   1. From, identities: Add user:[e-mail] to allow just your Redivis user to export, or ANY_USER_ACCOUNT to allow anyone to export from Redivis.

   2. From, sources: Add the following Google Cloud project: som-phs-redivis-prod. This project is managed by Redivis, and ensure that exports from only the Redivis platform can land in your bucket.

   3. To, resources: Add your BigQuery table's Google Cloud project.

   4. To, operations or IAM roles: Add the bigquery.googleapis.com to allow writes to your BigQuery table.

4. Save the policy, and then export any Redivis table to your BigQuery table, per the instructions above.

Configure with gcloud CLI tool

To do the above configuration using the gcloud CLI tool, you can use the following shell script, using the YAML file below. You'll need the VPC name and access policy ID to modify it, as well as the Google Cloud project number.

If your VPC has existing ingress policies, you should use perimeters describe (or dry-run describe) to get the existing YAML configuration, and merge it with the file below.

# Dry-run config (recommended first)
gcloud access-context-manager perimeters dry-run update PERIMETER_NAME \
  --set-ingress-policies=ingress.yaml \
  --policy=POLICY_ID

# Enforced config, once validated
gcloud access-context-manager perimeters update PERIMETER_NAME \
  --set-ingress-policies=ingress.yaml \
  --policy=POLICY_ID

Use the following ingress.yaml file, which contains ingress rule described above.

# ingress.yaml
- ingressFrom:
    identities:
      - user:[e-mail]
    sources:
      - resource: projects/936051303296
  ingressTo:
    resources:
      - projects/DESTINATION_PROJECT_NUMBER
    operations:
      - serviceName: bigquery.googleapis.com
        methodSelectors:
          - method: '*'