Data sources
Last updated
Last updated
By default, you may upload data from your local computer, a public URL, or from another dataset or project on Redivis. However, Redivis supports numerous integrations for data ingest across common sources. You'll need to enable data sources in your account workspace settings in order to import data they contain.
When you the enable the data source, you'll be prompted to log into the corresponding account. Redivis will only ever read data from these sources when explicitly requested, and it will never modify or overwrite content.
Once configured, you'll see any added data sources appear as an option when uploading data.
You may import any object that you have read access to in GCS by specifying a bucket name and path to that object, in the form /my-bucket/path/to/file. You may import multiple objects at once by providing a prefix followed by wildcard characters, e.g.: /my-bucket/my-folder/* .
The following wildcard characters are supported:
* : Match any number of characters within the current directory level. For example, /my-bucket/my-folder/d* matches my-folder/data.csv , but not my-folder/data/text.csv
** : Match any number of characters across directory boundaries. For example, my-folder/d** will match both examples provided above
? : Match a single character. For example, /my-bucket/da??.csv matches /my-bucket/data.csv
[chars] : Match any of the specified characters once. For example, /my-bucket/[aeiou].csv matches any of the vowel characters followed by .csv
[char range] : Match any of the range of characters once. For example, /my-bucket/[0-9].csv matches any number followed by .csv
You may import any object that you have read access to in S3 by specifying a bucket name and path to that object, in the form /my-bucket/path/to/file. You may import multiple objects at once by providing a prefix followed by a wildcard character, following the same syntax and rules as outlined for Google Cloud Storage above.
You may import any file of valid format that you have stored within your Drive, including Google Sheets. Upon choosing as your import source, a modal will open that will allow you to browse and select files from your Google Drive.
You may import any table that you have read access to in BigQuery, including views, materialized views, and external tables. You must specify the table in the form project.dataset.table . To import multiple tables within a dataset, you may use wildcards. E.g., project.dataset.* or project.dataset.prefix* .
You may import any file of valid format that you have stored within Box. Upon choosing as your import source, a modal will open that will allow you to browse and select files from your Box.
Coming soon. Please contact support@redivis.com if this integration would be helpful for your use case so that we can prioritize.
You can import any table, which can be particularly helpful with ETL workloads where you want to import a cleaned version of your data (example). You can also import any uploaded files into your table, supporting workflows where tabular data is initially loaded as a file before being loaded into a table.
You must both have data access and the ability to export any table or file that you import.
All files on Redivis belong to a file index table. To import a file, first specify the index table, followed by a forward slash (/), and then the file name. E.g.:
To import multiple files at once, you can use wildcard characters, following the same pattern rules as specified for Google Cloud Storage above. E.g.:
You can reference any table on Redivis using the form user|organization.dataset|project.table. That is, specify its owner (a user or organization), its containing entity (a dataset or project), as well as the table name, separated by periods.
