# Referencing resources

## Overview

In many parts of this API, we will be referencing [**tables**](https://docs.redivis.com/api/resource-definitions/table)**,** [**notebooks**](https://docs.redivis.com/api/resource-definitions/notebook)**,** and [**transforms**](https://docs.redivis.com/api/resource-definitions/transform) on Redivis, as well as their related [**datasets**](https://docs.redivis.com/api/resource-definitions/dataset) or [**workflows**](https://docs.redivis.com/api/resource-definitions/workflow). The API uses a consistent structure to uniquely identify these resources, as specified below. 

In the simplest case, resources are referenced by their name, alongside the name of the containing workflow or dataset, as well as the name the dataset or workflow owner. 

Additionally, [names may be escaped](#escaping-names) to handle whitespace and non-standard characters. You can also provide an [optional reference id](#reference-ids) to ensure that your references don't break when various resources get renamed.

## General structure

All tables on Redivis belong to either a dataset or a workflow. All datasets belong to either a user or organization, and all workflows belong to a user.

A table reference reflects this hierarchy, taking the following form:

```http
ownerName.workflowIdentifier|datasetIdentifier.tableIdentifer
```

The **ownerName** will represent the user or organization that owns the dataset / workflow.

The **workflowIdentifier** consists of the workflow name, followed by an optional `referenceId` prefaced by a colon. The name of the workflow may be [escaped](#escaping-names).

```http
workflowIdentifier = workflowName[:workflowReferenceId]
```

The **datasetIdentifier** consists of the dataset name, followed by an optional `referenceId` prefaced by a colon. The name of the dataset may be [escaped](#escaping-names). Additionally, the dataset identifier *may* contain a sample flag `:sample` as well as a version identifier. The version identifier will identify a particular version of the dataset (of the form, `v1_0`, the current version `current`, or the next (unreleased) version `next` . If no version is specified the current version will be used by default.

```http
datasetIdentifier = datasetName[:datasetReferenceId][:sample][:versionIdentifier]

versionIdentifier = v1_0|current|next
```

{% hint style="info" %}
Make sure to specify a **versionIdentifier** to avoid errors or inconsistent results when new versions get released, unless you explicitly want to always reference the latest version of the dataset.
{% endhint %}

The **tableIdentifier** consists of the table name, followed by an optional `referenceId` prefaced by a colon. The name of the table may be [escaped](#escaping-names).

```http
tableIdentifier = tableName[:tableReferenceId]
```

{% hint style="success" %}
Notebooks and transforms follow the same conventions as tables, though they always belong to workflows. A notebook / transform qualified reference will look like:

```
owner_name.workflow_identifier.notebook|transform_identifier
```

{% endhint %}

## Escaping names

The **user\_name** will never need to be escaped, as these names can only contain word characters (`[A-Za-z0-9_]`). These names are case-insensitive.

Dataset, workflow, and table names can contain a wide array of characters. To facilitate programmatic references, these names can be escaped with the following rules:

1. All non alpha-numeric and underscore characters in names and version tags are replaced by an underscore (`_`) character.
2. Multiple underscore characters are collapsed into one.
3. Leading and trailing underscores are removed.
4. All names are case-insensitive. 

For example:

* `Census dataset: 1940-1980` -> `census_dataset_1940_1980`
* `~~Leading and trailing characters.` -> `leading_and_trailing_characters`

{% hint style="info" %}
Uniqueness is enforced for all escaped names within the relevant scope. For example, all tables in a workflow and all datasets in an organization will have a unique escaped name.
{% endhint %}

{% hint style="warning" %}
If a name contains colons (`:`), periods (`.`), or backticks (`` ` ``), they **must** be escaped. 
{% endhint %}

## Reference Ids

While it is obviously convenient to reference tables, datasets, and workflows by their name, this presents a challenge if these resources get renamed over time. In order to avoid code breakage due to renames, each resource has a 4-character (lowercase, alphanumeric) **referenceId** associated with it. This identifier will always be unique to the relevant scope — that is, a table's `referenceId` is unique across all tables in its dataset / workflow, and the `referenceId` of a dataset or workflow is unique to all datasets and workflows for that user. 

{% hint style="info" %}
If you're writing code within Redivis (table queries, transforms, notebooks), the referenceId will generally be pre-populated for you. This section is generally only applicable if you're using the Redivis API from an external environment (e.g., a Jupyter notebook running on your computer). 
{% endhint %}

{% hint style="info" %}
For tables that belong to a dataset, the referenceId will be consistent across all versions of the dataset, as well as for sample tables. This allows for you to easily change the version / sample without needing to update any reference ids.
{% endhint %}

### Locating the reference id

#### In the URL bar

One of the easiest ways to find the referenceId is to look at your URL bar. For example, if we navigate to the daily observations table in the GHCN dataset, our URL will be: <https://redivis.com/datasets/7br5-41440fjzk/tables/6fff-2djw7v7mw>. Here you can see that the dataset's full identifier is `7br5-41440fjzk`. The referenceId is the first part of this identifier: `7br5`. Similarly, the referenceId for the table is `6fff`.

#### In the table export modal

You can also find this information by [navigating to the export modal](https://redivis.com/datasets/7br5-41440fjzk/tables?exportTable=6fff-2djw7v7mw\&exportDestination=programmatic) for a table, and clicking on the tab for programmatic export information. Make sure the "exact references" box is checked, and you'll see the referenceId included in the template.&#x20;

![Use the export interface to view referenceIds and example code](https://content.gitbook.com/content/GCgH8jTSmY8Vgwceiri5/blobs/65kOK4tnrIbfSNyVee9k/Screen%20Shot%202022-02-18%20at%2012.57.05.png)

#### On the dataset page

You can also find dataset-specific information by clicking on **API Information** link on the dataset overview page:

![](https://content.gitbook.com/content/GCgH8jTSmY8Vgwceiri5/blobs/rka3lUIhF5Bgim31G7v4/Screen%20Shot%20202a2-02-18%20at%2013.19.25.png)

#### Via the API

Finally, the `referenceId` is returned as a property on dataset and table resource via the API. Table's also have a `qualifiedReference` property, which represents the full reference to the table (`owner.dataset|workflow.table`), including all reference ids and version specifiers.

## Examples

We can reference the "Daily observations" table from the [GHCN Climatology dataset](https://redivis.com/datasets/7br5-41440fjzk) as:

{% tabs %}
{% tab title="SQL" %}

```sql
SELECT * FROM 
`demo.ghcn_daily_weather_data.daily_observations` 
LIMIT 100
```

{% endtab %}

{% tab title="Python" %}

```python
import redivis

user = redivis.user("demo")
dataset = user.dataset("ghcn_daily_weather_data")
table = dataset.table("daily_observations")

df = table.to_dataframe(max_results=100)
```

{% endtab %}

{% tab title="R" %}

```r
library(redivis)

user <- redivis$user("demo")
dataset <- user$dataset("ghcn_daily_weather_data")
table <- dataset$table("daily_observations")

data <- table$to_tibble(max_results=100)
```

{% endtab %}

{% tab title="JS" %}

```javascript
import * as redivis from 'redivis'

const table = redivis.user('demo')
    .dataset('ghcn_daily_weather_data')
    .table('daily_observations')

const rows = table.list_rows({ maxResults: 100} )
```

{% endtab %}

{% tab title="HTTP" %}

```http
https://redivis.com/api/v1
    /tables/demo.ghcn_daily_weather_data.daily_observations
```

{% endtab %}
{% endtabs %}

By default this uses the latest version of the dataset. If we want to work with version 1.0:

{% tabs %}
{% tab title="SQL" %}

```sql
SELECT * FROM 
`demo.ghcn_daily_weather_data:v1_0.daily_observations` 
LIMIT 100
```

{% endtab %}

{% tab title="Python" %}

```python
import redivis

user = redivis.user("demo")
dataset = user.dataset("ghcn_daily_weather_data", version="1.0")
# Alternatively, dataset = user.dataset("ghcn_daily_weather_data:v1_0")
table = dataset.table("daily_observations")

df = table.to_dataframe(max_results=100)
```

{% endtab %}

{% tab title="R" %}

```r
library(redivis)

user <- redivis$user("demo")
dataset <- user$dataset("ghcn_daily_weather_data", version="1.0")
# Alternatively, dataset <- user$dataset("ghcn_daily_weather_data:v1_0")
table <- dataset$table("daily_observations")

data <- table$to_tibble(max_results=100)
```

{% endtab %}

{% tab title="JS" %}

```javascript
import * as redivis from 'redivis'

const table = redivis.user('demo')
    .dataset('ghcn_daily_weather_data', { version: '1.0' })
    // Alternatively, dataset("ghcn_daily_weather_data:v1_0")
    .table('daily_observations')

const rows = table.list_rows({ maxResults: 100} )
```

{% endtab %}

{% tab title="HTTP" %}

```http
https://redivis.com/api/v1
    /tables/demo.ghcn_daily_weather_data:v1_0.daily_observations
```

{% endtab %}
{% endtabs %}

If we want to work with the 1% sample:

{% tabs %}
{% tab title="SQL" %}

```sql
SELECT * FROM 
`demo.ghcn_daily_weather_data:sample.daily_observations`
LIMIT 100
```

{% endtab %}

{% tab title="Python" %}

```python
import redivis

user = redivis.user("demo")
dataset = user.dataset("ghcn_daily_weather_data", sample=True)
# Alternatively, dataset = user.dataset("ghcn_daily_weather_data:sample")
table = dataset.table("daily_observations")

df = table.to_dataframe(max_results=100)
```

{% endtab %}

{% tab title="R" %}

```r
library(redivis)

user <- redivis$user("demo")
dataset <- user$dataset("ghcn_daily_weather_data", sample=TRUE)
# Alternatively, dataset <- user$dataset("ghcn_daily_weather_data:sample")
table <- dataset$table("daily_observations")

data <- table$to_tibble(max_results=100)
```

{% endtab %}

{% tab title="JS" %}

```javascript
import * as redivis from 'redivis'

const table = redivis.user('demo')
    .dataset('ghcn_daily_weather_data', { sample: true })
    // Alternatively, dataset("ghcn_daily_weather_data:sample")
    .table('daily_observations')

const rows = table.list_rows({ maxResults: 100} )
```

{% endtab %}

{% tab title="HTTP" %}

```http
https://redivis.com/api/v1
    /tables/demo.ghcn_daily_weather_data:sample.daily_observations
```

{% endtab %}
{% endtabs %}

Finally, we can provide [referenceIds](#reference-ids) to prevent things from breaking if a dataset or table is renamed. Make sure also to specify a specific dataset version to avoid changes when a new version is released:

{% tabs %}
{% tab title="SQL" %}

```sql
SELECT * FROM 
`demo.ghcn_daily_weather_data:7br5:v1_1:.daily_observations:6fff` 
LIMIT 100
```

{% endtab %}

{% tab title="Python" %}

```python
import redivis

user = redivis.user("demo")
dataset = user.dataset("ghcn_daily_weather_data:7br5:v1_1")
table = dataset.table("daily_observations:6fff")

df = table.to_dataframe(max_results=100)
```

{% endtab %}

{% tab title="R" %}

```r
library(redivis)

user <- redivis$user("demo")
dataset <- user$dataset("ghcn_daily_weather_data:7br5:v1_1")
table <- dataset$table("daily_observations:6fff")

data <- table$to_tibble(max_results=100)
```

{% endtab %}

{% tab title="JS" %}

```javascript
import * as redivis from 'redivis'

const table = redivis.user('demo')
    .dataset('ghcn_daily_weather_data:7br5:v1_1')
    .table('daily_observations:6fff')

const rows = table.list_rows({ maxResults: 100} )
```

{% endtab %}

{% tab title="HTTP" %}

```http
https://redivis.com/api/v1
    /tables/demo.ghcn_daily_weather_data:7br5:v1_1:.daily_observations:6fff
```

{% endtab %}
{% endtabs %}

Referencing [tables in a workflow](https://redivis.com/projects/t066-0pp19xtmt/tables/132398) is quite similar, though workflows don’t have versions or samples:

{% tabs %}
{% tab title="SQL" %}

```sql
SELECT * FROM 
imathews.demo_workflow:t066.annual_precipitation:vdwn
LIMIT 100
```

{% endtab %}

{% tab title="Python" %}

```python
import redivis

user = redivis.user("imathews")
workflow = user.workflow("demo_workflow:t066")
table = workflow.table("annual_precipitation:vdwn")

df = table.to_dataframe(max_results=100)
```

{% endtab %}

{% tab title="R" %}

```r
library(redivis)

user <- redivis$user("imathews")
workflow <- user$workflow("demo_workflow:t066")
table <- workflow$table("annual_precipitation:vdwn")

data <- table$to_tibble(max_results=100)
```

{% endtab %}

{% tab title="JS" %}

```javascript
import * as redivis from 'redivis'

const table = redivis.user('imathews')
    .dataset('demo_workflow:t066')
    .table('annual_precipitation:vdwn')

const rows = table.list_rows({ maxResults: 100} )
```

{% endtab %}

{% tab title="HTTP" %}

```http
https://redivis.com/api/v1
    /tables/imathews.demo_workflow:t066.annual_precipitation:vdwn
```

{% endtab %}
{% endtabs %}