Build your own site with Observable

This guide outlines an example static site hosted by the Redivis team. Follow along at the Observable on Redivis Github repository to see all the details and build your own!

Overview

It's easy to use of Redivis data anywhere on the web, either by interfacing programmatically via our client libraries, visualizing via one of our export integrations, or embedding a table directly in another webpage.

To present Redivis data in a fully customizable way, you may want to build your own dashboard-style site. The following example shows a static site built with Observable Framework, deployed via Github Pages, that connects to Redivis data using our client libraries.

To try a lighter-weight "dashboard" for analyzing and visualizing your data, you can use integrated Redivis notebooks. With notebooks, you can work directly in your Redivis project alongside relevant source data, leverage a breadth of techniques and libraries available in the Python, R, Stata or SAS ecosystems, and easily maintain a central asset to annotate, collaborate, and share.

1. Identify your Redivis data

Identify the data sources on Redivis that will provide the content to build your site. You may be interested in presenting facets of a dataset already hosted on Redivis, highlighting tables you uploaded yourself, or showing outputs generated in one of your projects. In our example, we'll use weather station locations contained in the GHCN Daily Weather dataset, and outputs from a public Redivis project that uses the same source data, to visualize locations and aggregate precipitation measurements from around the world.

To reference a Redivis table, choose the "Export table" option and navigate to the "Programmatic" tab (e.g., the GHCN Daily Weather "Stations" table), and see the code snippet for unique identifiers for the owner, dataset/project, and table. More details about referencing Redivis resources here.

2. Build your Observable project

With chosen data content in mind, the next step is to choose a set of tools to build our custom dashboard and deploy it to the web. Of the numerous choices, Observable Framework provides an approachable option with support for working with data in many programming languages (not just javascript), powerful visualization libraries built in, and excellent documentation. To start with a copy of our example dashboard, you can clone our public Github repo and follow the README.md instructions to develop a similar site in a local environment. To create a brand new project, and to reference additional details and development strategies, see the Observable Getting started guide.

3. Load data with a client library of choice

You'll need to generate a Redivis API access token with appropriate permissions. See Authorization in our API documentation to create your own token to read data from Redivis.

To successfully authorize data fetching functionality in your development environment, be sure to export a REDIVIS_API_TOKEN variable to your path, with the following terminal command: export REDIVIS_API_TOKEN = 'MY_API_TOKEN'

IMPORTANT: API access tokens operate like passwords, and can allow another party to access resources on your behalf.

You should never share tokens, and avoid committing to source control where collaborators may have access (either now or in the future). See the Deploy and share your site section below for details on how to use secrets to store a Redivis API token necessary for your project.

Writing a "data loader"

In Observable, a data loader is a file that accesses data and writes it to a standard output channel for consumption by renderers, to be ultimately displayed on your site.

In our example, we'll illustrate use of both javascript and python to pull data from Redivis into our dashboard, using the Redivis API via js and python client libraries.

Using the redivis-js client library

In a web environment, the redivis-js client library allows for fetching and manipulating data via simple javascript functions.

In our example, we first install the library by specifying the latest version in our package.json and running npm install in our development environment to install or update all specified packages.

Then, in the Observable data loader file redivis-js-precipitation.json.js , we authorize access, and then listVariables and listRows from our specified precipitation table.

Using the redivis-python client library

In a python environment, the redivis-python client library allows for fetching and manipulating data via simple python functions.

Using the redivis-python client library requires that you develop your web application in a virtual python environment. Observable documents this process here – essentially, create an environment with uv, activate it on your machine, and install relevant packages to the environment.

In our example, we'll create a requirements.txt file that specifies the latest version of the client library, both for development and future deployment. Running the following command in your virtual python environment will install pip, and then any package specified in the requirements.txt file.

python3 -m pip install --upgrade pip
pip install -r requirements.txt

Then, in the Observable data loader file redivis-python-geodata.json.py , we create a pandas dataframe from our specified stations table, and output the records.

4. Visualize data

With data loaded and parsed into JSON format, it can now be manipulated and presented as you see fit. Among a wealth of modern web visualization tools is Observable Plot, which provides a polished selection of simple visualizations and corresponding functions to group and filter data, and pairs nicely with our development framework.

Observable interactive visualzations

In our example, we use several Observable Plot functions to display our data on a map – using Plot.plot() with Plot.geo() to show a globe – and in a hex-binned histogram – using Plot.dot() and Plot.hexgrid() for data aggregation. We also add a range input to allow viewers to rotate the globe visualization.

Raw data used to build the above globe and histogram plots

Then, we use the built-in display command to print the JSON-shaped payloads, to give the viewer a quick look at the raw data.

5. Deploy and share your site

With our dashboard finalized, you can deploy your project anywhere on the web. Observable provides a quick command (npm run build) to generate static files to be hosted on a server of your choice, as well as a deployment service if you'd like to host with Observable.

Deploying to Observable

You can run npm run deploy in the command line, and follow the prompts to deploy your project to an address on Observable. Further configuration and permissioning is available through your Observable account.

Deploying to Github Pages

Github Pages provides a hosting service that's easy to integrate with your Github repository and any Github Actions needed to deploy your project.

In our example, we deployed our project to Github Pages in a few simple steps.

First, we need to specify a REDIVIS_API_TOKEN to support Redivis client library authorization in the live deployment. We used "action secret" within the Settings of the Github repo, which sets an environment variable REDIVIS_API_TOKEN to our appropriately scoped token, which is completely private while our repository remains public.

Next, we write a Github Action to build and deploy the project, which will run anytime we push to the main branch of our repository. The action is specified via the deploy.yaml file, and relies on a set of open-source actions and our own custom code to do the following steps:

• Checkout the code

• Set up a Node.js environment, with a specific node version (20.6)

• Install our python dependencies (via pip,specified in requirements.txt)

• Build the static files for the site

• Deploy the files (in the generated dist folder) to Github Pages via a specific branch (gh-pages)

After adding this file to your Github repository, any push to the main branch will trigger a deployment following the steps above, the status of which can be viewed under the Actions tab of the repository.

Finally, Redivis has hosted a number of different team projects at a custom endpoint specified in our organization's Github Pages settings, which you can see at labs.redivis.com.