Links

Core concepts

Overview

It might be useful to think of a project as a folder which contains all of the work you and your collaborators have done to reshape and analyze a data.
In a Redivis project this work is represented visually on the left side of the screen in a tree layout where you can build nodes and easily see their relationships. Clicking on a node will allow you to dive into that entity to explore or execute changes.

Project tree

Use this visual interface to create new nodes or view existing nodes in your project by clicking on them.

Node types

Dataset nodes represent datasets across Redivis.
Table nodes are either dataset tables, or the resulting output table of the upstream transform.
Transform nodes are queries that are used to reshape data by creating new tables.
Notebook nodes are code blocks (and their outputs) which are used to analyze data.

Project actions

From the black project tool bar you can:
  • Add data to the project workspace
  • Search by name for a node and jump to it
  • Run all transforms in bulk
  • Create value lists to centrally manage project parameters
Within the map you can right click on nodes to open its action menu where you can:
  • Update learn more about the node (Rename, add description, view access, delete)
  • Build your project (create new nodes, copy and paste nodes, insert nodes)

Layout

The project tree automatically creates a grid layout of all the nodes in your project as you go, helping to keep it organized as your project grows. To shift a dataset or notebook node, select the node and click the arrow that appears.
Note that shifting nodes is purely an organizational tool; it has no effect on the data produced in the project.

Node states

The state of the work you're doing in a project is captured in the project tree where you can get an overview of the project status at a glance.
State
Display
Text
Details
Empty
White background
Transforms Tables
A node will be empty when it has never been run (transform) or has not been created yet (transform output table).
Executed
Grey background
All nodes
A node will be grey when it has no action needed and has not changed.
Invalid
Black exclamation icon
Transforms
A transform will be invalid when it is unable to be run. This might be because you haven't finished building the steps or have input invalid information.
Errored
Red exclamation icon
Transforms
A transform will be errored when you run it and the run can't be completed. This might be due to an incorrect input you've set that our validator can't catch. Or something might have gone wrong while executing and you'll just need to rerun it.
Edited
Grey hash marks
Transforms
A transform will be edited when you revisit a successfully run transform and change a parameter. You can either Run this transform or Revert to its previously run state to resolve it. Editing a transform makes the downstream table stale.
Stale
Yellow background
Transforms Tables Notebooks
A node will be stale when an upstream change has been made. This means the content of the node does not match the content of the node above it.
Queued
Static circular icon
Transforms
When you run multiple nodes they will be executed in a logical order. Any nodes that are waiting for an upstream run to finish before running themselves are queued and will automatically run when possible.
Running
Spinning circular icon
Transforms
A node that is actively running. You can cancel running (or queued) transforms by clicking the Run menu in the top bar and selecting Cancel. If a node is currently running it might not be able to cancel, depending on what point in the process it's at.
Incomplete access
All black background, or dashed borders
All nodes
This means that you don't have full access the node. Click on these nodes and then the Incomplete access button in the top bar to begin applying for access to the relevant datasets.
Sampled
Black circle with 1% icon
Datasets Tables
This means that you are using a 1% sample of the data. When a dataset has a sample, it will automatically default to it when added to a project. You can change this to the full sample and back at any time in the dataset node
Outdated version
Purple background
Datasets
For datasets this means that you are not using the latest version. This means that you have either intentionally switched to using an older version, or that this dataset's administrator has released a new version that you can to.

Deleting nodes

To delete a node, right click on a dataset or transform node and select Delete.
When deleting a transform
  • The transform and output table will be deleted; every transform must have an output table to record results of that transform.
  • If the project tree has additional nodes downstream, the transform and output table will be 'spliced' out, i.e. the upstream node nearest the deleted transform will be connected to the downstream node nearest to the deleted output table. Note that this deletion will cause the next downstream transform to receive new input variables from the node that's directly upstream. (In the above example, deleting the selected transform will result in the 'Optum SES Inpatient Confinement' dataset being connected directly to the remaining transform, which will change the variables available to work with in that transform.)
When deleting a dataset or dataset table:
  • The dataset and all downstream nodes will be deleted. If additional branches are joined into the branch downstream of the deleted dataset, those branches will be retained up to but not including the transform located in the deleted branch.
Since you can't undo a deletion, you'll receive a warning message before proceeding.
As you make changes in a project you will change the status of different nodes connected to it. These changes in status are shown in the left panel of the project to help you keep track of any changes.