Tables
Last updated
Was this helpful?
Last updated
Was this helpful?
Table nodes in your workflow can either be generated by running transform, running code in a notebook to output a table, or will be part of a dataset you've added. The main purpose of table nodes in workflows is to store data which you can sanity check and further query.
Derivative tables in a workflow behave quite similarly to throughout Redivis, where you can preview cells, view summary statistics, and run quick queries.
All table nodes also have an associated export interface to see different ways that you can use this data off of the Redivis platform.
You can create multiple transforms to operate on a single table, which allows you to create various branches within your workflow. To create a new transform, select the table or dataset and click the small + icon that appears under the bottom right corner of the node.
After you run a transform, you can investigate the downstream output table to get feedback on the success and validity of your querying operation – both the filtering criteria you've applied and the new features you've created.
Understanding at the content of an output table allows you perform important sanity checks at each step of your research process, answering questions like:
Did my filtering criteria remove the rows I expected?
Do my new variables contain the information I expect?
Does the distribution of values in a given variable make sense?
Have I dropped unnecessary variables?
If you haven't interacted with tables in your workflow for a while, these tables may become archived, which will temporarily limit your ability to view cells and query data in that table. This is done to prevent runaway storage costs, while leveraging the built-in reproducibility of workflows to allow you to unarchive the table and pick up where you last left off.
Note that the transform immediately upstream (or any additional upstream transforms, if multiple sequential tables are archived) is invalid, you'll have to resolve the invalid state before un-archiving the table.
As you work in a workflow, nodes colors and symbols will change on the tree view to help you keep track of your work progress.
Empty
White background
A table node will be empty when it contains no data because the upstream node either has not been executed or its execution resulted in no data.
Executed
Grey background
A table node will be grey when it has data that aligns with the contents of its upstream node.
Stale
Yellow background
A node will be stale when an upstream change has been made. This means the content of the node does not match the content of the node above it.
Sampled
Black circle with 1% icon
Incomplete access
All black background, or dashed borders
You don't have full access the node. Click on the Incomplete access button in the top bar to begin applying for access to the relevant datasets.
All table nodes have one upstream parent. You can view the table's data and metadata similarly to other on Redivis. You cannot edit or update the metadata here.
To sanity check the contents of a table node, you can inspecting the general characteristics, checking the of different variables, looking at the table's , or create a for more in-depth analysis.
If a table is archived, you can still see the name, row count, and variable names/types. To access , , or downstream of an archived table, you'll have to reconstitute the table by re-running upstream transforms.
If you've added a dataset to a workflow that contains (storage for unstructured data) you will see a table with a label in that dataset's list of tables. This is an automatically generated table, where every row represents one file. You can work with this table just like any other in the workflow, but the file_id variable will remain linked to the files in that dataset for use in a .
Click the Export table button in the table's right panel to see your and manage any access restrictions associated with data this table is derived from.
This means that you are using a 1% of the data. When a dataset has a sample, it will automatically default to it when added to a workflow. You can change this to the full sample and back at any time in the .
