Tables
As you use transforms to manipulate data in your project, you will generate derivative tables that contain the output of your transform. These tables serve as helpful snapshots throughout your project, where you can sanity check your results and perform further analysis.
Derivative tables in a project behave quite similarly to dataset tables, where you can preview cells, view summary statistics, and run quick queries.
If you haven't interacted with tables in your project for a while, these tables may become archived, which will temporarily limit your ability to view and query data in that table. This is done to prevent runaway storage costs, while leveraging the built-in reproducibility of projects to allow you to unarchive the table and pick up where you last left off.
The archival algorithm prioritizes tables that are large, quick to regenerate, and intermediary (not at the bottom of the tree). It currently does not archive tables less than 1GB; in many cases you may never interact with archived tables.
Archived tables still provide much of the metadata. To unarchive a table, you can re-run upstream transforms.
If a table is archived, you can still see the name, row count, and variable names/types. To access variable summary statistics, view cells, or run transforms downstream of an archived table, you'll have to reconstitute the table by re-running upstream transforms.
Note that the transform immediately upstream (or any additional upstream transforms, if multiple sequential tables are archived) is invalid, you'll have to resolve the invalid state before un-archiving the table.
Last modified 1yr ago