Editing data

If the data you've uploaded needs to be edited, you can do so in Redivis. Users can view edits on a table to learn more about the data and how it was shaped.

Editing overview

In the dataset tables list, click the "Edit data" button on any table. (You might also end up here after you've finished uploading data for the first time.)

The first version of a dataset will always begin with an "Initial upload" block. Additional edit blocks are created by clicking the + Add new edit button on the current version.

Valid edit blocks will be automatically applied as you work, allowing you to audit your data as you go by clicking the "View table" button.

Edit block types

Initial upload

This block is only present on the first version, and cannot be removed. Click the "upload files" button to import data from various sources.

Replace files

Similar to the initial upload block, but only available on versions subsequent to v1.0. Replaces all content in the previous table with newly imported data. The replace block must always come first if it exists.

Append files

Similar to the "initial upload" block, with the same interface for importing data. However, the append block appends any uploaded data to the current table, matching on (case-insensitive) variable names.

The append block is the only edit to which other edits may be applied. You can add Recode, Delete, Retype, and Drop edit blocks to an append block, allowing you to clean only the records that are uploaded through that particular append block. This allows you to import data with different schema and clean them independently before they are all appended and combined into one coherent table.

Recode records

Allows you to change instances of a value within a variable to another value. For example, you can convert all instances of 'None' to NULL .

Delete records

Allows you to delete all records where a logical condition is true. For example, you could drop all records where PatientId IS NULL .

Drop variables

This edit block allows you to drop variables from the your upload and/or the previous version.

Retype variables

This edit block allows you to convert a variable to a different type, optionally replacing any invalid values with NULL when the "Set incompatible values to null" option is selected. Variable retyping follows standard variable retyping rules (note that converting floats to integers will automatically round them), and any invalid type transformations will prevent the version from being released.

When converting to date, date time, or time types, you will often need to specify a format string to read in data encoded in various display formats. Please consult the format string documentation for further information and examples.

Computing deltas

Each edit block can display the number of variables and records that were added, updated, and/or removed by that block. In some cases this can be computed immediately, but in other circumstances you will need to click the "Compute" text to view the modifications triggered by the particular edit. Note that any non-computed edits will automatically be computed when the version is released.

Auditing data

As edits are applied to a version, they will automatically be propagated to that version's table. It is highly recommended that you validate and audit this table before you release your version.

The version table behaves much like other tables across Redivis, with a few additional features specific to auditing data within the version. Note that the cell preview is not available on unreleased versions.

Variable changes

When viewing the table, variable level modifications will be highlighted, with added variables in green, modified variables in yellow, and deleted variables in red. Note that variable modifications only highlight changes at the variable level (that is, retyped or renamed), not whether individual values of that variable were modified.

Summary statistics

As in other tables, you can view variable summary statistics by clicking on any variable in your table. This allows you to see the general distribution of a variable and validate it against your expectations of the data.

Additionally, if you are inspecting a variable in a subsequent version that also existed in the previous version, clicking on that variable will open a comparison of the variable across the two versions. This can aid in identifying any unexpected changes due to any edits in the new version.

Querying data

Like other tables, the version table allows you to run custom SQL queries to create more customized statistics and metrics.

Additionally, anyone with edit access to the dataset will be able to use the unreleased version within a project, allowing them to merge the pending version with other tables and leverage various querying interfaces within the project.

Note that if any edits are made to a pre-released version, they will immediately propagate to any projects that are using that pre-released version.