Overview

Variables are present in all tables on Redivis. Conceptually, they represent something that is being measured in the data. When viewed in the table cells view, variables are represented by the table's columns.

In order to view variables, you must have metadata access the source dataset(s) for the given table.

Characteristics

The characteristics on a variable help researchers understand what that variable measures and how it was collected. All characteristics (except for variable type) are indexed by the Redivis search engine — better and accurate metadata will help researchers find your dataset!

These characteristics will only be considered in search results if the user has metadata access to the underlying dataset.

Univariate statistics

Redivis automatically computes certain statistics for all variables, depending on that variable's type and number of distinct values. To view summary statistics, click on a variable in a table.

Univariate plots

Histogram

A frequency chart of observations, sorted into 64 buckets, clamped to three standard deviations of the mean on either side. To switch between a linear and logarithmic y-axis, click on the bottom left corner of the chart.

Only shown for continuous variables with more than 64 distinct values.

Box plot

A visual display of the distribution of values by frequency. Shown are the minimum, 25%, median, 75% and maximum value. To include or exclude outliers in the calculations, click on the label on the bottom left corner of the chart.

Only shown for continuous variables with a meaningful number of discrete values.

Frequency table

A table showing the frequency of common values for that variable, limited to 10,000 values. If a variable's values are highly heterogenous, no frequency table will be displayed.

Editing metadata

To edit a variable's label, description, or value labels, click the Edit metadata button on the right of any table. This will allow you to any of the variables in this table, or you can navigate to the "All tables" tab to edit the same variable across multiple tables.

In the metadata editor, you can edit a variable's metadata fields as you would in any spreadsheet, and save changes when you're done.

Uploading metadata

Redivis determines variable names and types during data upload. Additionally, it will automatically parse certain metadata based on the uploaded file format:

• SAS (.sas7bdat): labels

• Stata (.dta): labels and value labels

• SPSS (.sav): labels and value labels

For other file types (e.g., csv), you will need to augment the metadata directly. To apply metadata in bulk, you can upload a file containing metadata information directly from your computer. This file can either be a CSV or JSON.

CSV metadata format

The CSV should be formatted without no header, each row corresponding to a variable, with column 1 as the name, 2 as the label, 3 as the description. If the variable doesn't have a label or description, leave these columns empty.

variable1_name,variable1_label,variable1_description
variable2_name,variable2_label,variable2_description

For example:

sex,patient sex,patient's recorded sex
id,patient identifier,unique patient identifier

JSON metadata format

When uploading a JSON file, specify the name, label, description, and valueLabels using the appropriately named attributes in the object corresponding to each variable. If the variable doesn't have a label, description, or value labels, you don't need to include these attributes.

For example:

// JSON format is an array of objects, with each object representing a variable
[
    {
        "name": "sex",
        "label": "patient sex",
        "description": "patient's recorded sex",
        "valueLabels": [
            {
                "value": 1,
                "label": "Male"
            },
            {
                "value": 2,
                "label": "Female"
            }
        ]
    }
]

Exporting metadata

You can export your metadata at any time (in either of the above formats) by clicking Export file in the variable metadata editor. Variable metadata is also available in JSON format via the variables.list API endpoint.

Variable types

All variables in Redivis have a type associated with them. All types in Redivis support NULL values; that is, an empty cell.

String

A string can be used to store any textual data (UTF-8 encoded). Moreover, a string is a "universal" data type — any other type can be converted to a string without error.

Size: 2 bytes + UTF-8 encoded string size (1 byte for ASCII characters).

Integer

A 64-bit signed integer. Supports any integer value between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 .

Size: 8 bytes per cell.

Float

A 64-bit (double precision) floating point decimal value.

Size: 8 bytes per cell.

Boolean

A representation of either TRUE or FALSE

Size: 1 byte per cell.

Date

Represents a calendar date independent of time zone. Supports any value between 0001-01-01 and 9999-12-31

Size: 8 bytes per cell

Format:

'YYYY-[M]M-[D]D'

• YYYY: Four digit year

• [M]M: One or two digit month

• [D]D: One or two digit day

All values in brackets are optional. If your Date is not in this format, you can convert it using format strings.

DateTime

Represents a year, month, day, hour, minute, second, and subsecond, independent of timezone. Supports any value between 0001-01-01 00:00:00 and 9999-12-31 23:59:59.999999

Size: 8 bytes per cell

Format:

'YYYY-[M]M-[D]D[( |T)[H]H:[M]M:[S]S[.DDDDDD]]'

• YYYY: Four digit year

• [M]M: One or two digit month

• [D]D: One or two digit day

• ( |T): A space or a Tseparator

• [H]H: One or two digit hour (valid values from 00 to 23)

• [M]M: One or two digit minutes (valid values from 00 to 59)

• [S]S: One or two digit seconds (valid values from 00 to 59)

• [.DDDDDD]: Up to six fractional digits (i.e. up to microsecond precision)

All values in brackets are optional. If your DateTime is not in this format, you can convert it using format strings.

Time

Represents a time, independent of a specific date. Supports values between 00:00:00 and 23:59:59.999999

Size: 8 bytes per cell

Format:

'[H]H:[M]M:[S]S[.DDDDDD]'

• [H]H: One or two digit hour (valid values from 00 to 23)

• [M]M: One or two digit minutes (valid values from 00 to 59)

• [S]S: One or two digit seconds (valid values from 00 to 59)

• [.DDDDDD]: Up to six fractional digits (i.e. up to microsecond precision)

All values in brackets are optional. If your Time is not in this format, you can convert it using format strings​.​

Conversion rules

Some types can be implicitly converted within a query (e.g., 1 (integer) < 2.2 (float)), in other circumstances, you will explicitly need to convert the type of a variable before performing certain operations (e.g., "1.0" (string) < 2.2 (float)).

Variable types can be converted within the transform interface in a project. The following conversions are supported: