Variables

Overview
Variables are present in all tables on Redivis. Conceptually, they represent something that is being measured in the data. When viewed in the table cells view, variables are represented by the table's columns.
In order to view variables, you must have metadata access the source dataset(s) for the given table.
Characteristics
The characteristics on a variable help researchers understand what that variable measures and how it was collected. All characteristics (except for variable type) are indexed by the Redivis search engine — better and accurate metadata will help researchers find your dataset! 
These characteristics will only be considered in search results if the user has metadata access to the underlying dataset.
Field
Notes
Name
The name of the variable, limited to 60 characters. Must be unique within the table. All variable names are case-insensitive, and can only use alpha-numeric characters and underscores, and cannot start with a number. 
Type
Required. The data type of the variable. Learn more about variable types.
Label
Optional. A short, human-readable description of the variable name. Limited to 256 characters.
Description
Optional. A longer space for notes about the variable's creation methods, coding, or supplementary information. Limited to 5000 characters.
Value labels                                                                                                  
Optional. A map of each value in the data to a longer string of information. For example, a variable with records 0 , 1 , and 2 might have value labels for 0 = No, 1 = Yes, 2 = Don't know. 

Value labels will be shown inline on the cells view of a table, alongside the frequency table values in the univariate statistics.
Univariate statistics
Redivis automatically computes certain statistics for all variables, depending on that variable's type and number of distinct values. To view summary statistics, click on a variable in a table.
Look at univariate statistics by selecting a variable
Statistic
Description
Count
The total number of values for this variable in the table. Does not include any null values.
Distinct
The number of unique values in the table. Does not include null as a value. (For example, a variable with values 0, 1, and null will have a distinct of 2.)
Non-null
The percentage of values in the table which are not null. Calculated by dividing the Count by the total number of records in the table.
Min, max                
For continuous variables (integer, float, date, dateTime, time): the minimum and maximum of this variable. 
​
For string variables: the minimum and maximum length of all values.
Min lng, Min lat, Max lng, Max lat
For geography variables: the bounding box containing all geospatial data in a variable
μ, σ
The mean and sample (unbiased) standard deviation of the variable. Only available for continuous variables (integer, float, date, dateTime, time).
Univariate plots
Histogram
A frequency chart of observations, sorted into 64 buckets, clamped to three standard deviations of the mean on either side. To switch between a linear and logarithmic y-axis, click on the bottom left corner of the chart.
Only shown for continuous variables with more than 64 distinct values.
Box plot
A visual display of the distribution of values by frequency. Shown are the minimum, 25%, median, 75% and maximum value. To include or exclude outliers in the calculations, click on the label on the bottom left corner of the chart.
Only shown for continuous variables with a meaningful number of discrete values.
The box plot uses the APPROX_QUANTILES BigQuery method, which will estimate quantiles (±0.57%, 95% CI) for larger datasets. As such, the box plot values should only be used as an approximation.
Frequency table
A table showing the frequency of common values for that variable, limited to 10,000 values. If a variable's values are highly heterogenous, no frequency table will be displayed. You can right click on values to get a quick entry point to filter on that value in the query tab. 
Map
A table with variable of type geography now shows a map view, which represents a boundary of all geospatial values contained in the table.
Editing metadata
To edit a variable's label, description, or value labels, click the Edit metadata button on the right of any table. This will allow you to any of the variables in this table, or you can navigate to the "All tables" tab to edit the same variable across multiple tables.
In the metadata editor, you can edit a variable's metadata fields as you would in any spreadsheet, and save changes when you're done.
Uploading metadata
Redivis determines variable names and types during data upload. Additionally, it will automatically parse certain metadata based on the uploaded file format:
SAS (.sas7bdat): labels
Stata (.dta): labels and value labels
SPSS (.sav): labels and value labels
For other file types (e.g., csv), you will need to augment the metadata directly. To apply metadata in bulk, you can upload a file containing metadata information directly from your computer. This file can either be a CSV or JSON.
Is your metadata stuck in a PDF? We're truly sorry — if you can, please let the data provider know that it is essential that they provide metadata in a machine-readable format; hopefully in time this will change.
While you can just upload the PDF to the dataset's documentation, you'll be doing your researchers a huge service if you can add structured metadata to your variables. That might mean some manual copy and paste, or you could consider the various (and imperfect) online PDF to CSV conversion tools, or this python library.
If you don't have the bandwidth, consider asking for your researchers to contribute and making them a dataset editor. 
CSV metadata format
The CSV should be formatted without no header, each row corresponding to a variable, with column 1 as the name, 2 as the label, 3 as the description. If the variable doesn't have a label or description, leave these columns empty.
variable1_name,variable1_label,variable1_description
variable2_name,variable2_label,variable2_description
For example:
sex,patient sex,patient's recorded sex
id,patient identifier,unique patient identifier
JSON metadata format
When uploading a JSON file, specify the name, label, description, and valueLabels using the appropriately named attributes in the object corresponding to each variable. If the variable doesn't have a label, description, or value labels, you don't need to include these attributes.
For example:
// JSON format is an array of objects, with each object representing a variable
[
    {
        "name": "sex",
        "label": "patient sex",
        "description": "patient's recorded sex",
        "valueLabels": [
            {
                "value": 1,
                "label": "Male"
            },
            {
                "value": 2,
                "label": "Female"
            }
        ]
    }
]
To upload value labels in bulk, you must use the JSON format. We no longer support bulk upload of value labels via CSV.
Exporting metadata
You can export your metadata at any time (in either of the above formats) by clicking Export file in the variable metadata editor. Variable metadata is also available in JSON format via the variables.list API endpoint.
Variable types
All variables in Redivis have a type associated with them. All types in Redivis support NULL values; that is, an empty cell.
String
A string can be used to store any textual data (UTF-8 encoded). Moreover, a string is a "universal" data type — any other type can be converted to a string without error.
Size: 2 bytes + UTF-8 encoded string size (1 byte for ASCII characters).
Integer
A 64-bit signed integer. Supports any integer value between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807 . 
Size: 8 bytes per cell.
Float
A 64-bit (double precision) floating point decimal value. 
Size: 8 bytes per cell.
Boolean
A representation of either TRUE or FALSE
Size: 1 byte per cell.
Date
Represents a calendar date independent of time zone. Supports any value between 0001-01-01 and 9999-12-31
Size: 8 bytes per cell
Format:
'YYYY-[M]M-[D]D'
YYYY: Four digit year
[M]M: One or two digit month
[D]D: One or two digit day
All values in brackets are optional. If your Date is not in this format, you can convert it using format strings.
DateTime
Represents a year, month, day, hour, minute, second, and subsecond, independent of timezone. Supports any value between 0001-01-01 00:00:00 and 9999-12-31 23:59:59.999999
Size: 8 bytes per cell
Format:
'YYYY-[M]M-[D]D[( |T)[H]H:[M]M:[S]S[.DDDDDD]]'
YYYY: Four digit year
[M]M: One or two digit month
[D]D: One or two digit day
( |T): A space or a Tseparator
[H]H: One or two digit hour (valid values from 00 to 23)
[M]M: One or two digit minutes (valid values from 00 to 59)
[S]S: One or two digit seconds (valid values from 00 to 59)
[.DDDDDD]: Up to six fractional digits (i.e. up to microsecond precision)
All values in brackets are optional. If your DateTime is not in this format, you can convert it using format strings.
Time
Represents a time, independent of a specific date. Supports values between 00:00:00 and 23:59:59.999999 
Size: 8 bytes per cell
Format:
'[H]H:[M]M:[S]S[.DDDDDD]'
[H]H: One or two digit hour (valid values from 00 to 23)
[M]M: One or two digit minutes (valid values from 00 to 59)
[S]S: One or two digit seconds (valid values from 00 to 59)
[.DDDDDD]: Up to six fractional digits (i.e. up to microsecond precision)
All values in brackets are optional. If your Time is not in this format, you can convert it using format strings.
Geography
Represents a Point, Multipoint, Linestring, MultiLinestring, Polygon, or MultiPolygon geometry, as specified by the GeoJSON standard.
You can create or manipulate geographic variables using various new variable methods.
Size: cell-content dependent
Format:
The WKT format is used to show cell values, though a feature can be viewed in a map modal (right-click on an individual cell)
'(POINT|MULTIPOINT|LINESTRING|MULTILINESTRING|POLYGON|MULTIPOLYGON)(...)'
​
-- Multiple of above geometries may be wrapped with
'(GEOMETRYCOLLECTION)(...)'
Conversion rules
Some types can be implicitly converted within a query (e.g., 1 (integer) < 2.2 (float)), in other circumstances, you will explicitly need to convert the type of a variable before performing certain operations (e.g., "1.0" (string) < 2.2 (float)).
Variable types can be converted within the transform interface in a project. The following conversions are supported:
Integer
Float
Returns a close but potentially not exact FLOAT64 value.
Integer
Boolean
Returns FALSE if x is 0, TRUE otherwise.
Float
Integer
Returns the closest INT64 value.
Halfway cases such as 1.5 or -0.5 round away from zero.
Float
String
Returns an approximate string representation.

Date
DateTime
Returns a DateTime at midnight on the corresponding date. For example, if x is 1970-01-01, returns 1970-01-01 00:00:00
DateTime
Date
Returns the part of the DateTime which is the calendar date. Note that this will not round the DateTime to the nearest Date. E.g., if x is 1970-01-01 23:59:59 , returns 1970-01-01 .
DateTime
Time
Returns the part of the DateTime which is the clock time. E.g., if x is 1970-01-01 23:59:59 , returns 23:59:59 .
Boolean
Integer
Returns 1 if x is TRUE, 0 otherwise.
Boolean
String
Returns "true" if x is TRUE, "false" otherwise.
String
Float
Returns x as a FLOAT64 value, interpreting it as having the same form as a valid FLOAT64 literal.
Also supports casts from "inf", "+inf", "-inf", and "nan".
Conversions are case-insensitive.
String
Boolean
Returns TRUE if x is "true" and FALSE if x is "false"
All other values of x are invalid and throw an error instead of casting to BOOL.
STRINGs are case-insensitive when converting to BOOL.
String
Geography
Use ST_GEOGFROM new variable method.
Geography
String
Use ST_ASTEXT new variable method.

Tables

Versions

Last modified 4mo ago

Statistic	Description
Count	The total number of values for this variable in the table. Does not include any null values.
Distinct	The number of unique values in the table. Does not include null as a value. (For example, a variable with values 0, 1, and null will have a distinct of 2.)
Non-null	The percentage of values in the table which are not null. Calculated by dividing the Count by the total number of records in the table.
Min, max	For continuous variables (integer, float, date, dateTime, time): the minimum and maximum of this variable. For string variables: the minimum and maximum length of all values.
Min lng, Min lat, Max lng, Max lat	For geography variables: the bounding box containing all geospatial data in a variable
μ, σ	The mean and sample (unbiased) standard deviation of the variable. Only available for continuous variables (integer, float, date, dateTime, time).

Field	Notes
Name	The name of the variable, limited to 60 characters. Must be unique within the table. All variable names are case-insensitive, and can only use alpha-numeric characters and underscores, and cannot start with a number.
Type	Required. The data type of the variable. Learn more about variable types.
Label	Optional. A short, human-readable description of the variable name. Limited to 256 characters.
Description	Optional. A longer space for notes about the variable's creation methods, coding, or supplementary information. Limited to 5000 characters.
Value labels	Optional. A map of each value in the data to a longer string of information. For example, a variable with records `0` , `1` , and `2` might have value labels for `0 = No`, `1 = Yes`, `2 = Don't know`. Value labels will be shown inline on the cells view of a table, alongside the frequency table values in the univariate statistics.

Integer	Float	Returns a close but potentially not exact FLOAT64 value.
Integer	Boolean	Returns `FALSE` if `x` is `0`, `TRUE` otherwise.
Float	Integer	Returns the closest INT64 value. Halfway cases such as 1.5 or -0.5 round away from zero.
Float	String	Returns an approximate string representation.
Date	DateTime	Returns a DateTime at midnight on the corresponding date. For example, if `x` is `1970-01-01`, returns `1970-01-01 00:00:00`
DateTime	Date	Returns the part of the DateTime which is the calendar date. Note that this will not round the DateTime to the nearest Date. E.g., if `x` is `1970-01-01 23:59:59` , returns `1970-01-01` .
DateTime	Time	Returns the part of the DateTime which is the clock time. E.g., if `x` is `1970-01-01 23:59:59` , returns `23:59:59` .
Boolean	Integer	Returns `1` if `x` is `TRUE`, `0` otherwise.
Boolean	String	Returns `"true"` if `x` is `TRUE`, `"false"` otherwise.
String	Float	Returns `x` as a FLOAT64 value, interpreting it as having the same form as a valid FLOAT64 literal. Also supports casts from `"inf"`, `"+inf"`, `"-inf"`, and `"nan"`. Conversions are case-insensitive.
String	Boolean	Returns `TRUE` if `x` is `"true"` and `FALSE` if `x` is `"false"` All other values of `x` are invalid and throw an error instead of casting to BOOL. STRINGs are case-insensitive when converting to BOOL.
String	Geography	Use `ST_GEOGFROM` new variable method.
Geography	String	Use `ST_ASTEXT` new variable method.