Parameters

Overview

Parameters allow you to centrally manage a set of values for use throughout a workflow. An update to a parameter will be immediately propagate to any transform or notebook where that parameter is referenced.

This may be useful in many situations, including where:

• Sets of inclusion and exclusion criteria are used to filter data across transforms, and these criteria may evolve as you build your analysis.

• You have a single value used in multiple places that you want to be able to quickly change everywhere at once.

Creating parameters

To create and update parameters, you can click the Parameters button at the top left of workflow tool top bar.

You can create, modify, search for, filter on, and delete parameters from this interface.

Parameters can have up to 10,000 values. Each individual value can be up to 255 characters. Any spaces included within a value are preserved as entered (e.g., Apple will retain the trailing space and would not be trimmed to Apple).

Parameter values should never be quoted, unless you specifically want to match against a quoted character.

Importing and exporting parameters

Values can be added to a parameter from copy+paste from any spreadsheet program. For bulk or scripted actions, you can also import multiple parameters from a JSON or CSV file. Parameters can also be exported in JSON and CSV formats.

CSV specification

Each column in your CSV will represent one parameter, with the first row containing the name of the parameter. E.g.:

Fruits	Vegetables	Item_codes
Strawberry	Broccoli	1
Tomato	Asparagus	2
Apple	Spinach	3

JSON specification

The same set of parameters can be imported via JSON, with the following format:

[
    { 
        "name": "Fruits",
        "values": ["Strawberry", "Tomato", "Apple"]
    },
    { 
        "name": "Vegetables",
        "values": ["Broccoli", "Asparagus", "Spinach"]
    },
    { 
        "name": "Item_codes",
        "values": ["1", "2", "3"]
        // This is identical:
        // "values": [1, 2, 3]
    },
]

Exporting

You can export all parameters at once as either .csv or .json files.

Parameter types

All variables in Redivis have a defined type. Redivis will automatically determine the most specific type for all the values in a parameter, falling back to the "universal type" of string.

You can also explicitly set the type of a parameter, for example if you have a set of numeric values that you want to be treated as strings.

Values	Type	Can compare to
1, 2, 3	Integer	Integer, Float, String
1, 1.0, 1.1	Float	Float, String
1, 2.0, a	String	String
1971-01-01, 1972-02-02, 1973-03-03	Date	Date, String
1971-01-01, a, b	String	String
1970-01-01 12:00:00, 1970-01-01 12:30:00	DateTime	DateTime, Date, String
14:35:22, 03:16:29	Time	Time, String

Using parameters

Parameters can be referenced in transforms and notebooks in the same workflow as the parameter.

When using parameters, the parameter type needs to be the same (or coercible to) the type required of for the input.

If you update the values in a parameter, all transforms and notebooks that reference the parameter will be marked as stale on the workflow tree.

Transforms

You can reference the parameter in transform interface anywhere you can input multiple values. Where you would input your value(s) you can scroll to the bottom of the menu to find your parameters.

You can also reference parameters in SQL steps and interactive SQL queries. E.g.;

SELECT * FROM my_table
WHERE text IN UNNEST(@my_param)

Notebooks

You can also create, read, and update parameters from within notebooks. See below for a simple example, and consult the relevant API documentation for full details.

• Learn more about programmatically referencing resources

• Python API docs

• R API docs

Python

import redivis

workflow = redivis.workflow("username.workflow_name")
parameter = workflow.parameter("my_param")
print(parameter.get_values()) # get the list of values stored in the parameter

# We can update parameter values programmatically
parameter.update(values=["hello","world"])
print(parameter.get_values()) # ["hello", "world"]

# We can can also reference parameters in SQL
df = redivis.query("""
    SELECT * FROM my_table
    WHERE text IN UNNEST(@my_param)
""").to_pandas_dataframe()

R

Parameter versioning

Each time a transform or notebook is run referencing a parameter, a snapshot of the parameter is created to reflect its values at that moment. You can view any previous versions of a parameter by opening its history and selecting the associated timestamp.

When viewing a historic version, you can compare values and potentially revert to the state of the parameter at that point in time.