Using synthetic data

When working with high-risk or restricted data, you can develop and test a workflow using a synthetic dataset before running the workflow on the non-synthetic dataset. In this example, we'll use a synthetic version of the CMS Patient Data to develop an analysis, then switch the data source to the corresponding non-synthetic dataset.

[Synthetic Version] CMS Sample Patient Data OMOPRedivis

CMS Sample Patient Data OMOPRedivis

1. Create a Workflow

Create a new workflow and add the CMS Synthetic Patient Dataset as a data source.

The synthetic dataset provides the tables and variables needed to develop the analysis without requiring access to the underlying patient data.

2. Build and test your analysis pipeline

Use the synthetic dataset to build your workflow as you normally would. For example, you might use the patient and claims data to identify patients who meet a particular set of criteria and summarize their claims by year.

Add the necessary transforms and notebooks to your workflow, then run the workflow to verify that your analysis produces the expected results.

At this stage, you can make changes and iterate on your analysis without working with the restricted dataset.

3. Replace with the non-synthetic dataset

Once the workflow is ready, replace the synthetic dataset with the corresponding CMS patient dataset. The dataset can be replaced by you or another user with edit access to the workflow and data access to the dataset

To replace the dataset, navigate to the dataset node, open the More menu in the resource bar, and select Replace dataset. Search for the non-synthetic dataset and select it as the new data source.

Then align each table used in your workflow to the corresponding tables in the non-synthetic dataset

Because the variables in the patient data have a compatible structure, you can preserve the transforms and notebooks you developed using the synthetic data.

If you do not have data access to the non-synthtic dataset, you will not be able to view its data or tables that you do not have access to. You can apply for access, if appropriate, or have an administrator with access replace the dataset and run it on your behalf.

4. Verify your workflow

Review the workflow to make sure that the expected tables and variables are available in the non-synthetic dataset and that the existing transformations still apply.

Make any necessary updates before running the workflow.

5. Run the workflow

Once you've verified the data source and made any necessary updates, some nodes will be stale. Rerun the workflow to apply the changes and materialize the updated results. To update all nodes at once, select Run all stale nodes from the workflow's Map menu.

The workflow now runs using the CMS patient data rather than the synthetic data used during development.