Table$to_data_table

Table$to_data_table(max_results=NULL, variables=NULL, batch_preprocessor=NULL, max_parallelization=parallely::availableCores()) → data.table

Returns a data.table representing a table.

Parameters:

max_results : int, default NULL The max number of records to load into the datatable. If not specified, the entire table will be loaded.

variables : list(str) | character vector The specific variables to return, e.g., variables = c("name", "date") . If not specified, all variables in the table will be returned.

batch_preprocessor : function, default NULL Function used to preprocess the data, invoked for each batch of records as they are initially loaded. This can be helpful in reducing the size of the data before the final table is loaded. The function accepts one argument, an Arrow RecordBatch, and must return a Arrow RecordBatch or NULL. If you prefer to work with the data solely in a streaming manner, see Table$to_arrow_batch_reader()

max_parallelization : int, default parallely::availableCores() The maximum parallelization when loading the table. Uses the future::multicore strategy when supported, falling back to future::multisession if not.

Returns:

data.table