Query.to_arrow_batch_iterator

Query.to_arrow_batch_iterator(max_results=None, *, progress=True) → pyarrow.RecordBatch iterator

Returns an iterator that can be used to consume query results in chunks of PyArrow RecordBatches. Allows for streaming workflows where only a small portion of the query results are read into memory at a time.

Parameters:

max_results : int, default None The maximum number of rows to return. If not specified, all rows in the query results will be read.

progress : bool, default True Whether to show a progress bar.

Yields:

pyarrow.RecordBatch

See also

• Query.to_arrow_dataset()

• Query.to_arrow_table()

• Query.to_geopandas_dataframe()

• Query.to_dask_dataframe()

• Query.to_pandas_dataframe()

• Query.to_polars_lazyframe()