vastdb.util

vastdb.util.create_table_from_files(schema: Schema, table_name: str, parquet_files: List[str], schema_merge_func: Callable | None = None, config: ImportConfig | None = None) → Table

vastdb.util.default_schema_merge(current_schema: pyarrow.Schema, new_schema: pyarrow.Schema) → pyarrow.Schema

This function validates a schema is contained in another schema Raises an InvalidArgument if a certain field does not exist in the target schema

vastdb.util.expand_ip_ranges(endpoints)

Expands endpoint strings that include an IP range in the format ‘http://172.19.101.1-16’.

vastdb.util.is_sorted(arr)

Check if the array is sorted.

vastdb.util.iter_serialized_slices(batch: pyarrow.RecordBatch | pyarrow.Table, max_rows_per_slice=None)

Iterate over a list of record batch slices.

vastdb.util.serialize_record_batch(batch: pyarrow.RecordBatch | pyarrow.Table)

Serialize a RecordBatch using Arrow IPC format.

vastdb.util.sort_record_batch_if_needed(record_batch, sort_column)

Sort the RecordBatch by the specified column if it is not already sorted.

vastdb.util.strict_schema_merge(current_schema: pyarrow.Schema, new_schema: pyarrow.Schema) → pyarrow.Schema

This function validates two Schemas are identical. Raises an InvalidArgument if schemas aren’t identical.

vastdb.util.union_schema_merge(current_schema: pyarrow.Schema, new_schema: pyarrow.Schema) → pyarrow.Schema

This function returns a unified schema from potentially two different schemas.