tabsdata.GCSDestination

class GCSDestination(

uri: str | list[str],

credentials: GCPCredentials,

format: str | FileFormat = None,

)

Bases: DestinationPlugin

Class for managing the configuration of GCS-file-based data outputs.

format

The format of the file to be created. If not provided, it will be inferred from the file extension.

Type:

FileFormat

uri

The URI of the files with format: ‘gs://path/to/files’. It can be a single URI or a list of URIs.

Type:

str | list[str]

credentials

The credentials required to access GCS.

Type:

GCPCredentials

__init__(

uri: str | list[str],

credentials: GCPCredentials,

format: str | FileFormat = None,

)

Initializes the GCSDestination with the given URI and the credentials

required to access GCS; and optionally a format.

Parameters:

• uri (str | list[str]) – The URI of the files to export with format: ‘gs://path/to/files’. It can be a single URI or a list of URIs.

• credentials (GCPCredentials) – The credentials required to access GCS. Must be a GCPCredentials object.

• format (str | FileFormat, optional) – The format of the file. If not provided, it will be inferred from the file extension. Can be either a string with the format or a FileFormat object. Currently supported formats are ‘csv’, ‘parquet’, ‘ndjson’ and ‘jsonl’.

Raises:

• OutputConfigurationError –

• FormatConfigurationError –

Methods

__init__(uri, credentials[, format])	Initializes the GCSDestination with the given URI and the credentials
chunk(working_dir, *results)	Trigger the exporting of the data to local parquet chunks. This method will
stream(working_dir, *results)	Trigger the exporting of the data. This method will receive the resulting data
write(files)	Given a file or a list of files, write to the desired destination. Note: this

Attributes

allow_fragments	Whether to allow fragments in the output.
credentials	The credentials required to access GCS.
format	The format of the file.
uri	'gs://path/to/files'.