tabsdata.BigQueryDest

class BigQueryDest(

conn: BigQueryConn,

tables: str | list[str] | None = None,

if_table_exists: Literal['append', 'replace'] = 'append',

schema_strategy: Literal['update', 'strict'] = 'update',

)

Bases: DestinationPlugin

BigQuery based data outputs. The data is first stored in parquet files in a GCS bucket, and then loaded into the BigQuery tables.

__init__(

conn: BigQueryConn,

tables: str | list[str] | None = None,

if_table_exists: Literal['append', 'replace'] = 'append',

schema_strategy: Literal['update', 'strict'] = 'update',

)

Initializes the BigQueryDest with the configuration desired to store the data.

Parameters:

• conn – The BigQuery connection configuration.

• tables – The table(s) to store the data in. If multiple tables are provided, they must be provided as a list. If None, the table names will be those of the input tables for the function. Defaults to None.

• if_table_exists –

  The strategy to follow when the table already exists.

  • ‘append’ will append to an existing table.

  • ‘replace’ will create a new table, overwriting an existing one. Defaults to ‘append’.

• schema_strategy –

  The strategy to follow for the schema when the table already exists.

  • ‘update’ will update the schema with the possible new columns that might exist in the TableFrame.

  • ‘strict’ will not modify the schema, and will fail if there is any difference. Defaults to ‘update’.

Methods

__init__(conn[, tables, if_table_exists, ...])	Initializes the BigQueryDest with the configuration desired to store the data.
chunk(working_dir, *results)	Trigger the exporting of the data to local parquet chunks.
stream(working_dir, *results)	Trigger the exporting of the data.
write(files)	Given a file or a list of files, write to the desired destination.

Attributes

conn
if_table_exists	The strategy to follow when the table already exists.
schema_strategy	The strategy to follow when appending to a table with an existing schema.
tables