tabsdata.BigQueryDest

class BigQueryDest(

conn: BigQueryConn,

tables: str | list[str] | None = None,

if_table_exists: Literal['append', 'replace'] = 'append',

schema_strategy: Literal['update', 'strict'] = 'update',

)

Bases: DestinationPlugin

Destination plugin to store data in BigQuery tables. The data is first stored in

parquet files in a GCS bucket, and then loaded into the BigQuery tables.

__init__(

conn: BigQueryConn,

tables: str | list[str] | None = None,

if_table_exists: Literal['append', 'replace'] = 'append',

schema_strategy: Literal['update', 'strict'] = 'update',

)

Initializes the BigQueryDest with the configuration desired to store

the data.

Parameters:

• conn (BigQueryConn) – The BigQuery connection configuration.

• tables (str | list[str] | None, optional) – The table(s) to store the data in. If multiple tables are provided, they must be provided as a list. If None, the table names will be those of the input tables for the function. Defaults to None.

• if_table_exists ({'append', 'replace'}, optional) – The strategy to follow when the table already exists. - ‘append’ will append to an existing table. - ‘replace’ will create a new table, overwriting an existing one. Defaults to ‘append’.

• schema_strategy ({'update', 'strict'}, optional) –

  The strategy to follow for the schema when the table already exists. - ‘update’ will update the schema with the possible new columns that

  might exist in the TableFrame.

  • ‘strict’ will not modify the schema, and will fail if there is any

    difference. Defaults to ‘update’.

Methods

__init__(conn[, tables, if_table_exists, ...])	Initializes the BigQueryDest with the configuration desired to store
chunk(working_dir, *results)	Trigger the exporting of the data to local parquet chunks. This method will
stream(working_dir, *results)	Trigger the exporting of the data. This method will receive the resulting data
write(files)	Given a file or a list of files, write to the desired destination. Note: this

Attributes

conn
if_table_exists	The strategy to follow when the table already exists.
schema_strategy	The strategy to follow when appending to a table with an existing schema.
tables