PostgresDest
class PostgresDest(
conn: PostgresConn,
destination_tables: str | list[str],
if_table_exists: IfTableExistStrategySpec = 'append',
transactional: bool = True,
chunk_size: PositiveInt = 100000,
loader: PostgresDestWriterSpec = 'polars_sqlalchemy',
)
Bases: DestinationPlugin
Categories: destination
Destination plugin for writing data to PostgreSQL.
Initialize a PostgresDest instance for writing data to PostgreSQL.
Parameters
connThe Postgres connection configuration.
destination_tablesThe tables to create. If multiple tables are provided, they must be provided as a list.
if_table_existsThe strategy to follow when the table already exists. Defaults to 'append'. - ‘replace’ will create a new database table, overwriting an existing one. - ‘append’ will append to an existing table.
transactionalWhether to use transactions for writing data to ensure consistency. Defaults to True
chunk_sizeThe chunk size for writing large results in batches. Defaults to 100000.
loaderThe data processing loader to use for executing DB write operation.
Properties
chunk_sizeintThe chunk size for writing large results in batches.
connPostgresConnThe Postgres connection configuration.
destination_tableslist[str]The table(s) to create. If multiple tables are provided, they must be provided as a list.
if_table_existsIfTableExistStrategySpecThe strategy to follow when the table already exists.
loaderPostgresDestWriterSpecThe data processing loader to use for executing DB write operation.
transactionalboolWhether to use transactions for writing data to ensure consistency.
Methods
streamdef stream(working_dir: str, *results: list[LazyFrame | None])
Store the results into the PostgreSQL database.
Parameters:
working_dirThe working directory where the results are stored.
resultsThe results to store in the SQL destination.