DruidDest

class

class DruidDest(
    tables: DruidTableNamesSpec,
    if_table_exists: IfTableExistsSpec = 'append',
    time_column: str | None = None,
    segment_granularity: SegmentGranularitySpec = 'ALL',
    max_concurrent_subtasks: int = 1,
    max_rows_per_segment: int = 5000000,
    max_rows_in_memory: int = 1000000,
    http_timeout_sec: int = 300,
    dest_cfg: DestCfgSpec | None = None,
)

Bases: Dest

Categories: destination

Druid destination for @subscriber -- one slot per datasource.

Examples

Overwrite the datasource on every run instead of appending:

@subscriber(
    destination=DruidDest(tables=["events"], if_table_exists="replace"),
    input_tables=["mycoll/events"],
)
def publish(events: TableFrameSpec) -> TableFrameSpec:
    return events

Parameters

parameter

tablesDruidTableNamesSpec (list[str])

Datasource names, one write slot per entry (each 1-128 chars, not starting with . or _, and containing no /, backslash or NUL).

parameter

if_table_existsIfTableExistsSpec (Literal['append', 'replace'])

"append" (default) or "replace" (overwrite the datasource).

parameter

time_columnstr | None

Name of the row-level timestamp column Druid uses as its primary __time field at ingest. Druid requires a time column on every datasource; if the rows carry none, leave unset and the ingestion spec falls back to the current ingest time.

parameter

segment_granularitySegmentGranularitySpec (Literal['ALL', 'NONE', 'SECOND', 'MINUTE', 'FIFTEEN_MINUTE', 'THIRTY_MINUTE', 'HOUR', 'SIX_HOUR', 'DAY', 'WEEK', 'MONTH', 'QUARTER', 'YEAR'])

Druid segment grain -- one of ALL (default), NONE, SECOND, MINUTE, FIFTEEN_MINUTE, THIRTY_MINUTE, HOUR, SIX_HOUR, DAY, WEEK, MONTH, QUARTER or YEAR.

parameter

max_concurrent_subtasksint

Druid ingestion-task concurrency (default 1).

parameter

max_rows_per_segmentint

Per-segment row cap (default 5,000,000).

parameter

max_rows_in_memoryint

Per-task in-memory row cap (default 1,000,000).

parameter

http_timeout_secint

HTTP request timeout in seconds (default 300).

parameter

dest_cfgDestCfgSpec | None (Mapping[Literal['druid.logging_level', 'druid.poll_interval_seconds', 'druid.retry_attempts', 'druid.retry_backoff_seconds'], Any] | None)

Optional connector config over the keys druid.logging_level, druid.poll_interval_seconds, druid.retry_attempts and druid.retry_backoff_seconds.