Workers

A worker is the process that executes the code defined in a Tabsdata Function

Each worker runs as an isolated process on the Tabsdata server's Kubernetes cluster. Performance optimizations such as Polars Streaming and Lazy Evaluation allow workers to process datasets incrementally, allowing them to handle datasets larger than the memory available on the machine's hardware.

Retries

Tabsdata deploys one worker for each execution attempt of the function code. If a worker fails, Tabsdata snapshots the worker's execution log and spins up a new worker to retry the execution. By default, a function run will attempt three executions before stalling.

A function is triggered

Function runGenerated by the trigger

Worker 1Failed

Worker 2Failed

Worker 3Succeeded

Three workers is the default limit. Had the third one failed as well, the function run would stall instead of starting a fourth.

Logs

A worker is where a function's output ends up, so its logs are the first place to look when a function run fails:

$ tdk worker logs --worker WID

Pass --file - to write the log to stdout.

Recovery

You can retry a single worker without re-running the whole plan:

$ tdk worker recover --worker WID

Finding a worker

tdk worker list filters down from any level of the plan, so you can start from whichever identifier you already have:

$ tdk worker list [--status R] [--coll C] [--fn F] [--plan P] [--trx T] [--fnrun FR]

See the CLI Guide for the full set of commands and the status codes each accepts.