Install Tabsdata on AWS EKS
Tabsdata runs on Kubernetes as a set of long-running deployment pods, the API server, the AI agent and their supporting services, all installed into a single namespace. User functions are not part of those deployments: every publisher, subscriber and transformer execution is dispatched as its own Kubernetes job that starts, runs the function and exits, so function capacity scales with the cluster rather than with the instance. Neither the pods nor the jobs hold state. Table data lives in AWS S3 and all metadata lives in AWS RDS for PostgreSQL.
Every step in this document is performed by the Tabsdata admin.
Requirements
- A Windows (x86), macOS (ARM64) or Linux (x86) client machine.
- Python 3.12.13 on the machine you install from.
- An AWS EKS Kubernetes cluster with x86 nodes.
- An AWS RDS for PostgreSQL instance running PostgreSQL 18.4, in the same region as the EKS cluster.
- An AWS S3 bucket, preferably in the same region as the EKS cluster.
- Network access: the EKS cluster must reach the RDS PostgreSQL instance, and the client machine must reach the EKS cluster.
Install the Tabsdata packages
You need to do these steps from a Windows, macOS or Linux machine that can access the EKS cluster.
Create a clean Python virtual environment, point pip and uv at the Tabsdata index, then install Tabsdata in it. Keep these variables set for the rest of the install: the instance resolves the same Tabsdata version for itself when it is created.
export PIP_EXTRA_INDEX_URL=https://pypi.tabsdata.com/simple/
export UV_INDEX=https://pypi.tabsdata.com/simple/
export UV_INDEX_STRATEGY=unsafe-best-match
uv pip install "tabsdata[all]"==2.0.0 tabsdata-agent==2.0.0
With pip instead of uv:
pip install "tabsdata[all]"==2.0.0 tabsdata-agent==2.0.0
Tabsdata 2.0.0 is not on PyPI yet, held up by an operational limit on uploads,
so it installs from pypi.tabsdata.com instead. The version is required too:
without it the resolver picks an older release from PyPI. tabsdata-agent ships
as its own package and the AI features need it.
Drop the variables and the pins once the packages reach PyPI.
If you do not have an EKS cluster and namespace ready, or you do not yet have a
<KUBECONFIG> that reaches them, AWS Services Setup for Tabsdata on
EKS explains how they must be configured.
The MySQL and PostgreSQL CDC connectors need database drivers that Tabsdata cannot redistribute. If you use either one, see Installing Third-Party Database Drivers before you continue, since the requirements file is read during the setup below.
Configuration you need at hand
Collect these values before you start. The install asks for all of them, and none of them can be filled in later without editing the instance.
| Value | Description |
|---|---|
| AWS RDS for PostgreSQL | |
<RDS_HOST> | Hostname of the RDS instance |
<RDS_PORT> | Port of the RDS instance |
<TABSDATA_DATABASE> | Database Tabsdata installs into |
<RDS_USERNAME> | User Tabsdata connects as during setup |
<RDS_PASSWORD> | Password for that user |
| AWS S3 | |
<S3_BUCKET> | Bucket used for data storage |
<S3_REGION> | Region of the bucket |
<S3_ACCESS_KEY> | Access key with full access to the bucket |
<S3_SECRET_KEY> | Secret key for that access key |
| AWS EKS | |
<TABSDATA_CLUSTER> | Cluster Tabsdata is installed into |
<TABSDATA_NAMESPACE> | Empty namespace reserved for this instance |
<KUBECONFIG> | Path to a kubeconfig file that reaches the cluster |
| LLM provider | |
<OPENAI_API_KEY> / <ANTHROPIC_API_KEY> | LLM API key, if the AI integration is enabled |
Chosen during the install
| Value | Description |
|---|---|
<TABSDATA_INSTANCE> | Name of the Tabsdata instance |
<TABSDATA_APISERVER_PASSWORD> | Password you pick for the API server database role |
<TABSDATA_AIAGENT_PASSWORD> | Password you pick for the AI agent database role |
Configure Tabsdata
Prepare Tabsdata configuration
tdkserver init --instance <TABSDATA_INSTANCE> --provider eks --kubeconfig <KUBECONFIG> --namespace <TABSDATA_NAMESPACE>
This will create a folder ~/.tabsdata/instances/<TABSDATA_INSTANCE>/ with files
that must be configured (details follow) to complete the Tabsdata installation.
The Tabsdata configuration files to edit are in the
~/.tabsdata/instances/<TABSDATA_INSTANCE>/init/input/configs/ folder.
Configure the Tabsdata database
The database configuration is split across two files, dbmanager.database.yaml
and apiserver.database.yaml. The first one is used during setup only. The
second one is used for regular operations.
The database credentials are extracted and stored encrypted in the EKS Kubernetes secret store.
The RDS information must be set in the dbmanager.database.yaml file.
Configuration file dbmanager.database.yaml:
database_server:
postgres:
host: <RDS_HOST>
port: <RDS_PORT>
database: <TABSDATA_DATABASE>
username: <RDS_USERNAME>
password: <RDS_PASSWORD>
ssl_mode: ${{system:tdk_database_ssl_mode}}
ssl_root_cert: ${{system:tdk_database_ssl_root_cert}}
pool:
min_connections: 1
max_connections: 2
acquire_timeout: 30
max_lifetime: 1800
idle_timeout: 30
test_before_acquire: true
The apiserver.database.yaml file requires two passwords, which you must
provide. The Tabsdata installation creates the <TABSDATA_DATABASE>_apiserver
and <TABSDATA_DATABASE>_aiagent roles and sets these passwords on them.
Configuration file apiserver.database.yaml:
database:
apiserver:
postgres:
password: <TABSDATA_APISERVER_PASSWORD>
pool:
min_connections: 5
max_connections: 50
acquire_timeout: 30
max_lifetime: 3600
idle_timeout: 60
test_before_acquire: true
aiagent:
postgres:
password: <TABSDATA_AIAGENT_PASSWORD>
pool:
min_connections: 1
max_connections: 10
acquire_timeout: 30
max_lifetime: 3600
idle_timeout: 60
test_before_acquire: true
Configure Tabsdata S3 storage
Tabsdata on EKS uses AWS S3 as its storage layer. It is recommended that the S3 bucket(s) be in the same region as the EKS cluster.
Three mounts must be configured: ROOT, BUNDLES and IMAGES. They can all use
the same bucket, as long as the paths differ. Each entry must be configured with
its corresponding region, access key and secret key.
The credentials are extracted and stored encrypted in the EKS Kubernetes secret store.
Configuration file apiserver.storage.yaml:
The S3 URI must end with /.
storage:
mounts:
- id: ROOT
path: /
uri: s3://<S3_BUCKET>/root/
options:
aws_region: <S3_REGION>
aws_access_key_id: <S3_ACCESS_KEY>
aws_secret_access_key: <S3_SECRET_KEY>
- id: BUNDLES
path: /bundles
uri: s3://<S3_BUCKET>/bundles/
options:
aws_region: <S3_REGION>
aws_access_key_id: <S3_ACCESS_KEY>
aws_secret_access_key: <S3_SECRET_KEY>
- id: IMAGES
path: /images
uri: s3://<S3_BUCKET>/images/
options:
aws_region: <S3_REGION>
aws_access_key_id: <S3_ACCESS_KEY>
aws_secret_access_key: <S3_SECRET_KEY>
Configure the AI agent
To enable the Tabsdata AI integration, an LLM API key must be provided. Tabsdata currently supports Anthropic and OpenAI.
The API keys are extracted and stored encrypted in the EKS Kubernetes secret store.
Configuration file aiagent.yaml:
port: ${{system:tdk_aiagent_public_port}}
allow_writes: true
ai_providers:
openai:
api_key: <OPENAI_API_KEY>
model: openai/gpt-5.4
anthropic:
api_key: <ANTHROPIC_API_KEY>
model: anthropic/claude-sonnet-4-6
hybrid_search:
embedder: fastembed
embedding_model: BAAI/bge-small-en-v1.5
device: cpu
cache_dir: null
offline: false
Configure the pod environment
Every pod resolves Python packages of its own, so the pods need the Tabsdata index as well as the machine you install from.
Configuration file tabsdata.yaml:
pods:
global:
# Every entry here is injected as an environment variable into every
# container of every pod. A variable a container declares itself is left
# alone, so these are defaults rather than overrides.
envs:
UV_INDEX: https://pypi.tabsdata.com/simple/
UV_DEFAULT_INDEX: https://pypi.org/simple/
UV_INDEX_STRATEGY: unsafe-best-match
Tabsdata 2.0.0 is not on PyPI yet due to an operational upload limit.
UV_DEFAULT_INDEX keeps PyPI as the default source for other dependencies, while
unsafe-best-match allows uv to resolve packages across both PyPI and the
Tabsdata index. Remove this section once Tabsdata 2.0.0 is available on PyPI.
Preflight check
Check that nothing is still missing before creating the instance.
tdkserver check-cfg --instance <TABSDATA_INSTANCE>
Create the Tabsdata instance
tdkserver create --instance <TABSDATA_INSTANCE>
Verify the Tabsdata instance state.
tdkserver inspect --instance <TABSDATA_INSTANCE>
Start the Tabsdata instance
tdkserver start --instance <TABSDATA_INSTANCE>
tdkserver status --instance <TABSDATA_INSTANCE>
Access Tabsdata
Use port forwarding to verify Tabsdata is running.
kubectl port-forward --kubeconfig <KUBECONFIG> --namespace <TABSDATA_NAMESPACE> service/net-apiserver 2457:2457
Leave that command running, and use http://127.0.0.1:2457 as <TABSDATA_URL>
in the next step. Port 2457 is the default API server port; if the instance was
given a different base port, use that one instead.
Port forwarding is meant for verifying the installation. Expose Tabsdata properly before using it.
Log in to the Tabsdata instance
tdk login --server <TABSDATA_URL> --user admin --password tabsdata
tdk status
Post-installation cleanup
The configuration is now stored in the instance, so you can delete the local setup files. They contain the passwords and access keys you entered.
rm -rf ~/.tabsdata/instances/<TABSDATA_INSTANCE>/init
Expose Tabsdata outside the cluster
The instance is now running, but only reachable through the port forward above. To reach the API server on an address outside the cluster, AWS Services Setup for Tabsdata on EKS explains how it must be exposed over HTTPS.
Next steps
Run your first data integration: Run Your First Data Integration.
Appendix
Sizing the service and function pods
All sizing lives in the instance's configs/tabsdata.yaml. Pod descriptors
carry none of their own, so what is written there is what the cluster receives.
service covers the pods that run alongside the instance; its own resources
apply to every service pod, and the named entries (apiserver, aiagent,
aiembeddings, aimetadata) override them for one component each, so a pod
with no entry of its own, a manager, inspector or loader, uses the service
values directly. function covers the pods created per execution and applies to
all of them uniformly. Service requests are a fixed per-instance cost: sum them
and multiply by the number of instances the cluster will hold. Function requests
are a per-concurrent-run cost. Disk is usually the binding constraint on both
sides, since aiagent alone asks for 64Gi, as does every function run, which is
why both node pools select instance types with local NVMe.
pods:
service:
priority: tabsdata-service
resources: # dbmanager / pymanager / inspector / venvloader
cpu: "2"
memory: 1Gi
disk: 16Gi
placement:
selectors: {}
tolerations: []
aiagent: # overrides the above, for this component only
resources:
cpu: "2"
memory: 8Gi
disk: 64Gi
function:
priority: tabsdata-function
resources:
cpu: "1"
memory: 16Gi
disk: 64Gi
placement:
selectors: {}
tolerations:
- key: workload # must match the functions pool taint
value: batch
effect: NoSchedule
Values are examples. Set them from the actual footprint of the components in
use, then confirm each pool's node class can still satisfy the largest single
pod. Retrieve and upload the file with tdkserver get-cfg / tdkserver put-cfg.