Operating Compose: installation, maintenance and backup

Installation and updates through Ansible#

The playbook under deploy/ansible prepares a Debian/Ubuntu VM with SSH access: install Docker, fetch a pinned release, generate missing secrets, build images sequentially and start Compose. It waits for a healthy agent and an existing backup. Enter the target host and SSH user in your inventory and set lavalake_version to the desired release tag.

cd deploy/ansible
cp inventar.beispiel.ini inventar.ini
# Set target host, SSH user and lavalake_version in inventar.ini
ansible-playbook -i inventar.ini install.yml

For an update, change the release tag and run the same playbook. Existing keys are preserved; database migrations run at startup. Images are built one at a time to limit peak memory use on smaller machines.

Schedules and Iceberg maintenance#

Regeltermine offers reports, threshold alerts and maintenance. Choose a schedule, inspect run history or trigger a run manually. Background runs use the stored offline authorization and permissions of their owner.

The “Wartung” type compacts small files, expires old snapshots, removes orphan files and finally refreshes table statistics (ANALYZE) — without them Trino's cost-based optimizer plans join order and distribution blind. Retention ranges from 7 to 365 days, with a default of 7. No automatic cleanup runs until a maintenance schedule has been configured. A table error is recorded in the result while processing continues for the remaining tables.

What the daily backup contains#

A dedicated backup service runs at startup and then daily. Backups are retained for 14 days by default; retention cleanup always preserves the latest backup. Betrieb → Überwachung shows its age and size and warns when it is missing or overdue.

Prepare for recovery#

The procedure is documented in docs/betrieb/WIEDERHERSTELLUNG.md in the product repository. Recovery requires compatible database backups, configuration and object storage data. A database backup alone does not restore the lakehouse data. Automatic site replication of the object store is not yet part of the delivered operational setup. Rehearse recovery on a separate test installation.

Query workload classes#

In Compose, roles assign queries to a Trino resource group. Limits apply to the combined queries of a class, not as an individual allowance per user. Memory percentages are soft limits for admitting further queries, not reserved memory shares.

Assign the role under Verwaltung → Nutzer & Rollen. If multiple workload roles apply, L takes precedence over M, then S; platform administrators use their own class. Selection takes effect through the role-to-group file. This does not provision additional machines, auto-suspend or automatically growing clusters.

Usage and limits#

Monitoring shows compute time and data read by user and purpose, together with daily quotas. AI usage, model prices and daily token budgets appear in “KI-Kosten”. The AI backend guide covers providers, price synchronization and budget limits in detail.

Public access over HTTPS#

Since v0.40.0 a single variable (PUBLIC_BASE_URL) switches the whole installation to a public hostname: the token issuer, issuer validation in the agent and the query engine, the sign-in address baked into the console and the permitted login redirects all move in lockstep. A TLS overlay for the gateway with HSTS and frame protection comes with it; local access over 127.0.0.1 remains available so SSH tunnels and operational scripts keep working. See docs/betrieb/OEFFENTLICHER-ZUGANG.md in the product repository.

Scaling out with more compute nodes#

Since v0.42.0 the query engine can be extended with workers — locally through a Compose profile for trying it out, and in production on additional machines through a separate Compose file. Each worker brings its own cores and memory and registers with the coordinator within seconds; workload classes continue to apply unchanged. Details in docs/betrieb/WORKER.md. Since v0.195.0 the console shows the connected compute nodes under Administration → Operations → System health and reports a missing node after five minutes. What a worker on a second machine gains is measured under Sizing.

What the console shows for operations#

Operations → Analytics bundles queries and their failures, data processed, compute time, AI cost in euros plus backup and license status, switchable across 7, 14 or 30 days. Operations → Load shows the state for humans: overall status and four traffic lights (running queries, queue, error rate, p90 response time over the last 15 minutes), calibrated against an overload test. The analytics board also lists tables without a physical layout and can apply the recommendation to all of them in sequence.