Hybrid search with a local model, metrics with one definition

Data discovery and metadata management

Data discovery starts by asking which data is available for an analysis. In lavalake, you explore schemas, tables and columns and search data and document contents. Metadata management adds context: names, data types and confirmed classifications help explain a table. The lineage view shows relationships recorded in the platform.

What is a dashboard?

A dashboard presents important metrics and charts in one place. A sales dashboard might show revenue, orders and trends by region. In lavalake, these analyses use defined metrics; dashboard tiles run under each viewer’s permissions. A shared definition helps teams interpret the same number consistently.

AI directly in SQL

Summarize, classify and translate text or extract information directly in queries. Administrators choose a local model or an API-backed provider for SQL functions. Forecasts and anomaly analysis extend the analytics tools.

Two searches, one ranking

Keyword search finds what is literally there — and misses everything phrased differently. Vector search finds meaning — and sometimes misses the exact part number somebody was actually looking for. Either one alone is unsatisfying, so the results are fused into a single ranking.

Search  "framework agreement notice period supplier"

  Mode    vector + keyword, fused
  Model   bge-m3  (local, CPU, no key)

  1  contracts/framework_2024.pdf   section 4.2
  2  lakehouse.raw.contracts        row 8817
  3  wiki/procurement.md            section "Notice periods"

The index is only an index

This is where search features in data platforms usually defeat governance. If the vector store holds the content itself, you have a second body of data with its own access control — and the second control knows nothing about the first one's policies.

In lavalake the search index stores embeddings, text chunks and references to source rows. The hit is then re-fetched from Trino under the searcher's token. Someone who may not see a row does not get it as a search result either; a masked column stays masked.

The search index finds candidates. Trino decides under your identity which content may be displayed.

Metrics with exactly one definition

The semantic layer carries curated metrics and multi-table semantic views. The point is not convenience but unambiguity: “revenue” is defined once, and the dashboard tile, the search and the AI agent all compute with that definition.

The assistant may propose and create metrics — with a confirmation and an audit entry. That lets the semantic model grow out of the data you already have, instead of waiting for somebody to find time for a definition workshop.

Queries and dashboards

Alongside search there are the ordinary tools: a data browser from catalog through schema to table, and a SQL worksheet that sends Trino SQL against Iceberg. Results can be placed on a dashboard as a tile.

One thing matters here: a tile stores the query, not the result. It computes for each viewer under that viewer's identity. Two people can open the same dashboard and see different numbers — and that is correct when their permissions differ.

• Data browser with a jump to lineage per table
• SQL worksheet over Trino, under your identity
• Dashboards from queries and metrics
• Tiles compute per viewer, not per author
• Materialized metrics with incremental watermark-based refresh
• Alerts against thresholds, reports stored as snapshots

Data analysis: from question to metric

Data analysis starts with a specific question: How is revenue changing, where are orders increasing, or which values stand out? Define the period, unit and comparison first. In lavalake, use SQL queries and defined metrics; Pyro can help with the analysis. Check the result against the underlying data before sharing it.

Data visualization for clear comparisons

Data visualization turns results into a readable presentation. A line suits trends over time, while a bar chart compares categories. An individual metric also needs a period and unit. An analytics view should answer the business question: a sales dashboard could show revenue trends alongside differences between regions.