Hybrid search with a local model, metrics with one definition

Two searches, one ranking

Keyword search finds what is literally there — and misses everything phrased differently. Vector search finds meaning — and sometimes misses the exact part number somebody was actually looking for. Either one alone is unsatisfying, so the results are fused into a single ranking.

Search  "framework agreement notice period supplier"

  Mode    vector + keyword, fused
  Model   nomic-embed-text  (local, CPU, no key)

  1  contracts/framework_2024.pdf   section 4.2
  2  lakehouse.raw.contracts        row 8817
  3  wiki/procurement.md            section "Notice periods"

The index is only an index

This is where search features in data platforms usually defeat governance. If the vector store holds the content itself, you have a second body of data with its own access control — and the second control knows nothing about the first one's policies.

In lavalake the vector index only remembers which columns are key and text. The hit is then re-fetched from Trino under the searcher's token. Someone who may not see a row does not get it as a search result either; a masked column stays masked.

A search index that holds content is a second body of data. A search index that only points at where something is, is not.

Metrics with exactly one definition

The semantic layer carries curated metrics and multi-table semantic views. The point is not convenience but unambiguity: “revenue” is defined once, and the dashboard tile, the search and the AI agent all compute with that definition.

The assistant may propose and create metrics — with a confirmation and an audit entry. That lets the semantic model grow out of the data you already have, instead of waiting for somebody to find time for a definition workshop.

Queries and dashboards

Alongside search there are the ordinary tools: a data browser from catalog through schema to table, and a SQL worksheet that sends Trino SQL against Iceberg. Results can be placed on a dashboard as a tile.

One thing matters here: a tile stores the query, not the result. It computes for each viewer under that viewer's identity. Two people can open the same dashboard and see different numbers — and that is correct when their permissions differ.

• Data browser with a jump to lineage per table
• SQL worksheet over Trino, under your identity
• Dashboards from queries and metrics
• Tiles compute per viewer, not per author
• Incremental materialization for expensive queriesIn development
• Alerting and scheduled reportsIn development