Insights: live dashboards on your warehouse¶

The Insights module lets you build live, governed dashboards directly on your Databricks SQL Warehouse. No data ingestion, no copy. A dashboard is a layout of cards — KPIs, charts, tables, matrices, multi-metric tiles — each backed by either a Databricks SQL query or a published Metric. Filters at the dashboard and card level let viewers slice data without leaving the page.

It's the platform's "tell me numbers" surface — the place a business user goes to see how the company is doing right now.

This page is the overview. Three deeper guides live alongside it:

Backing a card with a metric — how to use governed metrics as a card data source.
Matrix, heatmap & multi-metric cards — the two newer card types.
Cached query results — how the result cache works and when to bypass it.

When to choose this¶

Reach for Insights when you want to:

Stand up a dashboard fast. Pick a template (KPI overview, time series, data explorer) or ask AI to generate one for a brief.
Embed governed metrics. Drop your published Active Customers metric onto a card with one click — no SQL re-write, no drift.
Mix metrics and SQL. Some cards are metric-backed for governance; some are ad-hoc SQL for exploratory bits. The same dashboard.
Page and slice. Multi-page dashboards with global filters; per-card filter overrides.
Get a review gate before sharing. Submit a dashboard for review; an approver in the Tasks inbox approves; the dashboard moves from draft → under review → approved.
Move dashboards between environments. Export to JSON, re-import in another tenant, point at the new connection.

You do not need this module to:

Build governance models for the metrics themselves — that's Metrics.
Embed Databricks AI/BI (Lakeview) dashboards — that's Dashboards.
Run alerts when a number crosses a threshold — that's Logic Engine.

What Insights looks like¶

Surface	Where	What you see
List	`/insights`	Sidebar (search + status checkboxes + tag filter) on the left, dashboard cards on the right with status badges (Draft / Under review / Approved) and tag pills.
Create	`/insights/new`	Three on-ramps: pick a template, ask AI to generate one, or start blank.
Viewer	`/insights/{id}`	Multi-page tabs, filter bar, cards in their layout. Edit toggles inline edit-in-place. Submit for review opens the workflow trigger.
Editor	`/insights/{id}/edit`	Pages, sections, cards, layout picker, filter config, AI sidebar for editing card SQL / metric binding.
Card edit panel	Viewer in edit mode → click a card	Right-side slide-over (~760 px) with full edit form. The dashboard stays visible behind; a spotlight scrim highlights the focused card so you see the live preview as you change it.

Card types¶

Type	Use it for	Backing
Metric	Single KPI, with optional comparison or trend. Five layouts: simple, sparkline, target, segmented (2–3 sub-KPIs), bars.	SQL or metric.
Chart	Time series, bar, area, line. Y-axis formatting, custom tooltip, value labels, axis titles, legend at top-left.	SQL or metric.
Table	Tabular display with column-level number formatting.	SQL or metric.
Matrix	Pivot table over `(row, column, aggregation)`, with optional heatmap colouring (brand / success-danger / info colour scale).	SQL or metric.
Multi-metric	One query → grid of KPI tiles, each tile mapped to a column with its own format and trend.	SQL or metric.

Two ways to back a card¶

Every card has a data source discriminator:

sql — you write Databricks SQL. The platform parameterises filters, runs through the read-only validator, executes via statement_execution, and renders.
metric_ref — you pick a published metric for the same connection; the platform runs the metric's currently-published version and renders the result. Switching grain or compare-period is a chip on the card; the SQL is owned by the metric, not the dashboard.

A dashboard can mix both. The picker in the From Metric tab of the add-card modal only offers active, published metrics whose connection matches the dashboard's connection — you can't accidentally pair an Acme prod metric onto an Acme staging dashboard. The full contract is described in Backing a card with a metric.

Setup — what an admin needs to do once¶

Prereq	Where	Why
Databricks integration	`/admin/integrations` → Databricks	The SQL warehouse Insights queries against. Use Databricks per-user OAuth for per-user RLS + per-user audit.
AI provider key	`/admin/integrations` → AI Provider Keys	Powers the AI dashboard generator and the in-card Ask AI panel.
Roles	`/rolegroups`	`insights.read` to view; `insights.manage` to author dashboards / cards / ad-hoc queries / import / export / submit for review.
(Optional) Workflow	`/admin/workflows`	If you want a review gate before publishing, configure a workflow for `insight_dashboard`. The platform seeds a sane default.

Filters and parameters¶

Global filters at the dashboard level: date range, number, dropdown — auto-detected from the column type. Number filters carry a comparison operator (=, ≠, >, ≥, <, ≤) the viewer can change at runtime.
Card filters per card override the global filter for that card.
Filter drawer — every card has a funnel icon; clicking it shows exactly which filters apply.
Multi-page — pages share global filters; switching pages doesn't reset them.

Filters are bound through SQL parameters ({{name}}), and the platform's parameter expander handles the operator + value pair safely (no string interpolation into SQL).

Edit-in-place vs full editor¶

Inline edit — from the viewer, click Edit → cards get hover overlays (pencil + archive). Click pencil → the right-side slide-over opens the card edit form. The dashboard remains visible; you can click another card to edit it without leaving.
Add charts — in inline edit mode, an empty slot appears for any archived card; click Add charts to browse archived cards in a 3-column grid and batch-restore.
Full editor — click the gear → /insights/{id}/edit for page management, layout picker, dashboard settings.

Inline edit is the day-to-day flow; the full editor is for restructuring.

AI assistance¶

AI dashboard generation (/insights/new → AI tab). Describe the dashboard ("Sales pipeline by stage, last 90 days, with conversion KPIs"); the platform's insights agent reads the connection's published metrics first, then any SQL it needs to invent. Metrics-first is a hard rule: if a published metric covers the brief, the agent uses metric_ref; SQL only fills the gaps. The dashboard generator validates every generated query (column allowlist + EXPLAIN + actual LIMIT 1 execution) and includes auto-corrected fallbacks.
Per-card AI panel. Click Ask AI on a card in edit mode → slide-out chat tied to the card's context. The agent has tools to run discovery queries, set the SQL / display config / metric binding, and suggest title / type / colours. Great for "this query errors with 'column not found' — fix it" moments.

Submit for review¶

If a workflow is configured for insight_dashboard:

Click Submit for review on the dashboard → reviewer picker → task created in the reviewer's Tasks inbox.
The reviewer approves or rejects; the dashboard's status moves accordingly. The viewer page shows the workflow progress timeline.
An approved dashboard can be referenced as the canonical version of that report.

Result caching¶

SQL-backed cards are cached in PostgreSQL after each successful execution. The cache key includes the rendered SQL, parameters, integration, and (for OAuth U2M) the caller user — so per-user RLS is honoured. TTL cascade: per-card override → per-integration default → 24 h. Refresh-on-demand from the card's Refresh chip; admins can purge from /admin/insights-cache. Full mechanics in Cached query results.

Metric-backed cards rely on the metrics module's own cache, which uses the same key contract.

Export and import¶

Export (/insights/{id} → toolbar) downloads a portable JSON envelope with name, description, layout, cards, filters, and tag metadata. Tenant-local IDs (integration, governance, metric definition) are intentionally dropped.
Import (/insights → import) requires a destination integration; the platform creates a fresh dashboard with a (imported) suffix. Metric-backed cards are downgraded to a SQL placeholder so the validator passes — re-bind to the destination tenant's metrics.

Limitations¶

Limit	Why	Workaround
Queries are read-only — DDL / DML are blocked.	Safety.	Wire writes via Flows triggered by a Logic Engine alert.
Cold SQL warehouses can take 1–3 minutes to warm.	Databricks behaviour.	The platform polls up to 5 minutes; the viewer shows a "warehouse starting" hint. Use a serverless warehouse to avoid cold starts.
Metric-backed cards can't preview SQL in the modal.	The metric's compiled SQL is server-only.	The card runs immediately on save; the dashboard refreshes.
Export drops governance metadata (domain, department, team) and tenant-specific IDs.	Tenant boundary.	Re-tag in the destination.
Embedded view doesn't honour OAuth U2M for embedded Lakeview dashboards.	The embedding identity is per-organisation.	For per-user RLS, view the dashboard in the Dashboards module; for ad-hoc, build it in Insights with U2M.

Audit & compliance¶

Question a CISO might ask	Where to look
"Whose Databricks identity ran this query?"	OAuth U2M: the calling user's. PAT / SP: the integration's identity. The cache key reflects this.
"Did this dashboard run a write?"	Read-only validator blocks DDL / DML at the service layer; the response logs the actual SQL.
"Who approved this dashboard?"	Workflow progress card on the viewer; full audit at `/admin/audit-log` filtered by module `insights` and entity type `dashboard`.
"Who exported this dashboard?"	`/admin/audit-log` filtered by module `insights`, entity type `dashboard`, action `exported`. Every export records the actor and timestamp.
"Where did this exported dashboard end up?"	Export is a download — the recipient is whoever you sent the file to. The platform records who triggered the export but not the download destination.
"Can a non-author edit a dashboard?"	`insights.manage` required for any mutation.

Troubleshooting¶

Symptom	Likely cause	Fix
Card stuck on "Loading…"	Warehouse cold-starting.	Wait up to 5 minutes; switch to a serverless warehouse.
Card returns "0 rows" but you expected data	The query genuinely returned no rows; or the filter wiped them.	Check the filter drawer; check the Total rows badge in Insights logs.
"Column not found" from AI-generated card	The model hallucinated.	Open the card's Ask AI panel; the agent will auto-discover the actual columns and rewrite.
412 `DATABRICKS_OAUTH_CONSENT_REQUIRED`	OAuth U2M consent missing.	The card prompts you to re-consent.
Metric tab in Add Card is empty	No active, published metrics for this dashboard's connection.	Publish a metric for the same connection; refresh.
409 `METRIC_INTEGRATION_MISMATCH` on save	The metric is on a different connection than the dashboard.	Pick a different metric or change the dashboard's connection.
Submitted-for-review status stuck	Workflow misconfigured (no reviewer in the role group).	Add members to the role group; or cancel and re-submit.