Databricks: per-user authentication (U2M OAuth)¶

Run Databricks queries as the person who triggered them — so Unity Catalog row-level security, per-user audit, and least privilege follow the user into Datahub — without changing how your existing service-principal integrations behave.

Auth methodoauth_u2m

Query identityPer-user · UC RLS applies

Coexists withService principal · PAT

Token modelEncrypted refresh → short-lived access

When to choose this¶

Unity Catalog RLS depends on the user

Row filters or column masks keyed on current_user(), group membership, or any per-user attribute only work when the query runs as the real user.
Per-user audit attribution

Databricks audit logs should attribute each query to the end-user who ran it — not to one shared service principal.
Least privilege

Users see only the data they have UC privileges for, even when they query through Datahub.
Interactive querying

Insights, Metrics, Dashboards' native query builders, and Genie all honour oauth_u2m.

When to stick with the service principal

Embedding dashboards via mint_embed_token — embed tokens attest the viewer under a different identity model, so per-user UC RLS does not apply (see Limitations).
Batch / scheduled jobs — there is no end-user to attribute the query to.
Coarse-grained UC permissions that don't depend on the calling user — the extra per-user consent buys you nothing.

You don't have to choose — mix both

oauth_u2m is additive. Keep your existing PAT and service-principal integrations for embeds and batch jobs, and add a parallel oauth_u2m integration for interactive querying. A single tenant can run all three side by side without interference.

How per-user auth works¶

The first time Datahub needs your Databricks identity it opens a consent gate; once you've consented, an encrypted refresh token lives in your tenant's key vault and every query you trigger mints a short-lived access token and runs as you.

flowchart LR
    Q0["You trigger a query<br/>Insight · Metric · Genie"] --> GATE{Your consent<br/>on file?}
    GATE -- "no" --> TASK["'Connect to Databricks' task<br/>+ toast on the page"]
    TASK --> CONSENT["Consent page<br/>/databricks/consent/{id}"]
    CONSENT --> DBX["Databricks OAuth<br/>PKCE authorize + consent"]
    DBX --> CB["Callback → encrypted refresh<br/>token stored in key vault"]
    CB --> RUN
    GATE -- "yes" --> RUN(["Query runs as YOU<br/>UC row/column security applied"])
    classDef accent fill:#ff8400,stroke:#e67700,color:#ffffff,font-weight:600;
    class RUN accent

Figure 1 — The consent gate fires only on the first query (or after a revoke). Once your consent is on file, queries run straight through as your Unity Catalog identity, so row filters and column masks are evaluated for you.

Why teams reach for this

The same query returns different rows to different people — exactly what Unity Catalog RLS is for — and Databricks' own audit log names the real user, not a shared service principal. You get least-privilege querying through Datahub without re-implementing your UC access model.

How it coexists with the service principal¶

Per-user OAuth and the shared service identity run on parallel paths. Interactive query surfaces resolve to your consented token; embeds and batch work keep using the shared credential.

flowchart LR
    START["A Databricks operation"] --> KIND{What kind?}
    KIND -- "interactive query<br/>+ admin list / discovery" --> UT["Your (or the admin's)<br/>consented refresh token"]
    UT --> UID(["Runs as your UC identity<br/>per-user RLS applies"])
    KIND -- "embed (mint_embed_token)<br/>+ batch / scheduled jobs" --> ST["Shared credential<br/>in key vault"]
    ST --> SID["Runs as the service identity<br/>no per-user RLS"]
    classDef accent fill:#ff8400,stroke:#e67700,color:#ffffff,font-weight:600;
    class UID accent

Figure 2 — Per-user vs shared identity. Interactive queries (and admin list/discovery operations, which use the admin's own consent) run as the calling user; embedded dashboards and batch jobs run as the service principal.

Concepts¶

Concept	What it is
`oauth_u2m`	The per-user authentication method on a Databricks integration. Additive — it doesn't replace PAT or service-principal auth.
Consent	A one-time, per-user, per-integration authorisation. Grants Datahub a refresh token to act as you against Databricks.
Refresh token	Long-lived credential, stored encrypted in your tenant's key vault. Rotates transparently; never returned by any API.
Access token	Short-lived token minted from the refresh token for each Databricks request.
Consent gate	The check Datahub runs before a query; if your consent is missing/revoked it raises a task + toast instead of failing silently.
Consent page	The focused landing at `/databricks/consent/{integration-id}` — integration name, workspace host, status pill, one button.

Setup¶

One-time admin steps¶

The integration form in Datahub walks you through these four steps in order. The exact Redirect URI for your tenant is shown in the form with a copy button — no need to type it by hand.

Open the integration form in Datahub. Go to Admin → Integrations → New integration → Databricks, and set Authentication method to User-to-Machine OAuth. The form now shows the redirect URI you'll paste into Databricks in step 2.
Register an OAuth app in the Databricks Account Console. Open Settings → App connections (or User Management → OAuth apps, depending on your account UI) → Add connection. Paste the Redirect URI from the Datahub form into the Redirect URLs field — it must match character-for-character. Save the app and copy the Client ID.
Finish the integration in Datahub. Paste the Host (your Databricks workspace URL) and the Client ID from step 2.
Save. The integration is now ready for users to consent.

The client type you registered in Databricks decides what else you enter:

Public client (PKCE — recommended)Confidential client (has a secret)

Datahub mints PKCE-protected tokens, so no client secret is needed. Leave the Confidential client (has a client secret) toggle off and just save the Host + Client ID. This is the recommended path unless you have a specific reason to register a confidential client.

If you registered the OAuth app as a confidential client, turn the Confidential client (has a client secret) toggle on. The form then reveals an optional Client Secret field — paste the secret alongside the Client ID before saving.

Let the form give you the Redirect URI

Don't type the redirect URI by hand. Copy it from the Datahub form and paste it into the Databricks OAuth app's Redirect URLs — a mismatch (even a trailing slash) is the most common cause of a consent loop.

Per-user steps (each user, once per integration)¶

End users do not need access to the admin Integrations module. The first time Datahub needs to query Databricks on your behalf, it creates a high-priority Connect to Databricks task for you and surfaces a toast on the page that triggered the query.

Open the Connect to Databricks task from your inbox (or the bell), or click the toast on the dashboard / metric / insight that needs Databricks.
You land on the consent page at /databricks/consent/{integration-id} — it shows the integration name, the workspace host you're about to authorise, and a single Connect to Databricks button.
Click Connect to Databricks. You're sent to Databricks to consent.
After consenting, Databricks redirects you back to the same page. The status flips to Connected, the task auto-completes, and any pending toast disappears.

Admins get the same flow, plus a roster

Admins can consent from Admin → Integrations → {integration} (Authentication tab → Per-user OAuth section) and see a global view of who has consented per integration. End users only ever see their own consent state.

What users see¶

State	What you see	What it means
Before consenting (passive)	A Connect to Databricks task in your inbox, high priority	Datahub minted the task automatically the first time it needed your Databricks identity. The task's open action takes you straight to the consent page — you don't need admin permissions.
Before consenting (active)	A toast on a dashboard / metric / insight: "Connect your Databricks account to continue. We've added it to your tasks."	Background polling tripped the consent gate. The toast is rate-limited to once per 30 seconds per integration so it doesn't stack on auto-refreshing pages.
On the consent page	Integration name, workspace host, status pill (Connected / Not connected) and the single Connect to Databricks button	A focused, opinionated landing — no admin chrome, no other integration's settings.
After consenting	The status flips to Connected · last refreshed … and the task is marked done	You're connected. Tokens rotate transparently — you don't need to re-consent unless you revoke or your session is wiped.
When a query needs consent again later	The same task is re-opened (or a new one is created if the old one was completed/closed) and the toast returns	Datahub tried to query Databricks and your consent was missing or revoked. Open the task or the toast to restore access.

Revoking

Admins see a small revoke button on the integration's Per-user OAuth card. End users can revoke from the same consent page (/databricks/consent/{id}) — once you're connected, the page renders the same revoke control.

How rotation works¶

You consent once. Datahub stores an encrypted refresh token in your tenant's key vault and uses it to mint a short-lived access token for each Databricks request. The refresh token rotates transparently — you do not see it and do not re-consent on rotation.

flowchart LR
    C["Consent once"] --> V[("Encrypted refresh token<br/>tenant key vault")]
    V --> M(["Mint short-lived access<br/>token per request"])
    M -. "rotates transparently" .-> V
    R1["You revoke (X)"] --> RE["Re-consent needed"]
    R2["IdP session wipe /<br/>password reset"] --> RE
    R3["Admin removes the<br/>OAuth app in Databricks"] --> RE
    RE --> C
    classDef accent fill:#ff8400,stroke:#e67700,color:#ffffff,font-weight:600;
    class M accent

Figure 3 — Steady state is a silent loop: mint, rotate, repeat. Only three events break it and require a one-click re-consent — an explicit revoke, a Databricks-side token invalidation, or an admin removing the OAuth app.

You only need to re-consent when:

Trigger	What happened	What to do
You explicitly revoke	You clicked the X on the consent card	Consent again (one click) next time you query
Databricks invalidates your refresh token	Password reset, IdP session wipe	Consent again when the toast returns
An admin removes the OAuth app	The OAuth app was deleted in the Databricks Account Console	A new app must be registered; then consent again

In all three cases, the consent toast appears the next time you query — re-consenting takes one click.

Limitations¶

Limitation	Why	What to do
Embedded dashboards use the service-principal identity.	`mint_embed_token` attests the viewer under a different identity model, so per-user UC RLS does not apply to embedded dashboards.	For RLS-sensitive views, prefer the in-app query surfaces (Insights, Metrics, Dashboards' native query builders, Genie) — they all honour `oauth_u2m`.
Admin list / discovery uses the admin's own consent.	Listing workspaces, notebooks, or other resources runs as the admin, not a service principal.	If an admin hasn't consented yet, they get the same consent toast and click Connect once.
One refresh token per (user, integration).	The OAuth consent anchors on the first workspace; the refresh token is reused across all workspaces of the same integration.	If your tenant has multiple workspaces under one integration, a single consent covers all of them.
Service principal is not bypassed.	`oauth_u2m` is additive.	Existing PAT and service-principal integrations behave exactly as before; the three coexist without interference.

Embedded dashboards ≠ in-app queries

An embedded Databricks dashboard shows data as the integration's service principal sees it. If a view has per-user row filters, the same numbers in an embed and in Insights will differ — this is expected. Route RLS-sensitive data through the in-app query surfaces.

Audit & compliance¶

Concern	Datahub's behaviour
Where are refresh tokens stored?	Encrypted in your tenant's key vault. Only Datahub's backend can decrypt them, and only at query time.
Does Datahub leak the key-vault reference?	No. The reference (`encrypted_refresh_token_ref`) is never returned in any API response. An automated test fails CI if the field appears in any endpoint response body.
Can a user impersonate another user via the API?	No. Every endpoint that touches per-user state derives the calling user from the JWT exclusively. Decoy `user_id` fields in request bodies, queries, or headers are silently ignored — a regression suite (`*_no_spoof.py`) asserts this on every relevant endpoint.
Does Databricks see the actual end-user in audit logs?	Yes. Each query runs as the calling user's UC identity, so Databricks audit attributes the query to them — not to Datahub or a service principal.
Can I revoke a user's access centrally?	Yes — via the Databricks Account Console (revoke the user from the OAuth app or remove their UC privileges). The next Datahub query for that user fails and shows the consent toast.

Troubleshooting¶

Symptom	Likely cause	Fix
"Databricks consent required" toast on every query	Refresh token missing or revoked	Open the integration detail page → Connect to Databricks
Consent loop (consent → toast → consent again)	Redirect URI mismatch between the OAuth app and your Datahub base URL	Copy the Redirect URI shown in the Datahub integration form and paste it into the Databricks OAuth app's Redirect URLs — it must match character-for-character (scheme, host, path, no trailing slash).
`invalid_request — redirect_uri 'http://127.0.0.1:8000/...' not registered`	Build older than 2026-04-21: the backend derived the redirect URI from `request.base_url`, which becomes the upstream bind host behind a reverse proxy / Vite dev proxy and never matches what you pasted into Databricks.	Pull the latest build. The frontend now sends its own `window.location.origin`-derived URI; the backend uses it verbatim (after path validation) so the URI Databricks sees matches the one shown in the form. No re-registration needed if you already pasted the correct URI.
`Not authenticated` on the Databricks callback after consenting	Build older than 2026-04-21: the `datahub-token` cookie was set with `SameSite=Strict`, so the browser stripped it on the cross-site redirect from `databricks.net` back into Datahub; the callback saw no session and returned 401.	Pull the latest build. The auth cookie is now `SameSite=Lax` — it still blocks cross-site POST/PUT/DELETE and XHR/fetch (full CSRF protection for every mutating endpoint) but travels on legitimate inbound top-level navigations like the OAuth callback. After deploying, users log in once more for the cookie to be rewritten.
API or Warehouse connection test reports `'credentials_secret_ref' is required but not set` for an OAuth U2M integration	Build older than 2026-04-21: the test branched on `auth_type` and treated anything that wasn't a service principal as PAT (which has a `credentials_secret_ref`). OAuth U2M has no integration-level credential by design — auth is per-user at query time.	Pull the latest build. The API test now verifies workspace host reachability (`Workspace host reachable — … (per-user OAuth tested at consent)`); the Warehouse test reports a non-failing `OAuth U2M — warehouse connectivity is verified per-user at query time after consent`. Query auth is exercised the moment any user runs a query.
`OAUTH_CLIENT_ID_MISSING` when clicking Connect to Databricks	Either (a) the integration predates per-user OAuth and never had a Client ID saved, or (b) it was saved on a build older than 2026-04-21 where Key Vault silently rejected the secret name (an underscore was incompatible with Azure KV).	Open the integration → Authentication tab → re-paste the OAuth app's Client ID (and Client Secret for a confidential client) and save. The current build writes a KV-compatible name and fails loudly on storage errors, so the next save lands a working entry.
Embedded dashboard shows different data than the same query in Insights	`mint_embed_token` uses the SP identity, not your user identity	Expected. Use the in-app query surfaces for RLS-sensitive views.
Revoke button does nothing visible	Revoke is silent on success — the card flips back to "Connect to Databricks"	If it doesn't flip, refresh the page; the underlying token has already been deleted on the backend.