AlchemyLake for Databricks
Install a governed creative surface directly inside your Databricks workspace. AlchemyLake turns the data your lakehouse already trusts into finished deliverables Genie doesn't make: board-ready PowerPoint decks with a read-aloud script and Q&A prep under every slide, enterprise PDF dossiers with a statistical appendix and an Excel evidence workbook, deep-research dossiers (a planned multi-step investigation — Genie sources get governed SQL follow-ups in one continuing conversation), designed infographics, animated video briefings with narration (six formats), two-host data podcasts (five formats), and sonified scores (six genres) whose tempo follows your growth.
Get started free — no card required
- Sign up — 50 free credits land in your account immediately, nothing to enter but an email.
- Grab a developer key — sign in at app.alchemylake.com →
Studio → Developer · MCP & keys → Forge a new key (starts with
alk_…). - Clone this repo and deploy it into your Databricks workspace:
Trackinggit clone https://github.com/zorost/alchemylake-databricks.git cd alchemylake-databricks # edit the workspace host in databricks.yml (see Path 1 below), then: databricks bundle deploy -t prod databricks bundle run alchemylake_app -t prodmaingets you the latest build;git checkout v0.4.0first pins the deploy to a tagged release instead, if you'd rather not move withmain. - Open the App URL the CLI prints, paste your key, then Load governed sources → pick one → Transmute.
That's a governed PDF, deck, infographic, or video briefing — sealed to your own data, running inside your own workspace — for free, in about five minutes. No Databricks Apps on your workspace? Skip straight to Path 2 — register one URL and Genie, Claude, or Cursor get the same 50 free credits to work with, no bundle required.
Every lane binds to a governed source, and the numbers discipline holds throughout: the platform computes an analyst-grade statistical dossier from the bound rows by code — trend with fit quality, outliers, correlations, segment shares, concentration, pivots — the model never authors a figure, a verifier checks every claim after generation, and the result is sealed (source · row count · data sha256 · verification score) with the seal embedded in the file itself. Genie answers bind as sources too, and one-click recipes ship a board deck or a whole campaign pack from one table. Every credit accounted.
Where Genie stops, AlchemyLake starts. Genie answers questions about your data inside the workspace — tables and charts. AlchemyLake takes the same governed rows the last mile: "Ask Genie for Q2 revenue by region, then render an 8-slide deck titled Q2 Momentum" leaves the workspace as a .pptx the CFO can present cold — every figure verified against the rows, provenance sealed into the file.
This repository is a self-contained Databricks Asset Bundle. One command deploys a Databricks App (SSO-authenticated, running on your workspace) that calls the AlchemyLake platform over MCP. Nothing bypasses governance: the same credit ledger, provenance seals, and role checks that protect the web app protect every call made from inside Databricks. Your data never leaves the lakehouse except as the exact rows you choose to bind.
.
├── databricks.yml # the Asset Bundle (App resource + targets)
├── app/
│ ├── app.py # the Streamlit App (thin MCP client)
│ ├── app.yaml # App runtime config (command + env)
│ └── requirements.txt
└── sql/
└── ai_render.sql # optional UC function: call AlchemyLake from SQL/Genie
Three ways to use AlchemyLake with Databricks
What agents can ship from a table (things a Genie answer alone cannot):
render_deep_research— a planned multi-step investigation: the brief is decomposed into sub-questions, each answered with real evidence (Genie sources get governed SQL follow-ups in one continuing conversation), then synthesized into a sealed PDF dossier + Excel evidence workbook.render_presentation— a .pptx board deck, 5–20 slides, speaker script + Q&A in every notes pane, AI cover art, real charts from the rows.render_report— an enterprise PDF dossier (KPI band, chart sections, statistical appendix, citations, methodology) plus an Excel evidence workbook: raw rows, facts, statistics, pivot, correlations.render_infographic— a designed KPI poster with the exact figures rendered in-image and a branded provenance strip.render_video_briefing— an animated video briefing: platform-drawn charts, spoken narration, motion and crossfades. Six formats (styleparam): consultant walkthrough, newsroom segment, executive stand-up, documentary deep-dive, field report, social recap.render_podcast— a two-host audio briefing with a sealed transcript. Five formats: two-host interview, skeptic's debate, executive stand-up, narrative deep-dive, plain-language walkthrough.render_music— a sonified score (tempo ↔ momentum, mode ↔ trend) plus the literal data-motif WAV of the rows. Six genres: cinematic score, corporate uplift, ambient data fields, electronic pulse, orchestral arc, lo-fi data study.
Prefer plain REST? The same keys work on the public API:
https://app.alchemylake.com/api/public/v1 (OpenAPI at /api/public/v1/openapi.json).
Prefer a terminal? npx alchemylake is a zero-dependency CLI over that API:
export ALCHEMYLAKE_API_KEY=alk_YOUR_KEY
npx alchemylake upload ./q3-actuals.xlsx # CSV/Excel/PDF/Word/text → governed source
npx alchemylake render report --prompt "Board brief" --source up.1a2b3c
npx alchemylake render deep_research --prompt "Why did Q3 dip?" --source genie:dbx1
How it fits together
AlchemyLake never runs inside this bundle — the App is a thin, SSO'd client that calls the governed platform over HTTPS/MCP. Your tables stay in your lakehouse; only the exact rows you bind for a given render ever leave the workspace, and only to produce that one sealed deliverable.
flowchart LR
subgraph WS["Your Databricks workspace"]
UC[("Unity Catalog<br/>tables & volumes")]
Genie["Genie space"]
CSV["Uploaded CSV / Excel"]
App["AlchemyLake App<br/>(this bundle · Streamlit · SSO'd)"]
Agents["Genie / Agent Bricks<br/>Claude · Cursor"]
end
AL(["AlchemyLake<br/>governed render platform"])
UC -- "bind rows" --> App
Genie -- "bind an answer<br/>(conversation continues)" --> App
CSV -- "bind (BYO-data)" --> App
App <-- "MCP: render, verify, seal" --> AL
Agents <-- "13 MCP tools, same contract" --> AL
AL -. "no-egress text tier" .-> FM["Your Databricks<br/>Foundation Model endpoint"]
AL --> Out(["Sealed deliverable<br/>PDF · PPTX · image · video · audio"])
Every render — from the App, from an agent, or from ai_render() in SQL — goes
through the same discipline before it comes back:
flowchart LR
A["Bind<br/><sub>pick a governed source</sub>"] --> B["Analyze<br/><sub>compute the facts</sub>"]
B --> C["Design<br/><sub>plan the deliverable</sub>"]
C --> D["Transmute<br/><sub>generate</sub>"]
D --> E["Verify<br/><sub>check every claim</sub>"]
E --> F["Seal<br/><sub>embed provenance</sub>"]
The model is never the source of a number — every figure is computed from your rows first, and whatever the render says is checked against those figures afterward. The seal (source · row count · data hash · verification score) rides embedded in the file itself, in every format, so it survives being forwarded, downloaded, or printed.
Beyond a Genie answer
What people actually ask it for
Every lane binds to a governed source, so the figures in the output are the figures in your table — not a paraphrase. A sample of real workflows across the eight lanes:
Free to try: sign up for 50 credits, no card required, and point this bundle at your own key.
Prerequisites
- Databricks CLI ≥ 0.230 and authentication to your workspace:
databricks auth login --host https://<your-workspace>.cloud.databricks.com - Databricks Apps enabled on the workspace (Premium/Enterprise).
- An AlchemyLake developer key (
alk_…): sign up free (50 credits, no card required) or sign in at https://app.alchemylake.com → Studio → Developer · MCP & keys → Forge a new key.
Path 1 — Deploy the App (one command)
First set your workspace host in databricks.yml — replace the
your-workspace… placeholder under both targets.dev and targets.prod
(run databricks auth profiles if you need to find it). Then:
databricks bundle deploy -t prod # upload source + create the App
databricks bundle run alchemylake_app -t prod # start it; prints the App URL
CLI note (Terraform pin). Some Databricks CLI builds download a Terraform whose signing key has expired, which breaks
bundle deploy. If you hit that, point the CLI at a local Terraform binary and pin the version to match it:export DATABRICKS_TF_EXEC_PATH="$(command -v terraform)" export DATABRICKS_TF_VERSION="1.9.8" # set to your local `terraform version`This is the exact, verified path AlchemyLake's own reference App is deployed with — it runs live in-workspace as a Streamlit App under SSO.
Open the App URL that the CLI prints. Paste your alk_… key in the sidebar
(or bind it from a secret — see below), then Load governed sources → pick one
→ Transmute. Every result carries its provenance seal.
The pasted key is remembered in that browser (client-side only, until cleared or replaced) so it survives a refresh — a workspace-bound secret still always takes priority. The App also carries a self-contained Docs tab and an expanded About / Install tab (how to use it, why the verification model matters, where the render compute actually runs, and the fastest path to a key) so it's useful even where the public docs site isn't reachable.
Bind the key from a secret (recommended for teams)
So analysts never paste a key:
databricks secrets create-scope alchemylake
databricks secrets put-secret alchemylake api_key # paste alk_...
Then uncomment the ALCHEMYLAKE_API_KEY / valueFrom block in
app/app.yaml and databricks.yml, and redeploy.
Path 2 — Register AlchemyLake as an MCP server (Genie / Agent Bricks)
No bundle required. In your agent/MCP client configuration add:
{
"mcpServers": {
"alchemylake": {
"url": "https://app.alchemylake.com/api/mcp",
"headers": { "Authorization": "Bearer alk_YOUR_KEY" }
}
}
}
Genie / Agent Bricks, Claude, and Cursor will discover the thirteen tools
automatically. Ask, for example: “Using AlchemyLake, write a sealed 3-bullet
executive summary of gold_ridership_national_monthly” — or go bigger:
“Render an 8-slide presentation of that table titled Ridership Momentum”
(a .pptx with a presenter script under every slide), “Run deep research on
why ridership dipped this quarter” (render_deep_research returns a sealed
dossier + evidence workbook), or “Run the campaign-pack recipe”
(run_recipe renders the poster, the report, and the copy in one call, all
verified and sealed).
Path 3 — ai_render() in SQL / Genie
Edit sql/ai_render.sql (set your catalog/schema) and run it in a SQL editor
on serverless compute with external network access enabled. Then:
SELECT ai_render(
'One-sentence executive read of this quarter''s ridership',
'ntd_demo.ntd.gold_ridership_national_monthly'
) AS narrative;
See the header of sql/ai_render.sql for the network-egress and secret
prerequisites, and the MCP fallback if egress cannot be enabled.
Validate before deploying
databricks bundle validate -t prod
FAQ
Does my data leave the lakehouse? Only the exact rows you bind for a given render are sent to produce that render — nothing else is read, and bound data is never used to train models. A no-egress text tier can route even that narrative step to your own Databricks Foundation Model endpoint instead; see the residency docs.
Do I need Databricks Apps enabled for this? Only for Path 1 (the App). Path 2
(MCP) and Path 3 (ai_render()) need nothing beyond CLI/SQL access and a developer
key — no App, no extra compute resource.
Can agents use this without a human in the loop? Yes — that's Path 2. Register
/api/mcp once and Genie, Agent Bricks, Claude, and Cursor all gain the same thirteen
governed tools, metered and sealed exactly like the UI.
What does a Genie question cost? Genie compute bills inside your Databricks workspace (serverless DBUs — roughly 1.5 DBU ≈ $0.11 per question; Databricks gives every user 150 free Genie DBUs a month). AlchemyLake credits meter only the render itself, and the Studio shows the DBU estimate next to any Genie source.
I don't have Databricks — can I still use everything? Yes. Upload CSV, Excel,
PDF, Word, text or Markdown (in the web Studio, this App's Sources tab, the API,
the CLI, or the upload_source MCP tool) and every lane — including Deep
Research — runs with the same verification and seals.
Is AlchemyLake a Databricks product? No. AlchemyLake is an independent platform by Zorost Intelligence that plugs into Databricks; this repository is Apache-2.0 licensed and is how it plugs in. The hosted platform it talks to is a separate, metered service (50 free credits to start, no card required).
What does this bundle actually contain? Only a Streamlit client (app/app.py),
its App runtime config, and an optional SQL function — everything under
sql/ and app/. It holds no model weights, no proprietary
rendering code, and no secrets; it is a thin, auditable client over the same MCP/REST
contract documented at app.alchemylake.com/docs.
Something's broken. Open an issue in this repository, or email
[email protected] — see SECURITY.md for vulnerability reports
specifically.
Documentation
docs/DEVELOPER_GUIDE.md— CLI, REST API & MCP: get a key, every command, VS Code/Cursor/Claude setup, use cases, Python/Node examples, GitHub Actions.docs/GENIE_GUIDE.md— Genie integration: connect a space, health score, certification, all eight lanes, sharing model.- Platform docs: app.alchemylake.com/docs (Studio walkthrough, credits, security).
Uninstall
databricks bundle destroy -t prod
AlchemyLake is a product of Zorost Intelligence. Platform docs: https://app.alchemylake.com/docs. Support: [email protected].
