Skip to content
Vijay Work Resume Blog Contact

Project case study

Governed conversational data platform

A multi-LLM conversational data agent that retrieves business knowledge, runs governed queries, reasons over answers, and builds charts.

Moved analytics discovery toward self-service by connecting a multi-LLM graph runtime with knowledge retrieval, governed Trino execution, answer reasoning, chart generation, prompt tracing, and a richer Open WebUI experience. The natural-language analytics path reduced query time by about 40% for supported workflows.

LangGraph MCP (Model Context Protocol) Python FastAPI Context Engineering RAG LLM Orchestration Agent Orchestration Prompt Engineering Generative AI Langfuse Trino DataHub Open WebUI dbt Vector Search Multi-Agent Systems Agent Harness AI Evals AI Observability Tool Use & Function Calling Production AI

Context

The problem

Business users needed a safer way to ask data questions, understand business definitions, execute governed queries, inspect answers, and create charts without routing every request through analysts or treating the model as the authorization layer. Text-to-SQL specifically tends to fail in one of two places: the model cannot find the right tables once a warehouse outgrows any prompt, or generated SQL has no safe path to execution. This platform treated both as architectural problems to solve, not prompt-quality problems to retry.

My role covered both architecture and hands-on implementation across the full service topology: custom Open WebUI experience, multi-LLM agent runtime, graph subflows, knowledge system, query engine, platform control plane, and supporting services for identity, metadata, query execution, reasoning, and visualization. The text-to-SQL path inside the platform was taken through the full journey - discovery of why naive prompting fails at enterprise scale, retrieval-grounded schema linking, graph orchestration, governed execution, tracing, delivery, and post-launch evaluation.

System trace

How the work moved through the system

A high-level operating path: where the request starts, how the system shapes it, and how other teams consume the result.

  1. 1

    A customized Open WebUI experience captures the user question and supports richer data and chart interactions.

  2. 2

    A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.

  3. 3

    Schema linking follows the RASL design: build-time decomposition of schemas and metadata into semantic units indexed in a vector store, then inference-time keyword-based multi-stage retrieval with relevance calibration to narrow candidate tables within a context budget.

Ownership

Architect + developer

Designed and built the platform across agent runtime, query engine, knowledge system, service integration, and platform orchestration.

Agent architecture

Multi-LLM subgraphs

Specialized graph paths handle knowledge retrieval, business question answering, query generation, execution, answer reasoning, and visualization.

Governance principle

No raw data to LLM

The platform keeps enterprise data access behind governed execution paths instead of making the language model the data boundary.

User outcome

Text to data + charts

The product direction is natural-language analytics with governed query execution, answer reasoning, reusable result references, and visualization workflows.

Schema linking

Retrieval-grounded

Table selection uses RASL-style decomposition and multi-stage retrieval with relevance calibration (arXiv:2507.23104), replacing schema-stuffing approaches that break beyond a handful of tables.

Extension surface

Purpose-scoped MCP

Metadata lookup, controlled execution, and charting ship as MCP servers, giving the agent stable tools and giving the platform one place to audit each capability.

Time-to-answer

~40% lower

For supported analytics workflows, the conversational path reduced query time by about 40% while preserving governed execution.

Architecture

System shape

11
  1. 1 A customized Open WebUI experience captures the user question and supports richer data and chart interactions.
  2. 2 A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.
  3. 3 Schema linking follows the RASL design: build-time decomposition of schemas and metadata into semantic units indexed in a vector store, then inference-time keyword-based multi-stage retrieval with relevance calibration to narrow candidate tables within a context budget.
  4. 4 Structured state moves between graph nodes so the system remains deterministic, easier to trace, and less wasteful with tokens.
  5. 5 Drafted SQL is validated before submission: dialect checks, guardrails, and plan inspection happen outside the prompt rather than as model self-critique.
  6. 6 A knowledge system grounds the agent using metadata, business definitions, schema relationships, query logs, joins, filters, and usage patterns.
  7. 7 Business-question subgraphs map user intent such as geography, manager, time period, metric, and comparison type into grounded query plans.
  8. 8 MCP servers expose metadata lookup, controlled query execution, and chart generation to the agent runtime with explicit tool contracts.
  9. 9 A governed query engine submits SQL through Trino with user identity, Ranger authorization, cancellation, guardrails, and Parquet-style result persistence.
  10. 10 Answer-reasoning and visualization agents consume governed result references and produce explanations or charts without turning raw data into prompt context.
  11. 11 Langfuse supports prompt management and tracing so prompts, graph behavior, and production flows can be inspected and improved.

Ownership

What I handled

9
  1. 1 Defined the architecture and service boundaries for the conversational data platform.
  2. 2 Ran discovery for the text-to-SQL path and implemented retrieval-grounded schema linking following the RASL design (arXiv:2507.23104).
  3. 3 Implemented the core backend services around agents, subgraphs, query execution, knowledge context, reasoning, visualization, and platform integration.
  4. 4 Created MCP servers wrapping metadata access, governed execution, and visualization as auditable tools.
  5. 5 Built structured graph-state flows between nodes to improve determinism and reduce avoidable token usage.
  6. 6 Integrated Langfuse for prompt management and tracing across the conversational workflows.
  7. 7 Customized Open WebUI with richer user-facing interactions for data exploration and chart workflows.
  8. 8 Designed the safety model around metadata grounding, authorization, auditability, result references, and execution guardrails.
  9. 9 Built the developer and platform workflow needed to run multiple services together with reproducible setup.

Lessons

What carried forward

8
  1. 1 Enterprise AI becomes useful when it respects existing data governance instead of bypassing it.
  2. 2 Conversational analytics needs metadata, authorization, and observability as much as it needs prompt quality.
  3. 3 Text-to-SQL succeeds or fails at schema linking long before generation quality matters.
  4. 4 The execution path is the product: governed engines with real authorization turn a demo into something deployable.
  5. 5 Traces are evaluation data; without them, accuracy work is guesswork dressed up as engineering.
  6. 6 Most text-to-SQL projects fail for architectural reasons, not model-quality reasons - which means they are solvable.
  7. 7 A production data agent is a graph, state, tracing, and UX problem as much as it is an LLM problem.
  8. 8 The hard part is not generating SQL; it is making the full path explainable, cancellable, auditable, and safe for repeated use.

Engineering decisions

Keep business logic in governed data systems

The agent can help interpret intent, but KPI definitions, authorization, and query execution stay in governed platform layers such as dbt, Trino, and Apache Ranger-based controls.

Use metadata before data

The system grounds responses with catalogs, business definitions, schema relationships, query history, and usage patterns before attempting execution.

Retrieve schemas instead of stuffing them

Table selection follows the RASL pattern (arXiv:2507.23104): decompose schemas into indexed semantic units and retrieve per question with relevance calibration. That makes catalog growth an indexing task rather than a modeling or prompt-length problem.

Give tools protocol boundaries

Exposing metadata lookup, execution, and charting as MCP servers keeps tool contracts explicit and versioned, separates platform concerns from prompt concerns, and gives security one review surface per capability.

Validate SQL outside the prompt

Dialect checks, guardrails, and plan inspection run as deterministic pre-execution steps rather than model self-critique, because self-review is not a safety boundary.

Model business questions as structured intent

Questions such as top manager by city, month-over-month trends, and quarterly growth comparisons need explicit metric, geography, time, and entity slots before SQL generation is useful.

Split the agent into subgraphs

Knowledge retrieval, business question answering, query generation, execution, answer reasoning, and visualization each need different prompts, tools, state contracts, and failure handling.

Carry structured state between nodes

Passing structured data between graph nodes made behavior easier to reason about, improved determinism, and avoided spending tokens restating context that the system already knew.

Trace prompts and graph behavior

Langfuse made prompt versions and graph execution observable, which matters when a business-facing data agent needs repeatable behavior instead of one-off demos.

Return references, not unrestricted data

Query results move through persisted references and controlled previews, which keeps downstream visualization useful without expanding the prompt surface unnecessarily.

Split agent reasoning from query execution

Separating the conversational runtime from the governed query engine made cancellation, guardrails, auditing, and authorization easier to reason about.

Treat the chat UI as product surface

Customizing Open WebUI improved the user path for data workflows because business users need richer controls than a plain chat transcript when moving from question to query to chart.

What can be shown

Public evidence without internal names

The internal systems stay private. This section keeps the public parts: my role, system boundaries, technology context, scale, decisions, constraints, and what I learned.

Internal enterprise system High-level architecture Scale signal Open-source reference

Ownership

Architect + developer

Designed the architecture and built the core services and components across the agent runtime, query engine, knowledge system, and platform orchestration pieces.

Agent shape

Multi-LLM subgraphs

The platform uses multiple language models and specialized graph subflows instead of one generic prompt path.

Safety model

No raw data to LLM

Language models reason over metadata, query plans, and bounded outputs. Enterprise data access remains behind governed execution paths.

Schema linking

Retrieval-grounded

Table selection follows the RASL design (arXiv:2507.23104): decomposed semantic units, indexed retrieval, and multi-stage relevance calibration instead of stuffing schemas into prompts.

Adaptation cost

Zero fine-tuning

New or changed tables require re-indexing metadata rather than retraining or rewriting prompts, keeping the system operable by the data platform team.

User path

Text to data + charts

The system supports natural-language data questions and visualization workflows while keeping authorization and execution controls in the platform.

Analytics query time

~40% reduction

The natural-language path reduced time-to-answer for supported business analytics questions by about 40% compared with the older manual query path.

Query shape

Business questions

Example prompts include top-performing managers in Delhi, month-over-month gross-addition trends for Mumbai, and first-quarter manager growth comparisons.

Architecture shape

  • Business users ask through a customized Open WebUI experience with richer controls for data, charts, and follow-up workflows.
  • A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.
  • Schema linking is retrieval-based rather than prompt-stuffed: schemas and metadata are decomposed into semantic units, indexed, and retrieved per question with relevance calibration, following the RASL approach (arXiv:2507.23104) - zero fine-tuning, resilient to catalog changes.
  • Agents use metadata and knowledge services built from DataHub context, business definitions, SQL logs, schema relationships, and query patterns.
  • MCP servers expose metadata lookup, controlled query execution, and chart generation as discrete, auditable tools to the agent runtime.
  • A governed query engine executes through Trino with user identity, Ranger authorization, cancellation, guardrails, and persisted result references.
  • The product handles business questions such as top-performing managers by geography, month-over-month gross-addition trends, and quarter-level growth comparisons.
  • Visualization and answer-reasoning flows work from governed result references rather than sending raw enterprise data into the language model.

Responsibilities

  • Ran the discovery phase for the text-to-SQL path: audited why naive prompting and earlier attempts fail at enterprise catalog scale, and defined production non-negotiables.
  • Designed the platform architecture for governed conversational analytics.
  • Implemented retrieval-grounded schema linking for table selection, following the RASL design (arXiv:2507.23104).
  • Built the multi-LLM agent runtime, specialized subgraphs, query execution path, knowledge system, control-plane integration, and service orchestration pieces.
  • Created MCP servers exposing metadata lookup, controlled query execution, and chart generation as auditable tools.
  • Connected Trino, Apache Ranger-based authorization, DataHub metadata, Neo4j-backed knowledge context, Langfuse prompt management and tracing, and visualization workflows.
  • Used structured data flow between graph nodes to keep behavior deterministic and reduce unnecessary token usage.
  • Customized the Open WebUI experience with richer interaction patterns for business users working with data and charts.
  • Translated the internal implementation into a case study focused on ownership, topology, design decisions, and constraints.

Constraints

  • Internal product names, dataset names, business metrics, user groups, prompts, traces, and production screenshots are not published.
  • This page can describe architecture topology, technology choices, agent patterns, safety model, ownership scope, and implementation responsibilities.

Supporting context

High-level architecture

Governed conversational analytics topology

A shareable topology can show the custom conversation UI, multi-LLM graph runtime, specialized subgraphs, metadata and knowledge services, governed query engine, Trino/Ranger execution, persisted results, reasoning, and visualization outputs.

Article

Research grounding

The schema-linking design follows RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL (arXiv:2507.23104), which decomposes database schemas into semantic entities, indexes them for retrieval, and narrows candidate tables through multi-stage relevance calibration without fine-tuning.

Open-source reference

Technology references

The shareable technology context includes Trino, Apache Ranger-based authorization, DataHub, Neo4j, LangGraph-style orchestration with tool nodes, Langfuse prompt management and tracing, MCP servers, dbt, and Open WebUI customization.

Related case studies

Continue through related work or return to the full project index.

All projects

Related projects

Continue in the same area

Project index

DataHub + Python + AI

Access governance platform

Sole architect and lead developer of an enterprise access governance platform: centralized attribute management, policy authoring for OPA/Ranger/OpenFGA, cryptographically signed bundle distribution, and enforcement that keeps working with the control plane offline.

Python + Trino + Backend engineering

Browsing-log analytics and safe-browsing pipelines

Built browsing-log ingestion and analytics pipelines for safe-browsing classification, audience management, cohort creation, and pattern-based downstream data products.