A multi-LLM conversational data agent that retrieves business knowledge, runs governed queries, reasons over answers, and builds charts.
Moved analytics discovery toward self-service by connecting a multi-LLM graph runtime with knowledge retrieval, governed Trino execution, answer reasoning, chart generation, prompt tracing, and a richer Open WebUI experience. The natural-language analytics path reduced query time by about 40% for supported workflows.
Business users needed a safer way to ask data questions, understand business definitions, execute governed queries, inspect answers, and create charts without routing every request through analysts or treating the model as the authorization layer.
My role covered both architecture and hands-on implementation across the full service topology: custom Open WebUI experience, multi-LLM agent runtime, graph subflows, knowledge system, query engine, platform control plane, and supporting services for identity, metadata, query execution, reasoning, and visualization.
System trace
How the work moved through the system
A high-level operating path: where the request starts, how the system
shapes it, and how other teams consume the result.
1
A customized Open WebUI experience captures the user question and supports richer data and chart interactions.
2
A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.
3
Structured state moves between graph nodes so the system remains deterministic, easier to trace, and less wasteful with tokens.
Ownership
Architect + developer
Designed and built the platform across agent runtime, query engine, knowledge system, service integration, and platform orchestration.
Agent architecture
Multi-LLM subgraphs
Specialized graph paths handle knowledge retrieval, business question answering, query generation, execution, answer reasoning, and visualization.
Governance principle
No raw data to LLM
The platform keeps enterprise data access behind governed execution paths instead of making the language model the data boundary.
User outcome
Text to data + charts
The product direction is natural-language analytics with governed query execution, answer reasoning, reusable result references, and visualization workflows.
Time-to-answer
~40% lower
For supported analytics workflows, the conversational path reduced query time by about 40% while preserving governed execution.
Architecture
System shape
8
1 A customized Open WebUI experience captures the user question and supports richer data and chart interactions.
2 A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.
3 Structured state moves between graph nodes so the system remains deterministic, easier to trace, and less wasteful with tokens.
4 A knowledge system grounds the agent using metadata, business definitions, schema relationships, query logs, joins, filters, and usage patterns.
5 Business-question subgraphs map user intent such as geography, manager, time period, metric, and comparison type into grounded query plans.
6 A governed query engine submits SQL through Trino with user identity, Ranger authorization, cancellation, guardrails, and Parquet-style result persistence.
7 Answer-reasoning and visualization agents consume governed result references and produce explanations or charts without turning raw data into prompt context.
8 Langfuse supports prompt management and tracing so prompts, graph behavior, and production flows can be inspected and improved.
Ownership
What I handled
7
1 Defined the architecture and service boundaries for the conversational data platform.
2 Implemented the core backend services around agents, subgraphs, query execution, knowledge context, reasoning, visualization, and platform integration.
3 Built structured graph-state flows between nodes to improve determinism and reduce avoidable token usage.
4 Integrated Langfuse for prompt management and tracing across the conversational workflows.
5 Customized Open WebUI with richer user-facing interactions for data exploration and chart workflows.
6 Designed the safety model around metadata grounding, authorization, auditability, result references, and execution guardrails.
7 Built the developer and platform workflow needed to run multiple services together with reproducible setup.
Lessons
What carried forward
4
1 Enterprise AI becomes useful when it respects existing data governance instead of bypassing it.
2 Conversational analytics needs metadata, authorization, and observability as much as it needs prompt quality.
3 A production data agent is a graph, state, tracing, and UX problem as much as it is an LLM problem.
4 The hard part is not generating SQL; it is making the full path explainable, cancellable, auditable, and safe for repeated use.
Engineering decisions
Keep business logic in governed data systems
The agent can help interpret intent, but KPI definitions, authorization, and query execution stay in governed platform layers such as dbt, Trino, and Apache Ranger-based controls.
Use metadata before data
The system grounds responses with catalogs, business definitions, schema relationships, query history, and usage patterns before attempting execution.
Model business questions as structured intent
Questions such as top manager by city, month-over-month trends, and quarterly growth comparisons need explicit metric, geography, time, and entity slots before SQL generation is useful.
Split the agent into subgraphs
Knowledge retrieval, business question answering, query generation, execution, answer reasoning, and visualization each need different prompts, tools, state contracts, and failure handling.
Carry structured state between nodes
Passing structured data between graph nodes made behavior easier to reason about, improved determinism, and avoided spending tokens restating context that the system already knew.
Trace prompts and graph behavior
Langfuse made prompt versions and graph execution observable, which matters when a business-facing data agent needs repeatable behavior instead of one-off demos.
Return references, not unrestricted data
Query results move through persisted references and controlled previews, which keeps downstream visualization useful without expanding the prompt surface unnecessarily.
Split agent reasoning from query execution
Separating the conversational runtime from the governed query engine made cancellation, guardrails, auditing, and authorization easier to reason about.
Treat the chat UI as product surface
Customizing Open WebUI improved the user path for data workflows because business users need richer controls than a plain chat transcript when moving from question to query to chart.
What can be shown
Public evidence without internal names
The internal systems stay private. This section keeps the public parts:
my role, system boundaries, technology context, scale, decisions,
constraints, and what I learned.
Internal enterprise system High-level architecture Scale signal Open-source reference
Ownership
Architect + developer
Designed the architecture and built the core services and components across the agent runtime, query engine, knowledge system, and platform orchestration pieces.
Agent shape
Multi-LLM subgraphs
The platform uses multiple language models and specialized graph subflows instead of one generic prompt path.
Safety model
No raw data to LLM
Language models reason over metadata, query plans, and bounded outputs. Enterprise data access remains behind governed execution paths.
User path
Text to data + charts
The system supports natural-language data questions and visualization workflows while keeping authorization and execution controls in the platform.
Analytics query time
~40% reduction
The natural-language path reduced time-to-answer for supported business analytics questions by about 40% compared with the older manual query path.
Query shape
Business questions
Example prompts include top-performing managers in Delhi, month-over-month gross-addition trends for Mumbai, and first-quarter manager growth comparisons.
Architecture shape
Business users ask through a customized Open WebUI experience with richer controls for data, charts, and follow-up workflows.
A multi-LLM graph runtime routes work through specialized subgraphs for knowledge retrieval, business question answering, query generation, execution, reasoning, and visualization.
Agents use metadata and knowledge services built from DataHub context, business definitions, SQL logs, schema relationships, and query patterns.
A governed query engine executes through Trino with user identity, Ranger authorization, cancellation, guardrails, and persisted result references.
The product handles business questions such as top-performing managers by geography, month-over-month gross-addition trends, and quarter-level growth comparisons.
Visualization and answer-reasoning flows work from governed result references rather than sending raw enterprise data into the language model.
Responsibilities
Designed the platform architecture for governed conversational analytics.
Built the multi-LLM agent runtime, specialized subgraphs, query execution path, knowledge system, control-plane integration, and service orchestration pieces.
Connected Trino, Apache Ranger-based authorization, DataHub metadata, Neo4j-backed knowledge context, Langfuse prompt management and tracing, and visualization workflows.
Used structured data flow between graph nodes to keep behavior deterministic and reduce unnecessary token usage.
Customized the Open WebUI experience with richer interaction patterns for business users working with data and charts.
Translated the internal implementation into a case study focused on ownership, topology, design decisions, and constraints.
Constraints
Internal product names, dataset names, business metrics, user groups, prompts, traces, and production screenshots are not published.
This page can describe architecture topology, technology choices, agent patterns, safety model, ownership scope, and implementation responsibilities.
Supporting context
High-level architecture
Governed conversational analytics topology
A shareable topology can show the custom conversation UI, multi-LLM graph runtime, specialized subgraphs, metadata and knowledge services, governed query engine, Trino/Ranger execution, persisted results, reasoning, and visualization outputs.
Open-source reference
Technology references
The shareable technology context includes Trino, Apache Ranger-based authorization, DataHub, Neo4j, LangGraph-style orchestration, Langfuse prompt management and tracing, MCP-style tools, dbt, and Open WebUI customization.
Related case studies
Continue through related work or return to the full project
index.
Built browsing-log ingestion and analytics pipelines for safe-browsing classification, audience management, cohort creation, and pattern-based downstream data products.