PromptQL - Enforcing Agentic Reliability

Thoughts and excerpts post Agentic Summit BLR talk held on July 2, 2025

2025 is the year of Agents.

From copilots in enterprise workflows to autonomous agents navigating data lakes, we’re entering an era where decisions are made not by humans or traditional programs—but by models. Yet for all their promise, AI agents today struggle with a simple, unsexy problem: Reliability.

Hasura recently launched PromptQL, a completely new approach to tackling Agentic reliability. In continuation to Tiru's demo on stage at the Agentic Summit BLR specifically around building automations, this post unpacks the various AI architectures today, where they fall short, and how PromptQL’s unique approach of —Planning & Execution powered by an Agentic Semantic Layer—redefines what it means to build AI you can trust.

See the product demo given live at Agentic Summit BLR here:

Challenges in building Agentic Systems

There are a bunch of challenges that developers immediately run into as they start exploring building AI or Agentic systems for use-cases within their organisations:

1. Generalization Gap: LLMs haven’t seen your org’s data, your dashboards, your business logic - how can they make sense of it?
2. Data Messiness: Real-world data is inconsistent, semi-structured, and riddled with edge cases - current models are bound to fail
3. Non-Determinism: LLMs don’t behave like code. They generate different outputs even for the same inputs
4. Human Learning Curve: Prompting is a new developer skill—and brittle, its hard to get your prompts right without iteration

No, it's no longer about the Data

Data is the new oil - you've heard it being repeated many times over now. And like most things in life - it is partly true and partly not. When most people face the challenges above, they end up blaming others and start finger pointing, mostly alluding to one common thing: Data Quality.

But in our experience, also corroborated by the team at Hasura that's been playing with large quantums of data for a decade now, LLMs are smart enough to navigate data messiness owing to their semantic understanding capability. What then leads to this lack of reliable outputs if its not about challenges with Data quality?

Reliability Framework for Enterprise-grade AI

Agents leverage the ReAct (Reasoning and Acting) framework to combine their reasoning abilities with the capacity to take actions, enabling them to solve complex tasks and interact with the environment more effectively. While MCP and the likes are doing a great job of figuring out the "Acting" part of the framework, the first part - "Reasoning" - is starting to look like an orphaned child.

For AI to be truly Enterprise-grade, and to be able to work in production especially in mission-critical use-cases, we need systems that Reason while displaying these traits:

1. Predictable - Same input should give same output
2. Controllable - Should provide ability to direct the system towards the outcome
3. User-friendly - Easy to use and debug without technical skills requirement

This is the framework that we'll use to judge the various approaches taken towards building Agentic systems today.

Current Approaches

The current approaches towards building AI systems

1. RAG (Retrieval-Augmented Generation)

None of us are strangers to RAG, with its architectural pattern of relying on vector embeddings for retrieval of relevant context and providing it to the LLMs to fetch an answer.

Pattern: Search → Embed Context → Generate with LLM
Strengths: Grounded on data
Issues:
1. Search (cosine similarity) may return different contexts each time
2. Limited memory and context window
3. No clear interface for feedback or corrections, especially with humans' inability to read vector data

Evaluation:
Predictability: 🔴 Low
Controllability: 🔴 Low
User-Friendly: 🔴 Low

2. Tool Composition (MCP-style)

If you haven't explored MCP (Model Context Protocol) yet, you have definitely been living under the AI boulder. MCP standardises the way LLMs and (hence) agents can access data and tools exposed by providers and data stores.

Pattern: Multiple tool calls coordinated by an LLM
Strengths: Possible to stitch together multiple tool calls to get to a desired outcome, that too with a unified and standardised tool-calling framework
Issues:
1. Each step is handled by a non-deterministic LLM
2. Tool choice, order, and usage may vary run-to-run
3. Human correction is difficult mid-flow unless explicitly built in

Evaluation:
Predictability: 🔴 Low
Controllability: 🟡 Medium
User-Friendly: 🟡 Medium

3. Multi-Agent Orchestration

Pattern: Role-based agents working together or workflows built as directed acyclic graphs such as in LangGraph

Strengths: Possible to stitch together multiple tool calls to get to a desired outcome, that too with a unified and standardised tool-calling framework
Issues:
1. Faces the same non-determinism as MCP Tool Composition above
2. Adds further complexity in communication & coordination requiring technical expertise to troubleshoot

Evaluation:
Predictability: 🔴 Low
Controllability: 🟡 Medium
User-Friendly: 🔴 Low

4. Text-to-SQL Agents

Many vendors such as Defog but no clear winner yet

Pattern: User → LLM → SQL → DB
Strengths: SQL is interpretable and correctable - if output is wrong, analysts can inspect/fix it

Issues:
1. Needs technical ability to understand and parse SQL
2. Hard to embed business context unless done at the start
3. Doesn’t include agentic reasoning or sequencing across steps

Evaluation:
Predictability: 🟢 High
Controllability: 🟢 High
User-Friendly: 🔴 Low

Enter PromptQL

PromptQL is an AI framework built by Hasura that combines (1) a playground server iterating on a query plan using the power of LLMs and (2) a python runtime that executes the generated code over (3) a distributed query engine powered by Hasura's original DDN with (4) native data connectors to your internal and external data sources. In simpler words, think of it as a complex, iterative, text-2-sql engine that talks to your data.

The fundamental idea behind why PromptQL ends up with reliable AI is to be able to decouple Planning (which LLMs are great at, thanks to their reasoning capability over large amounts of data) and Execution (which LLMs suck at, and needs to be done deterministically every time). This is further augmented by bringing in an Agentic Semantic Layer (again driven by LLMs) that self improves its understanding of the metadata underneath for better Planning as more and more business context is fed in with usage.

🧠 Part 1: Planning

Create a Query Plan

Agents construct a plan in a domain-specific language (akin to SQL for agent tasks) known as PromptQL that helps decouple planning and execution.

The plan is interpretable, testable, and modular—separating logic from raw generation.

This reduces LLM involvement to only parts that require creativity or ambiguity.

🧭 Part 2: Execution

Execute Deterministically

The query plan is executed deterministically using a python runtime, ensuring consistent results across runs. This leverages the DDN that Hasura has mastered over a decade, stitching together databases and external data sources seamlessly.

Each step’s output is stored as an Artifact, creating traceability and reuse.

⚙ Part 3: Agentic Semantic Layer

Business Context to Metadata

PromptQL embeds metadata from the organization’s data systems—schemas, metrics, lineage, business rules into the query plan. As the user iterates with PromptQL, the system builds a semantic understanding of the metadata that it leverages as context for future outputs.

This provides the agent contextual grounding, tribal knowledge, and alignment with business semantics.

Benefits:

Predictability: 🟢 High. Plans are executed deterministically
Controllability: 🟢 High. Humans can edit plans, tweak DSL, or adjust the semantic layer
User-friendly: 🟡 Medium. Combines natural language flexibility with structured logic

PromptQL's approach towards data access clearly trumps the current approaches, especially in the ability to navigate LLMs towards finding the right answer to a user query. But there are few asks/unanswered questions that I would love for them to solve for over time -

(1) Supporting unstructured data, think multi-modal text/video/audio
(2) A more intuitive interface for iteration in the Planning phase
(3) Ability to take actions on behalf of the user (maybe via MCP integrations?)

Curious to learn more about PromptQL? See a quick summary here and read more in PromptQL's documentation:

Final Thoughts

Just like SQL abstracted databases for users, PromptQL abstracts AI agent complexity into something declarative, trustable, and powerful. But this time you don't need to learn a new syntax to get reliable answers.