Third part in my Ai series with databases.

When building AI solutions within the database realm the first thing that people do is a straightforward concept, connect the LLM endpoint to a SQL backend let it run queries and become a LLM wrapper.  It works for demos, fails in the real world.

Why is this pattern bad? Couple of reasons – It generates adhoc SQL, the model decides what it wants to code and how to query and parse the data back.  It’s a context window killer. Do you know how heavy the RAW result sets on a msdb query are when you have hundreds of jobs? I have seen token budgets get eaten up very fast. Probably the worst thing without any form of control is the hallucination connection. The model, whether Claude or GPT will absolutely make a casual link between a failed job to a breached backup RPO even when there is no link – I have even asked it why it did that it replied, apologies I just linked the two.  Most importantly, it’s a security issue. Imagine you  are sending back and forth sensitive metadata about key objects within your database system its massive area of concern.

Defence Layers

Architecturally I design to four layers to address these problems to provide a secure confident experience, these are – Entry layer, Context layer, Tool layer and the validation layer.

Each layer is a guardrail that doesn’t depending on the mood of the LLM. The model can’t run dangerous code because it never writes TSQL. It cannot flood the context window because of tool contracts, and it cannot link 2 things together just because it feels like it due to validation layer.

Key design principle?  Guardrails must be structural – NOT behavioural.

The trade off is complexity, this isn’t a something you can design and build in couple of days but how do you build trust if you don’t? The effort is absolutely needed.