Skip to content

h3nr7

Building guardrail with Jev for chat assistant h37

h37 is the assistant

h37 is a chat panel on every public page of h3nr7.com and iam.h3nr7.com. Ask it whether Henry can build a TypeScript web app, or what he's written about WebGL, and it streams back a short answer grounded in the public CV, the published articles and the recent activity feed, with links to them. When the record can't answer, or the visitor wants to hire or talk, it offers a button to email Henry or book a call. The page builds those links; the model never writes a URL for them.

Behind it is a service called brain: FastAPI and LangChain on one always-on machine on Fly.io. I'd hoped to run it as a Cloudflare Python Worker beside the website, but LangGraph doesn't install there (one of its dependencies has no Pyodide build).

The browser never talks to brain directly. The panel posts to /brain/messages on its own origin, and the site's Worker checks the body, adds a shared secret and forwards it. The reply streams back as Server-Sent Events, read with fetch rather than EventSource, which can only send GET requests and reconnects by itself: here, that would mean asking the same paid question twice.

Every 15 minutes brain rebuilds what it knows from the site's content API: a public summary for every prompt, the article bodies its tools can fetch, and a private layer from a secret-guarded endpoint, built per job from the full CV. That private layer is the reason most of this design exists.

Meet Jev

Jev is a decision model from TypeSafe. You don't chat with it. You give it a state, a JSON object describing the situation, plus a set of typed questions about that state, and it answers each one with a probability rather than prose. h37 uses three of its question types:

  • Noul: a yes-or-no question, with a sentence saying what yes means and one saying what no means. The answer is the probability of yes.

  • Score: a graded scale, such as four levels of harm. The answer is a number on that scale.

  • Choice: pick one of up to 255 options. The answer is a probability for each.

Here's one of h37's questions, exactly as brain asks it:

about_henry: Is question about Henry or his work?

Yes: It asks about Henry, or about something he has done, built, written or worked on.

No: It asks about something unconnected to Henry, such as general coding help, the news or a recipe.

Three properties made Jev the right tool for this job. It's fast: in my logs a request takes 130 to 320 ms. It's cheap: input costs $0.042 per million tokens, and output is free. And it can't be talked into writing anything. Its output is a set of numbers, so there's nothing for a clever question to hijack. The numbers go to code, and the code is a table I can read, test and change.

1. Ask about the question before the model sees it

Every message gets one Jev request before any LLM call. The state holds a sentence saying whose site this is, the visitor's earlier questions, the new question, and a catalogue of the private layer. Nine questions are asked about it at once: a guardrail battery with a severity score, whether it's probing for private details, whether it's about Henry at all, what it wants to know, and two that choose private chunks (Idea 2).

Then code decides, top to bottom, and the first rule that holds wins:

  • a jailbreak, an instruction override or a harmful request at 0.70 or above: blocked;

  • probing for private details at 0.70 or above: blocked, with the email button;

  • severity of 2 or more on the 0 to 3 scale: blocked;

  • about Henry below 0.35, unless it's about the site itself: off topic;

  • otherwise the agent answers.

2. Let the gate choose what the model may know

The private layer is about three dozen short chunks: one per job (the richest version of each role description), one per profile summary and one per skill group. Putting all of it in every prompt would be simple, and it's the wrong default. A model can only leak what it was given, so give it only what this question needs.

3. Check every paragraph before it leaves

Guard 1 decides whether to answer. Guard 2 decides whether what was written can be shown.

Each paragraph faces two checks. First, a string scan: a paragraph that repeats, word for word, a sentence found only in the private chunks is stopped without asking anyone. That catches a quote but not a paraphrase, which is Jev's job.

Then one Jev request per paragraph, with the public record (the summary plus whatever the tools returned), the chosen private chunks, the question and the paragraph as its state. Five Nouls are asked, and the first at 0.70 or above stops it:

  • reveals_private: a specific detail that is in the private notes but not in the public record;

  • personal_contact: an email address, a phone number or a postal address;

  • names_referee: someone presented as a referee;

  • broke_policy: playing a character, discussing its instructions, or helping with something unrelated;

  • unsupported_claim: something about Henry that the sources don't support.

4. When a check says no, take it back cleanly

Stopping a paragraph is easy. Ending the turn well is harder, because the visitor may already have read the first two paragraphs of the answer.

5. Fail closed, and test the guards like code

A guard you can route around isn't a guard, and the easiest way around one is an outage. If Jev can't be asked (no key, an error after the SDK's retries, or no answer within ten seconds), the turn ends as an error, with the email and call buttons. A failed check is never read as a pass. That makes TypeSafe a dependency of every answer, which is the point.