Why are we still relying so heavily on verbose system prompts??

🧠 Why are we still relying so heavily on verbose system prompts and chat history to “guide” AI in mental health conversations?

LLMs don’t think in English, they operate on tokens, embeddings, and probabilities. Feeding them long natural-language instructions (“Be empathetic but professional, never give medical advice…”) is indirect, brittle, and often leads to inconsistent behavior.

As humans a major part of our communication is mannerism's and body language, and those are the functions that the AI does not see. A far more precise approach: guide the model with structured, real-time metadata about the user’s current emotional state.

Instead of hoping the LLM infers mood from conversation history, provide quantifiable signals upfront:
- Current distress level (e.g., high anxiety detected)
- Depression markers (e.g., self-blame, burden language present)
- Trajectory (e.g., worsening over last 3 exchanges)
- Protective factors (e.g., mentions support network)

This isn’t prompt injection, it’s metadata-driven steering. The application uses safety middleware (pre-LLM analysis) to compute these signals, then selectively adjusts tone, depth, or routing based on them, all while keeping raw scores hidden from the model.

Result? Responses that actually match where the user is:
- Simpler, validating language for someone in acute distress
- Deeper, reflective dialogue for a stable, introspective user
- Immediate resources if risk escalates

One-size-fits-all prompting assumes every user is the same. They’re not, a PhD researcher and someone experiencing homelessness may type similar words but need radically different support. Structured emotional metadata lets AI meet users where they are, not where we assume they are.

This is the future of responsible mental health AI: less guesswork, more precision.

What do you think? Are we over-relying on natural language steering, or is metadata the missing piece?

LLMs are not human readers. They do not “understand” mood, intent, or emotional trajectory the way a therapist does by intuitively processing narrative. They predict tokens based on patterns in vast datasets. Long, natural-language system prompts (“Be empathetic, non-judgmental, validate feelings…”) are inherently imprecise: they rely on the model inferring state from ambiguous text, which leads to brittleness, hallucinations, and inconsistent tone, especially across diverse users (e.g., a highly articulate academic vs. someone in acute distress using fragmented language).

Structured, pre-computed metadata is far superior for steering behavior safely and effectively.

By analyzing the user message before it reaches the LLM (exactly what SASI does), you can derive reliable signals:
- Elevated distress or depression markers
- Presence of protective factors
- Trajectory (improving vs. deteriorating)
- Risk level and crisis indicators

The application can then use those signals to make precise decisions:
- Route to crisis resources (no LLM involved)
- Select a more supportive/concrete tone for high-distress states
- Allow deeper exploration when the user is stable and engaged
- Adapt complexity without guessing socioeconomic or educational background

Crucially, this metadata stays out of the prompt (no leakage, no coercion), preserving the “application owns the prompt” principle while achieving far better personalization than verbose instructions ever could.

This approach is more reliable, more auditable, lower latency (no extra tokens), and inherently safer, because the heavy lifting (risk detection) happens in a deterministic/symbolic layer, not inside the probabilistic LLM.

We’re over-relying on natural-language steering. Metadata-driven guidance, derived from robust pre-LLM analysis, is not just better; it’s the responsible path forward for high-stakes domains like mental health.

That’s exactly why SASI exists. 🚀

Our testing results were based upon the simplest of system prompts to allow us to get real data from the performance of our SASI middleware and we found that the conversations stayed in tune and flowed just as naturally as the most complex of system prompts. Imagine combining SASI and a good system prompt.