How Vex's tools work together

The agent loop in 5 lines

Vex's loop is conventional ReAct: classify intent, pick a model, run rounds where the model can call tools or finish with a reply. We didn't reinvent that. What's interesting is the guard rails: 6 grounded auto-fires that run before the model gets a chance to hallucinate, and 8 fallback paths that surface real data even when the model misfires.

Why "grounded auto-fires" exist

Small models (qwen2.5:3b, mistral:7b) confidently invent stock prices, weather numbers, and URLs. Telling them "use a tool" doesn't always work — they sometimes skip the tool and answer from training. We solved this by detecting the intent before calling the LLM and pre-fetching ground truth.

• Weather — wttr.in (no key) for any "weather in X" query
• News — Hacker News Algolia (no key) for "tech news / top stories"
• Search — SearXNG via pulse-search workflow for shopping/research
• Stocks — Yahoo Finance chart endpoint for tickers
• Crypto — CoinGecko free for BTC/ETH/SOL/etc
• Recipes — TheMealDB free for "recipe for X" / "how to make Y"
• Sports — TheSportsDB free for upcoming fixtures by team

Anti-hallucination digit-scan

Even after pre-fetching real data, the model sometimes produces text that ignoresthe fetched numbers. For prices, we scan the model's reply for any digit from the real price; if none match, we override with our own formatted summary. The user gets the real number every time.

const stocksIgnored = preStocks?.quotes?.length && (() => {
  for (const q of preStocks.quotes) {
    if (q.price != null) {
      const intPart = String(q.price).split('.')[0];
      if (raw.includes(intPart)) return false;
    }
  }
  return true; // model invented prices — override with our data
})();

URL stripping

Vex maintains an allowlist of known-good domains (komplett.no, prisjakt.no, bbcgoodfood.com, finance.yahoo.com, coingecko.com, themealdb.com, etc — 21 entries today). Any URL the model writes that isn't in the allowlist gets stripped. This catches things like the model inventing "alibaba.com/cheap-ram" — that link gets reduced to just the title text.

The tools, by category

• File ops: read_file, write_file, edit_file, list_dir, search_files
• Workspace: pulse_search_files, pulse_save_note, pulse_read_note
• Web: fetch_url, web_search, call_workflow (any n8n)
• Generation: make_pdf (pandoc + weasyprint, registers to Pulse Files)
• Data: pulse_calendar_events, daily_brief macro
• Comms: pulse_send_chat_message (autonomous Vex → user chat)
• Delegation: delegate_to_agent (10 specialist Pulse Agents)
• Shell: run_bash (sandboxed), execute_python
• Vision: analyze_image (llava with auto-fallback to qwen2.5:7b for tools)

Graceful recovery: when the model dies mid-loop

Ollama sometimes returns HTTP 400 mid-conversation (malformed JSON in tool args when the model gets overwhelmed by 30+ tool schemas). Instead of yielding an error to the user, we catch it and surface whatever ground truth we have:

1. chat_link from a successful tool (PDF, calendar, etc) → "Saved: [link]"
2. preNews hits → top 5 HN stories
3. preWeather → formatted Norwegian weather summary
4. preTool search hits → top 6 results
5. preStocks → formatted price block
6. preCrypto → formatted price block
7. preRecipe → top 3 recipes with ingredients
8. preSports → upcoming fixtures

Plus one more: a reduced-tools retry where we re-run the same model with only 4 critical tools instead of 30+. Often that's enough to get clean JSON.

Memory hygiene

Vex used to log every user message into Nebula (the personality store). PDF requests, weather queries, and "find me cheap RAM" all polluted the user's /app/memory page. We now have a 4-layer task-shape filter:

• Daemon: nebulaLogEpisode skip on task verbs
• Daemon: capture-endpoint skip
• Daemon: nebulaStore skip
• Server: autoCaptureFromChat short-circuit (defense in depth)

Plus a "Clean tasks" button on /app/memory that retroactively archives anything matching the same patterns.

What we got wrong

Lots. The PDF chat-link feature shipped with a 404 URL for two days because we didn't test the rendered link in a browser. Auto-pdf dropped file_id on the floor. The crypto pre-fire wrong-routed BTC to Yahoo Finance and returned a Bitcoin ETF stock at $35 instead of the real $79,800. Each one was a real bug, caught only by the user pushing back. The lesson: we can't trust the LLM's output to validate our own work — we need ground-truth tests.

Where this is going

More grounded auto-fires (translations, sports scores, stocks indices). A "fact card" UI block that renders grounded data as styled cards instead of prose. Better autoCapture rules so personality signals aren't drowned by task noise.

The thread that runs through all of it: verify with real bytes, not LLM claims.