Resolving Conflicts with Standing Instructions

At a Glance

Dimension	Details
Customer	Bucket Space — Personal knowledge, bookmarking, and content capture platform
Core Challenge	Support questions involve conditional policies, plan tiers, and platform guides that naive vector search conflated
Solution Deployed	Basegent support agent integrated with live Sanity Context Knowledge Base via Model Context Protocol (MCP)
Technologies	`@basegent/react`, `@basegent/client`, Sanity Context MCP (`kb3NuTkXw21o`), Sanity Studio, Next.js 15
Key Architectural Win	Structured policy precedence + 2-tier hybrid retrieval (index summaries + native MCP search)
Verified Verification	Live GROQ policy endpoint, public MCP tool verification script, and live integration on `mybucket.space`

About Bucket Space

Bucket Space is a personal knowledge and "second brain" web application designed for developers, researchers, and creators. It allows users to capture links, articles, notes, files, and rich media from any device, automatically organizing items with AI tagging, summaries, daily audio catch-ups, and custom shortlinks.

Because Bucket interacts deeply with browser extensions, mobile share sheets, and cross-platform devices, user support inquiries are inherently technical and multi-faceted:

Setup & Device Integrations: Installing the official Apple iOS Share Sheet shortcut, setting up browser extensions, and configuring Raycast / Windows hotkeys.
Account & Entitlements: Differentiating between Free and Pro workspace tiers, storage quotas, and region-specific terms.
Billing & Retention Policies: Understanding refund eligibility windows, data retention terms, and cancellation timelines across vintage accounts.

The Problem: Why Naive Vector RAG Broke Down

When exploring AI support for Bucket, our first thought was standard Retrieval-Augmented Generation (RAG): slice documentation markdown into chunks, compute embeddings, and retrieve top-$k$ nearest neighbors.

In practice, this approach failed on three fundamental classes of support questions:

1. Temporal & Precedence Collisions

Bucket updated its customer refund policy for Pro users in the United States from an older 14-day window to a 30-day window in 2026. A vector similarity search on "Can I get a refund after 21 days?" finds both the legacy 14-day paragraph and the canonical 30-day paragraph with nearly identical cosine similarity. A standard LLM prompt either averages them out or hallucinates an incorrect denial.

2. Multi-Dimensional Applicability

Support rules depend on customer attributes:

Plan tier (Free vs Pro)
Geographic region (US vs EU)
Custom enterprise contracts vs standard self-serve

Vector chunks have no concept of conditional logic or schema validation. They cannot reliably evaluate: "Does this customer qualify under the Pro US rule or the legacy global rule?"

3. Entity Hallucinations on Generic Names

When a user asked "How do I save links into Bucket from my iPhone?", an ungrounded LLM frequently conflated "Bucket" with Amazon AWS S3 buckets or Google Cloud Storage, outputting fabricated CLI commands (aws s3 cp ...) and broken download links instead of the real product guide.

The Solution: Basegent + Sanity Context MCP

Instead of storing duplicate copies of documentation in an isolated vector database, we connected Basegent directly to Sanity Context MCP.

Sanity Context treats documentation as structured, verifiable content with built-in conflict resolution and an open Model Context Protocol (MCP) interface:

[ User in Bucket Space (mybucket.space) ]
                    │
   Signed Customer Token (Plan, Market, User ID)
                    ▼
     [ @basegent/react Chat Drawer ]
                    │
                    ▼  SSE / WebSocket
  [ Basegent Platform (basegent.space) ]
                    │
      ┌─────────────┴──────────────────────────┐
      │ Multi-Provider BYOI / BYOK Engine      │
      │ (Groq, Anthropic, OpenAI, Google)      │
      └─────────────┬──────────────────────────┘
                    │
        1. Discover Outline (/initial-context)
        2. Score Candidates (knowledge_base_search)
        3. Read Document (knowledge_base_read)
                    ▼
 [ Sanity Context MCP Endpoint (api.sanity.io) ]
                    │
  ┌─────────────────┴──────────────────────────┐
  │ Sanity Knowledge Base (kb3NuTkXw21o)       │
  │ - Structured supportPolicy Schemas         │
  │ - Compiled Standing Instructions           │
  │ - Verified Canonical Citations             │
  └────────────────────────────────────────────┘
                    │
                    ▼
[ Grounded Answer with Citations & 1-Click Human Escalation ]

Implementation Details

1. Modeling Structured Support Policies in Sanity

In Sanity Studio (project jk662cms, dataset production), we defined a typed schema for supportPolicy documents rather than dumping plain text:

// schemas/supportPolicy.ts
import { defineType, defineField } from "sanity";

export const supportPolicy = defineType({
  name: "supportPolicy",
  title: "Support Policy & Guide",
  type: "document",
  fields: [
    defineField({ name: "policyKey", type: "string", title: "Policy Key" }),
    defineField({ name: "title", type: "string", title: "Title" }),
    defineField({ name: "claim", type: "text", title: "Core Claim / Rule" }),
    defineField({
      name: "appliesTo",
      type: "object",
      fields: [
        { name: "plans", type: "array", of: [{ type: "string" }] },
        { name: "markets", type: "array", of: [{ type: "string" }] },
        { name: "productIds", type: "array", of: [{ type: "string" }] },
      ],
    }),
    defineField({ name: "effectiveFrom", type: "datetime" }),
    defineField({ name: "effectiveUntil", type: "datetime" }),
    defineField({ name: "priority", type: "number", title: "Precedence (0-100)" }),
    defineField({
      name: "authority",
      type: "string",
      options: { list: ["canonical", "legacy", "advisory"] },
    }),
    defineField({ name: "supersedes", type: "reference", to: [{ type: "supportPolicy" }] }),
    defineField({ name: "sourceUrl", type: "url", title: "Canonical Source URL" }),
  ],
});

(You can inspect the live structured documents directly via Sanity's public GROQ endpoint: [jk662cms.api.sanity.io/.../query/production?query=*[_type=="supportPolicy"]](https://jk662cms.api.sanity.io/v2024-01-01/data/query/production?query=[_type==%22supportPolicy%22])).)*

2. Resolving Policy Conflicts with Standing Instructions

When Sanity Context compiled the Basegent Support Policies Knowledge Base (kb3NuTkXw21o), its analysis engine flagged overlapping rules in the Issues Review dashboard:

Conflict: The canonical 2026 Pro US policy specified a 30-day window (priority: 100, authority: canonical), whereas an older global entry specified 14 days (priority: 10, authority: legacy).
Resolution: Inside Sanity Context Lab, we selected the canonical source-backed claim. Sanity compiled this choice into a durable standing instruction that persists across future rebuilds without requiring manual code changes.

3. Connecting the Source Adapter in Basegent

Basegent connects to Sanity Context through a dedicated SanityContextSourceAdapter:

Outline Discovery: Queries /initial-context?mode=knowledge_base&knowledgeBases=kb3NuTkXw21o to discover all virtual outline entries.
Live Tool Execution: Issues JSON-RPC tool calls (knowledge_base_read) to fetch exact entry text over HTTP.
Security: The organization Context token is encrypted at rest in the tenant's workspace settings and is never exposed to the client browser.

4. Embedding the Support Assistant into Bucket Space

In Bucket's Next.js 15 App Router codebase, we integrated the @basegent/react client:

// components/shared/BasegentWidget.tsx
"use client";

import { useEffect, useState } from "react";
import { BasegentProvider, BasegentChat } from "@basegent/react";
import { useAuth } from "@/components/providers/auth-provider";

export function BasegentWidget() {
  const { user } = useAuth();
  const [customerToken, setCustomerToken] = useState<string | undefined>();

  useEffect(() => {
    if (!user || user.isAnonymous) return;

    // Fetch HMAC-signed customer token containing verified user attributes
    fetch("/api/support/basegent-token", { method: "POST" })
      .then((res) => res.json())
      .then((data) => setCustomerToken(data.token))
      .catch((err) => console.error("Could not sign Basegent token:", err));
  }, [user]);

  return (
    <BasegentProvider
      tenantId={process.env.NEXT_PUBLIC_BASEGENT_TENANT_ID!}
      apiBase="https://basegent.space"
      customerToken={customerToken}
    >
      <BasegentChat
        title="Bucket Support"
        placeholder="Ask about features, shortcuts, or refund policies..."
        className="bottom-20 sm:bottom-24"
        showFAB={false}
      />
    </BasegentProvider>
  );
}

Tracing a Real Support Interaction Step by Step

To see how the system operates under real conditions, consider this customer inquiry:

Customer Question: "I am on the Pro plan in the US and bought the standard product 21 days ago. Can I still request a refund, and how long will it take?"

Here is the exact runtime trace:

1. Customer Message Arrives
   ├── Verified Token: { plan: "pro", market: "US", signupDate: "2026-09-17" }
   └── Purchase Age: 21 days

2. Basegent Retrieval Strategy
   ├── Discovers Outline via /initial-context
   ├── Calls knowledge_base_search("refund eligibility processing")
   └── Reads entries:
       ├── refund_eligibility/standard_windows
       └── refund_processing

3. Policy Reasoning
   ├── Evaluates canonical rule: Pro US = 30 days (supersedes legacy 14-day rule)
   ├── Evaluates customer age: 21 days <= 30 days -> Eligible
   └── Distinguishes fulfillment: 5-7 business days processing timeline

4. Response Assembled & Streamed
   ├── Confirms eligibility with explicit citation back to Bucket settings
   ├── Clarifies that fulfillment takes 5-7 business days
   └── Provides 1-click button to escalate to human support inbox

What Broke in Production: 3 Real Postmortems

No production deployment goes smoothly on day one. During implementation and staging tests, we encountered three significant engineering failures:

Postmortem 1: The Bare Outline Retrieval Trap

The Failure: When Basegent first queried Sanity's /initial-context, the outline returned concise top-level slugs:
```
content_saving/desktop_and_mobile_methods [core]
plans_and_markets [core]
product_overview
refund_policy [core]
```
When a user asked "How do I set up the iOS shortcut to bookmark links from Safari?", our routing model evaluated the bare slug desktop_and_mobile_methods. Because the slug text didn't contain "iOS" or "shortcut", the router chose product_overview. Finding no setup steps, the agent replied: "I don't have documentation for an iOS shortcut."
The Root Cause: High-level outline slugs lacked sufficient semantic granularity to route specific feature keywords.
The Fix: We implemented a two-tier hybrid retrieval strategy:
1. We cached rich index metadata containing entry titles and leading summary paragraphs.
2. We hooked into Sanity's native knowledge_base_search MCP tool at query time. For the iOS query, Sanity's search scored desktop_and_mobile_methods at 13.55, ensuring it was included in the candidate pool. The agent now returns the authentic Apple iCloud shortcut link (https://www.icloud.com/shortcuts/57316fdc574b...) every time.

Postmortem 2: The Conversational Escalation Loop

The Failure: In @basegent/react, detecting intent keywords like "refund" prompted the user with:

"Would you like to speak with a human agent? [Yes, connect me] [No, ask the bot]" When a user clicked "No, ask the bot", the message was re-sent through the standard send() pipeline, which promptly re-triggered the same keyword detection—trapping the user in an infinite confirmation loop and generating duplicate chat bubbles.
The Root Cause: The chat state machine lacked a flag to differentiate an initial user query from a declined escalation retry.
The Fix: In @basegent/react (v0.2.9), we introduced a bypassKeywordCheck flag on re-submissions, cleared active escalation prompts upon conversation reset, and prevented duplicate user bubbles.

Postmortem 3: Multi-Provider Empty Streams

The Failure: Basegent uses a multi-provider BYOK architecture across Groq, OpenAI, Anthropic, and Google. Occasionally, token streams terminated prematurely with zero tokens emitted due to transient network resets or reasoning-model token packaging variations.
The Root Cause: Streaming pipelines that assume every open connection will successfully yield tokens leave users staring at blank loading spinners.
The Fix: Basegent's execution engine now detects zero-token stream terminations, transparently falls back to non-streaming complete text generation (generateText) across candidate models and API keys, and logs the event to the admin Review Queue for human audit.

Demonstrated Outcomes & Verified Capabilities

Rather than claiming unverified business metrics, here is what is verified and operational today:

Direct Verification of Content: All policy documents can be inspected live via Sanity's public GROQ API.
Reconciliation Without Code Changes: Policy precedence (e.g. 30-day Pro US vs legacy 14-day global) is resolved inside Sanity Context and persisted as durable standing instructions.
Verified Identity Grounding: Customer plan and market attributes are securely passed from Bucket's Next.js backend to Basegent via signed HMAC tokens.
Live Production Integration: The @basegent/react drawer is deployed and accessible on mybucket.space.
Human Escalation Fallback: When a question cannot be resolved or when the user requests a human operator, the session escalates into Basegent's unified team inbox with full conversation history and customer metadata.

Key Takeaways for Teams Building AI Support

Don't treat documentation as anonymous vector chunks: For business policies, structured fields (appliesTo, effectiveFrom, priority, authority) are essential to prevent temporal and entitlement hallucinations.
Outlines alone are not enough for routing: Combine high-level outline summaries with native knowledge-base search to capture deep keywords that aren't visible in slug names.
Plan for stateful escalation edge cases: A support widget is a state machine. When users decline human intervention or change their minds, ensure the conversation state resets cleanly.
Build multi-provider fallback into your streaming layer: Never let an empty or interrupted stream leave a customer hanging; always implement a non-streaming fallback and audit log.

Explore the Implementation

Live Application: Test the assistant on Bucket Space (mybucket.space).
Basegent Platform: Learn more about knowledge sources and BYOK workspaces on Basegent (basegent.space).
Technical Deep-Dive: Read the original engineering submission on DEV Community.
Source Packages: Inspect @basegent/react and @basegent/client on npm.