<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Knowledge-Base on twonines</title><link>https://twonines.codeberg.page/tags/knowledge-base/</link><description>Recent content in Knowledge-Base on twonines</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Wed, 08 Jul 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://twonines.codeberg.page/tags/knowledge-base/index.xml" rel="self" type="application/rss+xml"/><item><title>Building an Adversarial Knowledge Base for AI Agents</title><link>https://twonines.codeberg.page/posts/building-adversarial-knowledge-base/</link><pubDate>Wed, 08 Jul 2026 00:00:00 +0000</pubDate><guid>https://twonines.codeberg.page/posts/building-adversarial-knowledge-base/</guid><description>How we built @twonines/fact-store — a swamp extension that gives AI agents persistent, adversarially-reviewed operational memory.</description><content:encoded><![CDATA[<p>Every time an AI agent touches your infrastructure, it starts from scratch. It reads files, searches repos, follows reference chains, and then the next agent does it all over again. The operational knowledge evaporates between sessions, leaving no shared understanding.</p>
<p>We built <code>@twonines/fact-store</code> to fix that. It captures observation as typed, versioned, queryable data. Agents observe systems, record what they find with cited evidence, and the store makes those observations durable and searchable for every agent that follows.</p>
<p>This is the memory layer of an observe-don&rsquo;t-declare approach: instead of manually authoring documentation that drifts from reality, agents continuously observe the real state of systems and record what they find. The fact-store holds those observations, tracks their evidence strength, and serves them back at the moment they&rsquo;re needed.</p>
<h2 id="the-problem-repeated-discovery">The Problem: Repeated Discovery</h2>
<p>Consider a common scenario: an engineer asks an AI agent &ldquo;what do you know about how we deploy EKS clusters?&rdquo; The agent has to:</p>
<ol>
<li>Find the right repos</li>
<li>Search through CI configs, Terraform files, helm charts</li>
<li>Follow cross-repo references (this repo deploys to <em>that</em> account using <em>this</em> tool from <em>another</em> repo)</li>
<li>Piece together the full picture</li>
</ol>
<p>This takes minutes. And the next time someone asks a related question, or the same question in a different session, the entire process repeats.</p>
<h2 id="the-solution-discover--review--consume">The Solution: Discover → Review → Consume</h2>
<p><code>@twonines/fact-store</code> is a <a href="https://swamp-club.com">swamp</a> extension that implements a curated knowledge base with a three-role adversarial lifecycle:</p>
<h3 id="the-discoverer-internally-ferret">The Discoverer (internally: &ldquo;ferret&rdquo;)</h3>
<p>A discovery agent searches repository indexes with hypothesis-driven queries. When it finds something operationally useful (how repos connect, what deploys where, what consumes what) it proposes a fact:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp model method run facts propose <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="nv">kind</span><span class="o">=</span>repository_deploys_to_account <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="nv">scope</span><span class="o">=</span>infra/eks <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="s1">&#39;subjectRef={&#34;refType&#34;:&#34;repository&#34;,&#34;identityKind&#34;:&#34;gitlab_path&#34;,&#34;identityValue&#34;:&#34;infra/eks&#34;}&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="s1">&#39;value={&#34;accountId&#34;:&#34;123456789012&#34;,&#34;mechanism&#34;:&#34;terraform with helm charts&#34;}&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="nv">authorityBasis</span><span class="o">=</span>file_is_the_mechanism <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="nv">proposedBy</span><span class="o">=</span>kiro-ferret <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="s1">&#39;evidence=[&#34;k8s_config&#34;,&#34;services/&#34;]&#39;</span> --json
</span></span></code></pre></div><p>Every proposal carries:</p>
<ul>
<li><strong>A typed kind</strong> specific enough to close the lookup (<code>repository_deploys_to_account</code>, not <code>uses_cloud</code>)</li>
<li><strong>An authority basis</strong> declaring how strong the evidence actually is, from &ldquo;I queried the live system&rdquo; down to &ldquo;I inferred this from indirect clues&rdquo;</li>
<li><strong>Cited evidence</strong> pointing at the specific files that support the claim</li>
</ul>
<h3 id="the-reviewer-internally-mole">The Reviewer (internally: &ldquo;mole&rdquo;)</h3>
<p>An independent reviewer agent verifies each proposal against the raw evidence. It defaults to skepticism. False activations pollute the store, but false rejections are cheap since the discoverer can re-propose.</p>
<p>The reviewer checks four things:</p>
<ol>
<li><strong>Authority basis honesty</strong> is <code>file_is_the_mechanism</code> actually earned? A <code>.gitlab-ci.yml</code> <em>is</em> the pipeline. A README <em>mentioning</em> the pipeline is not.</li>
<li><strong>Value accuracy</strong> does the claimed value actually appear in the cited files?</li>
<li><strong>Specificity</strong> would someone still need to go look at the repo to understand what&rsquo;s happening?</li>
<li><strong>Deduplication</strong> does this add anything beyond what&rsquo;s already known?</li>
</ol>
<p>Rejections come with actionable feedback:</p>
<pre tabindex="0"><code>&#34;Account ID 123456789012 not found in .gitlab-ci.yml — only in
k8s_config which references external state. Authority basis should
be file_content_observation, not file_is_the_mechanism.&#34;
</code></pre><p>The discoverer can then re-propose with a corrected basis or better evidence.</p>
<h3 id="the-consumer">The Consumer</h3>
<p>Any agent about to do engineering work queries the store first:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp model method run facts query <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="nv">scope</span><span class="o">=</span>infra/eks <span class="se">\
</span></span></span><span class="line"><span class="cl">  --input <span class="s1">&#39;hints=[&#34;eks&#34;,&#34;cluster&#34;,&#34;deploy&#34;]&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  --json
</span></span></code></pre></div><p>And gets back a truth packet: all reviewed facts and constraints relevant to that scope, ranked by evidence tier.</p>
<p>The depth of what comes back depends on what&rsquo;s been discovered and reviewed. In our case, a single query about EKS deployment returned the provisioning toolchain, a full cluster topology table (eight clusters across four accounts with their environments and namespaces), workload deployment patterns, secrets management approach, and cluster add-ons. Facts originally scattered across dozens of repos, collected into a cohesive picture in a few seconds. The richer the fact-store grows, the less any agent has to rediscover.</p>
<p>Instead of minutes of repo exploration, the agent starts with verified operational knowledge and can immediately act on it.</p>
<h2 id="the-authority-tier-framework">The Authority Tier Framework</h2>
<p>Not all evidence is equal. The fact-store enforces honest assessment through six tiers:</p>
<table>
	<thead>
			<tr>
					<th>Tier</th>
					<th>Basis</th>
					<th>Meaning</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>0</td>
					<td><code>live_system_verification</code></td>
					<td>Queried the running system directly</td>
			</tr>
			<tr>
					<td>1</td>
					<td><code>file_is_the_mechanism</code></td>
					<td>A system enforces behavior by reading this file</td>
			</tr>
			<tr>
					<td>2</td>
					<td><code>file_content_observation</code></td>
					<td>File references external state that could be stale</td>
			</tr>
			<tr>
					<td>3a</td>
					<td><code>human_claim_in_file</code></td>
					<td>A human&rsquo;s claim recorded in a file</td>
			</tr>
			<tr>
					<td>3b</td>
					<td><code>human_claim_in_ticket</code></td>
					<td>A human&rsquo;s claim recorded in a ticket</td>
			</tr>
			<tr>
					<td>4</td>
					<td><code>agent_inference</code></td>
					<td>Logical conclusion from indirect evidence</td>
			</tr>
	</tbody>
</table>
<p>The same file can be different tiers for different claims. <code>.gitlab-ci.yml</code> is Tier 1 for &ldquo;this repo runs CI through GitLab&rdquo; but Tier 2 for &ldquo;this repo deploys to account 123456789012&rdquo; because the file references the account without creating it.</p>
<p>This matters because consumers can filter by tier. If you&rsquo;re about to modify infrastructure, you probably want Tier 0-1 facts. If you&rsquo;re onboarding to a new system, Tier 2-3 is fine.</p>
<h2 id="the-export-closing-the-loop">The Export: Closing the Loop</h2>
<p>After the reviewer activates facts, a workflow materializes them into a local SQLite database with full-text search and vector embeddings:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp workflow run refresh-fact-index
</span></span></code></pre></div><p>This writes <code>~/.jitter/facts.db</code>, a portable file that downstream tools can query with sub-100ms hybrid retrieval (BM25 via FTS5 + cosine similarity over embeddings). No network round-trip needed at query time.</p>
<h2 id="the-last-mile-injecting-facts-into-agent-context">The Last Mile: Injecting Facts into Agent Context</h2>
<p>The SQLite database is the interface. How you consume it is up to you. In our implementation, a Go tool called <code>jitter</code> handles the last mile. It does more than read rows from a table:</p>
<ol>
<li>A Kiro CLI hook fires at session start, passing the user&rsquo;s prompt to <code>jitter truth --query &quot;...&quot;</code></li>
<li>Jitter opens <code>~/.jitter/facts.db</code> and runs the same hybrid retrieval the fact-store uses internally: BM25 full-text search via FTS5 <em>and</em> cosine similarity over the pre-computed embedding vectors, fused with Reciprocal Rank Fusion (RRF) to combine both signals</li>
<li>It embeds the query on the fly against the same embedding endpoint used at index time, so semantic matches work even when the exact words differ</li>
<li>The top-ranked facts and all active constraints are assembled into a truth packet and emitted on stdout</li>
<li>The Kiro hook injects that truth packet into the agent&rsquo;s context as a user-prompt preamble</li>
</ol>
<p>The agent never calls the fact-store directly. It sees relevant operational knowledge pre-loaded in its context before it begins work, ranked by evidence strength. From the agent&rsquo;s perspective, it simply <em>knows things</em> about the system it&rsquo;s about to touch.</p>
<p>This is a deliberate separation of concerns: the fact-store manages truth, the export materializes it for fast access, and jitter performs real-time relevance retrieval at the moment it matters. Each layer is independently replaceable.</p>
<h2 id="what-makes-this-different">What Makes This Different</h2>
<p><strong>Adversarial by design.</strong> No single agent can pollute the store. The discoverer proposes, the reviewer challenges. Only evidence-backed facts survive.</p>
<p><strong>Honest about uncertainty.</strong> The authority tier framework prevents overstated confidence. A file that mentions an account ID is not the same as the live system confirming that account exists.</p>
<h2 id="getting-started">Getting Started</h2>
<p>Install the extensions:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp extension pull @twonines/fact-store
</span></span><span class="line"><span class="cl">swamp extension pull @twonines/repo-indexer
</span></span><span class="line"><span class="cl">swamp extension pull @twonines/fact-store-index
</span></span></code></pre></div><p>Create the model instances:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp model create @twonines/fact-store facts
</span></span><span class="line"><span class="cl">swamp model create @twonines/repo-indexer repo-indexer <span class="se">\
</span></span></span><span class="line"><span class="cl">  --global-arg <span class="nv">gitlabUrl</span><span class="o">=</span>https://gitlab.example.com <span class="se">\
</span></span></span><span class="line"><span class="cl">  --global-arg <span class="nv">gitlabToken</span><span class="o">=</span><span class="s1">&#39;${{ vault.get(&#34;secrets&#34;, &#34;gitlab-token&#34;) }}&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  --global-arg <span class="nv">embedUrl</span><span class="o">=</span>https://api.openai.com <span class="se">\
</span></span></span><span class="line"><span class="cl">  --global-arg <span class="nv">embedToken</span><span class="o">=</span><span class="s1">&#39;${{ vault.get(&#34;secrets&#34;, &#34;openai-key&#34;) }}&#39;</span>
</span></span></code></pre></div><p>Index some repos, then run a discovery pass using the shipped skill:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">swamp model method run repo-indexer index --input <span class="nv">projectPath</span><span class="o">=</span>myorg/myrepo
</span></span></code></pre></div><p>The <code>propose-facts</code> and <code>review-proposals</code> skills ship with the extension and load automatically. They guide agents through the full lifecycle.</p>
<h2 id="what-we-learned">What We Learned</h2>
<p>Building this taught us a few things:</p>
<ol>
<li>
<p><strong>Discovery without review is hallucination with citations.</strong> The adversarial step isn&rsquo;t overhead. It&rsquo;s what makes the output trustworthy.</p>
</li>
<li>
<p><strong>Evidence tiers matter more than confidence scores.</strong> An agent saying &ldquo;I&rsquo;m 95% sure&rdquo; means nothing. An agent saying &ldquo;this is Tier 1 because the file directly drives the behavior&rdquo; is verifiable.</p>
</li>
<li>
<p><strong>The feedback loop is the product.</strong> The discoverer reports what it couldn&rsquo;t do. The reviewer reports what was wrong. Both feed back into better tooling and better skills. The system improves itself.</p>
</li>
<li>
<p><strong>Exploration is expensive; storage is cheap.</strong> One well-evidenced fact saves minutes of discovery every time someone touches that system. The ROI compounds with every query.</p>
</li>
</ol>
]]></content:encoded></item></channel></rss>