AI Research Stack
Two research tools, built together — from crawling the open web to publishing governed, auditable research.
ai-web-research
MITA deterministic, robots-aware AI web crawler & research agent
Crawls and monitors the web with legal and ethical guardrails built in from the start — reads and honors robots.txt, respects each site's Crawl-delay and rate limits, and blocks SSRF targets before every fetch. On top of that foundation: LLM-based semantic extraction with source-quote verification, a Divergence–Resonance–Compression research loop, and an MCP server exposing every capability to compatible agents.
Source access is currently private. No public installation package is available from this page.
Veritaxa Workbench
MITThe first stable CHSA reference runtime for governed research
Takes evidence — from a crawler like ai-web-research, or any authorized source — through a governed pipeline: normalized Evidence IR, stable Evidence Spans, candidate Claims, governed Claim Review, evidence invalidation with dependency propagation, and deterministic, portable Research Bundles. Four boundaries hold throughout: Extraction ≠ Evidence, Evidence ≠ Verified Claim, Approved Claim ≠ Universal Truth, Recompilation ≠ Automatic Republication.
Source access is currently private. No public installation package is available from this page.
How they connect
ai-web-research is the Source Adapter — legally-scoped, rate-limited fetching and extraction. Veritaxa is the governance layer on top of it — turning fetched evidence into reviewed, versioned, publishable research. Both speak the same Model Context Protocol, so they connect without custom glue code.
Use it legally.
This page describes AI crawling technology — that is not a license to scrape anything, anywhere, without limits. ai-web-research honors robots.txt and per-site rate limits by design, but lawful use is still your responsibility: check each target site's Terms of Service and robots policy, respect copyright and database rights, comply with the data-protection law that applies to you, and never use these tools to bypass access controls or overload a server you don't operate. When authorization is unclear, don't scrape — ask first.