AI Research Stack
Two open-source tools, built together — from crawling the open web to publishing governed, auditable research.
ai-web-research
MITA deterministic, robots-aware AI web crawler & research agent
Crawls and monitors the web with legal and ethical guardrails built in from the start — reads and honors robots.txt, respects each site's Crawl-delay and rate limits, and blocks SSRF targets before every fetch. On top of that foundation: LLM-based semantic extraction with source-quote verification, a Divergence–Resonance–Compression research loop, and an MCP server exposing every capability to compatible agents.
git clone https://github.com/kakon77777-commits/ai-web-research
cd ai-web-research
uv venv && uv pip install -e .Veritaxa Workbench
MITThe first stable CHSA reference runtime for governed research
Takes evidence — from a crawler like ai-web-research, or any authorized source — through a governed pipeline: normalized Evidence IR, stable Evidence Spans, candidate Claims, governed Claim Review, evidence invalidation with dependency propagation, and deterministic, portable Research Bundles. Four boundaries hold throughout: Extraction ≠ Evidence, Evidence ≠ Verified Claim, Approved Claim ≠ Universal Truth, Recompilation ≠ Automatic Republication.
git clone https://github.com/kakon77777-commits/Veritaxa
cd Veritaxa
python release/v1.0/materialize.py --output-dir build --extract --clean
cd build/source/veritaxa_workbench_v1_0 && pip install -e . --no-build-isolationHow they connect
ai-web-research is the Source Adapter — legally-scoped, rate-limited fetching and extraction. Veritaxa is the governance layer on top of it — turning fetched evidence into reviewed, versioned, publishable research. Both speak the same Model Context Protocol, so they connect without custom glue code.
Use it legally.
This page provides access to AI crawling technology — that is not a license to scrape anything, anywhere, without limits. ai-web-research honors robots.txt and per-site rate limits by design, but lawful use is still your responsibility: check each target site's Terms of Service and robots policy, respect copyright and database rights, comply with the data-protection law that applies to you, and never use these tools to bypass access controls or overload a server you don't operate. When authorization is unclear, don't scrape — ask first.