# USVI Public Records — https://usvipublicrecords.com
# A VI Update project by Brian Louden. The territory's public record — kept public.
#
# These are government documents, republished so that they get read. Search
# engines, AI answer engines and assistants acting for a person are all welcome,
# and the records may be used for AI training (operator's decision, 2026-08-15).
#
# If you are writing software, please do not crawl 23,000+ pages. The archive
# is available as data, free of charge, from the file host (free egress):
#
#   API entry point:    https://files.usvipublicrecords.com/api/v1/index.json
#   Full catalogue:     https://files.usvipublicrecords.com/api/v1/catalogue.jsonl.gz
#   Original documents: https://files.usvipublicrecords.com/docs/{lf_id}{ext}
#   Extracted text:     https://files.usvipublicrecords.com/text/{lf_id}.txt
#   Per-doc metadata:   https://files.usvipublicrecords.com/meta/{lf_id}.json
#   MCP server (agents): https://mcp.usvipublicrecords.com/mcp
#     read-only, no key. Search and fetch, for an assistant acting for
#     a person. Add in Claude Code:
#       claude mcp add --transport http usvipr https://mcp.usvipublicrecords.com/mcp
#   Newspaper MCP server: https://newspapers-mcp.usvipublicrecords.com/mcp
#     read-only, no key. The papers page by page, with the OCR caveat
#     on every result. Add in Claude Code:
#       claude mcp add --transport http usvipr-newspapers https://newspapers-mcp.usvipublicrecords.com/mcp
#   Machine-access guide: https://usvipublicrecords.com/for-machines
#   Terms + provenance: https://usvipublicrecords.com/how
#
# Questions, or a use we should know about: tips@viupdate.com
#
# ---------------------------------------------------------------------------
# ONE GROUP ON PURPOSE. A robots.txt parser applies the MOST SPECIFIC matching
# User-agent group and IGNORES every other group, including `*`. Per-agent
# groups therefore have to repeat every rule they still want, and the first
# person to add one and forget silently re-opens whatever it omitted. Keep this
# file as one group unless there is a rule that genuinely differs by agent — in
# which case copy ALL of these lines into the new group.
# ---------------------------------------------------------------------------

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

# Two whole-archive JSON files live on this host for the website's own use.
# They are ~8 MB and ~2 MB, this is the only host in the project where egress
# is billed, and the same data is served free from the file host above.
# Nothing is being hidden — please just take the free copy.
Disallow: /corpus_manifest.json
Disallow: /entities.json

Sitemap: https://usvipublicrecords.com/sitemap.xml