GoGuides URL Request API

The URL Request capability lets a machine explicitly request GoGuides intelligence for a full public HTTP or HTTPS URL.

The caller does not write an authoritative GoGuides record directly. A request creates request intent. If the URL is unknown and safe, GoGuides attaches that intent to protected crawler work. When the normal GoGuides crawler successfully observes the target, GoGuides creates the authoritative observation and the request status becomes complete.

Capability: goguides.url_request
API version: url-request-v1
Data License: 2.0

Evaluate versus URL Request

GET /evaluate.php?url={absolute_url}&format=json is the exact-page evaluation surface. A known exact URL returns its stored page intelligence. For an eligible machine request, an unknown exact URL can generate asynchronous crawl demand for both the authoritative domain root and the requested exact page.

This Evaluate-generated demand is distinct from URL Request adoption. POST /request-url is the explicit attributed acquisition and status workflow, with request IDs and public lifecycle tracking.

Request

POST https://www.goguides.com/request-url
Content-Type: application/json

{
  "url": "https://example.com/article",
  "freshness": "use-current-or-crawl-on-miss"
}

Known URL

If the exact URL already has usable authoritative GoGuides intelligence, the endpoint returns HTTP 200 and does not create crawl work.

For URL Request responses, links.evaluate is the primary exact-page result and points to /evaluate.php?url={absolute_url}&format=json. The separate links.domain_evaluate field points to the authoritative root/domain Evaluate record. A page AI Rank therefore remains distinct from the site's root/domain AI Rank.

Unknown URL

A safe unknown URL may return HTTP 202 and a high-entropy gur_... request ID. Crawl work is deduplicated so multiple requests can share one crawler job.

Location: /request-url/status/gur_...
Retry-After: 5

Status

GET https://www.goguides.com/request-url/status/gur_...

Status polling is observational only. It does not create or promote crawl work.

Result surfaces

Surface	Purpose
Exact URL Evaluate	Exact-page intelligence. An eligible machine miss can schedule root + exact-page crawl demand.
Domain Evaluate	Authoritative root/domain evaluation and possible eligible machine discovery/freshness demand.
State	Current observed web state.
Changes	Observed change intelligence.
History	Domain provenance and observation history.

Safety

The request endpoint does not fetch target URLs synchronously. Crawl-on-miss uses the normal GoGuides crawler pipeline. URL validation rejects unsupported schemes, credentials, unsupported ports, and non-public network targets. Redirect destinations are revalidated by the crawler.

Identity

A User-Agent or Signature-Agent claim alone is never treated as verified identity. When Signature-Agent, Signature-Input, and Signature headers are supplied, GoGuides attempts cryptographic Web Bot Auth verification using RFC 9421 HTTP Message Signatures and discovered public verification keys. Successful verification establishes a cryptographically verified agent identity for internal attribution, quota, and priority decisions. Web Bot Auth is opportunistic: unsigned requests remain supported.