Why agents need a dedicated search API

An LLM's knowledge ends at its training cutoff. The moment a question involves "latest", "today", or "current pricing", the model can only improvise. Wiring an agent to the web sounds like calling a search box — until you actually try it.

Why fetching pages directly fails

The naive approach is to let the agent fetch full HTML for a URL. The problems show up immediately:

• Context explosion — a page of HTML is often hundreds of KB, with navigation, scripts, and ads drowning out the actual content.
• No ranking signal — search engines rank with hundreds of signals; raw fetching gives you no idea which result is trustworthy.
• Rendering and anti-bot walls — more sites require JS rendering or actively block plain fetches, returning empty shells.
• Unpredictable cost — tokens burned per call are impossible to budget.

What a search API for agents looks like

Search for agents is not search for humans. Humans want ten blue links; agents need structured evidence that drops straight into context:

{
  "query": "latest papers on retrieval",
  "results": [
    {
      "title": "Dense Passage Retrieval",
      "url": "https://arxiv.org/abs/2004.04906",
      "content": "DPR shows that dual-encoder retrievers...",
      "score": 0.95
    }
  ],
  "response_time": 0.42
}

Three things matter:

1. Content, not HTML — every result carries a cleaned snippet or body text the model can cite directly, no DOM parsing required.
2. Deterministic pricing — one call, one price (1 credit basic, 2 advanced), so agent frameworks can enforce budgets.
3. Predictable errors — 432 when credits run out, 429 on rate limits, always shaped as { "detail": { "error": "..." } } so retry logic is trivial.

Search vs Extract

Search answers "which pages exist". But often you already have the URL — a user pasted a link, or a previous search returned candidates. What you need then is point extraction: turn a specific URL into clean markdown.

WebSearch splits these into two endpoints:

• POST /api/search — query in, ranked results out
• POST /api/extract — URL list in, per-page content out (failed URLs listed separately, never billed)

Together they form the standard agent loop: search for candidates, extract the bodies, then let the model answer from real sources.

Next steps

New accounts get 1,000 credits. Try it in the Playground, or read the quickstart to make your first call with curl.