Skip to main content
POST
/
v1
/
web
Web services API
curl --request POST \
  --url https://api.gately.ai/v1/web \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "scrape",
  "params": {}
}
'
{
  "id": "<string>",
  "object": "<string>",
  "created": 123,
  "model": "<string>",
  "data": {},
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "total_tokens": 123
  },
  "system_fingerprint": "<string>"
}

Documentation Index

Fetch the complete documentation index at: https://docs.gately.ai/llms.txt

Use this file to discover all available pages before exploring further.

The Website Mapping service helps you discover all links and pages on a website. You can filter results using search queries and control the crawling behavior.

​
Required Parameters

​
url
string
required
The base URL to start mapping from

​
Optional Parameters

​
ignoreSitemap
boolean
default:"true"
Ignore the website sitemap when crawling
​
sitemapOnly
boolean
default:"false"
Only return links found in the website sitemap
​
includeSubdomains
boolean
default:"false"
Include subdomains of the website
​
search
string
Search query to filter mapped links. During Alpha, smart search is limited to 1000 results
​
limit
integer
default:"5000"
Maximum number of links to return (max: 5000)

​
Example Request

curl --request POST \
  --url https://crawl.taam.cloud/v1/web \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "https://example.com",
    "search": "blog",
    "ignoreSitemap": true,
    "includeSubdomains": false,
    "limit": 1000
  }'

​
Example Response

{
  "success": true,
  "links": [
    "https://example.com/blog/post-1",
    "https://example.com/blog/post-2",
    "https://example.com/blog/categories"
  ]
}

​
Status Codes

Links mapped successfully
Insufficient credits
Rate limit exceeded
Server error occurred during mapping

Authorizations

​
Authorization
string
header
required

Enter your API key prefixed with 'Bearer '

Body

application/json
​
model
enum<string>
required

Type of web service to use

Available options:
scrape,
crawl,
map,
taam-ai-search,
crawl-status
​
params
object

Parameters specific to the selected model

Response

200 - application/json

Successful response

​
id
string

Unique identifier for the request

​
object
string

Type of completion (e.g., scrape.completion)

​
created
integer

Unix timestamp of when the request was created

​
model
string

Model used for the request

​
data
object

Model-specific response data

​
usage
object
​
system_fingerprint
string