> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gately.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Website Mapping

> Discover and map all links on a website

The Website Mapping service helps you discover all links and pages on a website. You can filter results using search queries and control the crawling behavior.

### Required Parameters

<ParamField body="url" type="string" required>
  The base URL to start mapping from
</ParamField>

### Optional Parameters

<AccordionGroup>
  <Accordion title="Crawl Settings" icon="gear">
    <ParamField body="ignoreSitemap" type="boolean" default="true">
      Ignore the website sitemap when crawling
    </ParamField>

    <ParamField body="sitemapOnly" type="boolean" default="false">
      Only return links found in the website sitemap
    </ParamField>

    <ParamField body="includeSubdomains" type="boolean" default="false">
      Include subdomains of the website
    </ParamField>
  </Accordion>

  <Accordion title="Search & Limits" icon="magnifying-glass">
    <ParamField body="search" type="string">
      Search query to filter mapped links. During Alpha, smart search is limited to 1000 results
    </ParamField>

    <ParamField body="limit" type="integer" default="5000">
      Maximum number of links to return (max: 5000)
    </ParamField>
  </Accordion>
</AccordionGroup>

### Example Request

```bash theme={null}
curl --request POST \
  --url https://crawl.taam.cloud/v1/web \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "https://example.com",
    "search": "blog",
    "ignoreSitemap": true,
    "includeSubdomains": false,
    "limit": 1000
  }'
```

### Example Response

```json theme={null}
{
  "success": true,
  "links": [
    "https://example.com/blog/post-1",
    "https://example.com/blog/post-2",
    "https://example.com/blog/categories"
  ]
}
```

### Status Codes

<AccordionGroup>
  <Accordion title="200 - Success" icon="check">
    Links mapped successfully
  </Accordion>

  <Accordion title="402 - Payment Required" icon="credit-card">
    Insufficient credits
  </Accordion>

  <Accordion title="429 - Too Many Requests" icon="clock">
    Rate limit exceeded
  </Accordion>

  <Accordion title="500 - Server Error" icon="triangle-exclamation">
    Server error occurred during mapping
  </Accordion>
</AccordionGroup>


## OpenAPI

````yaml POST /v1/web
openapi: 3.0.1
info:
  title: TaamCloud API
  description: >-
    A collection of AI services including chat, embeddings, reranking, and media
    generation
  version: 1.0.0
servers:
  - url: https://api.gately.ai
    description: Main API server
  - url: https://uploud.taam.cloud
    description: File Processing server
security:
  - bearerAuth: []
paths:
  /v1/web:
    post:
      tags:
        - Web
      summary: Web services API
      description: Unified endpoint for web scraping, crawling, mapping and AI search
      operationId: web
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/WebRequest'
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebResponse'
components:
  schemas:
    WebRequest:
      type: object
      required:
        - model
      properties:
        model:
          type: string
          enum:
            - scrape
            - crawl
            - map
            - taam-ai-search
            - crawl-status
          description: Type of web service to use
        params:
          type: object
          description: Parameters specific to the selected model
    WebResponse:
      type: object
      properties:
        id:
          type: string
          description: Unique identifier for the request
        object:
          type: string
          description: Type of completion (e.g., scrape.completion)
        created:
          type: integer
          description: Unix timestamp of when the request was created
        model:
          type: string
          description: Model used for the request
        data:
          type: object
          description: Model-specific response data
        usage:
          type: object
          properties:
            prompt_tokens:
              type: integer
            completion_tokens:
              type: integer
            total_tokens:
              type: integer
        system_fingerprint:
          type: string
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: Enter your API key prefixed with 'Bearer '

````