> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gately.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Crawl Status

> Check the status and retrieve results of a crawl job

Get the current status and results of a crawl operation. For large crawls, results may be paginated.

### Path Parameters

<ParamField path="id" type="string" required>
  The ID of the crawl job to check
</ParamField>

### Response Fields

<ResponseField name="status" type="string">
  Current status of the crawl: `scraping`, `completed`, or `failed`
</ResponseField>

<ResponseField name="total" type="integer">
  Total number of pages attempted to crawl
</ResponseField>

<ResponseField name="completed" type="integer">
  Number of successfully crawled pages
</ResponseField>

<ResponseField name="creditsUsed" type="integer">
  Number of credits consumed by the crawl
</ResponseField>

<ResponseField name="expiresAt" type="string">
  ISO 8601 timestamp when the crawl data expires
</ResponseField>

<ResponseField name="next" type="string">
  URL to retrieve next batch of data (for paginated results)
</ResponseField>

<ResponseField name="data" type="array">
  <Expandable title="Show object structure">
    <ResponseField name="markdown" type="string">
      Markdown content of crawled pages
    </ResponseField>

    <ResponseField name="html" type="string">
      HTML content of crawled pages
    </ResponseField>

    <ResponseField name="links" type="array">
      List of discovered links
    </ResponseField>

    <ResponseField name="metadata" type="object">
      Page metadata including title, description, etc.
    </ResponseField>
  </Expandable>
</ResponseField>

### Example Request

```bash theme={null}
curl --request POST \
  --url https://crawl.taam.cloud/v1/web \
  --header 'Authorization: Bearer YOUR_API_KEY'
```

### Example Response

```json theme={null}
{
  "status": "completed",
  "total": 50,
  "completed": 48,
  "creditsUsed": 96,
  "expiresAt": "2024-03-20T10:00:00Z",
  "next": null,
  "data": [
    {
      "markdown": "# Page Title\nContent...",
      "html": "<h1>Page Title</h1><p>Content...</p>",
      "links": [
        "https://example.com/page1",
        "https://example.com/page2"
      ],
      "metadata": {
        "title": "Example Page",
        "description": "Page description",
        "language": "en",
        "sourceURL": "https://example.com/page",
        "statusCode": 200
      }
    }
  ]
}
```

### Status Codes

<AccordionGroup>
  <Accordion title="200 - Success" icon="check">
    Status retrieved successfully
  </Accordion>

  <Accordion title="402 - Payment Required" icon="credit-card">
    Insufficient credits
  </Accordion>

  <Accordion title="429 - Too Many Requests" icon="clock">
    Rate limit exceeded
  </Accordion>

  <Accordion title="500 - Server Error" icon="triangle-exclamation">
    Server error or crawl failed
  </Accordion>
</AccordionGroup>


## OpenAPI

````yaml POST /v1/web
openapi: 3.0.1
info:
  title: TaamCloud API
  description: >-
    A collection of AI services including chat, embeddings, reranking, and media
    generation
  version: 1.0.0
servers:
  - url: https://api.gately.ai
    description: Main API server
  - url: https://uploud.taam.cloud
    description: File Processing server
security:
  - bearerAuth: []
paths:
  /v1/web:
    post:
      tags:
        - Web
      summary: Web services API
      description: Unified endpoint for web scraping, crawling, mapping and AI search
      operationId: web
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/WebRequest'
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebResponse'
components:
  schemas:
    WebRequest:
      type: object
      required:
        - model
      properties:
        model:
          type: string
          enum:
            - scrape
            - crawl
            - map
            - taam-ai-search
            - crawl-status
          description: Type of web service to use
        params:
          type: object
          description: Parameters specific to the selected model
    WebResponse:
      type: object
      properties:
        id:
          type: string
          description: Unique identifier for the request
        object:
          type: string
          description: Type of completion (e.g., scrape.completion)
        created:
          type: integer
          description: Unix timestamp of when the request was created
        model:
          type: string
          description: Model used for the request
        data:
          type: object
          description: Model-specific response data
        usage:
          type: object
          properties:
            prompt_tokens:
              type: integer
            completion_tokens:
              type: integer
            total_tokens:
              type: integer
        system_fingerprint:
          type: string
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: Enter your API key prefixed with 'Bearer '

````