Search

Search Engine Results Page (SERP) API for querying search engines and retrieving structured results.

Introduction

AnyCrawl Search API enables you to programmatically access search engine results from multiple search engines. Built specifically for Large Language Models (LLMs), it returns clean, structured data that's optimized for AI processing and analysis.

The API returns data immediately and synchronously - no polling or webhooks required. It supports multiple search engines and multi-page result retrieval for comprehensive data collection.

Core Features

Multi-Engine Support: Currently supports Google search with plans for additional engines
LLM Optimized: Returns structured JSON data perfect for AI processing
Multi-Language Support: Search in different languages and regions
Multi-Page Results: Retrieve multiple pages of search results in a single request
High Performance: Native high concurrency support with asynchronous processing
Immediate Response: Synchronous API - get results instantly without polling
Comprehensive Results: Returns both search results and search suggestions

API Endpoint

POST https://api.anycrawl.dev/v1/search

Usage Examples

Basic Search (Default Google Engine)

curl -X POST "https://api.anycrawl.dev/v1/search" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type": "application/json" \
  -d '{
    "query": "OpenAI ChatGPT"
  }'

Multi-Page Search

curl -X POST "https://api.anycrawl.dev/v1/search" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type": "application/json" \
  -d '{
    "query": "machine learning tutorials",
    "pages": 3,
    "limit": 10
  }'

Search with Language/Region Specification

curl -X POST "https://api.anycrawl.dev/v1/search" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type": "application/json" \
  -d '{
    "query": "AnyCrawl",
    "lang": "zh",
    "pages": 2
  }'

Search with Pagination

curl -X POST "https://api.anycrawl.dev/v1/search" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type": "application/json" \
  -d '{
    "query": "web scraping tools",
    "offset": 20,
    "limit": 10
  }'

Search with Safe Search

curl -X POST "https://api.anycrawl.dev/v1/search" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type": "application/json" \
  -d '{
    "query": "educational content",
    "safe_search": 0
  }'

Request Parameters

Parameter	Type	Required	Default	Description
`query`	string	Yes	-	Search query string to be executed
`engine`	enum	No	`google`	Search engine to use. Currently supported: `google`
`limit`	number	No	10	Maximum number of results per page
`offset`	number	No	0	Number of results to skip (for pagination)
`pages`	number	No	1	Number of search result pages to retrieve (min: 1, max: 20)
`lang`	string	No	`en`	Language locale for search results (e.g., 'en', 'zh', 'es', 'fr')
`safe_search`	number \| null	No	null	Safe search filter level for Google: `0` (off), `1` (medium), `2` (high), or null (default). Only applicable to Google search engine.

Optional: `scrape_options` for result URLs

When provided, each search result URL can be scraped with the given options (same fields as /v1/scrape, excluding top-level url). Note: browser-only options are ignored for non-browser scraping.

| Field | Type | Default | Notes | | ------------------- | ------------------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | engine | enum | - | Scrape engine for result URLs: cheerio, playwright, puppeteer | | formats | array of enum | ["markdown"] | Output formats: markdown, html, text, screenshot, screenshot@fullPage, rawHtml, json | | timeout | number | 60000 | Per-request timeout (ms) | | retry | boolean | false | Whether to retry on failure | | wait_for | number | - | Delay before extraction (ms); browser engines only; lower priority than wait_for_selector | | wait_for_selector | string, object, or array | - | Wait for one or multiple selectors (browser engines only). Accepts a CSS selector string, an object { selector: string, state?: "attached" | "visible" | "hidden" | "detached", timeout?: number }, or an array mixing strings/objects. Each entry is awaited sequentially. Takes priority over wait_for. | | include_tags | array of string | - | Only include elements that match CSS selectors | | exclude_tags | array of string | - | Exclude elements that match CSS selectors | | proxy | string (URI) | - | Optional proxy URL | | json_options | object | - | Options for structured JSON extraction (schema, user_prompt, schema_name, schema_description) | | extract_source | enum | markdown | Source to extract JSON from: markdown (default) or html |

Supported Search Engines

`google` (Default)

Use Case: General web search with comprehensive results
Advantages: Most comprehensive search results, supports multiple languages and regions
Features: Returns both search results and search suggestions. Supports safe search filtering (off/medium/high)
Recommended For: General web searches, research, content discovery
Safe Search: Supports safe_search parameter with values 0 (off), 1 (medium), 2 (high), or null (default)

Response Format

Success Response (HTTP 200)

Successful Search

{
    "success": true,
    "data": [
        {
            "title": "AlsoAsked: People Also Ask keyword research tool",
            "url": "https://alsoasked.com/",
            "description": "Find the questions people also ask. Enter a question, brand or search query. e.g. 'keyword research'.",
            "source": "Google Search Result"
        },
        {
            "title": "OpenAI ChatGPT Features and Benefits",
            "url": "https://openai.com/chatgpt",
            "description": "ChatGPT is an AI assistant that can help with writing, research, and problem-solving.",
            "source": "Google Search Result"
        },
        {
            "title": "Keyword tool",
            "source": "Google Suggestions"
        }
    ]
}

Search Result Types

The API returns two types of results:

Search Results: Complete results with title, URL, description, and source
Search Suggestions: Related query suggestions from the search engine

Error Responses

400 - Validation Error

{
    "success": false,
    "error": "Validation error",
    "details": {
        "issues": [
            {
                "field": "engine",
                "message": "Invalid enum value. Expected 'google', received 'invalid'",
                "code": "invalid_enum_value"
            }
        ],
        "messages": ["Invalid enum value. Expected 'google', received 'invalid'"]
    }
}

401 - Authentication Error

{
    "success": false,
    "error": "Invalid API key"
}

402 - Payment Required

{
    "success": false,
    "error": "Insufficient credits. Please upgrade your plan or purchase more credits."
}

Best Practices

Search Engine Selection Guidelines

General Web Search → Use google (currently the only supported engine)
Multi-language Searches → Specify appropriate lang parameter
Comprehensive Research → Use multiple pages to get more results

Performance Optimization

Batch Requests: Use the pages parameter to get multiple pages in one request instead of making separate calls
Language Targeting: Specify language for region-specific and localized results
Query Optimization: Use specific keywords and phrases for better result quality

Error Handling

try {
    const response = await fetch("/v1/search", {
        method: "POST",
        headers: {
            Authorization: "Bearer YOUR_API_KEY",
            "Content-Type": "application/json",
        },
        body: JSON.stringify({
            query: "artificial intelligence",
            pages: 2,
        }),
    });

    const result = await response.json();

    if (result.success) {
        // Handle successful result
        console.log("Found results:", result.data.length);

        // Separate search results from suggestions
        const searchResults = result.data.filter((item) => item.url);
        const suggestions = result.data.filter((item) => !item.url);

        console.log("Search Results:", searchResults.length);
        console.log("Suggestions:", suggestions.length);
    } else {
        // Handle search failure
        console.error("Search failed:", result.error);
    }
} catch (error) {
    // Handle network error
    console.error("Request failed:", error);
}

High Concurrency Usage

The API natively supports high concurrency. You can make multiple simultaneous search requests without rate limiting concerns:

// Concurrent searching example
const queries = ["machine learning", "artificial intelligence", "data science"];

const searchPromises = queries.map((query) =>
    fetch("/v1/search", {
        method: "POST",
        headers: {
            Authorization: "Bearer YOUR_API_KEY",
            "Content-Type": "application/json",
        },
        body: JSON.stringify({ query, engine: "google", pages: 2 }),
    }).then((res) => res.json())
);

// All requests execute concurrently and return immediately
const results = await Promise.all(searchPromises);
console.log(`Completed ${results.length} searches simultaneously`);

Rate Limits and Credits

Each search request consumes credits based on the number of pages requested
Credit consumption = number of pages (e.g., pages: 3 = 3 credits)
No rate limiting for concurrent requests - the API natively supports high concurrency
Monitor your credit usage through the API response headers

Frequently Asked Questions

Q: When should I use different language settings?

English (en): For global search results and international content
Chinese (zh): For Chinese-language content and China-focused results
Regional variants (e.g., en-GB, fr-CA): For country-specific results and localized content

Q: Why do some searches return fewer results than expected?

A: Possible reasons include:

Search query is too specific or uses uncommon terms
Language mismatch between query and lang parameter
Search engine limitations for certain topics or regions
Network connectivity issues

Q: How to handle searches requiring specific regions?

A: Currently the API automatically selects appropriate regions based on the lang parameter. Recommendations:

Use language codes that match your target region (e.g., en-GB for UK results)
Craft search queries with region-specific terms when needed

Q: What are the differences between search results and suggestions?

Search Results: Complete entries with title, URL, description, and source
Search Suggestions: Related query suggestions without URLs, useful for query expansion

all - All languages

English 🌐

en - Global English
en-AU - English (Australia) 🇦🇺
en-CA - English (Canada) 🇨🇦
en-GB - English (United Kingdom) 🇬🇧
en-IE - English (Ireland) 🇮🇪
en-IN - English (India) 🇮🇳
en-NZ - English (New Zealand) 🇳🇿
en-PH - English (Philippines) 🇵🇭
en-PK - English (Pakistan) 🇵🇰
en-SG - English (Singapore) 🇸🇬
en-US - English (United States) 🇺🇸
en-ZA - English (South Africa) 🇿🇦

Chinese 🌐

zh - Global Chinese
zh-CN - Chinese (China) 🇨🇳
zh-HK - Chinese (Hong Kong) 🇭🇰
zh-TW - Chinese (Taiwan) 🇹🇼

Spanish 🌐

es - Global Spanish
es-AR - Spanish (Argentina) 🇦🇷
es-CL - Spanish (Chile) 🇨🇱
es-CO - Spanish (Colombia) 🇨🇴
es-ES - Spanish (Spain) 🇪🇸
es-MX - Spanish (Mexico) 🇲🇽
es-PE - Spanish (Peru) 🇵🇪

French 🌐

fr - Global French
fr-BE - French (Belgium) 🇧🇪
fr-CA - French (Canada) 🇨🇦
fr-CH - French (Switzerland) 🇨🇭
fr-FR - French (France) 🇫🇷

German 🌐

de - Global German
de-AT - German (Austria) 🇦🇹
de-BE - German (Belgium) 🇧🇪
de-CH - German (Switzerland) 🇨🇭
de-DE - German (Germany) 🇩🇪

Portuguese 🌐

pt - Global Portuguese
pt-BR - Portuguese (Brazil) 🇧🇷
pt-PT - Portuguese (Portugal) 🇵🇹

Italian 🌐

it - Global Italian
it-CH - Italian (Switzerland) 🇨🇭
it-IT - Italian (Italy) 🇮🇹

Japanese 🌐

ja - Global Japanese
ja-JP - Japanese (Japan) 🇯🇵

Korean 🌐

ko - Global Korean
ko-KR - Korean (South Korea) 🇰🇷

Russian 🌐

ru - Global Russian
ru-RU - Russian (Russia) 🇷🇺

European Languages

Dutch 🌐

nl - Global Dutch
nl-BE - Dutch (Belgium) 🇧🇪
nl-NL - Dutch (Netherlands) 🇳🇱

Polish 🌐

pl - Global Polish
pl-PL - Polish (Poland) 🇵🇱

Czech 🌐

cs - Global Czech
cs-CZ - Czech (Czech Republic) 🇨🇿

Hungarian 🌐

hu - Global Hungarian
hu-HU - Hungarian (Hungary) 🇭🇺

Romanian 🌐

ro - Global Romanian
ro-RO - Romanian (Romania) 🇷🇴

Greek 🌐

el - Global Greek
el-GR - Greek (Greece) 🇬🇷

Bulgarian 🌐

bg - Global Bulgarian
bg-BG - Bulgarian (Bulgaria) 🇧🇬

Croatian 🌐

hr - Global Croatian

Slovak 🌐

sk - Global Slovak

Slovenian 🌐

sl - Global Slovenian

Estonian 🌐

et - Global Estonian
et-EE - Estonian (Estonia) 🇪🇪

Latvian 🌐

lv - Global Latvian

Lithuanian 🌐

lt - Global Lithuanian

Finnish 🌐

fi - Global Finnish
fi-FI - Finnish (Finland) 🇫🇮

Swedish 🌐

sv - Global Swedish
sv-SE - Swedish (Sweden) 🇸🇪

Norwegian 🌐

nb - Norwegian Bokmål
nb-NO - Norwegian Bokmål (Norway) 🇳🇴

Danish 🌐

da - Global Danish
da-DK - Danish (Denmark) 🇩🇰

Icelandic 🌐

is - Global Icelandic

Other Languages

Arabic 🌐

ar - Global Arabic
ar-SA - Arabic (Saudi Arabia) 🇸🇦

Hebrew 🇮🇱

he - Hebrew

Turkish 🌐

tr - Global Turkish
tr-TR - Turkish (Turkey) 🇹🇷

Persian 🌐

fa - Persian

Hindi 🌐

hi - Hindi

Urdu 🌐

ur - Urdu

Thai 🌐

th - Global Thai
th-TH - Thai (Thailand) 🇹🇭

Vietnamese 🌐

vi - Global Vietnamese
vi-VN - Vietnamese (Vietnam) 🇻🇳

Indonesian 🌐

id - Global Indonesian
id-ID - Indonesian (Indonesia) 🇮🇩

Indian Languages

kn - Kannada 🌐
ml - Malayalam 🌐
mr - Marathi 🌐
ta - Tamil 🌐
te - Telugu 🌐

Other European Languages

af - Afrikaans 🌐
be - Belarusian 🌐
ca - Catalan 🌐
cy - Welsh 🌐
eu - Basque 🌐
ga - Irish 🌐
gd - Scottish Gaelic 🌐
gl - Galician 🌐
sq - Albanian 🌐
uk - Ukrainian 🌐

Usage Notes

Global codes (e.g., en, zh, es) provide broader search results without country restrictions
Country-specific codes (e.g., en-US, zh-CN, es-MX) return localized results for specific regions
The API automatically selects the appropriate Google domain and language settings based on your locale specification
For best results, match the lang parameter to your search query's language

Search

On this page