Search
Search Engine Results Page (SERP) API for querying search engines and retrieving structured results.
Introduction
AnyCrawl Search API enables you to programmatically access search engine results from multiple search engines. Built specifically for Large Language Models (LLMs), it returns clean, structured data that's optimized for AI processing and analysis.
Key Features: The API returns data immediately and synchronously - no polling or webhooks required. It supports multiple search engines and multi-page result retrieval for comprehensive data collection.
Core Features
- Multi-Engine Support: Currently supports Google search with plans for additional engines
- LLM Optimized: Returns structured JSON data perfect for AI processing
- Multi-Language Support: Search in different languages and regions
- Multi-Page Results: Retrieve multiple pages of search results in a single request
- High Performance: Native high concurrency support with asynchronous processing
- Immediate Response: Synchronous API - get results instantly without polling
- Comprehensive Results: Returns both search results and search suggestions
API Endpoint
POST https://api.anycrawl.dev/v1/search
Request Parameters
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
query | string | Yes | - | Search query string to be executed |
engine | enum | No | google | Search engine to use. Currently supported: google |
limit | number | No | 10 | Maximum number of results per page |
offset | number | No | 0 | Number of results to skip (for pagination) |
pages | number | No | 1 | Number of search result pages to retrieve (min: 1, max: 20) |
lang | string | No | en | Language locale for search results (e.g., 'en', 'zh', 'es', 'fr') |
safeSearch | number | null | No | null | Safe search filter level for Google: 0 (off), 1 (medium), 2 (high), or null (default). Only applicable to Google search engine. |
Supported Search Engines
google
(Default)
- Use Case: General web search with comprehensive results
- Advantages: Most comprehensive search results, supports multiple languages and regions
- Features: Returns both search results and search suggestions. Supports safe search filtering (off/medium/high)
- Recommended For: General web searches, research, content discovery
- Safe Search: Supports
safeSearch
parameter with values 0 (off), 1 (medium), 2 (high), or null (default)
Response Format
Success Response (HTTP 200)
Successful Search
{
"success": true,
"data": [
{
"title": "AlsoAsked: People Also Ask keyword research tool",
"url": "https://alsoasked.com/",
"description": "Find the questions people also ask. Enter a question, brand or search query. e.g. 'keyword research'.",
"source": "Google Search Result"
},
{
"title": "OpenAI ChatGPT Features and Benefits",
"url": "https://openai.com/chatgpt",
"description": "ChatGPT is an AI assistant that can help with writing, research, and problem-solving.",
"source": "Google Search Result"
},
{
"title": "Keyword tool",
"source": "Google Suggestions"
}
]
}
Search Result Types
The API returns two types of results:
- Search Results: Complete results with title, URL, description, and source
- Search Suggestions: Related query suggestions from the search engine
Error Responses
400 - Validation Error
{
"success": false,
"error": "Validation error",
"details": {
"issues": [
{
"field": "engine",
"message": "Invalid enum value. Expected 'google', received 'invalid'",
"code": "invalid_enum_value"
}
],
"messages": ["Invalid enum value. Expected 'google', received 'invalid'"]
}
}
401 - Authentication Error
{
"success": false,
"error": "Invalid API key"
}
402 - Payment Required
{
"success": false,
"error": "Insufficient credits. Please upgrade your plan or purchase more credits."
}
Usage Examples
cURL
Basic Search (Default Google Engine)
curl -X POST "https://api.anycrawl.dev/v1/search" \
-H "Authorization: Bearer <your-api-key>" \
-H "Content-Type: application/json" \
-d '{
"query": "OpenAI ChatGPT"
}'
Multi-Page Search
curl -X POST "https://api.anycrawl.dev/v1/search" \
-H "Authorization: Bearer <your-api-key>" \
-H "Content-Type: application/json" \
-d '{
"query": "machine learning tutorials",
"pages": 3,
"limit": 10
}'
Search with Language/Region Specification
curl -X POST "https://api.anycrawl.dev/v1/search" \
-H "Authorization: Bearer <your-api-key>" \
-H "Content-Type: application/json" \
-d '{
"query": "人工智能",
"lang": "zh",
"pages": 2
}'
Search with Pagination
curl -X POST "https://api.anycrawl.dev/v1/search" \
-H "Authorization: Bearer <your-api-key>" \
-H "Content-Type: application/json" \
-d '{
"query": "web scraping tools",
"offset": 20,
"limit": 10
}'
Search with Safe Search
curl -X POST "https://api.anycrawl.dev/v1/search" \
-H "Authorization: Bearer <your-api-key>" \
-H "Content-Type: application/json" \
-d '{
"query": "educational content",
"safeSearch": 0
}'
JavaScript/TypeScript
Basic Search
const response = await fetch("https://api.anycrawl.dev/v1/search", {
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({
query: "artificial intelligence trends",
engine: "google",
}),
});
const result = await response.json();
console.log(result.data); // Array of search results
Advanced Search with Multiple Pages
interface SearchOptions {
query: string;
engine?: "google";
pages?: number;
limit?: number;
lang?: string;
safeSearch?: number | null; // 0: off, 1: medium, 2: high, null: default (Google only)
}
async function searchWithPagination(options: SearchOptions) {
const response = await fetch("https://api.anycrawl.dev/v1/search", {
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({
query: options.query,
engine: options.engine || "google",
pages: options.pages || 1,
limit: options.limit || 10,
lang: options.lang || "en",
safeSearch: options.safeSearch,
}),
});
const result = await response.json();
if (result.success) {
return result.data.filter((item) => item.url); // Filter only results with URLs
} else {
throw new Error(result.error);
}
}
// Usage
const searchResults = await searchWithPagination({
query: "best web scraping APIs",
pages: 3,
limit: 10,
lang: "en",
});
Python
Basic Search
import requests
response = requests.post(
'https://api.anycrawl.dev/v1/search',
headers={
'Authorization': 'Bearer YOUR_API_KEY',
'Content-Type': 'application/json'
},
json={
'query': 'data science tools',
'engine': 'google'
}
)
result = response.json()
if result['success']:
for item in result['data']:
if 'url' in item: # Search result
print(f"Title: {item['title']}")
print(f"URL: {item['url']}")
print(f"Description: {item['description']}")
print("---")
else: # Search suggestion
print(f"Suggestion: {item['title']}")
else:
print(f"Search failed: {result.get('error', 'Unknown error')}")
Advanced Search with Error Handling
import requests
from typing import List, Dict, Any
def search_web(query: str, pages: int = 1, lang: str = "en") -> List[Dict[str, Any]]:
"""
Search the web using AnyCrawl Search API
Args:
query: Search query string
pages: Number of pages to retrieve
lang: Language locale
Returns:
List of search results
Raises:
Exception: If search fails
"""
try:
response = requests.post(
'https://api.anycrawl.dev/v1/search',
headers={
'Authorization': 'Bearer YOUR_API_KEY',
'Content-Type': 'application/json'
},
json={
'query': query,
'pages': pages,
'lang': lang,
'limit': 10
},
timeout=30
)
response.raise_for_status()
result = response.json()
if result['success']:
return result['data']
else:
raise Exception(f"Search failed: {result.get('error', 'Unknown error')}")
except requests.RequestException as e:
raise Exception(f"Request failed: {str(e)}")
# Usage
try:
results = search_web("machine learning frameworks", pages=2, lang="en")
print(f"Found {len(results)} results")
# Separate search results from suggestions
search_results = [r for r in results if 'url' in r]
suggestions = [r for r in results if 'url' not in r]
print(f"Search Results: {len(search_results)}")
print(f"Suggestions: {len(suggestions)}")
except Exception as e:
print(f"Error: {e}")
Best Practices
Search Engine Selection Guidelines
- General Web Search → Use
google
(currently the only supported engine) - Multi-language Searches → Specify appropriate
lang
parameter - Comprehensive Research → Use multiple pages to get more results
Performance Optimization
- Batch Requests: Use the
pages
parameter to get multiple pages in one request instead of making separate calls - Language Targeting: Specify language for region-specific and localized results
- Query Optimization: Use specific keywords and phrases for better result quality
Error Handling
try {
const response = await fetch("/v1/search", {
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({
query: "artificial intelligence",
pages: 2,
}),
});
const result = await response.json();
if (result.success) {
// Handle successful result
console.log("Found results:", result.data.length);
// Separate search results from suggestions
const searchResults = result.data.filter((item) => item.url);
const suggestions = result.data.filter((item) => !item.url);
console.log("Search Results:", searchResults.length);
console.log("Suggestions:", suggestions.length);
} else {
// Handle search failure
console.error("Search failed:", result.error);
}
} catch (error) {
// Handle network error
console.error("Request failed:", error);
}
High Concurrency Usage
The API natively supports high concurrency. You can make multiple simultaneous search requests without rate limiting concerns:
// Concurrent searching example
const queries = ["machine learning", "artificial intelligence", "data science"];
const searchPromises = queries.map((query) =>
fetch("/v1/search", {
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ query, engine: "google", pages: 2 }),
}).then((res) => res.json())
);
// All requests execute concurrently and return immediately
const results = await Promise.all(searchPromises);
console.log(`Completed ${results.length} searches simultaneously`);
Rate Limits and Credits
- Each search request consumes credits based on the number of pages requested
- Credit consumption = number of pages (e.g.,
pages: 3
= 3 credits) - No rate limiting for concurrent requests - the API natively supports high concurrency
- Monitor your credit usage through the API response headers
Frequently Asked Questions
Q: When should I use different language settings?
A:
- English (
en
): For global search results and international content - Chinese (
zh
): For Chinese-language content and China-focused results - Regional variants (e.g.,
en-GB
,fr-CA
): For country-specific results and localized content
Q: Why do some searches return fewer results than expected?
A: Possible reasons include:
- Search query is too specific or uses uncommon terms
- Language mismatch between query and
lang
parameter - Search engine limitations for certain topics or regions
- Network connectivity issues
Q: How to handle searches requiring specific regions?
A: Currently the API automatically selects appropriate regions based on the lang
parameter. Recommendations:
- Use language codes that match your target region (e.g.,
en-GB
for UK results) - Craft search queries with region-specific terms when needed
Q: What are the differences between search results and suggestions?
A:
- Search Results: Complete entries with title, URL, description, and source
- Search Suggestions: Related query suggestions without URLs, useful for query expansion
Q: Is there rate limiting for concurrent search requests?
A: No, the API natively supports high concurrency. You can make multiple simultaneous search requests without worrying about rate limits, and all requests return data immediately.
Language and Region Support
The Search API supports multiple languages and regions through the lang
parameter. Below are all supported locales:
Major Languages
all
- All languages
English 🌐
en
- Global Englishen-AU
- English (Australia) 🇦🇺en-CA
- English (Canada) 🇨🇦en-GB
- English (United Kingdom) 🇬🇧en-IE
- English (Ireland) 🇮🇪en-IN
- English (India) 🇮🇳en-NZ
- English (New Zealand) 🇳🇿en-PH
- English (Philippines) 🇵🇭en-PK
- English (Pakistan) 🇵🇰en-SG
- English (Singapore) 🇸🇬en-US
- English (United States) 🇺🇸en-ZA
- English (South Africa) 🇿🇦
Chinese 🌐
zh
- Global Chinesezh-CN
- Chinese (China) 🇨🇳zh-HK
- Chinese (Hong Kong) 🇭🇰zh-TW
- Chinese (Taiwan) 🇹🇼
Spanish 🌐
es
- Global Spanishes-AR
- Spanish (Argentina) 🇦🇷es-CL
- Spanish (Chile) 🇨🇱es-CO
- Spanish (Colombia) 🇨🇴es-ES
- Spanish (Spain) 🇪🇸es-MX
- Spanish (Mexico) 🇲🇽es-PE
- Spanish (Peru) 🇵🇪
French 🌐
fr
- Global Frenchfr-BE
- French (Belgium) 🇧🇪fr-CA
- French (Canada) 🇨🇦fr-CH
- French (Switzerland) 🇨🇭fr-FR
- French (France) 🇫🇷
German 🌐
de
- Global Germande-AT
- German (Austria) 🇦🇹de-BE
- German (Belgium) 🇧🇪de-CH
- German (Switzerland) 🇨🇭de-DE
- German (Germany) 🇩🇪
Portuguese 🌐
pt
- Global Portuguesept-BR
- Portuguese (Brazil) 🇧🇷pt-PT
- Portuguese (Portugal) 🇵🇹
Italian 🌐
it
- Global Italianit-CH
- Italian (Switzerland) 🇨🇭it-IT
- Italian (Italy) 🇮🇹
Japanese 🌐
ja
- Global Japaneseja-JP
- Japanese (Japan) 🇯🇵
Korean 🌐
ko
- Global Koreanko-KR
- Korean (South Korea) 🇰🇷
Russian 🌐
ru
- Global Russianru-RU
- Russian (Russia) 🇷🇺
European Languages
Dutch 🌐
nl
- Global Dutchnl-BE
- Dutch (Belgium) 🇧🇪nl-NL
- Dutch (Netherlands) 🇳🇱
Polish 🌐
pl
- Global Polishpl-PL
- Polish (Poland) 🇵🇱
Czech 🌐
cs
- Global Czechcs-CZ
- Czech (Czech Republic) 🇨🇿
Hungarian 🌐
hu
- Global Hungarianhu-HU
- Hungarian (Hungary) 🇭🇺
Romanian 🌐
ro
- Global Romanianro-RO
- Romanian (Romania) 🇷🇴
Greek 🌐
el
- Global Greekel-GR
- Greek (Greece) 🇬🇷
Bulgarian 🌐
bg
- Global Bulgarianbg-BG
- Bulgarian (Bulgaria) 🇧🇬
Croatian 🌐
hr
- Global Croatian
Slovak 🌐
sk
- Global Slovak
Slovenian 🌐
sl
- Global Slovenian
Estonian 🌐
et
- Global Estonianet-EE
- Estonian (Estonia) 🇪🇪
Latvian 🌐
lv
- Global Latvian
Lithuanian 🌐
lt
- Global Lithuanian
Finnish 🌐
fi
- Global Finnishfi-FI
- Finnish (Finland) 🇫🇮
Swedish 🌐
sv
- Global Swedishsv-SE
- Swedish (Sweden) 🇸🇪
Norwegian 🌐
nb
- Norwegian Bokmålnb-NO
- Norwegian Bokmål (Norway) 🇳🇴
Danish 🌐
da
- Global Danishda-DK
- Danish (Denmark) 🇩🇰
Icelandic 🌐
is
- Global Icelandic
Other Languages
Arabic 🌐
ar
- Global Arabicar-SA
- Arabic (Saudi Arabia) 🇸🇦
Hebrew 🇮🇱
he
- Hebrew
Turkish 🌐
tr
- Global Turkishtr-TR
- Turkish (Turkey) 🇹🇷
Persian 🌐
fa
- Persian
Hindi 🌐
hi
- Hindi
Urdu 🌐
ur
- Urdu
Thai 🌐
th
- Global Thaith-TH
- Thai (Thailand) 🇹🇭
Vietnamese 🌐
vi
- Global Vietnamesevi-VN
- Vietnamese (Vietnam) 🇻🇳
Indonesian 🌐
id
- Global Indonesianid-ID
- Indonesian (Indonesia) 🇮🇩
Indian Languages
kn
- Kannada 🌐ml
- Malayalam 🌐mr
- Marathi 🌐ta
- Tamil 🌐te
- Telugu 🌐
Other European Languages
af
- Afrikaans 🌐be
- Belarusian 🌐ca
- Catalan 🌐cy
- Welsh 🌐eu
- Basque 🌐ga
- Irish 🌐gd
- Scottish Gaelic 🌐gl
- Galician 🌐sq
- Albanian 🌐uk
- Ukrainian 🌐
Usage Notes
- Global codes (e.g.,
en
,zh
,es
) provide broader search results without country restrictions - Country-specific codes (e.g.,
en-US
,zh-CN
,es-MX
) return localized results for specific regions - The API automatically selects the appropriate Google domain and language settings based on your locale specification
- For best results, match the
lang
parameter to your search query's language