ContentEngine API Endpoint Reference
Complete mapping of every DataForSEO and Google API endpoint required for the three-layer agentic pipeline (Strategy → Content → Production), organized by capability and agent consumer.
DataForSEO APIs
SERP API — feeds Search Landscape Agent
Base URL: https://api.dataforseo.com/v3/serp/google/organic/
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/serp/google/organic/task_post | POST | Queue SERP snapshot (Standard Queue, async) | $0.0006/task | Weekly snapshots for 100 keywords (400 tasks/month) |
/serp/google/organic/task_get/advanced | GET | Retrieve queued SERP results with full feature parsing | (included) | Pull results with ai_overview, featured_snippet, people_also_ask, local_pack items |
/serp/google/organic/live/advanced | POST | Real-time SERP snapshot (synchronous) | $0.002/task | Daily rank checks for top 20 keywords (600 tasks/month) |
Key request parameters:
1{
2 "keyword": "service mesh kubernetes",
3 "location_code": 2840,
4 "language_code": "en",
5 "device": "desktop",
6 "os": "windows",
7 "depth": 100
8}Key response fields to extract:
result.items[]— each SERP item withtype(organic, featured_snippet, ai_overview, people_also_ask, local_pack, video, etc.)result.items[].rank_group— position in SERPresult.items[].rank_absolute— absolute position including SERP featuresresult.items[].ai_overview_element— structured AI Overview with text blocks, references, expandable sectionsresult.items[].featured_snippet— snippet text, URL, typeresult.items[].people_also_ask— array of PAA questions with answersresult.spell— spelling corrections Google appliedresult.refinement_chips— related search filters
Estimated monthly cost: 1.20 (Live) = ~$2/month
Keywords Data API — feeds Content Strategy Agent
Base URL: https://api.dataforseo.com/v3/keywords_data/google_ads/
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/keywords_data/google_ads/search_volume/live | POST | Bulk search volume + CPC + competition + 12-month trend | $0.05/location per task (up to 1,000 keywords per task) | Monthly refresh of 100 primary keywords |
/keywords_data/google_ads/keywords_for_site/live | POST | Keyword suggestions based on a domain/URL | $0.05/task | Expansion queries from competitor domains |
/keywords_data/google_ads/keywords_for_keywords/live | POST | Related keyword suggestions from seed keywords | $0.05/task | Expansion from primary keyword list |
/keywords_data/google_ads/ad_traffic_by_keywords/live | POST | Traffic estimates and CPC forecasts | $0.05/task | CPC data for content ROI modeling |
Key response fields:
1{
2 "keyword": "service mesh",
3 "search_volume": 12100,
4 "competition": 0.34,
5 "competition_level": "MEDIUM",
6 "cpc": 4.82,
7 "monthly_searches": [
8 {"year": 2025, "month": 1, "search_volume": 14800},
9 {"year": 2025, "month": 2, "search_volume": 13200}
10 ]
11}Estimated monthly cost: ~$3/month (5–6 bulk tasks covering 500 keyword lookups)
DataForSEO Labs API — feeds Content Strategy Agent + Competitive Intelligence Agent
Base URL: https://api.dataforseo.com/v3/dataforseo_labs/google/
Keyword Analysis Endpoints
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/dataforseo_labs/google/keyword_overview/live | POST | KD, volume, CPC, SERP features, backlink stats for ranking pages | $0.001/keyword | Per-keyword scoring for strategic_alignment × search_opportunity |
/dataforseo_labs/google/bulk_keyword_difficulty/live | POST | Batch KD scores for up to 1,000 keywords | $0.0001/keyword | Bulk difficulty scoring for prioritization |
/dataforseo_labs/google/related_keywords/live | POST | Semantically related keywords with metrics | $0.003/keyword | Topic cluster expansion |
/dataforseo_labs/google/keyword_suggestions/live | POST | Keyword ideas from seed term | $0.003/keyword | Content gap discovery |
/dataforseo_labs/google/keyword_ideas/live | POST | Broader keyword ideas with search intent | $0.003/keyword | Calendar planning input |
/dataforseo_labs/google/search_intent/live | POST | Intent classification (informational, navigational, transactional, commercial) | $0.001/keyword | Intent field for scoring model |
/dataforseo_labs/google/keywords_for_categories/live | POST | Keywords by industry/topic category | $0.005/category | Niche discovery |
/dataforseo_labs/google/historical_search_volume/live | POST | Monthly search volume going back 5+ years | $0.005/keyword | Long-term trend analysis |
Competitive Intelligence Endpoints
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/dataforseo_labs/google/domain_intersection/live | POST | Keyword gap analysis — keywords competitors rank for that you don't | $0.004/row | Weekly gap analysis across 5 competitors (core competitive signal) |
/dataforseo_labs/google/competitors_domain/live | POST | Discover organic competitors for a domain | $0.001/row | Identify new competitors entering your keyword space |
/dataforseo_labs/google/ranked_keywords/live | POST | All keywords a domain/URL ranks for with positions | $0.00015/keyword | Per-page keyword profile for competitor pages |
/dataforseo_labs/google/relevant_pages/live | POST | Top pages by estimated organic traffic | $0.001/page | Competitor top content identification |
/dataforseo_labs/google/domain_rank_overview/live | POST | Domain-level metrics: organic traffic, keywords count, rank | $0.001/domain | Weekly competitor health check |
/dataforseo_labs/google/subdomains/live | POST | Subdomain breakdown with traffic estimates | $0.001/subdomain | Detect competitor content restructuring |
/dataforseo_labs/google/domain_metrics_by_categories/live | POST | Domain performance segmented by topic category | $0.001/domain | Category-level competitive positioning |
/dataforseo_labs/google/categories_for_domain/live | POST | Topic categories a domain ranks in | $0.001/domain | Competitive scope mapping |
/dataforseo_labs/google/page_intersection/live | POST | Page-level keyword overlap between specific URLs | $0.004/row | Head-to-head content comparison |
Key request for domain intersection (keyword gap):
1{
2 "targets": {
3 "1": "yourdomain.com",
4 "2": "competitor1.com",
5 "3": "competitor2.com"
6 },
7 "target1_intersection": false,
8 "intersections": {
9 "1": false,
10 "2": true,
11 "3": true
12 },
13 "filters": [
14 ["keyword_data.keyword_info.search_volume", ">", 100]
15 ],
16 "order_by": ["keyword_data.keyword_info.search_volume,desc"],
17 "limit": 1000
18}Key request for page-level ranked keywords:
1{
2 "target": "https://competitor.com/blog/specific-post",
3 "target_type": "url",
4 "location_code": 2840,
5 "language_code": "en",
6 "limit": 100,
7 "order_by": ["ranked_serp_element.serp_item.rank_group,asc"]
8}Estimated monthly cost: ~$12/month (keyword analysis + competitive intelligence combined)
Backlinks API — feeds Content Strategy Agent (authority scoring)
Base URL: https://api.dataforseo.com/v3/backlinks/
Requires: $100/month subscription (deposits to shared balance)
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/backlinks/summary/live | POST | Domain or URL-level authority summary | $0.002/target | Monthly snapshot of 11 domains (own + 10 competitors) |
/backlinks/referring_domains/live | POST | List of referring domains with metrics | $0.0004/row | Referring domain count and quality distribution |
/backlinks/anchors/live | POST | Anchor text distribution | $0.0004/row | Anchor text analysis for authority scoring |
/backlinks/history/live | POST | New/lost backlinks over time | $0.002/target | Link velocity monitoring |
/backlinks/backlinks/live | POST | Individual backlink list | $0.0002/row | Deep-dive when needed |
/backlinks/domain_pages/live | POST | Pages of a domain sorted by backlinks | $0.001/row | Find competitor's most-linked content |
/backlinks/domain_pages_summary/live | POST | Page-level authority metrics across a domain | $0.001/row | Page authority distribution analysis |
/backlinks/competitors/live | POST | Domains competing for same backlinks | $0.005/row | Link-building opportunity identification |
/backlinks/page_intersection/live | POST | Domains linking to competitors but not to you | $0.005/row | Link gap analysis |
/backlinks/bulk_referring_domains/live | POST | Bulk referring domain count for up to 1,000 targets | $0.002/target | Quick authority comparison across all competitors |
/backlinks/bulk_backlinks/live | POST | Bulk backlink count for up to 1,000 targets | $0.002/target | Quick link profile overview |
/backlinks/bulk_ranks/live | POST | Bulk InLink Rank for up to 1,000 targets | $0.002/target | Batch authority scoring |
/backlinks/bulk_new_lost_backlinks/live | POST | New/lost link counts in bulk | $0.002/target | Detect link velocity changes across competitors |
Key response fields from summary:
1{
2 "target": "competitor.com",
3 "external_links_count": 245000,
4 "referring_domains": 8200,
5 "referring_main_domains": 7800,
6 "rank": 487,
7 "main_domain_rank": 492,
8 "broken_backlinks": 120,
9 "referring_domains_nofollow": 1200,
10 "anchor": null,
11 "referring_links_tld": {"com": 5400, "org": 800, "net": 340},
12 "referring_links_platform_types": {"cms": 4200, "blogs": 1800}
13}Estimated monthly cost: ~100 commitment
AI Optimization APIs — feeds Search Landscape Agent + ai_overview LangExtract
LLM Mentions API — Requires $100/month subscription
Base URL: https://api.dataforseo.com/v3/dataforseo_labs/ai/
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/dataforseo_labs/ai/llm_mentions/live | POST | Pre-indexed brand mentions across AI platforms with citations, sentiment | $0.005/task | Track brand + competitor mentions across ChatGPT, Google AI Overviews |
/dataforseo_labs/ai/ai_keyword_data/live | POST | AI-specific search volumes and overview prevalence | $0.0002/keyword | Identify which keywords trigger AI Overviews |
/dataforseo_labs/ai/ranked_urls_in_ai/live | POST | URLs cited in AI-generated answers | $0.001/task | Track which of your pages get cited in AI responses |
LLM Direct Query APIs — No subscription required
Base URL: https://api.dataforseo.com/v3/serp/ai/
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/serp/ai/llm_scraper/task_post | POST | Scrape live responses from ChatGPT, Gemini | $0.0012/task | Direct AI response monitoring for specific prompts |
/serp/ai/llm_scraper/task_get | GET | Retrieve scraped LLM responses | (included) | Parse structured response with citations |
/serp/ai/llm_responses/task_post | POST | Submit prompts to multiple LLMs, get structured output | $0.0006/task + LLM cost | Cross-platform brand mention testing |
/serp/ai/llm_responses/task_get | GET | Retrieve LLM response results | (included) | Parse multi-LLM comparison data |
Key request for LLM Mentions:
1{
2 "keyword": "best service mesh for kubernetes",
3 "location_code": 2840,
4 "language_code": "en",
5 "filters": [
6 ["mention_type", "=", "brand"],
7 ["platform", "in", ["chatgpt", "google_ai_overview", "gemini"]]
8 ]
9}Key response fields:
1{
2 "keyword": "best service mesh for kubernetes",
3 "mention": {
4 "brand": "Consul",
5 "platform": "chatgpt",
6 "position_in_response": 2,
7 "citation_url": "https://consul.io/docs/...",
8 "sentiment": "positive",
9 "context_snippet": "...",
10 "share_of_voice": 0.15,
11 "competitors_mentioned": ["Istio", "Linkerd", "Envoy"]
12 }
13}Estimated monthly cost: ~100 LLM Mentions commitment
Google Trends API — feeds Content Strategy Agent (calendar planning)
Base URL: https://api.dataforseo.com/v3/keywords_data/google_trends/
| Endpoint | Method | Purpose | Cost | ContentEngine Use |
|---|---|---|---|---|
/keywords_data/google_trends/explore/live | POST | Interest over time (relative 0–100 scale) | $0.001/task | Seasonal pattern detection for editorial calendar |
/keywords_data/google_trends/explore/task_post | POST | Async version for batch processing | $0.001/task | Batch trend analysis for 100 keywords |
/keywords_data/google_trends/explore/task_get | GET | Retrieve async results | (included) | Pull trend data |
Key request parameters:
1{
2 "keywords": ["service mesh", "api gateway", "kubernetes networking"],
3 "location_code": 2840,
4 "language_code": "en",
5 "date_from": "2024-03-01",
6 "date_to": "2025-03-01",
7 "type": "web"
8}Estimated monthly cost: ~$5/month
Google APIs (Free)
Google Search Console API — feeds Refresh Queue + Post-Publish Pipeline
Base URL: https://www.googleapis.com/webmasters/v3/
Auth: OAuth 2.0 with webmasters.readonly scope
Rate limits: 1,200 queries/minute per site
Search Analytics
| Endpoint | Method | Purpose | Rate Limit | ContentEngine Use |
|---|---|---|---|---|
/sites/{siteUrl}/searchAnalytics/query | POST | Page-level performance data (impressions, clicks, CTR, position) | 1,200/min | Core performance data for Refresh Queue scoring |
Key request for page-level performance:
1{
2 "startDate": "2025-02-01",
3 "endDate": "2025-03-01",
4 "dimensions": ["page", "query", "date", "device"],
5 "type": "web",
6 "dataState": "all",
7 "rowLimit": 25000,
8 "startRow": 0,
9 "dimensionFilterGroups": [{
10 "filters": [{
11 "dimension": "page",
12 "operator": "contains",
13 "expression": "/blog/"
14 }]
15 }]
16}Key response fields:
1{
2 "rows": [{
3 "keys": ["https://yourdomain.com/blog/service-mesh-guide", "service mesh", "2025-02-15", "DESKTOP"],
4 "clicks": 45,
5 "impressions": 1200,
6 "ctr": 0.0375,
7 "position": 4.2
8 }]
9}Feeds these Refresh Queue signals:
- Position Decay (30% weight): Compare
positionacross 30-day windows - Traffic Decline (25% weight): Compare
clicksandimpressionsacross periods - Query-level attribution shows which keywords are losing position
URL Inspection API
| Endpoint | Method | Purpose | Rate Limit | ContentEngine Use |
|---|---|---|---|---|
/sites/{siteUrl}/urlInspection/index:inspect | POST | Index status, last crawl date, crawl state, mobile usability | 2,000/day per property | 24h post-publish index verification |
Key request:
1{
2 "inspectionUrl": "https://yourdomain.com/blog/new-post",
3 "siteUrl": "https://yourdomain.com/",
4 "languageCode": "en-US"
5}Key response fields:
1{
2 "inspectionResult": {
3 "indexStatusResult": {
4 "verdict": "PASS",
5 "coverageState": "Submitted and indexed",
6 "indexingState": "INDEXING_ALLOWED",
7 "lastCrawlTime": "2025-03-05T14:22:00Z",
8 "pageFetchState": "SUCCESSFUL",
9 "googleCanonical": "https://yourdomain.com/blog/new-post",
10 "robotsTxtState": "ALLOWED",
11 "crawledAs": "DESKTOP"
12 },
13 "mobileUsabilityResult": {
14 "verdict": "PASS"
15 }
16 }
17}Post-publish pipeline usage:
- 24h check: Call URL Inspection → verify
indexingState= "INDEXING_ALLOWED" andpageFetchState= "SUCCESSFUL" - 7d check: Call Search Analytics with
startDate= publish date → verify initial impressions appearing - 30d check: Call Search Analytics for full 30-day window → run Refresh Queue scoring
Sitemaps API
| Endpoint | Method | Purpose | ContentEngine Use |
|---|---|---|---|
GET /sites/{siteUrl}/sitemaps | GET | List submitted sitemaps | Verify sitemap health |
PUT /sites/{siteUrl}/sitemaps/{feedpath} | PUT | Submit/resubmit sitemap | Trigger re-crawl after content update |
Indexing API (separate from Search Console)
Base URL: https://indexing.googleapis.com/v3/
| Endpoint | Method | Purpose | Rate Limit | ContentEngine Use |
|---|---|---|---|---|
/urlNotifications:publish | POST | Notify Google of new/updated URLs | 200/day | Trigger immediate crawl after publish |
Key request:
1{
2 "url": "https://yourdomain.com/blog/new-post",
3 "type": "URL_UPDATED"
4}Note: Officially only for JobPosting and BroadcastEvent schema types, but empirically accelerates crawling for all page types. Use alongside sitemap submission for belt-and-suspenders indexation.
Google Analytics 4 Data API — feeds 30-day Performance Review
Base URL: https://analyticsdata.googleapis.com/v1beta/
Auth: OAuth 2.0 with analytics.readonly scope
Rate limits: 25,000 tokens/day per property (each request costs 1–10 tokens based on complexity)
| Endpoint | Method | Purpose | ContentEngine Use |
|---|---|---|---|
/properties/{propertyId}:runReport | POST | Custom report with dimensions + metrics | Page engagement data for performance review |
/properties/{propertyId}:batchRunReports | POST | Multiple reports in single call (up to 5) | Efficient batch data pull |
/properties/{propertyId}:runRealtimeReport | POST | Real-time active users (last 30 min) | Post-publish immediate traffic verification |
Key request for page-level engagement:
1{
2 "dateRanges": [{"startDate": "30daysAgo", "endDate": "today"}],
3 "dimensions": [
4 {"name": "pagePath"},
5 {"name": "sessionDefaultChannelGroup"}
6 ],
7 "metrics": [
8 {"name": "screenPageViews"},
9 {"name": "averageSessionDuration"},
10 {"name": "bounceRate"},
11 {"name": "engagedSessions"},
12 {"name": "scrolledUsers"},
13 {"name": "conversions"},
14 {"name": "totalUsers"},
15 {"name": "newUsers"}
16 ],
17 "dimensionFilter": {
18 "filter": {
19 "fieldName": "pagePath",
20 "stringFilter": {
21 "matchType": "CONTAINS",
22 "value": "/blog/"
23 }
24 }
25 },
26 "orderBys": [{"metric": {"metricName": "screenPageViews"}, "desc": true}],
27 "limit": 500
28}Key metrics for 30-day review:
screenPageViews— total page viewsaverageSessionDuration— time on page (engagement proxy)bounceRate— single-page session rateengagedSessions— sessions lasting >10s or with conversion/2+ pageviewsscrolledUsers— users who scrolled 90%+ (content consumption signal)conversions— goal completions attributed to page
Bing Webmaster Tools API — supplementary indexation + Bing performance
Base URL: https://ssl.bing.com/webmaster/api.svc/json/
Auth: API key from Bing Webmaster Tools dashboard
| Endpoint | Method | Purpose | ContentEngine Use |
|---|---|---|---|
GetQueryStats | GET | Bing impressions, clicks, CTR, position by query | Supplementary search performance data |
GetPageStats | GET | Bing page-level performance | Bing-specific content performance |
GetUrlInfo | GET | Index status for a specific URL | Cross-engine index verification |
SubmitUrl | POST | Submit URL for crawling | Trigger Bing crawl on publish |
SubmitUrlBatch | POST | Submit up to 500 URLs | Batch indexation requests |
GetCrawlStats | GET | Crawl activity and errors | Monitor crawl health |
IndexNow endpoint (instant indexation):
1POST https://www.bing.com/indexnow
2{
3 "host": "yourdomain.com",
4 "key": "your-indexnow-key",
5 "keyLocation": "https://yourdomain.com/indexnow-key.txt",
6 "urlList": [
7 "https://yourdomain.com/blog/new-post",
8 "https://yourdomain.com/blog/updated-post"
9 ]
10}IndexNow is the fastest path to Bing indexation (near-instant). It also feeds Yandex, Seznam, and Naver automatically. Google does not officially support IndexNow but may honor it experimentally.
Endpoint-to-Agent Mapping
| Agent | Primary Endpoints | Data Frequency |
|---|---|---|
| Search Landscape Agent | SERP API (organic/live + task), AI Optimization (LLM Mentions, AI Keyword Data) | Weekly (SERP), Daily (top 20 rank checks) |
| Content Strategy Agent | Keywords Data (search_volume), Labs (keyword_overview, bulk_keyword_difficulty, search_intent, related_keywords, domain_intersection), Backlinks (summary, bulk_ranks), Google Trends | Weekly (competitive), Monthly (keyword refresh) |
| Competitive Intelligence Agent | Labs (ranked_keywords, relevant_pages, domain_rank_overview, competitors_domain, domain_intersection, page_intersection) | Weekly across 10 competitors |
| Refresh Queue Scoring | GSC Search Analytics (page + query + date dimensions), GA4 runReport (engagement metrics) | Daily polling, 30-day rolling window |
| Post-Publish Pipeline | GSC URL Inspection, Google Indexing API, Bing IndexNow, SERP API (live for rank verification) | 24h / 7d / 30d post-publish |
| Link Authority Scoring | Backlinks (summary, referring_domains, bulk_ranks, anchors, history) | Monthly snapshots |
Authentication Summary
| API | Auth Method | Setup |
|---|---|---|
| DataForSEO (all endpoints) | HTTP Basic Auth | Base64 encode login:password in Authorization header |
| Google Search Console | OAuth 2.0 | Service account with webmasters.readonly scope |
| Google Analytics 4 | OAuth 2.0 | Service account with analytics.readonly scope |
| Google Indexing API | OAuth 2.0 | Service account with indexing scope |
| Bing Webmaster Tools | API Key | Key from Bing Webmaster dashboard in query parameter |
| Bing IndexNow | API Key | Self-hosted key file at domain root |
Monthly Cost Summary
| API Group | Endpoints Used | Actual Usage Cost | Commitment Floor |
|---|---|---|---|
| DataForSEO SERP | 2 endpoints | ~$2 | — |
| DataForSEO Keywords Data | 4 endpoints | ~$3 | — |
| DataForSEO Labs | 17 endpoints | ~$12 | — |
| DataForSEO Backlinks | 14 endpoints | ~$2 | $100/month |
| DataForSEO AI Optimization | 6 endpoints | ~$20 | $100/month |
| DataForSEO Google Trends | 3 endpoints | ~$5 | — |
| Google Search Console | 4 endpoints | $0 | — |
| Google Analytics 4 | 3 endpoints | $0 | — |
| Google Indexing API | 1 endpoint | $0 | — |
| Bing Webmaster + IndexNow | 7 endpoints | $0 | — |
| Total | 61 endpoints | ~$44/month actual | $200/month floor |