How to Scrape Wikidata: Search, Entities & SPARQL 2026

Wikidata is the largest open knowledge graph on the web — 100+ million entities with structured facts about places, people, companies, chemicals, artworks, everything. Its APIs are fully public and its data is CC0 (public domain), so access isn’t the problem. The problem is shape: the data is a property graph, not a table, entities are referenced by opaque QIDs like Q64 and properties by codes like P31, and getting a clean flat dataset out means either looping thousands of entity lookups or writing SPARQL. This guide covers the three ways to extract it — keyword search, QID lookup, and SPARQL — and when to use each, no API key required.

What’s worth extracting

Wikidata Scraper returns one row per entity (or per SPARQL result row), with these fields:

• Identity — qid (the entity ID, e.g. Q64), label (human-readable name in your chosen language), and wikidataUrl (direct link to the entity page).
• Description — description (a short phrase, e.g. “capital and largest city of Germany”) and aliases (comma-separated alternative names and spellings).
• Classification — instanceOf (comma-separated P31 “instance of” values — the entity’s type).
• Structured values — claims (all claim values flattened as a JSON object, populated in entities mode) and sparqlBindings (all SPARQL result variable bindings as a JSON object, populated in sparql mode).
• Provenance — mode (which mode produced the row: search, entities, or sparql).

The critical thing to understand is that description, aliases, instanceOf and claims are only populated in entities mode. In SPARQL mode those are intentionally empty, and every piece of query result data lives in sparqlBindings as a JSON object holding each bound variable from your SELECT. This trips people up constantly — see Pitfalls.

How the source actually works

The actor talks to the official Wikidata APIs and the SPARQL endpoint directly, all keyless. There are three modes, each mapping to a specific Wikidata action.

SPARQL mode is the highest-volume option. It sends your raw query to the Wikidata Query Service and returns every result row. SPARQL is the graph query language for Wikidata — similar to SQL but built for linked data. This query pulls sovereign states with their capital and population:

{
  "mode": "sparql",
  "sparql": "SELECT ?item ?itemLabel ?population ?capital ?capitalLabel WHERE { ?item wdt:P31 wd:Q6256 . OPTIONAL { ?item wdt:P1082 ?population . } OPTIONAL { ?item wdt:P36 ?capital . } SERVICE wikibase:label { bd:serviceParam wikibase:language 'en'. } } LIMIT 500",
  "language": "en",
  "maxResults": 0
}

Entities mode takes a list of QIDs and calls wbgetentities in batches of 50, returning full labels, descriptions, aliases and flattened claims. Use it when you already know the QIDs:

{
  "mode": "entities",
  "qids": ["Q64", "Q90", "Q220", "Q84", "Q456"],
  "language": "en"
}

Search mode uses wbsearchentities to find entities matching a keyword, paginating automatically to your maxResults:

{
  "mode": "search",
  "query": "Olympic Games",
  "language": "en",
  "maxResults": 200
}

A few property codes recur so often they’re worth memorizing: P31 (instance of), P17 (country), P1082 (population), P36 (capital), P571 (inception date), P856 (website), P159 (headquarters location), P452 (industry). In SPARQL you reference a property as wdt: plus its code and an entity as wd: plus its QID.

▶ Open the Wikidata Scraper on Apify — one SPARQL query returns thousands of structured rows: countries, companies, scientists, chemicals, anything in the graph. Keyless, no login. Export to CSV, JSON or Excel.

The access reality

Wikidata actively encourages programmatic access, so there’s no anti-bot fight — the constraints are the query service’s limits and the graph’s shape:

• The SPARQL endpoint has a 60-second query timeout. A single run can return tens of thousands of rows, but the practical ceiling is whatever finishes inside 60 seconds. If a complex query times out, add a LIMIT, narrow the filter conditions, or split it with OFFSET pagination.
• Control volume with LIMIT, not just maxResults. Capping at the query level is more efficient than pulling everything and truncating with the maxResults field — you save the endpoint the work.
• Missing properties silently drop rows. Many entities lack a given property. If you don’t wrap optional properties in OPTIONAL { ... }, entities without that property vanish from the result entirely — a common source of “why is my count so low.”
• Labels aren’t guaranteed per language. Not every entity has a label in every language. Set language to en for the best coverage, or add SERVICE wikibase:label to your SPARQL so it falls back to the entity ID when no label exists.
• Etiquette. The actor sends a descriptive User-Agent and paces requests as Wikidata’s API guidelines recommend, rather than hammering the endpoint.

The naive approach — looping individual entity lookups for a large set — is slow and rate-limited. A single SPARQL query does the same work server-side in one shot, which is why it’s the recommended path for bulk extraction.

Example output

One fully-populated row from entities mode:

{
  "qid": "Q64",
  "label": "Berlin",
  "description": "capital and largest city of Germany",
  "aliases": "Berlin, DE",
  "instanceOf": "Q515, Q1549591, Q200250",
  "wikidataUrl": "https://www.wikidata.org/wiki/Q64",
  "claims": "{\"P31\":[\"Q515\",\"Q1549591\"],\"P17\":\"Q183\",\"P1082\":\"3677472\",\"P856\":\"http://www.berlin.de/\"}",
  "sparqlBindings": "",
  "mode": "entities"
}

And one from SPARQL mode, where the data lives in sparqlBindings:

{
  "qid": "Q183",
  "label": "Germany",
  "wikidataUrl": "https://www.wikidata.org/wiki/Q183",
  "sparqlBindings": "{\"itemLabel\":\"Germany\",\"population\":\"84607016\",\"capitalLabel\":\"Berlin\"}",
  "mode": "sparql"
}

Typical use cases

Who pulls Wikidata and for what:

• Structured list building — extract all capital cities with their countries and populations, all chemical elements with atomic numbers and discovery dates, or all books with author, year, genre and language, via a single SPARQL query.
• Company enrichment — look up hundreds of company QIDs in entities mode to get official names, founding dates, headquarters, and industry (P452) classifications.
• Entity dataset discovery — search “Nobel Prize” or “Olympic Games” to build a dataset of laureates or events, then feed the QIDs into entities mode for full detail.
• NLP and entity linking — pull canonical labels, aliases and types in bulk for entity recognition, disambiguation, or knowledge-base construction.
• Fact-checking and journalism — get structured factual data (populations, birth dates, office holders) without hand-navigating Wikidata.
• Grounded AI pipelines — supply an LLM with factual, structured data to reduce hallucination, e.g. “all UNESCO World Heritage Sites in Italy with year of inscription and site type.”

Cost and effort: build vs. managed

The DIY path splits by mode. Hand-rolling wbsearchentities pagination and wbgetentities 50-QID batching is a fair amount of glue code. SPARQL is a single POST, but then you own result parsing, the 60-second timeout handling, OFFSET pagination for big result sets, User-Agent etiquette, and flattening bindings and claims into a stable row shape. Doable, but it’s a component you’d have to maintain.

The managed actor is pay-per-result: a small per-run start fee plus a fraction of a cent per row, no subscription (Wikidata is free and CC0). The README puts a SPARQL query returning up to 10,000 rows at under 30 seconds. Because you pay per row and can cap with both LIMIT in the query and maxResults, a validation run is nearly free before you scale a bulk extraction.

Common pitfalls

Specific to Wikidata:

• SPARQL mode leaves description/aliases/instanceOf/claims empty — on purpose. All your query data is in sparqlBindings. If you expected those columns filled, you wanted entities mode. This is the number-one confusion.
• Wrap optional properties in OPTIONAL. Without it, entities missing a property are dropped from the result set entirely, silently shrinking your dataset.
• Test SPARQL before you run. The Wikidata Query Service editor at query.wikidata.org has autocomplete and instant feedback. Validate the query there, then paste it into the actor.
• Add SERVICE wikibase:label for readable names. Without it, SPARQL returns entity URIs (http://www.wikidata.org/entity/Q64) instead of “Berlin,” and rows may lack labels.
• Cap with LIMIT, not just the actor field. The 60-second timeout kills over-broad queries. A LIMIT clause is the efficient, reliable ceiling; use OFFSET to page a large result set across runs.
• Empty labels are a language issue. If you set language: 'de' and an entity has no German label, the field is blank. Fall back to en or rely on the label service.
• Parse the JSON-string fields. claims and sparqlBindings are JSON serialized as strings. Parse them before reading nested values.

Wrapping up

Wikidata is an extraordinary, fully-open, CC0 knowledge graph — the only real skill is choosing the right mode (search to discover, entities to detail QIDs, SPARQL for bulk) and, if you go SPARQL, writing a query that returns readable labels and doesn’t time out. If you’d rather skip building the pagination, batching and binding-flattening layer, the managed actor does all three modes keyless.

▶ Run the Wikidata Scraper on Apify — one SPARQL query yields thousands of structured rows with QIDs, labels, descriptions and bound variables; entities mode returns full flattened claims per QID. Keyless, no login. Export to CSV, JSON or Excel. Start on Apify’s free monthly credit.