How to Scrape OpenAlex Papers, Authors & Institutions in 2026

OpenAlex is the open-access answer to Scopus and Web of Science: a bibliographic graph of over 250 million scholarly works, 90 million authors, 100,000+ institutions, and 200,000+ journals, released under CC0 with a fully keyless API. For anyone doing a systematic literature review, a citation analysis, or a bibliometric study, it removes the paywall entirely. What it does not remove is the engineering: to pull a real corpus you need cursor-based pagination done correctly, OpenAlex’s filter syntax to narrow results, and a flattening step to turn nested authorship and concept objects into columns you can load into R, Python, or a spreadsheet. This guide covers the API shape and how to extract clean bibliographic data at scale.

What data is worth extracting

The scraper returns a unified flat schema across four entity types, with the relevant fields populated per mode.

Works (papers) — the most common target:

• Identity & type — id (OpenAlex work ID, URL format), doi (full URL), title, type (article, book-chapter, preprint, etc.).
• Dates & impact — publication_year, publication_date, cited_by_count.
• People & places — authors (semicolon-separated names), author_institutions (unique institutions across all authors).
• Venue & access — journal, oa_status (gold, green, hybrid, bronze, closed), language (ISO 639-1).
• Graph edges — concepts (comma-separated research topics), referenced_works_count.

Authors — display_name, works_count, cited_by_count, orcid, affiliations, x_concepts (top research areas).

Institutions — display_name, country, type (education, government, nonprofit, etc.), works_count, cited_by_count, homepage_url.

Sources (journals) — display_name, issn, publisher, works_count, is_oa, country.

Because all four modes share one schema, fields from the other entity types come back null on a given row. That is by design, not a bug.

How the source actually works

The scraper connects to the OpenAlex API at api.openalex.org — no auth, generous rate limits. It runs four modes: works, authors, institutions, and sources.

The pagination strategy differs by mode, and this is the detail that makes or breaks a bulk pull. Works use cursor-based pagination: you start with cursor=*, read the next_cursor from each response, and pass it to the next request, at per-page=200. That is how the actor fetches thousands of papers efficiently — roughly one API call per 200 records. Authors, institutions, and sources use ordinary page-based pagination.

The other lever is OpenAlex’s filter syntax, which lets you narrow works by structured attributes without keyword matching. A keyword search plus a filter for recent journal articles:

{
  "mode": "works",
  "search": "machine learning",
  "filter": "from_publication_date:2023-01-01,type:article",
  "maxResults": 1000,
  "mailto": "[email protected]"
}

Searching authors by name:

{
  "mode": "authors",
  "search": "Geoffrey Hinton",
  "maxResults": 50
}

The input fields: mode, search (full-text keyword query across titles, abstracts, and indexed content), filter (OpenAlex filter expression, works mode only), maxResults (default 300, up to 50,000), and mailto (your email for the polite pool). Datacenter proxy is sufficient — OpenAlex does not block Apify’s infrastructure.

The search versus filter distinction matters: search does keyword matching; filter narrows by structured attributes like from_publication_date, type, open_access.is_oa, concepts.id, and institutions.id. You can and often should combine both.

Four modes are one pipeline

The reason to keep all four entity types in one actor is that a real bibliometric project touches all of them. You search works for the papers on a topic, then pivot: pull the authors behind the most-cited results to profile the field’s key researchers (works_count, cited_by_count, orcid, affiliations); resolve their institutions to benchmark universities by output and citation impact; and check the sources — the journals — to compare venues by works_count, is_oa, and publisher before deciding where to submit. Because the output shares one flat schema, the four result sets join cleanly on the OpenAlex IDs. A single filter like institutions.id:I27837315 in works mode returns every paper from one institution — the same ID you got from an institutions-mode search — so the entity graph stays connected across runs.

▶ Open the OpenAlex Scraper on Apify — search works, authors, institutions, or journals and get flat rows with DOI, citations, and open-access status. Cursor pagination handles thousands of results. No API key.

The access reality

OpenAlex is one of the most accommodating large APIs in existence, but there are still things to engineer around:

• The polite pool is the difference between fast and throttled. Supply your email via mailto and OpenAlex routes you into a higher-rate-limit pool and can contact you if your usage causes problems. Omit it and you share the anonymous pool.
• The per-run ceiling is 50,000 records. Cursor pagination is efficient, but for a topic with hundreds of thousands of matching works you narrow with filters (by year, by type, by concept) or run multiple filtered passes and merge.
• Filter syntax is exact. A malformed filter expression silently returns nothing useful. from_publication_date:2020-01-01,open_access.is_oa:true is comma-separated with no spaces; get the attribute names right (they are documented) or you get zero results.
• Concept filtering needs IDs, not names. To filter by topic you use a concept ID like concepts.id:C41008148, not the word “machine learning”. Look the ID up first.
• Everything is stringified in the flat output. Numeric-looking fields like cited_by_count and works_count come back as strings for schema consistency — cast them downstream before you sort or sum.

Throughput is a non-issue: 200 records per request, roughly five calls per 1,000 papers, and a typical 1,000-paper run finishes in under 30 seconds.

Example output

One works-mode row, trimmed:

{
  "mode": "works",
  "id": "https://openalex.org/W2963403868",
  "doi": "https://doi.org/10.1038/s41586-019-1099-1",
  "title": "Solving the retrosynthetic analysis problem using neural network models",
  "publication_year": "2019",
  "publication_date": "2019-03-27",
  "type": "article",
  "cited_by_count": "1890",
  "authors": "Segler, Marwin H. S.; Preuss, Mike; Waller, Mark P.",
  "author_institutions": "University of Münster; IBM Research",
  "journal": "Nature",
  "oa_status": "closed",
  "concepts": "Retrosynthetic analysis, Machine learning, Neural network, Chemistry",
  "referenced_works_count": "48",
  "language": "en"
}

Use cases

• Systematic literature reviews — pull every paper on a topic published after a cutoff, filter by article type, and export to CSV for PRISMA screening.
• Citation analysis — fetch the most-cited papers in a field, identify key authors and their institutions, and map influence networks.
• Open-access monitoring — track what percentage of an institution’s output is published open-access using the oa_status field.
• Author profiling — search researchers by name and extract works count, citation count, ORCID, and affiliations.
• Journal benchmarking — compare journals by works count, citation count, and open-access rate to inform submission decisions.

Who uses it: academic researchers avoiding Scopus and Web of Science fees; data scientists and NLP engineers building citation networks and topic-modeling datasets; research libraries tracking publication output and OA adoption; journalists and policy analysts exploring publishing trends; and developers building research tools on a keyless pipeline.

Cost and effort: build vs. managed

Building this yourself means implementing cursor pagination correctly (the cursor=* → next_cursor loop is easy to get subtly wrong and either miss records or loop forever), wiring up the polite-pool mailto header, understanding the filter grammar well enough to not silently return nothing, and flattening nested authorship, institution, and concept arrays into stable columns across four entity types. It is a solid day of work, most of it in the flattening.

The actor is pay-per-result at a low per-record rate. A 1,000-paper review lands in the low cents, and since OpenAlex charges nothing for the data and datacenter proxy suffices, you are effectively paying only for extraction. Schedule monthly snapshots to watch citation counts grow over time.

Common pitfalls

• Null fields are mode-specific, not missing data. Works-mode fields (doi, journal, oa_status) are null in authors/institutions/sources rows and vice versa — the unified schema shares one row shape.
• Zero results usually means a bad filter or a niche query. Verify filter syntax and try a broader search term; check the query on openalex.org directly if unsure.
• Cast strings before numeric work. cited_by_count, works_count, and similar are strings in the output — sort and aggregate only after casting.
• search and filter are complementary, not interchangeable. Use search for topic keywords and filter for structured constraints; combine them for precision.
• Concepts are machine-assigned. The concepts field comes from an automated taxonomy at varying confidence — treat it as a useful signal, not ground truth.
• Attribute a CC0 source anyway. OpenAlex is public domain, so no legal obligation, but “Data from OpenAlex” is the norm in publications and products.

Wrapping up

OpenAlex hands you Scopus-scale bibliographic data for free; the only friction is cursor pagination, filter grammar, and flattening the graph into rows. For a one-off review the works mode with a good filter gets you a clean CSV; for ongoing monitoring, schedule it and let the actor manage the cursor loop and export.

▶ Run the OpenAlex Scraper on Apify — one run returns up to 50,000 works, authors, institutions, or journals with DOI, citations, and OA status as flat rows. No API key, export to JSON, CSV, Excel, or JSONL. Start on Apify’s free monthly credit.