How to Scrape Open Library Books, ISBNs & Ratings 2026

Assembling a clean book dataset usually means fighting one of two things: a commercial metadata API that charges per lookup and gates the good fields, or scraping retailer pages that fight bots and ship inconsistent HTML. Open Library — the Internet Archive’s catalog of over 20 million book records — sidesteps both. It has a free, keyless REST API with ISBNs, ratings, subjects and cover images already normalized. The work is in knowing which of its four endpoints to hit for what you want, and paginating them into a single tidy dataset. This guide covers how the Open Library API is shaped, what each mode returns, and how to pull thousands of book records in one run.

What’s worth extracting

Each book record carries around 20 fields, normalized into flat strings for easy tabular export:

• key — the Open Library work key (e.g. /works/OL45804W), the canonical identity of the book.
• title and authors (comma-separated names).
• firstPublishYear — the year the work first appeared.
• isbn — up to 5 ISBNs (ISBN-10 or ISBN-13) for the work.
• editionCount — total editions across all formats, a decent proxy for a book’s reach.
• publishers — comma-separated publisher names.
• subjects — up to 8 subject tags / genres.
• ratingsAverage (out of 5) and ratingsCount — community-sourced ratings.
• language — language codes (e.g. eng, fre).
• coverUrl — the large cover image URL.
• openLibraryUrl — the book’s page.
• description — the book blurb, populated in bookDetail mode.
• mode — which scraping mode produced the row.

How the source actually works

The actor talks to the official Open Library REST API at openlibrary.org, using got-scraping with descriptive User-Agent headers and Apify proxy support. No authentication — the API is designed and documented for programmatic access. There are four modes:

• search — full-text query across the whole catalog, optionally narrowed by title or author. Fetches 100 records per API call and paginates.
• subject — browse every book in a genre/topic. Subject slugs use underscores (science_fiction, mystery_and_detective_stories, history).
• author — search authors and their catalogs; here coverUrl returns the author’s photo.
• bookDetail — deep data for specific works by ID, including the description field that search mode omits.

A minimal search run:

{
  "mode": "search",
  "query": "science fiction",
  "maxResults": 500
}

Subject browsing (note the underscored slug):

{
  "mode": "subject",
  "subject": "mystery_and_detective_stories",
  "maxResults": 300
}

And deep detail for specific works — work IDs are accepted as OL27258W, /works/OL27258W or just the numeric part:

{
  "mode": "bookDetail",
  "workIds": ["OL27258W", "OL45804W", "OL262238W"]
}

One handy trick: Open Library’s search supports ISBN queries natively, so in search mode you can set query to isbn:9780345391803 to resolve a single book by its ISBN.

▶ Open the Open Library Scraper on Apify — one run pulls thousands of book records with ISBNs, ratings, publishers, subjects and cover URLs. Keyless, straight from the Internet Archive’s public API. Export to JSON, CSV or Excel.

The access reality

Open Library is one of the friendliest sources on the open web, but its coverage has a shape you need to plan around:

• Community-driven completeness. The catalog is crowd-maintained, so lesser-known books may lack ISBNs, ratings or covers. ratingsAverage and ratingsCount only appear once at least one user has rated a work — expect null on obscure titles.
• Work vs. edition. A work is the canonical book (e.g. The Hitchhiker’s Guide to the Galaxy); an edition is a specific physical version (hardcover 2000, paperback 2003). The actor returns work-level data by default; bookDetail mode also pulls edition data (ISBNs, publishers, languages).
• Descriptions are bookDetail-only. Search mode does not return description. If you need blurbs, collect work IDs from search, then run bookDetail on them.
• Subject slugs are exact. They must use underscores and match a valid Open Library subject. You can verify a slug at openlibrary.org/subjects/<slug>.json before a big run.
• English-heavy coverage. Open Library indexes best for English-language books. For non-English works, add the language to your query (e.g. "french poetry").
• maxResults caps at 5000 per run, with the default at 200. No auth token, no rate-limit workarounds needed — the API is public and the actor runs at low resource usage (512 MB, datacenter proxy).

Example output

A representative search-mode record:

{
  "key": "/works/OL27258W",
  "title": "The Hitchhiker's Guide to the Galaxy",
  "authors": "Douglas Adams",
  "firstPublishYear": "1979",
  "isbn": "9780345391803, 0345391802, 9780330508117",
  "editionCount": "487",
  "publishers": "Pan Books, Del Rey Books, Harmony Books",
  "subjects": "Science fiction, Humorous fiction, Space, Adventure, Comedy",
  "ratingsAverage": "4.21",
  "ratingsCount": "188432",
  "language": "eng",
  "coverUrl": "https://covers.openlibrary.org/b/id/8267040-L.jpg",
  "openLibraryUrl": "https://openlibrary.org/works/OL27258W",
  "description": null,
  "mode": "search"
}

Note description is null here because this came from search mode; run bookDetail on the key to populate it. Cover images come in three sizes — swap -L.jpg for -M.jpg (medium) or -S.jpg (small) in coverUrl.

Typical use cases

• Genre recommendation datasets — scrape all science-fiction books with their ratings to seed a recommender.
• Catalog population — extract ISBNs and publisher info for a subject area to fill a catalog database.
• Author authority files — collect author metadata (work count, top works, subjects) across a body of writers.
• Book-discovery sites — pull cover URLs and first-publish years to power a browsing UI.
• Rating and edition monitoring — track editionCount and community ratings for a set of classic works over time.
• NLP / corpus work — assemble large bibliographic datasets for classification or trend analysis, exported straight to CSV.

Build-it-yourself vs. managed actor

The API is open and well-documented, so a single lookup is trivial. What adds up is the four-mode routing (search vs. subject vs. author vs. detail), 100-per-call pagination to reach thousands of rows, the work-ID normalization that accepts three input formats, flattening nested JSON (authors, subjects, ISBNs) into clean comma-separated strings, and handling the pervasive null fields gracefully.

On cost, the actor is pay-per-result — a fraction of a cent per book row plus a tiny per-run start fee, and no proxy subscription since the API is public. A 1,000-book research pull is roughly 10 API calls and finishes in well under a minute; you’re paying cents for a dataset that’s otherwise tedious to assemble by hand. Open Library’s data is published under open licenses (primarily CC0 and CC BY), which makes it unusually clean to reuse — though for commercial use you should confirm your application aligns with that licensing.

Common pitfalls

• Underscored subject slugs, always. science_fiction, not science fiction. A space in the slug returns nothing. Verify at openlibrary.org/subjects/<slug>.json first.
• Null fields are the norm, not a bug. Obscure works legitimately lack ISBNs, ratings or covers. Design downstream code to tolerate null, and use bookDetail for the fullest data on a specific work.
• Search mode has no descriptions. Don’t expect description from search — it’s a bookDetail-only field. Two-step it: search for keys, then detail those keys.
• Ratings are sparse and community-sourced. They exist only on rated works, so a genre dataset will have plenty of unrated entries. Don’t filter to “rated only” unless you deliberately want the popular subset.
• Work vs. edition confusion. If you need per-edition ISBNs and publishers, use bookDetail; search returns work-level fields. Mixing the two silently will make your ISBN coverage look inconsistent.
• Set maxResults high for research pulls. The default is 200. For a real dataset, push it toward 1000+ and let the actor paginate.

Wrapping up

Open Library gives you a genuinely open, keyless, openly-licensed book catalog — the hard part is just routing its four endpoints and paginating them into one clean dataset with the null fields handled sanely. This actor packages that, so you can go from a subject slug or a search query to thousands of structured book records, ISBNs and cover URLs in a single run.

▶ Run the Open Library Scraper on Apify — thousands of book records with ISBNs, ratings, publishers, subjects and covers by search, subject, author or work ID. No API key, no login. Export to JSON, CSV or Excel, and start on Apify’s free monthly credit.