How to Scrape GBIF Species Occurrence & Biodiversity Data

The Global Biodiversity Information Facility (GBIF) is the largest open-access biodiversity database on the planet: over 2 billion georeferenced occurrence records from 1,700+ publishers across 190+ countries, spanning museum collections, citizen-science platforms like iNaturalist and eBird, government agencies, and research institutions. The API is genuinely open — no key, no account — which is a rare gift. The catch is that pulling a real dataset means resolving scientific names to GBIF’s internal taxon keys, driving offset pagination across hundreds of pages, and flattening a deeply nested Darwin Core record into something you can load into a GIS layer or a dataframe. This guide covers how the API is shaped and how to extract clean occurrence data at volume.

What data is worth extracting

Each occurrence record flattens to 25 structured fields. Grouped by what you would actually query on:

• Identity — key (GBIF unique occurrence key), gbifID, and taxonKey.
• Full taxonomy — scientificName (with authorship), plus the whole ladder: kingdom, phylum, class, order, family, genus, species.
• Geography — decimalLatitude and decimalLongitude (WGS84), country, countryCode (ISO 3166-1 alpha-2), and verbatim locality.
• Time — eventDate (a date or date range) and year.
• Provenance — basisOfRecord (how it was observed), datasetName, datasetKey (the GBIF dataset UUID), recordedBy (observer/collector), and institutionCode.
• Status & conservation — occurrenceStatus (PRESENT/ABSENT), iucnRedListCategory (LC, NT, VU, EN, CR, EW, EX, or null), and the license URL for proper attribution.

The basisOfRecord field is the quality lever most people miss: HUMAN_OBSERVATION (iNaturalist, eBird), PRESERVED_SPECIMEN (museum collections), MACHINE_OBSERVATION (camera traps, acoustic detectors), and MATERIAL_SAMPLE (DNA/environmental samples) are very different data.

How the source actually works

The scraper hits the official GBIF REST API at api.gbif.org/v1 — fully public, no auth. It runs in four modes:

Mode	What it does
occurrences	The flagship: resolves a species name to a taxon key, then paginates occurrence records.
speciesMatch	Resolves a taxon name to its accepted canonical name and GBIF usage key.
speciesSearch	Paginated species catalog results for a keyword.
datasets	Searches the GBIF dataset registry to find data sources by theme.

The clever part is taxon resolution. You do not pass an internal ID — you pass a plain scientific name, and the actor calls /species/match to resolve it (handling synonyms and fuzzy matches) before it ever queries occurrences. It then drives /occurrence/search with offset-based pagination at 300 records per page, respecting GBIF’s endOfRecords flag so it stops cleanly.

An occurrences pull for brown bears in Finland over a date range:

{
  "mode": "occurrences",
  "taxonName": "Ursus arctos",
  "country": "FI",
  "yearFrom": 2010,
  "yearTo": 2024,
  "maxResults": 1000
}

A dataset-registry search to discover what sources cover a topic:

{
  "mode": "datasets",
  "query": "coral reef monitoring",
  "maxResults": 200
}

The input fields: mode, taxonName (the scientific name to resolve and filter on), query (free-text for species/dataset search), country (ISO 2-letter code), yearFrom / yearTo (temporal filters), and maxResults (default 300, up to 100,000). Datacenter proxy is sufficient — GBIF does not block Apify’s infrastructure.

Why the taxon-resolution step matters

GBIF is a taxonomic backbone as much as an occurrence store, and species names are messy: the same animal has synonyms, misspellings, and authorship variants scattered across 1,700 publishers’ records. If you queried /occurrence/search with a raw name string you would get a fraction of the real records — or none. The speciesMatch step against /species/match normalizes the name to its accepted canonical form and a single GBIF usage key first, absorbing synonyms and fuzzy matches, so the occurrence query runs against the taxon key that actually ties all those records together. You can run speciesMatch on its own to inspect what a name resolves to before committing to a large pull. For discovering what data even exists on a theme, datasets mode searches the registry — find the sources covering “coral reef” or “pollinator monitoring”, note their datasetKey UUIDs, and use those to scope occurrence queries.

▶ Open the GBIF Biodiversity Scraper on Apify — pass a scientific name like Panthera leo, optionally filter by country and year, and get georeferenced occurrence records with full taxonomy. No key, no login.

The access reality

GBIF is one of the friendlier large APIs to work with, but there are real limits and quirks to plan around:

• The pagination ceiling is 100,000 records per run. GBIF’s own occurrence search paginates up to that offset. For a dataset larger than that — a common widespread species can have millions of records — you split by country or by year range across multiple runs and merge.
• Not every record is georeferenced. Older specimens, some museum collections, and deliberately coarsened coordinates leave decimalLatitude/decimalLongitude null. Filter on non-null coordinates downstream if you need a map.
• Coordinate precision varies. Some records are exact GPS points; others are generalized to a grid cell to protect sensitive species. If precision matters, check coordinateUncertaintyInMeters in the raw data.
• IUCN status is output-only, not a query filter. GBIF’s API does not accept IUCN category as a search parameter. The iucnRedListCategory field is present in results, so you filter for VU/EN/CR after export.
• Use scientific names, not common names. “Brown bear” produces vague matches; Ursus arctos resolves precisely to the right taxon key. This is the single biggest determinant of result quality.

Speed is not really a constraint here: the API fetches 300 records per request with no throttling for typical research workloads, so a 5,000-record run usually finishes in under two minutes.

Example output

One occurrence record, a lion sighting in Kenya:

{
  "key": "2304895122",
  "gbifID": "2304895122",
  "scientificName": "Panthera leo (Linnaeus, 1758)",
  "kingdom": "Animalia",
  "class": "Mammalia",
  "order": "Carnivora",
  "family": "Felidae",
  "genus": "Panthera",
  "species": "Panthera leo",
  "taxonKey": "5219404",
  "country": "Kenya",
  "countryCode": "KE",
  "locality": "Maasai Mara National Reserve",
  "decimalLatitude": -1.4061,
  "decimalLongitude": 35.0078,
  "eventDate": "2022-07-15",
  "year": 2022,
  "basisOfRecord": "HUMAN_OBSERVATION",
  "datasetName": "iNaturalist Research-grade Observations",
  "recordedBy": "kenya_wildlife_observer",
  "occurrenceStatus": "PRESENT",
  "iucnRedListCategory": "VU",
  "license": "http://creativecommons.org/licenses/by-nc/4.0/legalcode"
}

Use cases

• Conservation biologists & ecologists — track species distribution across time and geography, and model habitat range shifts under climate change.
• Environmental data scientists — build GIS layers, biodiversity maps, or machine-learning training sets from georeferenced occurrences.
• Academic researchers — pull bulk occurrence data for species distribution models (SDMs) like MaxEnt without paying for a commercial database.
• Nature NGOs & government agencies — monitor population trends and protected-area coverage using country and IUCN filters.
• Data engineers — build biodiversity data pipelines for public or private analytics platforms.

Concretely: pull 10,000 grizzly bear sightings across Europe to model range shifts; extract every insect occurrence in Germany from 2000–2024 to study pollinator decline; search datasets for “coral reef” to find marine sources; or build a country-level occurrence matrix for endangered amphibians.

Cost and effort: build vs. managed

Building this yourself is not just the pagination. It is the taxon-resolution step (call /species/match, handle the synonym/fuzzy-match response, extract the usage key), then the offset loop honoring endOfRecords, then flattening the nested Darwin Core structure into 25 clean columns, then handling null coordinates and mixed date formats. It works, but it is an afternoon of glue code before your first usable CSV.

The actor is pay-per-result at a low per-record rate. A 5,000-record occurrence pull lands in the low cents, and because GBIF imposes no bandwidth cost of its own, you are effectively paying only for the rows you keep. Schedule it weekly to catch new citizen-science observations for common species as publishers push updates.

Common pitfalls

• Empty results usually mean over-filtering. Combining taxonName + country + a narrow year range for a rare species can produce an empty intersection. Confirm records exist for the species with no filters first, then narrow.
• Common vs. rare species differ by orders of magnitude. A widespread sparrow has millions of records; a rare endemic may have hundreds. Set maxResults with that in mind.
• basisOfRecord is your quality filter. For distribution studies, HUMAN_OBSERVATION (especially research-grade iNaturalist) and PRESERVED_SPECIMEN are the most reliable; treat machine observations accordingly.
• Always honor the license. Most records are CC BY 4.0 or similar; the license and datasetName fields tell you how to attribute the original publisher, which the GBIF terms require.
• Discover datasets first for niche topics. Run datasets mode to find the sources covering your theme, then filter occurrences accordingly.
• The 100k cap is per run, not per query. Split large targets by geography or time and merge — do not expect one run to return millions of rows.

Wrapping up

GBIF is an unusually generous open resource, and the only thing standing between you and a clean occurrence dataset is the taxon-resolution and pagination plumbing. For a one-off species pull the flagship occurrences mode gets you there; for recurring monitoring, schedule it and let the actor handle name resolution, offset pagination, and flattening.

▶ Run the GBIF Biodiversity Scraper on Apify — one run resolves a species name and returns thousands of georeferenced occurrence records with full taxonomy and IUCN status. No key, no login, export to JSON, CSV, or Excel. Start on Apify’s free monthly credit.