How to Scrape GitLab Projects, Issues & MRs in 2026

GitHub gets all the scraping attention, but GitLab hosts a large, distinct ecosystem — infrastructure tooling, DevOps projects, and companies that never mirror to GitHub. Its public REST API is genuinely open and needs no token for public data, but if you want the top 1,000 Kubernetes projects by star count, or every open issue across a set of infra repos, you still have to handle pagination, rate limits, and the fact that projects, issues, and merge requests come back in different shapes. This guide covers what GitLab’s public API exposes, how the four scrape modes map onto it, and how to pull clean rows at scale.

What’s worth extracting

The actor returns one of three record types depending on the mode. Project records carry 14 fields:

• Identity — id (GitLab numeric ID), name, path (full path with namespace, e.g. gitlab-org/gitlab), and namespace (the owning user or group).
• Popularity & activity — starCount, forksCount, openIssuesCount, and lastActivityAt.
• Description & tags — description and topics (comma-separated tags).
• Metadata — webUrl, visibility (public, internal, private), createdAt, and defaultBranch.

Issue and merge request records add a second field set:

• projectId and projectPath (the parent repo), number (the internal iid), title, state (opened, closed, merged), author (GitLab username), labels, and timestamps: createdAtItem, updatedAt, closedAt.

Every row is tagged with a type field (project, issue, or merge_request) so mixed outputs stay separable downstream.

How the source actually works

The scraper hits GitLab’s official REST API v4 at https://gitlab.com/api/v4/, using only public, unauthenticated endpoints — no account, no Personal Access Token. A single mode parameter selects one of four operations:

• projects — search public projects by keyword, sorted by star count, last activity, or creation date.
• issues — fetch open issues for one or more project IDs or paths, with author, labels, and timestamps.
• mergeRequests — fetch open merge requests for given projects, including state, author, and merge timestamps.
• userProjects — list all public repositories for a specific GitLab username.

A typical competitive-research run — the most-starred projects for a topic — looks like this:

{
  "mode": "projects",
  "query": "kubernetes",
  "orderBy": "star_count",
  "maxResults": 500,
  "proxyConfiguration": { "useApifyProxy": true }
}

The orderBy field takes any of GitLab’s sort options: star_count, last_activity_at, created_at, id, or name. Switch to last_activity_at when you want the freshest repos rather than the most popular.

For issues or merge requests, you pass a list of project references — either numeric IDs or namespace paths — and the actor handles URL-encoding the paths:

{
  "mode": "issues",
  "projectIds": ["gitlab-org/gitlab", "278964"],
  "maxResults": 1000,
  "proxyConfiguration": { "useApifyProxy": true }
}

Listing a user’s repos is just a username:

{
  "mode": "userProjects",
  "username": "torvalds",
  "maxResults": 200
}

Under the hood the actor uses got-scraping for resilient HTTP with automatic retries, respects GitLab’s pagination headers, and fetches 100 records per API call, walking pages until it reaches your maxResults. That per-call batching is why throughput scales cleanly: roughly 10–20 seconds of runtime for 1,000 records, and around 1–2 minutes for a full 5,000, depending on API latency.

The four modes are meant to compose. There is no single endpoint that returns a topic’s repos and their issues, so the intended workflow is a two-step chain: run projects to collect the IDs of the repos you care about, then feed those IDs into an issues or mergeRequests run. Keeping the modes separate also keeps the output schema clean — you never get project rows and issue rows interleaved in a way you have to untangle, because the type field always tells you which shape a row is.

▶ Open the GitLab Scraper on Apify — one run turns a keyword, a set of project IDs, or a username into structured rows with stars, forks, topics, issues and merge requests. No key, no login. Export to CSV, JSON or Excel.

The access reality

GitLab’s public API doesn’t fight scrapers — the friction is rate limits and coverage, not anti-bot walls:

• ~60 requests/min unauthenticated. GitLab rate-limits the public API to roughly 60 requests per minute without a token. For large runs (over 2,000 results) the actor handles this gracefully with retries and backoff, so you don’t have to engineer the throttling.
• 5,000 results per run. maxResults caps at 5,000. GitLab supports deep pagination for projects, so the practical ceiling is how many repos actually match your query — a broad term like react has thousands.
• Public repos only. The unauthenticated API returns projects with visibility: public. Private and internal projects are invisible without a token, by design.
• Datacenter IPs are fine. GitLab’s public API doesn’t block datacenter ranges, so proxy is optional — it helps stability on long runs but isn’t required.
• Merge requests and issues default to open. The actor fetches state=opened items. If you need closed or merged history you’d need a modified deployment.

Example output

One project record:

{
  "type": "project",
  "id": "12584701",
  "name": "StackGres",
  "path": "ongresinc/stackgres",
  "namespace": "ongresinc",
  "description": "StackGres Operator, Full Stack PostgreSQL on Kubernetes",
  "starCount": 140,
  "forksCount": 63,
  "openIssuesCount": 45,
  "lastActivityAt": "2026-07-07T21:19:05.216Z",
  "webUrl": "https://gitlab.com/ongresinc/stackgres",
  "topics": "PgBouncer, envoy, java, k8s, kubernetes, operator, patroni, postgresql",
  "visibility": "public",
  "createdAt": "2019-05-29T11:46:15.548Z",
  "defaultBranch": "main"
}

Typical use cases

• Ecosystem research — extract the top 1,000 most-starred public projects for a topic (e.g. kubernetes) for a competitive-analysis dashboard.
• Centralized bug tracking — fetch all open issues across a set of infrastructure repos to build a single feed.
• Release-cycle monitoring — track merge-request activity on a set of repos to watch how fast projects ship.
• Developer profiling — list every public repo for a username as part of a talent-sourcing or enrichment pipeline.
• ML dataset building — assemble a dataset of GitLab projects by topic for a classifier that predicts project quality or maturity.
• Security recon — scan public repos for specific technologies or frameworks at scale.

The common thread across these is that GitLab’s own UI caps out at browsing a page at a time, while every one of these tasks needs the full result set — the top 1,000 repos, every open issue, an account’s entire public footprint — reduced to rows you can sort, filter, and join. That’s the gap the API closes and the scraper packages.

Build it yourself vs. use the actor

GitLab’s API is free and documented, so a first query is trivial. What you maintain is the surrounding machinery: pagination-header handling, the 60-req/min rate-limit backoff, path URL-encoding for issues/MR modes, retry logic for transient errors, and mapping three different response shapes (projects, issues, MRs) into consistent rows. It’s a small client you now own.

The actor is that client, built with got-scraping and retries already wired in. Pricing is pay-per-result — a small fraction of a cent per row plus a negligible start fee — and GitLab’s API is free. For a 500-project pull that’s a rounding-error cost, and the rate-limit handling is solved.

Common pitfalls

Specific gotchas with GitLab’s public API:

• Null fields are normal. openIssuesCount is null when a project has issue tracking disabled; topics is null when a repo has no tags. These are genuine gaps in GitLab’s response, not scraper errors.
• Use paths carefully. You can pass gitlab-org/gitlab directly in projectIds — the actor URL-encodes it. But a wrong path or private project silently yields nothing for that entry.
• Specific queries win. "kubernetes operator" returns far more targeted results than "k8s". Broad or abbreviated terms miss matches.
• Sort deliberately. star_count gives you the popular repos; last_activity_at gives you the active ones. They’re very different lists — pick based on your question.
• Chain modes for issue data. To get issues for the top repos in a topic, run projects first to collect IDs, then feed those IDs into an issues run. There’s no single call that does both.
• Open-only by default. Don’t expect closed/merged MRs or issues in the output — the actor pulls the open state.

Wrapping up

GitLab’s public API is open and unauthenticated, but a real pull still means rate-limit backoff, pagination, and reconciling three record shapes. If you need a one-off dataset, the endpoints are open and worth scripting. If you need refreshed GitLab data on a schedule — tracking a rival’s issue velocity, monitoring new repos in your niche — let the actor own the paging, retries, and rate-limit handling and hand you clean rows.

▶ Run the GitLab Scraper on Apify — reads GitLab’s public REST API v4 directly across projects, issues, merge requests and user repos, up to 5,000 rows per run. No API key, no login. Start on Apify’s free monthly credit.