How to Scrape Packagist PHP Package & Download Data

Packagist.org is the registry behind every composer require in the PHP world, which makes it the single best source for understanding the PHP ecosystem: which packages are dominant, what a dependency tree actually pulls in, how download counts trend, who maintains what. The registry exposes a public JSON API with no key and no login. The friction is in the shape of the work — you’re stitching together four different endpoints, paginating tens of thousands of search results, fetching per-package detail concurrently without tripping rate limits, and flattening nested require and authors structures into rows. This guide covers what Packagist exposes and how to pull it at scale.

What’s worth extracting

Each output row carries up to 17 fields spanning identity, popularity, dependencies, and authorship:

• Identity — name (the full vendor/package), vendor, description, url (Packagist page), and repository (usually the GitHub source).
• Popularity — downloadsTotal (all-time), downloadsMonthly (last 30 days), downloadsDaily (last 24 hours), favers (users who starred it), and dependents (packages that depend on it).
• Classification — type (library, project, composer-plugin, symfony-bundle, wordpress-plugin, etc.), language (usually PHP), and latestVersion.
• Compliance & provenance — license (e.g. MIT, Apache-2.0), requires (top-level require dependencies as a JSON object), authors (a JSON array of name/email/role), and time (the package’s creation timestamp).

The requires and authors fields come back as JSON strings, so they carry the full structure for downstream parsing rather than being flattened lossy.

How the source works

The scraper hits the official Packagist.org JSON API — the same endpoints that power Composer’s package resolution worldwide. It runs in four modes, each mapped to a real endpoint:

• search — keyword search via /search.json, paginated across tens of thousands of results, ranked by Packagist’s relevance algorithm.
• popular — the real-time trending feed via /explore/popular.json, ranked by total downloads.
• packageDetail — deep extraction for a known list of packages via /packages/{vendor}/{package}.json, including full version and dependency data.
• vendorPackages — enumerate every package under a namespace via /packages/list.json.

A keyword search with full detail fetching looks like this:

{
  "mode": "search",
  "query": "laravel",
  "maxResults": 500,
  "fetchDetails": true
}

Deep-diving a curated dependency set:

{
  "mode": "packageDetail",
  "packageNames": ["laravel/framework", "symfony/console", "guzzlehttp/guzzle"]
}

Enumerating a vendor’s whole catalog:

{
  "mode": "vendorPackages",
  "vendor": "symfony",
  "maxResults": 100,
  "fetchDetails": true
}

The fetchDetails flag is the important lever. Search and popular modes return basic fields cheaply; with fetchDetails: true (the default), the scraper fetches each package’s detail endpoint to fill in downloads, dependencies, license, and authors. It does this 5 requests at a time to balance speed against Packagist’s rate limits. Every request carries a descriptive User-Agent.

The two-endpoint pattern

Understanding why detail fetching exists explains how to use the tool efficiently. Packagist’s /search.json endpoint is a discovery endpoint — it returns matching package names and a few headline fields, but not the full picture. The per-package /packages/{vendor}/{package}.json endpoint holds the deep data: monthly and daily downloads, the dependent count, the require map, the license, and the author list. So any complete extraction is inherently two passes — first find the packages, then fetch each one’s detail — which is exactly what fetchDetails: true automates behind a single run. The consequence is a real cost/completeness dial: a 500-package search with fetchDetails: false is one paginated sweep and finishes in under a minute; the same search with detail on fires 500 extra requests (100 batches of 5) and takes several minutes. Set the flag to match what you actually need — names and basic stats, or the full dependency-and-license profile.

The access reality

Packagist is an open, well-behaved registry — the data is meant to be consumed, since it powers millions of composer install runs daily. The constraints are about detail-fetch economics and pagination:

• fetchDetails is the speed/completeness trade-off. With detail fetching off, you get only the basic search fields — name, description, url, repository, downloadsTotal, favers. Fields like downloadsMonthly, downloadsDaily, dependents, license, authors, and requires are null unless fetchDetails is on. Detail fetching is roughly 5× slower because it’s an extra request per package.
• Concurrency is capped for politeness. The scraper fetches detail 5 packages at a time. Packagist has generous rate limits for well-behaved bots, and staying at 5-concurrent keeps runs comfortably within them. Throughput with detail on is roughly 30–60 packages per minute — a 200-package run takes 3–7 minutes.
• The search index is 90,000+ packages. You can paginate the whole thing with search mode, up to 10,000 per run. The popular feed is smaller but always current.
• Only public packages. Private Packagist (private.packagist.com) requires authentication and is out of scope. Everything here is the open Packagist.org registry.
• Near-real-time data. Packagist updates download counts and listings every few minutes, so each run fetches live figures — no stale cache to worry about.

▶ Run the Packagist Scraper on Apify — one run turns a keyword like laravel into hundreds of PHP packages with downloads, dependencies, license, and authors. No API key, no login. Export to CSV, JSON, or Excel.

Example output

A detail-enriched record for guzzlehttp/guzzle:

{
  "name": "guzzlehttp/guzzle",
  "vendor": "guzzlehttp",
  "description": "Guzzle is a PHP HTTP client library",
  "url": "https://packagist.org/packages/guzzlehttp/guzzle",
  "repository": "https://github.com/guzzle/guzzle",
  "downloadsTotal": "580000000",
  "downloadsMonthly": "12500000",
  "downloadsDaily": "420000",
  "favers": "22850",
  "dependents": "42000",
  "type": "library",
  "language": "PHP",
  "latestVersion": "7.9.2",
  "license": "MIT",
  "requires": "{\"php\":\"^7.4 || ^8.0\",\"psr/http-client\":\"^1.0\"}",
  "authors": "[{\"name\":\"Michael Dowling\",\"role\":\"Lead developer\"}]",
  "time": "2011-10-05T20:55:15+00:00"
}

Notice requires and authors are JSON strings — parse them to get the structured dependency map and author list.

Use cases

• Dependency auditing — fetch full require maps for hundreds of packages at once via packageDetail to trace transitive dependency chains and flag deprecated or unmaintained packages.
• Competitive intelligence — search a niche (e.g. JWT authentication) and compare download counts, favers, and update frequency to pick the strongest library.
• Ecosystem trend tracking — schedule popular mode to watch which packages rise or fall in download rank over time.
• Vendor due diligence — use vendorPackages to extract a company’s complete namespace before onboarding a supplier or auditing a framework.
• License compliance — extract the license field across a dependency tree to check it against your organization’s open-source policy.
• AI-agent research — the actor is an MCP tool, so an agent can be asked to “find the top 50 PHP HTTP clients sorted by monthly downloads and compare their licenses.”

Cost and effort: build vs. managed

The Packagist API is free and keyless, so this is about the plumbing:

• Building from scratch — you’d wire up four different endpoints, write pagination for the search index, build the concurrent detail-fetch loop with a sane concurrency cap and 429 handling, and parse the nested require/authors JSON into usable fields. The two-endpoint pattern (search for names, then detail for stats) is the part most people underestimate.
• Using the managed actor — pick a mode, set maxResults, toggle fetchDetails, run. Pricing is pay-per-result: a fraction of a cent per package, plus a trivial per-run start fee. A 500-package search with detail costs a few cents.

The concurrent detail-fetch orchestration is the real engineering here — building it to be both fast and rate-limit-safe is fiddly, and the managed actor already tuned it to 5-concurrent.

Common pitfalls

Specifics of Packagist that will trip you up:

• Null fields mean fetchDetails was off. The single most common surprise. If downloadsMonthly, dependents, license, authors, or requires are null, you ran with fetchDetails: false. Turn it on to populate them — at the cost of ~5× the runtime.
• requires and authors are JSON strings, not objects. They’re stored as escaped JSON so the full structure survives. Parse them (JSON.parse / json.loads) before you can read individual dependencies or author names.
• Vendor names are case-sensitive and exact. vendorPackages returns zero if the vendor name doesn’t match Packagist exactly — it’s lowercase and precise. Verify it against the package’s Packagist URL first.
• downloadsTotal vs. downloadsMonthly measure different things. Total is all-time and favors old packages; monthly reflects current adoption. For “what’s popular now,” rank by downloadsMonthly, not downloadsTotal.
• Search vs. popular cover different populations. search captures the long tail by keyword; popular is the highest-download packages across all categories. Use search for niche discovery, popular for ecosystem-wide trending — they answer different questions.
• Skip detail when you only need names. If you’re building a name list or basic stats, set fetchDetails: false for a 5× faster run. Only pay for detail when you actually need dependencies, license, or authors.

Wrapping up

Packagist’s open API makes PHP ecosystem data accessible, but the four-endpoint structure and the concurrent detail-fetch pattern are more work than they first appear. If you want one package’s data, curl the detail endpoint. If you want a ranked search, a vendor’s full catalog, or a repeatable dependency-audit feed with license and author data, a managed actor handles the endpoints, concurrency, and JSON parsing so your rows land clean.

▶ Open the Packagist Scraper on Apify — one run yields hundreds of PHP packages with downloads, dependencies, favers, license, and authors; no API key, no login. Export to CSV, JSON, or Excel, and start on Apify’s free monthly credit.