How to Scrape the Federal Register: Rules & Notices 2026

If you track federal regulation for a living, you know the Federal Register website has a search box but no export button. You can find the rule you want, but getting a month of EPA final rules into a spreadsheet means copying document numbers by hand or writing a paginator against an API you have to reverse-engineer first. This guide covers what the Federal Register actually exposes, how its official public API is structured, and how to pull thousands of rules, proposed rules and notices in one pass without an API key.

The Federal Register is the official daily journal of the US federal government. Every agency rulemaking, public notice, executive order and regulatory proposal must be published there before it takes effect — over 80,000 documents a year. That makes it the authoritative source for compliance monitoring, regulatory intelligence and legal research. The good news for data work: the underlying API is free, keyless, and built for programmatic access.

What’s worth extracting

Each document record the scraper returns is a flat, ten-field row pulled straight from the official API:

• documentNumber — the official Federal Register document number, e.g. 2024-12345. This is your stable join key; the title and URL can shift, the document number does not.
• title — the full document title as published.
• type — one of RULE, PRORULE, NOTICE, or PRESDOCU. This single field lets you separate binding regulation from drafts and announcements.
• abstract — the agency’s brief summary of the document. Enough to triage relevance without opening the full text.
• agencies — a comma-separated list of the issuing agencies. Many documents are issued jointly.
• publicationDate — publication date in YYYY-MM-DD.
• effectiveDate — the date the rule takes legal effect, when applicable (blank for notices and most proposed rules).
• htmlUrl — the full document page on federalregister.gov.
• pdfUrl — the direct PDF on the GPO’s govinfo servers.
• citation — the official citation, e.g. 89 FR 12345, which is what legal briefs and dockets actually reference.

For most compliance and monitoring use cases the minimum viable record is documentNumber, title, type, agencies, and publicationDate. Add abstract and citation when you’re building a research corpus or feeding an NLP pipeline.

How the source actually works

The scraper is a thin, honest wrapper over the official Federal Register API:

https://www.federalregister.gov/api/v1/documents.json

This is a completely public, keyless government API maintained by the US Government Publishing Office (GPO). It is designed for programmatic access and documented at federalregister.gov/developers/api/v1. You can verify the no-key claim yourself in a browser:

https://www.federalregister.gov/api/v1/documents.json?conditions[term]=test&per_page=5

The API paginates results up to 1,000 documents per call and returns total_pages in every response, so the scraper knows exactly how many pages remain and walks them until it hits your maxResults cap or exhausts the matches.

Input is a small set of stackable filters. The mode field switches between two behaviors: search runs a keyword query, and agencyDocs returns everything an agency has published recently without a keyword. A keyword search looks like this:

{
  "mode": "search",
  "query": "safety",
  "maxResults": 1000
}

All filters combine. The API treats multiple values of the same filter as OR (any match) and different filters as AND (all must hold), so you can scope a query tightly:

{
  "query": "clean air",
  "agencies": ["epa"],
  "types": ["RULE", "PRORULE"],
  "dateFrom": "2024-01-01",
  "dateTo": "2024-12-31",
  "maxResults": 500
}

Agency and type codes

Agencies are addressed by lowercase, hyphenated slugs. Some are short (epa), most are descriptive (securities-and-exchange-commission, occupational-safety-and-health-administration, homeland-security-department). The full list lives at federalregister.gov/api/v1/agencies.json, or you can read a slug off any agency page URL after /agencies/. The API accepts partial slug matching.

Document types are four codes worth internalizing:

• RULE — Final Rules. Legally binding regulations that have completed the rulemaking process.
• PRORULE — Proposed Rules (NPRMs). Drafts open for public comment, not yet binding. This is the one lobbyists and compliance teams watch during comment windows.
• NOTICE — Notices. Announcements, meetings, grant awards, information collections.
• PRESDOCU — Presidential Documents. Executive orders and proclamations.

▶ Run the Federal Register Scraper on Apify — keyword or agency in, up to 10,000 rules/notices out with citations and PDF links. No API key, no login. Export to JSON, CSV or Excel.

The access reality

This is the rare scraping target with almost no friction, and it’s worth being clear about why. The Federal Register API is a public-domain government service with no authentication and no hard rate limit for reasonable usage. There is no anti-bot stack, no TLS fingerprinting, no CAPTCHA. Datacenter proxy is fine — the API is not blocked from cloud IPs. Content is US government work, which is not subject to copyright, so the legal position for reuse is unusually clean.

The real constraints are volume and shape, not access:

• Per-page cap of 1,000. The scraper paginates for you, but a broad term like "regulation" can match hundreds of thousands of documents. Always set maxResults (1–10,000) to bound the pull.
• Coverage window. The API covers documents from 1994 onward, with full-text search available from 2000 onward. Older material predates the digital archive.
• Publication cadence. The Federal Register publishes on federal business days, and documents typically appear in the API within 24 hours of publication. There is no intra-day streaming — a daily run is the natural rhythm.

Throughput is roughly 1,000–5,000 documents per minute, and 512 MB of memory handles any query size because the payloads are compact JSON.

Example output

One NOTICE record, straight from the schema:

{
  "documentNumber": "2024-28147",
  "title": "Artificial Intelligence Risk Management Framework for Federal Agencies",
  "type": "NOTICE",
  "abstract": "This notice announces the availability of an updated risk management framework for AI systems used in federal decision-making.",
  "agencies": "Office of Science and Technology Policy",
  "publicationDate": "2024-11-15",
  "effectiveDate": "",
  "htmlUrl": "https://www.federalregister.gov/documents/2024/11/15/2024-28147/artificial-intelligence-risk-management-framework",
  "pdfUrl": "https://www.gpo.gov/fdsys/pkg/FR-2024-11-15/pdf/2024-28147.pdf",
  "citation": "89 FR 90512"
}

Use cases

Who pulls Federal Register data programmatically, and why:

• Compliance teams tracking every final rule from the EPA, OSHA or FDA that touches their industry. Run types: ["RULE"] scoped to your agencies on a dateFrom delta.
• Policy and legal researchers monitoring regulatory activity across the government, exporting citations directly into briefs and memos.
• Lobbyists and advocacy groups watching PRORULE documents during their comment periods so they don’t miss a window.
• Journalists covering federal policy who need a searchable, dated feed rather than a manual site check.
• Data scientists building regulatory datasets for NLP and ML — the abstract plus htmlUrl gives you labeled text and a link to the full document for bulk retrieval.
• Government contractors staying ahead of procurement-related notices before they close.

Cost and effort math

Building this yourself is genuinely easy — it’s a keyless, well-documented API — but the effort is not zero. You still write the paginator, handle the total_pages loop, normalize the ten fields, dedupe on documentNumber, and schedule the delta runs. That’s an afternoon to build and a small tax to maintain every time you want a new agency or type filter.

The managed scraper collapses that to an input form and pay-per-result pricing: a tiny per-run start charge plus a fraction of a cent per document returned, and runs that yield zero documents cost effectively nothing. For a compliance monitor pulling a few hundred new documents a day across a watchlist of agencies, you’re at low single-digit dollars a month with no proxy bill and no maintenance. The API being free means the only thing you’re paying for is the wrapper, the pagination, and not having to think about it.

Common pitfalls

A few things specific to this source that will bite you if you don’t plan for them:

• Unbounded keyword queries. Terms like "regulation", "safety" or "health" match enormous result sets. Always pair a broad query with dateFrom/dateTo or an agency filter, and cap with maxResults.
• effectiveDate is often blank. Notices and most proposed rules have no effective date. Don’t treat an empty string as a data error — it’s semantically correct.
• Joint issuance. agencies is a comma-separated string, not a single value. A rule issued by two agencies appears once with both listed. Split on comma if you need per-agency counts, and dedupe on documentNumber so joint documents aren’t counted twice.
• Slug precision. epa works, but many agencies need the full slug (federal-communications-commission, not fcc). When a run comes back thin, check the slug against agencies.json first.
• Full text is not in the record. The API returns the abstract plus htmlUrl, not the complete document body. For bulk full-text extraction, feed the htmlUrl values into a web-to-text scraper as a second stage.

Wrapping up

The Federal Register is one of the cleanest public data sources on the open web: a free, keyless, public-domain API with predictable pagination and no anti-bot friction. If you only need one query occasionally, the API is simple enough to hit directly. If you’re running a compliance monitor or building a regulatory dataset, the value is in the delta-scheduling, the stacked filters and the flat ten-field schema handled for you.

▶ Open the Federal Register Scraper on Apify — search by keyword, agency, type and date; get document numbers, abstracts, citations and PDF links in one run. Keyless, public-domain data. Start on Apify’s free monthly credit.