How to Scrape SEC EDGAR Full-Text Filing Search in 2026

The SEC’s EDGAR full-text search is one of the most valuable public datasets on the internet and one of the most awkward to use in bulk. You can type a phrase into the web UI and see filings that contain it, but turning that into a dataset — every 10-K mentioning “artificial intelligence risk,” every 8-K naming an acquisition target — means hitting the same Elasticsearch backend the UI calls, paginating through it, and parsing hits into structured rows. This guide covers how EDGAR’s full-text search engine (efts.sec.gov) actually works, its real limits, and how to pull filing metadata at scale.

What’s worth extracting

Each matching filing document returns one row with up to 14 fields, grouped by use:

• Company identity — companyName (full registered name), ticker (one or more symbols, e.g. “AAPL” or “LAAI, VEST”), cik (the SEC Central Index Key — the stable company identifier).
• Filing metadata — formType (10-K, 8-K, S-1, 10-K/A, etc.), filingDate (when submitted, YYYY-MM-DD), periodEnding (the period covered, e.g. fiscal year end), accessionNo (the unique filing ID like “0001477932-26-002727”).
• Document links — fileUrl (direct URL to the primary filing document) and edgarUrl (the EDGAR index page for the full filing package).
• Descriptors — fileDescription (human-readable, e.g. “FORM 10-K ANNUAL REPORT”), bizLocations (business/mailing address from the header), incStates (state(s) of incorporation), sics (SIC industry code(s)).
• Ranking — relevanceScore (the Elasticsearch relevance score — higher means a closer match to your query).

The cik is your durable key for a company, and accessionNo uniquely identifies each filing. For most research you sort by relevanceScore to get the most on-topic filings first.

How EDGAR full-text search works

The actor queries efts.sec.gov — the same EDGAR Full-Text Search (EFTS) engine behind the official search interface. It’s an Elasticsearch index, so your query behaves like a search query, not a database filter. Results paginate via a from parameter, 100 hits per page, and the actor loops until it hits maxResults.

A keyword search across all form types:

{
  "query": "supply chain disruption",
  "maxResults": 500
}

Two filters sharpen the search. forms restricts to specific form types (empty means all forms); dateFrom / dateTo bound the filing date:

{
  "query": "cryptocurrency exposure",
  "forms": ["10-K", "8-K", "S-1"],
  "dateFrom": "2023-01-01",
  "dateTo": "2024-12-31",
  "maxResults": 1000
}

Because the backend is Elasticsearch, the query field supports the operators you’d expect. Quoted phrases ("deepfake risk") match exact language; boolean operators ("supply chain" AND "force majeure", "cybersecurity" OR "data breach") combine terms. That’s what lets you find filings using a precise regulatory or narrative phrase rather than everything loosely related.

Picking the right form types

The forms filter is where a search goes from noisy to targeted, because different forms serve different disclosures:

• 10-K — the annual report. Contains the most detailed risk disclosures (Item 1A), so it’s the go-to for risk-factor and narrative analysis. Use it for year-over-year language comparisons.
• 8-K — material-event disclosures, filed within days of the event. Best for near-real-time monitoring of acquisitions, executive changes, and other one-off events.
• S-1 — IPO registration statements. Useful for studying companies at the moment they go public — what they disclose about risk, revenue and holdings.
• DEF 14A — proxy statements. The place to find executive compensation, shareholder proposals and activist-investor references.
• 20-F / 6-K — the foreign private issuer equivalents of the 10-K and periodic reports. Filter to these when your study includes non-U.S. companies.

Leave forms empty to search every form type at once, then narrow once you see where the results concentrate.

▶ Run the SEC EDGAR Full-Text Search Scraper on Apify — search all SEC filings by keyword, filter by form type and date, and export company + filing metadata to CSV or JSON. No API key, no login.

The access reality

EDGAR is a U.S. government public database maintained for investor access, and it’s refreshingly unhostile to automation. There’s no bot protection and no login. But there are real boundaries you have to design around:

• The 10,000-result Elasticsearch ceiling. Any single query returns at most 10,000 results — that’s a hard limit of EDGAR’s ES index, not the scraper. If a broad query would exceed it, split by form type or date window (e.g. year by year) and combine the datasets. A full 10,000-result run makes 100 API calls at 100 hits each.
• A descriptive User-Agent is mandatory. The SEC’s access policy requires every automated client to send a meaningful User-Agent header identifying itself. The actor supplies one automatically; requests without one get rejected.
• Rate limits are lenient but real. The SEC recommends no more than 10 requests per second. The actor throttles automatically and won’t hit the limit under normal use, but this is a government resource — don’t hammer it.
• Full-text coverage isn’t the whole archive. The EDGAR database holds filings from 1993 onward, but full-text search depends on the document being submitted in text/HTML. Most filings from 1996 on are fully indexed; very old scanned PDFs may not be searchable at all.
• Metadata fields aren’t always populated. Foreign issuers often lack U.S. state fields; reporting-only entities may have no ticker. Expect nulls and handle them.

Datacenter proxies work fine — there’s no reason to spend on residential IPs for a public government API.

Example output

One filing record from a keyword search:

{
  "companyName": "Microsoft Corporation",
  "ticker": "MSFT",
  "cik": "0000789019",
  "formType": "10-K",
  "filingDate": "2024-07-30",
  "periodEnding": "2024-06-30",
  "accessionNo": "0000950170-24-087843",
  "fileUrl": "https://www.sec.gov/Archives/edgar/data/789019/000095017024087843/msft-20240630.htm",
  "edgarUrl": "https://www.sec.gov/Archives/edgar/data/789019/000095017024087843/000095017024087843-index.htm",
  "fileDescription": "FORM 10-K",
  "bizLocations": "Redmond, WA",
  "incStates": "WA",
  "sics": "7372",
  "relevanceScore": "22.4"
}

The fileUrl points straight at the primary document; edgarUrl gives you the full filing index if you need exhibits.

Who uses this

• Financial analysts and hedge funds tracking narrative shifts — AI risk, ESG disclosures, supply-chain language — across thousands of annual reports at once.
• Compliance and legal teams monitoring how specific regulatory language appears in filings across sectors.
• Journalists and investigative reporters searching for companies that mention a specific topic, event, executive or counterparty.
• Academic researchers building SEC-disclosure datasets for NLP, topic modeling and event studies.
• Corporate-intelligence professionals benchmarking competitors by searching for rival names or product mentions inside filings.

Concrete jobs this handles: find every 10-K mentioning “artificial intelligence risk” to build a risk-disclosure dataset; reconstruct an M&A timeline from 8-K filings naming a target; track which companies disclosed “cryptocurrency” holdings in S-1 prospectuses in a given year.

Cost and effort: build vs. managed

Building this yourself is more involved than it looks. You have to reverse the EFTS request format, paginate correctly with from, parse the Elasticsearch hit structure into the 14-field record, construct the document and index URLs from the accession number, set a compliant User-Agent, throttle under the rate limit, and split queries around the 10,000-result cap. Then you maintain it whenever the SEC adjusts the endpoint.

The managed actor is pay-per-result: a fraction of a cent per filing row plus a trivial start fee. A 200-result run completes in under 30 seconds; a full 10,000-result run takes about five minutes. With a public API and datacenter proxies, the entire cost is essentially per-row — market-wide studies across thousands of filings stay cheap.

Common pitfalls

EDGAR-specific things to get right:

• Split around the 10,000 cap up front. For broad queries, partition by year or form type before you run — hitting the ceiling mid-extraction silently truncates your dataset.
• 10-K and 10-K/A are different forms. An amendment (/A suffix) is a distinct form type. If your forms filter is too narrow you’ll miss amendments; if you get zero results, loosen the form filter first.
• Quote phrases for exact language. Unquoted multi-word queries match loosely. "force majeure" finds the exact phrase; force majeure matches filings containing either word.
• Sort by relevanceScore, not filing date, for topical work. The highest-scoring hits use your query language most directly. Date-sorting is for timelines, not topical relevance.
• Nulls are normal for foreign and reporting-only entities. Missing ticker, incStates or sics reflects the filer type, not a scrape failure.
• CIK is the company key; accession number is the filing key. Use cik to pull all of a company’s filings; use accessionNo to uniquely reference one filing or fetch its exhibit list.
• Full-text ≠ full history. Pre-1996 filings may not be text-indexed, so a phrase that existed in a 1994 filing might not surface. The searchable window is narrower than the archive window.

Wrapping up

EDGAR full-text search is a public, government-maintained goldmine with no access wall — the friction is entirely in the mechanics. Reversing the EFTS request format, paginating past its quirks, building URLs from accession numbers, and navigating the 10,000-result ceiling is real work you’d otherwise own forever. If you need a handful of filings, the web UI is fine. If you need a structured dataset of filings matching a phrase across form types and years, exported to a spreadsheet, a managed actor carries the plumbing for a fraction of a cent per row.

▶ Open the SEC EDGAR Full-Text Search Scraper on Apify — one run yields structured filing metadata — company, ticker, CIK, form type, dates and document URLs — for every filing matching your query, exportable to CSV, Excel or JSON. No API key, no login. Start on Apify’s free monthly credit.