Structured web data
46 datasets, all running on the Apify platform. Each one gives you rows with a fixed schema, which is the dull property that turns out to matter most: the parser you write today survives the next change upstream. When a row fails it arrives as a record with an error code. You are not charged for those.
ACTOR=steadydata~youtube-transcript-bulk URL="https://api.apify.com/v2/acts/$ACTOR/run-sync-get-dataset-items" curl -X POST "$URL?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"videos": ["https://youtu.be/dQw4w9WgXcQ"], "languages": ["en"]}' # rows come straight back as JSON, one object per delivered row
Pick the line that matches the job. They all bill the same way and return the same row shape, so moving from one to another costs you nothing to learn.
Transcripts, comments, channels and charts in bulk, for AI pipelines and content research.
11 datasetsTrends, autocomplete, news, jobs, shopping and local results, structured and repeatable.
15 datasetsApp Store, Google Play, Steam and Apple Podcasts metadata, rankings and charts.
3 datasetsDNS, SSL, deliverability, tech stack, sitemaps, redirects and page extraction.
10 datasetsShopify and WooCommerce catalogues and store reports.
4 datasetsSEC filings, finance quotes, careers pages, GitHub and Hacker News.
3 datasetsType what you need. The list filters as you type, so you do not have to guess which line it lives in.
Full App Store listing per app id, URL or bundle id, up to 500 apps per run, any storefront: rating and count, price, genre, age rating, version, release dates, size, minimum iOS, languages, description, release notes, artwork. From Apple's official lookup API. Pay per app.
$1.50 per 1,000 · App fetchedSearch the App Store for up to 200 keywords per run and get the ranked results with full listing data: position, title, developer, rating, price, genre, version and more, per storefront. Track your own apps' rank per keyword. Pay only for delivered rows.
$0.70 per 1,000 · App rankedOpen jobs from company boards on Greenhouse, Lever, Ashby, Workday, Workable, SuccessFactors, iCIMS, UKG, Oracle, Dayforce, JazzHR, BambooHR, Breezy, Paylocity, Teamtailor and Recruitee, up to 200 boards a run: title, department, location, posting date, salary, apply link. Pay per job.
$1.00 per 1,000 · Job listedOne health report per domain, up to 500 per run: registration and expiry from the official RDAP registry, DNS records with the mail and nameserver provider, SPF and DMARC policy, the TLS certificate with days remaining, and a ranked list of issues. Official protocols only.
$20.00 per 1,000 · Domain reportedOne deliverability report per domain, up to 500 per run: MX and mail provider, the SPF record with its DNS lookup count against the limit of ten, DKIM selectors that really exist, the DMARC policy and reporting, MTA-STS mode, TLS-RPT, BIMI and DNSSEC, plus ranked issues and a score.
$3.00 per 1,000 · Domain checkedDetails of any public GitHub repository, up to 500 per run: stars, forks, watchers, open issues, primary language, license, topics, default branch, size, homepage, created and last push dates, and the archived and fork flags, from GitHub's own API. No profiles, no emails. Pay per repository.
$0.40 per 1,000 · Repository detailedAds from Google's Ads Transparency Center per advertiser domain or advertiser id, per country or worldwide: creative id, advertiser name, preview image or preview URL, first and last shown dates and a link to the ad. Up to 200 advertisers per run. Pay only for delivered ads.
$1.40 per 1,000 · Ad listedGoogle's autocomplete suggestions for up to 500 seed keywords per run, per country and language, with optional a-to-z and question-word expansion for hundreds of long-tail keywords per seed. Ranked, with Google's relevance score and suggestion type. Pay only for delivered suggestions.
$0.35 per 1,000 · Suggestion listedThe cheapest fare per departure day for a route over a date range, from Google Flights: one row per day with the lowest price, airline, stops and duration, so you see when it is cheap. Up to 100 routes, up to 60 days each. Pay per delivered day.
$3.00 per 1,000 · Day pricedFlight options from Google Flights for up to 200 searches per run: price, airlines, flight numbers, departure and arrival times, duration, stops and layover airports, aircraft and CO2 estimate, in Google's order. One-way or round trip, any cabin class and currency. Pay only for delivered flights.
$1.40 per 1,000 · Flight listedThe hotel landscape of a city from Google Hotels, up to 200 cities per run: hotel name, Google rating, number of reviews, indicative nightly price and currency, in Google's own order. No dates, no booking, no personal data: the market overview, not the rate search. Pay per delivered hotel.
$4.00 per 1,000 · Hotel listedImage search results from Google per query: full-size image URL with width and height, title, source page, domain and file size. Up to 200 queries per run. Only delivered images are charged.
$1.40 per 1,000 · Image listedJob listings from Google Jobs per search query and location: title, company, location, source, posting date, salary when shown, and deduplicated apply links. Strict schema for pipelines and AI agents; only successfully delivered result pages are charged.
$20.00 per 1,000 · Result page deliveredLocal business results from Google per search query: name, rating, review count, category, price band, address and Google place id. Up to 200 queries per run. No phone numbers or contact details. Only delivered businesses are charged.
$2.00 per 1,000 · Business listedNews articles from Google News per search term, topic or country headlines: title, source, publication date, link and snippet. Up to 200 queries per run. Only delivered articles are charged.
$3.50 per 1,000 · Article listedOne structured record per patent publication number from Google Patents, up to 200 per run: title, abstract, assignee, country, priority, filing and publication dates, legal status and expiry, claim count, CPC codes and citation counts. No inventor names. Pay per delivered patent.
$5.00 per 1,000 · Patent deliveredFull Google Play listing per app id or URL, up to 500 apps per run and any country: title, developer, rating, ratings and reviews count, installs, price, in-app purchases, category, content rating, version, release and update dates, description. Pay per delivered app.
$2.00 per 1,000 · App fetchedProduct listings from Google Shopping per search query: title, current price, previous price, discount, merchant, delivery, returns, rating and review count, with the product id and URL. Up to 200 queries per run. Only delivered products are charged.
$1.75 per 1,000 · Product listedCompare up to 5 keywords in Google Trends, the way the Compare view does: relative interest over time for all terms on one scale, the average per term, the leader, and optionally which term wins in each country or region. Up to 200 comparisons per run. Pay per comparison.
$5.00 per 1,000 · Comparison fetchedWhere in the world, or where in a country, a keyword is searched: Google Trends interest by region for up to 200 keywords per run. Worldwide gives up to 250 countries per keyword; a country code gives its regions or cities. One row per keyword, regions ranked 0 to 100.
$2.00 per 1,000 · Keyword mappedBatch Google Trends data with a transparent success rate: interest-over-time and related queries per search term, with geo, timeframe and language control. Strict output schema for AI agents and pipelines; only successfully delivered terms are charged.
$3.00 per 1,000 · Search term deliveredTrending searches from Google Trends per country: every trend of the last 4 hours to 7 days with search volume, growth, start and end time, category and related queries. Up to 200 countries per run, pay only for delivered trends.
$0.60 per 1,000 · Trend listedText extraction (OCR) from images by URL, up to 200 per run: every line with its confidence and position, the full text joined, image size and format, for screenshots, scans, receipts, menus and product labels, without a browser or an API key. Pay per image with text.
$2.00 per 1,000 · Image readCore Web Vitals and Lighthouse scores for up to 300 URLs per run, from Google's own PageSpeed Insights API. Every row pairs the field data real Chrome users produced over 28 days (LCP, INP, CLS) with the lab scores and the heaviest fixes Lighthouse found. Bring your own free Google key. Pay per URL.
$1.40 per 1,000 · URL auditedPlain text extraction from PDF files by URL, up to 200 files per run: one row per page (or one per file) with the text, page count, title, author, dates, producer and file size, plus whether the page has a text layer at all. Pay per page delivered.
$1.00 per 1,000 · Page extractedRead up to 200 RSS or Atom feeds per run and get their items as clean rows: title, link, publication date, summary, categories and image. Give it a website instead of a feed URL and it finds the feed itself. Pay only for delivered items.
$0.40 per 1,000 · Item listedRecent SEC EDGAR filings per company by ticker or CIK, up to 200 companies a run and 1,000 filings each: form, filing and report dates, 8-K items, accession number, document and index links, plus SIC, state and exchange per company, from the SEC's JSON API. Filter on form and date. Pay per filing.
$1.00 per 1,000 · Filing listedProduct catalogue of any Shopify store from its public products feed, up to 200 stores a run and 5,000 products each: title, handle, URL, vendor, type, tags, dates, price range, sale and stock flags, every variant with SKU, price and options, images and description. Robots-aware. Pay per product.
$1.00 per 1,000 · Product listedOne report per Shopify store, up to 200 stores per run: catalogue size, price range and average, currency, share in stock, share on sale, product types, brands, newest and oldest product, plus ranked signals such as a stale catalogue or heavy discounting. Pay per delivered report.
$3.50 per 1,000 · Store reportedSitemap URL extractor: every URL a website lists in its XML sitemaps, up to 100 sites per run: sitemaps are found through robots.txt and the usual paths, sitemap indexes and gzip are followed, and each URL comes with its sitemap, last modification date, change frequency and priority. Pay per URL.
$0.50 per 1,000 · URL listedAudit up to 500 pages per run on the technical basics: title and description lengths, canonical, robots meta, hreflang, headings, word count, images without alt text, structured data, Open Graph, viewport and redirects, with a ranked list of issues per page. Respects robots.txt.
$30.00 per 1,000 · Page auditedThe main content of web pages as Markdown and plain text, up to 500 URLs per run: title, description, author, date, language, word count, links and images, without menus, cookie banners and footers, ready for LLMs, search indexes and archives. Pay per page.
$1.00 per 1,000 · Page convertedDetect the technologies behind up to 500 websites per run: CMS, ecommerce platform, analytics, tag managers, frameworks, server, CDN, payment providers, consent tools and more, with the evidence for every detection. Skips sites whose robots.txt forbids it, at no cost. Pay per site analysed.
$50.00 per 1,000 · Site analysedProduct catalogue of any WooCommerce store from its public Store API, up to 200 stores a run and 5,000 products each: name, URL, SKU, price, sale price, currency, stock and on-sale flags, rating, categories, tags, brands, attributes, images and description. Robots-aware. Pay per product.
$1.00 per 1,000 · Product listedOne report per WooCommerce store, up to 200 stores per run: the exact product count, price range and average, currency, share in stock, share on sale, categories, brands, average review rating, and ranked signals such as an empty catalogue or heavy discounting. Company data only, no personal data.
$3.50 per 1,000 · Store reportedChannel profiles for up to 500 YouTube channels per run, by handle, id or URL: name, handle, subscribers, exact total views and video count, country, description, verified badge, external links, avatar and banner. No e-mail addresses. Pay only for delivered channels.
$2.50 per 1,000 · Channel fetchedEvery stream on a YouTube channel's Live tab, up to 100 channels per run: live now with viewers watching, scheduled with the start time, and past streams with views and duration. One row per stream, in YouTube's order. No viewer or chat data. Pay per delivered stream.
$1.00 per 1,000 · Stream listedFind YouTube channels by keyword: channel id, name, handle, subscriber count, video count, verified badge and thumbnail, ranked as YouTube ranks them. Up to 200 keywords per run. No email addresses or contact details. Only delivered channels are charged.
$1.00 per 1,000 · Channel listedTranscripts for the recent videos of a YouTube channel in one run, up to 50 channels: give a handle or channel URL, get one row per video with full transcript text, language, duration and views. Videos without captions cost nothing. Pay per transcript.
$2.50 per 1,000 · Transcript fetchedList every video of a YouTube channel: title, video id, duration, view count, publish date and thumbnail, plus channel name, handle and subscriber count. Paste channel URLs, @handles or channel ids. Only delivered videos are charged.
$0.35 per 1,000 · Video listedComments from YouTube videos for sentiment, FAQ mining and research: text, likes, reply count, publication time, pinned and owner flags, top or newest first, up to 200 videos per run. Deliberately without commenter names, handles or channel ids. Pay only for delivered comments.
$0.35 per 1,000 · Comment listedEvery video in a YouTube playlist in order: position, title, duration, view count, publish date and thumbnail, plus the playlist title and owner. Paste playlist URLs or ids. Only delivered videos are charged.
$0.35 per 1,000 · Video listedRanked YouTube search results for up to 200 keywords per run: rank, title, video id, duration, view count, publish date and thumbnail. Sort by relevance, date, views or rating. Only delivered results are charged.
$0.40 per 1,000 · Result listedEvery Short of a YouTube channel: title, video id, view count and thumbnail, with the channel name on each row. Paste channel URLs, @handles or channel ids. Only delivered Shorts are charged.
$0.40 per 1,000 · Short listedFetch transcripts/subtitles for up to 500 YouTube videos per run. Returns plain text, timestamped snippets and optional SRT, with language selection. Built for AI agents and bulk pipelines: strict output schema, only successful transcripts are charged.
$2.50 per 1,000 · Transcript deliveredFull metadata for up to 500 YouTube videos per run: title, description, view count, duration, publish date, category, tags, channel and thumbnail. Paste URLs, short links, shorts or bare video ids. Only delivered videos are charged.
$1.50 per 1,000 · Video deliveredAnything works once. The interesting part starts when the same job has to run from a server every Monday morning, on an address the site has seen before.
A video with no captions, a domain that does not resolve, a store that went offline last week. Each one lands in your dataset with an error code. A run of ten thousand does not die on row four hundred.
Billing is per delivered row. Empty, failed and blocked results cost nothing, so the invoice and the data agree.
Stable field names, explicit types, an id on every row.
EDGAR's access rules want a User-Agent naming the party, with a contact address. Ours names the party and links the profile page, and so far every request comes back 200. It is not literally what they ask for, which is why it is on our list to fix.
Every dataset here shipped only after someone read the actual rows: dead columns, broken URLs, HTML entities left in the text. A run can finish successfully and still hand you a table you cannot use.
Feed in 500 domains where the limit is 200 and you used to get 200 rows and no explanation. The other 300 now come back as an error record that says so. You are billed for the 200 you received and nothing else.
Two terms fetched separately are each scaled against their own peak. Put them in one request or the comparison means nothing. This is the single most common reason a Trends dashboard is quietly wrong.
Written around questions people actually type. Each one starts with the free way to do it, because for a small job that is usually the right answer.
The free routes, the point where each one stops, and what it costs to pull transcripts for hundreds or thousands of videos.
What Google does and does not offer, why Trends numbers break when you compare them wrong, and the working routes.
Extensions, the official API and its quota, and how to pull comments for a whole channel without babysitting it.
The free RSS route, the Python route, and what changes once you need this every day without failures.
Why the indexing API is not what most people are looking for, and how to read Google for Jobs results instead.
Apple has an official lookup API, Google Play does not. What that means for your data, and the storefront mistake that quietly measures the wrong country.
Eight lookups per domain, the SPF chain that silently breaks past ten, and how to get one row per domain instead of eight tabs.
Why a valid-looking SPF record can be broken, why DKIM cannot be discovered from outside, and what p=none actually protects.
What tech lookup tools really do, the three things that make the answer wrong, and what it costs for a prospect list.
Your own store exports for free. For someone else's, decide first whether you need every product row or an answer at store level.
Digital versus scanned, the edge cases that ruin naive scripts, and the point where a library stops being worth the afternoon.
The User-Agent rule that stops most first attempts, the ten-per-second limit, and why starting from a ticker instead of a CIK causes silent mismatches.
There are two, and the official one is the wrong tool for almost every question people actually ask.