Guide

Is there a Hacker News API?

There are two, and the official one is the wrong tool for almost every question people actually ask.

There are two, and the good one is not the official one

Hacker News has an official API. It is free, needs no key, and it is a poor fit for almost everything people want to do with it.

It is built around individual items fetched by id. To find stories about a topic you would walk the id space and filter client side, which means thousands of requests to answer one question. There is no search in it at all.

The thing people actually end up using is the search index that powers the site's own search box. It has full text search, filters on points, comments, author and date, and it returns results rather than ids. That is why every tutorial quietly switches to it halfway through.

What that means for your rate limit

The official API has no documented limit because its shape makes hammering it the normal case. The search index does have limits and they are not generous, because it is there to serve a website rather than your batch job. A loop that pages through a year of stories will meet them.

The practical consequence is the same as everywhere else on this site: it works on your laptop for a few hundred results and starts failing on a server for a few thousand.

Three things worth knowing before you build on it

  • Points and comment counts move. A story scraped an hour after posting and the same story scraped a day later are different rows. If you are ranking anything, decide whether you want the snapshot or the settled number, and say so in your data.
  • Comments are a tree, not a list. Every comment has a parent, and a flat export loses the conversation. For sentiment work that is usually fine. For anything about how a discussion went, it is not.
  • Deleted and flagged items. They vanish from the index but the ids stay. Your totals will not match what you can see on the site, and that is normal rather than a bug.

What people use it for

Watching whether a product, a company or a technology gets mentioned, and how that changes over months. Reading what developers say about a competitor in the comments, which is blunter and more useful than any survey. And finding the threads where a specific problem was discussed, which is a research shortcut rather than a monitoring job.

Where a dataset helps

If you need a few dozen results once, use the search box on the site and copy them. If you need the same query every week, or thousands of rows with the fields already separated, the plumbing is the job and that is what a dataset removes.

The datasets behind this