Hacker News Scraper

Turn Hacker News into clean, structured data. Run a keyword search across every story and comment, pull the live Top / New / Best / Ask HN / Show HN / Jobs front pages, look up user profiles, or grab a story with its full comment thread — and get one tidy row per item. Export to JSON, CSV or Excel, or pull straight from the API.

Perfect for tech trend monitoring, startup and product research, developer sentiment analysis, building datasets for LLMs, tracking mentions of your company or competitors, lead lists from "Who is hiring?" threads, and feeding Hacker News data into your own app, dashboard or workflow.

What you get

One flat, ready-to-use record per item — story, comment, job or user:

How to use

1. Search by keyword

Add one or more keywords and the scraper searches across all of Hacker News. Narrow it down with type, score, comment count and date filters.

{
  "searchQueries": ["openai", "rust"],
  "tags": "story",
  "minPoints": 100,
  "sortByDate": false,
  "maxItems": 200
}

• Search: item type — Stories, Comments, Ask HN, Show HN, Polls, or Stories + comments.
• Search: min points / min comments — only keep items above a threshold.
• Search: from / to date — restrict to a date range (YYYY-MM-DD).
• Search: newest first — sort chronologically instead of by relevance.

2. Front-page feeds

Scrape the live front pages — pick any combination:

{
  "feeds": ["top", "ask", "show"],
  "maxItems": 100
}

Top stories, New stories, Best stories, Ask HN, Show HN and Jobs are all supported.

3. User profiles

Look up any Hacker News user — karma, account age, about text and submission count:

{
  "usernames": ["pg", "dang"]
}

4. Specific stories & full comment threads

Paste story URLs or IDs and optionally pull every comment:

{
  "itemUrls": ["https://news.ycombinator.com/item?id=1"],
  "includeComments": true,
  "maxCommentsPerItem": 300,
  "maxCommentDepth": 10
}

Turn on Include comments with any front-page feed or specific item to add each comment as its own row, with reply depth preserved.

Example output

{
  "id": 1,
  "type": "story",
  "title": "Y Combinator",
  "url": "http://ycombinator.com",
  "hnUrl": "https://news.ycombinator.com/item?id=1",
  "author": "pg",
  "points": 57,
  "numComments": 3,
  "createdAt": "2006-10-09T18:21:51.000Z",
  "source": "feed:top",
  "collectedAt": "2026-09-20T01:30:00.000Z"
}

Common use cases

• Tech trend & topic monitoring — track how often a technology, framework or company is discussed and how sentiment shifts over time.
• Startup & product research — mine Show HN launches, Ask HN threads and comment discussions.
• Hiring intelligence — extract companies and roles from the monthly "Who is hiring?" threads.
• LLM & research datasets — build clean corpora of stories and comments with points, authors and timestamps.
• Brand & competitor tracking — get alerted to mentions and read the full discussion around them.
• Newsletters & dashboards — pipe the daily Top or Best stories straight into your own product.

Frequently asked questions

Do I need any login, cookies or tokens? No. Just set your inputs and run.

How many results can I get? Set Max items to whatever you need (0 = no limit). Keyword search paginates through all matches; front-page feeds cover up to 500 stories each.

Can I get every comment on a story? Yes — enable Include comments and set the per-item cap and depth. Comments come back as individual rows with parentId, storyId and depth.

What format is the data? A flat table you can export as JSON, CSV, or Excel, or fetch from the Apify API.

Is the data live? Yes — front-page feeds and items reflect Hacker News in real time; search covers the full history.

Related scrapers

Looking for more data sources? Check out our other scrapers on Apify Store for news, tech communities, job boards and developer platforms — all with the same clean, one-row-per-item output.

You pay only for what the run returns. Set a maximum cost per run and it stops cleanly at that limit.