Building an Automated Reddit Sentiment Pipeline

Every morning before the market opens, a script on my server wakes up, scrapes a handful of stock-focused subreddits, and starts reading. Not literally reading — but close enough. It pulls the day’s hottest posts and comments, feeds them to an LLM, and asks a simple question: what is this crowd feeling about which tickers, and how confident should I be?

The pipeline has three stages. First, ingestion: pull posts via the Reddit API, filter noise (bots, mega-threads, obvious spam), and bucket mentions by ticker. Second, synthesis: rather than a naive keyword sentiment score, I pass aggregated comment clusters to an LLM with a tight prompt asking for a directional read and a confidence level, grounded in the actual text rather than vibes.

Third, delivery: results get formatted and pushed to a private Discord channel via webhook, so I get a readable digest before the opening bell instead of a raw CSV. The whole thing runs unattended on a cron schedule.

The part that took the longest to get right wasn’t the scraping or the LLM prompt — it was deciding what to ignore. Subreddits are loud, and most of the loudest posts are the least informative. Filtering for signal over noise ended up mattering more than any model tweak, and it’s the main reason the recommendations turned out usable rather than just entertaining.