Andrej Karpathy Built the YouTube Filter I Always Wanted
AI researcher Andrej Karpathy used Claude to build HackerTube, a site that surfaces timeless YouTube videos from 20 years of Hacker News comments.
The short version
Andrej Karpathy, one of the sharpest minds in AI, built a website called HackerTube. It finds the most-recommended YouTube videos from 20 years of Hacker News comments. This little weekend project is a perfect example of using AI for a real, focused problem instead of just chasing hype.
What is HackerTube, exactly?
It’s a website that looks deceptively simple. You go there, and you see a list of YouTube videos with titles like “A Programmer’s Guide to Western Art” or “The Longest Way,” a film about a guy walking across China. No trending topics. No sensational thumbnails. Just a list of things that a very specific community has found consistently valuable over a long period.
The magic of HackerTube is its source. The videos are curated from two decades of comments on Hacker News. If you’re not familiar, Hacker News is a simple forum run by the startup incubator Y Combinator. It’s where programmers, founders, and tech folks discuss the industry. The comment threads are legendary for being a goldmine of obscure knowledge and deep-cut recommendations. People share the single best resource they’ve ever found on a topic, and HackerTube is a project to automatically find and rank those recommendations.
This whole thing was built by Andrej Karpathy. He’s a founding member of OpenAI and was the head of AI at Tesla. For him to spend a weekend building this tells you something. He didn’t set out to boil the ocean or create a YouTube competitor. He just wanted to solve a personal problem: finding the signal in twenty years of noise. He built the tool he wished he had. It turns out, it’s the tool a lot of us wish we had.
How did he actually build it?
This is the most interesting part, because it’s a masterclass in practical data processing. You don’t start with the fancy, expensive AI. You start with the simplest tools that can do the job.
First, Karpathy got his hands on the entire 43 million comment archive from Hacker News. A massive blob of text. Trying to run an AI model over all of that would be incredibly slow and expensive. So he didn’t. He started with simple Python code to just scan for YouTube links. That’s step one. Find the needles in the haystack before you start asking which needles are the sharpest.
Next came the brilliant filtering rule. For a video to even be considered, it had to be recommended by at least three different people in three different months. Think about what that does. It instantly kills spam and self-promotion. It filters out videos that were just part of a fleeting news cycle. This simple rule is a proxy for timelessness. A video that people are still independently recommending months or years apart has proven its value. It has staying power. This is curation by consensus, stretched over time.
This filtering process narrowed the field from millions of comments down to about 40,000 potential video recommendations. Now the data was much cleaner, but still too large for a human to sift through. This is where the AI comes in. He used Anthropic’s large language model, Claude, to act as a judge. Karpathy fed each of the 40,000 comments to Claude and asked a simple question: Is this a positive recommendation? A negative one? Or just a neutral mention? Claude read the context—phrases like “this is the best explanation I’ve ever seen” or “avoid this video, it’s wrong”—and assigned a score.
The final website is just the result of that process. A list of videos that passed the simple code filter and then got a thumbs-up from the AI judge. It’s a beautiful pipeline: use cheap, fast code for the 99% of work that is just brute-force filtering, then deploy the smart, expensive AI for the final 1% of nuanced, qualitative judgment.
Why is this so much better than YouTube’s own algorithm?
YouTube’s algorithm knows what you watched thirty seconds ago. It knows what’s popular right now. Its goal is to keep you on the site for one more click, one more video. It is a machine optimized for immediate engagement, not long-term value. Ask YouTube for the best introduction to calculus, and it will likely show you a dozen videos posted in the last month with flashy graphics and click-friendly titles. It’s a popularity contest where the votes are cast today.
HackerTube is the complete opposite. It’s a system designed to find consensus over decades. It actively ignores what’s popular today. Its single-minded goal is to surface videos that were so good, a smart person took the time to recommend them to another smart person years after the video was published. It’s an algorithm for durability.
I run into this problem constantly. I’ll be trying to learn a new skill, maybe something about audio engineering or a specific coding library. My first stop is YouTube, and I’m met with a firehose of content. Most of it is shallow. A lot of it is just re-packaged versions of other videos. Finding the original, canonical explanation from a true expert is a job in itself. Hacker News, and by extension HackerTube, solves that by relying on a community of experts to do the filtering for you. It finds the video someone on a forum posted with the comment, “Forget all that other stuff. This is the one you need to watch.”
What does this tell us about using AI effectively?
This project is more than just a cool website. It’s a blueprint for how to use modern AI tools to solve real problems. The most important lesson is about restraint. Karpathy didn’t throw Claude at the entire 43-million-comment database. He used simple, traditional programming to do the heavy, dumb work first. The AI was the last step, the fine-toothed comb used after the bulldozers had cleared the area. This efficiency is something I see missing from so many AI projects, which try to make the model do everything.
Andrej Karpathy’s project shows a practical way to get value from huge datasets. His method of combining simple scripts with powerful LLMs is something anyone can learn from, and he even detailed his write-up about the process for others to follow.
It also highlights a shift in what’s possible for a single person. Building HackerTube was a “weekend hack.” A few years ago, a project involving this level of data processing and qualitative analysis would have required a team and significant resources. Today, thanks to accessible LLM APIs, one developer can build a genuinely useful, data-driven product in a couple of days. The AI acts as a force multiplier for individual creativity. It’s not about building a sentient robot; it’s about having a tireless, super-smart intern who can read 40,000 comments without getting bored.
So, what should you do with this? First, go browse HackerTube. Don’t just glance. Click on a video that looks vaguely interesting and give it five minutes. I bet you’ll feel the difference in quality. It feels like a library curated by experts, not an infinite scroll designed by an attention merchant.
Second, and more importantly, steal this pattern. Think about the firehoses of information in your own life or work. Is it a customer support inbox? A backlog of user research notes? A community Slack channel? The recipe is the same. Find your high-signal source of data. Apply a few simple, hard-coded rules to filter out the noise. Then, use an LLM to perform the final, nuanced sort. You can ask it to categorize feedback, summarize long threads, or, like Karpathy, just identify positive sentiment. This is one of the most practical and powerful applications of AI today, and it’s available to anyone with a little bit of code.
FAQ
Who is Andrej Karpathy? Andrej Karpathy is a world-renowned AI researcher. He was a founding member of OpenAI, the Director of AI at Tesla, and is known for his ability to make complex AI concepts accessible to a broad audience.
What is Hacker News? Hacker News (HN) is a social news website focused on technology and entrepreneurship, operated by the Y Combinator startup incubator. Its comment sections are famous for their high level of technical discourse.
What is Claude? Claude is a large language model created by Anthropic, an AI safety and research company. It’s a direct competitor to models like OpenAI’s GPT-4 and is capable of sophisticated conversation, text analysis, and generation.
Can I build something like this myself? Yes. Karpathy himself described this as a “weekend hack.” The core logic—filtering a large dataset with simple scripts and then using an LLM API for nuanced analysis—is a pattern that any developer can replicate for their own projects.