Persistent movie data
Structured resources preserve Letterboxd profiles, watchlists, viewing histories, ratings, social graphs, and films in PostgreSQL for reuse across requests.
P—003 / Data infrastructure
Watchboxd
01 / Overview
A persistent scrape-and-cache API that transforms Letterboxd profiles, watchlists, viewing histories, ratings, social graphs, and films into structured resources backed by PostgreSQL.
- Contributors
- Sanjay Kumar
- Project dates
- May 2025 - August 2026
02 / Tools & technologies
07 tools- Next.js
- Prisma
- PostgreSQL
- TypeScript
- Vercel Queues
- Supabase
- TMDB
03 / Technical highlights
Design / ImplementationDurable background jobs
A hierarchical job system combines Vercel Queues and PostgreSQL with database-enforced deduplication, idempotent queue delivery, transactional job claiming, bounded fan-out, retry policies, and atomic snapshot persistence.
Stale-while-revalidate caching
Resource-specific cache lifetimes allow stale data to be served immediately while deduplicated background jobs refresh Letterboxd and TMDB data. Valid snapshots are replaced atomically, avoiding partial updates.
Rating analysis
An ordinary least squares model characterizes how individual ratings vary relative to global movie ratings, providing a foundation for normalized taste comparisons and personalized recommendation features.