Econ Dashboard

CASE STUDY

Economists have to stay current: papers, working-paper series, newsletters, data releases, commentary, all scattered across platforms, easy to miss and easier to forget. This pipeline turns the 15+ sources I checked by hand into one verified morning read, searchable forever.

View the code on GitHub →

46 sources · 0 failuresevery source URL verified live; latest full run clean

39.1 s → 63 msfull background refresh; dashboard answer time

1,283 itemsarchived and full-text searchable after the first 5 days

01 · The problem

Staying current was a job of its own

I followed 15+ economics sources by hand across four surfaces: a bookmarks folder, 7 Substack newsletters in my inbox, NBER's weekly abstracts email, and 2 YouTube channels. Checked daily, with no memory of what I had already seen and no way to search across any of it.

02 · How it works

Fetch, rank, label, serve

Stage 0 · Python

Fetch

46 sources (RSS, an undocumented NBER JSON API, arXiv, World Bank API, RePEc digests, a Google News proxy for scraper-blocking sites) land in a local SQLite archive with full-text search.

Stage 1 · Deterministic

Rank

Cross-source phrase-surge detection builds a weekly trending panel; a scored daily shortlist displays the exact reasons every pick was chosen.

Stage 2 · Claude

Label

The LLM is used only where semantics genuinely need it: labelling trend clusters the data already found, and scoring papers 0-10 for the development-economics x AI intersection, justification shown beside every paper.

Stage 3 · PWA

Serve

A vanilla-JS dashboard, no framework and no build step, installable as a PWA. A scheduled job pre-fetches everything at 6:30am, before I'm up.

Items are classified once and cached, so a page reload never re-bills the API, and unparseable dates raise instead of defaulting: that rule exists because a lenient parser once reported "0 posts in 30 days" for a feed that publishes daily. That bug is now a regression test.

03 · Design decisions

Failures are loud; the LLM never decides

ReliabilityFailures are loudEvery fetch outcome is recorded per source. A broken feed renders as "source failed" in the UI, so an empty section can never be mistaken for a slow news day.

AI disciplineThe LLM labels, it never decidesThemes and candidates are found deterministically first; model output is grounded in shown evidence, and mid-confidence papers go to a visible "possibly relevant" tray instead of being silently included or dropped.

Human in the loop throughout: I choose the sources, papers scored 4-6 are left to my judgment, saving and note-taking are manual, and nothing is auto-published anywhere.

04 · Results

Measured from real runs

647items on first fetch, in 12.2 s

134automated tests, none needing network

18papers LLM-scored; 7 verified at 7+/10

Time saved per day: not measured. API cost: bounded by design to roughly one labelling call per fetch plus one-time scoring per new paper. And one real data-integrity incident, 248 duplicate rows from a mid-project hashing change, was detected, fixed, migration-tested, and documented.

05 · Under the hood

Stack

Python 3.13 · stdlib server SQLite · FTS5 Claude API · structured outputs Vanilla JS · PWA pytest · 134 tests

Public code, keys and personal reading data excluded, on GitHub.