All projects

Team project, Columbia University, May 2026

ELI5: Explain Like I'm 5

A news pipeline that turns dense articles into plain-language summaries.

From headline to plain English

  1. News APIs
  2. MongoDB Atlas
  3. PySpark
  4. PostgreSQL
  5. AI summaries
  6. Web app

In brief

The question
News is fast, dense and full of jargon. How do you turn thousands of articles a day into something anyone can understand?
The approach
A data pipeline that pulls articles from three news APIs, processes them with Spark, stores them in a cloud database, and generates simple summaries with an AI model, served through a web app.
The result
A working app processing over 20,000 articles from three sources, with pre-generated summaries, a switch back to the original article, and follow-up questions answered by searching across articles.

The idea

Most readers scroll past articles they don't fully understand. ELI5 takes real-time news from three sources and simplifies it: read the plain version, switch to the original, or ask a follow-up question.

How the pipeline works

  • Ingestion from three news APIs: The Guardian, GNews and NewsData.io.
  • Raw articles land in MongoDB Atlas, whose flexible schema copes with inconsistent JSON from different sources.
  • PySpark on Databricks standardizes the articles and loads them into PostgreSQL on Supabase, whose structured schema makes filtering by category, date and source fast.
  • Claude Haiku writes the plain-language summaries, and Pinecone, a vector database, supports search for follow-up questions.
  • A Flask front end displays the summarized articles.

Decisions and why

  • Each store was chosen for its job: a document database for messy raw data, a relational one for fast filtered reads, a vector one for semantic search.
  • The team weighed the cost of each choice too: Spark is more than a small dataset needs but scales, and per-call AI costs add up at volume.
  • A production version would add reading preferences, engagement data and subscriptions to personalize the feed.