NotesJanuary 20, 2025· 5 min
Replacing 15 hand-written parsers with one LLM ranker
The maintenance tax
The Superjoin Chrome extension needed to extract structured data from web pages: tables, lists, repeated elements. Each website had its own DOM structure, so we wrote custom parsers: one for Airtable, one for Notion, one for HubSpot, and so on.
Every time a site updated their markup, a parser broke. Fifteen parsers meant fifteen potential breakages per quarter.
Extract, then rank
The replacement is two stages:
- Extract: A general-purpose table extractor that finds all tabular data on a page, regardless of DOM structure.
- Rank: An LLM ranker that scores each extracted table by relevance to what the user is trying to import.
The extractor handles the structural variety. The ranker handles the semantic question of "which table does the user actually want?"