Retina

What happens to one article

Retina is the lab’s news perception. It is built embedding-first: every article is turned into a vector before anything else happens, so that similarity, clustering and search all work on meaning rather than keywords. A language model then extracts the entities each article mentions (people, places, organisations, events, assets) and links them to a curated catalogue. How often those links are right was checked on test names in July; the numbers and their limits are below. Storylines are clusters of articles that stay together over days; bursts are names that suddenly outrun their own baseline; tone is a sentiment score per article, aggregated per entity.

Where the reports come from

Sources are ordinary: RSS from newspapers, broadcasters, agencies and specialist outlets, a set of public Telegram channels, and a few sources that are only reachable over Tor, routed through the lab’s own gateway. Nothing is scraped that a publisher hides behind a login. Retina keeps derived data; the articles stay with their publishers.

What it replaced, and what that fixed

Retina replaced a much larger predecessor on 26 March 2026: fifty-seven containers and more than two hundred tools, retired in one move for a single service with one job. The links did not come for free either. Until July 2026 articles were linked to entities by vector similarity. Checked on one test name, Iraqi Kurdistan, 271 of 11,267 linked articles actually contained the name: 2.4 %. A stricter threshold did not save it; at 0.80 it reached 27.7 % and kept less than one percent of the right links. The language-model extraction had been running alongside all the time, its results thrown away; since 18 July it is the only source of the links. The same check afterwards: Iraqi Kurdistan 95.2 %, Bandar Abbas 96.4 %, Strait of Hormuz 88.0 %. The check is deliberately simple, a name in the text, so it measures links that are wrong, not names the model missed. Before any request is made, posts that carry only a picture or a video, and consent, login and bot pages, are recognised and set aside: stored, but not read further. That was 4.9 % of the inflow in the fourteen days to 15 October, nearly all of it picture-only posts.

Five questions per article

Since 23 September the language model that extracts the entities also answers five short questions about nearly every new article: topic, reach, violence, harm to civilians, and whether it matters at all. What it judges unimportant is set aside, not deleted. Together with the pattern filter above, 12.4 % of the inflow was set aside in the fourteen days to 15 October: 4.9 % by pattern, 7.5 % by the model.

For one day before that, a small fine-tuned decision model did this job (see The Model Lab). In its first run, with a reach threshold, it set aside 42 % of two days’ inflow, and on 23 September it was switched off. On 180 headlines labelled by one rater, the agent, the large model caught 68 % of the junk and lost 2.5 % of the relevant articles; the small model with its refined rule caught 43 % and lost 2.8 %.