A hobbyist coder trained TimeCapsuleLLM on texts from 1800–1875 London and aimed to reproduce a convincing Victorian voice. In a surprising turn, the model produced a passage referencing real protests that occurred in 1834 London.
TimeCapsuleLLM is a small language model built by Muhlenberg College computer science student Hayk Grigorian. He trained it from scratch on more than 7,000 Victorian-era texts, totaling around 6.25GB of data, using a method he calls Selective Temporal Training to emphasize historical wording over contemporary language. The model tends to generate text heavy with biblical allusions and period-appropriate rhetorical flourishes.
In a simple test, Grigorian fed the prompt “It was the year of our Lord 1834” and the model produced a passage describing London protests and petitions.
It was the year of our Lord 1834 and the streets of London were filled with protest and petition. The cause, as many recounted, was not bound in the way of private, but having taken up the same day in the day of Lord Palmerston…
He notes that the output also mentioned Lord Palmerston, and after a quick Google search he confirmed that his actions during that period contributed to the 1834 protests.
Grigorian positions TimeCapsuleLLM within the broader movement of Historical Large Language Models (HLLMs). Other efforts include MonadGPT, which is trained on thousands of texts spanning 1400–1700 CE, and XunziALLM, which generates classical Chinese poetry following ancient rules. These projects aim to offer researchers a way to interact with historic linguistic patterns in a modern AI framework.
To build a historical model, Grigorian trained on Victorian-era sources—books, legal documents, and newspapers from 1800–1875—without incorporating modern vocabulary. He describes his process as Selective Temporal Training (STT) and uses a custom tokenizer to focus the model on historical language. He notes that early versions produced period-appropriate prose but often slipped into surface-level imitation rather than genuine historical reasoning; the current 700-million-parameter iteration appears more coherent and capable of placing events in a historical context.
While enthusiasts hail the potential of such period-specific models for digital humanities and interactive history, experts caution that even small models can hallucinate or misstate facts. Still, Grigorian’s experiment underscores an intriguing possibility: AI that can emulate extinct vernaculars and illuminate stylistic aspects of past eras, even if factual reliability remains a work in progress.
