Formula Pipeline
Made formulas from textbooks structurally comparable across disciplines, with its own parser. Stopped in July 2026; the formulas are kept as an archive.
- Good for
- Makes formulas from textbooks of different fields comparable by their structure, with a parser of its own, as groundwork for spotting the same idea in two disciplines.
- Measured
- 225,607 formulas signed in the last full run on 26 July 2026, at 30 microseconds each.
- 274,244 formulas kept in a readable archive since the move to the new core.
- Open questions
- How many structural matches between fields are real analogies rather than repetition?
- Can the 48,559 formulas the parser could not read be recovered?
From textbook to signature
- PDF textbooks. Formulas pulled out by two converters.
- Parser. Built for the purpose.
- Structural signature. The same shape becomes findable across different fields.
Formulas were pulled from PDF textbooks by two converters over the spring and summer, and translated into structural signatures by a parser built for the purpose. The same shape becomes findable across different fields.
Stopped in July, kept as an archive
The pipeline stopped in July 2026. The last full signing run, on 26 July, covered 225,607 formulas at 30 microseconds each; 48,559 could not be parsed, and nothing has been signed since. When the lab moved its memory to a new core in September, the formulas were deliberately not carried over: 274,244 of them are kept in an archive of the old graph, readable, not growing.
What goes in
Snapshot 23 September 2026PDF textbooks, read through the text layer rather than image recognition. A LaTeX-set PDF says where its mathematics is through its fonts: one family for variables, one for blackboard letters, one for script; the converter reads the font names and lifts the formulas out as LaTeX. Then a parser built for the purpose turns each formula into a structural signature: the shape of the expression, independent of which letters were chosen for it. The signatures are the part that works. Turning them into reliable bridges between fields is not solved here, and the numbers below say so rather than implying otherwise.
In the graph
| formulas in the graph | 274,244 |
| variables | 169,907 |
| canonical variables the same quantity under different letters, merged | 12,558 |
Parser benchmark
Share of formulas parsed completely. A partial parse counts as failure; the parser refuses to sign a fragment.
- own parser57.9 %
- latex2sympy21.4 %
- hard cases: 4,000 real parse failures from the old parser
- own parser88.1 %
- latex2sympy24.8 %
- ordinary material
30 microseconds per formula; the whole stock in eight seconds. Measured 26 July 2026 on PDF-converted material; handwritten arXiv LaTeX not yet tested.
What comes out
Why signatures, not values
The parser does not evaluate anything. Unknown notation becomes an opaque node instead of an error, so two formulas can be compared by shape alone, whatever letters they use. What a signature is good for is finding the same expression again when it appears in another book under different letters.
What was learned the hard way
The first parser returned partial trees without saying so; measured, 51.5 % of its "successes" were fragments. The second one has three states, complete, partial, failed, and only complete gets a signature. The whole stock was re-signed on 26 July 2026.
What is actually in there
Not a sample chosen to look good: two formulas from each of the best represented subject areas, filtered only for being complete expressions rather than single symbols. Shown as they are stored, in LaTeX.
mathematics
x_i^{(1)} = \sum_{a_{ji} \neq 0} y_j^{(0)}y_i^{(1)} = \sum_{a_{ij} \neq 0} x_j^{(1)}
financial regimes
\Phi_{t+s}(z) = \Phi_{X_{t+s}}(z) = \Phi_{X_s}(z)\Phi_{X_{t+s}-X_s}(z)\Phi_{X_s}(z)\Phi_{X_t}(z) = \Phi_s\Phi_t
category theory
(f \circ g) \circ h = f \circ (g \circ h)G(f) \circ \alpha_{C'} = \alpha_C \circ F(f)
nonlinear science
q = 2\kappa^2 \operatorname{sech}^2 \kappa (x - 4\kappa^2 t - x_0)iq_t + q_{xx} + \sigma |q|^2 q = 0, \tag{3}
information theory
E(\mathbf{x}; J, H) = -\left[\frac{1}{2} \sum_{m,n} J_{mn} x_m x_n + \sum_n H x_n\right],P(\mathbf{x}|\beta, J, H) = \frac{1}{Z(\beta, J, H)} \exp[-\beta E(\mathbf{x}; J, H)], \qquad (31.2)
active inference
\sum_{x} P(x,y) = \sum_{x} P(y)P(x|y) = \underbrace{P(y)\sum_{x} P(x|y) = P(y)}_{\text{Sum rule}}\underbrace{P(x)P(y|x) = P(x,y)}_{\text{Product rule}} \underbrace{P(x)P(x|y)}_{\text{Product rule}}
physics
\mathbf{a} \bullet \mathbf{b} = a_1b_1 + a_2b_2 + a_3b_3\pi - (\alpha + \beta + \gamma) = C\Delta
probability theory
X_t = X_0 \exp\left(\left(a - \frac{1}{2}b^2\right)t + bW_t\right) ,f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) \mathrm{d}y
Filtered for complete expressions: entries that are a single symbol rather than a formula are left out of this list.