Experiments

Each experiment in the repository’s experiments/ directory ships with a write-up: hypothesis, setup, results table with the f32 baseline, and discussion. Finished write-ups are published here. Negative and inconclusive results are kept.

• SciFact: native truncation against post-hoc quantization at equal bytes (2026-10-12)
  A Matryoshka-trained 768-dim model truncated natively never beats a conventional 384-dim model quantized post hoc at the same bytes; product quantization is the best codec under 400 bytes on both (pq48 on bge-small: 0.678 nDCG@10 at 48 bytes, two points under f32).
• SciFact: codecs against the relevance judgments (2026-10-12)
  Binary loses 39 points of agreement with f32 on bge-small but only 4.4 points of nDCG@10 against the human labels; int8 is indistinguishable from f32; centring's gain barely registers.
• SciFact: mean-centred binary against plain binary (2026-10-09)
  Centring cuts bge-small's binary score error six-fold (0.307 to 0.048) but lifts recall@10 only from 0.611 to 0.656; the binary loss is a resolution problem, not a centring problem.
• SciFact: int8 and binary codecs against f32, two embedding models (2026-10-08)
  int8 keeps recall@10 at 0.99 at 3.9× smaller; binary keeps 0.61 (bge-small) and 0.71 (MiniLM) at 32× smaller, and widening k does not recover it.