Full write-up, raw results and a one-command reproduction are in the repository: experiments/2026-10-scifact-native-vs-posthoc. Follow-up to SciFact: codecs against the relevance judgments.

Hypothesis. A model trained with Matryoshka loss [1] carries its own codec: keep the first k dimensions. Truncated to 256 or 128 dimensions, nomic-embed-text-v1.5 will retrieve at least as well against the SciFact judgments [4] as bge-small-en-v1.5 [5] quantized post hoc to the same bytes. If native wins, the right codec is a different embedding model.

Result. It does not win. Two new codec families make the comparison: pq<m>, product quantization with trained codebooks [2][6], and mrl<k>, truncation composable with any other codec (mrl256+int8). nDCG@10 against the judgments, by byte budget:

bytes bge-small-en-v1.5 (384 d) nDCG@10 nomic-embed-text-v1.5 (768 d) nDCG@10
3072     f32 0.696
1536 f32 0.698    
1024 mrl256 0.684 mrl256 0.671
392 int8 0.698    
264     mrl256+int8 0.671
256 mrl64 0.520 mrl64 0.520
96     pq96 · binary-centred · binary 0.686 · 0.683 · 0.673
48 pq48 · binary-centred · binary 0.678 · 0.658 · 0.654 pq48 · mrl384+binary 0.634 · 0.632
32     mrl256+pq32 · mrl256+binary 0.638 · 0.586
24 pq24 · mrl192+binary 0.602 · 0.580 mrl192+binary 0.523

Reading. Matryoshka training works per dimension: dropping two thirds of nomic costs 2.5 points where dropping three quarters of bge-small costs 10.5. But the two models retrieve equally well at full precision and nomic needs twice the bytes to do it, so per byte the native lever never catches up: post-hoc truncation of bge-small wins at 1,024 bytes and the two tie at

  1. Product quantization is the best codec under 400 bytes on both models; at 48 bytes on bge-small it keeps 0.678 where the sign code [3] keeps 0.654, two points under f32 at 32× smaller. Fewer bits of more dimensions beats more bits of fewer: binary on all 768 nomic dimensions (96 bytes) beats mrl128+int8 (136 bytes). Nomic does tolerate binary much better than bge-small (2.3 points lost against 4.4), having twice the sign bits and more centred vectors.

Caveats. One corpus; only two models, one of each kind, so dimension and training are confounded; bge-small ran without its query instruction; plain PQ without a rotation; truncation of bge-small keeps the leading dimensions where a random projection would be the fairer post-hoc control. No chat model yet.

References

  1. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, Ali Farhadi. Matryoshka Representation Learning. Advances in Neural Information Processing Systems (NeurIPS), 2022. library
  2. Hervé Jégou, Matthijs Douze, Cordelia Schmid. Product Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(1), 2011. library
  3. Moses S. Charikar. Similarity Estimation Techniques from Rounding Algorithms. ACM Symposium on Theory of Computing (STOC), 2002. library
  4. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, Iryna Gurevych. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS Datasets and Benchmarks, 2021. library
  5. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff. C-Pack: Packed Resources For General Chinese Embeddings. arXiv:2309.07597, 2023. library
  6. Stuart P. Lloyd. Least Squares Quantization in PCM. IEEE Transactions on Information Theory 28(2), 1982. library

Terms used above are defined in the glossary.