2 articles
Embeddings
All posts tagged with #Embeddings
One Encoder, Two Jobs: UltraBERT and the End of Scaling Your Embedder
UltraBERT tops 9 of 10 encoder benchmarks at 150M and 380M parameters with a 32K context, and the research behind it proves more pretraining compute stopped helping encoders long ago. Here is what that changes for your RAG pipeline.
Eight Megabytes. All of Wikipedia. Seven Minutes on a Laptop.
Lattice is a static embedding model that scores 0.4749 on BEIR, compresses to 7.94 MB, and embeds all 6.4 million Wikipedia articles in 7 minutes on a laptop. Here is what that changes for retrieval pipelines.