as a tradeoff between reminiscence and recall. The usual is Float32 with excessive constancy and excessive reminiscence price. The essential answer is scalar quantization, which reduces every worth to fewer bits (round 4× compression) with a slight recall loss. Though binary quantization pushes a lot tougher, typically reaching 32× compression, the retrieval consequence may change into inconsistent resulting from info loss. However, product quantization could also be extra environment friendly, however it’s tougher to tune and function in actual manufacturing.
In early Might of 2026, Qdrant launched TurboQuant, a brand new quantization methodology. They usually claimed that “TurboQuant can scale back reminiscence use with out making retrieval high quality too unstable“. TurboQuant sounds just like the form of characteristic vector search groups need.
Nevertheless, I puzzled whether or not TurboQuant nonetheless holds up after we check it throughout totally different dataset sizes. Does it give an actual enchancment over frequent quantization strategies, or does its benefit depend upon the info?
I ran experiments to match it with extra acquainted quantization strategies akin to scalar and binary quantization. The aim was to grasp the place TurboQuant is helpful, the place it’s dangerous, and whether or not it may be handled as a severe default choice for vector search.
I imagine that it will assist engineers, ML practitioners, and vector database customers perceive the place TurboQuant suits in contrast with extra frequent quantization strategies, particularly when transferring from experiments to manufacturing.
1. What’s Quantization?
Each float32 quantity in a vector makes use of 4 bytes. In consequence, a 1536-dimension embedding takes 6 KB per vector; at one million vectors, the database takes as much as 6 GB only for the index.
That is after we want Quantization. Quantization shrinks every quantity in a vector to a smaller byte quantity. The usual strategy is Scalar quantization. It begins with discovering the min and max throughout every dimension. Then, that vary is split into 255 equal bins. Each worth within the vector is rounded to the closest bin, and the bin quantity is saved as a single byte as a substitute of 4.
The unique Float32 embedding now turns into a uint8 embedding at 4x compression, that means 4 occasions smaller in storage measurement.
Determine 1 under is a straightforward demonstration of this course of on a 6D vector.
The tiny error within the final row is known as quantization error, and it accumulates throughout 6 dimensions of the vector throughout dot product computation. That is what makes similarity scores barely fallacious.
Nevertheless, there are extra aggressive compressions akin to 8x (4-bit), 16x (2-bit), or 32x (1-bit). The extra the compression, the smaller the vector measurement, and the larger the error from the unique one. You’ll be able to see it in Determine 2 under, which demonstrates the error after reworking a Float32 quantity to totally different quantization areas.
The tradeoff between compression and recall (or reminiscence and recall) is apparent. Extra compression ends in decrease recall.
2. The Actual Query is Not Compression Ratio
The true query is: what vector geometry stays after compression?
Conventional quantizers, normally, instantly compress the vector. Scalar quantization applies the identical fastened grid to each dimension, whether or not that dimension accommodates a helpful sign or noise. Binary quantization retains solely the signal bit. Due to this fact, neither methodology first checks whether or not some dimensions carry extra sign than others.
Qdrant 1.18 modifications this sample with the brand new TurboQuant built-in. Primarily based on a Google Analysis algorithm offered at ICLR 2026, TurboQuant rotates the vector earlier than compression. This random rotation spreads variance extra evenly throughout dimensions, so every bit can protect extra helpful info.
TurboQuant will not be higher as a result of it makes use of fewer bits. It’s higher as a result of it makes the vector simpler to compress earlier than spending these bits.
The important thing variations between TurboQuant and others are proven in Determine 3 under.
- Scalar Quant forces one grid on all dimensions, like the identical pair of sneakers for everybody, no matter their foot size.
- Binary Quant transforms values to 0 or 1 with the principles: Values ≥ 0 change into 1; Values < 0 change into 0. That is like chopping each shoe to just one selection: left or proper, large or small, sure or no. This can be very low-cost, nevertheless it throws away virtually all form info, so the “match” turns into very crude.
- Product Quant learns per-subspace codebooks; it suits every of the pair of sneakers to every foot. It’s an excellent match for everybody, however extraordinarily pricey.
TurboQuant makes all dimensions look alike first, then makes use of one well-designed codebook. This is similar as altering all of the ft to the identical measurement and having one pair of sneakers for all.
3. TurboQuant in Quick: Rotate First, Compress Second
Each vector in an embedding mannequin has construction.
A 1536-dimensional embedding may carry most of its helpful sign in solely a small subset of coordinates. The remaining dimensions typically contribute a lot much less, however they nonetheless seem in each vector, which provides noise and makes distance comparisons much less dependable
3.1 The TurboQuant Pipeline
The concept is easy. Earlier than compressing, spin the vector by way of a random orthogonal rotation. That rotation doesn’t change distances - it simply redistributes power so each dimension carries roughly the identical quantity of knowledge. Then, a single precomputed codebook is utilized to the rotated vectors, and it may deal with all dimensions equally properly. No per-dimension tuning wanted. No coaching in your information.
Test Determine 4 under for a abstract of the method.
3.2 What Does Rotation Do to the Coordinates?
In Determine 5, earlier than rotation, a couple of dimensions carry many of the power. The remainder carry a lot much less sign and sometimes extra noise.
After rotation, each dimension carries roughly equal power and an equal quantity of knowledge.
Nevertheless, does this certainly imply that power transformation preserves vital info and maintains distance relative to a different vector, as with the unique one?
I made a easy computation between 2 4D vectors, with Vector A reworked utilizing TurboQuant, after which, at inference time, rotated Vector B with the identical matrix and measured the cosine similarity in the identical rotated area. This cosine similarity is in comparison with the unique vector A vs unique vector B cosine similarity.
3.3 Normal TurboQuant course of
In Determine 6, after making use of TurboQuant to the unique vector A, the space between the brand new vector A and Vector B barely modifications in comparison with the unique vector A and Vector B, proving that the vital geometry between vectors remains to be preserved, and recall is extremely maintained.
3.4 How precisely does Qdrant apply Turboquant within the Database?
There are 2 processes individually on Qdrant:
3.4.1. Indexing course of:
The overview of Indexing Move is visualized in Determine 7. Principally, the vector is processed as follows:
unique vector → normalize/put together relying on metric → pad if wanted → Hadamard rotation → non-compulsory per-coordinate calibration: x → (x + shift) · scale → Lloyd-Max centroid project → packed TurboQuant codes
For TurboQuant particularly, Qdrant shops the data under as written in Desk 1:
An vital issue launched by Qdrant is the Size Renormalization, aka Scaling issue. It occurs after quantization, when Qdrant measures how a lot shorter the quantized reconstruction grew to become vs the unique size, shops that ratio as a per-vector scaling issue, after which applies it throughout scoring at question time.
The scaling issue = original_length / centroid_reconstruction_length
Why do we want Size Renormalization?
There’s an statement after quantization
The quantized vector factors in the precise path however is just too quick
Which implies when quantizing a vector, there may be all the time a quantization error, and it systematically shrinks the size of each vector. In question time, while you compute a dot product between a quantized vector and a rotated & encoded question, you’re computing the dot product of a slightly-too-short vector, which provides a rating that’s persistently too low. Qdrant calls this the “recall-degrading bias”.
To repair this, we want an element to multiply it again in through the scoring part as a substitute of fixing the vectors. This tactic is easy and efficient.
3.4.2. Question Time Course of
Determine 8 reveals the method of querying with the TurboQuant vector database.
The question is rotated and transformed right into a SIMD scoring illustration, and Qdrant makes use of uneven scoring to match that encoded question instantly towards the packed TurboQuant codes saved for database vectors.
After that, the saved scaling issue is multiplied by the rating
4. Which Methodology to Strive First
Qdrant affords a number of selections for quantization, and TurboQuant additionally affords a number of bit-compression variants akin to bits4, bits2, bits1.5, and bits1.
As per their doc, decrease bit depths provide greater compression at the price of accuracy.
Determine 9 reveals some solutions for reference in case you continue to marvel which compression strategies to make use of.
5. Getting Began: The First Experiment
Change just one config within the present Qdrant code to allow TurboQuant. Your present collections stay untouched.
Please reference the code snippet under for particulars.
from qdrant_client import QdrantClient, fashions
consumer = QdrantClient("localhost", port=6333)
# New assortment — one config change
consumer.create_collection(
collection_name="my_collection",
vectors_config=fashions.VectorParams(
measurement=1536,
distance=fashions.Distance.COSINE,
),
quantization_config=fashions.TurboQuantization(
turbo=fashions.TurboQuantQuantizationConfig(
bits=fashions.TurboQuantBitSize.BITS4,
always_ram=True,
)
),
)
# Current assortment — patch with out recreating vectors
consumer.update_collection(
collection_name="existing_collection",
quantization_config=fashions.TurboQuantization(
turbo=fashions.TurboQuantQuantizationConfig(
bits=fashions.TurboQuantBitSize.BITS4,
always_ram=True,
)
),
)For extra configuration, please test the Qdrant documentation for TurboQuant here.
6. Benchmark: Does the idea maintain?
To check TurboQuant towards each different Qdrant quantizer on actual embeddings, I ran a number of checks at totally different sizes (10K, 50K, and 100K vectors) with totally different quantization strategies of Qdrant.
6.1 Why the DBpedia Dataset?
I selected the DBpedia embeddings dataset (License: CC-BY-SA 4.0 and GNU Free Documentation License) as a result of it has a coordinate variance ratio of 233.5x - extremely anisotropic. A couple of dimensions carry many of the sign; the remainder carry noise. That is precisely the distribution the place TurboQuant’s rotation ought to assist most, and the place scalar quantization’s fastened grid wastes essentially the most bits.
Please test the main points of the check setting within the Appendix part, half 9.2.
6.2 Recall throughout scale
Particulars of the testing recall efficiency are in Determine 10.
4 issues bounce out:
- TQ recall stays unchanged because the dataset grows. Whereas Binary Quantization drops from 0.916 to 0.78 when the dataset measurement doubles, the TurboQuant variants maintain up a lot better. The rotation step helps every bit protect extra info, making TQ much less delicate to corpus progress.
- Most TQ variants are near Float32 and Scalar Quantization in recall. Aside from TQ 1-bit and TQ 4-bit, the TurboQuant outcomes stay broadly corresponding to the Float32 baseline and Scalar Quantization.
- TQ 4-bit provides the very best accuracy–compression tradeoff. It reaches recall near Scalar Quantization whereas utilizing roughly half the storage: 8× compression vs Scalar’s 4×. At 100K vectors, TQ 4-bit reaches 0.965 recall, only one.5 factors under Scalar’s 0.980. With rescoring, the hole disappears: 0.996 for TQ 4-bit vs 0.993 for Scalar.
- Rescoring recovers a lot of the recall hole, even for aggressive compression (TQ 1-bit). TQ 1-bit improves considerably with rescoring. Binary Quantization with rescoring can work on smaller datasets, however its recall degrades quicker because the dataset grows.
6.3 Latency Throughout Scale
Particulars of the testing latency efficiency are in Determine 11.
- The latency story is evident: rescoring provides some price, however not a lot. At 100K vectors, TQ 4-bit + rescore runs in 6.4 ms, quicker than Float32 at 7.6 ms and solely barely behind Scalar Quantization at 6.8 ms.
- Throughout TQ variants, rescoring will increase latency however stays quicker than the Float32 baseline.
6.4 Storage Footprint
Determine 12 under reveals the testing storage measurement for every quantization methodology.
- TQ 1-bit has the identical storage footprint as Binary Quantization: each use 18 MB, or round 32× compression.
- TQ 2-bit and TQ 4-bit use extra storage to protect extra info. TQ 2-bit roughly doubles the storage of TQ 1-bit, whereas TQ 4-bit will increase it by about 4×. Even so, each are nonetheless a lot smaller than Scalar Quantization.
6.5 Index Constructing Time
Particulars of the testing index constructing time are in Determine 13.
- TQ is the quickest configuration at 64s for 50K vectors and 179s for 100K vectors, largely as a result of sign-bit extraction is affordable.
- TQ 4-bit takes 57s / 224s, and TQ 1.5-bit takes 75s / 239s. Each are corresponding to or quicker than Float32 (110s / 289s). This means that rotation and codebook calibration add solely a small indexing price.
- TQ 2-bit is the slowest configuration (73s / 357s). This can be resulting from a much less frequent bit-packing sample or implementation-specific overhead. Even so, it nonetheless completes indexing for 100K vectors in beneath 6 minutes.
Indexing time is extra environment-sensitive, so deal with these numbers as directional relatively than absolute. Outcomes can fluctuate relying on CPU, reminiscence bandwidth, disk I/O, parallelism, and the general machine load through the run.
7. What This Means in Follow
General, TurboQuant seems promising after we prioritize the steadiness of compression and secure retrieval high quality. The outcomes present that not all compressed codecs behave the identical because the dataset grows. Some strategies lose recall rapidly, whereas others keep a lot nearer to the Float32 baseline.
- TQ 2-bit and TQ 4-bit maintain recall comparatively secure because the corpus grows. Whereas Binary Quantization and TQ 1-bit drop extra noticeably because the dataset will get bigger. This means that TurboQuant’s rotation step helps protect extra helpful info in every bit. In consequence, these TQ 2-bit and TQ 4-bit variants are much less delicate to corpus progress.
- TQ 4-bit provides the very best steadiness between recall and compression. TQ 4-bit reaches recall near Scalar Quantization however with twice the compression (Scalar Quantization provides round 4× compression, whereas TQ 4-bit provides round 8× compression). This implies TQ4-bit can save at roughly half the reminiscence price.
- TQ 1.5-bit with rescoring is the strongest choice for excessive compression: It provides round 24× compression whereas protecting recall near Float32 after rescoring. That is helpful when storage is your main constraint, however the system nonetheless wants acceptable retrieval high quality. With out rescoring, aggressive compression can lose an excessive amount of info. With rescoring, a lot of that hole could be recovered.
- TQ with rescoring is the safer sample when it is advisable steadiness latency and accuracy. That is in keeping with different practices. Rescoring does add some latency, and it’s more practical in enhancing retrieval high quality beneath excessive compression. This makes rescoring an affordable tradeoff. It provides the system a means to make use of stronger compression with out taking a big hit in retrieval high quality.
In brief, TurboQuant will not be solely about lowering reminiscence. TQ 4-bit is essentially the most balanced choice for normal use. TQ 1.5-bit with rescoring is healthier when compression is the highest precedence. The efficient sample is to pair TurboQuant with rescoring.
Essential: These numbers shouldn’t be handled as a manufacturing rule. These act as a reference in your personal judgment. Measure the efficiency in your embeddings, your queries, your {hardware}, and your recall targets earlier than migrating to manufacturing.
8. TurboQuant’s Limitations
TurboQuant improves the compression tradeoff. Nevertheless it doesn’t take away the tradeoff utterly.
Additionally it is nonetheless new. It was launched Might 11, 2026. So actual manufacturing expertise remains to be restricted. The protected strategy is easy: benchmark it first, then resolve whether or not it ought to change into your default.
I wish to lay out some limitations that should be thought of. A abstract of the constraints could be present in Determine 14:
The primary limitation is maturity. Qdrant’s benchmark outcomes look promising. However your information might behave in a different way. Your embedding mannequin, question sample, filters, and information distribution might not match the benchmark datasets. So TurboQuant ought to be handled as a powerful choice, not an automated alternative.
TurboQuant may additionally be slower than Binary Quantization on the identical storage measurement. This issues in case your predominant aim is throughput or velocity. If you happen to care extra about velocity than recall, Binary Quantization remains to be be the higher selection. TurboQuant is extra helpful while you need higher recall from a small reminiscence finances.
There’s additionally a calibration price. TurboQuant wants a one-time calibration step for every section. This normally takes seconds, not minutes. However it’s nonetheless a price. In case your system creates many segments or rebuilds indexes typically, this additional step ought to be thought of.
Distance kind is one other limitation. TurboQuant works finest with L2, dot product, and cosine similarity. Rotation preserves these distance relationships properly. Nevertheless it doesn’t protect L1 or Manhattan distance in the identical means. L1 and Manhattan distance can nonetheless work, however they want full vector reconstruction for every comparability. That may make search slower. If Manhattan distance is vital in your system, Scalar Quantization is the safer selection.
As proven within the check consequence, TQ 1-bit will not be a protected selection. TQ 1-bit provides very excessive compression, however recall can drop an excessive amount of. The rotation step helps, however 1 bit per dimension is usually too little. It can not all the time protect sufficient geometry at scale. Take into account rescoring in case TQ 1-bit doesn’t offer you anticipated efficiency. Or TQ 1.5-bit seems like a extra sensible decrease restrict. It nonetheless provides robust compression, nevertheless it retains recall extra secure. For very aggressive compression, it’s a safer selection than TQ 1-bit.
The principle lesson will not be “all the time use TurboQuant.” The principle lesson is to check what issues in your personal information. TurboQuant shifts the tradeoff in a greater path. It helps scale back recall loss earlier than the bit finances is spent. Nevertheless it doesn’t make compression free. You continue to want to decide on between reminiscence, velocity, recall, and distance habits.
In brief, TurboQuant is a powerful new choice. It’s particularly helpful with rescoring and reasonable bit settings. Nevertheless it shouldn’t be used blindly. Benchmark it by yourself embeddings first and measure it fastidiously earlier than shifting into manufacturing.
9. Appendix:
9.1 Quantization Help in Common Vector Databases
Determine 15 under is a abstract of 4 quantization affords in well-liked vector databases in your reference.
Qdrant is without doubt one of the first providers to supply TurboQuant available in the market.
9.2 Take a look at setting
- Machine: Apple M3, 16 GB RAM, macOS 15.6.1
- Testing database:
- Qdrant v1.18.0, single-node Docker, no useful resource limits
- HNSW with Default (m=16, ef_construct=100)
- Distance: Cosine
- Dataset:
