Added PQ INT8 quantizer in Turbo by zzlin237 · Pull Request #554 · alibaba/zvec

zzlin237 · 2026-07-07T08:17:20Z

The basic PQ has been implemented, but its QPS requires further optimization. The recall test results for the current implementation are as follows:

HNSW

Configuration:

M: 50
efConstruction: 500
efSearch: 400
topk: 50

L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift	kNone	100	-
Sift	kPQ + int8 + `64`	92.02	100
Sift	kPQ + int8 + `32`	78.44	99.98
Sift	kPQ + int8 + `16`	63.68	98.82
Gist	kNone	98.576	-
Gist	kPQ + int8 + `480`	93.212	98.668
Gist	kPQ + int8 + `240`	76.676	98.688
Gist	kPQ + int8 + `120`	54.40	94.45

The results are generally consistent with previous tests. Under the int8 condition, using one subquantizer per 4 dimensions yields the best performance.

Cosine Metric

Option 1: Normalization + IP Metric

Option 2: Normalization + L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kPQ + int8 + `192` + Option 2	86.03	99.73
Openai	kPQ + int8 + `384` + Option 2	87.442	99.76

The performance of both options is nearly identical. Option 2 was ultimately adopted because the L2 metric also supports operations specific to L2, such as decentralization.

Inner Product Metric

Option 1: L2 for k-means + L2 encoding + IP for SDC calculation + IP for ADC calculation

Option 2: L2 for k-means + L2 encoding + L2 for SDC calculation + IP for ADC calculation |<---- faiss approach

Option 3: L2 for k-means + IP encoding + IP for SDC calculation + IP for ADC calculation

Failed approach: Due to mismatched encoding methods, recall dropped significantly.

Option 4: IP for k-means(sp) + IP encoding + IP for SDC calculation + IP for ADC calculation |<---- ivf porting approach

Failed approach: 1) Using IP in k-means leads to collapse, resulting in only one cluster containing points while all others remain empty; 2) Segmented clustering is unsuitable for spherical data. PQ performs segmentation before clustering. Even if the original vectors are normalized onto the unit sphere, their individual segments will no longer lie on the sphere.

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kPQ + int8 + `192` + Option 1	79.904	99.658
Openai	kPQ + int8 + `384` + Option 1	82.41	99.936
Cohere	kPQ + int8 + `192` + Option 2	79.532	98.062
Openai	kPQ + int8 + `384` + Option 2	82.266	99.65

Option 1 performs slightly better and is more semantically coherent: L2 is used for encoding, while IP is used for distance calculation.

Decentralization

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift + L2	kPQ + int8 + `32`	78.44	99.98
Sift + L2	kPQ + int8 + `32` + decentralization	78.56	99.96
Cohere + Cosine	kPQ + int8 + `192`	86.03	99.73
Cohere + Cosine	kPQ + int8 + `192` + decentralization	86.088	99.738
Openai + Cosine	kPQ + int8 + `384`	87.54	99.78
Openai + Cosine	kPQ + int8 + `384` + decentralization	87.68	99.78

Decentralization does not yield significant improvements, but support for it is retained nonetheless.

IVF

Configuration:

centroid_count: 128
nprobe: 16
topk: 50
ScaleFactor: 4

Option 1: Build codebooks using residuals, with each cluster assigned an independent codebook.

Option 2: Build codebooks using residuals, with all clusters sharing a single codebook. ｜<--- faiss approach

L2 Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Sift	kNone	99.7	-
Sift	IVFPQ + `int8` + `32` + Option 1	86.18	99.54
Sift	IVFPQ + `int8` + `32` + Option 1	78.52	99.44
Gist	kNone	96.412	-
Gist	IVFPQ + `int8` + `240` + Option 1	80.718	96.468
Gist	IVFPQ + `int8` + `240` + Option 2	77.49	96.244

Cosine Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kNone	99.576	-
Cohere	IVFPQ + `int8` + `192` + Option 1	85.174	94.37
Cohere	IVFPQ + `int8` + `192` + Option 2	83.916	94.09
Openai	kNone	96.024	-
Openai	IVFPQ + `int8` + `384` + Option 1	87.218	96.196
Openai	IVFPQ + `int8` + `384` + Option 2	83.916	94.09

Inner Product Metric

Datasets	Quantizer	Recall(%)	Refiner Recall (%)
Cohere	kNone	87.632	-
Cohere	IVFPQ + `int8` + `192` + Option 1	81.43	95.696
Cohere	IVFPQ + `int8` + `192` + Option 2	79.74	95.77

The original IVFPQ+IP implementation uses MIPS, which results in lower recall.

Although Option 2 shows a slight decrease in Recall compared to Option 1, it significantly reduces training time and storage requirements while maintaining comparable Refiner Recall. Therefore, Option 2 was ultimately adopted.

…efactor/turbo_preprocessor

…rocessor # Conflicts: # .gitignore

…o refactor/turbo_pq

Copilot

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR introduces Product Quantization (PQ) int8 support and an FHT-based rotator in Turbo, including ISA-dispatched kernels (scalar/SSE/AVX2/AVX512/NEON) and expanded test coverage for correctness and SIMD consistency.

Changes:

Added PqInt8Quantizer with training, encoding/decoding, ADC/SDC distance kernels, and (de)serialization.
Added FhtRotator preprocessor with ISA-dispatched rotate/unrotate kernels and serialization.
Updated Turbo dispatch (get_pq_kernels, get_rotator_kernels), build flags for SIMD sources, and added NEON runtime feature detection.

Reviewed changes

Copilot reviewed 47 out of 53 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.{h,cc}`	Implements PQ int8 quantizer, training, distances, and serialization.
`src/turbo/turbo.cc` / `src/include/zvec/turbo/turbo.h`	Adds kernel dispatch APIs/types for PQ and rotator; refactors arch matching.
`src/turbo/preprocessor/preprocessor.h` / `src/turbo/preprocessor/fht_rotator/*`	Introduces rotator interface and FHT rotator implementation with (de)serialization.
`src/turbo/distance/*/pq_distance` / `src/turbo/distance/*/rotate/fht/`	Adds scalar + SIMD kernels for PQ ADC/SDC and FHT rotation.
`src/ailego/internal/cpu_features.{h,cc}`	Adds NEON runtime detection to support ISA dispatch on ARM.
`src/turbo/CMakeLists.txt`	Applies per-file SIMD compile flags for SSE/AVX2/AVX512 sources.
`tests/turbo/*`	Adds extensive unit tests for PQ quantizer and FHT rotator / SIMD consistency.
Misc test/style files	Formatting tweaks and minor cleanups.

Comments suppressed due to low confidence (11)

src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1

For Cosine, the implementation returns squared L2 distance between unit-normalized vectors (range ~[0,4]), not cosine distance 1 - cos (range [0,2]). The comment says no conversion is needed, but a conversion factor is required if the public metric semantics are cosine distance. A common conversion for unit vectors is cosine_distance = 0.5f * l2_sq (since ||a-b||^2 = 2 - 2cos). This impacts calc_distance_dp_query(_batch), distance() (which currently returns adc_fn_ directly), and any SDC path built from dist_table_ when metric_name == \"Cosine\". Suggest wrapping ADC/Batch ADC (and SDC if exposed) in lambdas for Cosine to apply the conversion, or explicitly change the quantizer’s metric exposure to squared-L2-on-normalized-vectors so callers/tests aren’t misled.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
For Cosine, the implementation returns squared L2 distance between unit-normalized vectors (range ~[0,4]), not cosine distance 1 - cos (range [0,2]). The comment says no conversion is needed, but a conversion factor is required if the public metric semantics are cosine distance. A common conversion for unit vectors is cosine_distance = 0.5f * l2_sq (since ||a-b||^2 = 2 - 2cos). This impacts calc_distance_dp_query(_batch), distance() (which currently returns adc_fn_ directly), and any SDC path built from dist_table_ when metric_name == \"Cosine\". Suggest wrapping ADC/Batch ADC (and SDC if exposed) in lambdas for Cosine to apply the conversion, or explicitly change the quantizer’s metric exposure to squared-L2-on-normalized-vectors so callers/tests aren’t misled.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
Two concrete correctness issues around deserialization/SDC:\n1) calc_distance_dp_dp() always calls sdc_fn_ with dist_table_.data(), but dist_table_ is never rebuilt after deserialize(). That means SDC can read from an empty buffer (UB) or return garbage post-deserialization. If SDC is part of the supported API surface, either recompute dist_table_ during deserialize() (or lazily on first SDC use) or make SDC explicitly unavailable after deserialize (and return an error / invalid DistanceImpl).\n2) deserialize() only checks magic, but does not validate hdr.version, hdr.quant_type, hdr.payload_size vs len, nor does it restore the metric configuration from hdr.metric. On a fresh factory-created quantizer (without init()), meta_.metric_name() may be unset, which will make Cosine/IP behaviors and extra_meta_size_ inconsistent. Suggest: validate header fields, bounds-check payload reads using hdr.payload_size, and restore meta_ (including metric) from header before selecting fp32_batch_fn_/extra_meta_size_.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
Two concrete correctness issues around deserialization/SDC:\n1) calc_distance_dp_dp() always calls sdc_fn_ with dist_table_.data(), but dist_table_ is never rebuilt after deserialize(). That means SDC can read from an empty buffer (UB) or return garbage post-deserialization. If SDC is part of the supported API surface, either recompute dist_table_ during deserialize() (or lazily on first SDC use) or make SDC explicitly unavailable after deserialize (and return an error / invalid DistanceImpl).\n2) deserialize() only checks magic, but does not validate hdr.version, hdr.quant_type, hdr.payload_size vs len, nor does it restore the metric configuration from hdr.metric. On a fresh factory-created quantizer (without init()), meta_.metric_name() may be unset, which will make Cosine/IP behaviors and extra_meta_size_ inconsistent. Suggest: validate header fields, bounds-check payload reads using hdr.payload_size, and restore meta_ (including metric) from header before selecting fp32_batch_fn_/extra_meta_size_.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
thread_count_ is documented in the header as 0 = hardware_concurrency, but train() passes it through directly and constructs SingleQueueIndexThreads with thread_count == 0. If SingleQueueIndexThreads treats 0 as invalid (common), training will either run with zero workers (no tasks executed → untrained centroids), or fail/assert internally. Recommend normalizing thread_count at the start of train(holder, thread_count) (e.g., map <=0 to IndexThreads::hardware_concurrency() or 1) and using that normalized value consistently for logging and pool sizing.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
thread_count_ is documented in the header as 0 = hardware_concurrency, but train() passes it through directly and constructs SingleQueueIndexThreads with thread_count == 0. If SingleQueueIndexThreads treats 0 as invalid (common), training will either run with zero workers (no tasks executed → untrained centroids), or fail/assert internally. Recommend normalizing thread_count at the start of train(holder, thread_count) (e.g., map <=0 to IndexThreads::hardware_concurrency() or 1) and using that normalized value consistently for logging and pool sizing.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
There are two high-cost patterns in hot paths:\n- Training subsampling swaps vectors element-by-element (O(dim) swaps inside the shuffle loop). This is significantly slower than swapping contiguous blocks (e.g., std::swap_ranges over [i*dim, (i+1)*dim) or a temporary buffer/memcpy swap).\n- quantize_data() allocates std::vector<float> buffers (norm_vec_storage, centered_vec_storage) per call. Encoding is typically a throughput-critical path; repeated heap allocations can dominate runtime. Consider using a reusable scratch buffer strategy (e.g., thread-local buffers, small static buffers when dim is bounded, or passing an external scratch workspace) to avoid per-vector allocations.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
There are two high-cost patterns in hot paths:\n- Training subsampling swaps vectors element-by-element (O(dim) swaps inside the shuffle loop). This is significantly slower than swapping contiguous blocks (e.g., std::swap_ranges over [i*dim, (i+1)*dim) or a temporary buffer/memcpy swap).\n- quantize_data() allocates std::vector<float> buffers (norm_vec_storage, centered_vec_storage) per call. Encoding is typically a throughput-critical path; repeated heap allocations can dominate runtime. Consider using a reusable scratch buffer strategy (e.g., thread-local buffers, small static buffers when dim is bounded, or passing an external scratch workspace) to avoid per-vector allocations.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
There are two high-cost patterns in hot paths:\n- Training subsampling swaps vectors element-by-element (O(dim) swaps inside the shuffle loop). This is significantly slower than swapping contiguous blocks (e.g., std::swap_ranges over [i*dim, (i+1)*dim) or a temporary buffer/memcpy swap).\n- quantize_data() allocates std::vector<float> buffers (norm_vec_storage, centered_vec_storage) per call. Encoding is typically a throughput-critical path; repeated heap allocations can dominate runtime. Consider using a reusable scratch buffer strategy (e.g., thread-local buffers, small static buffers when dim is bounded, or passing an external scratch workspace) to avoid per-vector allocations.
src/turbo/quantizer/pq_int8_quantizer/pq_int8_quantizer.cc:1
There are two high-cost patterns in hot paths:\n- Training subsampling swaps vectors element-by-element (O(dim) swaps inside the shuffle loop). This is significantly slower than swapping contiguous blocks (e.g., std::swap_ranges over [i*dim, (i+1)*dim) or a temporary buffer/memcpy swap).\n- quantize_data() allocates std::vector<float> buffers (norm_vec_storage, centered_vec_storage) per call. Encoding is typically a throughput-critical path; repeated heap allocations can dominate runtime. Consider using a reusable scratch buffer strategy (e.g., thread-local buffers, small static buffers when dim is bounded, or passing an external scratch workspace) to avoid per-vector allocations.
src/turbo/quantizer/distance.h:1
DistanceImpl appears to be a generic wrapper used beyond PQ. Updating the comment to state it is specifically "the ADC path for PQ" is misleading for other quantizers/metrics that may use func() differently. Suggest rewording to describe the generic contract (distance between candidate and stored query representation), and mention PQ/ADC only as an example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot