Batch convert docs to achieve acceleration - #647
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR changes the Python query hot path to avoid per-document Python/C++ binding overhead by batch-materializing query results as lightweight tuples in C++, then constructing Doc objects from those tuples in Python.
Changes:
- Refactors
_Collection.Querybindings to return a batch of(id, score, fields, vectors)tuples, parameterized by schema. - Extracts per-doc materialization into a shared C++ helper (
ZVecPyDoc::doc_to_tuple) and reuses it for both per-doc and batch paths. - Updates the Python
QueryExecutorto consume tuples viaDoc._from_tuple, and adds correctness tests for the new batch-materialized behavior.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/binding/python/model/python_doc.cc | Extracts get_all logic into doc_to_tuple and binds get_all to it. |
| src/binding/python/model/python_collection.cc | Changes Query bindings to return batch-materialized tuples and adds docs_to_tuples helper. |
| src/binding/python/include/python_doc.h | Declares doc_to_tuple and adds schema dependency for the new signature. |
| python/zvec/executor/query_executor.py | Switches query execution to consume tuple batches and build Doc via Doc._from_tuple. |
| python/tests/test_query_executor.py | Adjusts unit test to patch Doc._from_tuple instead of convert_to_py_doc. |
| python/tests/test_batch_materialize.py | Adds integration-style correctness tests for batch-materialized query results. |
Suppressed comments (1)
src/binding/python/include/python_doc.h:1
- Including
<zvec/db/schema.h>in this public binding header increases compile-time coupling for every translation unit that includespython_doc.h. Sincedoc_to_tupleonly needsCollectionSchemaby reference in the declaration, you can forward-declareclass CollectionSchema;innamespace zvechere and move the schema include into the corresponding.ccfile.
// Copyright 2025-present the zvec project
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Query with the GIL released, then materialize all hits into | ||
| // (id, score, fields, vectors) tuples in one crossing (see docs_to_tuples). | ||
| col.def( | ||
| "Query", | ||
| [](const Collection &self, const SearchQuery &query, | ||
| const CollectionSchema &schema) { | ||
| Result<DocPtrList> result; | ||
| { | ||
| py::gil_scoped_release release; | ||
| result = self.Query(query); | ||
| } | ||
| return docs_to_tuples(unwrap_expected(result), schema); | ||
| }, | ||
| py::arg("query"), py::arg("schema"), | ||
| "Execute a query and return results as a list of " | ||
| "(id, score, fields, vectors) tuples materialized in one batch.") |
There was a problem hiding this comment.
确实有点问题，_Collection.Query到底算不算公共API
There was a problem hiding this comment.
_Collection.Query不算公共API。 作为pybind层的wrapper。用来封装py -> c++的调用
| // Query with the GIL released, then materialize all hits into | ||
| // (id, score, fields, vectors) tuples in one crossing (see docs_to_tuples). | ||
| col.def( | ||
| "Query", | ||
| [](const Collection &self, const SearchQuery &query, | ||
| const CollectionSchema &schema) { | ||
| Result<DocPtrList> result; | ||
| { | ||
| py::gil_scoped_release release; | ||
| result = self.Query(query); | ||
| } | ||
| return docs_to_tuples(unwrap_expected(result), schema); | ||
| }, | ||
| py::arg("query"), py::arg("schema"), | ||
| "Execute a query and return results as a list of " | ||
| "(id, score, fields, vectors) tuples materialized in one batch.") |
There was a problem hiding this comment.
_Collection.Query不算公共API。 作为pybind层的wrapper。用来封装py -> c++的调用
| Result<CollectionSchema> schema_result; | ||
| { | ||
| py::gil_scoped_release release; | ||
| result = self.Query(query); |
There was a problem hiding this comment.
self.Query(query) & self.Schema() 不是同一把锁吧？ 并发情况下，如果drop_column，会不会有解析丢字段的问题？
| if (!self.has_value(field)) { | ||
| continue; | ||
| } | ||
| py::tuple ZVecPyDoc::doc_to_tuple(Doc &self, const CollectionSchema &schema) { |
There was a problem hiding this comment.
get_any 里的 DataType -> Python switch 逻辑和 doc_to_tuple方法里的逻辑有重复。 这次抽一个doc_valude_to_py的方法，来复用相关逻辑
Change the query result materialization from converting each document individually across the Python/C++ boundary into a PyDoc, to batch-materializing results as lightweight tuples according to the schema in a single C++ call and then constructing Doc objects via Doc._from_tuple on the Python side.
This aims to eliminate the per-document language binding conversion overhead on the query hot path, thereby reducing query latency.
Dataset: cohere-1m, M: 15