Skip to content

fix: AnswerBuilder returned referenced documents in scrambled order#11953

Open
Otis0408 wants to merge 1 commit into
deepset-ai:mainfrom
Otis0408:fix/answer-builder-scrambled-doc-order
Open

fix: AnswerBuilder returned referenced documents in scrambled order#11953
Otis0408 wants to merge 1 commit into
deepset-ai:mainfrom
Otis0408:fix/answer-builder-scrambled-doc-order

Conversation

@Otis0408

Copy link
Copy Markdown
Contributor

What's the problem

AnswerBuilder with return_only_referenced_documents=True collects the referenced document indices into a set and iterates it directly to build Answer.documents. Because CPython sets store items in internal hash-table order, the referenced documents were emitted in that hash-table order rather than ascending source-index order.

For small non-contiguous int indices this order is deterministic across runs (int hashing is identity), but it does not match the intuitive source order a user expects. For example, citations [3] [10] [50] yielded documents ordered 10, 3, 50.

Anyone rendering citations, feeding the referenced documents to a downstream component, or asserting on their order hits this: the documents look shuffled relative to how they were cited.

The fix

Iterate the collected indices via sorted(doc_idxs) so the referenced documents come out in ascending source-index order.

- for idx in doc_idxs:
+ for idx in sorted(doc_idxs):

Before / after (public component API)

from haystack import Document
from haystack.components.builders import AnswerBuilder

builder = AnswerBuilder(reference_pattern=r"\[(\d+)\]", return_only_referenced_documents=True)

result = builder.run(
    query="test query",
    replies=["First [3], then [10], and finally [50]."],
    documents=[Document(content=f"doc{i}") for i in range(1, 51)],
)

order = [doc.meta["source_index"] for doc in result["answers"][0].documents]
print(order)
# before: [10, 3, 50]   # set hash-table order (deterministic, but not source order)
# after:  [3, 10, 50]    # ascending source-index order

Notes

Wording-only, behavior-preserving fix plus one regression test. No public API change.

The referenced document indices were kept in a set and iterated directly, so the
returned documents came out in the set's hash-table order rather than ascending
source-index order (e.g. citations [3] [10] [50] yielded documents ordered 10, 3,
50). This order was deterministic but did not match the source order. Iterate the
indices sorted so documents follow ascending source index.
@Otis0408
Otis0408 requested a review from a team as a code owner July 10, 2026 09:26
@Otis0408
Otis0408 requested review from julian-risch and removed request for a team July 10, 2026 09:26
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

@Otis0408 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Jul 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/builders
  answer_builder.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant