Skip to content

detect() writes converted sidecars into the scanned tree; cache_root does not redirect them #2787

Closed

Description

@sortakool

Summary

detect() writes into the tree it is scanning. When the root contains an Office
or Google-Workspace document, detection converts it and writes the markdown
sidecar to <root>/graphify-out/converted/. That makes detect() a mutating
operation on its own input, which breaks any caller that treats the scanned tree
as immutable.

Observed on 0.9.44.

Why this is a problem, not a quirk

We scan pinned third-party sources in disposable clones and assert afterwards
that the clone is byte-identical to the commit we materialized — the check exists
so that a detector can never silently alter reviewed source. Detection itself
trips that assertion:

disposable clone changed during detection: 2 dirty path(s):
  ['?? graphify-out/converted/example_8a530a1d.md',
   '?? graphify-out/converted/example_d1c39225.md']

The two sidecars come from cognee/tests/test_data/example.docx and
example.pptx. It is fully deterministic: every source shipping an Office
document drifts, and no other source does.

More generally, a read-shaped API that writes to its argument is surprising for
read-only mounts, immutable checkouts, concurrent scans of one tree, and anything
doing provenance or reproducibility checks.

There is no way to redirect it

converted_dir is hardcoded (detect.py, 0.9.44):

converted_dir = root / GRAPHIFY_OUT / "converted"

detect() does accept cache_root, which reads like the knob for this, but it
only reaches _cache.cached_word_count(...) — the word-count cache. Nothing
routes conversion output through it.

Reproduce

mkdir -p repro && cd repro && git init -q
cp /path/to/any.docx .
git add -A && git commit -qm fixture
python -c "from pathlib import Path; from graphify.detect import detect; detect(Path('.'))"
git status --porcelain --untracked-files=all
# ?? graphify-out/converted/any_<hash>.md

Suggested resolution

Any of these would resolve it for us, in preference order:

  1. Honour cache_root for conversion output as well, so a caller can direct
    sidecars outside the scanned tree.
  2. Add an explicit converted_dir / out_dir parameter to detect().
  3. Make conversion opt-in during detect(), given that detection's documented
    job is classification rather than materialization.

Happy to send a PR if you have a preference on the shape.

Activity

  1. sortakool commented on Aug 17, 2026

    @sortakool
    Author

    Flagging that #2794 ("fix(detect): respect cache root for converted sidecars") looks like it addresses this — it declares Fixes #2787 and redirects converted sidecars to <cache_root>/graphify-out/converted/, which matches the behaviour described here.

    It is still open / unmerged at the time of writing, so this is a pointer rather than a resolution.

    One data point that may be useful: the fix is not in the released 0.9.45. Checked against the installed wheel:

    • detect() does accept cache_root — graphify/detect.py:1460 — and threads it into the word-count cache at :1500.
    • But the sidecar destination is still anchored to the scanned root: converted_dir = root / GRAPHIFY_OUT / "converted" at graphify/detect.py:1647.

    So on 0.9.45 detect() can still write into a source tree that was passed a separate cache_root.

    Context for why we noticed: we pin graphify sources at an exact commit and verify the tree digest before extracting, so a detect() run that writes graphify-out/ into the scanned checkout shows up as source drift against a pinned SHA. We currently work around it by materialising each source into a disposable temporary tree before scanning.

    Happy to re-check against a release once #2794 lands, if that is useful.

  2. safishamsi commented on Aug 19, 2026

    @safishamsi
    Member

    Fixed in v0.9.47: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.47 (converted sidecars now write to the cache root, not the source tree). Closing as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions