feat(completion): value-based completion after pipes

[laurigates]

feat(completion): value-based completion after pipes

Implements completion beyond root JSON paths for #114, building on the design
discussion there between @ynqa, @01mf02, and @wader. It uses the value-based
analysis that thread converged on — run the filter prefix and derive
candidates from the values flowing at that point — and now follows the agreed
token-tree segmentation and cursor-aware model rather than the original
proof-of-concept's heuristics.

What works

Query (‸ = cursor)	Suggests	Notes
{first: 1} | .fi‸	.first	MUST case — candidate from a computed value, not the input
.foo | .b‸	.foo | .bar, .foo | .baz	relative to .foo, not root
.items[] | .‸	.name, .qty	element-relative
.items[] | .na‸me	.name (keeps me after cursor)	cursor mid-line — only the text left of the cursor is segmented
.f‸ (no pipe)	unchanged	existing root behavior preserved

How it works (reuses existing primitives)

1. Segment the text left of the cursor at the last top-level | into
   (base, segment) using jaq's own tokenizer (jaq_core::load::lex), so it
   can't disagree with how jq parses the query — string literals, interpolation,
   and (...)/[...]/{...} nesting are handled by the lexer, and |= is a
   distinct token rather than a boundary. Text right of the cursor is preserved
   verbatim.
2. Root case: unchanged — the existing incremental SuggestionStore.
3. Piped case: evaluate base with the existing run_jaq, enumerate its
   output's paths with get_all_paths, filter by segment, and splice the
   chosen suggestion back in as prefix + suggestion + suffix, landing the
   cursor right after the inserted text. Parsed input is cached so a piped
   completion doesn't re-deserialize per keypress.

Relative to the agreed design

The two items the #114 thread settled on as the right direction are now in:

• ✅ Token-based segmentation via load::lex token trees (replacing the
  PoC's byte scanner) — thanks @01mf02 for the pointer and the offer to help.
• ✅ Cursor-awareness — segmentation now acts on the text left of the
  cursor and preserves the right side, instead of assuming cursor-at-end.

Still intentionally out of scope:

• WANT (select(.a.b<cur> | ...), in-paren) depends on the new yield
  filter (Implement yield $x 01mf02/jaq#426), which is still unmerged and not in released jaq-core.
  jnv now tracks jaq 3.0.0; this case is a natural follow-up once yield lands.
• def interiors remain OUT, per @ynqa's scoping comment.

Testing

cargo test covers segmentation (strings, ()/[]/{} nesting, |=,
trailing-dot, unbalanced-input fallback), the cursor cases (mid-line, mid-pipe,
cursor-at-end regression, multi-byte boundary), the char→byte cursor
conversion, relative-path enumeration, array-element descent, and invalid-base
fallback. cargo fmt / cargo clippy clean.

Happy to adjust scope or approach — let me know if anything here should look
different.

---

🤖 Generated with Claude Code

[laurigates]

Pushed a follow-up commit: segmentation now uses jaq's own tokenizer (jaq_core::load::lex) instead of the hand-rolled byte scanner, per @01mf02's guidance in #114.

last_top_level_pipe now walks only the top-level Vec<Token>, keeping the last Tok::Sym == "|" — delimited groups (Tok::Block) and string literals/interpolation (Tok::Str) keep nested pipes off the top level for free, and |= lexes as a distinct Tok::Sym so the exclusion is structural rather than a manual = lookahead. All the manual string/escape/depth bookkeeping is deleted, so segmentation can no longer drift from how jq parses the query.

Faithful port — the analyze() -> CompletionCtx contract and all existing tests are unchanged; added token-specific tests (.a., an unbalanced select(.a that degrades to root, and a pipe inside "\(.a|.b)" interpolation). A lex failure on unbalanced mid-typing input degrades to the root context, which is never worse than the old scanner. Cursor-awareness and in-paren (WANT) completion remain explicit follow-ups.

[laurigates]

Pushed a follow-up implementing cursor-awareness (the next item on the roadmap from #114): completion now segments only the text left of the cursor and preserves the text to its right, instead of treating the whole line as the segment.

So TAB at .items[] | .na‸me (‸ = cursor) completes .na → .name and keeps me after the cursor, landing the cursor right after the inserted suggestion. Cursor-at-end is byte-identical to the previous behavior (the preserved suffix is empty), so it's regression-safe.

Implementation builds on the token-tree port: a pure segment_at_cursor(query, cursor) slices at the cursor byte offset and reuses the unchanged analyze on the left side; the editor converts promkit's char-index cursor to a byte offset and splices suggestions mid-line. New table-driven tests cover the cursor cases (including a multi-byte boundary) plus the char→byte conversion.

The in-paren select(...) WANT case remains a separate follow-up — it needs jaq's unreleased yield branch (jaq#426).

[laurigates]

Related PRs (no strict merge order) — #116 and #118 (opt-in vi-style editing) both touch src/query_editor.rs and src/main.rs, so whichever merges first, the other needs a quick rebase. Independent in intent — no required order. (#119, editor keybinds, is conflict-free with this PR.)

[01mf02]

I would make a separate PR for the jaq 3.0 upgrade, to keep this PR smaller.

[laurigates]

Agreed. The jaq 3.0 bump is only here because the token-based segmentation uses jaq_core::load::lex, which is jaq 3.0 API — so this PR currently carries the Cargo.toml/Cargo.lock upgrade as a dependency.

Since #115 (chore: use jaq v3.0) already does exactly that upgrade, I'll drop the Cargo.toml/Cargo.lock changes from this PR and rebase onto main once #115 merges. That leaves #116 as completion-only, depending on the jaq 3.0 that #115 brings in.