Skip to content

[fix](fe) Preserve column identity for forwarded USING keys - #69388

Merged
924060929 merged 2 commits into
apache:masterfrom
924060929:codex/fix-using-groupby
Oct 26, 2026
Merged

924060929 merged 2 commits into
apache:masterfrom
924060929:codex/fix-using-groupby

Conversation

@924060929

@924060929 924060929 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: N/A

Related PR: #64718

A qualified reference to a preserved-side USING key can be rejected even when the same key appears unqualified in GROUP BY. For example:

SELECT t1.a, COUNT(*)
FROM t1 LEFT JOIN t2 USING(a)
GROUP BY a;

The USING-key projection creates a fresh Alias for a column forwarded directly from the preserved side. SELECT t1.a references the original ExprId while GROUP BY a references the new one. The same mismatch affects HAVING and ORDER BY.

Keep the input ExprId for forwarded USING keys while clearing the qualifier of the unqualified view. Preserve qualified outputs and the explicit asterisk output list. FULL OUTER JOIN still builds a new COALESCE expression with its own ExprId, and aggregate validation remains unchanged.

Sharing the forwarded key identity can repeat a producer ExprId in CTE output, for example:

WITH q(x,y,z) AS (
    SELECT a,t1.b,t1.a FROM t1 JOIN t2 USING(a)
)
SELECT * FROM q;

Keep CTE consumer columns in their original positions using the ordered consumer-to-producer map. Flattening the reverse multimap groups equal producer slots and incorrectly changes [x,y,z] into [x,z,y]. Preserve the ordered map when updating nullability, replacing producer ExprIds and pruning variant subpaths, so those rewrites cannot reintroduce the ordering defect.

Release note

Fix spurious GROUP BY errors when qualified and unqualified references to a forwarded USING join key are mixed in SELECT, GROUP BY, HAVING, or ORDER BY. Preserve CTE output column order when the same source column appears more than once.

Check List (For Author)

  • Test: Unit Test
    • Official run-fe-ut.sh: 73 tests, 0 failures/errors across BindUsingJoinTest, LogicalCTEConsumerTest, AnalyzeCTETest and EliminateGroupByKeyTest.
    • Reproduced the CTE column-order failure before the CTE repair: expected [x,y,z], actual [x,z,y].
    • USING tests cover complete planning across retained sides, mixed grouping references, HAVING/ORDER BY, expressions, multiple keys, derived inputs and invalid nullable-side/FULL references.
    • Added CTE column-order cases for inner/left/right USING joins, an actual FE-computed result check for (1,10,1), and repeated-producer tests covering consumer copies, pruning, nullability changes and producer-ID replacement.
    • Added an EXPLAIN regression case to test_using_join_merge_key; the same SQL is covered by the FE full-planner test. The regression suite and BE data execution have not been run locally.
    • FE build through ./build.sh --fe -j4, integrated Checkstyle, and git diff --check.
  • Behavior changed: Yes, valid grouping queries plan successfully and CTE column positions remain intact.
  • Does this need documentation: No.

### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#64718

A qualified reference to a preserved-side USING key can be rejected even when the same key appears unqualified in GROUP BY. For example:

```sql
SELECT t1.a, COUNT(*)
FROM t1 LEFT JOIN t2 USING(a)
GROUP BY a;
```

The USING-key projection creates a fresh Alias for a column forwarded directly from the preserved side. SELECT t1.a then references the original ExprId while GROUP BY a references the new one. Aggregate analysis rejects the query. The same mismatch affects HAVING and ORDER BY.

Keep the input ExprId for forwarded USING keys while clearing the qualifier of the unqualified view. Preserve the original qualified outputs and the explicit asterisk output list. FULL OUTER JOIN still builds a new COALESCE expression with its own ExprId, and aggregate validation remains unchanged.

### Release note

Fix spurious GROUP BY errors when qualified and unqualified references to a forwarded USING join key are mixed in SELECT, GROUP BY, HAVING, or ORDER BY.

### Check List (For Author)

- Test: Unit Test
  - Complete BindUsingJoinTest suite passed through ./run-fe-ut.sh: 29 tests, 0 failures/errors.
  - Added complete-planner cases for both reference directions, HAVING/ORDER BY, compound grouping expressions, multiple USING keys, and derived inputs across INNER, LEFT/RIGHT, and SEMI/ANTI joins.
  - Existing coverage checks star expansion, qualified stars, chained joins, and FULL OUTER COALESCE keys; negative cases keep nullable-side and FULL OUTER key validation intact.
  - Added an EXPLAIN regression case to test_using_join_merge_key; its SQL is also exercised by the FE full-planner unit test. The regression suite itself has not been run locally.
  - Checkstyle and git diff --check.
- Behavior changed: Yes, valid grouping queries now plan successfully.
- Does this need documentation: No.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929

Copy link
Copy Markdown
Contributor Author

run buildall

@924060929

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes for one P1 data-correctness regression: sharing a forwarded USING key's ExprId with its qualified source can silently reorder CTE SELECT-star columns. A producer selecting merged key, payload, qualified key becomes [L,B,L]; LogicalCTEConsumer groups the equal producer keys and exposes [key,key,payload]. The inline finding includes the concrete SQL, reduced plan, expected versus incorrect row, baseline distinction and fix direction. One comment covers both preserved-side branches.

Reviewed all three authoritative changed paths at head 4c48cc1, including parser/binding, scope and slot identity, stars/chained joins/derived inputs, aggregate validation and normalization, CTE construction/inlining, properties, rewrites and physical/result translation. Existing review threads were empty; no existing P0/P1 IDs need carrying forward. No additional user focus was supplied.

Critical checkpoint conclusions:

  • Goal and evidence: preserving identity addresses the intended mixed qualified/unqualified GROUP BY/HAVING/ORDER BY rejection, and new identity/full-planner tests target it. The CTE ordering regression prevents accepting the implementation as complete.
  • Scope, clarity and reuse: production changes are focused and use the existing withQualifier API; no unrelated refactor.
  • Concurrency and locking: statement-local synchronous binding; no new thread, shared mutable state, lock scope or lock-order change.
  • Lifecycle and static initialization: immutable plan-owned expressions; no new resource ownership, cleanup or static initialization dependency.
  • Configuration: no configuration or session variable added or changed.
  • Compatibility: no storage format, function symbol or rolling-upgrade wire contract changed; intended query acceptance expands.
  • Thrift/protobuf: no definitions changed; field-ID/enum compatibility checks are inapplicable.
  • Parallel paths: preserved left/right inner/outer/semi/anti branches, left-oriented parsed ASOF, downstream mark paths and FULL COALESCE isolation were checked. The finding applies to both forwarded branches; no additional issue found in these paths.
  • Special conditions: existing side-selection conditions remain appropriate; preserved-side type/nullability are retained and opposite nullable/FULL identities stay separate. No new defensive or silent-error branch.
  • Test coverage: new cases cover seven join forms, both grouping directions, expressions, multiple keys, HAVING/ORDER BY, derived inputs and nullable/FULL negatives. Existing star/chained cases remain. The mixed-key CTE output-order trigger is missing; add a result-checking regression for the inline finding. The new Groovy case only EXPLAINs.
  • Expected results: no result files changed; the plain EXPLAIN requires no new result block. The existing nine ordered regression result blocks were inspected against their SQL, including unmatched-row NULLs.
  • Observability: existing planner diagnostics suffice for this local identity change; no new operational path requires logs or metrics.
  • Transactions, persistence and failover: no EditLog, transaction, version or replay changes.
  • Data writes and atomicity: no write/commit implementation changed, but misordered SELECT columns can feed wrong positional values into INSERT SELECT or set operations, as described inline.
  • FE/BE variables: none introduced; existing project/result translation and duplicate-aware maps were inspected.
  • Performance: standard qualifier copying replaces Alias creation; no added asymptotic work or other substantiated performance issue.
  • Other concerns/core invariants: view SQL occurrence indices, lineage/visibility, correlation, grouping-set null extension and ordinary duplicated projections were checked without another verified regression. Read-query data correctness is affected by the P1; visible-version, delete-bitmap and BE memory-accounting paths are untouched, and existing AnalysisException behavior remains.

Converged after 2 review rounds: all three assigned reviewers returned NO_NEW_VALUABLE_FINDINGS on the identical frozen inline set; all candidates and risk items have evidence-backed dispositions, and the final changed-path/anchor sweep is complete. No unresolved static coverage gaps remain. Validation was static only: no builds, tests or query execution were performed, as required by this review contract.

Review converged after 2 round(s).

} else {
// LEFT OUTER / INNER / LEFT SEMI / LEFT ANTI / CROSS: use left side
mergeExpr = new Alias(leftSlot, colName);
mergeExpr = leftSlot.withQualifier(ImmutableList.of());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve CTE column order when sharing the USING-key ExprId

WITH q(x,y,z) AS (SELECT a,t1.b,t1.a FROM t1 JOIN t2 USING(a)) SELECT * FROM q now exposes its columns in the wrong order. The producer becomes:

Project(a#L, t1.b#B, t1.a#L)
  Join(t1.a = t2.a)

AnalyzeCTE wraps this with column names [x#L,y#B,z#L], retaining the ExprIds. LogicalCTEConsumer then keys producerToConsumerOutputMap by producer Slot, whose equality compares only ExprId, and computeOutput() returns that multimap's grouped values(): [x,z,y]. Star expansion binds this order before CTE inlining, which does not repair it. With a matching t1(a,b)=(1,10) row, the result is (1,1,10) instead of (1,10,1); positional consumers such as UNION or INSERT SELECT can consequently receive wrong values. Previously the merged Alias had a separate ExprId, so this query preserved its column order. The right-preserved branch has the same issue.

Preserve producer column positions in the CTE consumer alongside this identity change, and add a result-checking regression with merged and qualified keys separated by a non-key column.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed locally: the reported SQL fails on the original PR head with expected column order [x,y,z] but actual [x,z,y]. Fixed in 363d34b.

LogicalCTEConsumer now takes its output from the insertion-ordered consumer-to-producer map, whose distinct consumer keys preserve each producer position. Nullability adjustment, producer ExprId replacement and variant subpath pruning also retain that order when rebuilding the map. The USING-key identity fix remains intact.

Added inner/left/right USING CTE order checks with full planning, an actual FE-computed repeated-column result check for (1,10,1), and consumer copy/pruning/nullability/ExprId replacement tests. All 73 focused FE tests passed through run-fe-ut.sh; the full FE build and Checkstyle also passed. The result check uses the FE result-computation path; no BE execution is claimed.

### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#69388

Problem Summary: A CTE selecting a forwarded USING key, a payload column,
and the qualified key can expose columns in the wrong order because the
reverse producer-to-consumer multimap groups equal producer ExprIds.
Use the ordered consumer-to-producer map for output positions and preserve
that order when rebuilding mappings for nullability, producer ExprId
replacement and variant subpath pruning. Cover inner/left/right USING CTEs,
FE-computed result values, consumer copies, pruning and mapping rewrites.

### Release note

Preserve CTE result column order when the same source column appears more
than once in its output.

### Check List (For Author)

- Test: Unit Test
    - Official run-fe-ut.sh: 73 tests passed across BindUsingJoinTest,
      LogicalCTEConsumerTest, AnalyzeCTETest and EliminateGroupByKeyTest.
    - Reproduced the column-order failure before the repair.
    - Integrated Checkstyle and git diff --check passed.
- Behavior changed: Yes, repeated CTE source columns retain their positions.
- Does this need documentation: No
@924060929

Copy link
Copy Markdown
Contributor Author

run buildall

@924060929

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head 363d34b70578aee08231e8082b6a0151ce199ac0: no new actionable findings. The existing P1 4334911111 is independently confirmed repaired: distinct consumer keys retain [x,y,z] when producer ExprIds repeat, and copies, pruning, nullability/ExprId/type rewrites, variant expansion, inlining and materialized translation preserve explicit column mappings. No existing blocking comment remains applicable from the supplied threads.

All eight changed files and their relevant upstream/downstream paths were covered. No additional user review focus was supplied.

Critical checkpoint conclusions:

  1. Goal/data correctness: preserved qualified and unqualified USING keys share identity; nullable-side keys remain distinct and FULL keeps a separate COALESCE Alias. Inspected tests address grouping and repeated CTE positions.
  2. Scope/reuse: focused changes reuse the existing binder, consumer maps and rewrite interfaces; the CTE repair is necessary for the identity change.
  3. Concurrency/locking: statement-local rewriting and immutable plan-map snapshots; no new threads, shared mutations, catalog IO or locks.
  4. Lifecycle/static initialization: traced consumer construction through reconstruction, inlining and physical translation; no new resource or static-initialization dependency.
  5. Configuration: no configuration additions or dynamic-update behavior changes.
  6. Compatibility: no storage-format, external-symbol or wire-contract change requiring rolling-upgrade handling.
  7. Thrift/protobuf: no changed definitions.
  8. Parallel paths: checked left/right inner, outer, semi/anti, FULL, nested USING, qualified/unqualified stars, both CTE inline implementations and materialized mappings. No new rule, stage, mark-join or distribution restriction.
  9. Conditional checks: preserved-side and FULL branches match their semantics; no speculative fallback or swallowed invariant failure introduced.
  10. Test coverage: inspected positive/negative grouping, expressions, multiple keys, derived inputs, INNER/LEFT/RIGHT CTE order, consumer copy/prune/nullability/ID tests and FE-computed (1,10,1) result assertion.
  11. Expected results: no golden result file changed; the new EXPLAIN case exercises planning and existing result cases are deterministic.
  12. Observability: no new operational path requiring logs/metrics; existing planner diagnostics remain applicable.
  13. Transactions/persistence/failover: no EditLog, transaction or persisted-metadata changes.
  14. Writes/atomicity/crash cleanup: no data-write protocol or resource-owning path changed.
  15. FE/BE propagation: no new transmitted variables or serialization fields; visible-version and delete-bitmap mechanisms are unaffected.
  16. Performance/memory: query-owned map snapshots retain existing ownership; forwarding removes redundant Alias creation, and the changed entry loops avoid repeated reverse-group traversal. No new obvious hot-path cost found; no benchmark claim.
  17. Other concerns: final sweep found no unresolved candidate, risk focus or uncovered changed file.

Validation limits: static review only; no builds, style checks or tests were run in this invocation. The author's reported test runs were not independently reproduced. The new aggregate regression checks EXPLAIN/full planning, and the value test computes a constant result in FE; added tests do not establish BE-executed results for materialized USING/variant CTE combinations. Those paths were traced statically with no concrete defect found.

Converged after 1 review round: both complete-coverage reviewers and the separate risk-focused reviewer returned NO_NEW_VALUABLE_FINDINGS; all candidate and risk dispositions were independently checked, and the final sweep is complete.

Review converged after 1 round(s).

924060929 added a commit to 924060929/incubator-doris that referenced this pull request Oct 21, 2026
### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#69388

Problem Summary: A CTE selecting a forwarded USING key, a payload column,
and the qualified key can expose columns in the wrong order because the
reverse producer-to-consumer multimap groups equal producer ExprIds.
Use the ordered consumer-to-producer map for output positions and preserve
that order when rebuilding mappings for nullability, producer ExprId
replacement and variant subpath pruning. Cover inner/left/right USING CTEs,
FE-computed result values, consumer copies, pruning and mapping rewrites.
Adapt the backported tests to the branch Java 8 API target.

### Release note

Preserve CTE result column order when the same source column appears more
than once in its output.

### Check List (For Author)

- Test: Unit Test
    - Official run-fe-ut.sh: 73 tests passed across BindUsingJoinTest,
      LogicalCTEConsumerTest, AnalyzeCTETest and EliminateGroupByKeyTest.
    - Reproduced the column-order failure before the repair.
    - Integrated Checkstyle and git diff --check passed.
- Behavior changed: Yes, repeated CTE source columns retain their positions.
- Does this need documentation: No

(cherry picked from commit 363d34b)
@924060929

Copy link
Copy Markdown
Contributor Author

run buildall

@924060929

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of PR #69388 at 363d34b: no new substantiated findings. All eight changed paths and relevant upstream/downstream behavior were reviewed. The review converged in one round; all three independent reviewers returned NO_NEW_VALUABLE_FINDINGS, and both initial risk focuses have evidence-backed dispositions.

Existing P1 4334911111 is repaired at this head. For producer output [x#L,y#B,z#L], LogicalCTEConsumer now exposes distinct consumer keys in [x,y,z] order. Nullability, producer-ID replacement and variant rebuilding retain forward-map order. Inlining uses that output order; materialization maps producer IDs to all corresponding consumers and constructs projections in consumer order. No existing blocking comment remains applicable from the supplied thread, and none is reposted.

Critical checkpoint conclusions:

  1. Goal and evidence: preserved-side USING references now share identity for GROUP BY/HAVING/ORDER BY; FULL OUTER retains an independent COALESCE result. Added FE tests cover grouping, invalid nullable-side/FULL references, repeated CTE positions and an FE-computed (1,10,1) result.
  2. Scope and reuse: changes are focused and reuse existing slot-copying and ordered-map APIs; the CTE repair directly supports the identity change.
  3. Concurrency and locks: these query-local transforms introduce no thread, shared-state or locking changes.
  4. Lifecycle: creation, copying, pruning, nullable/type/ID/variant rewriting, both inliners and physical lowering were traced; no new resource or static lifetime is introduced.
  5. Configuration: no configuration or dynamic-setting behavior changes.
  6. Compatibility: no external symbol, persisted-format or wire-format changes were introduced.
  7. Thrift/protobuf: no changes; field-ID compatibility checks are not applicable.
  8. Parallel paths: inner, left/right outer/semi/anti, FULL, qualified/unqualified stars, chained keys and consumer rebuild/materialization paths were checked. No new rule registration/property alternative or mark-join change.
  9. Conditions: the existing FULL-computed/right-preserved/left-preserved selection remains justified and documented; opposite-side identities remain distinct.
  10. Test coverage: positive, negative and full-planner FE cases are present. The added regression checks EXPLAIN success. This invocation ran no builds or tests, as required by the review contract.
  11. Expected results: [x,y,z] and (1,10,1) match the intended positional semantics; no regression result file was changed.
  12. Observability: existing analysis errors and plan output suffice for these local fixes; no new logging or metric requirement was identified.
  13. Persistence and transactions: no EditLog, replay, visible-version, delete-bitmap or transaction path changes.
  14. Writes and atomicity: no storage-write or commit mechanism changes; query identity and output positions were the applicable correctness concerns.
  15. FE-BE variables: no new transmitted field or variable; existing ID mapping and projection translation were traced.
  16. Performance: forward-entry rebuilding removes repeated reverse-group traversal for repeated producers; no demonstrated performance regression.
  17. Other concerns: identity collisions, per-slot metadata, SQL-index preservation and positional downstream consumers were checked; no unresolved code candidate remains.

No additional user focus was supplied. Validation limits: this is static analysis, and the author's reported test/build results were not independently executed. The new tests do not add BE aggregate-result, forced-materialization or variant-order execution assertions; those paths were inspected in source. No changed path remains unreviewed, and these validation limits are not presented as verified defects.

Review converged after 1 round(s).

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 11.59% (16/138) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28263 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17590	5127	5042	5042
q2	2093	318	255	255
q3	10245	1342	801	801
q4	4680	465	336	336
q5	7443	789	550	550
q6	181	188	155	155
q7	732	806	602	602
q8	9254	1325	1056	1056
q9	5480	4172	4152	4152
q10	6827	1279	1027	1027
q11	419	245	233	233
q12	640	400	281	281
q13	18102	3567	2961	2961
q14	277	265	243	243
q15	q16	716	714	665	665
q17	1182	769	592	592
q18	7048	6187	6101	6101
q19	1112	974	598	598
q20	382	330	225	225
q21	5156	2137	2320	2137
q22	343	291	251	251
Total cold run time: 99902 ms
Total hot run time: 28263 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5628	5634	5594	5594
q2	683	592	528	528
q3	5192	5332	4902	4902
q4	2018	2159	1546	1546
q5	5275	5164	5201	5164
q6	312	204	159	159
q7	2383	1986	1879	1879
q8	3079	2726	2733	2726
q9	7952	8032	7896	7896
q10	3687	3649	3257	3257
q11	622	449	405	405
q12	699	748	644	644
q13	3490	3941	3330	3330
q14	319	311	317	311
q15	q16	704	709	661	661
q17	8330	7460	6871	6871
q18	13043	12163	12856	12163
q19	899	761	782	761
q20	2156	2141	1917	1917
q21	5810	4720	4849	4720
q22	486	457	438	438
Total cold run time: 72767 ms
Total hot run time: 65872 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 150467 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query5	4320	589	447	447
query6	444	206	190	190
query7	4815	452	220	220
query8	319	173	155	155
query9	8769	3103	3122	3103
query10	493	299	263	263
query11	5915	3882	3545	3545
query12	141	95	88	88
query13	1251	428	324	324
query14	6098	3949	3699	3699
query14_1	3512	3438	3534	3438
query15	215	205	182	182
query16	958	437	416	416
query17	858	664	526	526
query18	2423	416	314	314
query19	196	166	135	135
query20	84	82	82	82
query21	214	128	106	106
query22	12940	13019	12722	12722
query23	13432	12872	12419	12419
query23_1	12533	12540	12507	12507
query24	7342	820	470	470
query24_1	478	477	470	470
query25	541	416	368	368
query26	1263	252	140	140
query27	2755	440	274	274
query28	4604	1954	1919	1919
query29	1583	629	430	430
query30	296	204	183	183
query31	996	920	805	805
query32	143	89	85	85
query33	514	284	230	230
query34	895	834	497	497
query35	720	731	641	641
query36	810	840	725	725
query37	119	95	84	84
query38	1765	1733	1673	1673
query39	785	796	741	741
query39_1	729	734	739	734
query40	212	116	104	104
query41	81	65	64	64
query42	88	84	83	83
query43	344	357	318	318
query44	1292	683	678	678
query45	170	174	157	157
query46	832	959	566	566
query47	2849	2946	2556	2556
query48	310	307	212	212
query49	582	393	289	289
query50	656	267	209	209
query51	10608	10492	10523	10492
query52	79	83	71	71
query53	194	205	149	149
query54	243	203	177	177
query55	80	70	65	65
query56	228	231	218	218
query57	1687	1628	1573	1573
query58	274	272	251	251
query59	2133	2137	1984	1984
query60	299	239	234	234
query61	148	147	153	147
query62	389	324	280	280
query63	194	157	157	157
query64	2737	911	763	763
query65	3317	3270	3261	3261
query66	1776	416	301	301
query67	20451	19952	20127	19952
query68	3047	1019	600	600
query69	380	290	237	237
query70	912	831	802	802
query71	310	242	211	211
query72	2737	2574	2292	2292
query73	525	518	299	299
query74	4648	4554	4324	4324
query75	2346	2305	1919	1919
query76	2322	1005	618	618
query77	357	393	291	291
query78	9636	9696	9102	9102
query79	1068	874	524	524
query80	557	464	369	369
query81	526	317	267	267
query82	590	148	109	109
query83	300	200	195	195
query84	308	131	104	104
query85	972	416	360	360
query86	329	232	230	230
query87	2020	1950	1796	1796
query88	3618	2694	2675	2675
query89	339	280	241	241
query90	1866	176	165	165
query91	162	150	131	131
query92	102	86	91	86
query93	996	1009	575	575
query94	429	288	276	276
query95	557	397	308	308
query96	660	488	231	231
query97	2460	2512	2313	2313
query98	163	180	149	149
query99	775	763	658	658
Total cold run time: 234104 ms
Total hot run time: 150467 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query1	0.00	0.00	0.00
query2	0.10	0.05	0.05
query3	0.27	0.09	0.09
query4	1.61	0.10	0.09
query5	0.24	0.25	0.25
query6	1.17	0.69	0.68
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.40	0.35	0.36
query10	0.58	0.56	0.61
query11	0.19	0.14	0.14
query12	0.18	0.16	0.14
query13	0.51	0.50	0.50
query14	1.02	1.03	1.00
query15	0.64	0.62	0.61
query16	0.34	0.35	0.36
query17	1.16	1.10	1.09
query18	0.20	0.19	0.20
query19	2.02	1.90	1.91
query20	0.02	0.01	0.02
query21	15.50	0.20	0.15
query22	4.85	0.06	0.05
query23	16.27	0.32	0.12
query24	2.66	0.59	0.44
query25	0.10	0.06	0.05
query26	1.55	0.21	0.15
query27	0.05	0.04	0.04
query28	3.88	0.80	0.36
query29	12.50	4.04	3.25
query30	0.28	0.12	0.10
query31	2.77	0.55	0.32
query32	3.21	0.60	0.49
query33	3.23	3.15	3.20
query34	15.39	4.03	3.36
query35	3.34	3.35	3.37
query36	0.55	0.46	0.43
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.15	0.14
query41	0.08	0.04	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 97.38 s
Total hot run time: 24.19 s

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28815 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17591	5137	5065	5065
q2	2084	306	267	267
q3	10256	1353	820	820
q4	4678	466	333	333
q5	7440	803	532	532
q6	184	183	155	155
q7	727	802	594	594
q8	9257	1311	1187	1187
q9	5474	4171	4149	4149
q10	6828	1277	1036	1036
q11	441	244	231	231
q12	635	394	277	277
q13	18102	3474	3006	3006
q14	270	271	244	244
q15	q16	737	726	663	663
q17	1175	744	571	571
q18	6885	6115	6093	6093
q19	1117	949	608	608
q20	376	330	218	218
q21	4768	2583	2487	2487
q22	374	313	279	279
Total cold run time: 99399 ms
Total hot run time: 28815 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5975	5851	5897	5851
q2	712	608	545	545
q3	5540	5720	5257	5257
q4	2001	2141	1561	1561
q5	5935	5726	5666	5666
q6	248	188	148	148
q7	2289	2000	1791	1791
q8	2980	2611	2692	2611
q9	7680	7472	7470	7470
q10	3613	3561	3172	3172
q11	581	409	424	409
q12	674	660	508	508
q13	3263	3604	2958	2958
q14	284	329	268	268
q15	q16	652	678	604	604
q17	7794	7021	6891	6891
q18	12955	12211	12828	12211
q19	891	764	767	764
q20	2129	2135	1911	1911
q21	5708	4734	4702	4702
q22	486	452	398	398
Total cold run time: 72390 ms
Total hot run time: 65696 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 149961 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query5	4308	579	440	440
query6	429	208	188	188
query7	4802	436	232	232
query8	313	161	162	161
query9	8772	3089	3099	3089
query10	499	312	255	255
query11	6038	3842	3580	3580
query12	144	89	85	85
query13	1251	461	311	311
query14	6108	3910	3644	3644
query14_1	3375	3393	3418	3393
query15	203	195	173	173
query16	936	428	388	388
query17	808	623	510	510
query18	2404	409	296	296
query19	191	159	131	131
query20	88	81	79	79
query21	208	128	103	103
query22	12985	12957	12862	12862
query23	13559	12892	12550	12550
query23_1	12482	12508	12504	12504
query24	7404	794	464	464
query24_1	460	467	462	462
query25	504	393	338	338
query26	1262	250	139	139
query27	2774	427	266	266
query28	4646	1906	1922	1906
query29	1516	582	432	432
query30	303	221	181	181
query31	1016	913	820	820
query32	130	88	88	88
query33	514	289	228	228
query34	890	833	493	493
query35	729	743	648	648
query36	824	811	699	699
query37	120	98	75	75
query38	1808	1738	1666	1666
query39	789	805	744	744
query39_1	731	725	728	725
query40	222	116	99	99
query41	72	68	69	68
query42	102	84	84	84
query43	340	364	307	307
query44	1287	684	688	684
query45	179	174	160	160
query46	821	957	570	570
query47	2896	2995	2440	2440
query48	298	309	221	221
query49	596	402	302	302
query50	671	286	213	213
query51	10547	10703	10136	10136
query52	80	80	75	75
query53	186	210	153	153
query54	245	203	203	203
query55	75	69	69	69
query56	275	239	238	238
query57	1630	1588	1572	1572
query58	287	270	266	266
query59	2075	2127	1918	1918
query60	308	254	234	234
query61	164	166	168	166
query62	392	334	323	323
query63	185	157	162	157
query64	2708	929	760	760
query65	3317	3265	3249	3249
query66	1769	428	298	298
query67	20187	20102	19839	19839
query68	3013	1002	596	596
query69	395	282	250	250
query70	828	760	763	760
query71	301	233	215	215
query72	2689	2589	2313	2313
query73	507	520	290	290
query74	4599	4526	4386	4386
query75	2330	2268	1877	1877
query76	2319	997	607	607
query77	347	360	272	272
query78	9825	9625	9152	9152
query79	991	857	506	506
query80	505	411	343	343
query81	522	316	267	267
query82	571	126	99	99
query83	301	193	174	174
query84	312	128	95	95
query85	813	415	357	357
query86	319	241	225	225
query87	1995	1948	1828	1828
query88	3617	2688	2671	2671
query89	326	275	235	235
query90	1896	158	168	158
query91	204	152	131	131
query92	97	78	90	78
query93	999	1001	561	561
query94	448	299	265	265
query95	588	399	301	301
query96	665	477	234	234
query97	2446	2460	2343	2343
query98	177	151	149	149
query99	775	765	657	657
Total cold run time: 233713 ms
Total hot run time: 149961 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.17 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query1	0.01	0.01	0.01
query2	0.09	0.05	0.05
query3	0.26	0.10	0.09
query4	1.61	0.10	0.10
query5	0.23	0.23	0.24
query6	1.17	0.68	0.68
query7	0.03	0.01	0.00
query8	0.06	0.03	0.04
query9	0.41	0.34	0.35
query10	0.58	0.59	0.61
query11	0.20	0.15	0.14
query12	0.18	0.14	0.15
query13	0.51	0.49	0.48
query14	1.02	0.99	1.01
query15	0.63	0.61	0.61
query16	0.36	0.33	0.34
query17	1.13	1.16	1.11
query18	0.21	0.19	0.19
query19	2.11	1.96	2.00
query20	0.02	0.01	0.01
query21	15.44	0.21	0.14
query22	4.81	0.06	0.06
query23	16.27	0.32	0.12
query24	3.02	0.58	0.34
query25	0.11	0.05	0.06
query26	1.16	0.22	0.16
query27	0.05	0.04	0.05
query28	3.11	0.72	0.36
query29	12.51	4.03	3.24
query30	0.30	0.13	0.10
query31	2.77	0.55	0.32
query32	3.23	0.59	0.50
query33	3.30	3.26	3.16
query34	15.34	3.98	3.40
query35	3.34	3.33	3.34
query36	0.54	0.45	0.43
query37	0.09	0.06	0.06
query38	0.06	0.03	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.64 s
Total hot run time: 24.17 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

2 similar comments
@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28212 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17601	5163	5014	5014
q2	2091	327	255	255
q3	10221	1373	810	810
q4	4682	456	333	333
q5	7453	792	529	529
q6	173	183	154	154
q7	737	786	598	598
q8	9277	1344	1036	1036
q9	5496	4144	4188	4144
q10	6830	1295	1011	1011
q11	441	250	228	228
q12	634	392	278	278
q13	18098	3560	3005	3005
q14	271	273	247	247
q15	q16	713	705	645	645
q17	1181	747	576	576
q18	6953	6178	6111	6111
q19	1118	949	602	602
q20	367	327	219	219
q21	4755	2156	2361	2156
q22	342	296	261	261
Total cold run time: 99434 ms
Total hot run time: 28212 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5634	5612	5616	5612
q2	708	584	516	516
q3	5171	5342	4948	4948
q4	2023	2128	1543	1543
q5	5303	5149	5441	5149
q6	252	204	163	163
q7	2339	2045	1888	1888
q8	3015	2618	2666	2618
q9	8043	8079	7984	7984
q10	3719	3706	3250	3250
q11	780	445	424	424
q12	723	694	539	539
q13	3418	3893	3183	3183
q14	320	316	287	287
q15	q16	698	753	658	658
q17	8346	7542	6948	6948
q18	13032	12185	12853	12185
q19	895	779	777	777
q20	2129	2159	1925	1925
q21	5608	4612	4720	4612
q22	502	442	390	390
Total cold run time: 72658 ms
Total hot run time: 65599 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 150296 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query5	4326	596	452	452
query6	439	204	186	186
query7	4807	452	256	256
query8	336	163	155	155
query9	8781	3097	3080	3080
query10	482	311	252	252
query11	6039	3875	3554	3554
query12	161	91	88	88
query13	1243	431	358	358
query14	6152	3929	3640	3640
query14_1	3429	3437	3434	3434
query15	214	193	182	182
query16	935	424	409	409
query17	847	652	536	536
query18	2434	426	317	317
query19	196	159	133	133
query20	87	91	82	82
query21	213	131	108	108
query22	13048	12979	12785	12785
query23	13430	12725	12534	12534
query23_1	12555	12490	12490	12490
query24	7378	851	463	463
query24_1	473	475	477	475
query25	541	417	355	355
query26	1253	262	140	140
query27	2762	450	276	276
query28	4625	1938	1985	1938
query29	1587	620	463	463
query30	301	221	187	187
query31	1017	938	821	821
query32	154	99	96	96
query33	526	314	243	243
query34	972	877	514	514
query35	759	773	646	646
query36	804	802	725	725
query37	124	100	86	86
query38	1790	1807	1690	1690
query39	789	810	759	759
query39_1	730	737	767	737
query40	223	121	100	100
query41	72	107	65	65
query42	90	84	79	79
query43	357	370	315	315
query44	1403	700	705	700
query45	180	184	165	165
query46	846	997	574	574
query47	2990	2819	2460	2460
query48	290	315	219	219
query49	579	398	285	285
query50	718	292	215	215
query51	10512	10501	10277	10277
query52	84	79	70	70
query53	198	207	155	155
query54	231	207	177	177
query55	72	65	65	65
query56	233	225	215	215
query57	1654	1616	1585	1585
query58	279	281	254	254
query59	2118	2181	1899	1899
query60	289	238	224	224
query61	147	143	136	136
query62	385	335	283	283
query63	195	162	160	160
query64	2759	908	746	746
query65	3331	3291	3283	3283
query66	1780	432	311	311
query67	20291	19923	19803	19803
query68	2976	1005	614	614
query69	397	287	246	246
query70	853	797	755	755
query71	299	237	215	215
query72	2736	2578	2284	2284
query73	535	549	297	297
query74	4600	4498	4398	4398
query75	2318	2295	1890	1890
query76	2297	1022	635	635
query77	347	386	282	282
query78	9804	9742	9163	9163
query79	1057	868	508	508
query80	1236	458	342	342
query81	591	317	266	266
query82	621	132	106	106
query83	306	198	183	183
query84	314	129	97	97
query85	847	411	371	371
query86	402	236	228	228
query87	2034	1945	1798	1798
query88	3759	2690	2672	2672
query89	331	276	237	237
query90	1855	171	172	171
query91	160	152	132	132
query92	93	88	89	88
query93	1089	1012	574	574
query94	644	304	265	265
query95	567	386	303	303
query96	678	511	231	231
query97	2439	2450	2336	2336
query98	169	157	147	147
query99	759	762	655	655
Total cold run time: 235767 ms
Total hot run time: 150296 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 363d34b70578aee08231e8082b6a0151ce199ac0, data reload: false

query1	0.00	0.00	0.01
query2	0.10	0.06	0.05
query3	0.26	0.08	0.08
query4	1.61	0.10	0.11
query5	0.24	0.24	0.24
query6	1.17	0.69	0.68
query7	0.04	0.01	0.00
query8	0.05	0.03	0.04
query9	0.42	0.36	0.36
query10	0.58	0.59	0.60
query11	0.19	0.14	0.15
query12	0.18	0.15	0.15
query13	0.49	0.50	0.49
query14	1.06	1.00	1.01
query15	0.62	0.61	0.60
query16	0.32	0.37	0.36
query17	1.11	1.11	1.15
query18	0.21	0.19	0.20
query19	2.05	1.90	2.02
query20	0.02	0.01	0.01
query21	15.49	0.20	0.13
query22	4.84	0.06	0.05
query23	16.28	0.31	0.12
query24	3.13	0.50	0.37
query25	0.10	0.05	0.05
query26	0.97	0.22	0.16
query27	0.04	0.03	0.04
query28	3.11	0.74	0.38
query29	12.49	4.14	3.23
query30	0.29	0.15	0.12
query31	2.76	0.56	0.32
query32	3.22	0.60	0.51
query33	3.16	3.19	3.31
query34	15.41	4.03	3.41
query35	3.34	3.33	3.34
query36	0.56	0.47	0.43
query37	0.10	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.04	0.03
query40	0.16	0.15	0.14
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 96.41 s
Total hot run time: 24.28 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 93.75% (15/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

2 similar comments
@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (16/16) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 4.55% (16/352) 🎉
Increment coverage report
Complete coverage report

yiguolei pushed a commit that referenced this pull request Oct 22, 2026
…ys (branch-4.2) (#69389)

Backport of #69388 to branch-4.2. The production repair is unchanged;
the backported tests use Java 8-compatible APIs.

### What problem does this PR solve?

Issue Number: N/A

Related PR: #69388

A qualified reference to a preserved-side USING key can be rejected even
when the same key appears unqualified in GROUP BY. For example:

```sql
SELECT t1.a, COUNT(*)
FROM t1 LEFT JOIN t2 USING(a)
GROUP BY a;
```

The USING-key projection creates a fresh Alias for a column forwarded
directly from the preserved side. SELECT t1.a references the original
ExprId while GROUP BY a references the new one. The same mismatch
affects HAVING and ORDER BY.

Keep the input ExprId for forwarded USING keys while clearing the
qualifier of the unqualified view. Preserve qualified outputs and the
explicit asterisk output list. FULL OUTER JOIN still builds a new
COALESCE expression with its own ExprId, and aggregate validation
remains unchanged.

Sharing the forwarded key identity can repeat a producer ExprId in CTE
output, for example:

```sql
WITH q(x,y,z) AS (
    SELECT a,t1.b,t1.a FROM t1 JOIN t2 USING(a)
)
SELECT * FROM q;
```

Keep CTE consumer columns in their original positions using the ordered
consumer-to-producer map. Flattening the reverse multimap groups equal
producer slots and incorrectly changes [x,y,z] into [x,z,y]. Preserve
the ordered map when updating nullability, replacing producer ExprIds
and pruning variant subpaths, so those rewrites cannot reintroduce the
ordering defect.

### Release note

Fix spurious GROUP BY errors when qualified and unqualified references
to a forwarded USING join key are mixed in SELECT, GROUP BY, HAVING, or
ORDER BY. Preserve CTE output column order when the same source column
appears more than once.

### Check List (For Author)

- Test: Unit Test
- Official run-fe-ut.sh: 73 tests, 0 failures/errors across
BindUsingJoinTest, LogicalCTEConsumerTest, AnalyzeCTETest and
EliminateGroupByKeyTest.
- Reproduced the CTE column-order failure before the CTE repair:
expected [x,y,z], actual [x,z,y].
- USING tests cover complete planning across retained sides, mixed
grouping references, HAVING/ORDER BY, expressions, multiple keys,
derived inputs and invalid nullable-side/FULL references.
- Added CTE column-order cases for inner/left/right USING joins, an
actual FE-computed result check for (1,10,1), and repeated-producer
tests covering consumer copies, pruning, nullability changes and
producer-ID replacement.
- Added an EXPLAIN regression case to test_using_join_merge_key; the
same SQL is covered by the FE full-planner test. The regression suite
and BE data execution have not been run locally.
- FE server build through DISABLE_BUILD_UI=ON ./build.sh --fe -j4,
integrated Checkstyle, and git diff --check. Unmodified Web UI excluded.
- Behavior changed: Yes, valid grouping queries plan successfully and
CTE column positions remain intact.
- Does this need documentation: No.
@924060929

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of all eight changed files at head 363d34b70578aee08231e8082b6a0151ce199ac0 found no new substantiated findings.

Existing P1 comment 4334911111 was independently revalidated as fixed. Each producer position receives a distinct consumer ExprId, and LogicalCTEConsumer.computeOutput() now reads the ordered consumer-to-producer keys. The nullable, producer-ID and variant rebuilds preserve that order; inlining and materialized CTE translation use explicit identity mappings and ordered projections. The original interleaved [x,y,z] case no longer follows the reported [x,z,y] binding path, so this comment is not carried as an existing blocker or reposted.

Critical checkpoint conclusions:

  1. Goal and supporting tests: forwarding preserved-side USING keys with their input ExprId makes qualified and unqualified references match in GROUP BY, HAVING and ORDER BY. FULL OUTER keys retain a separate COALESCE Alias. Added planner assertions and CTE order/result assertions match these goals; they were inspected, not executed here.
  2. Scope and reuse: the five production files and three test files form a focused identity/order repair using existing slot and ordered-map APIs.
  3. Concurrency and locks: no concurrency or locking behavior is changed; these are staged, query-local plan transformations.
  4. Lifecycle and initialization: fresh per-position consumer IDs, immutable map copies and existing plan-copy lifecycles are retained; no resource or static-initialization change.
  5. Configuration: no configuration item or dynamic-update path is changed.
  6. Compatibility: no public symbol signature, storage format or RPC contract is changed; the intended observable changes are query acceptance and column ordering.
  7. Thrift/protobuf: no gensrc definitions change, so cross-branch field-ID checks are inapplicable.
  8. Parallel paths: reviewed retained left/right join sides, semi/anti, FULL and chained joins, qualifier/star binding, CTE construction/copy/pruning, nullable/ExprId/type/variant rebuilding, inlining, materialization, and positional UNION/INSERT mapping. No separate defect was established.
  9. Conditions and errors: no new conditional guard is added. Existing preserved-side selection remains appropriate; nullable-opposite-side and FULL grouping references retain distinct identities and existing aggregate validation.
  10. Test coverage: added cases cover all seven forwarding join forms, both grouping-reference directions, HAVING/ORDER BY, arithmetic, multiple keys, derived inputs, invalid opposite-side/FULL references, inner/left/right CTE order, consumer copies/pruning/nullability/ID replacement and an FE-computed repeated-column result. The new Groovy case checks EXPLAIN acceptance.
  11. Expected results: [x,y,z], repeated producer identity, nullable positions and the literal result (1,10,1) agree with the traced semantics. No golden output file is added or changed.
  12. Observability: existing analysis errors and EXPLAIN remain appropriate; no new operation requires additional metrics or logging.
  13. Transactions and persistence: no EditLog, replay, version-visibility or failover path is changed.
  14. Writes and crash behavior: no data-write, atomicity or resource-release path is changed.
  15. FE/BE propagation: no new variable or protocol field is introduced. CTE physical output mappings and multicast projection/tuple order were traced explicitly.
  16. Performance: ordered output remains linear in consumer count; the new entry-based visitors avoid repeatedly visiting reverse-map groups for repeated producers. No introduced performance defect was found.
  17. Other correctness checks: slot name/qualifier handling, outer-join nullability, duplicate output positions and map consistency were checked. No unresolved source-level candidate remains; committed-data visibility, delete bitmaps and native memory-accounting mechanisms are untouched.

No additional user-provided focus was supplied. Coverage is complete for the static review scope. No builds, tests, or BE execution were performed. The new tests do not demonstrate BE execution of a materialized USING/variant CTE, and the FE constant-result test does not contain a USING join; execution coverage of those combinations remains a validation limitation. Static tracing resolved the associated source-level risk questions without treating those tests as execution proof.

Review converged after 1 round(s).

@924060929
924060929 merged commit 89f2ed3 into apache:master Oct 26, 2026
53 of 54 checks passed
@924060929
924060929 deleted the codex/fix-using-groupby branch October 26, 2026 09:24
yiguolei pushed a commit that referenced this pull request Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants