[Fix](fold_const) MAKE_SET constant folding should clear#64907
Merged
morrySnow merged 1 commit intoJun 29, 2026
Conversation
linrrzqqq
requested review from
924060929,
englefly,
morrySnow and
starocean999
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
run buildall |
Collaborator
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review result: no blocking issue found.
Critical checkpoints:
- Goal: the PR fixes FE constant folding for MAKE_SET when clearing bit positions >= 32 by changing the mask to
1L << pos; this addresses the Java int-shift masking issue described in the PR. - Scope: the authoritative GitHub PR file list shows a focused one-line FE fix plus one regression invocation. Local commit history in this checkout is a shallow/root snapshot, so I used the GitHub PR file metadata as the review scope authority.
- Parallel paths: BE MAKE_SET already clears with an unsigned 64-bit mask, so the FE fold path now matches the runtime path for high bits.
- Tests: the added
testFoldConstcase exercisesMAKE_SET(4294967296, ...)with enough arguments to reach bit 32 and compares FE-folded output against no-fold output. - Concurrency, lifecycle, config, persistence, and compatibility: not implicated by this FE executable-function mask change.
- User focus: no additional user focus was provided.
Subagent conclusions:
- optimizer-rewrite: no candidate findings; convergence round 1 returned
NO_NEW_VALUABLE_FINDINGS. - tests-session-config: no candidate findings; convergence round 1 returned
NO_NEW_VALUABLE_FINDINGS. - No candidates were accepted for inline comments; two suspicious points were dismissed in the shared ledger with code evidence.
Validation limits: I did not run regression tests or FE build locally because this checkout is not worktree-initialized and thirdparty/installed / thirdparty/installed/bin/protoc are absent.
Contributor
TPC-H: Total hot run time: 28955 ms |
Contributor
TPC-DS: Total hot run time: 171049 ms |
Contributor
ClickBench: Total hot run time: 25.17 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Collaborator
Author
|
run feut |
morrySnow
approved these changes
Jun 29, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
morrySnow
added
dev/4.0.x
dev/4.1.x
and removed
approved
Contributor
|
PR approved by anyone and no changes requested. |
github-actions Bot
pushed a commit
that referenced
this pull request
Jun 29, 2026
related PR: #56367 Problem Summary: `MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions in the FE constant folding path, which leads to incorrect clearing of high bits when `pos >= 32` due to integer shift modulo. For Java `int` shifts, the shift distance is masked with 0x1F, which means only the low 5 bits are used: - `pos = 0..31` -> normal - `pos = 32` -> treated as `0` - `pos = 33` -> treated as `1` - `pos = 64` -> treated as `0` again For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32) expect: `bit &= ~(1 << 32)` got: `bit &= ~(1 << 0)` That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32 - the loop never makes progress - the same string is appended again and again - Java throws OutOfMemoryError: Java heap space before(FE constant folding failed or FE OOM): ```text Doris> EXPLAIN SELECT MAKE_SET(4294967296, -> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', -> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', -> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', -> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', -> 'a32', 'a33') AS ms; +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | ms[#0] | | PARTITION: UNPARTITIONED | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 0:VUNION(11) | | constant exprs: | | make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') | | | | | | | | ========== STATISTICS ========== | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 17 rows in set (28.822 sec) ``` ```text 2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a 18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail ed. org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time: {"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1} ``` now: ```text Doris> SET debug_skip_fold_constant = 0; Doris> EXPLAIN SELECT MAKE_SET(4294967296, -> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', -> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', -> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', -> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', -> 'a32', 'a33') AS ms; +----------------------------------+ | Explain String(Nereids Planner) | +----------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | ms[#0] | | PARTITION: UNPARTITIONED | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 0:VUNION(12) | | constant exprs: | | 'a32' | | | | | | | | ========== STATISTICS ========== | +----------------------------------+ ```
github-actions Bot
pushed a commit
that referenced
this pull request
Jun 29, 2026
related PR: #56367 Problem Summary: `MAKE_SET` uses `bit &= ~(1 << pos)` for clearing bits at high positions in the FE constant folding path, which leads to incorrect clearing of high bits when `pos >= 32` due to integer shift modulo. For Java `int` shifts, the shift distance is masked with 0x1F, which means only the low 5 bits are used: - `pos = 0..31` -> normal - `pos = 32` -> treated as `0` - `pos = 33` -> treated as `1` - `pos = 64` -> treated as `0` again For inputs like: `MAKE_SET(4294967296, ...)`(4294967296 == 1L << 32) expect: `bit &= ~(1 << 32)` got: `bit &= ~(1 << 0)` That does **not** clear bit 32 at all. So: - `bit` stays unchanged and `pos` stays 32 - the loop never makes progress - the same string is appended again and again - Java throws OutOfMemoryError: Java heap space before(FE constant folding failed or FE OOM): ```text Doris> EXPLAIN SELECT MAKE_SET(4294967296, -> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', -> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', -> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', -> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', -> 'a32', 'a33') AS ms; +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | ms[#0] | | PARTITION: UNPARTITIONED | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 0:VUNION(11) | | constant exprs: | | make_set(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') | | | | | | | | ========== STATISTICS ========== | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 17 rows in set (28.822 sec) ``` ```text 2026-06-26 23:27:50,285 INFO (mysql-nio-pool-0|303) [StmtExecutor.executeByNereids():821] Command(EXPLAIN SELECT MAKE_SET(4294967296, 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', 'a16', 'a17', 'a 18', 'a19', 'a20', 'a21', 'a22', 'a23', 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', 'a32', 'a33') AS ms) process fail ed. org.apache.doris.nereids.exceptions.AnalysisException: Nereids cost too much time (32s > 30s). You should increment timeout by set 'nerei ds_timeout_second' or disable check timeout by set 'enable_nereids_timeout' to false. Time consuming details, parse time: 6ms, plan time: {"plan":-1,"garbage_collect":-1,"lock_tables":0,"analyze":2,"rewrite":-1,"fold_const_by_be":0,"collect_partitions":-1,"optimize":-1,"tra nslate":-1,"init_scan_node":-1,"finalize_scan_node":-1,"create_scan_range":-1,"distribute":-1} ``` now: ```text Doris> SET debug_skip_fold_constant = 0; Doris> EXPLAIN SELECT MAKE_SET(4294967296, -> 'a00', 'a01', 'a02', 'a03', 'a04', 'a05', 'a06', 'a07', -> 'a08', 'a09', 'a10', 'a11', 'a12', 'a13', 'a14', 'a15', -> 'a16', 'a17', 'a18', 'a19', 'a20', 'a21', 'a22', 'a23', -> 'a24', 'a25', 'a26', 'a27', 'a28', 'a29', 'a30', 'a31', -> 'a32', 'a33') AS ms; +----------------------------------+ | Explain String(Nereids Planner) | +----------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | ms[#0] | | PARTITION: UNPARTITIONED | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 0:VUNION(12) | | constant exprs: | | 'a32' | | | | | | | | ========== STATISTICS ========== | +----------------------------------+ ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related PR: #56367
Problem Summary:
MAKE_SETusesbit &= ~(1 << pos)for clearing bits at high positions in the FE constant folding path, which leads to incorrect clearing of high bits whenpos >= 32due to integer shift modulo.For Java
intshifts, the shift distance is masked with 0x1F, which means only the low 5 bits are used:pos = 0..31-> normalpos = 32-> treated as0pos = 33-> treated as1pos = 64-> treated as0againFor inputs like:
MAKE_SET(4294967296, ...)(4294967296 == 1L << 32)expect:
bit &= ~(1 << 32)got:
bit &= ~(1 << 0)That does not clear bit 32 at all. So:
bitstays unchanged andposstays 32before(FE constant folding failed or FE OOM):
now: