Adding v1 operator ARRAY_JOIN

[xiangfu0]

• Extend the SQL parser, PinotQuery thrift schema, and request contexts with ArrayJoin* objects and parsing logic for the new ARRAY JOIN syntax.
• Add integration with the planner and execution layer: include ARRAY JOIN expressions in projection planning and introduce an ArrayJoinOperator placeholder that currently delegates to its child.
• Cover the new behavior with parser/unit tests plus an integration test exercising ARRAY JOIN queries.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 32.79915% with 629 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.33%. Comparing base (2bcbdfe) to head (dedfd87).

Files with missing lines	Patch %	Lines
...not/core/operator/transform/ArrayJoinOperator.java	26.11%	484 Missing and 31 partials ⚠️
...va/org/apache/pinot/core/plan/ProjectPlanNode.java	36.92%	35 Missing and 6 partials ⚠️
...ava/org/apache/pinot/core/plan/FilterPlanNode.java	14.28%	22 Missing and 2 partials ⚠️
...ot/core/query/pruner/ColumnValueSegmentPruner.java	11.76%	12 Missing and 3 partials ⚠️
...sthandler/BaseSingleStageBrokerRequestHandler.java	39.13%	12 Missing and 2 partials ⚠️
...org/apache/pinot/sql/parsers/CalciteSqlParser.java	76.92%	5 Missing and 7 partials ⚠️
.../sql/parsers/parser/SqlArrayJoinTableFunction.java	46.15%	7 Missing ⚠️
.../org/apache/pinot/core/operator/ColumnContext.java	66.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17182      +/-   ##
============================================
- Coverage     65.47%   65.33%   -0.15%     
  Complexity     1421     1421              
============================================
  Files          3426     3429       +3     
  Lines        217315   218241     +926     
  Branches      34509    34765     +256     
============================================
+ Hits         142283   142581     +298     
- Misses        63513    64083     +570     
- Partials      11519    11577      +58     

Flag	Coverage Δ
custom-integration1	?
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	65.33% <32.79%> (-0.15%)	⬇️
temurin	65.33% <32.79%> (-0.15%)	⬇️
unittests	65.32% <32.79%> (-0.15%)	⬇️
unittests1	56.67% <32.63%> (-0.16%)	⬇️
unittests2	37.74% <2.77%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR introduces support for ARRAY JOIN syntax in Apache Pinot, enabling the flattening of array columns into individual rows.

Changes:

• Extends the SQL parser to recognize ARRAY JOIN and LEFT ARRAY JOIN syntax
• Adds Thrift schema definitions for ArrayJoinSpec, ArrayJoinOperand, and ArrayJoinType
• Integrates ARRAY JOIN into the query execution pipeline through new operator and context classes
• Includes comprehensive test coverage across parser, operator, and integration tests

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pinot-common/src/thrift/query.thrift	Defines Thrift schema for ARRAY JOIN structures (spec, operand, type enum)
pinot-common/src/main/codegen/templates/Parser.jj	Adds parser grammar rules for ARRAY JOIN syntax and bracket array constructors
pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java	Implements extraction and compilation of ARRAY JOIN clauses from SQL
pinot-common/src/main/java/org/apache/pinot/sql/parsers/parser/SqlArrayJoinTableFunction.java	Custom Calcite SQL node representing ARRAY JOIN table function
pinot-core/src/main/java/org/apache/pinot/core/query/request/context/ArrayJoinContext.java	Request context for ARRAY JOIN operations
pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java	Extended to include ARRAY JOIN contexts
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/ArrayJoinOperator.java	Core operator implementing array expansion logic
pinot-core/src/main/java/org/apache/pinot/core/plan/ProjectPlanNode.java	Modified to incorporate ARRAY JOIN operators in projection planning
pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java	Updated to skip filter evaluation for ARRAY JOIN columns
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java	Enhanced column name resolution to respect ARRAY JOIN aliases
pinot-integration-tests/.../ArrayTest.java	Integration tests for ARRAY JOIN queries with various combinations

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 36 out of 36 changed files in this pull request and generated no new comments.

[xiangfu0]

Closing this. CROSS JOIN UNNEST landed in the multi-stage engine in #17168 (with column pruning added in #18782) and covers this functionality, plus WITH ORDINALITY which this PR doesn't support. Adding a parallel implementation to the single-stage engine isn't the right investment given the direction toward MSE-by-default.

Two gaps remain worth tracking separately:

• ClickHouse-compatible ARRAY JOIN / LEFT ARRAY JOIN syntax, implemented as a parser-level desugaring to CROSS JOIN UNNEST so it runs on MSE.
• Outer unnest semantics (keep rows whose array is empty) — RelToPlanNodeConverter#convertLogicalCorrelate currently ignores the correlate's join type.