Skip to content

perf: optimize greatest in datafusion-functions#23538

Draft
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/greatest-datafusion-20260713-150618
Draft

perf: optimize greatest in datafusion-functions#23538
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/greatest-datafusion-20260713-150618

Conversation

@andygrove

Copy link
Copy Markdown
Member

This PR was created by an LLM as a draft PR. I will mark it as ready for review after human review.

Which issue does this PR close?

N/A — autonomous exploratory PR.

Rationale for this change

Vectorize greatest's nullable comparison path — replace the per-row boxed make_comparator with cmp::gt_eq plus bitwise null masking, keeping the comparator only for floats where NaN ordering differs.

What changes are included in this PR?

Vectorize greatest's nullable comparison path — replace the per-row boxed make_comparator with cmp::gt_eq plus bitwise null masking, keeping the comparator only for floats where NaN ordering differs.

Are these changes tested?

Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs main).

Benchmark (criterion):

  • greatest_utf8_nullable: 9.023% faster (base 10651ns -> cand 9690ns)
  • greatest_i64_nullable: 21.744% faster (base 7310ns -> cand 5721ns)
  • greatest_i64_nullable_3_args: 21.667% faster (base 14078ns -> cand 11027ns)
  • greatest_i64_no_nulls: 1.398% faster (base 3163ns -> cand 3119ns)

Full criterion output:

greatest_i64_nullable   time:   [5.7071 µs 5.7111 µs 5.7164 µs]
                        change: [−21.848% −21.744% −21.617%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe

greatest_i64_no_nulls   time:   [3.1184 µs 3.1210 µs 3.1238 µs]
                        change: [−1.4889% −1.3979% −1.3102%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

greatest_i64_nullable_3_args
                        time:   [11.012 µs 11.022 µs 11.033 µs]
                        change: [−21.773% −21.667% −21.583%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  12 (12.00%) high mild
  1 (1.00%) high severe

greatest_utf8_nullable  time:   [9.6840 µs 9.6950 µs 9.7074 µs]
                        change: [−9.2000% −9.0226% −8.8658%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

Are there any user-facing changes?

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant