Skip to content

lower min number of resolved forecasts for inclusion in futureeval le…#4942

Merged
colesussmeier merged 6 commits into
mainfrom
fix/lower-min-forecasts-for-futureeval
Jun 25, 2026
Merged

lower min number of resolved forecasts for inclusion in futureeval le…#4942
colesussmeier merged 6 commits into
mainfrom
fix/lower-min-forecasts-for-futureeval

Conversation

@colesussmeier

@colesussmeier colesussmeier commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

…aderboard

Summary by CodeRabbit

  • Bug Fixes

    • Lowered the minimum “resolved forecasts” requirement for leaderboard visibility, so additional entries can appear sooner.
    • Improved which models are included for “upcoming” lists by adjusting the default gating threshold.
  • New Features

    • Added support for recognizing the “MiniMax” model family, including its label and light/dark icons.
    • Enhanced model-family detection for Google-related names to match additional patterns.

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@colesussmeier, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 14 minutes and 58 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5cc19d86-02b9-476d-9e14-d4b955cb37fa

📥 Commits

Reviewing files that changed from the base of the PR and between 2c93580 and e2bad35.

⛔ Files ignored due to path filters (1)
  • front_end/src/app/(main)/aib/assets/ai-models/minimax.svg is excluded by !**/*.svg
📒 Files selected for processing (1)
  • front_end/src/app/(futureeval)/futureeval/components/leaderboard/bot_meta.ts
📝 Walkthrough

Walkthrough

The leaderboard utility lowers its default minimum resolved forecast threshold from 190 to 150, and bot metadata adds MiniMax family recognition with matching icons and updated Google family matching.

Changes

Leaderboard updates

Layer / File(s) Summary
Lower resolved forecast threshold
front_end/src/app/(futureeval)/futureeval/components/leaderboard/utils.ts
The exported MIN_RESOLVED_FORECASTS constant changes from 190 to 150, updating the default value used by shouldDisplayEntry filtering.
Add MiniMax family metadata
front_end/src/app/(futureeval)/futureeval/components/leaderboard/bot_meta.ts
MiniMax icon imports, family detection, and metadata are added, and the Google family matching branch is expanded to include username and model-prefix checks.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • Metaculus/metaculus#4085: Changes the same MIN_RESOLVED_FORECASTS constant in the same leaderboard utility, with a directly related threshold adjustment.

Poem

🐰 I nibbled the gate from high to low,
And MiniMax hopped in with a glowing show.
Google got a wider family map,
The leaderboard gave its forecasts a lap.
Ears twitch, paws pat—what a tidy glow!

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly reflects the main change of lowering the resolved-forecast threshold for FutureEval leaderboard inclusion.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/lower-min-forecasts-for-futureeval

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Cleanup: Preview Environment Removed

The preview environment for this PR has been destroyed.

Resource Status
🌐 Preview App Deleted
🗄️ PostgreSQL Branch Deleted
⚡ Redis Database Deleted
🔧 GitHub Deployments Removed
📦 Docker Image Retained (auto-cleanup via GHCR policies)

Cleanup triggered by PR close at 2026-06-25T15:58:51Z

Comment on lines +37 to +40
u.includes("gemini") ||
u.includes("gemma") ||
m.startsWith("gemini") ||
m.startsWith("gemma")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.includes("gemini") will always catch .startsWith("gemini"), hence redundant

@hlbmtc hlbmtc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a small nit and gtg

@colesussmeier colesussmeier merged commit 41dff09 into main Jun 25, 2026
15 checks passed
@colesussmeier colesussmeier deleted the fix/lower-min-forecasts-for-futureeval branch June 25, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants