fix WeightedRandomizedLoadBalancer skipping the last server - #3426
Open
Nas01010101 wants to merge 1 commit into
Open
fix WeightedRandomizedLoadBalancer skipping the last server#3426Nas01010101 wants to merge 1 commit into
Nas01010101 wants to merge 1 commit into
Conversation
Nas01010101
force-pushed
the
fix/wr-lb-off-by-one
branch
from
August 3, 2026 01:24
d792023 to
e0cf753
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes an off-by-one error in brpc::policy::WeightedRandomizedLoadBalancer::SelectServer() that prevented the last server in the weighted prefix-sum list from ever being selected (and slightly over-selected the first), and adds a targeted unit test to catch this regression.
Changes:
- Fix selection logic by searching
lower_bound(random_weight + 1)against inclusive prefix sums. - Add
weighted_randomized_equal_weightunit test with equal weights and tighter distribution bounds to reliably detect the bug.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/brpc/policy/weighted_randomized_load_balancer.cpp |
Adjusts the weighted random selection to correctly map random draws onto prefix-sum ranges. |
test/brpc_load_balancer_unittest.cpp |
Adds a new equal-weight distribution test to expose the prior off-by-one behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1071
to
+1072
| // 0.9x ~ 1.1x of the expected rate, which is more than 20 standard | ||
| // deviations away from the mean at this number of runs. |
SelectServer() draws random_weight from fast_rand_less_than(weight_sum), i.e. from [0, weight_sum - 1], and then lower_bound()s it against Server::current_weight_sum, which Add() fills with an inclusive prefix sum. lower_bound() returns the first server whose prefix sum is >= random_weight, but a server owns the half-open range [prefix(i-1), prefix(i)), so the predicate has to be > random_weight. Because of that the first server in the list also serves random_weight == prefix(0) and the last server never serves anything at all, since random_weight can never reach weight_sum. With four servers of equal weight the measured distribution is 49.8/25.1/25.1/0.0 percent instead of 25 percent each. Search for random_weight + 1 so that lower_bound() lands on the first prefix sum strictly greater than random_weight. The existing weighted_randomized test does not catch this: its servers have weights 3/2/5/10 and it only asserts that each rate is within 0.5x~2x of the expected one. The weight-10 server measures 0.448 before this change and 0.494 after it, both inside that band. Add weighted_randomized_equal_weight, which uses equal weights so that a single misplaced slot is visible, and check the rates within 0.9x~1.1x. Signed-off-by: Anas <156536069+Nas01010101@users.noreply.github.com>
Nas01010101
force-pushed
the
fix/wr-lb-off-by-one
branch
from
August 3, 2026 03:50
e0cf753 to
9637f6b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
WeightedRandomizedLoadBalancer::SelectServer()never selects the server that was added last, and over-selects the first one.Add()fillsServer::current_weight_sumwith an inclusive prefix sum.SelectServer()drawsrandom_weightfrombutil::fast_rand_less_than(weight_sum), i.e. from[0, weight_sum - 1], and thenlower_bound()s that value against those prefix sums.lower_bound()returns the first server whose prefix sum is>= random_weight, but a server owns the half-open range[prefix(i-1), prefix(i)), so the predicate has to be> random_weight.Two consequences: the first server also serves
random_weight == prefix(0), so it gets one slot too many; and the last server is never selected at all, becauserandom_weightcan never reachweight_sum.With four servers of equal weight the measured distribution is 49.8 / 25.1 / 25.1 / 0.0 percent instead of 25 percent each. The last server only receives traffic through the fallback loop, when every server picked at random happens to be unavailable.
What is changed and the side effects?
Changed: search for
random_weight + 1, solower_bound()lands on the first prefix sum strictly greater thanrandom_weight.random_weight + 1is at mostweight_sum, which is exactly the last prefix sum, so the iterator is still always valid.Side effects:
The existing
weighted_randomizedtest does not catch this. Its servers have weights 3/2/5/10 and it only asserts that every rate is within 0.5x~2x of the expected one. The weight-10 server measures 0.453 before this change and 0.5055 after it, and both are inside that band. The off-by-one predicts that number exactly: the server should own 10 of 20 slots but owns 9, i.e. 0.45.This PR adds
weighted_randomized_equal_weight, which uses equal weights so a single misplaced slot is visible, and checks the rates within 0.9x1.1x. On master the last server is selected 0 times out of 40000 and the test fails; with this change all four land within 0.24760.25435.Check List:
brpc_load_balancer_unittestpasses in full:[ PASSED ] 17 tests.--gtest_repeat=40without a failure.Tested on macOS/arm64 with clang. I was not able to build on Linux/gcc locally; CI will cover that.