Repository navigation
fix(sender): add connect and read timeouts to SyncSender - #387
buongarzoni wants to merge 5 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 423bf81149
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| public ConfigBuilder connectTimeoutMillis(int connectTimeoutMillis) { | ||
| this.connectTimeoutMillis = requireNonNegative(connectTimeoutMillis, "connectTimeoutMillis"); |
There was a problem hiding this comment.
Rebuild inherited senders when changing timeouts
When either timeout setter is used through Rollbar.configure, which starts with ConfigBuilder.withConfig, the constructor has already copied the current sender into this.sender. Consequently, build() skips its sender == null branch, so the returned Config reports the new timeout while the reused SyncSender continues using the old value. This makes runtime timeout reconfiguration ineffective; the builder needs to rebuild or otherwise update an inherited default sender when these values change.
Useful? React with 👍 / 👎.
| try (ServerSocket silentServer = new ServerSocket(0, 1, InetAddress.getLoopbackAddress())) { | ||
| String endpoint = "http://127.0.0.1:" + silentServer.getLocalPort() + "/api/1/item/"; |
There was a problem hiding this comment.
Use the bound loopback address in the endpoint
On JVMs started with IPv6 preferred, InetAddress.getLoopbackAddress() resolves to ::1, but the client still connects to 127.0.0.1. The connection is then refused rather than timing out, so the test fails with ConnectException instead of the expected SocketTimeoutException. Construct the endpoint from the address actually used to bind the server.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
AI Agent Review (anthropic, CLI default model; requested openai/openai-astra)
Scrutiny: standard (failure path)
Before this change, an app that used the default Rollbar sender could stop reporting errors entirely if one network connection was accepted but never answered, for example by a stuck proxy or load balancer. All reports go out one at a time from a single background worker. That worker would wait on the stuck connection forever. Nothing was logged, reports kept piling up in memory, and a shutdown that waits for pending reports could hang.
With this change, the default sender gives up after 30 seconds of trying to connect and after 30 seconds of waiting for a reply. The report that timed out fails like any other network error: error listeners are told about it and the sender moves on to the next one. Apps can change both limits with two new settings, connectTimeoutMillis and readTimeoutMillis, on the config builder or the sender builder. 0 restores the old wait-forever behaviour, and a negative value is rejected immediately. The settings have no effect when an app supplies its own sender. Android needs no changes and picks up the 30-second defaults. Apps that implement the configuration interface themselves keep compiling and also get the defaults. The reactive-streams module has its own configuration and HTTP client, so it is not affected.
I found one blocking problem. The new code is fine, but the build will fail: one new comment line in the sender is 106 characters long. The project's lint rules allow at most 100 characters and treat any violation as an error. CI runs those rules as part of ./gradlew clean build. The PR description says lint passes locally, but the lint configuration in this checkout does not support that. Wrapping the comment fixes it.
I read the tests; I did not run them. They check that both default limits are applied, that custom values are applied, that custom values survive copying a configuration, and that negative values are rejected. One test uses a real local socket that accepts the connection and never answers. It checks that the send fails with a timeout error instead of hanging, which covers the scenario the PR is about.
Non-blocking things worth knowing (none of these code paths are in this PR):
- Changing the limits later has no effect. If an app reconfigures Rollbar at runtime with new timeouts, the existing sender is reused with its original limits, but the configuration reports the new values. Proxy and compression settings already behave this way.
- Shutdown is no longer endless, but can still be long. A shutdown that waits for pending reports still sends them one at a time. Against a stuck endpoint, each queued report can take up to 30 seconds (up to 60 if connecting also stalls), so a large backlog can make shutdown take a long time.
- On Android, a timed-out report is dropped, not retried. Android's "pause while the network is down" logic only recognises socket and unknown-host errors as network problems, and a timeout is neither.
- A stalled upload is not covered. The read limit applies while waiting for a reply. A send that stalls partway through uploading is not limited by these settings. With compressed reports this is unlikely to matter.
| // By default, this will use the HTTPS Rollbar API URL defined in DEFAULT_API_ENDPOINT. | ||
| HttpURLConnection connection = (HttpURLConnection) url.openConnection(this.proxy); | ||
|
|
||
| // Without these calls HttpURLConnection never times out, which could block the sender thread forever. |
There was a problem hiding this comment.
Medium: This comment line is 106 characters long, and the project's lint rules allow at most 100. Lint violations are configured as errors, the line contains no URL that would exempt it, and lint runs on this module's main source code. CI runs ./gradlew clean build, which includes the lint check, so the build job will fail. In this module's main code, the only other lines over 100 characters are exempt URL comments. That contradicts the PR checklist entry saying lint passes locally. Wrapping the comment onto two lines fixes it.
| // Without these calls HttpURLConnection never times out, which could block the sender thread forever. | |
| // Without these calls HttpURLConnection never times out, which could block the sender | |
| // thread forever. |
There was a problem hiding this comment.
AI Agent Review (anthropic, CLI default model; requested openai/openai-astra)
Scrutiny: standard (failure path)
What this changes for users
The Java notifier sends every error report through one background sender, and until now that sender had no time limit on its network calls. Suppose the Rollbar endpoint, or a proxy or load balancer in front of it, accepts a connection and then never answers. The sender then waits forever. No more errors get reported, unsent reports pile up in memory, and a shutdown that flushes pending reports first can hang. That includes stopping the Logback and Log4j2 appenders.
With this PR, the default sender stops trying to connect after 30 seconds. Once connected, it gives up after 30 seconds without a response. A timed-out report then fails like any other network error: it is logged, error listeners are notified, and the sender moves on to the next report. Apps can change both limits on the configuration builder or on the sender builder (connectTimeoutMillis / readTimeoutMillis). Negative values are rejected. 0 brings back the old wait-forever behaviour.
| Situation | Before | After |
|---|---|---|
| Endpoint accepts the connection but never answers | Reporting stops until the process restarts | That report fails after about 30 s; the next one is tried |
| Endpoint unreachable during connect | Could wait indefinitely | Fails after about 30 s |
| Healthy endpoint | Unchanged | Unchanged |
| App supplies its own sender | Unchanged | Unchanged (the new settings are ignored, as the docs say) |
Existing custom implementations of the configuration interface still compile, because the two new settings come with built-in defaults. The compression option already works this way (rollbar-java/src/main/java/com/rollbar/notifier/config/CommonConfig.java:225).
Conclusion
I found no blocking problem. Here is what I checked in the code:
- The configured values are applied to every connection the sender opens (
SyncSender.java:81-88). - The default sender built from the configuration builder receives both values (
ConfigBuilder.java:648-660). - Copying a configuration keeps the values (
ConfigBuilder.java:156-157). - The Logback and Log4j2 appenders create their sender through the same builder (
rollbar-logback/.../RollbarAppender.java:103,rollbar-log4j2/.../RollbarAppender.java:114), so they get the 30 s defaults automatically. - Android builds its own sender with the sender builder's defaults (
rollbar-android/.../Rollbar.java:463-465), so it gets them too. - A timeout error is caught and passed to listeners like any other failure (
AbstractSender.java:35-46), and the background loop keeps going with the next report (BufferedSender.java:274-284).
I read the tests but did not run them. The mock-based tests check that the default and the custom values are applied to the connection and that negative values are rejected. The real-socket test points the sender at a local server that accepts the connection and never replies, and checks that the send fails with a timeout instead of hanging. The one gap is that no test checks that values set on the configuration builder reach the sender it creates (finding below).
Worth knowing (not blocking)
-
Shutdown is not capped overall. A shutdown that flushes pending reports still sends them one at a time (
BufferedSender.java:143-147). With many queued reports and an unresponsive endpoint, it can take roughly 30–60 seconds per queued report. That is far better than waiting forever, but it is not a fixed limit, so the description's point about shutdown hangs is only partly solved. -
Changing timeouts on a running notifier has no live effect. Reconfiguring through
Rollbar.configurereuses the existing sender (Rollbar.java:96-108,ConfigBuilder.java:146). The configuration will report the new timeout, but the live sender keeps the old one. The Javadoc says so, and compression and proxy settings already behave the same way. -
Two Android effects.
- The configuration builder that Android hands to apps already contains Android's own sender (
rollbar-android/.../Rollbar.java:482-488). The two new builder settings therefore do nothing there; Android always uses the 30 s defaults. - Android's network-aware retry logic only treats socket and unknown-host errors as network problems (
ConnectionAwareSenderFailureStrategy.java:151-161). A timed-out report is therefore dropped, not queued for retry (:103-106).
That Android code is outside this PR. I'm mentioning it so the team can decide whether that behaviour is what they want.
- The configuration builder that Android hands to apps already contains Android's own sender (
| @Test | ||
| public void shouldBuildWithTimeoutsAndCopyThem() { | ||
| Config config = ConfigBuilder.withAccessToken(ACCESS_TOKEN) | ||
| .connectTimeoutMillis(1234) | ||
| .readTimeoutMillis(5678) | ||
| .build(); | ||
|
|
||
| Config copy = withConfig(config).build(); | ||
|
|
||
| assertThat(config.connectTimeoutMillis(), is(1234)); | ||
| assertThat(config.readTimeoutMillis(), is(5678)); | ||
| assertThat(copy.connectTimeoutMillis(), is(1234)); | ||
| assertThat(copy.readTimeoutMillis(), is(5678)); | ||
| } |
There was a problem hiding this comment.
Low: Most users will set the new timeouts on the configuration builder, but the new test only checks that the values are stored on the configuration and survive a copy. Nothing checks that the sender the builder creates actually uses them. If the two lines that pass the values to the sender (ConfigBuilder.java:654-655) were dropped or broken, every test would still pass. A user who sets, for example, a 5-second read timeout would silently get 30 seconds, while the configuration still reports 5 seconds. The values can't be read back from the sender directly because they are private. However, the default sender wraps the real one and exposes it through BufferedSender.sender(), so the existing silent-server test can be reused through the builder.
AI Agent Review (anthropic, CLI default model; requested openai/openai-astra)Scrutiny: standard (failure path) If Rollbar's API, or a proxy or load balancer in front of it, accepts a connection and then never answers, the Java SDK used to wait for that reply forever. One background worker sends all error reports, one at a time. So a single stuck request stopped error reporting until the process restarted. Nothing was logged, and unsent reports kept piling up in memory. Shutting down while waiting for pending reports could also hang; the Logback and Log4j2 appenders do this when they stop. With this change, the SDK stops trying to connect after 30 seconds. Once connected, it stops waiting for a reply after another 30 seconds. A report that times out fails like any other network error: the failure is logged, anyone listening for send failures is told the cause was a timeout, and the worker moves on to the next report. This happens automatically for anyone using the SDK's built-in sender, including the Android client. Nothing needs to be configured.
Both limits can be changed through two new settings, Conclusion: I found no blocking problem. I traced the configured values from the configuration builder into the sender, and from there onto every outgoing connection. They are applied before the connection is used, so they cover every report, not just the first. Copying an existing configuration keeps the values. A timeout ends up in the sender's normal failure handling, which logs it, notifies listeners and lets the worker carry on. The project already targets Java 8 and already adds settings to the configuration interface this way, so compatibility does not change. Android builds its sender with the builder's defaults, so it gets the 30-second limits too. I read the new tests but did not run them. They check that:
Some things worth knowing, none of them caused by this PR:
Code locations checked
|
Description of the change
SyncSendernever set a connect or read timeout, andHttpURLConnectionwaits forever by default.All payloads go through a single sender thread, so one connection that is accepted but never
answered (a stuck proxy or load balancer, for example) stopped all error reporting until the process
restarted, without logging anything. Meanwhile the in-memory queue kept growing, and
close(true), including the Logback and Log4j2 appenders'stop(), could hang shutdown.This adds both timeouts, with a default of 30 s each:
SyncSender.Builder.connectTimeoutMillis(int)/readTimeoutMillis(int)ConfigBuilder.connectTimeoutMillis(int)/readTimeoutMillis(int), passed to the default senderConfig.connectTimeoutMillis()/readTimeoutMillis()asdefaultmethods, so existingConfigimplementations still compileNegative values throw
IllegalArgumentException.0keeps the old "wait forever" behaviour.Android builds its
SyncSenderwith the builder defaults, so it gets the timeouts with no changes.The timeouts are ignored when a custom
senderis configured.When a timeout expires, the send fails like any other network error: listeners receive a
SenderExceptioncaused bySocketTimeoutException, and the sender moves on to the next payload.Type of change
Checklists
Development
checkstyleMain)one against a real socket that accepts the connection and never responds
:rollbar-java:test)