Skip to content

fix(arrow flight): skip amp keep-alive frames instead of decoding them - #31

Draft
benface wants to merge 1 commit into
mainfrom
benface/skip-keep-alive-frames
Draft

benface wants to merge 1 commit into
mainfrom
benface/skip-keep-alive-frames

Conversation

@benface

@benface benface commented Nov 2, 2026

Copy link
Copy Markdown

On long-lived streams, Amp sends a keep-alive every keep_alive_interval (30 seconds by default): an empty FlightData with no data_header. streamQuery read every frame's header as a flatbuffer, so the first keep-alive killed the stream. It now skips frames with an empty data_header, the same check Amp's Rust client (amp-client-flight) makes in decode_flight_data.

  • streamQuery with stream: true, and so ProtocolStream, TransactionalStream and CdcStream, died 30 seconds in with a defect: RangeError: Offset is outside the bounds of the DataView. Busy streams too, since Amp sends the keep-alives on a fixed interval.

I reproduced it with ampd v0.0.42 (ampd solo) over an Anvil chain with 2-second blocks: streamQuery('SELECT block_num FROM "_/anvil".blocks', { stream: true }) got a frame every 2 seconds, then died at 30 seconds with that RangeError. With the fix, it streamed past both keep-alives (30 and 60 seconds) with no gaps in the block numbers.

You might prefer checking dataBody too (dataHeader.length === 0 && dataBody.length === 0). I only check the header, like the Rust client.

Checks:

  • pnpm lint, pnpm check:tsgo, pnpm test (234 passed, 1 new) and pnpm circular pass.
  • The new test puts keep-alive frames before the schema, between the schema and a record batch, and after it. Without the fix it fails with the same RangeError.
  • Against ampd v0.0.42, the stream that died at 30 seconds was still going at 75 seconds.

See edgeandnode/amp#2661 for the keep-alive frames on Amp's side.

🤖 Generated with Claude Code

Amp sends FlightData frames with an empty data header as keep-alives on
long-lived streams. streamQuery read every frame's header as a flatbuffer,
so the first keep-alive killed the stream with a RangeError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant