Skip to content

lnwire: remove legacy zlib encoding for channel queries - #10980

Open
TechLateef wants to merge 3 commits into
lightningnetwork:masterfrom
TechLateef:remove-zlib-compression
Open

lnwire: remove legacy zlib encoding for channel queries#10980
TechLateef wants to merge 3 commits into
lightningnetwork:masterfrom
TechLateef:remove-zlib-compression

Conversation

@TechLateef

Copy link
Copy Markdown

Change Description

This PR removes support for the deprecated EncodingSortedZlib format (type 1) which was officially dropped from the BOLT 7 specification in lightning/bolts#981.

Changes:

  • Removes all zlib compression/decompression logic from lnwire messages.
  • Marks the EncodingSortedZlib constant as Deprecated to retain protocol documentation.
  • Updates the decode path to explicitly return an error when a peer attempts to send zlib-encoded data, providing a clear error rather than falling through to a generic "unknown encoding" error.
  • Removes zlib-specific chunking logic (maxQueryChanRangeRepliesZlibFactor) from the discovery syncer.
  • Cleans up all testing suites (message_test.go, reply_channel_range_test.go, query_short_chan_ids_test.go) that previously verified zlib scenarios.
  • Leaves channeldb/migration files untouched to ensure historical snapshots remain intact.

Fixes #10694

Steps to Test

  1. Run make unit to ensure all lnwire and discovery tests pass.
  2. Review the code to verify that no active code paths rely on EncodingSortedZlib.

Pull Request Checklist

Testing

  • Your PR passes all CI checks.
  • Tests covering the positive and negative (error paths) are included.
  • Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

📝 Please see our Contribution Guidelines for further guidance.

@github-actions github-actions Bot added the severity-critical Requires expert review - security/consensus critical label Jul 20, 2026
@github-actions

Copy link
Copy Markdown

🔴 PR Severity: CRITICAL

Automated classification | 8 files | 269 lines changed

🔴 Critical (3 files)
  • lnwire/encoding.go - Lightning wire protocol message encoding; marks EncodingSortedZlib as deprecated
  • lnwire/query_short_chan_ids.go - Wire message encode/decode logic; removes zlib chunking/compression paths for query_short_channel_ids/reply_channel_range
  • lnwire/test_message.go - lnwire package helper (not a _test.go file, so classified by package path)
🟠 High (1 file)
  • discovery/syncer.go - Gossip protocol syncer; removes zlib-specific reply chunking factor (maxQueryChanRangeRepliesZlibFactor)
🟢 Low (4 files)
  • docs/release-notes/release-notes-template.md - Docs
  • lnwire/message_test.go - Test-only change
  • lnwire/query_short_chan_ids_test.go - Test-only change
  • lnwire/reply_channel_range_test.go - Test-only change

Analysis

The highest-severity files are in lnwire/*, which defines the Lightning wire protocol messages exchanged with peers — changes here directly affect interoperability and message parsing for gossip queries (query_short_channel_ids, reply_channel_range). This PR removes the deprecated EncodingSortedZlib (type 1) encoding per BOLT 7's removal of zlib support, replacing the decode fallback with an explicit error. Although the change is a net deletion of legacy code, it touches wire-level decode paths that peers rely on, so it warrants expert review to confirm no peer compatibility regressions (e.g. correctly erroring rather than silently mis-parsing legacy zlib-encoded payloads) and that channeldb/migration snapshots (explicitly left untouched per the PR description) remain unaffected.

No severity bump applied: only 5 non-test files and ~189 non-test lines changed (well under the 20-file/500-line thresholds), and only one critical package (lnwire) is touched.


To override, add a severity-override-{critical,high,medium,low} label.

@saubyk saubyk added this to lnd v0.22 Jul 20, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in lnd v0.22 Jul 20, 2026
@saubyk saubyk added this to the v0.22.0 milestone Jul 20, 2026
@saubyk saubyk moved this from Backlog to In progress in lnd v0.22 Jul 20, 2026

@saubyk saubyk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please correct the file for the release note entry

Comment thread docs/release-notes/release-notes-template.md
@TechLateef
TechLateef force-pushed the remove-zlib-compression branch from 0982ef4 to 7611d8f Compare July 21, 2026 05:55
@TechLateef
TechLateef requested a review from saubyk July 21, 2026 18:53
@saubyk

saubyk commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

/gateway review

@lightninglabs-gateway lightninglabs-gateway Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gateway review — 2 findings

🔴 0 Blocker · 🟠 1 Major · 🟡 1 Minor · 🔵 0 Nit

Summary

This PR removes the deprecated EncodingSortedZlib (type 1) gossip-query encoding, matching its removal from BOLT 7. The bulk of the change is clean deletion: the zlib encode block, the maxZlibBufSize/zlibDecodeMtx memory guards, the maxQueryChanRangeRepliesZlibFactor reply multiplier, and the corresponding test fixtures. The processChanRangeReply and encode-side removals are effectively dead-code cleanup — both key off g.cfg.encodingType, which only ever holds EncodingSortedPlain (encodingTypeToChunkSize has no other entry).

The one change that alters behavior on untrusted input is the decode path: decodeShortChanIDs now returns a hard error for type 1 instead of decompressing, and this function backs both QueryShortChanIDs.Decode and ReplyChannelRange.Decode. That behavior change, and its effect on peers that still emit zlib, is the main thing to confirm before merge. A secondary API-hygiene issue remains around the retained-but-unencodable exported constant.

Bot commands
  • /gateway re-review — re-run after pushing changes
  • /gateway dismiss <id> — silence a finding (maintainers)
  • /gateway explain <id> — elaborate on a finding (anyone)

Comment thread lnwire/query_short_chan_ids.go Outdated
Comment thread lnwire/query_short_chan_ids.go
@lightninglabs-gateway

lightninglabs-gateway Bot commented Jul 21, 2026

Copy link
Copy Markdown

🤖 gateway audit metadata for this PR — auto-generated, please don't edit.

@TechLateef
TechLateef force-pushed the remove-zlib-compression branch from 7611d8f to d0a3fa5 Compare July 22, 2026 08:50
@saubyk

saubyk commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

/gateway re-review

@lightninglabs-gateway lightninglabs-gateway Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gateway review — 2 findings

🔴 0 Blocker · 🟠 1 Major · 🟡 1 Minor · 🔵 0 Nit

Summary

Since the last review the author addressed F2: encodeShortChanIDs now has an explicit case EncodingSortedZlib returning ErrZlibNotSupported instead of falling through to the misleading ErrUnknownShortChanIDEncoding, and the EncodingSortedZlib doc in encoding.go now states the constant is "retained for detection on the decode side only — it must never be used for encoding."

F1 remains unresolved. The decode side still hard-rejects a peer-controlled type-1 encoding byte with a wire-decode error. The specialist pass sharpens, but does not clear, the concern: the guard removal itself is safe within lnwire (the maxZlibBufSize limit and zlibDecodeMtx only ever protected the now-deleted inflate loop, and both were unexported so channeldb/migration could never have referenced them), so the live risk is the interop/propagation behavior against not-yet-upgraded peers, which still cannot be confirmed from the loaded context.

One genuinely-new minor finding: the new encode-side error branch has no test.


Status of prior findings

  • F2 addressed: Fixed in lnwire/query_short_chan_ids.goencodeShortChanIDs now has an explicit case EncodingSortedZlib returning ErrZlibNotSupported (no longer falling through to ErrUnknownShortChanIDEncoding), and lnwire/encoding.go tightens the Deprecated doc to state the constant is "retained for detection on the decode side only — it must never be used for encoding."
Bot commands
  • /gateway re-review — re-run after pushing changes
  • /gateway dismiss <id> — silence a finding (maintainers)
  • /gateway explain <id> — elaborate on a finding (anyone)

Comment thread lnwire/query_short_chan_ids.go
Comment thread lnwire/query_short_chan_ids.go Outdated
@saubyk
saubyk requested a review from morehouse July 23, 2026 18:01
Comment thread lnwire/query_short_chan_ids.go Outdated
@TechLateef
TechLateef force-pushed the remove-zlib-compression branch from d0a3fa5 to 7b27191 Compare July 23, 2026 22:04
@saubyk

saubyk commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

/gateway dismiss F1 per #10980 (comment)

@lightninglabs-gateway

Copy link
Copy Markdown

🚫 Dismissed F1 (major) by @saubyk — per #10980 (comment)

Open findings on this PR: 🟡 F3 (minor)

@saubyk

saubyk commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

/gateway re-review

@lightninglabs-gateway lightninglabs-gateway Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gateway review

No findings.


Status of prior findings

  • F2 addressed: Addressed — encodeShortChanIDs carries an explicit case EncodingSortedZlib returning ErrZlibNotSupported (lnwire/query_short_chan_ids.go), and the Deprecated: doc comment in encoding.go now states the constant is decode/detection-only.
  • F3 addressed: Addressed — TestEncodeShortChanIDsZlibRejection (lnwire/query_short_chan_ids_test.go) asserts encodeShortChanIDs(&b, EncodingSortedZlib, nil) returns ErrZlibNotSupported via require.ErrorIs, covering the encode branch that was previously untested.
Bot commands
  • /gateway re-review — re-run after pushing changes
  • /gateway dismiss <id> — silence a finding (maintainers)
  • /gateway explain <id> — elaborate on a finding (anyone)

@saubyk

saubyk commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

/gateway approve

@lightninglabs-gateway lightninglabs-gateway Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — all prior findings addressed.

Findings recap

Findings:

  • F1 (major) — unresolved: decodeShortChanIDs now returns fmt.Errorf("zlib encoding (type %d) is no longer supported", ...) for a type-1 payload instead of decodin…
  • F2 (minor) — addressed: EncodingSortedZlib remains an exported constant, but the EncodingSortedZlib case was fully removed from encodeShortChanIDs, so it now …
  • F3 (minor) — addressed: The new EncodingSortedZlib branch in encodeShortChanIDs returns ErrZlibNotSupported, but only the decode-side rejection is covered by …

Dismissed:


Approved by @saubyk via /gateway approve. Last reviewed at 7b27191. Skill v0.3.0, model claude-opus-4-8.

@Abdulkbk Abdulkbk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass with a few feedback items:

  • Consider splitting into logical commits, e.g., lnwire: remove legacy zlib encoding for channel queries, discovery: drop zlib reply chunking factor, and docs: add release note for zlib removal.
  • Ensure the linter is happy (cause is here).
  • Look at how release note entries are added. For example, the author is missing from the release note, and some unrelated formatting was added.

@TechLateef
TechLateef force-pushed the remove-zlib-compression branch 2 times, most recently from 5907b81 to cc1432a Compare July 27, 2026 10:37
@TechLateef

Copy link
Copy Markdown
Author

First pass with a few feedback items:

  • Consider splitting into logical commits, e.g., lnwire: remove legacy zlib encoding for channel queries, discovery: drop zlib reply chunking factor, and docs: add release note for zlib removal.
  • Ensure the linter is happy (cause is here).
  • Look at how release note entries are added. For example, the author is missing from the release note, and some unrelated formatting was added.

Thanks for the review!
All feedback have been resolved

@ziggie1984 ziggie1984 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through each commit and traced the remaining zlib paths — the removal itself is correct and safe. Recording the verification here since none of it is in the PR description, then some cleanup nits below.

Why this is safe (worth folding into the PR/commit bodies)

  • lnd never sent zlib: all three construction sites are plain (discovery/syncer.go:888 hardcoded, :1210/:1283 via cfg.encodingType, which sync_manager.go:663 pins to EncodingSortedPlain). The new error in encodeShortChanIDs is therefore unreachable in production.
  • lnd never echoes a peer's encoding type back into replies — replyChanRangeQuery uses g.cfg.encodingType, not query.EncodingType, and peer/brontide.go:2826 only logs it. If we did echo it, a zlib query would have made our own Encode fail.
  • Nobody on the network sends type 1: eclair removed sending in ACINQ/eclair#2244 and decoding in ACINQ/eclair#3263; lnd was the last decoder standing (per #10694).
  • The timestamps TLV already rejected everything but plain (lnwire/timestamps.go:92), and lnd doesn't implement query_flags, so no other encoding_type byte is left behind.
  • The empty-list case is a non-issue: eclair's codec always encoded empty lists as uncompressed "for compatibility with other implementations", and lnd never sent zlib, so nothing on the wire produces an empty type-1 body. The removed len(queryBody) == 0 tolerance isn't needed.
  • The chunking-factor removal in 8f01ba8 is a semantic no-op: that switch is on g.cfg.encodingType (ours, always plain), never the reply's encoding, so the zlib arm was already dead code.
  • Security win beyond the decompression bomb: this also deletes zlibDecodeMtx, a package-global mutex that serialized zlib decodes across all peers.
  • No new DoS surface. A peer could already force a fatal decode with encoding type 2, an unsorted SCID list, or a bad body length; type 1 just joins that existing class, which is already rate-limited by nextPeerBackoff (server.go:4069, decaying to MaxBackoff).
  • .golangci.yml:195 disables -SA1019, so the new Deprecated: marker won't trip lint on the internal references to the constant.
  • Verified both code commits build standalone and that ./lnwire/... and ./discovery/... pass at each.

Cleanup items

Dead fuzz targets left behind. lnwire/fuzz_test.go:315 FuzzZlibQueryShortChanIDs and :349 FuzzZlibReplyChannelRange both hardcode the encoding byte to \x01, so ReadMessage now always errors and the harness returns early at fuzz_test.go:45 — every assertion after that is unreachable. I ran one for 3s: 131k execs, and the only coverage it finds is stdlib zlib compression inside the harness itself. Worth deleting both, along with the then-unused compress/zlib import.

Commit messages. Body lines run 130 and 149 chars in 9413a7f and 154 in 8f01ba8; the commit guidelines ask for 72. Both would also read better as prose than as bullet lists. More substantively, neither commit nor the PR body states why the removal is safe, or that receiving type 1 now disconnects the peer — that's a reviewer's first question, and the points above would cover it.

Inline comments follow. None of them are correctness problems.

Comment thread lnwire/query_short_chan_ids.go Outdated
Comment thread lnwire/query_short_chan_ids.go Outdated
Comment thread lnwire/reply_channel_range_test.go
Comment thread lnwire/query_short_chan_ids_test.go
Comment thread docs/release-notes/release-notes-0.22.0.md Outdated
Comment thread discovery/syncer.go
Comment thread lnwire/encoding.go Outdated
Comment thread lnwire/query_short_chan_ids.go
Comment thread discovery/syncer.go
@TechLateef
TechLateef force-pushed the remove-zlib-compression branch 2 times, most recently from 8c243b3 to 3ba646b Compare July 30, 2026 01:53
@TechLateef

Copy link
Copy Markdown
Author

Went through each commit and traced the remaining zlib paths — the removal itself is correct and safe. Recording the verification here since none of it is in the PR description, then some cleanup nits below.

Why this is safe (worth folding into the PR/commit bodies)

  • lnd never sent zlib: all three construction sites are plain (discovery/syncer.go:888 hardcoded, :1210/:1283 via cfg.encodingType, which sync_manager.go:663 pins to EncodingSortedPlain). The new error in encodeShortChanIDs is therefore unreachable in production.
  • lnd never echoes a peer's encoding type back into replies — replyChanRangeQuery uses g.cfg.encodingType, not query.EncodingType, and peer/brontide.go:2826 only logs it. If we did echo it, a zlib query would have made our own Encode fail.
  • Nobody on the network sends type 1: eclair removed sending in ACINQ/eclair#2244 and decoding in ACINQ/eclair#3263; lnd was the last decoder standing (per Remove zlib support #10694).
  • The timestamps TLV already rejected everything but plain (lnwire/timestamps.go:92), and lnd doesn't implement query_flags, so no other encoding_type byte is left behind.
  • The empty-list case is a non-issue: eclair's codec always encoded empty lists as uncompressed "for compatibility with other implementations", and lnd never sent zlib, so nothing on the wire produces an empty type-1 body. The removed len(queryBody) == 0 tolerance isn't needed.
  • The chunking-factor removal in 8f01ba8 is a semantic no-op: that switch is on g.cfg.encodingType (ours, always plain), never the reply's encoding, so the zlib arm was already dead code.
  • Security win beyond the decompression bomb: this also deletes zlibDecodeMtx, a package-global mutex that serialized zlib decodes across all peers.
  • No new DoS surface. A peer could already force a fatal decode with encoding type 2, an unsorted SCID list, or a bad body length; type 1 just joins that existing class, which is already rate-limited by nextPeerBackoff (server.go:4069, decaying to MaxBackoff).
  • .golangci.yml:195 disables -SA1019, so the new Deprecated: marker won't trip lint on the internal references to the constant.
  • Verified both code commits build standalone and that ./lnwire/... and ./discovery/... pass at each.

Cleanup items

Dead fuzz targets left behind. lnwire/fuzz_test.go:315 FuzzZlibQueryShortChanIDs and :349 FuzzZlibReplyChannelRange both hardcode the encoding byte to \x01, so ReadMessage now always errors and the harness returns early at fuzz_test.go:45 — every assertion after that is unreachable. I ran one for 3s: 131k execs, and the only coverage it finds is stdlib zlib compression inside the harness itself. Worth deleting both, along with the then-unused compress/zlib import.

Commit messages. Body lines run 130 and 149 chars in 9413a7f and 154 in 8f01ba8; the commit guidelines ask for 72. Both would also read better as prose than as bullet lists. More substantively, neither commit nor the PR body states why the removal is safe, or that receiving type 1 now disconnects the peer — that's a reviewer's first question, and the points above would cover it.

Inline comments follow. None of them are correctness problems.

Thanks for the review! I've addressed all the feedback in the latest push: Updated the release notes with the PR link and the operator context. Cleaned up the two stale comments regarding the encoding types.

@TechLateef
TechLateef force-pushed the remove-zlib-compression branch 2 times, most recently from 68cb210 to d5f55de Compare July 30, 2026 21:22
@litbot-9000

Copy link
Copy Markdown
Collaborator

@saubyk: review reminder
@Abdulkbk: review reminder
@ziggie1984: review reminder

@TechLateef
TechLateef force-pushed the remove-zlib-compression branch from d5f55de to 667cab5 Compare August 6, 2026 07:58
This commit removes support for the deprecated EncodingSortedZlib
format (type 1) which was dropped from the BOLT 7 specification.
All zlib compression and decompression logic has been removed from
lnwire messages. The EncodingSortedZlib constant is now marked as
Deprecated to retain protocol documentation. The decode path now
explicitly returns an error when a peer attempts to send zlib-encoded
data, rather than a generic unknown encoding error. All tests that
previously verified zlib scenarios have been cleaned up or deleted.
This removal is safe because lnd has never sent zlib-encoded data,
and nobody on the network sends type 1.
Additionally, dropping this encoding removes the package-global
zlibDecodeMtx, improving performance and security without introducing
new DoS vectors, as receiving type 1 now simply triggers an instant
disconnect.
This commit removes zlib-specific chunking logic from the discovery
syncer. Since LND no longer supports zlib compression for gossip
queries, the corresponding chunking factors and parsing cases in the
syncer are no longer needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway-active gateway-approved severity-critical Requires expert review - security/consensus critical

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

Remove zlib support

5 participants