Skip to content

replace go-dsqueue#1177

Draft
gammazero wants to merge 11 commits into
mainfrom
cascadeq-for-provide
Draft

replace go-dsqueue#1177
gammazero wants to merge 11 commits into
mainfrom
cascadeq-for-provide

Conversation

@gammazero

@gammazero gammazero commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Replace the provide queue use of go-dsqueue with cascadeq, in boxo and go-libp2p-kad-dht

The intended benefits are:

  • Overall efficiency (faster, less memory overhead)
  • Reliable performance regardless of datastore in use

The efficiency of go-dsqueue depends on the ability of the datastore in use to support ordered query responses without reading all queued items into memory to provide that ordering of queued item(s). With go-dsqueue is it also necessary to store additional ordering information with each queued item, taking up additional space.

Using cascadeq eliminates the need for a datastore implementation to efficiently return ordered query results as these are always stored in queue order when persisted to disk. Storage is also more efficient as there is no need to store additional ordering information with each item.

A potential downside to this PR is the case where is user wants all data written to S3, so an S3-backed datastore is used. In this case, cascadeq would not allow the user to write all data to S3, without some additional work.

Kubo PR: ipfs/kubo#11364

go-libp2p-kad-dht PR: libp2p/go-libp2p-kad-dht#1266

@socket-security

socket-security Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgithub.com/​gammazero/​cascadeq@​v0.2.099100100100100
Updatedgithub.com/​libp2p/​go-libp2p-kad-dht@​v0.40.0 ⏵ v0.41.1-0.20260703233757-7e03fc4a9736100 +1100100100100

View full report

@gammazero gammazero changed the title Cascadeq for provide replace go-dsqueue Jun 17, 2026
@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 47.05882% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.80%. Comparing base (ddf8b81) to head (059bd3e).

Files with missing lines Patch % Lines
provider/reprovider.go 53.33% 4 Missing and 3 partials ⚠️
provider/noop.go 0.00% 2 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1177      +/-   ##
==========================================
- Coverage   63.83%   63.80%   -0.04%     
==========================================
  Files         269      269              
  Lines       27047    27054       +7     
==========================================
- Hits        17266    17262       -4     
- Misses       8071     8080       +9     
- Partials     1710     1712       +2     
Files with missing lines Coverage Δ
provider/provider.go 90.16% <ø> (ø)
provider/noop.go 0.00% <0.00%> (ø)
provider/reprovider.go 58.27% <53.33%> (+0.67%) ⬆️

... and 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gammazero gammazero force-pushed the cascadeq-for-provide branch from 61157c9 to bbc1704 Compare July 2, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant