feat: rotate fraction based on in-memory size by dkharms · Pull Request #422 · ozontech/seq-db

dkharms · 2026-05-07T12:39:49Z

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)

I have read and followed all requirements in CONTRIBUTING.md;
I used LLM/AI assistance to make this pull request;

If you have used LLM/AI assistance please provide model name and full prompt:

Model: {{model-name}}
Prompt: {{prompt}}

dkharms · 2026-05-07T12:40:04Z

@seqbenchbot up main bulk

seqbenchbot · 2026-05-07T12:40:07Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - f49fa4e7.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-07T12:54:06Z

@seqbenchbot down f49fa4e7

seqbenchbot · 2026-05-07T12:54:09Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator f49fa4e7 was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`62.06`	`63.06`	`+1.60%`	`20.72`	`18.73`	`-9.64%`	`57.00`	`59.00`	`+3.51%`	`103.00`	`97.00`	`-5.83%`	`136.00`	`124.00`	`-8.82%`	`15653.00`	`15580.00`	`-0.47%`

Have a great time!

dkharms · 2026-05-07T12:55:59Z

@seqbenchbot up main mixed

seqbenchbot · 2026-05-07T12:56:01Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - c8788b4c.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-07T13:21:41Z

@seqbenchbot down c8788b4c

seqbenchbot · 2026-05-07T13:21:44Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator c8788b4c was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`67.73`	`66.01`	`-2.54%`	`28.40`	`25.12`	`-11.57%`	`60.00`	`60.00`	`0.00%`	`119.00`	`108.00`	`-9.24%`	`180.50`	`168.50`	`-6.65%`	`33031.00`	`32935.00`	`-0.29%`
`service:payment-backend-eu AND k8s_namespace:prod AND level:[0 to 3] AND ( message:'failed' OR message:'timeout' )`	warm	`59.63`	`57.16`	`-4.14%`	`18.22`	`23.68`	`+29.96%`	`56.00`	`47.00`	`-16.07%`	`89.50`	`103.00`	`+15.08%`	`131.00`	`135.00`	`+3.05%`	`6461.00`	`6435.00`	`-0.40%`

Have a great time!

dkharms · 2026-05-07T13:22:15Z

@seqbenchbot up main mixed

seqbenchbot · 2026-05-07T13:22:18Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - ab552b62.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-07T13:26:57Z

@seqbenchbot down ab552b62

seqbenchbot · 2026-05-07T13:27:00Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator ab552b62 was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Have a great time!

dkharms · 2026-05-07T13:27:32Z

@seqbenchbot up main search-keyword-exact-match-warm

seqbenchbot · 2026-05-07T13:27:35Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 9529f46e.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-07T13:37:12Z

@seqbenchbot down 9529f46e

seqbenchbot · 2026-05-07T13:37:15Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator 9529f46e was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`65.17`	`67.72`	`+3.91%`	`22.04`	`21.05`	`-4.48%`	`60.00`	`64.00`	`+6.67%`	`107.00`	`105.00`	`-1.87%`	`147.50`	`146.00`	`-1.02%`	`2450.00`	`2450.00`	`0.00%`
`service:payment-backend-eu AND k8s_namespace:prod`	warm	`94.07`	`101.87`	`+8.29%`	`46.19`	`58.73`	`+27.14%`	`93.00`	`89.00`	`-4.30%`	`169.00`	`204.00`	`+20.71%`	`197.50`	`225.00`	`+13.92%`	`3819.00`	`3811.00`	`-0.21%`

Have a great time!

…ence counting

In this commit I've fixed the rebase issues when rebasing onto 0-fm-compaction-api

In this commit I've unexported methods for several entities: - `lidAccumulator` -- it is accessible outside of `indexwriter`; - `indexBlock` -- the same reason;

dkharms · 2026-05-27T15:10:43Z

@seqbenchbot up main bulk

seqbenchbot · 2026-05-27T15:10:46Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 2346f269.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-27T15:27:49Z

@seqbenchbot down 2346f269

seqbenchbot · 2026-05-27T15:27:53Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator 2346f269 was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`65.76`	`66.20`	`+0.67%`	`22.69`	`20.65`	`-8.97%`	`60.00`	`62.00`	`+3.33%`	`112.00`	`106.00`	`-5.36%`	`152.50`	`140.50`	`-7.87%`	`20407.00`	`20484.00`	`+0.38%`

Have a great time!

dkharms · 2026-05-27T15:28:09Z

@seqbenchbot up main mixed

seqbenchbot · 2026-05-27T15:28:12Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 00e3e423.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-27T15:43:48Z

@seqbenchbot down 00e3e423

seqbenchbot · 2026-05-27T15:43:59Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator 00e3e423 was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`65.99`	`66.31`	`+0.48%`	`24.14`	`19.64`	`-18.66%`	`60.00`	`62.00`	`+3.33%`	`114.00`	`103.00`	`-9.65%`	`164.00`	`138.00`	`-15.85%`	`18826.00`	`18914.00`	`+0.47%`
`service:payment-backend-eu AND k8s_namespace:prod AND level:[0 to 3] AND ( message:'failed' OR message:'timeout' )`	warm	`66.81`	`64.50`	`-3.46%`	`24.49`	`27.32`	`+11.57%`	`61.00`	`53.00`	`-13.11%`	`107.00`	`119.00`	`+11.21%`	`155.00`	`156.00`	`+0.65%`	`2907.00`	`2929.00`	`+0.76%`

Have a great time!

dkharms · 2026-05-27T15:46:48Z

@seqbenchbot up main mixed

seqbenchbot · 2026-05-27T15:46:51Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 384a5e44.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-05-27T15:59:53Z

@seqbenchbot down 384a5e44

seqbenchbot · 2026-05-27T15:59:56Z

Nice, @dkharms <(-^,^-)=b!

The benchmark with identificator 384a5e44 was finished.
I've prepared a summary for you. Click on Show summary button to see it:

Show summary

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`bulk`	warm	`78.19`	`78.31`	`+0.16%`	`31.98`	`27.13`	`-15.16%`	`69.00`	`72.00`	`+4.35%`	`145.00`	`131.00`	`-9.66%`	`204.00`	`176.00`	`-13.73%`	`14865.00`	`14903.00`	`+0.26%`
`service:payment-backend-eu AND k8s_namespace:prod AND level:[0 to 3] AND ( message:'failed' OR message:'timeout' )`	warm	`81.52`	`78.38`	`-3.85%`	`27.83`	`34.94`	`+25.55%`	`74.00`	`67.00`	`-9.46%`	`141.00`	`144.00`	`+2.13%`	`187.00`	`203.00`	`+8.56%`	`1917.00`	`1926.00`	`+0.47%`

Have a great time!

eguguchkin added 2 commits May 20, 2026 12:28

feat(fracmanager): implement fraction snapshots with wait group refer…

c092251

…ence counting

review fixes

e447d7f

dkharms force-pushed the 336-call-info-once branch from aa9e94e to 77a537e Compare May 21, 2026 12:17

dkharms added 11 commits May 22, 2026 14:26

feat: new api for compaction

69c1e69

chore: remove garbage code

dcdeec2

chore: remove evict local fix

96ba81f

refactor: introduce blockbuilder and indexwriter package

0978ee4

refactor: filename similar to package name

3abf863

refactor: remove BlockBuilder type

6bf88e3

refactor: move unexported functions

b9f77b8

chore: fix rebase conflicts

bb339f6

refactor: merge indexwriter and blockbuilder

7484ac0

chore: fix rebase conflicts

e0dfb13

In this commit I've fixed the rebase issues when rebasing onto 0-fm-compaction-api

chore: do not export some methods

8bc6dff

In this commit I've unexported methods for several entities: - `lidAccumulator` -- it is accessible outside of `indexwriter`; - `indexBlock` -- the same reason;

dkharms added 13 commits May 25, 2026 12:47

feat: calculate information correctly

eac4423

feat: use linear scan for k-way merge

465bc20

fix: calculate offsets and info once

9944b81

feat: build distribution for compacted fraction

21a854b

refactor: consistent naming

d3e1c9e

feat: implement stcs

1cb6543

feat: add FractionName method for FracManager

3118a7d

feat: first iteration on planner

e0628dd

feat: first iteration on Executor

688cb8c

feat: add compaction executor startup

14001ce

refactor: use local fraction interface

b7ae92a

refactor: move to frac_test package

ec7be15

feat: rotate fraction based on in-memory size

65b228e

dkharms force-pushed the 0-mem-size branch from 55a05b6 to 65b228e Compare May 27, 2026 15:08

dkharms changed the base branch from 336-call-info-once to 336-compaction May 27, 2026 15:08

chore: frac-size=1MiB

d33fd80

chore: group by creation time

fdde91d

dkharms force-pushed the 336-compaction branch from ace0b94 to 126b15f Compare June 11, 2026 15:48

Conversation

dkharms commented May 7, 2026

Description

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 7, 2026

Uh oh!

seqbenchbot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented May 27, 2026

Uh oh!

seqbenchbot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 7, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading

seqbenchbot commented May 27, 2026 •

edited

Loading