alexey-milovidov · Oct 12, 2026 · Oct 9, 2026 · Oct 9, 2026 · Oct 12, 2026 · Oct 12, 2026
diff --git a/ravel/README.md b/ravel/README.md
@@ -68,18 +68,37 @@ container, less a fixed reserve), with a fixed segment cap and engine deadline.
 Every resolved value and its source is logged on a `performance default
 resolved` line in `server.log`; a published result should record those lines,
 because they are the configuration the numbers were measured at. On
-c6a.4xlarge, with the store on loopback, the fetch cache resolves to 12.3 GB and
-the per-query SQL pool to 16.5 GB. That is the tenant's whole SQL share since
-0.21.0; it was 8.2 GB before. `./start` also raises the process's open-file
+c6a.4xlarge, with the store on loopback, v0.23.0 resolved the fetch cache to
+11.4 GB, the per-query SQL pool to 14.2 GB and fetch concurrency to 32.
+`./start` also raises the process's open-file
 soft limit to the hard limit, since the server has exited with `EMFILE` at
 startup under the common default of 1024.
 
-`./load` declares the typed attribute columns and then loads the Parquet file.
-Object size is set at ingest by `--batch-rows`: one batch becomes one object per
-involved shard, so 150,000 rows over 4 shards gives ~4 MB objects and roughly
-2,600 objects for the 100M-row dataset. **There is no post-load step**: no
-compaction, no catalog fold, no VACUUM equivalent, so the layout the queries
-run against is the layout ingest produced.
+`./load` declares the typed attribute columns and then loads the Parquet file
+in 100,000-row batches. Since 0.23.0 a shard merges several batches into one
+stored object until its buffer's estimated uncompressed content reaches
+`--target-bytes`; at 1,850,000,000 that gives objects of about 25 MB on this
+dataset (about 410 objects in all; a stored object is about 1/74 of the
+estimate here). Before 0.23.0 each batch was its own object, about 4 MB. Large
+objects are what make a cold scan cheap. The loader holds its batches under
+`--load-memory-bytes`, so the load fits the machine. `./load` picks the
+pipeline depth, read cursors and that budget from `MemTotal`:
+
+| MemTotal | `--pipeline-depth` | `--read-cursors` | `--load-memory-bytes` |
+|---|---|---|---|
+| 15 GB and up | 32 | 16 | 6,500,000,000 |
+| 7 to 15 GB | 24 | 16 | 3,500,000,000 |
+| 3 to 7 GB | 16 | 2 | 1,200,000,000 |
+| under 3 GB | 8 | 1 | derived by the loader |
+
+On a 4 GB machine the budget is smaller than the objects need, so the loader
+flushes early to stay inside it and the objects come out smaller, about 17 MB
+median. `--max-flush-delay 30s` lets an object fill before the age trigger
+closes it. Every value can be overridden from the environment (`RAVEL_BATCH_ROWS`,
+`RAVEL_TARGET_BYTES`, `RAVEL_PIPELINE_DEPTH`, `RAVEL_READ_CURSORS`,
+`RAVEL_LOAD_MEMORY_BYTES`, `RAVEL_MAX_FLUSH_DELAY`). **There is no post-load
+step**: no compaction, no catalog fold, no VACUUM equivalent, so the layout the
+queries run against is the layout ingest produced.
 
 `./data-size` reports the tenant's whole durable footprint in the bucket (data
 objects, commit records, manifests, catalog snapshots), which is where all of
@@ -90,61 +109,70 @@ Ravel's state lives.
 The reference machine's disk is a 500 GB gp2 volume. Ravel holds no data on
 local disk, so a cold run (server restarted, page cache dropped) fetches what it
 needs from RustFS, and RustFS reads it from that volume. The stock fetch policy
-reads whole objects, so a statement that touches the table reads the whole
-dataset from the volume, whatever the statement computes. The dataset is
-9.8 GB since 0.21.0, whose log format stores the same rows smaller; it was
-11.2 GB before. Measured on v0.16.1, 40 of the 42 statements that return a
-number take 42.6 to 46.6 s cold, consistent with reading the dataset at about
-250 MB/s each time; the other two (q1 and q7) take about 0.5 s. A 0.19.0 build
-measured 42.6 to 43.3 s for the same 40. On v0.21.0, in the same fresh-VM
-setup, the cold sum over those 42 statements was 1,252.7 s, against 1,712.1 s
-for 0.19.0.
-
-Ravel holds no data on local disk, so a warm run is served from the read cache
-or from RustFS. Since 0.19.0, a server whose store is on loopback derives its
-fetch cache at 40% of its memory budget instead of 25%: 12.3 GB on this
-machine, above the dataset, so the warm runs are served from the cache.
-Before 0.19.0 the derived cache was 7.7 GB, and warm runs re-read the dataset
-from RustFS through the page cache.
+reads an object whole when it is below the ranged-read break-even (about 19 MB
+here) and only the columns it needs above it. The 4 MB objects before 0.23.0
+were all below it, so a statement that touched the table read the whole
+dataset from the volume, whatever it computed; the 25 MB objects are above it. On c6a.4xlarge, in this repository's fresh-VM setup, the
+cold sum over the 43 statements was 486.6 s on v0.23.0, against 1,291.0 s on
+v0.21.0. The dataset is 10.2 GB (9.8 GB on v0.21.0; fewer, larger objects
+compress slightly differently).
+
+A warm run is served from the read cache or from RustFS. A server whose store
+is on loopback derives its fetch cache at 40% of its memory budget: 11.4 GB on
+this machine, above the dataset, so the warm runs are served from the cache.
 
 ### The tuned configuration
 
 This directory publishes one result, the stock one. A tuned run belongs in its
 own top-level entry the way the other tuned entries in this repository do, and
-that is left for a follow-up. The tuned configuration is three server flags
-passed through `RAVEL_TUNED_ARGS` in `./start`:
+that is left for a follow-up. The tuned configuration is server flags passed
+through `RAVEL_TUNED_ARGS` in `./start`:
 
 ```
-RAVEL_TUNED_ARGS="--logs-fetch-policy latency-first --fetch-concurrency 256 --sql-max-query-bytes 12884901888" ./benchmark.sh
+RAVEL_TUNED_ARGS="--logs-fetch-policy latency-first --fetch-concurrency 256 --sql-max-query-bytes <stock value>" ./benchmark.sh
 ```
 
 `latency-first` is a named policy, not a tuning constant: it says spend
 requests to save wall time, and resolves the byte quantities exactly as
 `byte-minimal` does, so a logs read takes ranged reads wherever they save
 bytes. It only pays off once fetch concurrency is raised with it, which is what
 the second flag does (it sets the object-store GET permits, the SQL partition
-count and the PromQL fan-out together). The third flag lifted the per-query
-memory pool from the 8.2 GB that versions before 0.21.0 derived to 12 GiB,
-which let the widest `GROUP BY` in the set (q33) complete instead of being
-refused. Since 0.21.0 the derived pool is 16.5 GB on this machine, and q33
-completes without it. The trade is more object-store requests for less cold
-wall-clock. It has not been measured against RustFS; the real-S3 figures below
-are the reference for it.
+count and the PromQL fan-out together). Raising the permits shrinks the derived
+per-query pool, so the third flag holds it at the value a stock start logs on
+its `sql_max_query_bytes` line (about 14.2 GB on c6a.4xlarge); without it the
+widest `GROUP BY` (q33) is refused.
+
+Measured on v0.23.0 against RustFS on c6a.4xlarge, the tuned configuration
+took the cold sum from 486.6 s to 384.6 s and left the hot sum unchanged
+(56.9 s and 57.1 s). It is not safe on smaller machines. In-flight fetches at
+256 permits draw on the same process memory budget the queries do, so on
+c6a.2xlarge (16 GB) three statements are refused that the stock configuration
+answers, and more on 8 and 4 GB.
 
 ### Reference: the same binaries on real S3
 
-Measured by us on the same machine type against an S3 bucket in the instance's
-region, credentials from the instance role, with the same driver and the same
-true-cold protocol, on the released v0.16.1 binaries and a freshly loaded
-tenant. One pass each. Not reproducible by this harness, which does not run
-entries on real S3, and therefore not a results file. Cold and hot are the sums
-of the first and third run over the statements that returned a number.
-
-| configuration | load | cold | hot | statements |
-|---|---|---|---|---|
-| stock, real S3 | 1,191 s | 483 s | 430 s | 42 of 43, q33 refused |
-| tuned, real S3 | (same tenant) | 201 s | 99 s | 43 of 43 |
-| stock, local RustFS, v0.16.1 | 1,236 s | 1,720 s | 272 s | 42 of 43, q33 refused |
+Measured by us on c6a.4xlarge against a fresh S3 bucket in the instance's
+region (us-east-1), credentials from the instance role, with the same driver
+and the same true-cold protocol, on the released v0.23.0 binaries and this
+entry's `./load` and `./start` unchanged except for the credential source
+(`--s3-auth instance-role`). One pass. Not reproducible by this harness, which
+does not run entries on real S3, and therefore not a results file. Cold is the
+sum of the first runs; hot is the sum of the better of the second and third
+runs.
+
+| configuration | load | cold | hot | hot geomean | statements |
+|---|---|---|---|---|---|
+| stock, real S3, v0.23.0 | 932 s | 186 s | 63 s | 1.01 s | 43 of 43 |
+| stock, local RustFS, v0.23.0 | 924 s | 487 s | 57 s | 0.74 s | 43 of 43 |
+
+Cold is faster on S3 because nothing waits on the 250 MB/s gp2 volume: S3
+serves the ranged reads in parallel at network speed. Hot is slightly slower
+because the larger fetch cache share a loopback store gets does not apply to
+a remote endpoint: the derived cache was 7.5 GB on this machine, below the
+10.2 GB dataset, so part of each warm run goes back to S3.
+
+The earlier reference, v0.16.1 with ~4 MB objects, measured a 483 s cold and
+430 s hot sum on real S3, with q33 refused.
 
 ## Notes
 
@@ -156,5 +184,8 @@ of the first and third run over the statements that returned a number.
   endpoint's window defaults to the last hour.
 - The heaviest whole-table aggregates can exceed the derived per-query memory
   pool on a small instance and are reported as errors rather than being run
-  with a raised limit. q33 reserves about 10.9 GB. Since 0.21.0 it completes on
-  c6a.4xlarge; on instances with less memory it can still be refused.
+  with a raised limit. q33 reserves about 10.9 GB. It completes on c6a.4xlarge
+  and larger; on instances with less memory it can still be refused.
+- q28 averages `octet_length("URL")`. ClickHouse's `length()` counts bytes and
+  DataFusion's counts characters, so `octet_length` is the faithful
+  translation; q29 still uses `length("Referer")`.
diff --git a/ravel/install b/ravel/install
@@ -42,7 +42,7 @@ if ! command -v aws >/dev/null 2>&1; then
 fi
 aws --version
 
-RAVEL_VERSION="${RAVEL_VERSION:-v0.21.0}"
+RAVEL_VERSION="${RAVEL_VERSION:-v0.23.0}"
 RAVEL_REPO="${RAVEL_REPO:-https://github.com/NOFireAI/ravel}"
 
 case "$(uname -m)" in

diff --git a/ravel/load b/ravel/load
@@ -29,21 +29,44 @@ MAPPING="${RAVEL_MAPPING:-hits.mapping.toml}"
 "$RAVEL_CLI" --store s3 --s3-auth static \
     typed-attr-column set "$RAVEL_TENANT" --from-mapping "$MAPPING"
 
-# Load geometry. One batch becomes one object per involved shard, so
-# --batch-rows sets the object size directly: 150,000 rows over 4 shards is
-# ~37.5k rows (~4 MB) per object and ~2,600 objects for the 100M-row dataset.
-# That is the whole object-sizing mechanism; --target-bytes stays at its
-# default of 1 (flush every write) because accumulating instead makes each
-# batch's acknowledgement wait for a later batch and the load runs several
-# times longer for the same layout.
+# Load geometry. Small batches merge into large stored objects: a shard's
+# buffer accumulates batches until its estimated uncompressed content reaches
+# --target-bytes, and on this dataset a stored object holds about 1/74 of that
+# estimate, so 1,850,000,000 stores objects of about 25 MB (about 410 objects
+# for the 100M rows). Large objects are what make a cold scan cheap here.
+# --max-flush-delay must outlast one object's fill time or the age trigger
+# closes objects early, and --pipeline-depth must cover the batches one object
+# needs on a shard, since each batch's acknowledgement waits for the flush that
+# holds it. --load-memory-bytes caps what the loader holds; the depth and the
+# budget are sized to the machine's memory below (measured on 128, 16, 8 and
+# 4 GB hosts, NOFireAI/ravel#2627). Every value can be overridden from the
+# environment.
+mem_kb=$(awk '/^MemTotal:/ {print $2}' /proc/meminfo)
+if [ "$mem_kb" -ge 15000000 ]; then
+    tier_depth=32; tier_budget=6500000000; tier_cursors=16
+elif [ "$mem_kb" -ge 7000000 ]; then
+    tier_depth=24; tier_budget=3500000000; tier_cursors=16
+elif [ "$mem_kb" -ge 3000000 ]; then
+    tier_depth=16; tier_budget=1200000000; tier_cursors=2
+else
+    # Below 3 GB the loader derives its own budget from the host's memory and
+    # admits one batch at a time when that is below one batch.
+    tier_depth=8; tier_budget=; tier_cursors=1
+fi
+load_budget="${RAVEL_LOAD_MEMORY_BYTES-$tier_budget}"
+echo "load: MemTotal ${mem_kb} kB, pipeline depth ${RAVEL_PIPELINE_DEPTH:-$tier_depth}, read cursors ${RAVEL_READ_CURSORS:-$tier_cursors}, memory budget ${load_budget:-derived}" >&2
+
 "$RAVEL_CLI" --store s3 --s3-auth static load \
     --parquet "$PARQUET" \
     --tenant "$RAVEL_TENANT" \
     --mapping "$MAPPING" \
     --shards "$RAVEL_SHARDS" \
-    --read-cursors "${RAVEL_READ_CURSORS:-16}" \
-    --batch-rows "${RAVEL_BATCH_ROWS:-150000}" \
-    --target-bytes 1
+    --read-cursors "${RAVEL_READ_CURSORS:-$tier_cursors}" \
+    --batch-rows "${RAVEL_BATCH_ROWS:-100000}" \
+    --pipeline-depth "${RAVEL_PIPELINE_DEPTH:-$tier_depth}" \
+    ${load_budget:+--load-memory-bytes "$load_budget"} \
+    --max-flush-delay "${RAVEL_MAX_FLUSH_DELAY:-30s}" \
+    --target-bytes "${RAVEL_TARGET_BYTES:-1850000000}"
 
 sync
 

diff --git a/ravel/queries.sql b/ravel/queries.sql
@@ -25,7 +25,7 @@ SELECT * FROM logs WHERE "URL" LIKE '%google%' ORDER BY ts LIMIT 10;
 SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY ts LIMIT 10;
 SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY "SearchPhrase" LIMIT 10;
 SELECT "SearchPhrase" FROM logs WHERE "SearchPhrase" <> '' ORDER BY ts, "SearchPhrase" LIMIT 10;
-SELECT "CounterID", AVG(length("URL")) AS l, COUNT(*) AS c FROM logs WHERE "URL" <> '' GROUP BY "CounterID" HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25;
+SELECT "CounterID", AVG(octet_length("URL")) AS l, COUNT(*) AS c FROM logs WHERE "URL" <> '' GROUP BY "CounterID" HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25;
 SELECT regexp_replace("Referer", '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS k, AVG(length("Referer")) AS l, COUNT(*) AS c, MIN("Referer") FROM logs WHERE "Referer" <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25;
 SELECT SUM("ResolutionWidth"), SUM("ResolutionWidth" + 1), SUM("ResolutionWidth" + 2), SUM("ResolutionWidth" + 3), SUM("ResolutionWidth" + 4), SUM("ResolutionWidth" + 5), SUM("ResolutionWidth" + 6), SUM("ResolutionWidth" + 7), SUM("ResolutionWidth" + 8), SUM("ResolutionWidth" + 9), SUM("ResolutionWidth" + 10), SUM("ResolutionWidth" + 11), SUM("ResolutionWidth" + 12), SUM("ResolutionWidth" + 13), SUM("ResolutionWidth" + 14), SUM("ResolutionWidth" + 15), SUM("ResolutionWidth" + 16), SUM("ResolutionWidth" + 17), SUM("ResolutionWidth" + 18), SUM("ResolutionWidth" + 19), SUM("ResolutionWidth" + 20), SUM("ResolutionWidth" + 21), SUM("ResolutionWidth" + 22), SUM("ResolutionWidth" + 23), SUM("ResolutionWidth" + 24), SUM("ResolutionWidth" + 25), SUM("ResolutionWidth" + 26), SUM("ResolutionWidth" + 27), SUM("ResolutionWidth" + 28), SUM("ResolutionWidth" + 29), SUM("ResolutionWidth" + 30), SUM("ResolutionWidth" + 31), SUM("ResolutionWidth" + 32), SUM("ResolutionWidth" + 33), SUM("ResolutionWidth" + 34), SUM("ResolutionWidth" + 35), SUM("ResolutionWidth" + 36), SUM("ResolutionWidth" + 37), SUM("ResolutionWidth" + 38), SUM("ResolutionWidth" + 39), SUM("ResolutionWidth" + 40), SUM("ResolutionWidth" + 41), SUM("ResolutionWidth" + 42), SUM("ResolutionWidth" + 43), SUM("ResolutionWidth" + 44), SUM("ResolutionWidth" + 45), SUM("ResolutionWidth" + 46), SUM("ResolutionWidth" + 47), SUM("ResolutionWidth" + 48), SUM("ResolutionWidth" + 49), SUM("ResolutionWidth" + 50), SUM("ResolutionWidth" + 51), SUM("ResolutionWidth" + 52), SUM("ResolutionWidth" + 53), SUM("ResolutionWidth" + 54), SUM("ResolutionWidth" + 55), SUM("ResolutionWidth" + 56), SUM("ResolutionWidth" + 57), SUM("ResolutionWidth" + 58), SUM("ResolutionWidth" + 59), SUM("ResolutionWidth" + 60), SUM("ResolutionWidth" + 61), SUM("ResolutionWidth" + 62), SUM("ResolutionWidth" + 63), SUM("ResolutionWidth" + 64), SUM("ResolutionWidth" + 65), SUM("ResolutionWidth" + 66), SUM("ResolutionWidth" + 67), SUM("ResolutionWidth" + 68), SUM("ResolutionWidth" + 69), SUM("ResolutionWidth" + 70), SUM("ResolutionWidth" + 71), SUM("ResolutionWidth" + 72), SUM("ResolutionWidth" + 73), SUM("ResolutionWidth" + 74), SUM("ResolutionWidth" + 75), SUM("ResolutionWidth" + 76), SUM("ResolutionWidth" + 77), SUM("ResolutionWidth" + 78), SUM("ResolutionWidth" + 79), SUM("ResolutionWidth" + 80), SUM("ResolutionWidth" + 81), SUM("ResolutionWidth" + 82), SUM("ResolutionWidth" + 83), SUM("ResolutionWidth" + 84), SUM("ResolutionWidth" + 85), SUM("ResolutionWidth" + 86), SUM("ResolutionWidth" + 87), SUM("ResolutionWidth" + 88), SUM("ResolutionWidth" + 89) FROM logs;
 SELECT "SearchEngineID", "ClientIP", COUNT(*) AS c, SUM("IsRefresh"), AVG("ResolutionWidth") FROM logs WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "ClientIP" ORDER BY c DESC LIMIT 10;

diff --git a/ravel/results/20261012/c6a.2xlarge.json b/ravel/results/20261012/c6a.2xlarge.json
@@ -0,0 +1,60 @@
+{
+      "system": "Ravel",
+      "date": "2026-10-12",
+      "machine": "c6a.2xlarge",
+      "cluster_size": 1,
+      "proprietary": "no",
+      "hardware": "cpu",
+      "tuned": "no",
+      "tags": ["Rust","column-oriented","S3"],
+      "load_time": 1076,
+      "data_size": 10206509683,
+      "concurrent_qps": 0.27,
+      "concurrent_error_ratio": 0.426,
+      "result": [
+        [0.107, 0.046, 0.046],
+        [7.201, 0.717, 0.715],
+        [10.113, 0.996, 1.009],
+        [11.372, 1.09, 1.088],
+        [11.515, 1.801, 1.786],
+        [10.644, 2.225, 2.211],
+        [0.092, 0.048, 0.046],
+        [7.193, 0.715, 0.715],
+        [14.3, 2.351, 2.334],
+        [16.555, 2.687, 2.704],
+        [17.283, 1.59, 1.591],
+        [17.533, 1.785, 1.787],
+        [10.634, 2.012, 2.034],
+        [16.42, 3.146, 3.052],
+        [10.951, 2.293, 2.315],
+        [11.554, 1.915, 1.902],
+        [16.236, 3.832, 3.794],
+        [16.036, 3.654, 3.6],
+        [16.126, 4.631, 4.627],
+        [9.531, 0.684, 0.696],
+        [16.706, 2.651, 2.666],
+        [21.854, 3.596, 3.563],
+        [23.745, 11.355, 8.669],
+        [17.371, 3.173, 3.209],
+        [10.487, 1.678, 1.669],
+        [10.484, 1.635, 1.613],
+        [10.491, 1.677, 1.7],
+        [16.996, 3.301, 3.267],
+        [17.366, 9.067, 9.193],
+        [9.865, 0.857, 0.843],
+        [18.361, 3.094, 3.032],
+        [22.265, 3.691, 3.455],
+        [null, null, null],
+        [17.502, 5.71, 5.744],
+        [17.457, 5.752, 5.774],
+        [8.714, 1.799, 1.751],
+        [0.423, 0.133, 0.124],
+        [0.44, 0.1, 0.1],
+        [0.377, 0.106, 0.106],
+        [1.792, 0.221, 0.21],
+        [1.472, 0.09, 0.092],
+        [0.545, 0.091, 0.092],
+        [0.326, 0.081, 0.079]
+]
+  } 
+