Skip to content

ci: shard coverage and upload report to S3 - #504

Merged
p0rtale merged 3 commits into
masterfrom
add-s3-coverage-upload
Sep 23, 2026
Merged

p0rtale merged 3 commits into
masterfrom
add-s3-coverage-upload

Conversation

@p0rtale

@p0rtale p0rtale commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Coverage reports are now uploaded to S3 on pushes to master and tag releases.

Since a single coverage job took too long, the coverage run is sharded across 6 parallel jobs: CE coverage on Tarantool master and EE coverage on Tarantool Enterprise 3.6, each split into 3 shards. Each job runs a subset of the test files with luacov and publishes its luacov.stats*.out as an artifact, then a dedicated job merges them with test/luacov-merger.lua and uploads the generated report.

Also server wait timeouts became configurable via LUATEST_SERVER_WAIT_TIMEOUT (set to 180 in coverage jobs) to accommodate the slower luacov-instrumented storage.

Additionally, get_other_storage_bucket_id now retries vshard.storage.buckets_info until it responds or the deadline expires, instead of calling next() on a nil result when the storage times out under coverage.

@p0rtale
p0rtale requested a review from vakhov July 14, 2026 20:23
@p0rtale p0rtale self-assigned this Jul 14, 2026
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch 6 times, most recently from 370c5d8 to 71b147c Compare July 16, 2026 10:49
Comment thread .github/workflows/test_on_push.yaml Resolved
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch from 71b147c to 0fe4a89 Compare July 16, 2026 11:27
@p0rtale
p0rtale requested a review from vakhov July 16, 2026 11:31

@vakhov vakhov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch 2 times, most recently from 565bd51 to 7ba90c4 Compare July 16, 2026 13:10
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch 3 times, most recently from 1f0a348 to 57c1b8e Compare July 29, 2026 08:11
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch from 57c1b8e to 72429fd Compare September 14, 2026 12:17
@p0rtale p0rtale changed the title ci: upload luacov report to S3 ci: shard coverage and upload report to S3 Sep 14, 2026
@p0rtale
p0rtale requested review from a1div0 and vakhov September 14, 2026 12:21
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch from 72429fd to 321e8c7 Compare September 14, 2026 20:36
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch from 321e8c7 to 1977535 Compare September 21, 2026 08:22
- Upload the luacov report to S3 on master pushes and tag releases.
- Run CE and EE coverage in parallel sharded jobs and merge the
  per-shard luacov stats in a dedicated job before generating the
  report. CE runs on Tarantool master, EE on Tarantool Enterprise 3.6.
- Make server wait timeouts configurable via LUATEST_SERVER_WAIT_TIMEOUT
  and set it to 180 in coverage jobs.
- Retry vshard.storage.buckets_info in get_other_storage_bucket_id to
  avoid next() on a nil result when storage is slow under coverage.
- Use LUATEST_SERVER_WAIT_TIMEOUT instead of a hardcoded 60s timeout
  for bucket wait helpers and the double-buckets rebalancing retries.
@p0rtale
p0rtale force-pushed the add-s3-coverage-upload branch from 3d0e93c to f30a7d5 Compare September 22, 2026 18:21
@p0rtale
p0rtale requested a review from vakhov September 23, 2026 08:23
vakhov

This comment was marked as resolved.

@vakhov
vakhov self-requested a review September 23, 2026 09:19
Write a trailing space after the last counter, otherwise luacov
skips it and reports the last line as uncovered.
@p0rtale
p0rtale merged commit 13523e3 into master Sep 23, 2026
72 of 73 checks passed
@p0rtale
p0rtale deleted the add-s3-coverage-upload branch September 23, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants