Skip to content

sql-150: reclaim storage metadata for ephemeral items on crash - #38160

Open
SangJunBak wants to merge 1 commit into
jun/move-temp-to-catalog-split-3from
jun/move-temp-to-catalog-split-4
Open

sql-150: reclaim storage metadata for ephemeral items on crash#38160
SangJunBak wants to merge 1 commit into
jun/move-temp-to-catalog-split-3from
jun/move-temp-to-catalog-split-4

Conversation

@SangJunBak

@SangJunBak SangJunBak commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Motivation

sql-150

Description

catalog: reclaim storage metadata for ephemeral items on crash restart (SQL-150)

When a process crash-restarted, shards from temp tables were never cleaned up. However, on graceful close, we do clean up these shards. This was an existing bug from before the durable temporary objects change.

Verification

Extended the temp object cleanup test to check for shard finalization

@SangJunBak
SangJunBak requested a review from a team as a code owner August 11, 2026 14:25
@SangJunBak SangJunBak added the ci-nightly PR CI control: also trigger Nightly label Aug 11, 2026
@SangJunBak SangJunBak changed the title catalog: reclaim storage metadata for ephemeral items on crash restar… sql-150: reclaim storage metadata for ephemeral items on crash Aug 11, 2026
@linear-code

linear-code Bot commented Aug 11, 2026

Copy link
Copy Markdown

SQL-150

Comment thread test/restart/mzcompose.py
Comment on lines +1483 to +1484
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should actually check something

Suggested change
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")
# The comment row dies with its item.
comments = c.sql_query(temp_comment_count, port=6877, user="mz_system")
assert comments == [
(0,)
], f"the temp table's comment survived the restart: {comments}"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! 🙈

@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch 3 times, most recently from 5efd494 to 3b9c0fb Compare August 13, 2026 01:35
…t (SQL-150)

When a process crash-restarted, shards from temp tables were never cleaned up. However, on graceful close, we do clean up these shards. This was an existing bug from before the durable temporary objects change.
@SangJunBak
SangJunBak force-pushed the jun/move-temp-to-catalog-split-4 branch from 3b9c0fb to 1537f61 Compare August 13, 2026 15:19
@SangJunBak
SangJunBak requested a review from def- August 13, 2026 15:23

@mtabebe mtabebe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, change looks good. Great work

/// Used to reclaim temporary items when the catalog is opened with write
/// intent, at which point every session that could own one is dead.
///
/// This must mirror everything the graceful `Op::DropObjects` path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-nightly PR CI control: also trigger Nightly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants