fix: sync s3 metadata writes#382
Conversation
Persist metadata writes before updating local S3 backend state so storage errors are returned to callers instead of being hidden by deferred flushes.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de381755a3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| backend: s, | ||
| name: name, | ||
| state: make(map[string]any), | ||
| done: make(chan struct{}), | ||
| } |
There was a problem hiding this comment.
With the old syncLoop no longer started for each namespace, an S3-backed process only sees remote metadata changes after an explicit Namespace.Flush; I checked the production wiring and Flush is not called by the tiered cache or git strategy after config.Load wires the store in. In a multi-replica tiered cache, a replica that only serves reads will therefore keep its local cache-etags map stale indefinitely, so Tiered.invalidateStale can continue serving an out-of-date lower tier even after another replica synchronously persisted a new authoritative ETag.
Useful? React with 👍 / 👎.
| func (n *Namespace) apply(ops ...Op) error { | ||
| return errors.Wrap(n.backend.Apply(context.Background(), n.name, ops...), "apply metadata op") | ||
| } |
There was a problem hiding this comment.
Now that S3 Apply performs the lock/load/store path synchronously, using context.Background() for every public metadata write discards the caller's request deadline/cancellation. Under an S3 outage or a stuck namespace lock, callers such as tiered cache create/delete can remain blocked in lockNamespace or MinIO calls even after the HTTP request context has been canceled, because this write path has no cancellable context to observe.
Useful? React with 👍 / 👎.
1 participant
Persist metadata writes before updating local S3 backend state so storage errors are returned to callers instead of being hidden by deferred flushes.