Search before asking
Motivation
Although we has coverd most of standard metrics defined in FLIP-33: Standardize Connector Metrics, we still miss the metrics of pendingRecords which help us to trace data backlog.
Solution
pending records = latest log offset - fetched log offset.
For snapshot read phase of primary key table, it should be total records in kv snapshot - records emited, but to simplify, we can just trace the phase of incremental reading which read from change log.
Anything else?
No response
Willingness to contribute
Search before asking
Motivation
Although we has coverd most of standard metrics defined in FLIP-33: Standardize Connector Metrics, we still miss the metrics of
pendingRecordswhich help us to trace data backlog.Solution
pending records =
latest log offset - fetched log offset.For snapshot read phase of primary key table, it should be total records in kv snapshot - records emited, but to simplify, we can just trace the phase of incremental reading which read from change log.
Anything else?
No response
Willingness to contribute