Repository navigation
feature: add install-time logtime timestamp precision choice (#325) - #449
Merged
Merged
Conversation
TheWitness
force-pushed
the
feature/logtime-precision
branch
from
November 13, 2026 16:54
0e67a72 to
4295aff
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Remote-collector conversion defects can lose messages or prevent processed-row deletion, while partial conversions can report success and missing precision helpers break the schema CI test.
4 open issues
What changed in this PR
Adds install-time timestamp precision selection for Cacti Syslog to address #325, supporting millisecond storage on MariaDB while retaining second precision on MySQL.
Changes:
- Adds precision selection to interactive and silent installs without automatically converting existing schemas.
- Introduces a confirmation-gated, destructive MariaDB conversion utility.
- Documents precision choices and forwarder configuration, and updates translatable strings.
| File | Description |
|---|---|
setup.php |
Rejects explicit millisecond requests on MySQL. |
README.md |
Documents precision selection, conversion, and forwarder setup. |
locales/po/cacti.pot |
Updates installer and validation translation strings. |
includes/schema.php |
Resolves precision and applies it during table creation. |
includes/installer.php |
Adds the MariaDB timestamp precision selector. |
config.php.dist |
Documents the silent-install precision option. |
config_local.php.dist |
Documents the local silent-install precision option. |
cli/updatedb_convert_ms.php |
Adds destructive conversion to millisecond timestamps. |
CHANGELOG.md |
Records the feature and platform restrictions. |
Review effort: Balanced
TheWitness
force-pushed
the
feature/logtime-precision
branch
from
November 13, 2026 17:22
c149b4f to
794d2ed
Compare
Allow the syslog logtime column to be stored at millisecond (TIMESTAMP(3)) or second (TIMESTAMP) precision, chosen at install time. Millisecond precision is only offered on MariaDB, where a fractional TIMESTAMP(3) column can still be RANGE partitioned by UNIX_TIMESTAMP(logtime) and pruned. MySQL rejects that partition function on a fractional column (ERROR 1491 on 8.0 and 9.4), so MySQL installs are fixed to second precision and an explicit millisecond request on MySQL is rejected with a clear message instead of being silently downgraded. New MariaDB installs default to millisecond precision. Upgrades never migrate the schema automatically: the existing column type is authoritative, so an inadvertent config change cannot trigger a long, blocking table rewrite. Switching an existing production MariaDB install is a deliberate, destructive CLI operation (cli/updatedb_convert_ms.php) run during a maintenance window. - includes/schema.php: add syslog_db_is_mariadb(), syslog_logtime_sql_type(), syslog_detect_logtime_precision(), syslog_resolve_logtime_precision(); thread the resolved precision into the syslog and syslog_incoming tables (syslog_removed inherits via LIKE) and store syslog_install_logtime_precision - includes/installer.php: add the Timestamp Precision advisor field (MariaDB dropdown defaulting to millisecond; hidden/second on MySQL; hidden on upgrade) - setup.php: reject an explicit millisecond request on MySQL - cli/updatedb_convert_ms.php: new MariaDB-only, confirm-gated converter that truncates syslog/syslog_incoming/syslog_removed and widens logtime to TIMESTAMP(3) - config.php.dist / config_local.php.dist: document $syslog_install_options['logtime_precision'] - README.md: add a Millisecond Timestamps section with rsyslog/Cisco/Juniper/ Arista/syslog-ng forwarder guidance - locales/po/cacti.pot: regenerate for the new translatable strings Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- tests/regression/cli_schema_upgrade_test.php: stub syslog_resolve_logtime_precision()
and syslog_logtime_sql_type(); the harness evaluates syslog_setup_table_new() in
isolation and did not load the new precision helpers, so the schema CLI regression
check fatally errored with an undefined function (the CI failure)
- includes/schema.php: create syslog_replication_output with the effective logtime
precision, so a MariaDB remote collector's outbox matches the millisecond incoming
timestamps and the exact-logtime archival join keeps deleting processed rows
- cli/updatedb_convert_ms.php:
- also convert syslog_replication_output, widening it in place (no truncate) so
undelivered rows are preserved
- preserve the AUTO_INCREMENT high-water marks of the truncated tables so new rows
never reuse sequence values that remote replication identities reference
- make the conversion all-or-nothing and re-runnable: every TRUNCATE/ALTER result
is checked, the already-converted early check covers every target table, and the
stored precision setting is only recorded once every target column is TIMESTAMP(3)
- README.md: document the outbox widening, sequence preservation and all-or-nothing
behaviour of the converter
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TheWitness
force-pushed
the
feature/logtime-precision
branch
from
November 13, 2026 17:26
794d2ed to
6e07a98
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…325) cli/updatedb_convert_ms.php bootstraps Cacti at the top level (includes cli_check.php), so like the other CLI entry points it cannot be loaded into the isolated unit process and is absent from Clover. Add it to the patch-coverage unmeasured allowlist so the 100% changed-line gate passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TheWitness
force-pushed
the
feature/logtime-precision
branch
from
November 13, 2026 17:40
9a42504 to
83d8601
Compare
|
xmacan
approved these changes
Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an install-time choice for the precision of the syslog
logtimecolumn:millisecond (
TIMESTAMP(3)) or second (TIMESTAMP). Millisecond precisionpreserves the sub-second ordering of high-rate messages.
Because of an intrinsic engine difference, millisecond precision is offered
only on MariaDB:
RANGEpartition aTIMESTAMP(3)column byUNIX_TIMESTAMP(logtime)and still prune partitions.(
ERROR 1491 (HY000): The PARTITION function returns the wrong type),verified on MySQL 8.0.46 and 9.4.0.
FLOOR()/DIV-wrapped variants compilebut defeat partition pruning, and
CAST(... * 1000)is rejected outright(
ERROR 1564). So MySQL installs are fixed to second precision.Key behaviours:
defaults to millisecond; MySQL is fixed to second.
$syslog_install_options['logtime_precision'] = 'ms'|'sec'.A millisecond request on MySQL is rejected with a clear message rather than
silently downgraded.
logtimecolumn type isauthoritative, so changing the configured precision cannot trigger a long,
blocking table rewrite.
cli/updatedb_convert_ms.phptruncatessyslog,syslog_incoming, andsyslog_removedand widenslogtimetoTIMESTAMP(3)(confirm-gated).from Cisco IOS/NX-OS, Juniper, Arista, and syslog-ng.
Related Issue
#325
Motivation and Context
Issue #325 requests sub-second syslog timestamps. A naive switch to
TIMESTAMP(3)breaks the only supported (partitioned) table architecture on MySQL, because its
partition function cannot be fractional and cannot prune. This change delivers
millisecond precision where it actually works (MariaDB) while keeping MySQL on a
safe, pruning-capable second-granularity schema, and it treats the switch on an
existing production system as an explicit, data-destroying maintenance operation
rather than an automatic upgrade step.
How Has This Been Tested?
php -lon all changed PHP files.TIMESTAMP(3)partitionedsyslogtablecreates, stores milliseconds, and prunes (
EXPLAIN PARTITIONSselects a singleday partition). The CLI path (
TRUNCATEthenALTER TABLE ... MODIFY logtime timestamp(3)on the partitioned table) succeedsand still prunes afterward.
TIMESTAMP(3)+UNIX_TIMESTAMP(logtime)partitioning is rejected (ERROR 1491), justifying theMariaDB-only gate.
PartitionPrecreateBoundaryTestsource-extraction logic toconfirm partition names are unchanged and that the new
$precisionargumentyields plain
timestampby default andtimestamp(3)forms.locales/po/cacti.potregenerated via thexgettextstep; only the.potisincluded (per the Weblate convention).
Types of changes
Checklist: