diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..cd8ab5e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,151 @@ +# CTraces repository guide + +This is the canonical operating guide for humans and coding agents working in +CTraces. Tool-specific files may point here but must not redefine these rules. + +## Project role + +CTraces is a C library for constructing trace contexts and encoding or decoding +them as the library's MessagePack representation, text, or OpenTelemetry +protobuf. Fluent Bit consumes this library, so changes can affect a larger +downstream runtime. + +## Repository map + +- `include/ctraces/`: public headers and public data structures. +- `src/`: lifecycle, IDs, resources, scopes, spans, attributes, logging, and + format encoders/decoders. +- `tests/`: Acutest-based unit and round-trip tests registered with CTest. +- `examples/`: small API and OpenTelemetry examples, built by `CTR_DEV`. +- `cmake/`: dependency paths and CMake helper macros. +- `lib/cfl`: pinned CFL submodule used for variants and containers. +- `lib/fluent-otel-proto`: pinned OpenTelemetry protobuf C bindings. +- `lib/mpack`: vendored MPack source. +- `.github/workflows/`: platform, lint, package, and release automation. +- `docs/ai/`: detailed investigation and change workflows. + +Start architecture work at `include/ctraces/ctraces.h`, then follow the owning +public header into the matching `src/ctr_*.c` file. Encoding changes begin at +`ctr_encode_*`; parsing and input validation begin at `ctr_decode_*`. + +## Dependencies and downstream use + +Dependency direction is CTraces -> CFL, fluent-otel-proto, and MPack. These +dependencies do not own CTraces lifecycle or wire-format behavior. Fluent Bit +is the verified downstream consumer. + +Do not edit files inside a submodule for a CTraces-only change. Land a change in +the owning dependency first, update its pinned revision separately, validate +the standalone dependency, and then validate CTraces and Fluent Bit. Follow +[`docs/ai/cross-repository.md`](docs/ai/cross-repository.md). + +## Build and test + +Initialize dependencies once: + +```sh +git submodule update --init --recursive +``` + +Canonical development build: + +```sh +scripts/agent-build.sh +``` + +Run the full test suite after building: + +```sh +scripts/agent-test.sh +``` + +Run a targeted CTest selection: + +```sh +scripts/agent-test.sh -R ctr-test-span +scripts/agent-test.sh -R ctr-test-opentelemetry +``` + +Run the complete local verification wrapper: + +```sh +scripts/agent-verify.sh +``` + +The wrappers use `build/agent/` by default. Set `CTR_BUILD_DIR` to select a +different out-of-source directory. Direct equivalents are: + +```sh +cmake -S . -B build/agent -DCTR_DEV=On -DCTR_TESTS=On +cmake --build build/agent --parallel +ctest --test-dir build/agent --output-on-failure +``` + +Windows CI uses `scripts/win_build.bat`. Packaging is exercised by +`.github/workflows/packages.yaml`; it is not part of routine local verification. + +## Lint and analysis + +Check repository shell scripts locally with: + +```sh +bash -n scripts/agent-build.sh scripts/agent-test.sh scripts/agent-verify.sh +``` + +CI runs ShellCheck and actionlint. The repository does not currently provide a +formatter, static-analysis target, working sanitizer preset, Valgrind target, +fuzzer, or CTraces benchmark. Do not claim those checks ran unless a concrete +command was added and executed. CFL's benchmarks are dependency-local, not +CTraces benchmarks. + +## Correctness and compatibility + +- Treat `include/ctraces/` as public API. Avoid gratuitous signature, struct + layout, enum value, or ownership changes. +- Preserve create/destroy pairing and list ownership. Audit allocation failure, + partial initialization, unlinking, and cleanup paths together. +- A `ctrace` owns its linked resources and spans. Direct child destruction must + leave owner lists valid and make later parent destruction safe. +- Encoders and decoders are compatibility-sensitive. Preserve field meaning, + integer widths, IDs, flags, ordering where output comparisons depend on it, + and rejection of malformed or incomplete input. +- OpenTelemetry changes must be checked against the pinned generated bindings; + do not hand-edit generated protobuf sources in the submodule. +- Exercise null, empty, truncated, duplicate, invalid-enum, oversized, and + allocation/error cleanup cases when they are relevant to the change. +- Keep compiler portability covered by CMake and CI in mind, including MSVC and + the CentOS 7 GCC build. + +## Generated and vendored files + +`include/ctraces/ctr_info.h` and `include/ctraces/ctr_version.h` are generated +from their `.in` templates during configuration and are ignored by Git. Modify +the templates or CMake inputs, not generated copies. Do not make incidental +changes under `lib/`; update pinned submodules or vendored MPack only when the +task explicitly owns that dependency change. + +## Change discipline + +- Keep changes in the owning subsystem and avoid unrelated cleanup. +- Preserve existing user changes and untracked files. +- Add a focused regression test for behavior changes when practical. +- Use the existing commit prefix for the touched interface when history makes + it clear; examples include `core:`, `span:`, `decode_msgpack:`, `tests:`, and + `workflows:`. +- Do not infer compatibility promises or release procedures absent from the + repository. + +## Definition of done + +A change is complete when its owning code path is understood, compatibility and +ownership effects are accounted for, focused tests cover the changed behavior, +`scripts/agent-verify.sh` passes, and the final report states validation and any +remaining downstream risk. Cross-repository changes also require standalone +dependency validation and final Fluent Bit integration validation. + +Detailed workflows: + +- [`docs/ai/investigate.md`](docs/ai/investigate.md) +- [`docs/ai/bug-fix.md`](docs/ai/bug-fix.md) +- [`docs/ai/code-review.md`](docs/ai/code-review.md) +- [`docs/ai/cross-repository.md`](docs/ai/cross-repository.md) diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/README.md b/README.md index 7caca25..9fb6407 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,112 @@ # CTraces -The [CTraces](https://github.com/calyptia/ctraces) project is a tiny library to create and maintain Traces contexts and provide utilities for data manipulation, including encoding/decoding for compatibility with OpenTelemetry and other formats. +CTraces is a small C library for creating and managing distributed trace data. +It provides an in-memory trace model plus encoders and decoders for: -This project is a core library for [Fluent Bit](https://fluentbit.io): agent and aggregator for Observability. +- OpenTelemetry protobuf +- CTraces MessagePack +- human-readable text + +CTraces is a core library used by [Fluent Bit](https://fluentbit.io/). + +## Requirements + +- a C compiler +- CMake 3.20 or newer +- Git, including submodule support +- libcurl development files when building all examples + +The repository pins [CFL](https://github.com/fluent/cfl) and +[fluent-otel-proto](https://github.com/fluent/fluent-otel-proto) as Git +submodules. MPack is vendored under `lib/mpack`. ## Build -Clone the repository: +Clone the repository and its pinned dependencies: -```shell -git clone https://github.com/calyptia/ctraces +```sh +git clone --recurse-submodules https://github.com/fluent/ctraces.git +cd ctraces ``` -Get into the project directory and retrieve submodules: +For an existing checkout, initialize the dependencies with: -```shell -cd ctraces -git submodule update --init --recursive --remote +```sh +git submodule update --init --recursive ``` -Compile: +Configure and build the static library and tests out of source: -```shell -cd build/ -cmake -DCTR_DEV=on ../ -make +```sh +cmake -S . -B build -DCTR_TESTS=On +cmake --build build --parallel ``` -> CTR_DEV flag enables debugging mode, examples and the unit tests +Available project options: -## Usage +| Option | Default | Purpose | +| --- | --- | --- | +| `CTR_DEV` | `Off` | Enable a Debug build, tests, and examples | +| `CTR_TESTS` | `Off` | Build the unit tests | +| `CTR_EXAMPLES` | `Off` | Build the examples | +| `CTR_INSTALL_TARGETS` | `On` | Enable library and header installation | -In the [examples](examples/) directory, you will find a _simple_ example that describes how to use the API. +To build the complete development configuration: -## License +```sh +scripts/agent-build.sh +``` + +The wrapper uses `build/agent` by default. Set `CTR_BUILD_DIR` to use another +out-of-source build directory. + +## Test + +Run all registered tests: + +```sh +ctest --test-dir build --output-on-failure +``` -This program is under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0). +Or use the repository wrappers: -## Authors +```sh +scripts/agent-test.sh +scripts/agent-test.sh -R ctr-test-opentelemetry +scripts/agent-verify.sh +``` + +`agent-test.sh` uses the build produced by `agent-build.sh`. The verification +wrapper checks the shell scripts, builds the development configuration, and +runs the complete test suite. + +## Use the library + +Public headers are installed under `include/ctraces`. Start with +[`ctraces.h`](include/ctraces/ctraces.h) for context lifecycle and follow the +resource, scope, span, attribute, encoder, and decoder headers for focused APIs. + +[`examples/simple-c-api.c`](examples/simple-c-api.c) demonstrates construction +and cleanup of a complete trace. The other programs under [`examples/`](examples/) +show OpenTelemetry encoding and decoding. + +Install a configured build with: + +```sh +cmake --install build +``` + +## Contributing + +Repository architecture, ownership rules, compatibility-sensitive areas, and +canonical validation commands are documented in [`AGENTS.md`](AGENTS.md). +Detailed investigation, bug-fix, review, and cross-repository workflows live in +[`docs/ai/`](docs/ai/). + +Keep changes focused and add regression coverage for behavior changes. Public +headers and encoded formats are consumed outside this repository, particularly +by Fluent Bit. + +## License -[Calyptia Team](https://www.calyptia.com) +CTraces is licensed under the [Apache License 2.0](LICENSE). diff --git a/docs/ai/bug-fix.md b/docs/ai/bug-fix.md new file mode 100644 index 0000000..510ad55 --- /dev/null +++ b/docs/ai/bug-fix.md @@ -0,0 +1,45 @@ +# Bug-fix workflow + +## Purpose + +Produce the smallest verified correction with a regression test. + +## When to use + +Use for crashes, leaks, invalid output, rejected valid input, accepted malformed +input, ownership corruption, and behavior regressions. + +## Investigation + +1. Restate observed behavior and separate facts from assumptions. +2. Identify the owning subsystem and repository. +3. Trace the relevant public API through its complete code and cleanup paths. +4. Locate existing focused and round-trip tests. +5. Reproduce the failure when practical. + +## Implementation + +1. Add a regression test that fails for the reported reason. +2. Implement the smallest reasonable fix in the owning subsystem. +3. Audit sibling error paths for the same ownership or validation mistake. +4. Avoid API, struct layout, enum, and wire-format changes unless required and + explicitly justified. +5. Keep dependency and downstream changes in separate commits or repositories. + +## Validation + +Run the focused test with `scripts/agent-test.sh -R `, then run +`scripts/agent-verify.sh`. For dependency-facing behavior, also validate the +standalone dependency and Fluent Bit consumer as described in +`cross-repository.md`. + +## Final report + +State root cause, fix, regression coverage, exact commands and results, +compatibility impact, downstream validation, and unresolved risks. + +## Stop conditions + +Escalate when the expected behavior is ambiguous, the fix requires a public or +wire-format break, the issue belongs elsewhere, or a reliable regression test +cannot be constructed from available evidence. diff --git a/docs/ai/code-review.md b/docs/ai/code-review.md new file mode 100644 index 0000000..0a54b19 --- /dev/null +++ b/docs/ai/code-review.md @@ -0,0 +1,43 @@ +# Code-review workflow + +## Purpose + +Find actionable regressions and risks in a proposed change. + +## When to use + +Use for pull requests, patch series, dependency updates, and pre-merge audits. + +## Investigation + +1. Establish the merge base and read the complete diff and commit intent. +2. Map changed public headers to implementations, tests, and consumers. +3. Trace success and failure paths, including partial initialization and cleanup. +4. Check ownership, double unlink/free, use-after-free, null handling, integer + widths, bounds, malformed input, and deterministic encoding. +5. Check API/ABI and MessagePack/OpenTelemetry compatibility. +6. Look for missing regression and round-trip coverage. +7. Reject unnecessary complexity and call out measurable performance impact, + especially new allocation or repeated traversal in hot encode/decode paths. + +## Implementation + +Review does not authorize edits. Suggest the smallest concrete correction and +the test that demonstrates it. + +## Validation + +Run focused tests when practical and `scripts/agent-verify.sh` when assessing +merge readiness. Do not treat green CI as proof that untested ownership or +compatibility behavior is correct. + +## Final report + +List findings by severity with file/line, failure scenario, impact, and fix. +Then note validation performed and residual risk. If there are no findings, say +so explicitly and identify any coverage gaps. + +## Stop conditions + +Escalate when the diff is incomplete, generated output lacks its source change, +the downstream contract is unavailable, or required CI logs cannot be obtained. diff --git a/docs/ai/cross-repository.md b/docs/ai/cross-repository.md new file mode 100644 index 0000000..efbbe09 --- /dev/null +++ b/docs/ai/cross-repository.md @@ -0,0 +1,47 @@ +# Cross-repository workflow + +## Purpose + +Coordinate changes across CTraces, its pinned dependencies, and Fluent Bit +without obscuring behavior ownership. + +## When to use + +Use for CFL or fluent-otel-proto updates, public API changes consumed by Fluent +Bit, and bugs first observed in Fluent Bit whose implementation is in CTraces. + +## Investigation + +1. Identify which repository owns the behavior and why. +2. Record dependency direction, pinned revisions, affected public APIs, and + format or ABI implications. +3. Inspect actual Fluent Bit call sites; do not infer consumer lifecycle. +4. Define the landing order and temporary branch or revision strategy. + +## Implementation + +1. Implement and test the fix in the owning repository first. +2. Keep each repository's change in a separate commit and pull request when + practical. +3. Update submodule or vendored revisions explicitly; do not mix dependency + source edits into the consumer repository. +4. Avoid temporary compatibility shims unless landing order requires them and + their removal is tracked. + +## Validation + +Validate the standalone owner first. Then build and test CTraces at the proposed +dependency revision. Finally build and exercise the relevant Fluent Bit +consumer path, including the trace sampling processor when that path is affected. + +## Final report + +Report ownership, repositories and revisions, landing order, API/ABI or format +impact, standalone results, consumer integration results, and remaining rollout +risk. + +## Stop conditions + +Stop when write authority is missing for another repository, a required branch +or revision is unavailable, landing order would break a consumer, or the +downstream behavior cannot be exercised reliably. diff --git a/docs/ai/investigate.md b/docs/ai/investigate.md new file mode 100644 index 0000000..ea57eec --- /dev/null +++ b/docs/ai/investigate.md @@ -0,0 +1,44 @@ +# Investigation workflow + +## Purpose + +Establish evidence about CTraces behavior without silently expanding into an +implementation task. + +## When to use + +Use for bug reports, unclear ownership, compatibility questions, unexplained +test failures, and behavior observed through Fluent Bit. + +## Investigation + +1. Restate the observed and expected behavior; separate facts from assumptions. +2. Identify whether CTraces, CFL, fluent-otel-proto, or Fluent Bit owns it. +3. Start at the relevant public header and trace create, mutate, encode/decode, + and destroy paths through `src/`. +4. Locate tests and recent history for the same interface. +5. Reproduce with the smallest existing test executable or a temporary harness. +6. Inspect allocation failures, partial objects, owner lists, malformed input, + and format boundaries relevant to the path. +7. Compare downstream usage in Fluent Bit when consumer assumptions matter. + +## Implementation + +Do not change production code unless the task also authorizes a fix. Keep any +temporary instrumentation or harness out of the final diff. + +## Validation + +Record the exact reproduction command and distinguish reproduced facts from +source-based inference. + +## Final report + +Report the owning repository, root cause or leading hypothesis, affected APIs +and formats, reproduction evidence, likely fix scope, and unresolved questions. + +## Stop conditions + +Stop and request direction if ownership requires a different repository, the +expected behavior would change a public contract, or reproduction needs data or +environment access that is unavailable. diff --git a/scripts/agent-build.sh b/scripts/agent-build.sh new file mode 100755 index 0000000..5e91b1d --- /dev/null +++ b/scripts/agent-build.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +build_dir=${CTR_BUILD_DIR:-"${repo_root}/build/agent"} + +cmake -S "${repo_root}" -B "${build_dir}" -DCTR_DEV=On -DCTR_TESTS=On +cmake --build "${build_dir}" --parallel diff --git a/scripts/agent-test.sh b/scripts/agent-test.sh new file mode 100755 index 0000000..6f53569 --- /dev/null +++ b/scripts/agent-test.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +build_dir=${CTR_BUILD_DIR:-"${repo_root}/build/agent"} + +if [[ ! -f "${build_dir}/CTestTestfile.cmake" ]]; then + echo "test build not found: run scripts/agent-build.sh first" >&2 + exit 1 +fi + +ctest --test-dir "${build_dir}" --output-on-failure "$@" diff --git a/scripts/agent-verify.sh b/scripts/agent-verify.sh new file mode 100755 index 0000000..7b1454c --- /dev/null +++ b/scripts/agent-verify.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) + +bash -n \ + "${repo_root}/scripts/agent-build.sh" \ + "${repo_root}/scripts/agent-test.sh" \ + "${repo_root}/scripts/agent-verify.sh" + +"${repo_root}/scripts/agent-build.sh" +"${repo_root}/scripts/agent-test.sh"