Waf challenge mode fix difficulty forgery

[buixor]

No description provided.

[github-actions]

@buixor: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind fix
• /kind chore
• /kind dependencies

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions]

@buixor: There are no area labels on this PR. You can add as many areas as you see fit.

• /area agent
• /area local-api
• /area cscli
• /area appsec
• /area security
• /area configuration

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[Copilot]

Pull request overview

This PR hardens and extends the AppSec “challenge mode” (bot-detection) pipeline by introducing a split-bundle design (public static JS + per-epoch obfuscated key module), adding replay protection and secret/keyring handling, and wiring new observability (Prometheus + cscli metrics) around challenge lifecycle and fingerprint mismatch signals. It also adjusts the AppSec acquisition/runtime wiring and adds regression tests (including a postoverflow whitelist stage-skipping case).

Changes:

• Add challenge-runtime building blocks: keyring/secret parsing, per-challenge ticket verification (difficulty MAC binding + replay burn), split static/dynamic JS obfuscation with caching/pools, and related build-time generators.
• Add new challenge lifecycle + fingerprint mismatch metrics and cscli metrics sections/aliases to surface them.
• Update AppSec acquisition/runner flow to support on_challenge, emit challenge events, and add targeted regression/unit tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/protobufs/notifier_grpc.pb.go	Regenerated gRPC stubs (tooling/version header + minor generated-code adjustments).
pkg/parser/postoverflow_whitelist_test.go	Adds regression tests ensuring postoverflow whitelisting isn’t skipped by earlier-stage “no pass” behavior.
pkg/metrics/acquisition_appsec.go	Introduces Prometheus metrics for fingerprint mismatch and challenge lifecycle/infra counters.
pkg/exprhelpers/geoip.go	Adds IPToCountryString + expr wrapper for country-only GeoIP lookups.
pkg/exprhelpers/expr_lib.go	Registers IPToCountry in the expr function library.
pkg/appsec/waf_helpers_test.go	Tests for parsing the optional TTL argument of GrantChallengeCookie.
pkg/appsec/patcher.go	Adds an expr AST visitor to detect challenge-runtime helper usage and force WASM runtime init.
pkg/appsec/patcher_test.go	Tests that config build correctly toggles NeedWASMVM based on hook usage.
pkg/appsec/fingerprint_dump.go	Adds on-disk JSONL fingerprint dumping with label sanitization + per-file locking.
pkg/appsec/event.go	Defines AppSec event sources and constructs WAF vs challenge lifecycle events.
pkg/appsec/event_test.go	Tests for challenge event shaping and channel emission behavior.
pkg/appsec/cookie/cookie.go	Adds an AppSec cookie builder/type and renderer for Set-Cookie output.
pkg/appsec/challenge/ticket.go	Implements ticket nonce/PoW salt handling, difficulty-auth MAC, and fingerprint payload deobfuscation helpers.
pkg/appsec/challenge/static_bundle.go	Adds embedded initial static bundle seeding + optional runtime obfuscation pool refresher.
pkg/appsec/challenge/split_bundle_test.go	Regression/security tests for split-bundle invariants (sentinel survival, key non-leakage, caching, concurrency).
pkg/appsec/challenge/spent_set.go	Adds a replay-protection spent nonce set with atomic check+insert.
pkg/appsec/challenge/spent_set_test.go	Tests for spent-set correctness, concurrency, and expiry behavior.
pkg/appsec/challenge/secret.go	Adds master-secret parsing/validation and random fallback generation.
pkg/appsec/challenge/secret_test.go	Tests secret parsing and runtime rejection of short secrets.
pkg/appsec/challenge/pow-worker.js	Adds the PoW web worker implementation (pure JS SHA-256 loop).
pkg/appsec/challenge/pb/generate.go	Adds go:generate instructions for challenge fingerprint protobuf generation.
pkg/appsec/challenge/obfuscator.go	Adds wazero-based JS obfuscator wrapper configured for real entropy/time.
pkg/appsec/challenge/obfuscation_pools_test.go	Tests default/option behavior for library + crypto obfuscation pools.
pkg/appsec/challenge/keyring_test.go	Adds deterministic/rotation/live-window tests and known vectors for key derivation.
pkg/appsec/challenge/js/README.md	Documents the JS build pipeline and regeneration instructions.
pkg/appsec/challenge/js/obfuscate/obfuscate.js	Obfuscator driver with reservedStrings sentinel and console-output handling for CDP detection.
pkg/appsec/challenge/js/js.go	Embeds the generated fpscanner bundle in Go.
pkg/appsec/challenge/js/generate.go	Adds go:generate steps for bundling/obfuscation/initial bundle generation.
pkg/appsec/challenge/js/fpscanner/src/types.ts	Adds TS types for signals, detections, and fingerprint payload shape.
pkg/appsec/challenge/js/fpscanner/src/signals/worker.ts	Adds worker-context fingerprint signal collection with timeout/cleanup.
pkg/appsec/challenge/js/fpscanner/src/signals/webgpu.ts	Adds WebGPU adapter info signal collection.
pkg/appsec/challenge/js/fpscanner/src/signals/webGL.ts	Adds WebGL vendor/renderer signal collection.
pkg/appsec/challenge/js/fpscanner/src/signals/webdriverWritable.ts	Adds webdriver-writable signal check.
pkg/appsec/challenge/js/fpscanner/src/signals/webdriver.ts	Adds webdriver signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/utils.ts	Adds shared signal constants/utilities and a Firefox detection helper.
pkg/appsec/challenge/js/fpscanner/src/signals/userAgent.ts	Adds user-agent signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/url.ts	Adds page URL signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/toSourceError.ts	Adds toSource error signal collection.
pkg/appsec/challenge/js/fpscanner/src/signals/time.ts	Adds time signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/seleniumProperties.ts	Adds selenium-property presence signal.
pkg/appsec/challenge/js/fpscanner/src/signals/screenResolution.ts	Adds screen/viewport resolution signals.
pkg/appsec/challenge/js/fpscanner/src/signals/plugins.ts	Adds plugin-array consistency/overflow signals.
pkg/appsec/challenge/js/fpscanner/src/signals/playwright.ts	Adds Playwright signal detection.
pkg/appsec/challenge/js/fpscanner/src/signals/platform.ts	Adds platform signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/nonce.ts	Adds nonce signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/navigatorPropertyDescriptors.ts	Adds navigator property-descriptor bitmask signal.
pkg/appsec/challenge/js/fpscanner/src/signals/multimediaDevices.ts	Adds multimedia device enumeration signals.
pkg/appsec/challenge/js/fpscanner/src/signals/memory.ts	Adds device-memory signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/mediaQueries.ts	Adds media query capability signals.
pkg/appsec/challenge/js/fpscanner/src/signals/mediaCodecs.ts	Adds codec support hashing + RTC capability hashing signals.
pkg/appsec/challenge/js/fpscanner/src/signals/maths.ts	Adds math-function fingerprinting hash.
pkg/appsec/challenge/js/fpscanner/src/signals/languages.ts	Adds language(s) signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/internationalization.ts	Adds locale/timezone signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/iframe.ts	Adds iframe-context signal collection with cleanup.
pkg/appsec/challenge/js/fpscanner/src/signals/highEntropyValues.ts	Adds UA high-entropy values collection.
pkg/appsec/challenge/js/fpscanner/src/signals/etsl.ts	Adds ETSL (eval-toString length) signal.
pkg/appsec/challenge/js/fpscanner/src/signals/cpuCount.ts	Adds CPU count signal getter.
pkg/appsec/challenge/js/fpscanner/src/signals/cdp.ts	Adds CDP signal detection logic (overrides prepareStackTrace).
pkg/appsec/challenge/js/fpscanner/src/signals/canvas.ts	Adds canvas fingerprint + modified-canvas signal.
pkg/appsec/challenge/js/fpscanner/src/signals/browserFeatures.ts	Adds browser feature bitmask signal.
pkg/appsec/challenge/js/fpscanner/src/signals/browserExtensions.ts	Adds extension presence signals + bitmask.
pkg/appsec/challenge/js/fpscanner/src/globals.d.ts	Declares the build-time __FP_ENCRYPTION_KEY__ constant for TS typing.
pkg/appsec/challenge/js/fpscanner/src/detections/hasWebdriverWritable.ts	Adds detection rule helper for webdriver-writable.
pkg/appsec/challenge/js/fpscanner/src/detections/hasWebdriverWorker.ts	Adds detection rule helper for webdriver in worker context.
pkg/appsec/challenge/js/fpscanner/src/detections/hasWebdriverIframe.ts	Adds detection rule helper for webdriver in iframe context.
pkg/appsec/challenge/js/fpscanner/src/detections/hasWebdriver.ts	Adds detection rule helper for webdriver.
pkg/appsec/challenge/js/fpscanner/src/detections/hasUTCTimezone.ts	Adds detection rule helper for UTC timezone.
pkg/appsec/challenge/js/fpscanner/src/detections/hasSwiftshaderRenderer.ts	Adds detection rule helper for SwiftShader renderer.
pkg/appsec/challenge/js/fpscanner/src/detections/hasSeleniumProperty.ts	Adds detection rule helper for selenium property presence.
pkg/appsec/challenge/js/fpscanner/src/detections/hasPlaywright.ts	Adds detection rule helper for Playwright.
pkg/appsec/challenge/js/fpscanner/src/detections/hasPlatformMismatch.ts	Adds detection rule helper for platform mismatch checks.
pkg/appsec/challenge/js/fpscanner/src/detections/hasMissingChromeObject.ts	Adds detection rule helper for missing Chrome object heuristic.
pkg/appsec/challenge/js/fpscanner/src/detections/hasMismatchWebGLInWorker.ts	Adds detection rule helper for WebGL mismatch between main/worker.
pkg/appsec/challenge/js/fpscanner/src/detections/hasMismatchPlatformWorker.ts	Adds detection rule helper for platform mismatch vs worker.
pkg/appsec/challenge/js/fpscanner/src/detections/hasMismatchPlatformIframe.ts	Adds detection rule helper for platform mismatch vs iframe.
pkg/appsec/challenge/js/fpscanner/src/detections/hasMismatchLanguages.ts	Adds detection rule helper for language mismatch.
pkg/appsec/challenge/js/fpscanner/src/detections/hasInconsistentEtsl.ts	Adds detection rule helper for ETSL inconsistencies.
pkg/appsec/challenge/js/fpscanner/src/detections/hasImpossibleDeviceMemory.ts	Adds detection rule helper for impossible device memory values.
pkg/appsec/challenge/js/fpscanner/src/detections/hasHighCPUCount.ts	Adds detection rule helper for high CPU count values.
pkg/appsec/challenge/js/fpscanner/src/detections/hasHeadlessChromeScreenResolution.ts	Adds detection rule helper for headless screen resolution heuristic.
pkg/appsec/challenge/js/fpscanner/src/detections/hasGPUMismatch.ts	Adds detection rule helper for GPU mismatch heuristic.
pkg/appsec/challenge/js/fpscanner/src/detections/hasContextMismatch.ts	Adds helper for context mismatch (indicator).
pkg/appsec/challenge/js/fpscanner/src/detections/hasCDP.ts	Adds detection rule helper for CDP signal.
pkg/appsec/challenge/js/fpscanner/src/detections/hasBotUserAgent.ts	Adds detection rule helper for bot/headless UA pattern.
pkg/appsec/challenge/js/fpscanner/src/crypto-helpers.ts	Adds simple XOR-based string obfuscation helpers.
pkg/appsec/challenge/js/cmd/obfuscate/main.go	Build-time tool to bundle the obfuscator driver and compile to WASM via javy.
pkg/appsec/challenge/js/cmd/initialbundle/main.go	Build-time tool to generate the baked-in initial obfuscated bundle.
pkg/appsec/challenge/js/cmd/bundle/main.go	Build-time tool to bundle challenge.js + TS sources via esbuild.
pkg/appsec/challenge/initial_bundle_test.go	Ensures challenge runtime startup stays within a budget and baked bundle seeding works.
pkg/appsec/challenge/grant_redirect.html.tmpl	Adds redirect HTML template used by challenge flows.
pkg/appsec/challenge/fingerprint_mismatch.go	Implements atomic mismatch checks and aggregates mismatch reports.
pkg/appsec/challenge/fingerprint_mismatch_data.go	Defines stable reason keys, severities, and detection tables.
pkg/appsec/challenge/fingerprint_helpers_test.go	Adds tests for fingerprint helpers/category aggregation.
pkg/appsec/challenge/dynamic_module.js.tmpl	Adds the per-epoch dynamic key module template.
pkg/appsec/challenge/dynamic_module.go	Implements dynamic module generation, caching, pre-warming, and eviction.
pkg/appsec/challenge/config.go	Adds YAML-facing challenge config + merge + option building.
pkg/appsec/appsec_config_test.go	Extends config tests for on_challenge and the challenge: config block merge/strictness.
pkg/acquisition/modules/appsec/utils.go	Removes duplicated EventFromRequest in acquisition module (now uses pkg/appsec).
pkg/acquisition/modules/appsec/testhelpers_test.go	Adds shared challenge-runtime test helper for appsec acquisition tests.
pkg/acquisition/modules/appsec/run.go	Wires output channel for challenge events and closes challenge runtime on shutdown.
pkg/acquisition/modules/appsec/metrics.go	Registers the new challenge/fingerprint mismatch metrics for the appsec source.
pkg/acquisition/modules/appsec/init.go	Reuses appsec.ModuleName constant for module naming.
pkg/acquisition/modules/appsec/config.go	Builds/initializes challenge runtime when needed and shares AppsecRuntime across runners.
pkg/acquisition/modules/appsec/appsec_test.go	Updates acquisition tests to support on_challenge and shared challenge runtime wiring.
pkg/acquisition/modules/appsec/appsec_runner.go	Adds allowlist bypass early, wires on_challenge processing, and switches to pkg/appsec EventFromRequest.
Makefile	Adds generate-challenge-js target for regenerating committed JS/WASM artifacts.
go.sum	Adds checksums for new Go dependencies (esbuild, etc.).
go.mod	Adds esbuild and promotes wazero to a direct dependency (plus indirect updates).
cmd/crowdsec-cli/climetrics/statappsecengine.go	Extends appsec engine table with challenge lifecycle columns.
cmd/crowdsec-cli/climetrics/statappsecchallengeinfra.go	Adds cscli metrics section for challenge runtime infrastructure counters.
cmd/crowdsec-cli/climetrics/statappsecchallenge.go	Adds cscli metrics section for challenge funnel + reason breakdown.
cmd/crowdsec-cli/climetrics/show.go	Adds new appsec-challenge and bot-detection alias expansions.
.golangci.yml	Adjusts gocritic exclusions for UnmarshalJSON pointer receivers.
.gitignore	Ignores local agent/tooling directories.
.github/workflows/go-tests.yml	Updates CI go generate setup: newer protoc + installs javy, and refreshes generator tool versions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[buixor]

/kind fix
/area appsec

[codecov]

Codecov Report

❌ Patch coverage is 0% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.90%. Comparing base (8a98a44) to head (5abb976).

Files with missing lines	Patch %	Lines
pkg/appsec/challenge/challenge.go	0.00%	23 Missing ⚠️
pkg/appsec/challenge/ticket.go	0.00%	4 Missing ⚠️
pkg/appsec/appsec.go	0.00%	2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (8a98a44) and HEAD (5abb976). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (8a98a44)	HEAD (5abb976)
unit-windows	1	0
unit-linux	1	0

Additional details and impacted files

@@                   Coverage Diff                   @@
##           waf-challenge-mode    #4510       +/-   ##
=======================================================
- Coverage               63.55%   41.90%   -21.65%     
=======================================================
  Files                     504      478       -26     
  Lines                   38349    37026     -1323     
=======================================================
- Hits                    24371    15515     -8856     
- Misses                  11648    19610     +7962     
+ Partials                 2330     1901      -429     

Flag	Coverage Δ
bats	41.90% <0.00%> (-0.01%)	⬇️
unit-linux	?
unit-windows	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.