Skip to content

inspector,http: support builtin http request bodies#64369

Open
GrinZero wants to merge 2 commits into
nodejs:mainfrom
GrinZero:feat/inspector-http-body-tracking
Open

inspector,http: support builtin http request bodies#64369
GrinZero wants to merge 2 commits into
nodejs:mainfrom
GrinZero:feat/inspector-http-body-tracking

Conversation

@GrinZero

@GrinZero GrinZero commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR supersedes #62915 with the same feature work rebased onto current main.

It adds builtin http/https request-body support to network inspection, so Network.getRequestPostData can return text request bodies while preserving the existing rejection behavior for binary request bodies.

It also moves builtin http response-body tracking to a raw-byte hook before IncomingMessage decoding, so response inspection remains correct even when user code calls response.setEncoding(...).

In addition, this PR lets Network.requestWillBeSent carry JS-provided initiator data captured at request construction time. The C++ inspector side validates structured initiator.stack objects against the inspector protocol schema before forwarding them to DevTools.

Problem

Builtin http/https network inspection currently emits request and response lifecycle events, but the builtin client path does not expose request-body bytes to the inspector. As a result, Network.getRequestPostData cannot return POST data for builtin http/https requests.

There are two related gaps:

  • IncomingMessage 'data' events are not a stable raw-byte source. If user code calls response.setEncoding('utf8'), chunks observed through userland become strings, while the inspector protocol expects byte-oriented payloads.
  • Delaying requestWillBeSent until request-body state is known means a C++ stack captured at emission time would point into Node.js internals rather than the user call site. The initiator needs to be captured when the request is created and validated before being emitted.

Approach

  1. Reuse the existing Network.dataSent(...) -> NetworkAgent::getRequestPostData(...) buffering pipeline instead of changing the CDP schema.
  2. Add builtin HTTP diagnostics events for request body chunks and request body completion:
    • http.client.request.bodyChunkSent
    • http.client.request.bodySent
  3. Capture builtin HTTP response body bytes through a parser-side raw-byte diagnostics event before stream decoding:
    • http.client.response.bodyChunkReceived
  4. Defer requestWillBeSent until body state is known, so hasPostData is accurate for both body and body-less requests.
  5. Capture the initiator stack in JS at request creation time, pass it through the inspector binding, and validate initiator.stack with generated inspector protocol schema support in C++.
  6. Document the new diagnostics channels and message shapes.

Behavior

After this change:

  • builtin http and https POST requests with UTF-8 text bodies are available through Network.getRequestPostData
  • binary request bodies still reject with the existing inspector error behavior
  • builtin http response inspection continues to work even if user code calls response.setEncoding('utf8')
  • Network.requestWillBeSent reports accurate hasPostData
  • Network.requestWillBeSent can carry JS-captured initiator stack data from the request construction site
  • malformed initiator stack input is rejected by C++ schema validation

Tests

This PR adds and extends coverage in:

  • test/parallel/test-diagnostics-channel-http.js
  • test/parallel/test-inspector-network-http.js
  • test/parallel/test-inspector-emit-protocol-event.js
  • test/parallel/test-inspector-emit-protocol-event-errors.js

The updated tests cover:

  • request body chunk and request body finished diagnostics events
  • response body raw-byte diagnostics events
  • text request bodies split across write() and end()
  • binary request bodies
  • http and https Network.getRequestPostData
  • binary request-body rejection semantics
  • response inspection when the client calls response.setEncoding('utf8')
  • accurate hasPostData for body-less requests
  • JS-provided initiator stack parsing and forwarding
  • malformed initiator stack error handling

Verification

Validated with the targeted test suite:

python3 tools/test.py \
  parallel/test-diagnostics-channel-http \
  parallel/test-inspector-network-http \
  parallel/test-inspector-emit-protocol-event \
  parallel/test-inspector-emit-protocol-event-errors

Also validated manually with an end-to-end inspector run:

./out/Release/node --inspect --experimental-network-inspection ./test.js

Refs

@GrinZero

GrinZero commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

e2e same like: #62915 (comment)
image

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http
  • @nodejs/inspector
  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jul 8, 2026
@GrinZero GrinZero force-pushed the feat/inspector-http-body-tracking branch from cce325b to 7865a8d Compare July 8, 2026 17:13
Signed-off-by: GrinZero <774933704@qq.com>
@GrinZero GrinZero force-pushed the feat/inspector-http-body-tracking branch from 7865a8d to 4bd2033 Compare July 8, 2026 17:41
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.73463% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.26%. Comparing base (f4c85d5) to head (929ddfe).
⚠️ Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
src/inspector/network_agent.cc 93.51% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #64369      +/-   ##
==========================================
+ Coverage   90.24%   90.26%   +0.01%     
==========================================
  Files         741      741              
  Lines      240976   241495     +519     
  Branches    45408    45495      +87     
==========================================
+ Hits       217477   217979     +502     
- Misses      15070    15085      +15     
- Partials     8429     8431       +2     
Files with missing lines Coverage Δ
lib/_http_client.js 97.63% <100.00%> (+0.04%) ⬆️
lib/_http_common.js 99.44% <100.00%> (+0.02%) ⬆️
lib/_http_outgoing.js 97.65% <100.00%> (+0.03%) ⬆️
lib/internal/http.js 92.33% <100.00%> (+0.05%) ⬆️
lib/internal/inspector/network_http.js 97.57% <100.00%> (+1.87%) ⬆️
src/inspector/network_agent.cc 89.37% <93.51%> (+1.34%) ⬆️

... and 44 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: GrinZero <774933704@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants