(Openinference Migration: Langchain): Capture multimodal image content (OpenAI image_url and Anthropic image blocks) as Blob/Uri message parts.

[rads-1996]

Description

Part of the langchain migration PRs - #272

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. List any relevant details for your test
configuration.

• uv run tox -e py312-test-instrumentation-genai-langchain -- -q
• uv run tox -e py312-test-instrumentation-genai-langchain-conformance -- -q
• uv run --python 3.12 tox -e lint-instrumentation-genai-langchain

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

• Followed the style guidelines of this project
• Changelog updated if the change requires an entry
• Unit tests added
• Documentation updated

[opentelemetry-pr-dashboard]

Pull request dashboard status

Waiting on the author · refreshed 2026-08-04 22:32 UTC

Respond to 1 review item (e.g. link a commit, explain why not, ask a follow-up):

• Top-level threads: 1

Status above doesn't look right?

• Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
• Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
• Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

[Copilot]

Pull request overview

Adds multimodal image support to the LangChain GenAI instrumentation by converting OpenAI image_url and Anthropic image blocks into Blob / Uri message parts, and extends the unit test suite to validate the new parsing behavior.

Changes:

• Add multimodal image parsing helpers (_media_part, _image_from_url) to convert LangChain image blocks into Blob/Uri parts.
• Extend callback-handler tests to cover OpenAI and Anthropic image content shapes (including data URIs and base64 sources).
• Add a changelog fragment documenting the new capability.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py	Adds image block parsing and base64 decoding to emit Blob/Uri message parts.
instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py	Adds unit tests for data-URI, HTTP-URI, and Anthropic image/source parsing into message parts.
instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/296.added	Documents the new multimodal image capture support.

[AgentGymLeader]

@rads-1996 One thing I wanted to check on ordering. In on_chat_model_start, to_input_messages() runs before anything consults TelemetryHandler.should_capture_content(), so with the default NO_CONTENT the data URI and Anthropic base64 images still get decoded to bytes via _decode_base64.

That means a config that doesn't record content still copies large images. If that isn't intended, either skip the media decode when capture is off, or add a test asserting _decode_base64 isn't called under NO_CONTENT.

If the parse is meant to run unconditionally so the completion hook has something to work with, that's fine, I just wanted to confirm that's the assumption.

[rads-1996]

@rads-1996 One thing I wanted to check on ordering. In on_chat_model_start, to_input_messages() runs before anything consults TelemetryHandler.should_capture_content(), so with the default NO_CONTENT the data URI and Anthropic base64 images still get decoded to bytes via _decode_base64.

That means a config that doesn't record content still copies large images. If that isn't intended, either skip the media decode when capture is off, or add a test asserting _decode_base64 isn't called under NO_CONTENT.

If the parse is meant to run unconditionally so the completion hook has something to work with, that's fine, I just wanted to confirm that's the assumption.

@AgentGymLeader Thanks for the callout. I have gated decoding and will skipped when the content capture mode is NO_CONTENT.