Skip to content

example(mcap): Add live replay and use CloudXRLauncher#695

Open
jiwenc-nv wants to merge 2 commits into
NVIDIA:mainfrom
jiwenc-nv:jiwenc/live-pose-viz
Open

example(mcap): Add live replay and use CloudXRLauncher#695
jiwenc-nv wants to merge 2 commits into
NVIDIA:mainfrom
jiwenc-nv:jiwenc/live-pose-viz

Conversation

@jiwenc-nv

@jiwenc-nv jiwenc-nv commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

Add live replay and use CloudXRLauncher for the mcap record / replay examples.

Fixes #(issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Testing

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix/feature works (or explained why not)
  • I have signed off all my commits (git commit -s) per the DCO

Summary by CodeRabbit

  • New Features

    • Added three new example scripts for real-time visualization of hand tracking, controller tracking, and full-body skeleton poses
    • Added option to automatically launch CloudXR runtime when recording data
    • Added CloudXR environment configuration file with preset settings

  • Refactor

    • Consolidated visualization components into shared utilities for consistency across examples

@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

📝 Docs preview is not auto-deployed for fork PRs.

A maintainer with write access to NVIDIA/IsaacTeleop can deploy a preview by
commenting /preview-docs on this PR. Once deployed, the preview
will live at:

https://nvidia.github.io/IsaacTeleop/preview/pr-695/

jiwenc-nv added 2 commits June 23, 2026 18:02
@jiwenc-nv
mcap examples: add live pose viz + refactor CloudXRLauncher + consoli…
7de1b99 
…date viz classes

- Add live_hand.py, live_controller.py, live_full_body.py: real-time viser
  visualization of XR pose data using CloudXRLauncher + TeleopSession (no MCAP)
- Refactor record_* scripts to use CloudXRLauncher so the CloudXR runtime
  starts automatically; add --accept-eula / --install-dir args
- Move HandViz, ControllerViz, FullBodyViz and shared color constants into
  common.py; replay_* scripts now import from there instead of duplicating

Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
@jiwenc-nv
mcap examples: add --env-file/--launch-cloudxr-runtime and fix missed…
edbbd8c 
…-frame counting

- Add --env-file (default: default.env) to pass a KEY=value override file
  to CloudXRLauncher; ship default.env with sensible defaults for Quest 3
- Add --launch-cloudxr-runtime / --no-launch-cloudxr-runtime
  (BooleanOptionalAction, default true) to optionally skip CloudXRLauncher
  and connect to a system OpenXR runtime directly
- Report missed frames per interval in live_hand/controller
- Fix missed-frame counting to accumulate per-frame gaps correctly

Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
@jiwenc-nv jiwenc-nv force-pushed the jiwenc/live-pose-viz branch from b178b8f to edbbd8c Compare June 23, 2026 18:09
@coderabbitai

coderabbitai Bot commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR centralizes all Viser visualization primitives — HandViz, ControllerViz, FullBodyViz, color constants, and geometry helpers — into common.py, removing duplicate implementations previously scattered across the three replay scripts. Three new live example scripts (live_hand.py, live_controller.py, live_full_body.py) are added, each connecting a TeleopSession to the shared viz classes at ~60 Hz. All record scripts are migrated from positional argv parsing to argparse, gaining --accept-eula, --install-dir, --env-file, and --launch-cloudxr-runtime flags backed by an optional CloudXRLauncher context manager. A default.env file is introduced with CloudXR environment variable defaults.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 34.62% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main changes: adding live replay capabilities and integrating CloudXRLauncher into the MCAP example scripts.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

aristarkhovNV
aristarkhovNV reviewed Jun 23, 2026
View reviewed changes
Comment thread examples/mcap_record_replay/python/record_controller.py
Comment thread examples/mcap_record_replay/python/live_hand.py
Comment on lines +66 to +71
server = viser.ViserServer(host=args.host, port=args.port)
server.scene.set_up_direction("+y")
server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1)

viz_left = HandViz(server, "hand_left", LEFT_COLOR)
viz_right = HandViz(server, "hand_right", RIGHT_COLOR)

@aristarkhovNV aristarkhovNV Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start the viz as late as possible after all checks have passed and the session is live - to avoid a misleading "viser running" when something is misconfigured

Comment thread examples/mcap_record_replay/python/live_hand.py
Comment thread examples/mcap_record_replay/python/live_hand.py
nv-jakob
nv-jakob reviewed Jun 24, 2026
View reviewed changes
Comment thread examples/mcap_record_replay/python/live_hand.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nv-jakob nv-jakob nv-jakob left review comments
@aristarkhovNV aristarkhovNV aristarkhovNV approved these changes
@nv-mhaselton nv-mhaselton Awaiting requested review from nv-mhaselton

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jiwenc-nv @aristarkhovNV @nv-jakob