### Describe the bug

Two `Camera` sensors with equal OVRTX renderer configs share the same native camera binding and render product. With the Kuka duo-camera task's default `update_latest_camera_pose=False`, the base and wrist sensors return almost the same stale view despite having different camera prims and poses.

**Reproduced on a real RTX 6000 Ada GPU with the unmodified upstream task and renderer, not just mocks or source inspection.** No renderer patches were used.

Updating the base camera's pose also changes the image returned by the wrist sensor when its pose refresh is disabled. **GPU-verified task-level fix: setting `update_latest_camera_pose=True` for both cameras before construction restores correct viewpoints during normal environment stepping, including a moving wrist camera.** See the flag validation below. Independent native views are not required to fix the incorrect images in this tested task; their potential performance benefit is a separate, unmeasured question.

### Image evidence

Untouched native `rgb64` captures from environment 0; displayed larger for readability. Physics was held still during these captures. Full captures for all four environments and metrics are in [the evidence bundle](https://github.com/nblauch/IsaacLab/tree/712d2424038bfe49db658f3e897ca9f709deaa5d).

| Condition | Base sensor | Wrist sensor |
| --- | --- | --- |
| Reference defaults: both return the same stale view | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/default-base-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/default-wrist-env0.png" width="256" /> |
| Refresh each sensor's pose: distinct viewpoints | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/refresh-base-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/refresh-wrist-env0.png" width="256" /> |
| Publish base pose, then capture wrist without refreshing: wrist now returns base view | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/base-published-base-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/images/base-published-wrist-env0.png" width="256" /> |

Across all four environments, mean absolute pixel-channel difference on the 0–255 scale:

- Default base vs wrist: **0.137**.
- Independently refreshed base vs wrist: **84.908**.
- Wrist after base publication vs base view: **0.961**; vs its own refreshed wrist view: **84.922**.

The default images are nearly identical, not bit-identical; separate renders can differ because of temporal rendering.

### Verified fix using the existing flag

Set both camera flags before `gym.make()`:

```python
cfg.scene.base_camera.update_latest_camera_pose = True
cfg.scene.wrist_camera.update_latest_camera_pose = True
```

A second native GPU run validated normal `env.step()` captures over five steps, four environments, with nonzero joint actions and a moving wrist camera. After each step, physics is held still and each camera is settled independently for a same-state reference. **All 40 normal camera images are closer to their own reference than to the other camera's view.** Mean absolute channel error is 2.57 for base and 3.58 for wrist against their own references, versus 85.81 and 86.12 against the other view. Both runs exited successfully with no competing compute process on the selected GPU.

| Pose refresh enabled from initialization; normal-step output | Independently settled same-state reference |
| --- | --- |
| Base: <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/flag-images/step4-normal-base-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/flag-images/step4-reference-base-env0.png" width="256" /> |
| Wrist: <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/flag-images/step4-normal-wrist-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/flag-images/step4-reference-wrist-env0.png" width="256" /> |

[Flag-validation script](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/validate_flag_gpu.py), [all per-image metrics](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/flag-results.json). This establishes a working fix for the tested task's default configuration; its measured performance cost is reported below. The renderer still shares one native view and switches its pose for each capture.

### Performance impact at 4 096 environments

Changing only both cameras' `update_latest_camera_pose` flags from `False` to `True` increased mean full environment step time by **28.3 ms (18.5 %)** and reduced environment FPS by **15.6 %** on the RTX 6000 Ada.

| Both camera flags | Mean step time | Environment FPS |
| --- | ---: | ---: |
| `False` — incorrect/stale views | 152.4 ms | 26 875 |
| `True` — correct views | 180.7 ms | 22 671 |

Task: `Isaac-Reorient-KukaAllegro-Camera`, presets `newton_mjwarp,ovrtx,cube,duo_camera,rgb64`; same tested commit and pinned dependencies listed below. Each eye requests RGB at 64 by 64. Physics dt is 1/120 s, decimation and render interval are both 4. Newton CUDA graph capture remains enabled; external ovstage remains disabled.

Measurement: two fresh-process runs per setting, in order **off → on → on → off**. Each run uses 30 warm-up steps and 80 measured steps. Timings cover complete `env.step()` calls, CUDA-synchronized before and after each step, with no policy/training. Environment FPS is the number of environment transitions divided by total measured step time; aggregate rows pool the 160 measured steps per setting.

| Run order | Mean step time | Environment FPS | Terminated environments during measurement |
| --- | ---: | ---: | ---: |
| Off A | 151.27 ms | 27 077 | 75 |
| On A | 179.84 ms | 22 776 | 74 |
| On B | 181.50 ms | 22 567 | 74 |
| Off B | 153.55 ms | 26 676 | 73 |

All runs used environment seed 42 and the same pre-generated uniform random actions in [-1, 1], from a separate action generator seeded with 12345. Action arrays and initial joint states had identical hashes across runs. Later physics states were not bitwise identical, including repeated runs with the same flag setting; termination counts were comparable as shown above, with no timeouts. Saved configurations differed only in the two flags and native log output paths. All runs exited successfully, and NVML monitoring found no competing compute process on the selected GPU.

**Interpretation:** this measures the total cost of enabling correct camera captures, including rendering different viewpoints. It does not isolate pose-upload cost, temporal-history effects, or the potential benefit of independent native camera views. The flags-off result is not a valid correct-duo-camera throughput baseline.

### IsaacRTX control: default flag does not reproduce the bug

**GPU-validated on the same pinned IsaacLab revision with `newton_mjwarp,isaacsim_rtx,cube,duo_camera,rgb64`: both flags remain at their default `False` during normal stepping, and the base/wrist images are correct and distinct.** This control uses Isaac Sim 6.1.0.0 / Kit; the OVRTX reproduction above remains kit-free.

Four environments, five capture points after ten normal steps each (50 total), with nonzero joint actions and a moving wrist camera. At each point, normal outputs are copied before any diagnostic changes. Physics is then frozen, both flags enabled, and rendering pumped for 24 reference frames. Joint positions and physics-step count are asserted unchanged during the reference captures. Both flags return to `False` before the next normal steps. **All 40 normal images are closer to their own same-state reference than to the other camera's view.** No competing compute process was observed.

| Camera | Mean absolute channel error vs own reference | Vs other camera reference |
| --- | ---: | ---: |
| Base | 2.24 | 86.53 |
| Wrist | 2.54 | 86.97 |

Untouched environment-zero images from the first capture, after ten normal steps:

| IsaacRTX, default `False` | Same-state pose-refreshed reference |
| --- | --- |
| Base: <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/step0-normal-base-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/step0-reference-base-env0.png" width="256" /> |
| Wrist: <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/step0-normal-wrist-env0.png" width="256" /> | <img src="https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/step0-reference-wrist-env0.png" width="256" /> |

Runtime diagnostics show a shared Python renderer but **separate render products for base and wrist**. [`IsaacRtxRenderer.update_camera`](https://github.com/isaac-sim/IsaacLab/blob/9adf4831384dbe9320de72e9b7809c0d87252b9a/source/isaaclab_physx/isaaclab_physx/renderers/isaac_rtx_renderer.py#L543) is a no-op: rendering uses the camera prims directly. With the flag disabled, cached `CameraData` pose metadata becomes stale, but the rendered wrist view still tracks its current pose. The observed OVRTX image-routing failure is therefore not reproduced by this IsaacRTX control.

[Validation script](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/validate_isaacrtx_default_gpu.py), [full image metrics](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx/results.json), [capture bundle](https://github.com/nblauch/IsaacLab/tree/712d2424038bfe49db658f3e897ca9f709deaa5d/isaacrtx).

### Renderer timing comparison at 4 096 environments

Same task, Newton/MJWarp, GPU, actions and initial joint states, RGB64 duo cameras, and synchronized full-step timing protocol as above. Non-renderer configuration fields match when the flags match.

| Renderer | Both pose flags | Mean step time | Environment FPS | Included fresh-process runs |
| --- | --- | ---: | ---: | ---: |
| IsaacRTX | Default `False` | 424.3 ms | 9 654 | 1 |
| IsaacRTX | `True` | 427.4 ms | 9 584 | 2 |
| OVRTX | `True`, corrected views | 180.7 ms | 22 671 | 2 |

The two IsaacRTX flags-on runs measured 428.02 and 426.75 ms; the uncontended flags-off run measured 424.27 ms. Each run includes 30 warm-up and 80 measured steps. A fourth IsaacRTX run was excluded because monitoring detected another compute process; it contributes to none of these aggregates. All included runs exited successfully without detected competing compute processes.

Corrected OVRTX is **2.35× faster than the measured default IsaacRTX baseline** in this full-environment-step benchmark. This is a comparison of upstream renderer defaults, **not matched image-quality settings**: IsaacRTX runtime settings report DLSS Quality, while the pinned OVRTX schema defaults to DLSS Auto. OVRTX's effective internal render resolution was not measured. These results do not isolate renderer-only time or the cost of pose publication. IsaacRTX's small apparent flag difference should not be treated as a precise isolated overhead estimate with only one uncontended flags-off run.

[Benchmark script](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/benchmark_pose_flag.py), [comparison metadata and aggregates](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/performance/renderer-comparison-4096.json), [all included per-step samples and GPU checks](https://github.com/nblauch/IsaacLab/tree/712d2424038bfe49db658f3e897ca9f709deaa5d/performance).

### Steps to reproduce

1. Use IsaacLab commit `9adf4831384dbe9320de72e9b7809c0d87252b9a` and its pinned OVRTX/Newton dependencies.
2. Download [validate_gpu.py](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/validate_gpu.py), and select an idle GPU UUID:

```bash
export CUDA_DEVICE_ORDER=PCI_BUS_ID
export CUDA_VISIBLE_DEVICES=GPU-YOUR-IDLE-GPU-UUID
export ISAAC_LAB_OVRTX_USE_OVSTAGE=0
export OVRTX_SKIP_USD_CHECK=1
python validate_gpu.py --output ./captures --visualizer none
```

The script resolves `Isaac-Reorient-KukaAllegro-Camera` with `presets=newton_mjwarp,ovrtx,cube,duo_camera,rgb64`, uses four environments and seed 42, runs five zero-action environment steps, then stops advancing physics. It captures each condition after 24 camera-only updates. `OVRTX_SKIP_USD_CHECK=1` was used for this pip OpenUSD/pinned SDK environment.

The control only changes the public `CameraCfg.update_latest_camera_pose` flag. Renderer internals are read for diagnostics, not modified. [Full measured results](https://raw.githubusercontent.com/nblauch/IsaacLab/712d2424038bfe49db658f3e897ca9f709deaa5d/results.json).

Expected: the base sensor renders the base camera prim and the wrist sensor renders the wrist camera prim. Updating one camera must not change the viewpoint returned by the other.

Actual native diagnostics:

```json
{
  "shared_renderer": true,
  "native_render_products": ["/Render/RenderProduct"],
  "native_camera_relative_path": "Camera"
}
```

Sharing the renderer/scene is appropriate; sharing the bound view across distinct sensors is the issue.

### System Info

- GPU-tested IsaacLab commit: `9adf4831384dbe9320de72e9b7809c0d87252b9a`, clean checkout.
- Upstream develop `487b5cac4a2a94ddd60070b3de21a16904ef0679` has byte-for-byte identical `ovrtx_renderer.py`; source-checked only. Its camera routing is unchanged.
- OVRTX reproduction: Isaac Sim/Kit not launched; kit-free Newton + OVRTX. Separate IsaacRTX control: Isaac Sim 6.1.0.0 / Kit launched.
- OVRTX: `0.4.1.364340`; ovstage: `0.1.1.355824`, external ovstage path disabled.
- Newton: `1.6.0.dev0`, commit `24bd863528d6b91137408930d0fbe8fa216ad962`.
- Warp: `1.16.0`; PyTorch: `2.11.0+cu128`.
- OS: Ubuntu 24.04.4 LTS; Python 3.12.
- GPU: NVIDIA RTX 6000 Ada Generation; driver `580.173.02`.
- Run exited successfully. NVML monitoring found no other compute process on the selected GPU.

### Additional context

Relevant code at the tested revision:

- [`RenderContext.get_renderer`](https://github.com/isaac-sim/IsaacLab/blob/9adf4831384dbe9320de72e9b7809c0d87252b9a/source/isaaclab/isaaclab/renderers/render_context.py#L106) reuses a backend for equal configs.
- [`OVRTXRenderer.create_render_data`](https://github.com/isaac-sim/IsaacLab/blob/9adf4831384dbe9320de72e9b7809c0d87252b9a/source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py#L909) initializes native camera/render-product state only for the first spec.
- [`_update_camera_legacy`](https://github.com/isaac-sim/IsaacLab/blob/9adf4831384dbe9320de72e9b7809c0d87252b9a/source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py#L1066) writes one shared camera binding regardless of `render_data`.
- [`_render_legacy`](https://github.com/isaac-sim/IsaacLab/blob/9adf4831384dbe9320de72e9b7809c0d87252b9a/source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py#L1502) extracts the first shared render product for either sensor.
- The task's base and wrist camera configs inherit `update_latest_camera_pose=False`.

The optional external ovstage path has similar shared state in source, but **this GPU reproduction tested only the default renderer-owned path**.

The comparison above measures the total performance impact of enabling both pose-refresh flags. It does not establish how much of that cost comes from pose publication, rendering the corrected viewpoints, or temporal-history effects.

### Checklist

- [x] I have checked for similar issues in the repo.
- [x] The reproduction exercises IsaacLab's adapter without running Isaac Sim/Kit.

### Acceptance Criteria

- [ ] The default Kuka duo-camera configuration returns the correct view for each sensor with OVRTX (enabling both existing pose-refresh flags fixes the tested task).
- [ ] Document or enforce the required per-camera pose publication when multiple sensors share a native binding, so the default pose-cache option cannot silently return the wrong view.
- [ ] The Kuka duo-camera task returns correct base/wrist views, including tracking the articulation-mounted wrist camera.
- [ ] Add a multi-camera regression test that checks rendered viewpoints, not only output shapes or separate Python output buffers.


[Bug Report] OVRTX Kuka duo-camera defaults return the same stale view; enabling pose updates fixes captures

### 🐛 Bug Report

Joint effort targets are **silently not applied** by the PhysX backend on the isaacsim 6.0.1.0 stack: a constant effort command produces **zero** cart motion. The Isaac Lab-side buffers and actuator bookkeeping are all correct — the effort reaches `root_view.set_dof_actuation_forces()` and is then ignored by the simulation. The same test on the **Newton (MJWarp) backend works correctly**, so effort-controlled tasks (e.g. `Isaac-Cartpole-Direct-v0`) are untrainable on PhysX while appearing to run normally.

### Environment

- **OS:** Windows 11 Home (10.0.26200)
- **Python:** 3.12 (standalone venv)
- **isaacsim:** 6.0.1.0 (pip)
- **isaaclab:** 6.1.14 (source checkout, commit `fff603eafc6b74264a5261cc0183d6a65390d78`, tag `v3.0.0-beta2.patch1`)
- **isaaclab_physx:** 1.1.3 (source, same checkout)
- **torch:** 2.11.0+cu128
- **GPU:** NVIDIA GeForce RTX 4060 Laptop GPU
- **skrl:** 2.1.0 (used for the training-impact test below)

### Steps to reproduce

Run the attached `physx_effort_repro.py` against the **official** `Isaac-Cartpole-Direct-v0` task (no custom code involved). It applies a constant `+0.5` action (= +50 N on the cart joint, well below the 400 N effort limit) for 120 env steps and prints the cart state:

```bash
# PhysX backend (default preset) — affected
python physx_effort_repro.py

# Newton backend — works
python physx_effort_repro.py presets=newton_mjwarp
```

Actual output (PhysX):

```
[REPRO] t=  0: cart_pos=+0.00000 cart_vel=+0.00000
[REPRO] t= 59: cart_pos=-0.00000 cart_vel=+0.00037
[REPRO] t=119: cart_pos=-0.00000 cart_vel=+0.00041
[REPRO] RESULT: cart DID NOT MOVE (PhysX effort targets are silently dropped)
```

With `presets=newton_mjwarp` the cart accelerates immediately (`cart_vel=+0.51` after the first step, up to `+6.2 m/s`).

### Diagnostics (all on the PhysX backend)

1. **Isaac Lab-side state is correct at the API boundary.** After stepping with the constant effort:
   - `articulation._joint_effort_target_sim == [50.0, 0.0]` (staging buffer written by the `update_targets` kernel)
   - `articulation.data.joint_effort_target == [50.0, 0.0]`
   - `articulation.data.applied_torque ≈ [49.997, 0.0]`

   So the effort flows correctly through `_apply_actuator_model()` (ImplicitActuator `compute()` is a passthrough) into `write_data_to_sim()` → `root_view.set_dof_actuation_forces(self._joint_effort_target_sim, ...)`. Nothing downstream acts on it.

2. **Solver-independent.** Setting `solver_type=0` (PGS) → cart still exactly at 0.0000.

3. **Not the external-forces timing flag.** Setting `enable_external_forces_every_iteration=True` (the flag from the "TGS solver … may cause noisy velocities" warning) → same result.

4. **No drive-mode setup in the new backend.** There are no `drive_mode` / `set_dof_drive*` references anywhere in `isaaclab_physx` — I suspect `set_dof_actuation_forces` is ignored for DOFs whose PhysX drive is not in force mode on this stack, or the call is otherwise a silent no-op in isaacsim 6.0.1.0's tensor API.

5. **Newton backend unaffected.** The Newton actuation path (`_apply_actuator_model_newton` → `w.joint_f_2d` → `set_dof_actuation_forces`) applies the same effort targets correctly, so the problem is specific to the PhysX staging path on this stack.

### Impact on RL training

This makes effort-controlled Direct RL environments **impossible to learn** while looking like a normal training run. With the official `Isaac-Cartpole-Direct-v0` + skrl PPO (official agent config, 4096 envs), the mean total reward plateaus at ~10 (mean episode ≈ 38 of 300 steps) and never improves, regardless of hyperparameters (learning rate schedules, entropy scales, batch sizes all tested — identical plateau). The identical task/config on the Newton preset converges to **~296/300 mean reward within ~1000 steps**. The pole visibly swings only under gravity in both cases; the cart never responds.

### Expected behavior

Per the assets documentation, joint effort targets are feed-forward efforts applied in addition to the implicit-drive PD terms; a constant +50 N on the cart joint should accelerate the cart along the rail (as it does on the Newton backend).

### Notes

- Reproduces headless and with 16/4096 envs, on both the official task and a freshly generated extension-template copy of the same env.
- I could not test older isaacsim releases for comparison (no installable Isaac Lab 6.x/5.x artifacts exist on PyPI — only `3.0.0b2`), but the same workflow trained fine on earlier stacks, so this reads like a regression in the isaacsim 6.0.1.0 / isaaclab_physx 1.1.3 combination.
- Happy to run additional diagnostics (e.g. raw `set_dof_actuation_forces` outside of Isaac Lab, drive-mode readback) on request.

<details>
<summary>📄 physx_effort_repro.py (full source)</summary>

```python
# Copyright (c) 2022-2026, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
"""Minimal reproduction: joint effort targets are silently not applied on the PhysX backend.

Applies a constant +0.5 action (=> +50 N on the cart joint of Isaac-Cartpole-Direct-v0)
for 120 env steps and prints the cart state. On a healthy stack the cart accelerates;
on the affected stack the cart stays exactly at 0.

Usage (run from any directory with `isaaclab` importable):
    python physx_effort_repro.py                     # PhysX backend (affected)
    python physx_effort_repro.py presets=newton_mjwarp  # Newton backend (works)

The `presets=...` token is consumed by `resolve_task_config` (Isaac-Cartpole-Direct-v0
ships CartpolePhysicsCfg with `physx` / `newton_mjwarp` presets).
"""

import argparse
import sys

parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--num_envs", type=int, default=16)
args_cli, _hydra_args = parser.parse_known_args()

import isaaclab_tasks  # noqa: F401
from isaaclab.app import AppLauncher
from isaaclab_tasks.utils import resolve_task_config

app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app

import gymnasium as gym
import torch

TASK = "Isaac-Cartpole-Direct-v0"
env_cfg, _ = resolve_task_config(TASK, "skrl_cfg_entry_point")
env_cfg.scene.num_envs = args_cli.num_envs
env_cfg.headless = True
env = gym.make(TASK, cfg=env_cfg)
num_envs = env.unwrapped.num_envs
device = env.unwrapped.device

obs, _ = env.reset()
o = obs["policy"]
print(f"[REPRO] t=  0: cart_pos={o[0, 2]:+.5f} cart_vel={o[0, 3]:+.5f}")
for t in range(120):
    actions = torch.full((num_envs, 1), 0.5, device=device)  # +50 N on the cart joint
    obs, _, _, _, _ = env.step(actions)
    o = obs["policy"]
    if t in (59, 119):
        print(f"[REPRO] t={t:3d}: cart_pos={o[0, 2]:+.5f} cart_vel={o[0, 3]:+.5f}")

moved = abs(o[0, 2]) > 1e-3
print(f"[REPRO] RESULT: cart {'MOVED' if moved else 'DID NOT MOVE'} "
      f"({'PhysX effort targets are working' if moved else 'PhysX effort targets are silently dropped'})")
env.close()
simulation_app.close()

```

</details>


Search — env-api

Issues