Search — usd

MuJoCo's USD decoder can change inferred mass/inertia even when bodies have explicit inertials and there are visual-only geoms. This breaks MJCF->USD->MuJoCo validation that compares compiled inertials and deterministic rollouts.

USD cable import currently applies only stretch and bend stiffness mappings while ignoring parsed shear and twist stiffness when building the rod. The builder falls back to defaults (shear=stretch, twist=bend) and emits a warning, leading to incorrect cable behavior versus authored USD parameters.

A ROS2 camera OmniGraph driven by OnPlaybackTick stops publishing after 1–2 frames when added as an `over` on a payload-referenced robot. Render products and image data appear valid, but ROS2CameraHelper output stalls.

MuJoCo’s USD decoder misses physics-purpose material bindings on colliders when different materials are used for visual vs physics purposes. This breaks contact property preservation in MJCF↔USD roundtrip validation.

MuJoCo’s USD decoder does not preserve several semantics needed for roundtrip validation: sites, model names, unlimited joints, and disabled-collider metadata. This undermines compiled-model comparisons and deterministic rollouts after import.

MuJoCo’s sample/compile cannot load USD decoder plugins unless the plugin is manually registered/loaded. Users want the sample to auto-load plugins from standard build/install directories before parsing.

User hits ERROR_OUT_OF_DEVICE_MEMORY on a GCP setup using an NVIDIA L4 while following the recommended Isaac Sim cloud setup guide. The issue occurs with both compiled-from-source and precompiled binaries, preventing progress.

MuJoCo’s native island discovery uses quadratic temporary structures (ntree x ntree adjacency and column-index arrays) even for sparse incidence. Proposal is to remove the quadratic scratch by constructing connected components directly from constraint/tree incidence.

Mesh SDF generation settings are partially configurable and inconsistent across USD/URDF/MJCF imports. Request is to expose configuration paths for Mesh.build_sdf() settings and avoid import paths that drop sdf_* options before expensive cooks occur.

Automatic USD mesh approximation calls ModelBuilder.approximate_meshes() without forwarding method-specific settings. This blocks users from applying required settings (e.g., CoACD threshold migration) through ModelBuilder.add_usd() while respecting USD-authored approximation routing.

Newton viewers currently apply viewport-style USD purpose defaults (show default/proxy; hide guide/render) with no user override. Request is to add configurable purpose visibility so users can reveal guide geometry (e.g., MuJoCo sites) or hide other purposes.

In Isaac Sim 6.0.1, a generated IRI SpillEvent puddle appears in RGB but is not captured in semantic segmentation even after applying semantic labels to the puddle mesh. The leaking source object is labeled correctly, but the generated spill is not.

parse_usd's custom-frequency traversal does not honor ignore_paths while other traversals do. This causes prims that should be ignored to be processed during the custom-frequency pass.

Streaming Client 2.0 on Windows auto-resizes when OS scaling is set to 125%, producing an oversized GUI despite a requested 1920x1080 resolution. Logs indicate dynamic resize adjustments due to alignment and constraint requirements.

In Isaac Lab camera output example, point cloud generation fails because camera pose fields become zero/NaN. The report uses Isaac Lab main with Isaac Sim 5.1 and the documented camera output tutorial script.

Feature request to import USD deformable rest shapes, requiring both USD parsing and Newton API support for separate live/rest geometry and rest-based properties. It calls out specific USD attributes and asks for validation/fallback behavior and tests.

Newton's import_usd.py has grown beyond 4500 lines and is requested to be split into multiple files. The goal is improved readability and maintainability of the USD import pipeline.

Request to add a first-class ModelBuilder operation to convert an already-added joint to a different type before finalize, at least to FIXED while preserving transforms. The use case is vehicle wheels authored with revolute joints but simulated analytically to reduce solver DOFs.

Documentation/support is inconsistent for free joints applied to non-root links: URDF loader allows it, MJCF does not, USD loader cannot produce it, while the Newton API supports it. The issue asks to clarify intended support, align loaders, and add tests/warnings.

Proposal to add an Allegro hand in-hand cylinder rotation RL task with grasp-cache initialization, tuned reset poses, contact-aware rewards, a gravity curriculum, and RSL-RL train/play support. The goal is coordinated multi-finger rotation rather than relying on a single pinch contact.

USD importer appears to mishandle authored center-of-mass transforms, yielding incorrect COM and unstable simulations during migration from PhysX USD assets. The report references OpenUSD rigid body physics expectations that COM is in local space.

Isaac Lab TerrainImporter fails to import a plane terrain when users set documented physics/visual material overrides because it assumes legacy collision/shader prim paths that aren’t present in the current ground-plane USD. This breaks valid TerrainImporterCfg usage during environment creation.

Isaac Sim/Kit forces PXR_WORK_THREAD_LIMIT=16, preventing the Newton/OpenUSD workaround (set to 1 before pxr init) for a known heap corruption crash when loading/cloning rigid bodies with many colliders.

Newton MJCF importer uses assertions and low-context exceptions for malformed user input instead of deterministic validation errors, making debugging difficult.

Isaac Lab resolves ISAAC_NUCLEUS_DIR to a Nucleus cloud URL even when Isaac Sim is configured for local assets, leading to FileNotFoundError when cloud assets are inaccessible.

Differentiable simulators have advanced policy learning and model-based control, yet actuator dynamics remain an important source of sim-to-real error. This is particularly acute on low-cost platforms, where the linear current-to-torque relation $τ= K_tI$ becomes unreliable during commanded-target tracking because of friction, hysteresis, backlash, and thermal effects. We present NeuralActuator, a neural actuator model that jointly predicts (i) a simulator-equivalent generalized-effort surrogate for trajectory propagation on low-cost servo platforms, (ii) external force with a contact-probability gate for sensorless force perception, and (iii) a motor-condition score for the supervised joint. We also introduce the Neural Actuation Dataset (NAD), collected with a twin-arm teleoperation system that records robot states and actuator telemetry together with external-force labels. The torque-surrogate head is trained through differentiable simulation from pose trajectories without direct generalized-effort labels, while the force, gate, and motor-condition heads receive direct supervision. A Transformer captures temporal dependencies while supporting real-time inference. We evaluate NeuralActuator on a 5-DoF OpenManipulator-X, a 6-DoF SO-101, and a 7-DoF Franka Emika Panda, spanning three actuator families and platforms costing approximately USD 500 to over USD 30,000. The low-cost platforms support dynamics and force evaluation, while the offline Franka experiment provides an additional payload-force-estimation benchmark. Experiments further demonstrate its application for motor condition estimation on OpenManipulator-X and improved behavior-cloning performance when NeuralActuator is used as a pretrained module.

3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class-level CAD templates, which limits instance-specific shape detail. This paper presents Hydra++, a system-level investigation into how learning-based object shape estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category-agnostic shape estimation and a reprojection-mask consistency check to reject degenerate predictions from partial observations or imprecise segmentation. In its default CRISP-based configuration, Hydra++ performs online scene graph construction; slower estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization-latency trade-offs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid LiDAR-camera configuration for large-scale operation, improving scene-level reconstruction quality. Experiments in both simulation and real-world outdoor campus scenarios demonstrate that Hydra++ improves object- and scene-level reconstruction quality. Project page is available at https://hydra-plusplus.github.io/.

Open-vocabulary 3D scene understanding aims to segment 3D scenes beyond predefined categories by transferring semantic knowledge from vision-language models. Existing methods have advanced this task by lifting language-aligned 2D features into 3D, yet they often rely on context-independent semantic representations, leaving object relationships underexplored for contextual refinement. We propose RelGraphOV, a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding. Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships and prune geometrically implausible connections, without manual relationship annotations. To aggregate relational context while avoiding feature interference, we introduce an Adaptive Gated Dual-Stream Contextual GAT that separates dense geometric features and semantic CLIP embeddings, performs edge-guided message passing, and adaptively fuses complementary semantics. A hierarchical contrastive objective further promotes instance-level consistency and category-level discrimination. Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability. Project Page: https://cxavireh.github.io/relgraphov-projectpage

Safe landing-site assessment in unstructured environments remains a key challenge for autonomous UAV deployment, as vision-only learning approaches often degrade under terrain variability and provide limited transparency in safety decisions. We present NEUROSYMLAND, a neuro-symbolic landing-site assessment system that integrates lightweight perception with explicit safety reasoning. The framework constructs a probabilistic semantic scene graph from onboard visual input and evaluates candidate landing regions using symbolic constraints capturing terrain flatness, obstacle clearance, and spatial consistency, enabling structured reasoning under perceptual uncertainty while maintaining edge-feasible execution. Across 72 simulated landing scenarios spanning diverse terrains, NEUROSYMLAND achieves 61 successful assessments, outperforming four competitive baselines (37-57 successes). To evaluate deployability, we further conduct 100 hardware-in-the-loop trials with randomized initial poses, profiling end-to-end latency, stage-wise execution time, and system-level metrics including CPU/GPU utilization, memory footprint, and power consumption. Results demonstrate improved robustness and interpretability with bounded edge-resource usage. Profiling shows that symbolic reasoning contributes only a small fraction of end-to-end latency, while the main computational cost arises from perception and PSSG construction. These results demonstrate the feasibility of deploying the landing-site assessment stack on edge-constrained UAV hardware, and all source code, datasets, prompts, and symbolic rule refinement examples are released in an open-source repository

While robotic manipulation capabilities have advanced rapidly, physical safety remains a major barrier to deploying household robots: task success is insufficient if the robot damages itself or its surroundings. Simulation offers a harm-free alternative to costly and dangerous real-world training and evaluation, yet existing simulators lack general mechanisms to detect, quantify, and represent damage. To address this gap, we introduce OOPSIEVERSE, a unified simulation framework and benchmark for damage-aware household manipulation. OOPSIEVERSE provides damage as an explicit, physically-grounded, and taskagnostic signal by converting sources such as contact forces, temperature changes, and liquid interactions into corresponding mechanical, thermal or fluid damage. OOPSIEVERSE comprises two core elements: (1) DAMAGESIM, a simulator-agnostic framework for detecting and quantifying damage during navigation and manipulation, and (2) a suite of household tasks designed to evaluate common damage modes and distinguish between task completion and safe execution. We demonstrate the generality of our framework by instantiating DAMAGESIM in two simulators with different physics backends, OmniGibson (Nvidia Omniverse) and RoboCasa (MuJoCo). We further showcase the utility of OOPSIEVERSE across multiple use cases, including (1) guiding safer demonstration collection via real-time damage feedback, (2) learning safer manipulation policies through damage-conditioned imitation learning and reinforcement learning, (3) benchmarking the safety of state-of-the-art Vision Language Action policies, and (4) improving real-world safety of sim-to-real transferred policies. Together, our results highlight the potential of OOPSIEVERSE as an open-source foundation for systematic, scalable research on safe robot manipulation. For code and more information, please refer to https://robin-lab.cs.utexas.edu/oopsieverse/