If you're learning robotics, building a robot, or have an idea but don't have access to the
hardware, OmniSim is built for you.
OmniSim is more than a simulator. It's an open-source robotics workshop designed for agentic
development — a place where an agent has everything it needs to work on any robotic system. You
can simulate complete robotic systems with high-fidelity physics, build digital twins, connect
simulation to real robots, and give AI agents a workshop to program, test, and debug your system.
Open-source robotics simulator for coding agents: HTTP/JSON + MCP control, Newton physics, wgpu rendering, ROS 2, and reproducible benchmarks.
星标
190
工具
—
可用率
—
You can do all of that simply by talking to it.
Give it an instruction. See it move.
One Husky. One arena. Live AI instructions, follow-up questions, and a return
to base. Watch the full 100-second video
· Run this demo. Silent native OmniSim footage;
waiting removed and motion time-compressed. The preview shows the opening 32 seconds.
OmniSim is the free robotics workshop. OmniLink
is the connected AI agent. OmniLink requires an OmniKey and a model connection,
including on Free; model usage is billed separately.
Connect your account.
The recorded result is simulation, not a hardware demonstration.
No robot? Simulate one. Newton physics — rigid
bodies, contacts, cloth, soft bodies, CUDA granular media — 52 demo worlds, and a procedural
world generator.
Already have a robot? Build its digital twin. Native URDF import, CAD import from STEP, and
20+ vendor robot packages under projects/robots/.
Want it to reach the real machine? The same agent tools and the same bridge protocol address
the simulated robot and the physical one, alongside a ROS 2 sidecar
speaking the simulation_interfaces standard. What is portable is the software above the driver:
the driver itself is yours to wire, and no policy trained here has been validated on hardware —
see the sim-to-real guide.
Don't know how to program it? Let an agent help build and debug it. The first-party
MCP server gives Claude Code and Cursor 46 tools onto a running
simulation, and when the robot misbehaves the agent can ask what actually happened — every
contact, every joint limit, every grip, every line your controller printed, on one cursor-paged
HTTP stream. On the CPU solver (newtonSolver "mujoco", the default) the same scene runs
bitwise identically across cold launches — measured on contact-rich ten-robot scenes, and
refuted on the GPU solver — so a failure you can reproduce is a failure you can fix.
The goal is simple: make robotics accessible to anyone with an idea. OmniSim is completely free
and open source, Apache-2.0.
Built by agents, for agents: the HTTP harness, the Newton physics integration, the cloth and
soft-body stack, the RL pipeline and the ROS 2 sidecar were written by an AI agent under human
direction. And when the instruments cannot see, they say so instead of guessing: GET /sim/contacts
returns its own completeness and empty_set_reasons[] rather than an empty list that reads as
"nothing touched". Time control is partial: v9 added a held pause and break-on-event, but there are
no watch conditions, no record/replay/diff, and snapshot is a pose teleport rather than a checkpoint. And
Windows has the only prebuilt package: Linux is a source build, macOS is not supported. Those gaps
are itemised in what OmniSim is worse at.
Public beta: we are looking for the first ten external developers willing
to spend 20 minutes installing OmniSim, running one demo, and reporting the
first confusing or broken step — and, better still, to bring us something that
misbehaves and find out whether OmniSim can explain it.
Windows has the first downloadable package;
Linux is a source build; macOS is not supported.
Take the 20-minute challenge →
How OmniLink compares in OmniSim
In the original study, OmniLink kept the most commitments among seven configurations in a
blind 30-minute warehouse shift: 89.9%, versus 47.9–78.8% for the other
configurations. Every agent used Gemini 3.5 Flash on the same simulated Husky,
with 33 scored checks and three repeats each.
A later retrospective run of Codex with GPT-6.1 Sol, high reasoning, scored 76.8%
on the same shift across three repeats, with one check flagged unsafe in
one of three episodes. It used the same full robot tools and instructions, but a
different model and later build; three simulators ran together rather than seven.
Each Codex session was isolated from the test script and previous results.
This is a system comparison, not a fresh blind or model-controlled test.
Codex conditions and evidence.
A matching retrospective run of Claude Code with Opus 5.5, high effort, scored 80.8%
across three repeats, with no check flagged unsafe. It used the same protocol,
machine, engine binary and robot tools as the Codex run. Its episodes ran
closer to real time than Codex's (worst window 0.97× vs 0.15–0.26×), which may
have favoured it. Its scores overlap Codex's, so neither is shown to be better.
The adapter was written by a Claude Code session on the same model.
Claude Code conditions and downloadable evidence
· Repository report.
OmniLink also had the lowest estimated model cost in the original study:$0.90 per scored shift, 32–59% below the six original comparison configurations.
Codex's and Claude Code's actual costs are unavailable and are excluded from the cost ranking.
Codex's recorded tokens produce a conditional $0.91-per-shift Standard API pricing
scenario with 98.1% cached input. This does not establish its subscription charge,
API cache-write cost or a cost tie with OmniLink.
Claude Code's own CLI reports an unranked list-price estimate of $1.24 per shift;
it does not establish the actual subscription charge or a comparable API cost.
The cost graph counts all model usage recorded for the three scored shifts,
including failed checks. Original infrastructure errors and the superseded Codex
wave are reported separately in the cost ledger and pricing conditions.
Subscriptions, hosting, local compute and tax are excluded. Cost estimates are
based on recorded usage, not invoices; interrupted Codex calls may lack final usage.
OmniSim is the simulator for every configuration; this compares tested agent
integrations, not physics engines. One shift, one robot, one Windows laptop
(RTX 3060 Laptop GPU; Newton/MuJoCo physics on CPU). The benchmark, grader and
integrations were built by the OmniLink team. OmniLink led on the preregistered
score, but its equal-tools lead weakens when reply checks are excluded; that
comparison remains unsettled. Seven errored competitor episodes were re-run
under a recorded amendment. Simulation only; no hardware validation.
Three minutes on Windows. About half an hour on Linux, because you build it.
Get OmniSim.
Windows 10/11 — install the asset from the
latest release
(~600 MB). This is the only prebuilt package.
Linux — Ubuntu 24.04 or 22.04, built from source:
bash scripts/install/linux_bootstrap.sh. Budget 25–45 minutes; most of it
is the compile. Details: quickstart.
On 22.04 the bootstrap installs Python 3.12 (deadsnakes) and the engine
embeds it — the system 3.10 cannot run newton, and the build refuses to
link it rather than produce a simulator where nothing moves.
macOS — not supported. There is no package, no verified build, and
Newton physics is unverified. Use Windows or Ubuntu 24.04.
Check the install. Open a terminal in the OmniSim directory:
python -m omnisim doctor
It ends on a VERDICT line, and exits non-zero if the install cannot run —
most usefully when the Newton runtime is absent, which is not a degraded
mode but an install where nothing ever falls. On Windows without Python,
run omnisim.bat doctor instead: it uses the interpreter in the package.
See a robot move.
python -m omnisim demo
This runs the self-contained OmniArm friction-grasp demo. For the Husky
conversation shown above, follow the OmniLink setup guide. python -m omnisim demos lists
all of them (52 as of 2026-09-01), by category — the launcher's demos.json
is the live catalogue.
Then choose the shortest route to your own work:
Coding agent or MCP client: read AGENTS.md. Opening this
directory in Claude Code registers the first-party
OmniSim MCP server automatically — the checked-in
.mcp.json does it, with no install. Start the harness it proxies with
python -m omnisim harness. There is also a packaged
OmniSim Codex plugin.
Your robot or research task: use the URDF/STEP workflows,
or ask OmniLink to port one bounded, measurable pilot.
If any step is confusing or fails, that is exactly what the
public beta is designed to capture.
Real recordings and simulation comparisons
Two supporting studies show how real robot tasks can be reconstructed in
OmniSim. These are authored simulation motions beside real recordings,
not validated transfer to hardware.
Watch the 28-second comparison
· Evidence, source attribution and reproduction.
The simulation demonstrates grasping, spring compression and fingertip seating.
Geometry and mechanism parameters are estimates; recorded actions do not reproduce
the insertion. No electrical or physical-hardware validation is claimed.
Every OmniSim cell is measured by us and is reproducible from this repo. Every row names the
machines that produced it and we never average across them: the §3 throughput rows are three
machines, the §3 workload rows and the capability matrix are two, and a row that is still one
machine says so in the row. The machines are a laptop RTX 3060 (6 GB), Ryzen 16-thread, Windows
11 (9722d23d12a3), an RTX A4500 / AMD EPYC 7352, Ubuntu 22.04 (8ab788c4c833), and RunPod
RTX 4000 Ada (c72ce5632c81) and RTX 4090 (b5dadd645b1f). Competitor cells are their own
published documentation, dated and linked; we did not measure their engines. Where we lose is in
its own section.
1. Observability, and the agent-native surface it rides on
The first two rows are the debugger; the rest are how an agent reaches it.
OmniSim
Gazebo Jetty
Isaac Sim 6.0.1
Typed runtime events
10, with drop counters
—
—
Structured load diagnostics
50+ codes (open enum; GET /capabilities serves the live set)
—
—
First-party HTTP/JSON scene API
38 endpoints
none
none
First-party MCP server
46 tools, stdio, zero deps — each one HTTP call to the same harness a human drives
none
5 tools — docs search; none touch a running sim
Typed external control
38 harness + 15 capture verbs, plus ROS 2 simulation_interfaces (15 svc + 1 action) and a ros2_controlSystemInterface
ROS 2 + ros2_control, plus raw Python over TCP :8226
AGENTS.md at repo root
701 lines
none
yes, + 25 SKILL.md skills
That event-type list is not maintained by hand. The simulator regex-scans its own emit call sites and
reports any drift between code and declaration as undeclared / declared_not_emitted on
GET /capabilities
(event_bus.py) — so the row above is
a count of what the code emits, not of what the docs remember. Reproduction is scoped the same way:
on the CPU MuJoCo path (newtonSolver "mujoco", the default) OmniSim reproduces contact-rich
ten-robot scenes bitwise across cold launches on one machine, and across two machines running the
same binary; on the GPU mujoco_warp path it does not, and that is refuted rather than
unmeasured — 0 of 24 same-config pairs
(determinism-scope.md). We do not compare that against other
simulators here: determinism only compares like with like, solver against solver
(why).
CUDA kernel, 100k particles @ 4.50 ms/step — robot↔particle coupling currently dead
none
GPU PBD; no CPU support
Particles
via cloth / soft / granular
visual + lidar scattering only
GPU PBD; schema "not finalized"
Fluids
none (removed with ODE)
analytic drag only
PBD position-based
The grasp numbers behind that first row — tracking error −1.50 mm on a 616-particle T-shirt
against −173.06 mm for a negative control whose jaws never close, corroborated by a second
instrument, plus the hem-edge target that misses and is reported as a miss — are in
cloth-simulation.md, with the three disclosures that travel
with them: the fabric is pinned, so these are tracking and not load-bearing numbers; self-contact
must be off to grasp and on to drape; and the composed fold is not demonstrated.
5. Newton maturity
OmniSim
Isaac Sim 6.0.1
Isaac Lab
Newton version
1.5.0
1.2.1
3.0 beta
Solvers driven
2 (MuJoCo + VBD)
1 of 8 (MJWarp)
primarily MJWarp
Status
only backend, shipping
"experimental"
beta since 2026-03-17, no GA
6. Licence
Engine licence
Runtime restrictions
OmniSim
Apache-2.0
none
Gazebo
Apache-2.0
none
Isaac Lab
BSD-3-Clause
inherits Isaac Sim's when used with it
Isaac Sim
Apache-2.0 source only
NVIDIA ASML: no redistribution, no derivative works, use confined to "systems with NVIDIA GPUs"
Command checks are not collision avoidance or a safety certification. Direct
simulator controls have different protections from the connected chat path; see
actuation coverage. A successful Husky
recording does not establish safe unattended operation or equivalent performance
on every robot class.
Repeated motion can drift. The earlier square-loop experiment accumulated
position error. A short successful return to base does not establish reliable
operation over a shift without localisation.
Time control is partial. Since v9 a client can hold the engine across calls (POST /sim/pause,
leased: 30 s by default, 300 s cap, so a dead client cannot freeze the sim) and arm a breakpoint on an
event (POST /sim/break); POST /sim/step then single-steps and stops early on a break. Detection is
never sub-step, and there are two latency regimes: held plus /sim/step costs at most one basic step
of engine time, but free-running costs one supervisor tick, which has measured 8 ms to ~600 ms of
engine time depending on load — so pause first, then step. Still missing: watch conditions,
record, replay and run-diff. And POST /sim/snapshot is not a checkpoint: it saves poses and
joint angles only. Velocity is never captured (OmSolid::saveHiddenFieldValues() is an empty
function) and solver state is never touched, so restore teleports bodies to the saved poses while
they keep their live Newton velocities, and does not rewind the clock. Identical forward evolution
from a restore is not achievable, and we do not claim it.
Light mode is the default, and it silences 5 of the 11 event types (contact.*, grip.*,
joint.limit_hit); a breakpoint armed on a silenced type is refused rather than accepted and never
fired. Load the world with {"light": false} for a debugging session; /sim/contacts answers either
way. Separately, controller logs and physics events cannot be put on one
timeline: log events carry a wall clock (t_wall), supervisor events carry sim milliseconds
(t_sim_ms), and seq collides across the two sides.
Sensors are not readable through the debug surface.GET /robot/<def>/sensor/<name> is a
deliberate 501 — cameras, lidar and IMUs are read by the controller that owns them, not by the
instrument. Joints, contacts, grips, devices and bounds are readable; sensor samples are not.
Fault injection is structural impact damage only. No sensor dropout, no encoder drift, no
actuator degradation, no injected latency, no thermal derating. It binds to a robot named husky
by default and idles silently otherwise, and its WHEEL_TORQUE_SCALE is written into customData
for a cooperating controller to honour — the motor is never touched.
A green run-headless is a log verdict, not a physics verdict. A hologram floor lets a body
reach z = −69 km and still PASS. Add --duration N --fail-on-runaway when the claim is about
physics.
ROS 2 support is new and incomplete. OmniSim implements the ROS 2
simulation_interfaces standard plus /clock, /tf, JointState,
/odom, cmd_vel, sensor topics (Imu, LaserScan, GPS) and a ros2_controlSystemInterface — but that last one is verified for velocity-commanded bases only
(diff_drive_controller on the Husky). MoveIt is still out of reach, because OmniSim's arm
bridge treats a joint command as a goal and answers 409 busy to a setpoint arriving while the
previous one is still interpolating — a trajectory would land in pieces. Nav2 now runs
end-to-end on the Husky for planning and goal execution, but the verified case uses
ground-truth odometry and is not a SLAM, AMCL, or obstacle-avoidance benchmark. OmniSim is not in
the ros2_control simulator registry. Sensor
coverage is partial by measurement, not by omission: OmniSim's Gyro and Accelerometer produce
no usable data, so Imu ships a real orientation and declares those two components absent, and no
robot in the tree has a camera. For a lab whose stack is ROS 2 end to end, Gazebo remains better
integrated.
Not photoreal. Isaac Sim's RTX renderer is a different class. Ours is wgpu-native (Vulkan / D3D12 / Metal): a real-time raster stack whose global illumination is baked — an off-frame path trace into an irradiance probe volume, not per-frame ray tracing.
Sim-to-real is unproven — zero physical-robot transfer.
No free-standing humanoid walk — every G1 result uses a weight-bearing balance harness.
Quadrupeds carry none.
Newton is young and we removed our fallback. macOS is untested, so it has no verified physics.
Single-GPU training only. No multi-GPU or multi-node; Isaac Lab publishes a 16-GPU ladder.
Our figures are one GPU, and every number above says which one.
95.7% of measured capabilities work (51 probes as of 2026-09-01 round 3 -- eight rows flipped to works in one day, measured: IMU carrier, propeller inflow, ball probe, limit-less servo promotion, cloth/FEM/granular particle readback, connector weld, runtime deletion via rebuild; the
45-probe set as it stood on 2026-08-17 also ran on a second machine — 43 of 45 verdicts
agreed, and both disagreements are attributed in the
campaign, neither to the hardware).
Restitution is unimplemented, and runtime scene mutation is non-physical in both directions by
default: deleted nodes keep colliding, and spawned nodes are never registered with the solver at
all (since 2026-09-01 an opt-in mid-run physics rebuild — POST /sim/rebuild_physics, 97–267 ms
measured — fixes both, refused on deformable worlds and dropping engaged welds).
Compound colliders drop all but the first child by default. A motorised BallJoint was measured
not to actuate, but a 2026-09-01 probe review found that measurement could not have detected
actuation — unproven either way pending re-measurement (its Hinge2Joint sibling does actuate).
Capability matrix · CHANGELOG.
A mujoco_warp scene silently drops constraint rows past 256. Measured on both machines:
16 driven four-wheeled rovers peak at 336 and 328 rows against a cap that does not grow, with a
clean log and exit code 0. Raise WorldInfo.newtonNjmax before you build a fleet.
Smaller ecosystem than Gazebo — fewer robots, fewer worlds, no package index.
Get started
The three-minute path above is the fastest start. Contributors who want a
source build can clone it, open the folder in a coding agent, and ask:
Set up OmniSim from this fresh clone — install whatever the toolchain needs, build it,
and then launch the warehouse Husky demo so I can watch it.
The agent follows AGENTS.md (auto-loaded) and the
quickstart. First build is 5–25 minutes. By hand instead:
build_omni.bat on Windows, bash scripts/install/linux_bootstrap.sh on Linux.
Then run python -m omnisim doctor — it reports the ground truth about your clone. ⚠️ Newton is
the only backend, so a build without its runtime has no physics at all and otherwise fails
quietly.
Windows 10/11
Linux
macOS
Build from source
✅ documented
✅ scripted
⚠️ untested
Simulator, URDF import, harness, capture
✅
✅ verified
⚠️ untested
Newton physics (the only physics)
✅ CPU default; GPU needs CUDA
✅ wheels in the systempython3
⚠️ untested — no fallback exists
RL training + locomotion demos
✅
✅
❌
What you can ask for
Find out what happened:
"The gripper drops the box when it starts 2 cm left of centre. Run both placements and tell me what differs."
"Which joints hit their limits during that run, and at what sim time?"
"Did the fork and the pallet ever actually touch? Show me the contact set, not a screenshot."
"Why isn't the camera seeing the red cylinder?"
"Run this world twice on the CPU solver and tell me whether the two runs are bitwise identical."
Build the thing you need to debug:
"Launch the warehouse demo."
"Generate a Mars world with a 5-Husky fleet and run it headless for 30 seconds."
"Wire a Jackal on a flat platform, expose it on HTTP, and drive it forward 2 m."
For the first kind, the agent loads the world with {"light": false}, steps it, and reads
/sim/events, /sim/contacts and /robot/<def>/joints — the same HTTP surface you can curl
yourself (endpoints ·
protocol). For the second, it builds the world, edits controllers,
runs the simulator, and verifies its own work through the
validation harness. For runtime control of robots in a live scene, point
OmniLink agents at the per-robot HTTP bridges
(protocol). The chat demos — right-click a robot, type drive forward 1 m —
are one of those bridges with a panel on the front (guide).
OmniLink requires an OmniKey and a connected model. Missing setup or model
errors do not switch chat into a basic-command fallback. The robot's direct
Stop control remains independent of the model. See the
v9 release preview for the current scope.
These controls are not a safety certification or a guarantee of collision-free
operation. Actuation coverage documents
the limits of the shipped interfaces.
Lite3 · X30 quadrupeds · M20 · M20S · M20 + Piper wheeled-legged · DR02 Standard · DR02 Pro humanoids
OmniLink
OmniQuad quadruped
Clearpath
Husky · Jackal
Universal Robots
UR3e · UR5e · UR10e
Husarion
Rosbot · Rosbot XL
Robotis
TurtleBot3
DJI
Mavic 2 Pro
Per-robot status is in DEMOS.md. One caveat on OmniSim's own reference robots:
OmniTug 500 is a visual-only prop with no collider, so it is positioned kinematically and does
not participate in contact. OmniArm 6 and OmniArm 7 are fully simulated, and OmniArm 6 is
verified holding pose under gravity.
Teaching robots to move
Legged policies are made by Shadowing — train against a reference trajectory verified feasible
before learning starts, in-engine, so train and deploy share one solver. Fifteen skills ship across
G1, H1, Go2 and OmniQuad; BATON composes them into sequences.
python -m omnisim policy list # the catalogue
python -m omnisim policy sequence box_delivery # run a BATON demo
⚠️ Our humanoid demos run on a weight-bearing balance harness and are not free-standing walks.
Unitree's own policies re-hosted unchanged in OmniSim do walk free-standing, so the engine carries
it — training a policy that does is the open problem.
Apache 2.0 (LICENSE · NOTICE). The OmniSim name and orb mark are OmniLink
trademarks (TRADEMARKS.md) — open code, protected brand: fork and ship, but rename a
modified fork. Bugs go to GitHub Issues; sponsorship funds the engine work
(Sponsor · SPONSORS.md).
Repository provenance
This public repository is a curated release mirror. Its visible commit graph is
the history of the published mirror, not the complete private development
history, so commit count and first-public-commit dates should not be used as a
measure of total engineering work. For auditable public provenance, use the
tagged releases,
CHANGELOG, source commit recorded by python -m omnisim doctor,
and the binary/world hashes carried by the benchmark artifacts.