Sim-to-Real Transfer
Short answer. Sim-to-real transfer is training a policy in simulation and running it on real hardware. The gap comes from everything the simulator gets wrong: friction, compliance, sensor noise, latency and the small physical details nobody modelled. Domain randomisation is the usual answer, and it has two failure modes that pull in opposite directions. Too narrow a range and the real world falls outside it. Too wide a range and the policy learns for a worst case that does not exist, which can be as destructive as no randomisation at all.
How to do it
Where the gap comes from
A simulator models the parts of physics the task depends on and approximates the rest. Rigid body contact is usually modelled well, friction and deformation less so, and the small compliance in a real gripper or a real joint is often left out entirely. Sensor models add their own error: a simulated camera does not have the same noise, exposure or motion blur as the real one.
The consequence is that a policy can be optimal in simulation and wrong on hardware for reasons that have nothing to do with the algorithm. A grasp that works at one friction coefficient fails at a lower one, and the failure looks like a control problem rather than a modelling problem. Finding which parameter is responsible is the first step, and it usually means measuring that parameter on the real system.
- contact parameters: friction, restitution, compliance
- actuator and sensor models: latency, noise, calibration error
- geometry and mass: the details of the actual object and the actual arm
Domain randomisation and its two failure modes
Randomising the uncertain parameters during training exposes the policy to a range of worlds, so it learns a behaviour that works across them rather than one tuned to a single setting. If the range covers the real value, the policy transfers. If it does not, the policy is still tuned to a world that is not the real one, and the transfer fails exactly as it would without randomisation.
Widening the range is the intuitive fix and it has a cost. A policy that must succeed at the most extreme setting in the range will behave conservatively everywhere, and the conservative behaviour can be physically destructive: a grasp that has to hold at almost zero friction will crush a delicate object on a normal surface. The range is a claim about the real world and has to be measured, not chosen for safety.
Measuring the parameters you are randomising
Every randomised parameter is a measurement you owe the project. Friction can be estimated by sliding the object and measuring the deceleration. Latency can be measured by commanding a step and recording when the joint moves. Compliance can be measured by applying a known force and recording the deflection.
The measurements do not have to be precise. What matters is that the range contains the real value with a stated margin, and that the margin is written down. A range of plus or minus fifty percent around a measured value is a defensible engineering choice. A range from near zero to five is not a choice, it is an admission that nobody measured anything, and the policy trained on it will behave accordingly.
Code
A grasp model reduced to one friction coefficient, showing no randomisation failing, a measured range transferring, and an over-wide range destroying the object.
# a grasp model simplified to the point of being computable: object mass m,
# friction coefficient mu
# the minimum grip force must satisfy 2 * mu * F >= m * g; above F_MAX the object is crushed
M = 0.5
G = 9.81
SAFETY = 1.2
F_MAX = 20.0
MU_REAL = 0.35
def needed(mu):
return M * G / (2 * mu) * SAFETY
print("on the real robot the surface has mu = %.2f, so a grasp needs %.2f N"
% (MU_REAL, needed(MU_REAL)))
print()
CASES = [
("no randomisation, mu = 0.5", 0.5, 0.5),
("randomised mu in [0.3, 0.7]", 0.3, 0.7),
("randomised mu in [0.01, 5.0]", 0.01, 5.0),
]
print(" %-30s %10s %10s %s" % ("training setup", "grasp", "required", "outcome"))
for name, lo, hi in CASES:
f = needed(lo)
req = needed(MU_REAL)
if f < req:
outcome = "slips"
elif f > F_MAX:
outcome = "crushes the object"
else:
outcome = "holds"
print(" %-30s %8.2f N %8.2f N %s" % (name, f, req, outcome))
print()
print("the first row is the classic failure: the policy learned one friction")
print("coefficient and the real surface is slipperier than the simulation.")
print("the second row covers the real value and holds.")
print("the third row is the failure that looks like a fix. widening the range until")
print("it certainly covers reality makes the policy learn for the worst case in the")
print("range, and the resulting force destroys the object it was meant to hold.")
print()
for mu in (0.35, 0.28, 0.25):
f = needed(0.3)
req = needed(mu)
print(" if the real mu is %.2f, the second setup needs %.2f N and outputs %.2f N -> %s"
% (mu, req, f, "holds" if f >= req else "slips"))
print()
print("randomisation only helps if the range covers the real distribution. it is a")
print("statement about what the real world contains, and it has to be measured on")
print("the real robot, not chosen because a wider range feels safer.")- Standard library only. The model is deliberately reduced to one parameter so the two failure modes are arithmetically visible; a real grasp involves contact geometry, compliance and object mass distribution.
- Expected: the real surface at mu 0.35 needs 8.41 N. Training at a fixed mu of 0.5 produces a 5.89 N grasp, which slips. Randomising over [0.3, 0.7] produces 9.81 N, which holds. Randomising over [0.01, 5.0] produces 294.30 N, which crushes the object against a 20 N limit.
- The third row is the point of the example. It is the setup that looks most careful, because the range certainly contains reality, and it fails worse than no randomisation at all. A policy has to be safe at the hardest setting it was trained on.
- Expected: the last block shows the other edge. If the real friction is 0.28 or 0.25 rather than 0.35, the measured range no longer covers it and the same policy slips. The range is a claim about the real distribution and it has a boundary.
- Change SAFETY from 1.2 to 1.0 and the middle setup lands at 8.18 N, just below the 8.41 N it needs, which shows how little margin separates holding from slipping at the edge of the range.
Where this goes wrong
Randomising parameters nobody measured
A range chosen by intuition rather than measurement cannot be checked against reality, so a transfer failure gives no clue which parameter was wrong. Measure the parameters the task depends on, state the margin, and write down where each range came from. A range without a source is a guess that will be debugged later at hardware cost.
Widening the range until transfer succeeds
Widening the range is the fix that appears to work, because the real value is eventually inside it. The policy then trains for the worst case in the range and behaves conservatively everywhere, which for a manipulation task can mean crushing the object. Check the force or speed the policy commands on a normal surface, not only whether it succeeds.
Judging transfer from success rate alone
A policy that succeeds at every trial by applying excessive force has transferred in the sense that the task completes, and has not produced something deployable. Record the forces, torques and speeds alongside the success rate, because the failure that matters may be the one that does not show up as a failure.
Tuning the simulation to match one real trajectory
Fitting the simulator until it reproduces a recording makes it accurate for that recording and no other. The parameters that match one trajectory are not necessarily the physical ones, and a policy trained on them inherits the overfitting. Validate on trajectories held out from the fitting, and expect the simulator to be approximate rather than exact.
When to buy the data instead of building the pipeline
Measure the parameters yourself, because they describe your hardware and no supplier can know them. Buy the data when the simulation cannot produce the variation you need: real contact, real sensor noise, real human demonstrations, or objects your simulator does not model. Specify the sensor, the platform, the parameters that have to vary, and the measurement that establishes each one. We collect real robot data with the platform, the sensor and the parameter coverage documented per delivery.
Need data for Sim-to-Real Transfer?
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.