Point Cloud Annotation
Short answer. Point cloud annotation draws three-dimensional boxes around objects in lidar or depth data. A 3D box has nine numbers: the centre in three axes, the size in three axes, and a heading, with two more rotations when the object can tilt. How many of those nine the data actually determines depends on how many points landed on the object, and that count falls with the square of distance. A pedestrian at fifty metres is a few scan lines, and several of the nine numbers are then inferred from the object class rather than measured.
How to do it
Nine numbers, and which ones the points support
The centre and the size are constrained by the point extent in each axis, though only on the sides the sensor can see. The heading is constrained by the shape of the point distribution, which works when there are enough points and fails when there are not. The remaining rotations are usually set to zero on the assumption that vehicles and pedestrians stay upright, which is an assumption about the world rather than a measurement.
The practical consequence is that annotation quality is not uniform across a frame. Near objects have hundreds of points and the box is measured. Far objects have a handful and the box is interpolated from the class and from the previous frame. Both end up in the dataset with the same format and the same apparent authority, and only the point count distinguishes them.
- centre xyz and size lwh: constrained by the extent of the points
- heading: constrained by point distribution shape, needs enough points
- extra rotations: usually set to zero by assumption, not measured
Why distance dominates the difficulty
A rotating lidar samples the scene at a fixed angular resolution, so the number of points landing on an object falls with the square of distance. An object twice as far away is covered by a quarter of the points, for the same reason a distant wall looks smaller. The effect is large: on a sixty-four line sensor, a car goes from over a thousand points at ten metres to a few dozen at fifty.
The scan line count is the number that matters for the vertical extent, and it falls linearly with distance. Below roughly five lines the object is a stack of a few horizontal slices and its height and heading are poorly determined. This is why the error analysis of a 3D detector shows a strong distance dependence even when the model is unchanged, and why the annotation guideline needs a rule for objects that are too sparse to box.
Temporal consistency across frames
Point clouds are usually delivered as sequences, and the same object appears in many consecutive frames. Annotating each frame independently produces a box that jitters as the point count changes, which teaches a tracker to expect noise that is an artifact of the annotation rather than of the sensor. Most pipelines track an object across frames and annotate it once, then propagate.
Propagation introduces its own failure: an object that was annotated while close and then tracked into the distance carries a box that is no longer supported by points. The guideline has to say at what point the object stops being annotated, and the dataset has to record it. A sequence where objects vanish at a fixed distance is easier to learn from than one where they vanish at a fixed point count, and the choice should be deliberate.
Code
Compute how many laser returns land on three object sizes at three distances, using the angular resolution of a 64 line sensor.
import math
# 64 line lidar: vertical field of view -24 to +2 degrees, horizontal angular
# resolution 0.2 degrees
LINES = 64
V_FOV = 26.0
H_RES = 0.2
OBJECTS = {"pedestrian": (1.7, 0.6), "car": (1.5, 1.8), "truck": (3.5, 2.5)}
DISTANCES = (10, 30, 50)
def returns(height, width, dist):
v_span = 2 * math.degrees(math.atan(height / 2 / dist))
h_span = 2 * math.degrees(math.atan(width / 2 / dist))
rings = v_span / (V_FOV / LINES)
cols = h_span / H_RES
return rings, cols, rings * cols
print("points falling on one object, by size and distance")
print(" %-11s %7s %9s %9s %10s" % ("object", "dist", "rings", "columns", "points"))
for name, (h, w) in OBJECTS.items():
for d in DISTANCES:
r, c, n = returns(h, w, d)
print(" %-11s %5dm %9.1f %9.1f %10.0f" % (name, d, r, c, n))
print()
print("the same car yields about %.0f points at 10m and about %.0f at 50m."
% (returns(1.5, 1.8, 10)[2], returns(1.5, 1.8, 50)[2]))
print("the count falls with the square of distance, so the annotation difficulty is")
print("not uniform across a frame. the near objects are easy and the far ones are")
print("where the label errors concentrate.")
print()
h, w = OBJECTS["pedestrian"]
for d in (10, 30, 50):
r, _, n = returns(h, w, d)
print(" a pedestrian at %2dm is covered by %.1f scan lines and %.0f points" % (d, r, n))
print()
print("below roughly five scan lines the object is a handful of dots and the yaw")
print("angle of its bounding box is not determined by the data. a 3D box has nine")
print("numbers (centre xyz, size lwh, heading, and often two more rotations), and")
print("for the pedestrian at 50m, sixteen points have to support all nine. several")
print("of them are being guessed from the object class rather than measured. ask")
print("for the point count distribution with any 3D dataset, not just the frame count.")- Standard library only. The point counts are a geometric upper bound: they assume every angular cell inside the object silhouette returns a measurement, which overestimates what a real sensor delivers at grazing angles and on dark or wet surfaces.
- Expected: a pedestrian at 10m is covered by 23.9 scan lines and 411 points, at 30m by 8.0 lines and 46 points, at 50m by 4.8 lines and 16 points. The car goes from 1086 points at 10m to 44 at 50m, and the truck from 3482 to 141.
- The rings column is the one to read for vertical extent. A pedestrian at 50m sits on under five scan lines, so its height is determined by four measurements, and its heading by a point cloud that is essentially a few arcs.
- Change V_FOV to 40 and LINES to 128, which is a newer sensor, and the counts roughly double. The square-law distance dependence does not change, so the far objects stay the hard ones on any sensor.
Where this goes wrong
Reporting a frame count instead of a point count distribution
A dataset of a hundred thousand frames says nothing about how many objects in those frames are supported by enough points to box. The frames are cheap and the well-observed objects are what the model learns from. Ask for the distribution of points per annotated object, and for the share of objects under a stated threshold.
Letting annotators infer a box the points do not support
A human looking at a sparse cluster still knows it is a pedestrian and will draw a plausible box for it. That box is a label derived from the annotator knowledge, not from the data, and a model trained on it learns to reproduce the annotator priors. Either mark those objects as low confidence or stop annotating below a stated point count, and record which rule was used.
Annotating each frame independently
Independent per-frame annotation makes the box jitter as the point count changes between frames, and the jitter is an artifact of the annotation rather than of the sensor. A tracker trained on it learns to smooth noise that should not be there. Track the object across frames and propagate one box, then review the frames where the object enters or leaves.
Ignoring the ego vehicle motion
In a driving sequence the sensor is moving, so a static object moves through the point cloud frame to frame. Boxes annotated in the sensor frame and boxes annotated in a world frame are different labels, and mixing them produces a dataset where the same object has inconsistent coordinates. State the coordinate frame with the delivery.
When to buy the data instead of building the pipeline
Decide the point count threshold and the coordinate frame yourself, because they define what counts as an annotated object and a supplier cannot guess them. Buy the annotation when the sequence volume is beyond your team or the class list needs domain knowledge, and specify the sensor, the coordinate frame, the minimum point count and the tracking rule. Ask for the point count distribution on a pilot before agreeing to a total. We deliver 3D boxes with the sensor specification, the coordinate frame, the tracking rule and the point count distribution documented.
Need data for Point Cloud Annotation?
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.