Bounding Box Annotation
Short answer. Bounding box annotation marks each object with an axis-aligned rectangle. The work is not drawing: it is deciding the rule for the cases the guideline did not anticipate, which is why two teams annotating the same images with the same tool agree at roughly 0.7 to 0.8 IoU rather than 0.95. The deliverable is a box list plus a written rule set covering occlusion, truncation, minimum size, and whether a box covers the visible extent or the estimated full extent of the object.
How to do it
The four rules that decide most of the disagreements
Occlusion: when a person is half behind a car, the box can cover the visible part only or the estimated full body. Both are defensible and they are not compatible. The downstream difference is large, because a model trained on visible-extent boxes learns to fire on fragments, and a model trained on full-extent boxes predicts regions that are not in the image.
Truncation at the image edge is the same question in a different place, and it usually gets a different answer, which is itself a rule that has to be written. Minimum size decides whether a distant crowd produces thousands of three-pixel boxes that no model can learn from. Class boundaries decide whether a boxed object is a van or a truck, and whether a person on a bicycle is one box or two.
- Occluded objects: visible extent, or estimated full extent
- Objects at the image edge: box the visible part, or drop the object
- Minimum box size in pixels, and what to do below it
- Objects that are one thing or two: person on a bicycle, truck with trailer
Format conversion is where the errors enter
Boxes are stored in at least three conventions and every conversion is a chance to lose a coordinate. COCO stores top-left plus width and height. Pascal VOC stores a corner pair. YOLO stores the centre plus size, normalized by the image dimensions. A dataset that moves between tools will pass through two of these, and a mistake in any step survives into training without an error message.
The round trip is the test: convert to the other convention and back, and assert that the original values return within a small tolerance. Normalization introduces floating point error, so the tolerance is not zero, and it should be far below one pixel. If the round trip is exact, the conversion is right; if it drifts by more than a pixel, something in the pipeline is rounding to integers too early.
How to specify a box job so it can be accepted
An acceptance test needs three things that a bare request does not have: the rule set, a sample that both sides annotate before the batch starts, and a measured agreement number on that sample. Without the sample there is no evidence that the two sides read the rules the same way, and the disagreement surfaces after the full delivery when it is expensive to redo.
The agreement number should be reported per class, because the hard classes are usually the small ones and the ones with unclear boundaries. A single overall figure hides that, and the classes that drag the model down are exactly the ones the average conceals.
Code
Convert one box between the COCO, Pascal VOC and YOLO conventions and prove the round trip is exact.
def coco_to_voc(b):
x, y, w, h = b
return [x, y, x + w, y + h]
def voc_to_coco(b):
x1, y1, x2, y2 = b
return [x1, y1, x2 - x1, y2 - y1]
def voc_to_yolo(b, W, H):
x1, y1, x2, y2 = b
return [((x1 + x2) / 2.0) / W, ((y1 + y2) / 2.0) / H,
(x2 - x1) / float(W), (y2 - y1) / float(H)]
def yolo_to_voc(b, W, H):
cx, cy, w, h = b
return [(cx - w / 2.0) * W, (cy - h / 2.0) * H,
(cx + w / 2.0) * W, (cy + h / 2.0) * H]
W, H = 1920, 1080
coco = [412.0, 233.0, 96.0, 210.0]
voc = coco_to_voc(coco)
yolo = voc_to_yolo(voc, W, H)
back_voc = yolo_to_voc(yolo, W, H)
back_coco = voc_to_coco(back_voc)
print("coco ", [round(v, 2) for v in coco])
print("voc ", [round(v, 2) for v in voc])
print("yolo ", [round(v, 6) for v in yolo])
print("back voc ", [round(v, 2) for v in back_voc])
print("back coco ", [round(v, 2) for v in back_coco])
print("round-trip max error: %.3e px"
% max(abs(a - b) for a, b in zip(coco, back_coco)))
print()
x, y, w, h = coco
print("forgetting to normalize for yolo:")
print(" cx %.1f cy %.1f w %.1f h %.1f" % (x + w / 2.0, y + h / 2.0, w, h))
print(" a yolo loader multiplies by the image size, so it would read the box as")
print(" %.0f x %.0f px at (%.0f, %.0f)" % (w * W, h * H, (x + w / 2.0) * W, (y + h / 2.0) * H))- No dependencies beyond the standard library.
- Expected: the round-trip error prints 5.684e-14 px, which is floating point noise at this coordinate scale. Anything larger than about 1e-9 px means a conversion step is wrong.
- The second block shows what an unnormalized box does to a YOLO loader. A 96 by 210 pixel box becomes 184,320 by 226,800, which is larger than the image and will be clipped to it, so the model trains on a box covering the whole frame without any error being raised.
Where this goes wrong
Not writing the occlusion rule down
Two annotators given the same images and the instruction to box the people will produce two different datasets if nobody said whether a half-hidden person is boxed to the visible edge or to the estimated full body. The agreement number on the pilot sample is the evidence that they read it the same way, and it is the only cheap moment to find out.
Rounding boxes to integers too early
Storing pixel boxes as integers loses sub-pixel precision and, more importantly, hides conversion bugs: a round trip through a normalized format will come back slightly different and the error will look like noise. Keep boxes as floats through every conversion and round once, at the point where the file is written for a format that requires integers.
Letting the class list grow during annotation
Annotators meet objects the class list does not cover and add a category, and by the end of the batch there are two labels for the same thing. Freeze the class list before the batch, collect the unmatched objects in a separate bucket, and decide the additions between batches so they can be applied consistently to everything already done.
Accepting a delivery without the agreement figure
A box list with no measured agreement is unauditable. You can look at a sample and form an impression, but you cannot tell whether the hard classes were done to the same standard as the easy ones. Require a pilot of a few hundred images annotated by two people, with per-class agreement reported, before the full batch is produced.
When to buy the data instead of building the pipeline
Build the viewer and the acceptance test yourself, because they encode your rule set. Buy the annotation when you need volume, a class balance your own team cannot sustain, or annotators who are not the engineers who will use the data. What you are buying is the rule set applied consistently, so ask for the guideline, the pilot agreement figure per class, and a sample of the hard cases before the batch starts. We run vision annotation against a written guideline, with a two-annotator pilot and per-class agreement reported before the full batch is produced.
Need data for Bounding Box Annotation?
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.