COCO Format
Short answer. COCO is a JSON annotation format built on three top-level arrays: images, annotations, and categories, joined by id fields. Each annotation carries an image_id, a category_id, a bounding box as [x, y, width, height] with the origin at the top-left pixel, an area, and an iscrowd flag. The pixel convention is what breaks pipelines: boxes are top-left plus size, while keypoints and segmentation polygons are absolute coordinates in the same file. A file can be valid COCO and still not contain the field your loader expects.
How to do it
The three arrays and the ids that join them
The images array holds one row per image with an id, a file name, and the width and height. The annotations array holds one row per object, and its image_id points back at the image row. The categories array holds the class list with an id and a name. Nothing enforces the joins, so a wrong image_id produces an annotation that belongs to no image and is silently dropped by most loaders.
Ids are integers and are usually not contiguous. Tools renumber them, splits are merged, and a dataset assembled from two sources will have gaps and occasionally collisions. Treat the id as an opaque key and never as an index into an array, because a collision merges two images into one and the loader reports fewer images than the folder holds.
- images[]: id, file_name, width, height
- annotations[]: id, image_id, category_id, bbox, area, iscrowd
- categories[]: id, name, supercategory
- optional per annotation: segmentation, keypoints, num_keypoints
Two coordinate conventions in one file
The bbox field is [x, y, width, height], where x and y are the top-left corner in pixels. Reading it as [x1, y1, x2, y2] gives a box that starts at the right place and has the wrong size, and the error grows with the object. The segmentation field, when it is a polygon, is a flat list of absolute x and y coordinates that trace the outline. Keypoints are absolute coordinates too, in triples of x, y, and a visibility flag.
The practical consequence is that a conversion script has to know which field it is reading. A single helper that assumes corners will be right for the polygons and wrong for the boxes. Write the conversion once, test it with a round trip, and keep the convention in the variable name.
iscrowd and area, and why the evaluator needs both
The iscrowd flag marks a region that is annotated as a group rather than as individual objects, such as a crowd of people or a shelf of identical items. The evaluator treats these regions differently: a prediction inside a crowd region is not counted as a false positive, and the crowd region is not required to be matched. A dataset that leaves the flag at zero everywhere is not wrong, but it will report false positives for every detection inside a crowd.
The area field is redundant for boxes, since width times height gives it, and it is not redundant for segmentation masks, where the mask area differs from the box area. Evaluators use area to bucket results by object size, so an incorrect area silently moves objects into the wrong bucket and changes the reported score for small and large objects without changing the boxes.
Code
Build a minimal COCO file, assert the required keys are present on every row, and show what the bbox convention does to a conversion that assumes corners.
import json
coco = {
"images": [
{"id": 1, "file_name": "street_0001.jpg", "width": 1920, "height": 1080},
],
"annotations": [
{"id": 1, "image_id": 1, "category_id": 1,
"bbox": [412.0, 233.0, 96.0, 210.0], "area": 96.0 * 210.0, "iscrowd": 0},
{"id": 2, "image_id": 1, "category_id": 2,
"bbox": [1280.0, 700.0, 310.0, 180.0], "area": 310.0 * 180.0, "iscrowd": 0},
],
"categories": [
{"id": 1, "name": "person", "supercategory": "person"},
{"id": 2, "name": "car", "supercategory": "vehicle"},
],
}
REQUIRED = {
"images": {"id", "file_name", "width", "height"},
"annotations": {"id", "image_id", "category_id", "bbox", "area", "iscrowd"},
"categories": {"id", "name"},
}
for key, fields in REQUIRED.items():
for row in coco[key]:
missing = fields - set(row)
assert not missing, "%s id=%s missing %s" % (key, row.get("id"), missing)
print("required keys present on every row")
print("serialized size: %d bytes" % len(json.dumps(coco)))
print()
print("bbox is [x, y, w, h], so the corners are:")
for a in coco["annotations"]:
x, y, w, h = a["bbox"]
print(" annotation %d -> corners (%.0f, %.0f) to (%.0f, %.0f), area %.0f"
% (a["id"], x, y, x + w, y + h, a["area"]))
print()
a = coco["annotations"][0]
x1, y1, x2, y2 = a["bbox"]
print("read as [x1, y1, x2, y2] instead:")
print(" width %.0f instead of %.0f" % (x2 - x1, a["bbox"][2]))
print(" height %.0f instead of %.0f" % (y2 - y1, a["bbox"][3]))- No dependencies beyond the standard library.
- Expected: the assertion passes, the serialized size prints 455 bytes, and the last block shows what reading bbox as a corner pair does. The person box comes out with width -316 and height -23, because x2 and y2 are being read from the width and height fields. A negative width is easy to notice; the dangerous version of the same mistake is a box whose width happens to exceed its x, where the error stays positive and the box is silently the wrong size.
- Add a segmentation polygon to one annotation and print its length. It is a flat list, so the length is twice the number of points, and a script that expects pairs of pairs will read it wrong.
Where this goes wrong
Assuming bbox is a corner pair
This is the single most common conversion bug, and it is quiet: the box still starts in the right place, so a viewer that also assumes corners draws something and nobody notices until the metrics come out at zero. Check with a round trip, and check the converted file in a tool that was not written by the same person as the converter.
Merging two datasets without renumbering
Concatenating two annotation files leaves two images with id 1 and two annotations with id 1. Most loaders build a dictionary keyed by id, so the second silently replaces the first and the dataset is smaller than the file list says. Renumber every id on import, and assert that the number of distinct image ids equals the number of image rows.
Forgetting that area must match the mask, not the box
For a segmentation dataset the evaluator buckets by area, and the area of a mask is smaller than the area of its bounding box for anything that is not a rectangle. Leaving the box area in place moves every object into a larger size bucket, which changes the small-object score without changing a single mask. Recompute area from the mask.
Expecting one COCO file to carry everything
Panoptic annotations use a different file layout with a per-image segment map, and keypoint datasets add fields the detection loader ignores. A file can pass a schema check and still lack the field the training code needs. Validate against the fields your loader reads, not against the format in general.
When to buy the data instead of building the pipeline
Write the converter yourself, because the convention has to match your training code. Buy the data when you need annotations in this format at a scale and class balance you cannot produce: a class list with instance counts per class, images that cover the conditions your product meets, and a guideline that says how partly hidden objects were boxed. Ask for a sample in the exact format you will load, with the schema documented, so the converter is tested before the full delivery arrives. We deliver vision corpora in COCO and in the corner-based formats, with the conversion checked by round trip and the class list documented per split.
Need data for COCO Format?
Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline, not a range. If we cannot source it well, we will tell you that instead.
- Pilot batch before the full run, so problems surface early.
- Consent documentation delivered with the data.
- No medical or clinical data. No recorded telephone calls.
- You send the spec Language, hours, recording conditions, and the deadline.
- We reply within two business days A real number and a real timeline, not a range.
- You approve a pilot batch A small run first, so problems surface before the full order.
- Full run, then delivery with documents Audio, transcripts, and the consent record for every speaker.