SBX COCO Dataset Format

Our datasets are compatible with vanilla COCO, and contain extra information.
Folder contents
Example_SBX_Dataset/
...color/
...grayscale/
...depth/
...ir/
...meshes/
...sbx_coco.json
Each SBX Dataset will contain a folders of images as well as a JSON file in COCO format. The images are grouped into folders by their channel. The json file will  be named sbx_coco.json.
If a channel is not present in the dataset (ie, if there are is no depth data), then the folder for that channel will not be present.
For 6D Pose estimation projects, SBX includes STL models that can be used to QA projections into the synthetic images using the K and RT matrices.
SBX COCO JSON file format
Our datasets store meta data and annotations in the JSON file sbx_coco.json, which is compatible with the COCO format.

A COCO dataset is formatted in JSON and is a collection of “info”, “images”, “annotations”, “categories.”
{
    "info": {...},
    "images": [...],
    "annotations": [...],
    "categories": [...]
}
SBX has added custom fields and meta data to the categories, annotations and images.
Annotations Section
The annotations section is a list of JSON objects, one for annotation. SBX supports a wide variety of use-cases, and the content of each annotation object varies depending on the use-case.
[
    ...
    {
     "RT": [..],
     "area": <float>,
     "bbox": [..],
     "bbox_3D": [..],
     "category_id": <int>,
     "id": <int>,
     "image_id": <int>,
     "iscrowd": <int>,
     "segmentation": {..},
     "visible_perc": <float>
    },
    ...
    {
     "RT": [..],
     "area": <float>,
     "bbox": [..],
     "bbox_3D": [..],
     "category_id": <int>,
     "id": <int>,
     "image_id": <int>,
     "iscrowd": <int>,
     "keypoints": [..],
     "keypoints_bbox": [..],
     "keypoints_xyz": [..],
     "num_keypoints": <int>,
     "segm_bbox": [..],
     "segmentation": {..},
     "visible_perc": <float>
    },
    ...
]
Decoding the masks
Each annotation has a mask stored in the segmentation field, and can easily be decoded to a numpy array using pycocotools. 
import pycocotools.mask as mask_utils
​
compressed_seg = annotation['segmentation']
mask_np = mask_utils.decode(compressed_seg)json
Images Section
The images section contains meta data about each image contained in the dataset. It is a list of JSON objects, one for each image.
[
    ...
    {
     "channel": "<color|depth|ir>",
     "file_name": "<color|depth|ir>/<string>_XXXXXXXX.png",
     "width": <int>,
     "height": <int>,
     "id": <int>,
     "scene_id": <int>,
     "frame_tag_list": [],
     "K": [..],
     ...
    },
    ...
    {
     "channel": "depth",
     "file_name": "depth/SBXDepthSensor_Right_00009808.png",
     "width": 1920,
     "height":: 1080,
     "id": 39617,
     "scene_id": 71,
     "depth_factor": <float>,
    },
    {
     "channel": "color",
     "file_name": "color/SBXCameraSensor_Movable_00000631.png",
     "frame_id": 631,
     "frame_tag_list": [],
     "has_annot": True,
     "width": 1920,
     "height": 1080,
     "id": 160,
     "scene_id": 45,
     "view_id": 0,
     "K': [[2058.7265625, 0.0, 960.0], [0.0, 2058.7265625, 540.0], [0.0, 0.0, 1.0]],
    }
    ...
]
The file_name field is relative to the dataset's folder.
Each annotation will have a field image_id with corresponds to an image in this list.
The camera intrinsic parameters are in a field K on each image object. 
If there are multiple sensors in the scene, each image can be joined with the others using the scene_id field.
Categories Section
The categories section contains meta data about each category (or class) of label that exists in the dataset. 
[
    ...
    {
     "id": <int>, 
     "name": "<string>", 
     "supercategory": "<string>"
    },
    ...
    {
     "id": <int>,
     "keypoints": [..],
     "mesh": "meshes/<string>.stl",
     "name": "<string>",
     "supercategory": "<string>",
     "sbx_transform": "stl_trans",
     "skeleton": [..]
    },
    ...
]
Each annotation will have a field: category_id which will correspond to a category in this list.
Info Section
The info section contains meta data about the dataset. It is a JSON object (key/value pairs) describing the dataset.
{
    "contributor": "SBX Robotics Inc.",
    "date_created": "YYYY/MM/DD UTC",
    "url": "http://sbxrobotics.com",
    "year": "YYYY",
    "description": "<string>",
    "name": "<string>",
    "project_version": "X.X.X",
    "sbxcoco_version": "Y.Y.Y"
}
Image format
SBX synthetic images are rendered into PNG format.
COCO Resources
​cocodataset.org​
​opencv.org​
​
​
Last modified 6mo ago