Export Ground Truth Object to Custom and COCO JSON Files

Open Live Script

This example shows you how to export a ground truth object to a custom data format JavaScript Object Notation (JSON) file, and to a COCO [1] data format JSON file.

You can parse JSON files to use for training and inference in machine learning frameworks. The format of the custom data format JSON file is similar to the COCO data format. The difference is that the format for representing label annotations for the JSON file is modified to include various label types supported by the ground truth object.

This example uses a ground truth object that contains labels previously generated using the Video Labeler app.

To export a ground truth object from the Labeler app to the JSON file, follow these steps:

Use the Image Labeler or the Video Labeler App to label images or videos respectively.
Export the labels to file or the workspace to generate a ground truth object.
Follow the steps in this example.

Load Ground Truth Object

Load a ground truth object

load("VehicleImageSequenceGroundTruth.mat")

Display the label definitions

VehicleImageSequenceGroundTruth.LabelDefinitions

ans=3×6 table
      Name        Type               LabelColor                 Group       Description     Hierarchy  
    _________    _______    _____________________________    ___________    ___________    ____________

    {'Car'  }    Polygon     0.5862     0.8276     0.3103    {'Vehicle'}    {0x0 char}     {1x1 struct}
    {'Lane' }    Line       0.49412    0.18431    0.55686    {'Road'   }    {0x0 char}     {0x0 double}
    {'Sunny'}    Scene            1    0.41176    0.16078    {'None'   }    {0x0 char}     {0x0 double}

Add Info and License Information (Optional)

Add the metadata to the annotations by specifying the Info and the License Name-Value pairs

info = struct( ...
    'year', 2020, ...
    'version', '1', ...
    'description', 'Vehicles Dataset', ...
    'date_created', datetime ...
    );

licenses = struct( ...
    'url', {'www.mathworks.com', 'www.mathworks.com'}, ...
    'name', {'License 1', 'License 2'} ...
    );

Export the Ground Truth Object to a custom data format JSON file

Use exportGroundTruthToJSON function to export the ground truth object to a JSON file.

The exportGroundTruthToJSON function parses the data in the ground truth object, encodes it in a custom data format that is described below, and then generates the JSON file.

annotationsFileName = 'vehicles_annotations.json';
exportGroundTruthToJSON(VehicleImageSequenceGroundTruth, annotationsFileName, 'Info', info, 'License', licenses)

Export the Object Detection Ground Truth Object to a COCO data format JSON file

Use exportGroundTruthToJSON with the 'COCO' Name-Value set to true to export object detection data that was labeled using polygons to the COCO object detection data format JSON file.

To create a ground truth object for object detection that can be exported to COCO data format JSON file, follow these steps:

Use the Polygon label type to label the object instances.
Use the Pixel label type to label the crowd regions of the object. Its label name should be as follows: polygonLabelName_crowd (Where polygonLabelName is the label name of the corresponding polygon label).

For example:

To annotate an image that contains both person instances and crowd regions,

Create a label of type polygon with name as 'person' for annotating person instances.
Create a corresponding Pixel label with name 'person_crowd' to annotate person-crowd regions.

The '_crowd' suffix is used to bind the two labels together. Therefore, ensure that for each crowd pixel label there is a corresponding polygon label.

Note: While exporting to COCO object detection data format, an expected warning is thrown specifying the label types that are being ignored. In this case, since the ground truth object has Line labels they will be ignored.

annotationsFileName = 'vehicles_annotations_coco.json';
exportGroundTruthToJSON(VehicleImageSequenceGroundTruth, annotationsFileName, ...
    'Info', info, 'License', licenses, 'COCO', true)

Warning: Label types, Line, Scene, are not supported and are ignored during export to a COCO data format JSON file

Custom JSON Data Format

The exported JSON file contains these five fields:

info
licenses
categories
images/ video
annotations

Info

The Info field specifies the dataset and annotation information by using the 'Info' name-value pair argument. The value must be a scalar structure.

Default Value:

{
  "year": Current year,
  "version": "1",
  "description": "Created using MATLAB",
  "date_created": Current datetime
}

Licenses

The Licenses field specifies information about the licenses by using the License name-value pair argument. The value must be a structure array.

Default Value: []

Images/ Video

The Images/ Video field contains a list of the images (or video) objects and related information.

Data Format:

{
  "id": Unique Image ID,
  "time_stamp": Time stamp of the image frame (Only applicable to data sources with timestamps),
  "width": Width of the image/video,
  "height": Height of the image/video,
  "file_name": Name of the file,
  "file_path": Absolute file path,
  "date_captured": Last modified date of the image/ Current datetime
}

Annotations

The Annotations field contains a list of annotation objects. The data format of the annotation objects depends on the label type and is as follows:

Rectangle ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID , 
  "category_id": Category ID,
  "position":  [x,y,width,height] bounding box location,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

x and y specify the upper-left corner of the rectangle and are 0-indexed.
w specifies the width of the rectangle, which is its length along the x-axis.
h specifies the height of the rectangle, which is its length along the y-axis.

Polygon ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID , 
  "category_id": Category ID,
  "position":  [[x1 y1 x2 y2 ...  xN yN]] for N points in the polygon,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

PolyLine ROI

{
  "id": Unique Annotation ID,
  "image_id": Image ID,
  "category_id": Category ID,
  "position": [x1 y1 x2 y2 ...  xN yN] for N points in the polyline.,
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

x1, y1,... specify the point location and are 0-indexed.

ProjectedCuboid ROI

{
"id": Unique Annotation ID,
"image_id": Image ID,
"category_id": Category ID,
"position": List of the form [xctr, yctr, zctr, xlen, ylen, zlen, xrot, yrot, zrot],
  "attributes": Contains attributes data,
  "sublabels": Contains sublabels data
}

xctr, yctr, and zctr specify the center of the projected cuboid and are 0-indexed.
xlen, ylen, and zlen specify the length of the projected cuboid along the x-axis, y-axis, and z-axis, respectively, before rotation has been applied.
xrot, yrot, and zrot specify the rotation angles for the projected cuboid along the x-axis, y-axis, and z-axis, respectively. These angles are clockwise-positive when looking in the forward direction of their corresponding axes.

PixelLabel ROI

{
  "image_id": Image ID,
  "file_name": Name of the pixel label image file,
  "file_path": Path to the pixel label image file,
  "segments_info": List of segment objects
}

Segment Objects format: Each segment in the pixel label image is representated as a segment object.

{
"id": PixelLabelID of the label or R+G*256+B*256^2 in case if PixeLabelID is an array,
"category_id": Category ID,
"area": Area covered by the pixels
}

Scene

{
  "id": Unique Annotation ID,
  "image_id": Image ID,
  "category_id": Category ID of the scene label applicable to the image
}

Limitations

Custom label types cannot be exported to the JSON file.
You cannot export data from a custom reader ground truth datasource to a JSON file format.
Only Polygon and crowd Pixel Label ROIs can be exported to the COCO object detection data format.
The JSON file generated by the "exportGroundTruthToJSON" helper function cannot be imported back into MATLAB as a ground truth object.

References

[1] Lin TY. et al. (2014) Microsoft COCO: Common Objects in Context. In: Fleet D., Pajdla T., Schiele B., Tuytelaars T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8693. Springer, Cham. https://doi.org/10.1007/978-3-319-10602-1_48