Create Class for Loading Custom Ground Truth Data Sources

In the Ground Truth Labeler app, you can label signals from image and point cloud data sources. These sources include videos, image sequences, point cloud sequences, Velodyne^® packet capture (PCAP) files, and rosbags. To load data sources that the app does not natively support, you can create a class to load that source into the app.

This example shows how to use one of the predefined data source classes that load signals from data sources into the Ground Truth Labeler app: the vision.labeler.loading.PointCloudSequenceSource class. The app uses this specific class to load sequences of point cloud data (PCD) or polygon (PLY) files from a folder.

To get started, open the vision.labeler.loading.PointCloudSequenceSource class. Use the properties and methods described for this class to help you write your own custom class.

edit vision.labeler.loading.PointCloudSequenceSource

Custom Class Folder

The Ground Truth Labeler app recognizes data source classes only if those files are in a +vision/+labeler/+loading folder that is on the MATLAB^® search path.

The vision.labeler.loading.PointCloudSequenceSource class and other predefined data source classes are stored in this folder.

matlabroot\toolbox\vision\vision\+vision\+labeler\+loading

In this path, matlabroot is the root of your MATLAB folder.

Save the data source classes that you create to this folder. Alternatively, create your own +vision/+labeler/+loading folder, add it to the MATLAB search path, and save your class to this folder.

Class Definition

Data source classes must inherit from the vision.labeler.loading.MultiSignalSource class. View the class definition for the vision.labeler.loading.PointCloudSequenceSource class.

classdef PointCloudSequenceSource < vision.labeler.loading.MultiSignalSource

When you load a point cloud sequence signal into the Ground Truth Labeler app, the app creates an instance of the class, that is, a PointCloudSequenceSource object. After labeling this signal in the app, when you export the labels, the exported groundTruthMultisignal object stores this PointCloudSequenceSource object in its DataSource property.

When defining your data source class, replace PointCloudSequenceSource with the name of your custom data source class.

Class Properties

Data source classes must define these abstract, constant properties.

• Name — A string scalar specifying the type of the data source

• Description — A string scalar describing the class

In the Ground Truth Labeler app, when you load signals from the Add/Remove Signal dialog box, the Name string appears as an option in the Source Type parameter. This figure shows the Name string for the vision.labeler.loading.PointCloudSequenceSource class.

The Description string does not appear in the dialog box. However, both the Name and Description strings are stored as read-only properties in instances of this class.

This code shows the Name and Property strings for the vision.labeler.loading.PointCloudSequenceSource class.

    properties (Constant)
        Name = "Point Cloud Sequence"
        
        Description = "A PointCloud sequence reader"
    end

When defining your data source class, define the Name and Description property values to match the name and description of your custom data source. You can also define any additional private properties that are specific to loading your data source. The source-specific properties for the vision.labeler.loading.PointCloudSequenceSource class are not shown in this example, but you can view them in the class file.

Method to Customize Load Panel

In data source classes, the customizeLoadPanel method controls the display of the panel for loading signals in the Add/Remove Signal dialog box of the app. This panel is a Panel object created by using the uipanel function. The panel contains the parameters and controls needed to load signals from data sources.

This figure shows the loading panel for the vision.labeler.loading.PointCloudSequenceSource class. In the Source Type list, when you select Point Cloud Sequence, the app calls the customizeLoadPanel method and loads the panel for point cloud sequences.

This code shows the customizeLoadPanel method for the vision.labeler.loading.PointCloudSequenceSource class. It uses the computePositions method to calculate the position values where the UI components such as text, buttons and parameters must be placed. The addUIComponents method then defines the panel by adding the UI components accordingly. For complete implementation of these methods, refer to the code for the vision.labeler.loading.PointCloudSequenceSource class.

        function customizeLoadPanel(this, panel)
            this.Panel = panel;
            
            computePositions(this);
            
            addUIComponents(this);
        end

When developing this method or other data source methods, you can use the static method loadPanelChecker to preview the display and functionality of the loading dialog box for your custom data source. This method does not require you to have an app session open to use it. For example, use the loadPanelChecker method with the vision.labeler.loading.PointCloudSequence class.

vision.labeler.loading.PointCloudSequenceSource.loadPanelChecker

Methods to Get Load Panel Data and Load Data Source

In the Add/Remove Signal dialog box, after you browse for a signal, set the necessary parameters, and click Add Source, the app calls these two methods in succession.

• getLoadPanelData — Get the data entered into the panel.

• loadSource — Load the data into the app.

This figure shows the relationship between these methods and the Add Source button when loading a point cloud sequence signal by using the vision.labeler.loading.PointCloudSequenceSource class.

When defining a custom data source, you must define the getLoadPanelData method, which returns these outputs.

• sourceName — The name of the data source

• sourceParams — A structure containing fields with information required to load the data source

This code shows the getLoadPanelData method for the vision.labeler.loading.PointCloudSequenceSource class. This method sets sourceName to the name entered in the Folder Name parameter of the dialog box and sourceParams to an empty structure. If the Timestamps parameter is set to From Workspace and has timestamps loaded, then the app populates this structure with those timestamps.

        function [sourceName, sourceParams] = getLoadPanelData(this)
            sourceParams = struct();
            
            if isUIFigureBased(this)
                sourceName = string(this.FolderPathBox.Value);
            else
                sourceName = string(this.FolderPathBox.String);
            end            
        end

You must also define the loadSource method in your custom data class. This method must take the sourceName and sourceParams returned from the getLoadPanelData method as inputs. This method must also populate these properties, which are stored in the instance of the data source object that the app creates.

• SignalName — String identifiers for each signal in a data source

• SignalType — An array of vision.labeler.loading.SignalType enumerations defining the type of each signal in the data source

• Timestamp — A vector or cell array of timestamps for each signal in the data source

• SourceName — The name of the data source

• SourceParams — A structure containing fields with information required to load the data source

This code shows the loadSource method for the vision.labeler.loading.PointCloudSequenceSource class. This method performs these actions.

1. Check that the point cloud sequence has the correct extension and save the information required for reading the point clouds into a fileDatastore object.

2. Set the Timestamp property of the data source object.

   • If timestamps are loaded from a workspace variable (Timestamps = From workspace), then the method sets Timestamp to the timestamps stored in the sourceParams input.

   • If timestamps are derived from the point cloud sequence itself (Timestamps = Use Default), then the method sets Timestamp to a duration vector of seconds, with one second per point cloud.

3. Validate the loaded point cloud sequence.

4. Set the SignalName property to the name of the data source folder.

5. Set the SignalType property to the PointCloud signal type.

6. Set the SourceName and SourceParams properties to the sourceName and sourceParams outputs, respectively.

        function loadSource(this, sourceName, sourceParams)
            
            % Load file
            ext = {'.pcd', '.ply'};
            this.Pcds = fileDatastore(sourceName,'ReadFcn', @pcread, 'FileExtensions', ext);
            
            % Populate timestamps
            
            if isempty(this.Timestamp)
                if isfield(sourceParams, 'Timestamps')
                    setTimestamps(this, sourceParams.Timestamps);
                else
                    this.Timestamp = {seconds(0:1:numel(this.Pcds.Files)-1)'};
                end
            else
                if ~iscell(this.Timestamp)
                    this.Timestamp = {this.Timestamp};
                end
            end
            
            import vision.internal.labeler.validation.*
            checkPointCloudSequenceAndTimestampsAgreement(this.Pcds,this.Timestamp{1});
            
            % Populate signal names and types
            [~, folderName, ~] = fileparts(sourceName);
            
            this.SignalName = makeValidName(this, string(folderName), "pointcloudSequence_");
            this.SignalType = vision.labeler.loading.SignalType.PointCloud;
            
            this.SourceName = sourceName;
            this.SourceParams = sourceParams;
        end

Method to Read Frames

The last required method that you must define is the readFrame method. This method reads a frame from a signal stored in the data source. The app calls this method each time you navigate to a new frame. The index to a particular timestamp in the Timestamp property is passed to this method.

This code shows the readFrame method for the vision.labeler.loading.PointCloudSequenceSource class. The method reads frames from the point cloud sequence by using the pcread function.

        function frame = readFrame(this, signalName, index)
            if ~strcmpi(signalName, this.SignalName)
                frame = [];
            else
                frame = pcread(this.Pcds.Files{index});
            end
        end

You can also define any additional private properties that are specific to loading your data source. The source-specific methods for the vision.labeler.loading.PointCloudSequenceSource class are not shown in this example but you can view them in the class file.

Use Predefined Data Source Classes

This example showed how to use the vision.labeler.loading.PointCloudSequenceSource class to help you create your own custom class. This table shows the complete list of data source classes that you can use as starting points for your own class.

Class	Data Source Loaded by Class	Command to View Class Source Code
vision.labeler.loading.VideoSource	Video file	edit vision.labeler.loading.VideoSource
vision.labeler.loading.ImageSequenceSource	Image sequence folder	edit vision.labeler.loading.ImageSequenceSource
vision.labeler.loading.VelodyneLidarSource	Velodyne packet capture (PCAP) file	edit vision.labeler.loading.VelodyneLidarSource
vision.labeler.loading.RosbagSource	Rosbag file	edit vision.labeler.loading.RosbagSource
vision.labeler.loading.PointCloudSequenceSource	Point cloud sequence folder	edit vision.labeler.loading.PointCloudSequenceSource
vision.labeler.loading.CustomImageSource	Custom image format	edit vision.labeler.loading.CustomImageSource