Convert Ultrasound Image Series into Training Data for 2-D Semantic Segmentation Network

This example shows how to create training data for a 2-D semantic segmentation network using a groundTruthMedical object that contains a multiframe ultrasound image series. To train a semantic segmentation network, you must have pairs of data source images and label images, stored in an imageDatastore object and pixelLabelDatastore object, respectively. To train a 2-D network using data from a multiframe image series, you must convert the series into individual frames stored in separate files.

Create Ground Truth Object

You can export a groundTruthMedical object from the Medical Image Labeler app or create one programmatically. For illustrative purposes, this example creates a groundTruthMedical object by using the createGroundTruthMed2D helper function. The helper function is attached to this example as a supporting file. The groundTruthMedical object references a multiframe echocardiogram data source and its corresponding label image series. The data source and label images are stored as a single DICOM file and MAT file, respectively.

gTruthMed = createGroundTruthMed2D;

Extract Data from Ground Truth Object

Extract the data source and label image filenames from the groundTruthMedical object.

dataSource = gTruthMed.DataSource.Source;
labelData = gTruthMed.LabelData;

Remove any data sources that are missing label images, if present.

labelsIdx = labelData ~= "";
dataSource = dataSource(labelsIdx);
labelData = labelData(labelsIdx);

Extract the label definitions from the groundTruthMedical object. Add a label definition for the background region, corresponding to a pixel value of 0.

labelDefs = gTruthMed.LabelDefinitions;
labelDefs(2,:) = {"background",[0 1 0],0};

Convert Multiframe Series into 2-D Frames

The data source is a multiframe ultrasound image series stored in a single DICOM file. Convert the DICOM file into individual 2-D images, stored as MAT files, by using the convertImageSeriesToFrames supporting function. The supporting function is defined at the end of this example.

newDataSource = convertImageSeriesToFrames(dataSource);
newDataSource = string(newDataSource);

The label data is a multiframe image series stored in a single MAT file. Convert each frame of the image series into an individual MAT file by using the convertLabelSeriesToFrames supporting function. The supporting function is defined at the end of this example.

newLabelData = convertLabelSeriesToFrames(labelData);

Create Datastores

Load the individual MAT file data sources into an imageDatastore object.

imds = imageDatastore(newDataSource, ...
       ReadFcn=@readFramesLabels, ...
       FileExtensions=[".mat",".dcm"]);

Load the individual MAT file label images into a pixelLabelDatastore (Computer Vision Toolbox) object. Use the label definitions from the groundTruthMedical object to map pixel values to categorical labels in the datastore.

pxds = pixelLabelDatastore(cellstr(newLabelData),labelDefs.Name,labelDefs.PixelLabelID, ...
       ReadFcn=@readFramesLabels, ...
       FileExtensions=".mat");

Preview one image and its corresponding label. Display the labeled image by using the labeloverlay function.

im = preview(imds);
label = preview(pxds);
imOverlay = labeloverlay(im,label);
imshow(imOverlay)

Create a CombinedDatastore object that pairs each data source image with its corresponding label image.

trainingData = combine(imds,pxds);

Supporting Functions

The convertImageSeriesToSlices function converts a multiframe DICOM image series, such as ultrasound data, into individual 2-D frames stored in separate MAT files. The function returns a cell array of the new MAT filenames.

function newDataSource = convertImageSeriesToFrames(labelDataSource)
% Create a data directory to store MAT files
dataFileDir = fullfile(pwd,"GroundTruthData");

if ~isfolder(dataFileDir)
    mkdir(dataFileDir)
end

image = medicalImage(labelDataSource);
data3d = image.Pixels;

% Assumption that time is the third dimension
numFrames =  size(data3d,3);
newDataSource = cell(numFrames,1);

for frame = 1:numFrames
    data = squeeze(data3d(:,:,frame,:));
    [~,name,~] = fileparts(labelDataSource);
    matFileName = strcat(fullfile(dataFileDir,name),"_",num2str(frame),".mat");
    save(matFileName,"data");
    newDataSource{frame} = string(matFileName);
end

end

The convertLabelSeriesToFrames function converts a multiframe image series stored in a single file into individual frames stored as separate MAT files. The function returns a cell array of the new MAT filenames.

function newlabelData = convertLabelSeriesToFrames(labelSource)
% Create a label directory to store MAT files
labelFileDir = fullfile(pwd,"GroundTruthLabel");

if ~isfolder(labelFileDir)
    mkdir(labelFileDir)
end

labelData = load(labelSource);

% Assume that the third dimension contains frames taken at different times
numFrames =  size(labelData.labels,3);
newlabelData = cell(numFrames,1);

[~,name,~] = fileparts(labelSource);
for frame = 1:numFrames
    data = squeeze(labelData.labels(:,:,frame,:));
    matFileName = strcat(fullfile(labelFileDir,name),"_",num2str(frame),".mat");
    save(matFileName,"data");
    newlabelData{frame} = string(matFileName);
end

end

The readFramesLabels function reads data from the data variable of a MAT file.

function data = readFramesLabels(filename)
d = load(filename);
data = d.data;
end